1
0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-25 14:12:13 +00:00
Commit Graph

11985 Commits

Author SHA1 Message Date
Christopher Faulet
fe410d685a BUG/MAJOR: mux-fcgi: Stop sending loop if FCGI stream is blocked for any reason
Because of a typo error in conditions to exit the sending loop, it is possible
to loop infinitely in fcgi_snd_buf() function. Instead of checking the FCGI
stream is not blocked to continue sending data, the FCGI connection is used. So
it is possible to have a stream blocked because there is not enough space in the
mux buffers to copy more data but continue to loop to send more data.

This patch should fix the issue . It must be backported to 2.1.
2020-05-19 16:00:55 +02:00
Willy Tarreau
2bdcc70fa7 MEDIUM: hpack: use a pool for the hpack table
Instead of using malloc/free to allocate an HPACK table, let's declare
a pool. However the HPACK size is configured by the H2 mux, so it's
also this one which allocates it after post_check.
2020-05-19 11:40:39 +02:00
Emeric Brun
957ec59571 MEDIUM: sink: add global statement to create a new ring (sink buffer)
This patch adds the new global statement:
ring <name> [desc <desc>] [format <format>] [size <size>] [maxlen <length>]
  Creates a named ring buffer which could be used on log line for instance.

  <desc> is an optionnal description string of the ring. It will appear on
         CLI. By default, <name> is reused to fill this field.

  <format> is the log format used when generating syslog messages. It may be
           one of the following :

    iso       A message containing only the ISO date, followed by the text.
              The PID, process name and system name are omitted. This is
              designed to be used with a local log server.

    raw       A message containing only the text. The level, PID, date, time,
              process name and system name are omitted. This is designed to be
              used in containers or during development, where the severity only
              depends on the file descriptor used (stdout/stderr). This is
              the default.

    rfc3164   The RFC3164 syslog message format. This is the default.
              (https://tools.ietf.org/html/rfc3164)

    rfc5424   The RFC5424 syslog message format.
              (https://tools.ietf.org/html/rfc5424)

    short     A message containing only a level between angle brackets such as
              '<3>', followed by the text. The PID, date, time, process name
              and system name are omitted. This is designed to be used with a
              local log server. This format is compatible with what the systemd
              logger consumes.

    timed     A message containing only a level between angle brackets such as
              '<3>', followed by ISO date and by the text. The PID, process
              name and system name are omitted. This is designed to be
              used with a local log server.

  <length> is the maximum length of event message stored into the ring,
           including formatted header. If the event message is longer
           than <length>, it would be truncated to this length.

  <name> is the ring identifier, which follows the same naming convention as
         proxies and servers.

  <size> is the optionnal size in bytes. Default value is set to BUFSIZE.

Note: Historically sink's name and desc were refs on const strings. But with new
configurable rings a dynamic allocation is needed.
2020-05-19 11:04:11 +02:00
Emeric Brun
e709e1e777 MEDIUM: logs: buffer targets now rely on new sink_write
Before this path, they rely directly on ring_write bypassing
a part of the sink API.

Now the maxlen parameter of the log will apply only on the text
message part (and not the header, for this you woud prefer
to use the maxlen parameter on the sink/ring).

sink_write prototype was also reviewed to return the number of Bytes
written to be compliant with the other write functions.
2020-05-19 11:04:11 +02:00
Emeric Brun
bd163817ed MEDIUM: sink: build header in sink_write for log formats
This patch extends the sink_write prototype and code to
handle the rfc5424 and rfc3164 header.

It uses header building tools from log.c. Doing this some
functions/vars have been externalized.

facility and minlevel have been removed from the struct sink
and passed to args at sink_write because they depends of the log
and not of the sink (they remained unused by rest of the code
until now).
2020-05-19 11:04:11 +02:00
Emeric Brun
9e8ea0ae6f BUG/MINOR: logs: prevent double line returns in some events.
Historically some messages used to already contain the trailing LF but
not all, and __do_send_log adds a new one in needed cases. It also does
trim a trailing LF in certain cases while computing the max message
length, as a result of subtracting 1 to the available room in the
destination buffer. But the way it's done is wrong since some messages
still contain it.

So the code was fixed to always trim the trailing LF from messages if
present, and then only subtract 1 from the destination buffer room
instead of the size..

Note: new sink API is not designed to receive a trailing LF on
event messages

This could be backported to relevant stable versions with particular
care since the logic of the code changed a bit since 1.6 and there
may be other locations that need to be adjusted.
2020-05-19 10:59:53 +02:00
Christopher Faulet
62f79fe68a MEDIUM: checks: Make post-41 the default mode for mysql checks
MySQL 4.1 is old enough to be the default mode for mysql checks. So now, once a
username is defined, post-41 mode is automatically used. To do mysql checks on
previous MySQL version, the argument "pre-41" must be used.

Note, it is a compatibility breakage for everyone using an antique and
unsupported MySQL version.
2020-05-18 18:32:09 +02:00
Christopher Faulet
784063eeb2 MINOR: config: Don't dump keywords if argument is NULL
Helper functions are used to dump bind, server or filter keywords. These
functions are used to report errors during the configuration parsing. To have a
coherent API, these functions are now prepared to handle a null pointer as
argument. If so, no action is performed and functions immediately return.

This patch should fix the issue . It is not a bug. There is no reason to
backport it.
2020-05-18 18:30:06 +02:00
Christopher Faulet
2a37cdbe6b BUG/MINOR: cache: Don't needlessly test "cache" keyword in parse_cache_flt()
parse_cache_flt() is the registered callback for the "cache" filter keyword. It
is only called when the "cache" keyword is found on a filter line. So, it is
useless to test the filter name in the callback function.

This patch should fix the issue . It may be backported as far as 1.9.
2020-05-18 17:47:18 +02:00
Christopher Faulet
3b78809de0 DOC: SPOE is no longer experimental
The SPOE was marked as experiemental since the begining. But, it is no longer
true. This can be an obstacle to its use.
2020-05-18 17:47:18 +02:00
William Lallemand
5520d6f3c7 BUILD: ssl: fix build without OPENSSL_NO_ENGINE
The CI revealed that the boringssl build is still broken because of some
ifdef misplacement.

Bug introduced by dad3105 ("REORG: ssl: move ssl configuration to
cfgparse-ssl.c").

No backport needed.

Fix issue .
2020-05-18 14:19:26 +02:00
Aleksandar Lazi
6112f5ccd2 DOC/MINOR: halog: Add long help info for ic flag
Add missing long help text for the ic (ip count) flag
2020-05-18 09:30:43 +02:00
Ilya Shipitsin
bd7212be10 CLEANUP: acl: remove unused assignment
Coverity found unused variable assignment

    CID 1299671 ( of 1): Unused value (UNUSED_VALUE)assigned_pointer:
    Assigning value from args[arg + 1] to word here, but that stored
    value is overwritten before it can be used.
 958                        word = args[arg + 1];
 959                        arg = arg_end;
2020-05-18 09:21:27 +02:00
William Lallemand
212e993e71 BUILD: ssl: include errno.h in ssl_crtlist.c
In issue  boringssl build were broken by the lack of errno.h
include in ssl_crtlist.c

Bug introduced by 6e9556b ("REORG: ssl: move crtlist functions to src/ssl_crtlist.c").

No backport needed.
2020-05-18 08:39:31 +02:00
William Dauchy
1665c43fd8 BUILD: ssl: include buffer common headers for ssl_sock_ctx
since commit c0cdaffaa3 ("REORG: ssl: move ssl_sock_ctx and fix
cross-dependencies issues"), `struct ssl_sock_ctx` was moved in
ssl_sock.h. As it contains a `struct buffer`, including
`common/buffer.h` is now mandatory. I encountered an issue while
including ssl_sock.h on another patch:

include/types/ssl_sock.h:240:16: error: field ‘early_buf’ has incomplete type
  240 |  struct buffer early_buf;      /* buffer to store the early data received */

no backport needed.

Fixes: c0cdaffaa3 ("REORG: ssl: move ssl_sock_ctx and fix
cross-dependencies issues")
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-05-18 08:29:32 +02:00
Marcin Deranek
a8dbdf3c4b MEDIUM: stats: Enable more accurate moving average calculation for stats
Enable more accurate generation of moving averages for partially
populated samples window.
2020-05-16 22:40:06 +02:00
Marcin Deranek
3c27ddabec MINOR: stats: Expose native cum_req metric for a server
Expose native cum_req metric for a server: so far it was calculated as a
sum or all responses. Rename it from Cum. HTTP Responses to Cum. HTTP
Requests to be consistent with Frontend and Backend.
2020-05-16 22:40:03 +02:00
Marcin Deranek
4dc2b57d51 MINOR: stats: Prepare for more accurate moving averages
Add swrate_add_dynamic function which is similar to swrate_add, but more
accurate when calculating moving averages when not enough samples have
been processed yet.
2020-05-16 22:40:00 +02:00
Martin Tzvetanov Grigorov
5eb5e253e6 CI: travis-ci: switch arm64 builds to use openssl from distro
there are ongoing arm64 failures in travis-ci. we suspect that
build stucks on scripts/build-ssl.sh which is called during build.
let us try to use openssl from ubuntu distro and not build it
ourselves. to achive that we add "install" section without
build-ssl.sh
2020-05-15 20:52:52 +02:00
William Lallemand
2f14aeb0cc DOC: ssl: update MAINTAINERS file
Update the MAINTAINERS file with new files and maintainer.
2020-05-15 14:11:54 +02:00
William Lallemand
6a66a5ec9b REORG: ssl: move utility functions to src/ssl_utils.c
These functions are mainly used to extract information from
certificates.
2020-05-15 14:11:54 +02:00
William Lallemand
15e169447d REORG: ssl: move sample fetches to src/ssl_sample.c
Move all SSL sample fetches to src/ssl_sample.c.
2020-05-15 14:11:54 +02:00
William Lallemand
c0cdaffaa3 REORG: ssl: move ssl_sock_ctx and fix cross-dependencies issues
In order to move all SSL sample fetches in another file, moving the
ssl_sock_ctx definition in a .h file is required.

Unfortunately it became a cross dependencies hell to solve, because of
the struct wait_event field, so <types/connection.h> is needed which
created other problems.
2020-05-15 14:11:54 +02:00
William Lallemand
ef76107a4b MINOR: ssl: remove static keyword in some SSL utility functions
In order to move the the sample fetches to another file, remove the
static keyword of some utility functions in the SSL fetches.
2020-05-15 14:11:54 +02:00
William Lallemand
dad3105157 REORG: ssl: move ssl configuration to cfgparse-ssl.c
Move all the configuration parsing of the ssl keywords in cfgparse-ssl.c
2020-05-15 14:11:54 +02:00
William Lallemand
da8584c1ea REORG: ssl: move the CLI 'cert' functions to src/ssl_ckch.c
Move the 'ssl cert' CLI functions to src/ssl_ckch.c.
2020-05-15 14:11:54 +02:00
William Lallemand
c756bbd3df REORG: ssl: move the crt-list CLI functions in src/ssl_crtlist.c
Move the crtlist functions for the CLI to src/ssl_crtlist.c
2020-05-15 14:11:54 +02:00
William Lallemand
fa1d8b4eaa REORG: ssl: move ckch_inst functions to src/ssl_ckch.c
Move ckch_inst_new() and ckch_inst_free() to src/ssl_ckch.c
2020-05-15 14:11:54 +02:00
William Lallemand
03c331c80a REORG: ssl: move the ckch_store related functions to src/ssl_ckch.c
Move the cert_key_and_chain functions:

int ssl_sock_load_files_into_ckch(const char *path, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_pem_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch , char **err);
void ssl_sock_free_cert_key_and_chain_contents(struct cert_key_and_chain *ckch);

int ssl_sock_load_key_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch , char **err);
int ssl_sock_load_ocsp_response_from_file(const char *ocsp_path, char *buf, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_sctl_from_file(const char *sctl_path, char *buf, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_issuer_file_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch, char **err);

And the utility ckch_store functions:

void ckch_store_free(struct ckch_store *store)
struct ckch_store *ckch_store_new(const char *filename, int nmemb)
struct ckch_store *ckchs_dup(const struct ckch_store *src)
ckch_store *ckchs_lookup(char *path)
ckch_store *ckchs_load_cert_file(char *path, int multi, char **err)
2020-05-15 14:11:54 +02:00
William Lallemand
c1c50b46e9 CLEANUP: ssl: avoid circular dependencies in ssl_crtlist.h
Add forward declarations in types/ssl_crtlist.h in order to avoid
circular dependencies. Also remove the listener.h include which is not
needed anymore.
2020-05-15 14:11:54 +02:00
William Lallemand
6e9556b635 REORG: ssl: move crtlist functions to src/ssl_crtlist.c
Move the crtlist functions to src/ssl_crtlist.c and their definitions to
proto/ssl_crtlist.h.

The following functions were moved:

/* crt-list entry functions */
void ssl_sock_free_ssl_conf(struct ssl_bind_conf *conf);
char **crtlist_dup_filters(char **args, int fcount);
void crtlist_free_filters(char **args);
void crtlist_entry_free(struct crtlist_entry *entry);
struct crtlist_entry *crtlist_entry_new();

/* crt-list functions */
void crtlist_free(struct crtlist *crtlist);
struct crtlist *crtlist_new(const char *filename, int unique);

/* file loading */
int crtlist_parse_line(char *line, char **crt_path, struct crtlist_entry *entry, const char *file, int linenum, char **err);
int crtlist_parse_file(char *file, struct bind_conf *bind_conf, struct proxy *curproxy, struct crtlist **crtlist, char **err);
int crtlist_load_cert_dir(char *path, struct bind_conf *bind_conf, struct crtlist **crtlist, char **err);
2020-05-15 14:11:54 +02:00
William Lallemand
c69973f7eb CLEANUP: ssl: add ckch prototypes in proto/ssl_ckch.h
Remove the static definitions of the ckch functions and add them to
ssl_ckch.h in order to use them outside ssl_sock.c.
2020-05-15 14:11:54 +02:00
William Lallemand
d4632b2b6d REORG: ssl: move the ckch structures to types/ssl_ckch.h
Move all the structures used for loading the SSL certificates in
ssl_ckch.h
2020-05-15 14:11:54 +02:00
William Lallemand
be21b663cd REORG: move the crt-list structures in their own .h
Move the structure definitions specifics to the crt-list in
types/ssl_crtlist.h.
2020-05-15 14:11:54 +02:00
William Lallemand
336c4bbb08 CLEANUP: ssl: remove the shsess_* macros
The shsess_* macros where already defined in proto/ssl_sock.h, remove
them from ssl_sock.c
2020-05-15 14:11:54 +02:00
William Lallemand
7fd8b4567e REORG: ssl: move macros and structure definitions to ssl_sock.h
The ssl_sock.c file contains a lot of macros and structure definitions
that should be in a .h. Move them to the more appropriate
types/ssl_sock.h file.
2020-05-15 14:11:54 +02:00
Dragan Dosen
2dec6a3bf1 MEDIUM: ssl: use ssl_sock_get_ssl_object() in fetchers where appropriate
Doing this also makes sure that conn->xprt_ctx is always checked before
using it.
2020-05-14 13:13:14 +02:00
Dragan Dosen
eb607fe6a1 MINOR: ssl: add a new function ssl_sock_get_ssl_object()
This one can be used later to get a SSL object from connection. It will
return NULL if connection is not established over SSL.
2020-05-14 13:13:14 +02:00
Dragan Dosen
9ac9809cb9 MEDIUM: ssl: split ssl_sock_msgcbk() and use a new callback mechanism
Make use of ssl_sock_register_msg_callback(). Function ssl_sock_msgcbk()
is now split into two dedicated functions for heartbeat and clienthello.
They are both registered by using a new callback mechanism for SSL/TLS
protocol messages.
2020-05-14 13:13:14 +02:00
Dragan Dosen
1e7ed04665 MEDIUM: ssl: allow to register callbacks for SSL/TLS protocol messages
This patch adds the ability to register callbacks for SSL/TLS protocol
messages by using the function ssl_sock_register_msg_callback().

All registered callback functions will be called when observing received
or sent SSL/TLS protocol messages.
2020-05-14 13:13:14 +02:00
Jerome Magnin
5ce3c14aa9 DOC: retry-on can only be used with mode http
The documentation for retry-on hints at it being meant to be used
in conjuction with mode http, but since we've a had bug report
involving mode tcp and retry-on, lets make it explicit in the
documentation that it only works with mode http and will be
ignored otherwise.
2020-05-14 09:43:13 +02:00
Olivier Houchard
7dd7b908db BUG/MEDIUM: stream: Only allow L7 retries when using HTTP.
Only allow L7 retries when using HTTP, it only really makes sense for HTTP,
anyway, and as the L7 retries code assume the message will be HTX, it will
crash when used with mode TCP.
This should fix github issue .

This should be backported to 2.1 and 2.0.
2020-05-13 19:07:20 +02:00
Olivier Houchard
8cabc9783a BUG/MEDIUM: streams: Remove SF_ADDR_SET if we're retrying due to L7 retry.
In do_l7_retry(), remove the SF_ADDR_SET flag. Otherwise,
assign_server_address() won't be called again, which means for 2.1 or 2.2,
we will always retry to connect to the server that just failed, and for 2.0,
that we will try to use to whatever the address is for the connection,
probably the last server used by that connection before it was pool_free()
and reallocated.

This should be backported to 2.1 and 2.0.
2020-05-13 19:02:38 +02:00
Jerome Magnin
c9c475e40d BUILD: select: only declare existing local labels to appease clang
Commit 42a50bd19 ("BUG/MINOR: pollers: remove uneeded free in global
init") removed the 'fail_revt' label from the _do_init() function
in src/ev_select.c but left the local label declaration, which makes
clang unhappy and unable to build.

These labels are only historic and unneeded anyway so let's remove them.

This should be backported where 42a50bd19 is backported.
2020-05-13 16:25:26 +02:00
Willy Tarreau
d645574fd4 MINOR: soft-stop: let the first stopper only signal other threads
When the first thread stops and wakes others up, it's possible some of
them will also start to wake others in parallel. Let's make give this
notification task to the very first one instead since it's enough and
can reduce the amount of needless (though harmless) wakeup calls.
2020-05-13 14:30:25 +02:00
Willy Tarreau
d7a6b2f742 BUG/MINOR: soft-stop: always wake up waiting threads on stopping
Currently the soft-stop can lead to old processes remaining alive for as
long as two seconds after receiving a soft-stop signal. What happens is
that when receiving SIGUSR1, one thread (usually the first one) wakes up,
handles the signal, sets "stopping", goes into runn_poll_loop(), and
discovers that stopping is set, so its also sets itself in the
stopping_thread_mask bit mask. After this it sees that other threads are
not yet willing to stop, so it continues to wait.

From there, other threads which were waiting in poll() expire after one
second on poll timeout and enter run_poll_loop() in turn. That's already
one second of wait time. They discover each in turn that they're stopping
and see that other threads are not yet stopping, so they go back waiting.

After the end of the first second, all threads know they're stopping and
have set their bit in stopping_thread_mask. It's only now that those who
started to wait first wake up again on timeout to discover that all other
ones are stopping, and can now quit. One second later all threads will
have done it and the process will quit.

This is effectively strictly larger than one second and up to two seconds.

What the current patch does is simple, when the first thread stops, it sets
its own bit into stopping_thread_mask then wakes up all other threads to do
also set theirs. This kills the first second which corresponds to the time
to discover the stopping state. Second, when a thread exists, it wakes all
other ones again because some might have gone back sleeping waiting for
"jobs" to go down to zero (i.e. closing the last connection). This kills
the last second of wait time.

Thanks to this, as SIGUSR1 now acts instantly again if there's no active
connection, or it stops immediately after the last connection has left if
one was still present.

This should be backported as far as 2.0.
2020-05-13 14:11:18 +02:00
William Dauchy
32fba0a629 CLEANUP: select: enhance readability in init
while reading the code, I thought it was clearer to put one instruction
per line as it is mostly done elsewhere

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-05-13 11:52:30 +02:00
William Dauchy
42a50bd19b BUG/MINOR: pollers: remove uneeded free in global init
Since commit d4604adeaa ("MAJOR: threads/fd: Make fd stuffs
thread-safe"), we init pollers per thread using a helper. It was still
correct for mono-thread mode until commit cd7879adc2 ("BUG/MEDIUM:
threads: Run the poll loop on the main thread too"). We now use a deinit
helper for all threads, making those free uneeded.

Only poll and select are affected by this very minor issue.

it could be backported from v1.8 to v2.1.

Fixes: cd7879adc2 ("BUG/MEDIUM: threads: Run the poll loop on the main
thread too")
Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-05-13 11:45:37 +02:00
Willy Tarreau
cece694bcc BUG/MINOR: pools: use %u not %d to report pool stats in "show pools"
In dump_pools_to_trash() we happen to use %d to display unsigned ints!

This has probably been there since "show pools" was introduced so this
fix must be backported to all versions. The impact is negligible since
no pool uses 2 billion entries. It could possibly affect the report of
failed allocation counts but in this case there's a bigger problem to
solved!
2020-05-13 10:10:54 +02:00
Christopher Faulet
325504cf89 BUG/MINOR: sample/ssl: Fix digest converter for openssl < 1.1.0
The EVP_MD_CTX_create() and EVP_MD_CTX_destroy() functions were renamed to
EVP_MD_CTX_new() and EVP_MD_CTX_free() in OpenSSL 1.1.0, respectively. These
functions are used by the digest converter, introduced by the commit 8e36651ed
("MINOR: sample: Add digest and hmac converters"). So for prior versions of
openssl, macros are used to fallback on old functions.

This patch must only be backported if the commit 8e36651ed is backported too.
2020-05-12 16:30:41 +02:00