The loop to find the id corresponding to the current rule in
tcpcheck_get_step_id() function has been simplified. And
tcpcheck_get_step_comment() function now only relies on the current rule to find
the rigth comment string. The step id is no longer used. To do so, we iterate
backward from the current step to find the first COMMENT rule immediately
preceedding the expect rule chain.
The rbinary match works similarly to the rstring match type, however the
received data is rewritten as hex-string before the match operation is
done.
This allows using regexes on binary content even with the POSIX regex
engine.
[Cf: I slightly updated the patch. mem2hex function was removed and dump_binary
is used instead.]
On the input buffer, it was mainly done to call regex_exec() function. But
regex_exec2() can be used instead. This way, it is no more required to add the
terminating null byte. For the output buffer, it was only done for debugging
purpose.
Allow declaring tcpcheck connect commands with a new parameter,
"linger". This option will configure the connection to avoid using an
RST segment to close, instead following the four-way termination
handshake. Some servers would otherwise log each healthcheck as
an error.
Some expect rules cannot be satisfied due to inherent ambiguity towards
the received data: in the absence of match, the current behavior is to
be forced to wait either the end of the connection or a buffer full,
whichever comes first. Only then does the matching diagnostic is
considered conclusive. For instance :
tcp-check connect
tcp-check expect !rstring "^error"
tcp-check expect string "valid"
This check will only succeed if the connection is closed by the server before
the check timeout. Otherwise the first expect rule will wait for more data until
"^error" regex matches or the check expires.
Allow the user to explicitly define an amount of data that will be
considered enough to determine the value of the check.
This allows succeeding on negative rstring rules, as previously
in valid condition no match happened, and the matching was repeated
until the end of the connection. This could timeout the check
while no error was happening.
[Cf: I slighly updated the patch. The parameter was renamed and the value is a
signed integer to support -1 as default value to ignore the parameter.]
Reduce copy of parsing portions that is common to all three types of
expect actions.
This reduces the amount of code, helping maintainability and reducing
future change spread.
Functionality is identical.
When receiving additional data while chaining multiple tcp-check expects,
previous inverse expects might have a different result with the new data. They
need to be evaluated again against the new data.
Add a pointer to the first inverse expect rule of the current expect chain
(possibly of length one) to each expect rule. When receiving new data, the
currently evaluated tcp-check rule is set back to this pointed rule.
Fonctionnaly speaking, it is a bug and it exists since the introduction of the
feature. But there is no way for now to hit it because when an expect rule does
not match, we wait for more data, independently on the inverse flag. The only
way to move to the following rule is to be sure no more data will be received.
This patch depends on the commit "MINOR: mini-clist: Add functions to iterate
backward on a list".
[Cf: I slightly updated the patch. First, it only concerns inverse expect
rule. Normal expect rules are not concerned. Then, I removed the BUG tag
because, for now, it is not possible to move to the following rule when the
current one does not match while more data can be received.]
Replace the generic integer with an enumerated list. This allows light
type check and helps debugging (seeing action = 2 in the struct is not
helpful).
TCP check expect matching strings or binary patterns are able to know
prior to applying their match function whether the available data is
already sufficient to attempt the match or not.
As such, on insufficient data the expect is postponed. This behavior
avoids unnecessary matches when the data could not possibly match.
When chaining expect, upon passing the previous and going onto the next
however, this length check is not done again. Then the match is done and
will necessarily fail, triggering a new wait for more data. The end
result is the same for a slightly higher cost.
Check received data length for all expects in a chain.
This bug exists since the introduction of the feature:
Fixes: 5ecb77f4c7 ("MEDIUM: checks: add send/expect tcp based check")
Version 1.5+ impacted.
The options and directives related to the configuration of checks in a backend
may be defined after the servers declarations. So, initialization of the check
of each server must not be performed during configuration parsing, because some
info may be missing. Instead, it must be done during the configuration validity
check.
Thus, callback functions are registered to be called for each server after the
config validity check, one for the server check and another one for the server
agent-check. In addition deinit callback functions are also registered to
release these checks.
This patch should be backported as far as 1.7. But per-server post_check
callback functions are only supported since the 2.1. And the initcall mechanism
does not exist before the 1.9. Finally, in 1.7, the code is totally
different. So the backport will be harder on older versions.
This options is used to force a non-SSL connection to check a SSL server or to
invert a check-ssl option inherited from the default section. The use_ssl field
in the check structure is used to know if a SSL connection must be used
(use_ssl=1) or not (use_ssl=0). The server configuration is used by default.
The problem is that we cannot distinguish the default case (no specific SSL
check option) and the case of an explicit non-SSL check. In both, use_ssl is set
to 0. So the server configuration is always used. For a SSL server, when
no-check-ssl option is set, the check is still performed using a SSL
configuration.
To fix the bug, instead of a boolean value (0=TCP, 1=SSL), we use a ternary value :
* 0 = use server config
* 1 = force SSL
* -1 = force non-SSL
The same is done for the server parameter. It is not really necessary for
now. But it is a good way to know is the server no-ssl option is set.
In addition, the PR_O_TCPCHK_SSL proxy option is no longer used to set use_ssl
to 1 for a check. Instead the flag is directly tested to prepare or destroy the
server SSL context.
This patch should be backported as far as 1.8.
Error codes ERR_WARN and ERR_ALERT are used to signal that the error
given is of the corresponding level. All errors are displayed as ALERT
in the display_parser_err() function.
Differentiate the display level based on the error code. If both
ERR_WARN and ERR_ALERT are used, ERR_ALERT is given priority.
The 'http-check send' directive have been added to add headers and optionnaly a
payload to the request sent during HTTP healthchecks. The request line may be
customized by the "option httpchk" directive but there was not official way to
add extra headers. An old trick consisted to hide these headers at the end of
the version string, on the "option httpchk" line. And it was impossible to add
an extra payload with an "http-check expect" directive because of the
"Connection: close" header appended to the request (See issue #16 for details).
So to make things official and fully support payload additions, the "http-check
send" directive have been added :
option httpchk POST /status HTTP/1.1
http-check send hdr Content-Type "application/json;charset=UTF-8" \
hdr X-test-1 value1 hdr X-test-2 value2 \
body "{id: 1, field: \"value\"}"
When a payload is defined, the Content-Length header is automatically added. So
chunk-encoded requests are not supported yet. For now, there is no special
validity checks on the extra headers.
This patch is inspired by Kiran Gavali's work. It should fix the issue #16 and
as far as possible, it may be backported, at least as far as 1.8.
list_for_each_entry_rev() and list_for_each_entry_from_rev() and corresponding
safe versions have been added to iterate on a list in the reverse order. All
these functions work the same way than the forward versions, except they use the
.p field to move for an element to another.
Server address and port may change at runtime. So the address and port passed as
arguments and as environment variables when an external check is executed must
be updated. The current number of connections on the server was already updated
before executing the command. So the same mechanism is used for the server
address and port. But in addition, command arguments are also updated.
This patch must be backported to all stable versions. It should fix the
issue #577.
It is the intended behaviour. But because of a bug, the 500 error resulting of a
rewrite failure during http-after-response ruleset evaluation is also
rewritten. So if at this step, if there is also a rewrite error, the session is
closed and no error message is returned.
Instead, we must be sure to not evaluate the http-after-response rules on an
error message if it is was thrown because of a rewrite failure on a previous
error message.
It is a 2.2-dev2+ bug. No need to backport. This patch should fix the issue
The wireshark dissector could only be build within wireshark, which
means maintaining a wireshark binary just for this dissector. It was not
really convenient to update wireshark because of this.
This patch converts the dissector into a .so plugin which is built with
the .h found in distributions instead of the whole wireshark sources.
pool_gc() causes quite some stress on the memory allocator because
it calls a lot of free() calls while other threads are using malloc().
In addition, pool_gc() needs to take care of possible locking because
it may be called from pool allocators. One way to avoid all this is to
use thread_isolate() to make sure the gc runs alone. By putting less
pressure on the pools and getting rid of the locks, it may even take
less time to complete.
The documentation for option logasap misleads into thinking it is
only valid for mode http. It is actually valid for mode tcp too,
so this patch tries to disambiguate the current wording.
The url_decode() function used by the url_dec converter and a few other
call points is ambiguous on its processing of the '+' character which
itself isn't stable in the spec. This one belongs to the reserved
characters for the query string but not for the path nor the scheme,
in which it must be left as-is. It's only in argument strings that
follow the application/x-www-form-urlencoded encoding that it must be
turned into a space, that is, in query strings and POST arguments.
The problem is that the function is used to process full URLs and
paths in various configs, and to process query strings from the stats
page for example.
This patch updates the function to differentiate the situation where
it's parsing a path and a query string. A new argument indicates if a
query string should be assumed, otherwise it's only assumed after seeing
a question mark.
The various locations in the code making use of this function were
updated to take care of this (most call places were using it to decode
POST arguments).
The url_dec converter is usually called on path or url samples, so it
needs to remain compatible with this and will default to parsing a path
and turning the '+' to a space only after a question mark. However in
situations where it would explicitly be extracted from a POST or a
query string, it now becomes possible to enforce the decoding by passing
a non-null value in argument.
It seems to be what was reported in issue #585. This fix may be
backported to older stable releases.
A typo resulted in '||' being used to concatenate trace flags, which will
only set flag of value '1' there. Noticed by clang 10 and reported in
issue #588.
No backport is needed, this trace was added in 2.2-dev.
As reported in issue #596, the edx register isn't marked as clobbered
in div64_32(), which could technically allow gcc to try to reuse it
if it needed a copy of the 32 highest bits of the o1 register after
the operation.
Two attempts were tried, one using a dummy 32-bit local variable to
store the intermediary edx and another one switching to "=A" and making
result a long long. It turns out the former makes the resulting object
code significantly dirtier while the latter makes it better and was
kept. This is due to gcc's difficulties at working with register pairs
mixing 32- and 64- bit values on i386. It was verified that no code
change happened at all on x86_64, armv7, aarch64 nor mips32.
In practice it's only used by the frequency counters so this bug
cannot even be triggered but better fix it.
This may be backported to stable branches though it will not fix any
issue.
When checking www-authenticate headers, we don't want to just accept
"NTLM" as value, because the server may send "HTLM <base64 value>". Instead,
just check that it starts with NTLM.
This should be backported to 2.1, 2.0, 1.9 and 1.8.
Documentation states that default settings for ssl server options can be set
using either ssl-default-server-options or default-server directives. In practice,
not all ssl server options can have default values, such as ssl-min-ver, ssl-max-ver,
etc..
This patch adds the missing ssl options in srv_ssl_settings_cpy() and srv_parse_ssl(),
making it possible to write configurations like the following examples, and have them
behave as expected.
global
ssl-default-server-options ssl-max-ver TLSv1.2
defaults
mode http
listen l1
bind 1.2.3.4:80
default-server ssl verify none
server s1 1.2.3.5:443
listen l2
bind 2.2.3.4:80
default-server ssl verify none ssl-max-ver TLSv1.3 ssl-min-ver TLSv1.2
server s1 1.2.3.6:443
This should be backported as far as 1.8.
This fixes issue #595.
This option activate the feature introduce in commit 16739778:
"MINOR: ssl: skip self issued CA in cert chain for ssl_ctx".
The patch disable the feature per default.
When trying to insert a new certificate into a directory with "add ssl
crt-list", no check were done on the path of the new certificate.
To be more consistent with the HAProxy reload, when adding a file to
a crt-list, if this crt-list is a directory, the certificate will need
to have the directory in its path.
Allowing the use of SSL options and filters when adding a file in a
directory is not really consistent with the reload of HAProxy. Disable
the ability to use these options if one try to use them with a directory.
This patch adds more explanation on how to use "http-request set-src"
and a link to "option forwardfor".
This patch can be applied to all previous version starting at 1.6
Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
This patch adds the sysname, release, version and machine fields from
the uname results to the version output. It intentionally leaves out the
machine name, because it is usually not useful and users might not want to
expose their machine names for privacy reasons.
May be backported if it is considered useful for debugging.
Released version 2.2-dev6 with the following main changes :
- BUG/MINOR: ssl: memory leak when find_chain is NULL
- CLEANUP: ssl: rename ssl_get_issuer_chain to ssl_get0_issuer_chain
- MINOR: ssl: rework add cert chain to CTX to be libssl independent
- BUG/MINOR: peers: init bind_proc to 1 if it wasn't initialized
- BUG/MINOR: peers: avoid an infinite loop with peers_fe is NULL
- BUG/MINOR: peers: Use after free of "peers" section.
- CI: github actions: add weekly h2spec test
- BUG/MEDIUM: mux_h1: Process a new request if we already received it.
- MINOR: build: Fix build in mux_h1
- CLEANUP: remove obsolete comments
- BUG/MEDIUM: dns: improper parsing of aditional records
- MINOR: ssl: skip self issued CA in cert chain for ssl_ctx
- MINOR: listener: add so_name sample fetch
- MEDIUM: stream: support use-server rules with dynamic names
- MINOR: servers: Add a counter for the number of currently used connections.
- MEDIUM: connections: Revamp the way idle connections are killed
- MINOR: cli: add a general purpose pointer in the CLI struct
- MINOR: ssl: add a list of bind_conf in struct crtlist
- REORG: ssl: move SETCERT enum to ssl_sock.h
- BUG/MINOR: ssl: ckch_inst wrongly inserted in crtlist_entry
- REORG: ssl: move some functions above crtlist_load_cert_dir()
- MINOR: ssl: use crtlist_free() upon error in directory loading
- MINOR: ssl: add a list of crtlist_entry in ckch_store
- MINOR: ssl: store a ptr to crtlist in crtlist_entry
- MINOR: ssl/cli: update pointer to store in 'commit ssl cert'
- MEDIUM: ssl/cli: 'add ssl crt-list' command
- REGTEST: ssl/cli: test the 'add ssl crt-list' command
- BUG/MINOR: ssl: entry->ckch_inst not initialized
- REGTEST: ssl/cli: change test type to devel
- REGTEST: make the PROXY TLV validation depend on version 2.2
- CLEANUP: assorted typo fixes in the code and comments
- BUG/MINOR: stats: Fix color of draining servers on stats page
- DOC: internals: Fix spelling errors in filters.txt
- MINOR: connections: Don't mark conn flags 0x00000001 and 0x00000002 as unused.
- REGTEST: make the unique-id test depend on version 2.0
- BUG/MEDIUM: dns: Consider the fact that dns answers are case-insensitive
- MINOR: ssl: split the line parsing of the crt-list
- MINOR: ssl/cli: support filters and options in add ssl crt-list
- MINOR: ssl: add a comment above the ssl_bind_conf keywords
- REGTEST: ssl/cli: tests options and filters w/ add ssl crt-list
- REGTEST: ssl: pollute the crt-list file
- BUG/CRITICAL: hpack: never index a header into the headroom after wrapping
- BUG/MINOR: protocol_buffer: Wrong maximum shifting.
- CLEANUP: src/fd.c: mask setsockopt with DISGUISE
- BUG/MINOR: ssl/cli: initialize fcount int crtlist_entry
- REGTEST: ssl/cli: add other cases of 'add ssl crt-list'
- CLEANUP: assorted typo fixes in the code and comments
- DOC: management: add the new crt-list CLI commands
- BUG/MINOR: ssl/cli: fix spaces in 'show ssl crt-list'
- MINOR: ssl/cli: 'del ssl crt-list' delete an entry
- MINOR: ssl/cli: replace dump/show ssl crt-list by '-n' option
- CI: use better SSL library definition
- CI: travis-ci: enable DEBUG_STRICT=1 for CI builds
- CI: travis-ci: upgrade openssl to 1.1.1f
- MINOR: ssl: improve the errors when a crt can't be open
- CI: cirrus-ci: rename openssl package after it is renamed in FreeBSD
- CI: adopt openssl download script to download all versions
- BUG/MINOR: ssl/cli: lock the ckch structures during crt-list delete
- MINOR: ssl/cli: improve error for bundle in add/del ssl crt-list
- MINOR: ssl/cli: 'del ssl cert' deletes a certificate
- BUG/MINOR: ssl: trailing slashes in directory names wrongly cached
- BUG/MINOR: ssl/cli: memory leak in 'set ssl cert'
- CLEANUP: ssl: use the refcount for the SSL_CTX'
- CLEANUP: ssl/cli: use the list of filters in the crtlist_entry
- BUG/MINOR: ssl: memleak of the struct cert_key_and_chain
- CLEANUP: ssl: remove a commentary in struct ckch_inst
- MINOR: ssl: initialize all list in ckch_inst_new()
- MINOR: ssl: free instances and SNIs with ckch_inst_free()
- MINOR: ssl: replace ckchs_free() by ckch_store_free()
- BUG/MEDIUM: ssl/cli: trying to access to free'd memory
- MINOR: ssl: ckch_store_new() alloc and init a ckch_store
- MINOR: ssl: crtlist_new() alloc and initialize a struct crtlist
- REORG: ssl: move some free/new functions
- MINOR: ssl: crtlist_entry_{new, free}
- BUG/MINOR: ssl: ssl_conf always set to NULL on crt-list parsing
- MINOR: ssl: don't alloc ssl_conf if no option found
- BUG/MINOR: connection: always send address-less LOCAL PROXY connections
- BUG/MINOR: peers: Incomplete peers sections should be validated.
- MINOR: init: report in "haproxy -c" whether there were warnings or not
- MINOR: init: add -dW and "zero-warning" to reject configs with warnings
- MINOR: init: report the compiler version in haproxy -vv
- CLEANUP: assorted typo fixes in the code and comments
- MINOR: init: report the haproxy version and executable path once on errors
- DOC: Make how "option redispatch" works more explicit
- BUILD: Makefile: add linux-musl to TARGET
- CLEANUP: assorted typo fixes in the code and comments
- CLEANUP: http: Fixed small typo in parse_http_return
- DOC: hashing: update link to hashing functions
Bret Mulvey, the author of the article cited in this pulication
has migrated his work to papa.bretmulvey.com. I was able to
view an archival version of Bret M.'s original post
(http://home.comcast.net/~bretm/hash/3.html) and have validated
that this is the same paper that is originally cited.
Other users are using musl, namely on Docker. It builds fine with
linux-glibc-legacy but not linux-glibc, which needs to first disable
USE_BACKTRACE. Better add a valid entry for it instead of hacking
around another libc.
People are often misled and think that this option can redirect
connections to backup servers.
This patch makes the documentation more specific about how the option
handles backup servers.
If haproxy fails to start and emits an alert, then it can be useful
to have it also emit the version and the path used to load it. Some
users may be mistakenly launching the wrong binary due to a misconfigured
PATH variable and this will save them some troubleshooting time when it
reports that some keywords are not understood.
What we do here is that we *try* to extract the binary name from the
AUX vector on glibc, and we report this as a NOTICE tag before the
very first alert is emitted.
Some portability issues were met a few times in the past depending on
compiler versions, but this one was not reported in haproxy -vv output
while it's trivial to add it. This patch tries to be the most accurate
by explicitly reporting the clang version if detected, otherwise the
gcc version.
Since some systems switched to service managers which hide all warnings
by default, some users are not aware of some possibly important warnings
and get caught too late with errors that could have been detected earlier.
This patch adds a new global keyword, "zero-warning" and an equivalent
command-line option "-dW" to refuse to start in case any warning is
detected. It is recommended to use these with configurations that are
managed by humans in order to catch mistakes very early.
This helps quickly checking if the config produces any warning. For
this we reuse the "warned" bit field to add a new WARN_ANY bit that is
set by ha_warning(). The rest of the bit field was also cleaned from
unused bits.
Before supporting "server" line in "peers" section, such sections without
any local peer were removed from the configuration to get it validated.
This patch fixes the issue where a "server" line without address and port which
is a remote peer without address and port makes the configuration parsing fail.
When encoutering such cases we now ignore such lines remove them from the
configuration.
Thank you to Jérôme Magnin for having reported this bug.
Must be backported to 2.1 and 2.0.
