Interim responses are by definition bodyless. But we must not set the
corresponding HTX start-line flag, beecause the start-line of the final
response is still expected. Setting the flag above too early may lead the
multiplexer on the sending side to consider the message is finished after
the headers of the interim message.
It happens with the H2 multiplexer on frontend side if a "100-Continue" is
received from the server. The interim response is sent and
HTX_SL_F_BODYLESS_RESP flag is evaluated. Then, the headers of the final
response are sent with ES flag, because HTX_SL_F_BODYLESS_RESP flag was seen
too early, leading to a protocol error if the response has a body.
Thanks to grembo for this analysis.
This patch should fix the issue #2587. It must be backported as far as 2.9.
CLASS_CERTCACHE is used to declare CertCache global object, not Regex one
This copy-paste typo introduced was in 30fcca18 ("MINOR: ssl/lua:
CertCache.set() allows to update an SSL certificate file")
Using CertCache.set() from init context wasn't explicitly supported and
caused the process to crash:
crash.lua:
core.register_init(function()
CertCache.set{filename="reg-tests/ssl/set_cafile_client.pem", ocsp=""}
end)
crash.conf:
global
lua-load crash.lua
listen front
bind localhost:9090 ssl crt reg-tests/ssl/set_cafile_client.pem ca-file reg-tests/ssl/set_cafile_interCA1.crt verify none
./haproxy -f crash.conf
[NOTICE] (267993) : haproxy version is 3.0-dev2-640ff6-910
[NOTICE] (267993) : path to executable is ./haproxy
[WARNING] (267993) : config : missing timeouts for proxy 'front'.
| While not properly invalid, you will certainly encounter various problems
| with such a configuration. To fix this, please ensure that all following
| timeouts are set to a non-zero value: 'client', 'connect', 'server'.
[1] 267993 segmentation fault (core dumped) ./haproxy -f crash.conf
This is because in hlua_ckch_set/hlua_ckch_commit_yield, we always
consider that we're being called from a yield-capable runtime context.
As such, hlua_gethlua() is never checked for NULL and we systematically
try to wake hlua->task and yield every 10 instances.
In fact, if we're called from the body or init context (that is, during
haproxy startup), hlua_gethlua() will return NULL, and in this case we
shouldn't care about yielding because it is ok to commit all instances
at once since haproxy is still starting up.
Also, when calling CertCache.set() from a non-yield capable runtime
context (such as hlua fetch context), we kept doing as if the yield
succeeded, resulting in unexpected function termination (operation
would be aborted and the CertCache lock wouldn't be released). Instead,
now we explicitly state in the doc that CertCache.set() cannot be used
from a non-yield capable runtime context, and we raise a runtime error
if it is used that way.
These bugs were discovered by reading the code when trying to address
Svace report documented by @Bbulatov GH #2586.
It should be backported up to 2.6 with 30fcca18 ("MINOR: ssl/lua:
CertCache.set() allows to update an SSL certificate file")
As reported by @Bbulatov in GH #2586, stktable_data_ptr() return value is
used without checking it isn't NULL first, which may happen if the given
type is invalid or not stored in the table.
However, since date_type is set by table_prepare_data_request() right
before cli_io_handler_table() is invoked, date_type is not expected to
be invalid: table_prepare_data_request() normally checked that the type
is stored inside the table. Thus stktable_data_ptr() should not be failing
at this point, so we add a BUG_ON() to indicate that.
In ticket #785, people are still confused about how to use the crt-store
load parameters in a crt-list.
This patch adds an example.
This must be backported in 3.0
In GH issue #2586 @Bbulatov reported a theoretical null-deref in
env_expand() in case there's no memory anymore to expand an environment
variable. The function should return NULL in this case so that the only
caller (str2sa_range) sees it. In practice it may only happen during
boot thus is harmless but better fix it since it's easy. This can be
backported to all versions where this applies.
When parsing tcp-check expect-header, a copy-paste error in the error
message causes the name of the header to be reporetd as the invalid
format string instead of its value. This is really harmless but should
be backported to all versions to help users understand the cause of the
problem when this happens. This was reported in GH issue #2586 by
@Bbulatov.
In GH issue #2586 @Bbulatov reported a bug where the http-check
send-state flag is removed from options instead of options2 when
http-check is disabled. It only has an effect when this option is
set and http-check disabled, where it displays a warning indicating
this will be ignored. The option removed instead is srvtcpka when
this happens. It's likely that both options being so minor, nobody
ever faced it.
This can be backported to all versions.
This patch removes the old README file and replaces it with a more
modern markdown version which allows clickable links on the github page.
It also adds some of the Github Actions worfklow Status.
This patch includes the HAProxy png in the doc directory.
Released version 3.0.0 with the following main changes :
- MINOR: sample: implement the uptime sample fetch
- CI: scripts: fix build of vtest regarding option -C
- CI: scripts: build vtest using multiple CPUs
- MINOR: log: rename 'log-format tag' to 'log-format alias'
- DOC: config: document logformat item naming and typecasting features
- BUILD: makefile: yearly reordering of objects by build time
- BUILD: fd: errno is also needed without poll()
- DOC: config: fix two typos "RST_STEAM" vs "RST_STREAM"
- DOC: config: refer to the non-deprecated keywords in ocsp-update on/off
- DOC: streamline http-reuse and connection naming definition
- REGTESTS: complete http-reuse test with pool-conn-name
- DOC: config: add %ID logformat alias alternative
- CLEANUP: ssl/ocsp: readable ifdef in ssl_sock_load_ocsp
- BUG/MINOR: ssl/ocsp: init callback func ptr as NULL
- CLEANUP: ssl_sock: move dirty openssl-1.0.2 wrapper to openssl-compat
- BUG/MINOR: activity: fix Delta_calls and Delta_bytes count
- CI: github: upgrade the WolfSSL job to 5.7.0
- DOC: install: update quick build reminders with some missing options
- DOC: install: update the range of tested openssl version to cover 3.3
- DEV: patchbot: prepare for new version 3.1-dev
- MINOR: version: mention that it's 3.0 LTS now.
The quick build reminders claimed to present "all options" but were
still missing QUIC. It was also the moment to split FreeBSD and
OpenBSD apart since the latter uses LibreSSL and does not require
the openssl compatibility wrapper. We also replace the hard-coded
number of cpus for the parallel build, by the real number reported
by the system.
Thanks to the commit 5714aff4a6
"DEBUG: pool: store the memprof bin on alloc() and update it on free()", the
amount of memory allocations and memory "frees" is shown now on the same line,
corresponded to the caller name. This is very convenient to debug memory leaks
(haproxy should run with -dMcaller option).
The implicit drawback of this solution is that we count twice same free_calls
and same free_tot (bytes) values in cli_io_handler_show_profiling(), when
we've calculed tot_free_calls and tot_free_bytes, by adding them to the these
totalizators for p_alloc, malloc and calloc allocator types. See the details
about why this happens in a such way in __pool_free() implementation and
also in the commit message for 5714aff4a6.
This double addition of free counters falses 'Delta_calls' and 'Delta_bytes',
sometimes we even noticed that they show negative values.
Same problem was with the calculation of average allocated buffer size for
lines, where we show simultaneously the number of allocated and freed bytes.
Valentine noticed this ugly SSL_CTX_get_tlsext_status_cb() macro
definition inside ssl_sock.c that is dedicated to openssl-1.0.2 only.
It would be better placed in openssl-compat.h, which is what this
patch does. It also addresses a missing pair of parenthesis and
removes an invalid extra semicolon.
In ssl_sock_load_ocsp() it is better to initialize local scope variable
'callback' function pointer as NULL, while we are declaring it. According to
SSL_CTX_get_tlsext_status_cb() API, then we will provide a pointer to this
'on stack' variable in order to check, if the callback was already set before:
OpenSSL 1.x.x and 3.x.x:
long SSL_CTX_get_tlsext_status_cb(SSL_CTX *ctx, int (**callback)(SSL *, void *));
long SSL_CTX_set_tlsext_status_cb(SSL_CTX *ctx, int (*callback)(SSL *, void *));
WolfSSL 5.7.0:
typedef int(*tlsextStatusCb)(WOLFSSL* ssl, void*);
WOLFSSL_API int wolfSSL_CTX_get_tlsext_status_cb(WOLFSSL_CTX* ctx, tlsextStatusCb* cb);
WOLFSSL_API int wolfSSL_CTX_set_tlsext_status_cb(WOLFSSL_CTX* ctx, tlsextStatusCb cb);
When this func ptr variable stays uninitialized, haproxy comipled with ASAN
crushes in ssl_sock_load_ocsp():
./haproxy -d -f haproxy.cfg
...
AddressSanitizer:DEADLYSIGNAL
=================================================================
==114919==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x5eab8951bb32 bp 0x7ffcdd6d8410 sp 0x7ffcdd6d82e0 T0)
==114919==The signal is caused by a READ memory access.
==114919==Hint: address points to the zero page.
#0 0x5eab8951bb32 in ssl_sock_load_ocsp /home/vk/projects/haproxy/src/ssl_sock.c:1248:22
#1 0x5eab89510d65 in ssl_sock_put_ckch_into_ctx /home/vk/projects/haproxy/src/ssl_sock.c:3389:6
...
This happens, because callback variable is allocated on the stack. As not
being explicitly initialized, it may contain some garbage value at runtime,
due to the linked crypto library update or recompilation.
So, following ssl_sock_load_ocsp code, SSL_CTX_get_tlsext_status_cb() may
fail, callback will still contain its initial garbage value,
'if (!callback) {...' test will put us on the wrong path to access some
ocsp_cbk_arg properties via its pointer, which won't be set and like this
we will finish with segmentation fault.
Must be backported in all stable versions. All versions does not have
the ifdef, the previous cleanup patch is useful starting from the 2.7
version.
Due to the support of different TLS/SSL libraries and its different versions,
sometimes we are forced to use different internal typedefs and callback
functions. We strive to avoid this, but time to time "#ifdef... #endif"
become inevitable.
In particular, in ssl_sock_load_ocsp() we define a 'callback' variable, which
will contain a function pointer to our OCSP stapling callback, assigned
further via SSL_CTX_set_tlsext_status_cb() to the intenal SSL context
struct in a linked crypto library.
If this linked crypto library is OpenSSL 1.x.x/3.x.x, for setting and
getting this callback we have the following API signatures
(see doc/man3/SSL_CTX_set_tlsext_status_cb.pod):
long SSL_CTX_get_tlsext_status_cb(SSL_CTX *ctx, int (**callback)(SSL *, void *));
long SSL_CTX_set_tlsext_status_cb(SSL_CTX *ctx, int (*callback)(SSL *, void *));
If we are using WolfSSL, same APIs expect tlsextStatusCb function prototype,
provided via the typedef below (see wolfssl/wolfssl/ssl.h):
typedef int(*tlsextStatusCb)(WOLFSSL* ssl, void*);
WOLFSSL_API int wolfSSL_CTX_get_tlsext_status_cb(WOLFSSL_CTX* ctx, tlsextStatusCb* cb);
WOLFSSL_API int wolfSSL_CTX_set_tlsext_status_cb(WOLFSSL_CTX* ctx, tlsextStatusCb cb);
It seems, that in OpenSSL < 1.0.0, there was no support for OCSP extention, so
no need to set this callback.
Let's avoid #ifndef... #endif for this 'callback' variable definition to keep
things clear. #ifndef... #endif are usually less readable, than
straightforward "#ifdef... #endif".
Add new test cases in http_reuse_conn_hash vtest. Ensure new server
parameter "pool-conn-name" is used as expected for idle connection name,
both alone and mixed with a SNI.
With the introduction of "pool-conn-name", documentation related to
http-reuse was rendered more complex than already, notably with multiple
cross-references between "pool-conn-name" and "sni" server keywords.
Took the opportunity to improve all http-reuse related documentation.
First, "http-reuse" keyword general purpose has been greatly expanded
and reordered.
Then, "pool-conn-name" and "sni" have been clarified, in particular the
relation between them, with the foremost being an advanced usage to the
default SSL SNI case in the context of http-reuse. Also update
attach-srv rule documentation as its name parameter is directly linked
to both "pool-conn-name" and "sni".
Some large files have been split since 2.9 (e.g. stats) and build times
have moved and become less smooth, causing a less even parallel build.
As usual, a small reordering cleans all this up. The effect was less
visible than previous years though.
The ability to give a name to a logformat_node (known as logformat item in
the documentation) implemented in 2ed6068f2a ("MINOR: log: custom name for
logformat node") wasn't documented.
The same goes for the ability to force the logformat_node's output type to
a specific type implemented in 1448478d62 ("MINOR: log: explicit
typecasting for logformat nodes")
Let's quickly describe such new usages at the start of the custom log
format section.
In 2.9 we started to introduce an ambiguity in the documentation by
referring to historical log-format variables ('%var') as log-format
tags in 739c4e5b1e ("MINOR: sample: accept_date / request_date return
%Ts / %tr timestamp values") and 454c372b60 ("DOC: configuration: add
sample fetches for timing events").
In fact, we've had this confusion between log-format tag and log-format
var for more than 10 years now, but in 2.9 it was the first time the
confusion was exposed in the documentation.
Indeed, both 'log-format variable' and 'log-format tag' actually refer
to the same feature (that is: '%B' and friends that can be used for
direct access to some log-oriented predefined fetches instead of using
%[expr] with generic sample expressions).
This feature was first implemented in 723b73ad75 ("MINOR: config: Parse
the string of the log-format config keyword") and later documented in
4894040fa ("DOC: log-format documentation"). At that time, it was clear
that we used to name it 'log-format variable'.
But later the same year, 'log-format tag' naming started to appear in
some commit messages (while still referring to the same feature), for
instance with ffc3fcd6d ("MEDIUM: log: report SSL ciphers and version
in logs using logformat %sslc/%sslv").
Unfortunately in 2.9 when we added (and documented) new log-format
variables we officially started drifting to the misleading 'log-format
tag' naming (perhaps because it was the most recent naming found for
this feature in git log history, or because the confusion has always
been there)
Even worse, in 3.0 this confusion led us to rename all 'var' occurrences
to 'tag' in log-format related code to unify the code with the doc.
Hopefully William quickly noticed that we made a mistake there, but
instead of reverting to historical naming (log-format variable), it was
decided that we must use a different name that is less confusing than
'tags' or 'variables' (tags and variables are keywords that are already
used to designate other features in the code and that are not very
explicit under log-format context today).
Now we refer to '%B' and friends as a logformat alias, which is
essentially a handy way to print some log oriented information in the
log string instead of leveraging '%[expr]' with generic sample expressions
made of fetches and converters. Of course, there are some subtelties, such
as a few log-format aliases that still don't have sample fetch equivalent
for historical reasons, and some aliases that may be a little faster than
their generic sample expression equivalents because most aliases are
pretty much hardcoded in the log building function. But in general
logformat aliases should be simply considered as an alternative to using
expressions (with '%[expr']')
Also, under log-format context, when we want to refer to either an alias
('%alias') or an expression ('%[expr]'), we should use the generic term
'logformat item', which in fact designates a single item within the
logformat string provided by the user. Indeed, a logformat item (whether
is is an alias or an expression) always starts with '%' and may accept
optional flags / arguments
Both the code and the documentation were updated in that sense, hopefully
this will clarify things and prevent future confusions.
On Linux, GNU make emits "w" at the beginning of the MAKEFLAGS
variable if -C is passed, which happens since vtest d6d228bcb3.
In fact it emits any of the command line flags without the leading
'-' in this case. gmake doesn't do that on BSD apparently. It's
documented under Options/Recursion in the GNU make doc. There's
also MFLAGS that could work but it does not contain the variables
definitions. So let's just avoid the -C that we don't really need.
This needs to be backported to stable versions.
Released version 3.0-dev13 with the following main changes :
- CLEANUP: ssl/cli: remove unused code in dump_crtlist_conf
- MINOR: ssl: check parameter in ckch_conf_cmp()
- BUG/MINOR: ring: free ring's allocated area not ring's usable area when using maps
- DOC: configuration: rework the crt-store load documentation
- DEBUG: tools: add vma_set_name() helper
- DEBUG: shctx: name shared memory using vma_set_name()
- DEBUG: sink: add name hint for memory area used by memory-backed sinks
- DEBUG: pollers: add name hint for large memory areas used by pollers
- DEBUG: errors: add name hint for startup-logs memory area
- DEBUG: fd: add name hint for large memory areas
- MEDIUM: ssl: don't load file by discovering them in crt-store
- DOC: configuration: update the crt-list documentation
- DOC: configuration: add the supported crt-store options in crt-list
- BUG/MEDIUM: proto: fix fd leak in <proto>_connect_server
- MINOR: sock: set conn->err_code in case of EPERM
- BUG/MINOR: http-ana: Don't crush stream termination condition on internal error
- MAJOR: spoe: Let the SPOE back into the game
- BUG/MINOR: connection: parse PROXY TLV for LOCAL mode
- BUG/MINOR: server: free PROXY v2 TLVs on srv drop
- MINOR: rhttp: add log on connection allocation failure
- BUG/MEDIUM: rhttp: fix preconnect on single-thread
- BUG/MINOR: rhttp: prevent listener suspend
- BUG/MINOR: rhttp: fix task_wakeup state
- MINOR: session: define flag to explicitely release listener on free
- MEDIUM: rhttp: create session for active preconnect
- MINOR: rhttp: support PROXY emission on preconnect
- MINOR: connection: support PROXY v2 TLV emission without stream
- MINOR: traces: enumerate the list of levels/verbosities when not found
- BUG/MINOR: sock: fix sock_create_server_socket
- MINOR: proto: fix coding style
- BUG/MAJOR: quic: Crash with TLS_AES_128_CCM_SHA256 (libressl only)
- REGTESTS: scripts: allow to change the vtest timeout
- BUG/MEDIUM: quic_tls: prevent LibreSSL < 4.0 from negotiating CHACHA20_POLY1305
- CI: scripts/build-ssl.sh: loudly fail on unsupported platforms
- BUG/MEDIUM: mux-quic: Create sedesc in same time of the QUIC stream
- MINOR: mux-quic: Set abort info for SC-less QCS on STOP_SENDING frame
- CI: scripts/build-ssl: add a DESTDIR and TMPDIR variable
- CI: scripts/buil-ssl: cleanup the boringssl and quictls build
- MINOR: config: add thread-hard-limit to set an upper bound to nbthread
- BUILD: quic: fix unused variable warning when threads are disabled
- BUG/MEDIUM: stick-tables: Fix race with peers when trashing oldest entries
- BUG/MEDIUM: stick-tables: Fix race with peers when killing a sticky session
- BUG/MEDIUM: stick-tables: make sure never to create two same remote entries
- CLEANUP: stick-tables: remove a few unneeded tests for use_wrlock
- MINOR: stick-tables: remove the uneeded read lock in stksess_free()
- CLEANUP: tools: fix vma_set_name() function comment
- DEBUG: tools: add vma_set_name_id() helper
- DEBUG: pollers/fd: add thread id suffix to per-thread memory areas name hints
- DOC: config: fix aes_gcm_enc() description text
- BUILD: trace: fix warning on null dereference
- MEDIUM: config: prevent communication with privileged ports
- MAJOR: config: prevent QUIC with clients privileged port by default
- BUG/MINOR: quic: adjust restriction for stateless reset emission
- MINOR: quic: clarify doc for quic_recv()
- MINOR: server: generalize sni expr parsing
- MINOR: server: define pool-conn-name keyword
- MEDIUM: connection: use pool-conn-name instead of sni on reuse
- BUG/MINOR: rhttp: initialize session origin after preconnect reversal
- BUG/MEDIUM: server/dns: preserve server's port upon resolution timeout or error
- BUG/MINOR: http-htx: Support default path during scheme based normalization
- BUG/MINOR: server: Don't reset resolver options on a new default-server line
- DOC: quic: specify that connection migration is not supported
- DOC: config: fix incorrect section reference about custom log format
- DOC: config: uniformize the naming and description of custom log format args
- DOC: config: clarify the fact that custom log format is not just for logging
- REGTESTS: acl_cli_spaces: avoid a warning caused by undefined logs
There's a warning being reported in this reg test in the detailed startup
logs because of "log global" and "option httplog" while there's no global
section hence no logger. Let's just drop both options since they're not
relevant to this test.
The wording in the Custom log format section was still extremely centered
on logging, but it's about time to mention that these are usable for other
actions as well, otherwise it's very confusing for newcomers who try to
define a variable or header. The updated text also reminds about the risks
of safe encodings that may (rarely) mangle an output string, and encourages
to migrate away from the unquoted definition which is full of backslashes.
It would definitely deserve further improvements and refinements.
A significant number of actions now take arguments that are evaluated as
log-format expressions. Some of them are called "fmt", others "string".
The description of the argument sometimes just says "the log-format
string" or "log format" or "custom log format" etc. Most of them do not
mention the section to visit, and section 8.2 speaking about log-format
is very centric on logs usage (the primary use case), making all of this
very confusing for newcomers.
Since section 8.2.6 is titled "Custom log format" and describes the syntax
to be used with the "log-format" (and other) directives, let's call this
"Custom log format" everywhere and mention section 8.2.6. When the field
was called "string", it was also renamed to "fmt".
It doesn't seem worth backporting this, unless it applies fine.
Since 2.5 with commit 98b930d043 ("MINOR: ssl: Define a default https
log format"), some log-format sections were shifted a bit without having
been renumberred, causing 8.2.4 to be referenced as the custom log
format while it's in fact 8.2.6. This patch fixes the affected
locations.
In addition two places mentioned 8.2.6 instead of 8.2.5 for the error
log format.
This can be backported to 2.6.
Currently haproxy does not support QUIC connection migration. This is
advertized to clients on their connections. Document this in the first
QUIC related paragraph.
This should be backported up to 2.6.
