First adjusted some typos in comments inside the function. Second,
change the naming of some variable to reduce confusion.
A special case has been inserted when advance is done inside a GAP block
and this block is the last of the buffer. In this case, the whole buffer
will be emptied, equivalent to a ncb_init() operation.
ncb_is_empty() was plainly incorrect as it directly dereferences the
memory to read offset blocks instead of ncb_read_off(). The result is
undefined.
Also, BUG_ON() statement is wrong when the buffer starts with a data
block. In this case, ncb_head() is not the first gap offset but instead
just random data. The calculated sum in BUG_ON() statement has thus no
meaning and may cause an abort. Adjust this by reorganizing the whole
function. Only the first data block size is read. If and only if not
nul, the first gap size is then checked.
ncb_is_full() has been rewritten to share the same model as
ncb_is_empty().
quic_rx_pkts_del() function removes packets from QUIC RX buffer. In most
cases, the buffer will be emptied after it. In this case, it's useful to
realign it. This will avoid future data wrapping and use of an
unnecessary junk to fill a too small contiguous space.
The quic-conn manages a buffer to store received QUIC packets. When the
buffer wraps, the gap is filled until the end with junk and packets can
be inserted at the start of the buffer.
On the other end, deletion is implemented via quic_rx_pkts_del().
Packets are removed one by one if their refcount is nul. If junk is
found, the buffer is emptied until its wrap.
This seems to work in most cases but a bug was found in a particular
case : on insertion if buffer gap is not at the end of the buffer. In
this case, the gap was filled, which is useless as now the buffer is
full and the packet cannot be inserted. Worst, on deletion, when junk is
removed there is a risk to removed new packets. This can happens in the
following case :
1. buffer contig space is too small, junk is inserted in the middle of
it
2. on quic_rx_pkts_del() invocation, a packet is removed, but not the
next one because its refcount is still positive. When a new packet is
received, it will be stored after the junk.
3. on next quic_rx_pkts_del(), when junk is removed, all contig data is
cleared, with newer packets data too.
This will cause a transfer between a client and haproxy to be stalled.
This can be reproduced with big enough POST requests. I triggered it
with ngtcp2 and 10M of posted data.
Hopefully, the solution of this bug is simple. If contig space is not
big enough to store a packet, but the space is not at the end of the
buffer, no junk is inserted and the packet is dropped as we cannot
buffered it. This ensures that junk is only present at the end of the
buffer and when removed no packets data is purged with it.
In httpclient_applet_init() function, ss_dst variable is always defined
before the call to sockaddr_alloc(). There is no reason to test it.
This patch should fix the issue #1706.
labels used in goto statement was not called in the right order. Thus if
there is an error during the appctx startup, it is possible to leak a task.
This patch should fix the issue #1703. No backport needed.
Even if `unique_id` and `s->unique_id` are identical it is a bit odd to
`isttest()` `unique_id` and then use `s->unique_id` in the call to `http_add_header()`.
This "issue" was introduced in a17e66289c,
because before that commit the function returned the length of the ID, as it
was not an ist.
When loading providers with 'ssl-provider' global options, this
ssl-provider-path option can be used to set the search path that is to
be used by openssl. It behaves the same way as the OPENSSL_MODULES
environment variable.
Depending on the timing, the second client that should be reported as a
client abort during connection attempt ("CC--" termination state) is
sometime logged as a server close ("SC--" termination state) instead. It
happens because sometime the connection failure to the server s1 is detected
by haproxy before the client c2 aborts. There is no retries and the
connection timeout is set to 100ms. So, to work, the client abort must be
performed and detected by haproxy in less than 100ms.
To fix the issue, the c2 client is now routed to a backend with a connection
timeout set to 1 second and 10 retries. It should be large enough to detect
the client aborts (~10s)
In addition, there is another race condition when the script is
started. sometime, server s1 is not stopped when the first client sends its
request. So a barrier was added to be sure it is stopped before starting to
send requests. And we wait to be sure the server is detected as DOWN to
unblock the barrier. It is performed by a dedicated backend with an
healthcheck on the server s1.
This patch should solve issue #1664.
negation or default modifiers are not supported for this option. However,
this was already tested earlier in cfg_parse_listen() function. Thus, when
"http-restrict-req-hdr-names" option is parsed, the keyword modifier is
always equal to KWM_STD. It is useless to test it again at this place.
This patch should solve the issue #1702.
The appctx is already the endpoint target. It is confusing to also use it to
set the endpoint context. So, never set the endpoint ctx when an appctx is
created or attached to an existing conn-stream.
When creating a new applet for peer outgoing connection, we check
the load on each thread. Threads with least applet count are
preferred.
With this solution we avoid a situation when many outgoing
connections run on the same thread causing significant load on
single CPU core.
It is now possible to start an appctx on a thread subset. Some controls were
added here and there. It is forbidden to start a backend appctx on another
thread than the local one. If a frontend appctx is started on another thread
or a thread subset, the applet .init callback function must be defined. This
callback function is responsible to finalize the appctx startup. It can be
performed synchornously. In this case, the appctx is started on the local
thread. It is not really useful but it is valid. Or it can be performed
asynchronously. In this case, .init callback function is called when the
appctx is woken up for the first time. When this happens, the appctx
affinity is set to the current thread to be able to start the session and
the stream.
In the same way than for the tasks, the applets api was changed to be able
to start a new appctx on a thread subset. For now the feature is
disabled. Only appctx_new_here() is working. But it will be possible to
start an appctx on a specific thread or a subset via a mask.
A .init callback function is defined for the peer_applet applet. This
function finishes the appctx startup by calling appctx_finalize_startup()
and its handles the stream customization.
A .init callback function is defined for the sink_forward_applet applet.
This function finishes the appctx startup by calling
appctx_finalize_startup() and its handles the stream customization.
A .init callback function is defined for the httpclient_applet applet. This
function finishes the appctx startup by calling appctx_finalize_startup()
and its handles the stream customization.
A .init callback function is defined for the update_applet applet. This
function finishes the appctx startup by calling appctx_finalize_startup()
and its handles the stream customization.
A .init callback function is defined for the spoe_applet applet. This
function finishes the spoe_appctx initialization. It also finishes the
appctx startup by calling appctx_finalize_startup() and its handles the
stream customization.
A .init callback function is defined for the dns_session_applet applet. This
function finishes the appctx startup by calling appctx_finalize_startup()
and its handles the stream customization.
appctx_free_on_early_error() must be used to release a freshly created
frontend appctx if an error occurred during the init stage. It takes care to
release the stream instead of the appctx if it exists. For a backend appctx,
it just calls appctx_free().
appctx_finalize_startup() may be used to finalize the frontend appctx
startup. It is responsible to create the appctx's session and the frontend
conn-stream. On error, it is the caller responsibility to release the
appctx. However, the session is released if it was created. On success, if
an error is encountered in the caller function, the stream must be released
instead of the appctx.
This function should ease the init stage when new appctx is created.
It is just a helper function that call the .init applet callback function,
if it exists. This will simplify a bit the init stage when a new applet is
started. For now, this callback function is only used when a new service is
started.
The session created for frontend applets is now totally owns by the
corresponding appctx. It means the appctx is now responsible to release
it. This removes the hack in stream_free() about frontend applets to be sure
to release the session.
Applets were moved at the same level than multiplexers. Thus, gradually,
applets code is changed to be less dependent from the stream. With this
commit, the frontend appctx are ready to own the session. It means a
frontend appctx will be responsible to release the session.
If no private key can be found in a bind line's certificate and
ssl-load-extra-files is set to none we end up trying to call
X509_check_private_key with a NULL key, which crashes.
This fix should be backported to all stable branches.
Found with -Wmissing-prototypes:
src/hlua_fcn.c:53:5: fatal error: no previous prototype for function 'hlua_checkboolean' [-Wmissing-prototypes]
int hlua_checkboolean(lua_State *L, int index)
^
src/hlua_fcn.c:53:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
int hlua_checkboolean(lua_State *L, int index)
^
static
1 error generated.
Found with -Wmissing-prototypes:
src/ssl_utils.c:22:5: fatal error: no previous prototype for function 'cert_get_pkey_algo' [-Wmissing-prototypes]
int cert_get_pkey_algo(X509 *crt, struct buffer *out)
^
src/ssl_utils.c:22:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
int cert_get_pkey_algo(X509 *crt, struct buffer *out)
^
static
1 error generated.
When HAProxy is linked to an OpenSSLv3 library, this option can be used
to load a provider during init. You can specify multiple ssl-provider
options, which will be loaded in the order they appear. This does not
prevent OpenSSL from parsing its own configuration file in which some
other providers might be specified.
A linked list of the providers loaded from the configuration file is
kept so that all those providers can be unloaded during cleanup. The
providers loaded directly by OpenSSL will be freed by OpenSSL.
This option can be used to define a default property query used when
fetching algorithms in OpenSSL providers. It follows the format
described in https://www.openssl.org/docs/man3.0/man7/property.html.
It is only available when haproxy is built with SSL support and linked
to OpenSSLv3 libraries.
The random generator initialization needs to be performed before the
chroot but it is not needed before. If we want to add provider
configuration option to the configuration file, they need to be
processed before any call to a crypto-related OpenSSL function.
We can then delay the initialization until after the configuration file
is parsed and processed.
The "http-restrict-req-hdr-names" option can now be set to restrict allowed
characters in the request header names to the "[a-zA-Z0-9-]" charset.
Idea of this option is to not send header names with non-alphanumeric or
hyphen character. It is especially important for FastCGI application because
all those characters are converted to underscore. For instance,
"X-Forwarded-For" and "X_Forwarded_For" are both converted to
"HTTP_X_FORWARDED_FOR". So, header names can be mixed up by FastCGI
applications. And some HAProxy rules may be bypassed by mangling header
names. In addition, some non-HTTP compliant servers may incorrectly handle
requests when header names contain characters ouside the "[a-zA-Z0-9-]"
charset.
When this option is set, the policy must be specify:
* preserve: It disables the filtering. It is the default mode for HTTP
proxies with no FastCGI application configured.
* delete: It removes request headers with a name containing a character
outside the "[a-zA-Z0-9-]" charset. It is the default mode for
HTTP backends with a configured FastCGI application.
* reject: It rejects the request with a 403-Forbidden response if it
contains a header name with a character outside the
"[a-zA-Z0-9-]" charset.
The option is evaluated per-proxy and after http-request rules evaluation.
This patch may be backported to avoid any secuirty issue with FastCGI
application (so as far as 2.2).
Using -Wall reveals several warning when building ncbuf testing API. One
of them was about the signedness mismatch. The other one was with an
incorrect print format.
ncbuf public API functions were not ready to deal with a NCBUF_NULL as
parameter. Strenghten these functions by handling it properly.
Most of the functions will consider the buffer as empty and silently
returns. The only exception is ncb_init(buf) which cannot be called
with a NCBUF_NULL. This seems legitimate to consider this as a bug and
not silently failed in this case.
this change introduce "LIBRESSL_VERSION=latest" semantic, which scans
http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/ and detects latest release.
LIBRESSL_VERSION=2.9.2 is removed from the matrix.
Released version 2.6-dev10 with the following main changes :
- MINOR: ssl: ignore dotfiles when loading a dir w/ ca-file
- MEDIUM: ssl: ignore dotfiles when loading a dir w/ crt
- BUG/MINOR: ssl: Fix typos in crl-file related CLI commands
- MINOR: compiler: add a new macro to set an attribute on an enum when possible
- BUILD: stats: conditionally mark obsolete stats states as deprecated
- BUILD: ssl: work around bogus warning in gcc 12's -Wformat-truncation
- BUILD: debug: work around gcc-12 excessive -Warray-bounds warnings
- BUILD: listener: shut report of possible null-deref in listener_accept()
- BUG/MEDIUM: ssl: fix the gcc-12 broken fix :-(
- DOC: install: update gcc version requirements
- BUILD: makefile: add -Wfatal-errors to the default flags
- BUG/MINOR: server: Make SRV_STATE_LINE_MAXLEN value from 512 to 2kB (2000 bytes).
- BUG/MAJOR: dns: multi-thread concurrency issue on UDP socket
- BUG/MINOR: mux-h2: mark the stream as open before processing it not after
- MINOR: mux-h2: report a trace event when failing to create a new stream
- DOC: configuration: add the httpclient keywords to the global keywords index
- MINOR: quic: Add a debug counter for sendto() errors
- BUG/MINOR: quic: Dropped peer transport parameters
- BUG/MINOR: quic: Wrong unit for ack delay for incoming ACK frames
- MINOR: quic: Congestion controller event trace fix (loss)
- MINOR: quic: Add correct ack delay values to ACK frames
- MINOR: config: Add "cluster-secret" new global keyword
- MINOR: quic-tls: Add quic_hkdf_extract_and_expand() for HKDF
- MINOR: quic: new_quic_cid() code moving
- MINOR: quic: Initialize stateless reset tokens with HKDF secrets
- MINOR: qc_new_conn() rework for stateless reset
- MINOR: quic: Stateless reset token copy to transport parameters
- MINOR: quic: Send stateless reset tokens
- MINOR: quic: Short packets always embed a trailing AEAD TAG
- CLEANUP: quic: wrong use of eb*entry() macro
- CLEANUP: quic: Useless use of pointer for quic_hkdf_extract()
- CLEANUP: quic_tls: QUIC_TLS_IV_LEN defined two times
- MINOR: ncbuf: define non-contiguous buffer
- MINOR: ncbuf: complete API and define block interal abstraction
- MINOR: ncbuf: optimize storage for the last gap
- MINOR: ncbuf: implement insertion
- MINOR: ncbuf: define various insertion modes
- MINOR: ncbuf: implement advance
- MINOR: ncbuf: write unit tests
- BUG/MEDIUM: lua: fix argument handling in data removal functions
- DOC/MINOR: fix typos in the lua-api document
- BUG/MEDIUM: wdt: don't trigger the watchdog when p is unitialized
- MINOR: mux-h1: Add global option accpet payload for any HTTP/1.0 requests
- CLEANUP: mux-h1: Fix comments and error messages for global options
- MINOR: conn_stream: make cs_set_error() work on the endpoint instead
- CLEANUP: mux-h1: always take the endp from the h1s not the cs
- CLEANUP: mux-h2: always take the endp from the h2s not the cs
- CLEANUP: mux-pt: always take the endp from the context not the cs
- CLEANUP: mux-fcgi: always take the endp from the fstrm not the cs
- CLEANUP: mux-quic: always take the endp from the qcs not the cs
- CLEANUP: applet: use the appctx's endp instead of cs->endp
- MINOR: conn_stream: add a pointer back to the cs from the endpoint
- MINOR: mux-h1: remove the now unneeded h1s->cs
- MINOR: mux-h2: make sure any h2s always has an endpoint
- MINOR: mux-h2: remove the now unneeded conn_stream from the h2s
- MINOR: mux-fcgi: make sure any stream always has an endpoint
- MINOR: mux-fcgi: remove the now unneeded conn_stream from the fcgi_strm
- MINOR: mux-quic: remove the now unneeded conn_stream from the qcs
- MINOR: mux-pt: remove the now unneeded conn_stream from the context
- CLEANUP: muxes: make mux->attach/detach take a conn_stream endpoint
- MINOR: applet: replace cs_applet_shut() with appctx_shut()
- MINOR: applet: add appctx_strm() and appctx_cs() to access common fields
- CLEANUP: applet: remove the unneeded appctx->owner
- CLEANUP: conn_stream: merge cs_new_from_{mux,applet} into cs_new_from_endp()
- MINOR: ext-check: indicate the transport and protocol of a server
- BUG/MEDIUM: mux-quic: fix a thinko in the latest cs/endpoint cleanup
- MINOR: tools: improve error message accuracy in str2sa_range
- MINOR: config: make sure never to mix dgram and stream protocols on a bind line
- BUG/MINOR: ncbuf: fix coverity warning on uninit sz_data
- MINOR: xprt_quic: adjust flow-control according to bufsize
- MEDIUM: mux-quic/h3/hq-interop: use ncbuf for bidir streams
- MEDIUM: mux-quic/h3/qpack: use ncbuf for uni streams
- CLEANUP: mux-quic: remove unused fields for Rx
- CLEANUP: quic: remove unused quic_rx_strm_frm
quic_rx_strm_frm type was used to buffered STREAM frames received out of
order. Now the MUX is able to deal directly with these frames and
buffered it inside its ncbuf.
Rx has been simplified since the conversion of buffer to a ncbuf. The
old buffer can now be removed. The frms tree is also removed. It was
used previously to stored out-of-order received STREAM frames. Now the
MUX is able to buffer them directly into the ncbuf.
This commit is the equivalent for uni-streams of previous commit
MEDIUM: mux-quic/h3/hq-interop: use ncbuf for bidir streams
All unidirectional streams data is now handle in MUX Rx ncbuf. The
obsolete buffer is not unused and will be cleared in the following
patches.
Add a ncbuf for data reception on qcs. Thanks to this, the MUX is able
to buffered all received frame directly into the buffer. Flow control
parameters will be used to ensure there is never an overflow.
This change will simplify Rx path with the future deletion of acked
frames tree previously used for frames out of order.
