Commit Graph

21140 Commits

Author SHA1 Message Date
Willy Tarreau
db09cd6ad4 [RELEASE] Released version 2.9-dev10
Released version 2.9-dev10 with the following main changes :
    - CLEANUP: Re-apply xalloc_size.cocci (3)
    - BUG/MEDIUM: stconn: Report send activity during mux-to-mux fast-forward
    - BUG/MEDIUM: stconn: Don't report rcv/snd expiration date if SC cannot epxire
    - MINOR: stconn: Don't queue stream task in past in sc_notify()
    - BUG/MEDIUM: Don't apply a max value on room_needed in sc_need_room()
    - BUG/MINOR: stconn: Sanitize report for read activity
    - CLEANUP: htx: Properly indent htx_reserve_max_data() function
    - DOC: stconn: Improve comments about lra and fsb usage
    - BUG/MEDIUM: quic: fix actconn on quic_conn alloc failure
    - BUG/MEDIUM: quic: fix sslconns on quic_conn alloc failure
    - BUG/MEDIUM: mux-h1: Be sure xprt support splicing to use it during fast-forward
    - MINOR: proto_reverse_connect: use connect timeout
    - BUG/MINOR: mux-h1: Release empty ibuf during data fast-forwarding
    - BUG/MINOR: stick-table/cli: Check for invalid ipv4 key
    - MEDIUM: stktable/cli: simplify entry key handling
    - MINOR: stktable/cli: support v6tov4 and v4tov6 conversions
    - BUG/MINOR: mux-h1: Properly handle http-request and http-keep-alive timeouts
    - BUG/MEDIUM: freq-ctr: Don't report overshoot for long inactivity period
    - BUG/MEDIUM: pool: fix releasable pool calculation when overloaded
    - BUG/MINOR: pool: check one other random bucket on alloc conflict
    - BUG/MEDIUM: pool: try once to allocate from another bucket if empty
    - MEDIUM: stconn/muxes: Loop on data fast-forwarding to forward at least a buffer
    - MINOR: stconn/mux-h2: Use a iobuf flag to report EOI to consumer side during FF
    - MEDIUM: quic: Heavy task mode during handshake
    - MEDIUM: quic: Heavy task mode with non contiguously bufferized CRYPTO data
    - MINOR: quic: release the TLS context asap from quic_conn_release()
    - MINOR: quic: Add idle timer task pointer to traces
    - BUG/MINOR: quic: idle timer task requeued in the past
    - CLEANUP: quic: Indentation fix in qc_do_build_pkt()
    - MINOR: quic: Avoid zeroing frame structures
    - BUG/MEDIUM: quic: Too short Initial packet sent (enc. level allocation failed)
    - BUG/MEDIUM: quic: Avoid trying to send ACK frames from an empty ack ranges tree
    - BUG/MEDIUM: quic: Possible crashes when sending too short Initial packets
    - BUG/MEDIUM: quic: Avoid some crashes upon TX packet allocation failures
    - BUG/MEDIUM: quic: Possible crashes during secrets allocations (heavy load)
    - BUG/MEDIUM: stconn: Don't update stream expiration date if already expired
    - MINOR: errors: ha_alert() and ha_warning() uses warn_exec_path()
    - MINOR: errors: does not check MODE_STARTING for log emission
    - MEDIUM: errors: move the MODE_QUIET test in print_message()
    - DOC: management: -q is quiet all the time
    - MEDIUM: mworker: -W is mandatory when using -S
    - BUG/MEDIUM: mux-h1: Exit early if fast-forward is not supported by opposite SC
    - MEDIUM: quic: adjust address validation
    - MINOR: quic: reduce half open counters scope
    - MEDIUM: quic: limit handshake per listener
    - MEDIUM: quic: define an accept queue limit
    - BUG/MINOR: quic: fix retry token check inconsistency
    - MINOR: task/debug: explicitly support passing a null caller to wakeup functions
    - MINOR: task/debug: make task_queue() and task_schedule() possible callers
    - OPTIM: mux-h2: don't allocate more buffers per connections than streams
    - BUG/MINOR: quic: remove dead code in error path
    - MEDIUM: quic: respect closing state even on soft-stop
    - MEDIUM: quic: release conn socket before using quic_cc_conn
    - DOC: config: use the word 'backend' instead of 'proxy' in 'track' description
    - BUG/MEDIUM: applet: Remove appctx from buffer wait list on release
    - MINOR: tools: make str2sa_range() directly return type hints
    - BUG/MEDIUM: server: invalid address (post)parsing checks
    - BUG/MINOR: sink: don't learn srv port from srv addr
    - CLEANUP: sink: bad indent in sink_new_from_logger()
    - CLEANUP: sink: useless leftover in sink_add_srv()
    - BUG/MINOR: quic: Useless use of non-contiguous buffer for in order CRYPTO data
    - MINOR: server: always initialize pp_tlvs for default servers
    - BUG/MEDIUM: proxy: always initialize the default settings after init
    - MEDIUM: startup: 'haproxy -c' is quiet when valid
    - BUG/MINOR: sample: Fix bytes converter if offset is bigger than sample length
    - BUG/MINOR: log: keep the ref in dup_logger()
    - BUG/MINOR: quic: fix crash on qc_new_conn alloc failure
    - BUG/MINOR: quic: fix decrement of half_open counter on qc alloc failure
    - BUG/MEDIUM: quic: fix FD for quic_cc_conn
    - DOC: config: Fix name for tune.disable-zero-copy-forwarding global param
    - REGTESTS: startup: -conf-OK requires -V with current VTest
    - BUG/MEDIUM: quic: Non initialized CRYPTO data stream deferencing
    - MINOR: quic: Add a max window parameter to congestion control algorithms
    - MINOR: quic: Maximum congestion control window for each algo
    - DOC: quic: Wrong syntax for "quic-cc-algo" keyword.
    - DOC: quic: Maximum congestion control window configuration
    - BUG/MINOR: quic: maximum window limits do not match the doc
    - BUG/MEDIUM: connection: report connection errors even when no mux is installed
    - BUG/MINOR: stconn: Handle abortonclose if backend connection was already set up
    - MINOR: connection: Add a CTL flag to notify mux it should wait for reads again
    - MEDIUM: mux-h1: Handle MUX_SUBS_RECV flag in h1_ctl() and susbscribe for reads
    - BUG/MEDIUM: stream: Properly handle abortonclose when set on backend only
    - MINOR: stconn: Use SC to detect frontend connections in sc_conn_recv()
    - REGTESTS: http: Improve script testing abortonclose option
    - MINOR: activity: report profiling duration and age in "show profiling"
    - BUG/MEDIUM: mworker: set the master variable earlier
    - BUG/MEDIUM: stream: Don't call mux .ctl() callback if not implemented
    - MINOR: connection: update rhttp flags usage
    - BUG/MINOR: mux_h2: reject passive reverse conn if error on add to idle
    - MINOR: server: force add to idle on reverse
    - MINOR: shctx: Set last_append to NULL when reserving block in hot list
    - MEDIUM: shctx: Move list between hot and avail list in O(1)
    - MEDIUM: shctx: Simplify shctx_row_reserve_hot loop
    - MINOR: shctx: Remove explicit 'from' param from shctx_row_data_append
    - MEDIUM: cache: Use dedicated cache tree lock alongside shctx lock
    - MINOR: cache: Remove expired entry delete in "show cache" command
    - MINOR: cache: Add option to avoid removing expired entries in lookup function
    - MEDIUM: cache: Use rdlock on cache in cache_use
    - MEDIUM: shctx: Remove 'hot' list from shared_context
    - MINOR: cache: Use dedicated trash for "show cache" cli command
    - MEDIUM: cache: Switch shctx spinlock to rwlock and restrict its scope
    - MEDIUM: cache: Add refcount on cache_entry
    - MEDIUM: shctx: Descend shctx_lock calls into the shctx_row_reserve_hot
    - MINOR: shctx: Add new reserve_finish callback call to shctx_row_reserve_hot
    - MAJOR: cache: Delay cache entry delete in reserve_hot function
    - MINOR: shctx: Remove redundant arg from free_block callback
    - MINOR: shctx: Remove 'use_shared_mem' variable
    - DOC: cache: Specify when function expects a cache lock
    - BUG/MEDIUM: stconn: Update fsb date on partial sends
    - MINOR: htx: Use a macro for overhead induced by HTX
    - MINOR: channel: Add functions to get info on buffers and deal with HTX streams
    - BUG/MINOR: stconn: Fix streamer detection for HTX streams
    - BUG/MINOR: stconn: Use HTX-aware channel's functions to get info on buffer
    - BUG/MINOR: stconn/applet: Report send activity only if there was output data
    - BUG/MINOR: stconn: Report read activity on non-indep streams for partial sends
    - BUG/MINOR: shctx: Remove old HA_SPIN_INIT
    - REGTESTS: try to activate again the seamless reload test with the master CLI
    - MINOR: proxy: Add "handshake" new timeout (frontend side)
    - MEDIUM: quic: Add support for "handshake" timeout setting.
    - MINOR: quic: Dump the expiration date of the idle timer task
    - BUG/MINOR: quic: Malformed CONNECTION_CLOSE frame
    - MEDIUM: session: handshake timeout (TCP)
    - DOC: proxy: Add "handshake" timeout documentation.
    - MINOR: quic: Rename "handshake" timeout to "client-hs"
    - CLEANUP: haproxy: remove old comment from 1.1 from the file header
    - BUG/MEDIUM: mux-h2: fail earlier on malloc in takeover()
    - BUG/MEDIUM: mux-h1: fail earlier on malloc in takeover()
    - BUG/MEDIUM: mux-fcgi: fail earlier on malloc in takeover()
    - MINOR: rhttp: remove the unused outgoing connect() function
    - MINOR: backend: without ->connect(), allow to pick another thread's connection
    - BUG/MINOR: stream/cli: report correct stream age in "show sess"
    - MINOR: stream/cli: add an optional "older" filter for "show sess"
    - MINOR: stream/cli: add another filter "susp" to "show sess"
    - MINOR: stktable: add stktable_deinit function
    - BUG/MINOR: proxy/stktable: missing frees on proxy cleanup
    - CLEANUP: backend: removing unused LB param
    - MEDIUM: lbprm: store algo params on 32bits
    - MEDIUM: log/balance: merge tcp/http algo with log ones
    - Revert "MINOR: proxy: report a warning for max_ka_queue in proxy_cfg_ensure_no_http()"
    - Revert "MINOR: tcp_rules: tcp-{request,response} requires TCP or HTTP mode"
    - Revert "MINOR: stktable: "stick" requires TCP or HTTP mode"
    - Revert "MINOR: cfgparse-listen: "http-send-name-header" requires TCP or HTTP mode"
    - Revert "MINOR: cfgparse-listen: "dynamic-cookie-key" requires TCP or HTTP mode"
    - Revert "MINOR: cfgparse-listen: "http-reuse" requires TCP or HTTP mode"
    - Revert "MINOR: fcgi-app: "use-fcgi-app" requires TCP or HTTP mode"
    - Revert "MINOR: http_htx/errors: prevent the use of some keywords when not in tcp/http mode"
    - Revert "MINOR: flt_http_comp: "compression" requires TCP or HTTP mode"
    - Revert "MINOR: filter: "filter" requires TCP or HTTP mode"
    - MINOR: log/backend: ensure log exclusive params are not used in other modes
    - MINOR: log/backend: prevent tcp-{request,response} use with LOG mode
    - MINOR: log/backend: prevent stick table and stick rules with LOG mode
    - MINOR: log/backend: prevent "http-send-name-header" use with LOG mode
    - MINOR: log/backend: prevent "dynamic-cookie-key" use with LOG mode
    - REGTESTS: http: add a test to validate chunked responses delivery
2023-11-18 12:00:37 +01:00
Willy Tarreau
42faad48f6 REGTESTS: http: add a test to validate chunked responses delivery
I've had this test here never committed over the last 2.5 years, that
works fine and I didn't notice it was not part of the tree. It makes a
server return odd-sized chunked responses with short pauses between half
of thems and verifies they're not truncated on the client. It may detect
eventually state machine breakages, so better commit it.
2023-11-18 11:16:41 +01:00
Aurelien DARRAGON
82f4bcafae MINOR: log/backend: prevent "dynamic-cookie-key" use with LOG mode
It doesn't make sense to set "dynamic-cookie-key" inside a log backend,
thus we report a warning to the user and reset the setting.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
c7783fb32b MINOR: log/backend: prevent "http-send-name-header" use with LOG mode
It doesn't make sense to use the "http-send-name-header" directive inside
a log backend so we report a warning in with case and reset the setting.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
4b2616f784 MINOR: log/backend: prevent stick table and stick rules with LOG mode
Report a warning and prevent errors if user tries to declare a stick table
or use stick rules within a log backend.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
5335618967 MINOR: log/backend: prevent tcp-{request,response} use with LOG mode
We start implementing some postparsing compatibility checks for log
backends.

Here we report a warning if user tries to use tcp-{request,response} rules
with log backend, and we properly ignore such rules when inherited from
defaults section.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
6a29888f60 MINOR: log/backend: ensure log exclusive params are not used in other modes
add proxy_cfg_ensure_no_log() function (similar to
proxy_cfg_ensure_no_http()) to ensure at the end of proxy parsing that
no log exclusive options are found if the proxy is not in log mode.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
42d7d1bd47 Revert "MINOR: filter: "filter" requires TCP or HTTP mode"
This reverts commit f9422551cd since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
c8948fb7ac Revert "MINOR: flt_http_comp: "compression" requires TCP or HTTP mode"
This reverts commit 225526dc16 since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
33e5c4055f Revert "MINOR: http_htx/errors: prevent the use of some keywords when not in tcp/http mode"
This reverts commit b41b77b4cc since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
0f9b475333 Revert "MINOR: fcgi-app: "use-fcgi-app" requires TCP or HTTP mode"
This reverts commit 0ba731f50b since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
7d59730100 Revert "MINOR: cfgparse-listen: "http-reuse" requires TCP or HTTP mode"
This reverts commit 65f1124b5d since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
f1a072d077 Revert "MINOR: cfgparse-listen: "dynamic-cookie-key" requires TCP or HTTP mode"
This reverts commit 0b09727a22 since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
a0a7dd1ee7 Revert "MINOR: cfgparse-listen: "http-send-name-header" requires TCP or HTTP mode"
This reverts commit d354947365 since we
cannot perform the test during parsing as the proxy mode is not yet
known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
c90d7dc46b Revert "MINOR: stktable: "stick" requires TCP or HTTP mode"
This reverts commit 098ae743fd since we
cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
8e20fdbb1c Revert "MINOR: tcp_rules: tcp-{request,response} requires TCP or HTTP mode"
This reverts commit 09b15e4163 since
we cannot perform the test during parsing as the effective proxy mode is
not yet known.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
b6e1e9ec8b Revert "MINOR: proxy: report a warning for max_ka_queue in proxy_cfg_ensure_no_http()"
This reverts commit 3934901 since it makes no sense to report a warning
in this case given that max-keepalive-queue will also work with TCP
backends.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
b61147fd2a MEDIUM: log/balance: merge tcp/http algo with log ones
"log-balance" directive was recently introduced to configure the
balancing algorithm to use when in a log backend. However, it is
confusing and it causes issues when used in default section.

In this patch, we take another approach: first we remove the
"log-balance" directive, and instead we rely on existing "balance"
directive to configure log load balancing in log backend.

Some algorithms such as roundrobin can be used as-is in a log backend,
and for log-only algorithms, they are implemented as "log-$name" inside
the "backend" directive.

The documentation was updated accordingly.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
f42dfaa214 MEDIUM: lbprm: store algo params on 32bits
Make sure lbprm.algo can store 32bits by declaring it as uint32_t

Then, use all 32 available bits to offer 4 extra bits for the BE_LB_NEED
inputs. This will allow new required inputs to be easily added (up to 4
new ones, plus one that wasn't used yet if we keep them exclusive)

This required some cleanup: all ALGO bitfields were rewritten in the
32bits format and the high ones were shifted to make room for the
new BE_LB_NEED bits.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
a327b80f1f CLEANUP: backend: removing unused LB param
BE_LB_HASH_RND was introduced with 760e81d35 ("MINOR: backend: implement
random-based load balancing") but was never used since. Removing it
to regain an extra slot for future types.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
2c4943c18b BUG/MINOR: proxy/stktable: missing frees on proxy cleanup
In 1b8e68e ("MEDIUM: stick-table: Stop handling stick-tables as proxies.")
we forgot to free the table pointer which is now dynamically allocated.

Let's take this opportunity to also fix a missing free in the table itself
(the table expire task wasn't properly destroyed)

This patch depends on:
 - "MINOR: stktable: add sktable_deinit function"

It should be backported in every stable versions.
2023-11-18 11:16:21 +01:00
Aurelien DARRAGON
e10cf61099 MINOR: stktable: add stktable_deinit function
Adding sktable_deinit() helper function to properly cleanup a sticktable
that was initialized using stktable_init().
2023-11-18 11:16:21 +01:00
Willy Tarreau
6c7771f1b4 MINOR: stream/cli: add another filter "susp" to "show sess"
This one reports streams considered as "suspicious", i.e. those with
no expiration dates or dates in the past, or those without a front
endpoint. More criteria could be added in the future.
2023-11-17 19:30:07 +01:00
Willy Tarreau
3ffcf7beb1 MINOR: stream/cli: add an optional "older" filter for "show sess"
It's often needed to be able to refine "show sess" when debugging, and
very often a first glance at old streams is performed, but that's a
difficult task in large dumps, and it takes lots of resources to dump
everything.

This commit adds "older <age>" to "show sess" in order to specify the
minimum age of streams that will be dumped. This should simplify the
identification of blocked ones.
2023-11-17 19:30:04 +01:00
Willy Tarreau
ec76e0138b BUG/MINOR: stream/cli: report correct stream age in "show sess"
Since 2.4-dev2 with commit 15e525f49 ("MINOR: stream: Don't retrieve
anymore timing info from the mux csinfo"), we don't replace the
tv_accept (now accept_ts) anymore with the current request's, so that
it properly reflects the session's accept date and not the request's
date. However, since then we failed to update "show sess" to make use
of the request's timestamp instead of the session's timestamp, resulting
in fantasist values in the "age" field of "show sess" for the task.

Indeed, the session's age is displayed instead of the stream's, which
leads to great confusion when debugging, particularly when it comes to
multiplexed inter-proxy connections which are kept up forever.

Let's fix this now. This must be backported as far as 2.4. However,
for 2.7 and older, the field was named tv_request and was a timeval.
2023-11-17 18:59:12 +01:00
Willy Tarreau
662565ddb4 MINOR: backend: without ->connect(), allow to pick another thread's connection
If less connections than threads are established on a reverse-http gateway
and these servers have a non-nul pool-min-conn, then conn_backend_get()
will refrain from picking available connections from other threads. But
this makes no sense for protocols for which there is no ->connect(),
since there's no way the current thread will manage to establish its own
connection. For such situations we should always accept to use another
thread's connection. That's precisely what this patch does.
2023-11-17 18:13:04 +01:00
Willy Tarreau
f592a0d5dd MINOR: rhttp: remove the unused outgoing connect() function
A dummy connect() function previously had to be installed for the log
server so that a reverse-http address could be referenced on a "server"
line, but after the recent rework of the server line parsing, this is
no longer needed, and this is actually annoying as it makes one believe
there is a way to connect outside, which is not true. Let's now get rid
of this function.
2023-11-17 18:10:16 +01:00
Willy Tarreau
d069825c5f BUG/MEDIUM: mux-fcgi: fail earlier on malloc in takeover()
This is the equivalent of the previous "BUG/MEDIUM: mux-h1: fail earlier
on malloc in takeover()".

Connection takeover was implemented for fcgi in 2.2 by commit a41bb0b6c
("MEDIUM: mux_fcgi: Implement the takeover() method."). It does have one
corner case related to memory allocation failure: in case the task or
tasklet allocation fails, the connection gets released synchronously.

Unfortunately the situation is bad there, because the lower layers are
already switched to the new thread while the tasklet is either NULL or
still the old one, and calling fcgi_release() will also result in
touching the thread-local list of buffer waiters, calling unsubscribe(),
There are even code paths where the thread will try to grab the lock of
its own idle conns list, believing the connection is there while it has
no useful effect. However, if the owner thread was doing the same at the
same moment, and ended up trying to pick from the current thread (which
could happen if picking a connection for a different name), the two
could even deadlock.

No tests were made to try to reproduce the problem, but the description
above is sufficient to see that nothing can guarantee against it.

This patch takes a simple but radically different approach. Instead of
starting to migrate the connection before risking to face allocation
failures, it first pre-allocates a new task and tasklet, then assigns
them to the connection if the migration succeeds, otherwise it just
frees them. This way it's no longer needed to manipulate the connection
until it's fully migrated, and as a bonus this means the connection will
continue to exist and the use-after-free condition is solved at the same
time.

This should be backported to 2.2. Thanks to Fred for the initial analysis
of the problem!
2023-11-17 18:10:16 +01:00
Willy Tarreau
95fd2d6801 BUG/MEDIUM: mux-h1: fail earlier on malloc in takeover()
This is the h1 equivalent of previous "BUG/MEDIUM: mux-h2: fail earlier
on malloc in takeover()".

Connection takeover was implemented for H1 in 2.2 by commit f12ca9f8f1
("MEDIUM: mux_h1: Implement the takeover() method."). It does have one
corner case related to memory allocation failure: in case the task or
tasklet allocation fails, the connection gets released synchronously.

Unfortunately the situation is bad there, because the lower layers are
already switched to the new thread while the tasklet is either NULL or
still the old one, and calling h1_release() will call some unsubscribe
and and possibly other things whose safety is not guaranteed (and the
ambiguity here alone is sufficient to be careful). There are even code
paths where the thread will try to grab the lock of its own idle conns
list, believing the connection is there while it has no useful effect.
However, if the owner thread was doing the same at the same moment, and
ended up trying to pick from the current thread (which could happen if
picking a connection for a different name), the two could even deadlock.

Contrary to mux-h2, a few tests were not sufficient to try to crash the
process, but there's nothing that indicates it couldn't happen based on
the description above.

This patch takes a simple but radically different approach. Instead of
starting to migrate the connection before risking to face allocation
failures, it first pre-allocates a new task and tasklet, then assigns
them to the connection if the migration succeeds, otherwise it just
frees them. This way it's no longer needed to manipulate the connection
until it's fully migrated, and as a bonus this means the connection will
continue to exist and the use-after-free condition is solved at the same
time.

This should be backported to 2.2. Thanks to Fred for the initial analysis
of the problem!
2023-11-17 18:10:16 +01:00
Willy Tarreau
4f02e3da67 BUG/MEDIUM: mux-h2: fail earlier on malloc in takeover()
Connection takeover was implemented for H2 in 2.2 by commit cd4159f03
("MEDIUM: mux_h2: Implement the takeover() method."). It does have one
corner case related to memory allocation failure: in case the task or
tasklet allocation fails, the connection gets released synchronously.
Unfortunately the situation is bad there, because the lower layers are
already switched to the new thread while the tasklet is either NULL or
still the old one, and calling h2_release() will also result in
h2_process() and h2_process_demux() that may process any possibly
pending frames. Even the session remains the old one on the old thread,
so that some sess_log() that are called when facing certain demux errors
will be associated with the previous thread, possibly accessing a number
of elements belonging to another thread. There are even code paths where
the thread will try to grab the lock of its own idle conns list, believing
the connection is there while it has no useful effect. However, if the
owner thread was doing the same at the same moment, and ended up trying
to pick from the current thread (which could happen if picking a connection
for a different name), the two could even deadlock.

The risk is extremely low, but Fred managed to reproduce use-after-free
errors in conn_backend_get() after a takeover() failed by playing with
-dMfail, indicating that h2_release() had been successfully called. In
practise it's sufficient to have h2 on the server side with reuse-always
and to inject lots of request on it with -dMfail.

This patch takes a simple but radically different approach. Instead of
starting to migrate the connection before risking to face allocation
failures, it first pre-allocates a new task and tasklet, then assigns
them to the connection if the migration succeeds, otherwise it just
frees them. This way it's no longer needed to manipulate the connection
until it's fully migrated, and as a bonus this means the connection will
continue to exist and the use-after-free condition is solved at the same
time.

This should be backported to 2.2. Thanks to Fred for the initial analysis
of the problem!
2023-11-17 18:10:16 +01:00
Willy Tarreau
c7a90cc181 CLEANUP: haproxy: remove old comment from 1.1 from the file header
There was still a totally outdated comment speaking about issues
affecting solaris on 1.1.8pre4 (April 2002, 21 year-old)! This
proves that comments in headers are never read, so let's take this
opportunity for also removing the outdated one recommending to read
the "updated" RFC7230.
2023-11-17 18:10:16 +01:00
Frdric Lcaille
888d1dc3dc MINOR: quic: Rename "handshake" timeout to "client-hs"
Use a more specific name for this timeout to distinguish it from a possible
future one on the server side.
Also update the documentation.
2023-11-17 18:09:41 +01:00
Frédéric Lécaille
957085e85d DOC: proxy: Add "handshake" timeout documentation.
Document the "handshake" timeout new setting available one frontend side.
This should at least be helpful for QUIC client connections to prevent
an attacker from refreshing plenty of connections without completing
the handshake step, leading haproxy to consume memory for nothing.
2023-11-17 17:31:42 +01:00
Frédéric Lécaille
373e40f0c1 MEDIUM: session: handshake timeout (TCP)
Adapt session_accept_fd() called on accept() to set the handshake timeout from
"hanshake-timeout" setting if set by configuration. If not set, continue to use
the "client" timeout setting.
2023-11-17 17:31:42 +01:00
Frédéric Lécaille
392640a61b BUG/MINOR: quic: Malformed CONNECTION_CLOSE frame
This bug arrived with this commit:
      MINOR: quic: Avoid zeroing frame structures
Before this latter, the CONNECTION_CLOSE was zeroed, especially the "reason phrase
length".

Restablish this behavior.

No need to backport.
2023-11-17 17:31:42 +01:00
Frédéric Lécaille
953c7dc2b9 MINOR: quic: Dump the expiration date of the idle timer task
This date is shared between the idle timer and hanshake timeout. So, it should be
useful to dump the expiration date of the idle timer task itself, in place of the
idle timer expiration date. This way, the handshake timeout value will be visible
during the handshake from CLI "show quic full" command.
2023-11-17 17:31:42 +01:00
Frédéric Lécaille
e3e0bb90ce MEDIUM: quic: Add support for "handshake" timeout setting.
The idle timer task may be used to trigger the client handshake timeout.
The hanshake timeout expiration date (qc->hs_expire) is initialized when the
connection is allocated. Obviously, this timeout is taken into an account only
during the handshake by qc_idle_timer_do_rearm() whose job is to rearm the idle timer.

The idle timer expiration date could be initialized only one time, then
never updated until the hanshake completes. But this only works if the
handshake timeout is smaller than the idle timer task timeout. If the handshake
timeout is set greater than the idle timeout, this latter may expire before the
handshake timeout.

This patch may have an impact on the L1/C1 interop tests (with heavy packet loss
or corruption). This is why I guess some implementations with a hanshake timeout
support set a big timeout during this test. This is at least the case for ngtcp2
which sets a 180s hanshake timeout! haproxy will certainly have to proceed the
same way if it wants to have a chance to pass this test as before this handshake
timeout.
2023-11-17 17:31:42 +01:00
Frédéric Lécaille
b33eacc523 MINOR: proxy: Add "handshake" new timeout (frontend side)
Add a new timeout for the handshake, on the frontend side only. Such a hanshake
will be typically used for TLS hanshakes during client connections to TLS/TCP or
QUIC frontends.
2023-11-17 17:31:42 +01:00
William Lallemand
3379441e26 REGTESTS: try to activate again the seamless reload test with the master CLI
Since the reload is now synchronous over the master CLI, try to reload
with it. This was a problem before with the signals because it wasn't
possible to wait for the end of the reload before sending the requests.

This activate again this test, we will see if it's more stable or we
will deactivate it again..
2023-11-17 17:11:42 +01:00
Remi Tricot-Le Breton
d5cce92a46 BUG/MINOR: shctx: Remove old HA_SPIN_INIT
The shctx lock was changed from a SPINLOCK to a RWLOCK in commit ed35b94
"MEDIUM: cache: Switch shctx spinlock to rwlock and restrict its scope"
but a SPIN_INIT was left behind.

This patch does not need to be backported.
2023-11-17 16:56:18 +01:00
Christopher Faulet
5ed101e09c BUG/MINOR: stconn: Report read activity on non-indep streams for partial sends
Partial sends is an activity, not a full blocking. Thus a read activity must
be reported for non-independent stream. It is especially important for very
congested stream where full sends are uncommon.

This patch must be backported to 2.8.
2023-11-17 15:36:43 +01:00
Christopher Faulet
7676a2cdf6 BUG/MINOR: stconn/applet: Report send activity only if there was output data
For applets and connection, when a send attempt is performed, we must be
sure to not report a send activity if there was no output data at all before
the attempt.

It is not important for the <fsb> date itself but for the <lra> date for
non-independent stream.

This patch must be backported to 2.8.
2023-11-17 15:36:43 +01:00
Christopher Faulet
ab5ecaa2ea BUG/MINOR: stconn: Use HTX-aware channel's functions to get info on buffer
Some channel function are used to check if the channel's buffer is full, not
empty or if there are input data. However, functions used are not
HTX-aware. So it is not accurate and may prevent some actions to be
performed (However, not sure there are really issues). Because HTX-aware
versions now exist, use them instead.

This patch may be backported as far as 2.2. It relies on

    * "MINOR: channel: Add functions to get info on buffers and deal with HTX streams"
    * "MINOR: htx: Use a macro for overhead induced by HTX"
2023-11-17 15:09:33 +01:00
Christopher Faulet
24409a5caa BUG/MINOR: stconn: Fix streamer detection for HTX streams
Since the HTX was introduced, the streamer detection is broken for HTX
streams because the HTX overhead was not counted in the test to set
CF_STREAMER and CF_STREAMER_FAST flags.

The consequence was that the consumer side was no longer able to send more
than tune.ssl.maxrecord at a time in SSL.

To fix the issue, we now count the HTX overhead of HTX streams to be able to
set CF_STREAMER/CF_STREAMER_FAST flags on a channel.

This patch relies on folloing commits:

  * "MINOR: channel: Add functions to get info on buffers and deal with HTX streams"
  * "MINOR: htx: Use a macro for overhead induced by HTX"

The series must be backported as far as 2.2.
2023-11-17 15:09:17 +01:00
Christopher Faulet
020231ea79 MINOR: channel: Add functions to get info on buffers and deal with HTX streams
This patch adds HXT-aware versions of the functions c_data(), ci_data() and
c_empty(). channel_data() function returns the amount of data in the
channel, channel_input_data() returns the amount of input data and
channel_empty() returns true if the channel's buffer is empty. These
functions handles HTX buffers.

In addition, channel_data_limit() function, still HTX-aware, can be used to
get the maximum absolute amount of data that can be copied in a buffer,
independently on data already present in the buffer.
2023-11-17 15:08:15 +01:00
Christopher Faulet
7393bf7e42 MINOR: htx: Use a macro for overhead induced by HTX
The overhead induced by the HTX format was set to the HTX structure itself
and two HTX blocks. It was set this way to optimize zero-copy during
transfers. This value may (and will) be used at different places. Thus we
now use a macro, called HTX_BUF_OVERHEAD.
2023-11-17 12:13:00 +01:00
Christopher Faulet
b68c579eda BUG/MEDIUM: stconn: Update fsb date on partial sends
The first-send-blocked date was originally designed to save the date of the
first send of a series where some data remain blocked. It was relaxed
recently (3083fd90e "BUG/MEDIUM: stconn: Report a send activity everytime
data were sent") to save the date of the first full blocked send. However,
it is not accurrate.

When all data are sent, the fsb value must be reset to TICK_ETERNITY. When
nothing is sent and if it is not already set, it must be set. But when data
are partially sent, the value must be updated and not reset. Otherwise the
write timeout may be ignored because fsb date is never set.

So, changes brought by the patch above are reverted and
sc_ep_report_blocked_send() was changed to know if some data were sent or
not. This way we are able to update fsb value.
l
This patch must be backported to 2.8.
2023-11-17 12:13:00 +01:00
Remi Tricot-Le Breton
f1f8e2b3df DOC: cache: Specify when function expects a cache lock
Some functions are built on the fact that the cache lock must be already
taken by the caller. This patch adds this information in the functions'
descriptions.
2023-11-16 19:35:10 +01:00
Remi Tricot-Le Breton
45a2ff0f4a MINOR: shctx: Remove 'use_shared_mem' variable
This global variable was used to avoid using locks on shared_contexts in
the unlikely case of nbthread==1. Since the locks do not do anything
when USE_THREAD is not defined, it will be more beneficial to simply
remove this variable and the systematic test on its value in the shared
context locking functions.
2023-11-16 19:35:10 +01:00
Remi Tricot-Le Breton
4fe6c1365d MINOR: shctx: Remove redundant arg from free_block callback
The free_block callback does not get called on blocks that are not row
heads anymore so we don't need too shared_block parameters.
2023-11-16 19:35:10 +01:00