Commit Graph

22784 Commits

Author SHA1 Message Date
Frederic Lecaille
4abaadd842 MINOR: quic: Dump TX in flight bytes vs window values ratio.
Display the ratio of the numbers of bytes in flight by packet number spaces
versus the current window values in percent.
2024-07-26 16:42:44 +02:00
Frederic Lecaille
76ff8afa2d MINOR: quic: Add information to "show quic" for CUBIC cc.
Add ->state_cli() new callback to quic_cc_algo struct to define a
function called by the "show quic (cc|full)" commands to dump some information
about the congestion algorithm internal state currently in use by the QUIC
connections.

Implement this callback for CUBIC algorithm to dump its internal variables:
   - K: (the time to reach the cubic curve inflexion point),
   - last_w_max: the last maximum window value reached before intering
     the last recovery period. This is also the window value at the
     inflexion point of the cubic curve,
   - wdiff: the difference between the current window value and last_w_max.
     So negative before the inflexion point, and positive after.
2024-07-26 16:42:44 +02:00
Willy Tarreau
2dab1ba84b MEDIUM: h1: allow to preserve keep-alive on T-E + C-L
In 2.5-dev9, commit 631c7e866 ("MEDIUM: h1: Force close mode for invalid
uses of T-E header") enforced a recently arrived new security rule in the
HTTP specification aiming at preventing a class of content-smuggling
attacks involving HTTP/1.0 agents. It consists in handling the very rare
T-E + C-L requests or responses in close mode.

It happens it does have an impact of a rare few and very old clients
(probably running insecure TLS stacks by the way) that continue to send
both with their POST requests. The impact is that for each and every
request they'll have to reconnect, possibly negotiating a full TLS
handshake that becomes harmful to the machine in terms of CPU computation.

This commit adds a new option "h1-do-not-close-on-insecure-transfer-encoding"
that does exactly what it says, it just asks not to close on such messages,
even though the message continues to be sanitized and C-L dropped. It means
that the risk is only between the sender and haproxy, which is limited, and
might be the only acceptable solution for such environments having to deal
with broken implementations.

The cases are so rare that it should not need to be backported, or in the
worst case, to the latest LTS if there is any demand.
2024-07-26 15:59:35 +02:00
Amaury Denoyelle
85131f91bf BUG/MEDIUM: quic: fix invalid conn reject with CONNECTION_REFUSED
quic-initial rules were implemented just recently. For some actions, a
new flags field was added in quic_dgram structure. This is used to
report the result of the rules execution.

However, this flags field was left uninitialized. Depending on its
value, it may close the connection to be wrongly rejected via
CONNECTION_REFUSED. Fix this by properly set flags value to 0.

No need to backport.
2024-07-26 15:24:35 +02:00
Amaury Denoyelle
08515af9df MINOR: quic: implement send-retry quic-initial rules
Define a new quic-initial "send-retry" rule. This allows to force the
emission of a Retry packet on an initial without token instead of
instantiating a new QUIC connection.
2024-07-25 15:39:39 +02:00
Amaury Denoyelle
69d7e9f3b7 MINOR: quic: implement reject quic-initial action
Define a new quic-initial action named "reject". Contrary to dgram-drop,
the client is notified of the rejection by a CONNECTION_CLOSE with
CONNECTION_REFUSED error code.

To be able to emit the necessary CONNECTION_CLOSE frame, quic_conn is
instantiated, contrary to dgram-drop action. quic_set_connection_close()
is called immediatly after qc_new_conn() which prevents the handshake
startup.
2024-07-25 15:39:39 +02:00
Amaury Denoyelle
f91be2657e MINOR: quic: pass quic_dgram as obj_type for quic-initial rules
To extend quic-initial rules, pass quic_dgram instance to argument for
the various actions. As such, quic_dgram is now supported as an obj_type
and can be used in session origin field.
2024-07-25 15:39:39 +02:00
Amaury Denoyelle
1259700763 MINOR: quic: support ACL for quic-initial rules
Add ACL condition support for quic-initial rules. This requires the
extension of quic_parse_quic_initial() to parse an extra if/unless
block.

Only layer4 client samples are allowed to be used with quic-initial
rules. However, due to the early execution of quic-initial rules prior
to any connection instantiation, some samples are non supported.

To be able to use the 4 described samples, a dummy session is
instantiated before quic-initial rules execution. Its src and dst fields
are set from the received datagram values.
2024-07-25 15:39:39 +02:00
Amaury Denoyelle
cafe596608 MEDIUM: quic: implement quic-initial rules
Implement a new set of rules labelled as quic-initial.

These rules as specific to QUIC. They are scheduled to be executed early
on Initial packet parsing, prior a new QUIC connection instantiation.
Contrary to tcp-request connection, this allows to reject traffic
earlier, most notably by avoiding unnecessary QUIC SSL handshake
processing.

A new module quic_rules is created. Its main function
quic_init_exec_rules() is called on Initial packet parsing in function
quic_rx_pkt_retrieve_conn().

For the moment, only "accept" and "dgram-drop" are valid actions. Both
are final. The latter drops silently the Initial packet instead of
allocating a new QUIC connection.
2024-07-25 15:39:39 +02:00
Amaury Denoyelle
a72e82c382 MINOR: quic: delay Retry emission on quic-force-retry
Currently, quic Retry packets are emitted for two different reasons
after processing an Initial without token :
- quic-force-retry is set on bind-line
- an abnormal number of half-open connection is currently detected

Previously, these two conditions were checked separately in different
functions during datagram parsing. Uniformize this by moving
quic-force-retry check in quic_rx_pkt_retrieve_conn() along the second
condition check.

The purpose of this patch is to uniformize datagram parsing stages. It
is necessary to implement quic-initial rules in
quic_rx_pkt_retrieve_conn() prior to any Retry emission. This prevents
to emit unnecessary Retry if an Initial is subject to a reject rule.
2024-07-25 15:29:50 +02:00
Aurelien DARRAGON
e328056ddc MEDIUM: sink: assume sft appctx stickiness
As mentioned in b40d804 ("MINOR: sink: add some comments about sft->appctx
usage in applet handlers"), there are few places in the code where it
looks like we assumed that the applet callbacks such as
sink_forward_session_init() or sink_forward_io_handler() could be
executing an appctx whose sft is detached from the appctx
(appctx != sft->appctx).

In practise this should not be happening since an appctx sticks to the
same thread its entire lifetime, and the only times sft->appctx is
effectively assigned is during the session/appctx creation (in
process_sink_forward()) or release.

Thus if sft->appctx wouldn't point to the appctx that the sft was bound
to after appctx creation, it would probably indicate a bug rather than
an expected condition. To further emphasize that and prevent the
confusion, and since 3.1-dev4 was released, let's remove such checks and
instead add a BUG_ON to ensure this never happens.

In _sink_forward_io_handler(), the "hard_close" label was removed since
there are no more uses for it (no hard errors may be caught from the
function for now)
2024-07-25 14:56:19 +02:00
William Lallemand
28cb01f8e8 MEDIUM: quic: implement CHACHA20_POLY1305 for AWS-LC
With AWS-LC, the aead part is covered by the EVP_AEAD API which
provides the correct EVP_aead_chacha20_poly1305(), however for header
protection it does not provides an EVP_CIPHER for chacha20.

This patch implements exceptions in the header protection code and use
EVP_CIPHER_CHACHA20 and EVP_CIPHER_CTX_CHACHA20 placeholders so we can
use the CRYPTO_chacha_20() primitive manually instead of the EVP_CIPHER
API.

This requires to check if we are using EVP_CIPHER_CTX_CHACHA20 when
doing EVP_CIPHER_CTX_free().
2024-07-25 13:45:39 +02:00
William Lallemand
177c84808c MEDIUM: quic: add key argument to header protection crypto functions
In order to prepare the code for using Chacha20 with the EVP_AEAD API,
both quic_tls_hp_decrypt() and quic_tls_hp_encrypt() need an extra key
argument.

Indeed Chacha20 does not exists as an EVP_CIPHER in AWS-LC, so the key
won't be embedded into the EVP_CIPHER_CTX, so we need an extra parameter
to use it.
2024-07-25 13:45:39 +02:00
William Lallemand
d55a297b85 MINOR: quic: rename confusing wording aes to hp
Some of the crypto functions used for headers protection in QUIC are
named with an "aes" name even thought they are not used for AES
encryption only.

This patch renames these "aes" to "hp" so it is clearer.
2024-07-25 13:45:38 +02:00
William Lallemand
31c831e29b MEDIUM: ssl/quic: implement quic crypto with EVP_AEAD
The QUIC crypto is using the EVP_CIPHER API in order to achieve
authenticated encryption, this was the API which was used with OpenSSL.
With libraries that inspires from BoringSSL (libreSSL and AWS-LC), the
AEAD algorithms are implemented using the EVP_AEAD API.

This patch converts the call to the EVP_CIPHER API when called in the
contex of AEAD cryptography for QUIC.

The patch defines some QUIC_AEAD macros that can be either EVP_CIPHER or
EVP_AEAD depending on the library.

This was mainly done for AWS-LC but this could be useful for other
libraries. This should finally allow to use CHACHA20_POLY1305 with
AWS-LC.

This patch allows to use the following ciphers with the EVP_AEAD API:
- TLS1_3_CK_AES_128_GCM_SHA256
- TLS1_3_CK_AES_256_GCM_SHA384

AWS-LC does not implement TLS1_3_CK_AES_128_CCM_SHA256 and
TLS1_3_CK_CHACHA20_POLY1305_SHA256 requires some hack for headers
protection which will come in another patch.
2024-07-25 13:45:38 +02:00
Frederic Lecaille
a6d40e09f7 BUG/MINOR: quic: Lack of precision when computing K (cubic only cc)
K cubic variable is stored in ms. But it was a formula with the second as unit
for the window difference parameter which was used to compute K without
considering the loss of information. Then the result was converted in ms (K *= 1000).
This leaded to a lack of precision and multiples of 1000 as values.

To fix this, use the same formula but with the window difference in ms as parameter
passed to the cubic function and remove the conversion.

Must be backported as far as 2.6.
2024-07-24 18:24:39 +02:00
Willy Tarreau
7eca16921b [RELEASE] Released version 3.1-dev4
Released version 3.1-dev4 with the following main changes :
    - MINOR: limits: prepare to keep limits in one place
    - REORG: fd: move raise_rlim_nofile to limits
    - CLEANUP: fd: rm struct rlimit definition
    - REORG: global: move rlim_fd_*_at_boot in limits
    - MINOR: haproxy: prepare to move limits-related code
    - REORG: haproxy: move limits handlers to limits
    - MINOR: limits: add is_any_limit_configured
    - CLEANUP: quic: remove obsolete comment on send
    - MINOR: quic: extend detection of UDP API OS features
    - MINOR: quic: activate UDP GSO for QUIC if supported
    - MINOR: quic: define quic_cc_path MTU as constant
    - MINOR: quic: add GSO parameter on quic_sock send API
    - MAJOR: quic: support GSO when encoding datagrams
    - MEDIUM: quic: implement GSO fallback mechanism
    - MINOR: quic: add counters of sent bytes with and without GSO
    - BUG/MEDIUM: bwlim: Be sure to never set the analyze expiration date in past
    - CLEANUP: proto: rename TID affinity callbacks
    - CLEANUP: quic: rename TID affinity elements
    - BUG/MINOR: limits: fix license type in limits.h
    - BUG/MINOR: session: Eval L4/L5 rules defined in the default section
    - CLEANUP: stconn: Fix a typo in comments for SE_ABRT_SRC_*
    - MEDIUM: spoe: Remove fragmentation support
    - MEDIUM: spoe: Remove async mode support
    - MINOR: spoe: Use only a global engine-id per agent
    - MINOR: spoe: Remove debugging
    - MAJOR: spoe: Remove idle applets and pipelining support
    - MINOR: spoe: Remove the dedicated SPOE applet task
    - MEDIUM: proxy/spoe: Add a SPOP mode
    - MEDIUM: applet: Add a .shut callback function for applets
    - MINOR: connection: No longer include stconn type header in connection-t.h
    - MINOR: stconn: Use a dedicated function to get the opposite sedesc
    - MINOR: spoe: Rename some flags and constant to use SPOP prefix
    - MINOR: spoe: Dynamically alloc the message list per event of an agent
    - MINOR: spoe: Move all stuff regarding the filter/applet in the C file
    - MINOR: spoe: Move spoe_str_to_vsn() into the header file
    - MEDIUM: mux-spop: Introduce the SPOP multiplexer
    - MEDIUM: check/spoe: Use SPOP multiplexer to perform SPOP health-checks
    - MAJOR: spoe: Rewrite SPOE applet to use the SPOP mux
    - CLEANUP: spoe: Uniformize function definitions
    - MINOR: spoe: Add internal sample fetch to retrieve the SPOE engine ID
    - MEDIUM: spoe: Set a specific name for the connection pool of SPOP servers
    - MINOR: backend: Remove test on HTX streams to reuse idle connections on connect
    - MEDIUM: spoe: Force the reuse 'always' mode for SPOP backends
    - MINOR: mux-spop: Use a dedicated function to update the SPOP connection timeout
    - MAJOR: mux-spop: Make the SPOP connections reusable
    - MINOR: stats-html: Display reuse ratio for spop connections
    - MEDIUM: spoe: Directly xfer NOTIFY frame when SPOE applet is created
    - MEDIUM: spoe: Directly receive ACK frame in the SPOE context buffer
    - MEDIUM: mux-spop/spoe: Save negociated max-frame-size value in the mux
    - MINOR: spoe: Remove the spop version from the SPOE appctx context
    - MEDIUM: mux-spop: Add checks on received frames
    - MEDIUM: mux-spop: Announce the pipeling support if possible
    - MEDIUM: spoe: Forward SPOE context error to the SPOE applet
    - MEDIUM: spoe: Make the SPOE applet use its own buffers
    - DOC: spoe: Update SPOE documentation to reflect recent refactoring
    - BUILD: mux-spop: fix build failure on gcc 4-10 and clang
    - MINOR: fd: don't scan the full fdtab on all threads
    - MINOR: server: better mt_list usage for node migration (prev_deleted handling)
    - BUG/MINOR: do not close uninit FD in quic_test_socketops()
    - BUG/MEDIUM: debug/cli: fix "show threads" crashing with low thread counts
    - MINOR: debug: prepare feed_post_mortem_late
    - CLEANUP: debug: fix indents in debug_parse_cli_show_dev
    - MINOR: debug: store runtime uid/gid in postmortem
    - MINOR: debug: keep runtime capabilities in post_mortem
    - MINOR: debug: use LIM2A to show limits
    - MINOR: debug: prepare to show runtime limits
    - MINOR: debug: keep runtime limits in postmortem
    - DOC: install: don't reference removed CPU arg
    - BUG/MEDIUM: ssl_sock: fix deadlock in ssl_sock_load_ocsp() on error path
    - BUG/MAJOR: mux-h2: force a hard error upon short read with pending error
    - MEDIUM: sink: start applets asynchronously
    - OPTIM: sink: balance applets accross threads
    - MEDIUM: ocsp: fix ocsp when the chain is loaded from 'issuers-chain-path'
    - MEDIUM: ssl: add extra_chain to ckch_data
    - MINOR: ssl: change issuers-chain for show_cert_detail()
    - REGTESTS: ssl: test the issuers-chain-path keyword
    - DOC: configuration: issuers-chain-path not compatible with OCSP
    - DOC: configuration: issuers-chain-path is compatible with OCSP
    - BUG/MEDIUM: startup: fix zero-warning mode
    - BUILD: tree-wide: cast arguments to tolower/toupper to unsigned char (2)
    - MINOR: cfgparse-global: move mode's keywords in cfg_kw_list
    - MINOR: cfgparse-global: move no<poller_name> in cfg_kw_list
    - DOC: config: improve the http-keep-alive section
    - BUG/MINOR: stick-table: fix crash for src_inc_gpc() without stkcounter
    - BUG/MINOR: server: Don't warn fallback IP is used during init-addr resolution
    - BUG/MINOR: cli: Atomically inc the global request counter between CLI commands
    - MINOR: stream: Add a pointer to set the parent stream
    - MINOR: vars: Fill a description instead of hash and scope when a name is parsed
    - MINOR: vars: Use a description to set/unset a variable instead of its hash and scope
    - MEDIUM: vars: Be able to parse parent scopes for variables
    - MINOR: vars: Use a variable description to get variables of a specific scope
    - MEDIUM: vars: Be able to retrieve variable of the parent stream, if any
    - MEDIUM: spoe: Set the parent stream for SPOE streams
    - BUG/MINOR: quic: Non optimal first datagram.
    - DOC: config: Add a dedicated section about variables
    - DOC: config: Add info about variable scopes referencing the parent stream
    - DOC: config: Explicitly state the SPOE streams have a usable parent stream
    - MINOR: quic: Avoid cc priv buffer overflow.
    - MINOR: spoe: Add a function to validate a version is supported
    - MINOR: spoe: export the list of SPOP error reasons
    - MEDIUM: spoe/tcpcheck: Reintroduce SPOP check as a customized tcp-check
    - REGTESTS: check/spoe: Re-enable the script performing SPOP health-checks
    - BUG/MEDIUM: sink: properly init applet under sft lock
    - MINOR: sink: unify and sink_forward_io_handler() and sink_forward_oc_io_handler()
    - MINOR: sink: Remove useless test on SE_FL_SHR/SHW flags
    - MINOR: sink: merge sink_forward_io_handler() with sink_forward_oc_io_handler()
    - MINOR: sink: add some comments about sft->appctx usage in applet handlers
    - MINOR: sink: distinguish between hard and soft close in _sink_forward_io_handler()
    - MEDIUM: sink: don't set NOLINGER flag on the outgoing stream interface
    - MINOR: ring: count processed messages in ring_dispatch_messages()
    - MINOR: sink: add processed events counter in sft
    - MEDIUM: sink: "max-reuse" support for sink servers
    - OPTIM: sink: consider threads' current load when rebalancing applets
2024-07-24 18:20:24 +02:00
Aurelien DARRAGON
2513bd257f OPTIM: sink: consider threads' current load when rebalancing applets
In c454296f0 ("OPTIM: sink: balance applets accross threads"), we already
made sure to balance applets accross threads by picking a random thread
to spawn the new applet.

Also, thanks to the previous commit, we also have the ability to destroy
the applet when a certain amount of messages were processed to help
distribute the load during runtime.

Let's improve that by trying up to 3 different threads in the hope to pick
a non-overloaded one in the best scenario, and the least over loaded one
in the worst case. This should help to better distribute the load over
multiple threads when high loads are expected.

Logic was greatly inspired from thread migration logic used by server
health checks, but it was simpliflied for sink's use case.
2024-07-24 17:59:18 +02:00
Aurelien DARRAGON
237849c911 MEDIUM: sink: "max-reuse" support for sink servers
Thanks to the previous commit, it is now possible to know how many events
were processed for a given sft/server sink pair. As mentioned in commit
c454296 ("OPTIM: sink: balance applets accross threads"), let's provide
the ability to restart a server connection when a certain amount of events
were processed to help better balance the load over multiple threads.

For this, we make use the of "max-reuse" server keyword which was only
relevant under "http" context so far. Under sink context, "max-reuse"
corresponds to the number of times the tcp connection can be reused
for sending messages, which in fact means that "max-reuse + 1" is the
number of events (ie: messages) that are allowed to be sent using the
same tcp server connection: when this threshold is met, the connection
will be destroyed and a new one will be created on a random thread.
The value is not strict: it is the minimum value above which the
connection may be destroyed since the value is checked after
ring_dispatch_messages() which may process multiple messages at once.

By default, no limit is enforced (the connection will be reused for as
long as it is available).

The documentation was updated accordingly.
2024-07-24 17:59:14 +02:00
Aurelien DARRAGON
709b3db941 MINOR: sink: add processed events counter in sft
Add a new struct member to sft structure named e_processed in order to
track the total number of events processed by sft applets.

sink_forward_oc_io_handler() and sink_forward_io_handler() now make use
of ring_dispatch_messages() optional value added in the previous commit
in order to increase the number of processed events.
2024-07-24 17:59:08 +02:00
Aurelien DARRAGON
47323e64ad MINOR: ring: count processed messages in ring_dispatch_messages()
ring_dispatch_messages() now takes an optional argument <processed> which
must point to a size_t counter when provided.

When provided, the value is updated to the number of messages processed
by the function.
2024-07-24 17:59:03 +02:00
Aurelien DARRAGON
0821460e3f MEDIUM: sink: don't set NOLINGER flag on the outgoing stream interface
Given that sink applets are responsible for conveying messages from the
ring to the tcp server endpoint, there are no protocol timeout or errors
expected there, it is an unidirectional flow of data over TCP.

As such, NOLINGER flag which was inherited from peers applet, see
dbd026792 ("BUG/MEDIUM: peers: set NOLINGER on the outgoing stream
interface") is not desirable under sink context:

The reason why we have the NOLINGER flag set is to ensure the connection
is closed right away and avoid 60s TIME_WAIT delay on closed sockets.
The downside is that messages sent right before closing the socket are
not guaranteed to make it to the server because closing with NOLINGER
flag set will result in RST packet being emitted right away, which could
prevent in-flight messages from being properly delivered.

Unlike peers applets, the only cases were sink applets are expected to
close the connection are upon unexpected error or upon stopping, which are
relatively rare events. Thanks to previous commit, ERROR flag is already
set in case of error, so the use of NOLINGER is not mandatory for the
RST to be sent. Now for the stopping case, it only happens once in the
process lifetime so it's acceptable to close the socket using EOS+EOI
flags without the NOLINGER option set.

So in our case, it is preferable to ensure messages get properly delivered
knowning that closed sockets should be piling up in TIME_WAIT, this means
removing the NOLINGER flag on the outgoing stream interface for sink
applets. It is a prerequisite for upcoming patches in order to cleanly
shut the applet during runtime without risking to send the RST packet
before all pending messages were sent to the endpoint.
2024-07-24 17:58:58 +02:00
Aurelien DARRAGON
c6ab0e14e2 MINOR: sink: distinguish between hard and soft close in _sink_forward_io_handler()
Aborting the socket on soft-stop is not the same as aborting it due to
unexpected error. As such, let's leverage the granularity offered by
sedesc flags to better reflect the situation: abort during soft-stop is
handled as a soft close thanks to EOI+EOS flags, while abort due to
unexpected error is handled as hard error thanks to ERROR+EOS flags.

Thanks to this change, hard error will always emit RST packet even if
the NOLINGER option wasn't set on the socket.
2024-07-24 17:58:52 +02:00
Aurelien DARRAGON
b40d804c7f MINOR: sink: add some comments about sft->appctx usage in applet handlers
There seem to be an ambiguity in the code where sft->appctx would differ
from the appctx that was assigned to it upon appctx creation.

In practise, it doesn't seem this could be happening. Adding a few notes
to come back to this later and try to see if we can remove this
ambiguity.
2024-07-24 17:58:47 +02:00
Aurelien DARRAGON
10811fdfd6 MINOR: sink: merge sink_forward_io_handler() with sink_forward_oc_io_handler()
Now that sink_forward_oc_io_handler() and sink_forward_io_handler() were
unified again thanks to the previous commit, let's take a chance to merge
code that is common to both functions in order to ease code maintenance.

Let's add _sink_forward_io_handler() internal function which takes the
applet and a message handler as argument: sink_forward_io_handler() and
sink_forward_oc_io_handler() leverage this internal function by passing
the correct message handler for the desired format.
2024-07-24 17:58:41 +02:00
Aurelien DARRAGON
f2848e6146 MINOR: sink: Remove useless test on SE_FL_SHR/SHW flags
Re-apply dcd917d972 ("MINOR: applet: Remove uselelss test on SE_FL_SHR/SHW
flags") for sink_forward_oc_io_handler() function as it was probably
overlooked given that sink_forward_oc_io_handler() and
sink_forward_io_handler() follow the same logic.
2024-07-24 17:58:35 +02:00
Aurelien DARRAGON
901a66b3fc MINOR: sink: unify and sink_forward_io_handler() and sink_forward_oc_io_handler()
In a739dc2 ("MEDIUM: sink: Use the sedesc to report and detect end of
processing"), we added a drain after close in sink_forward_oc_io_handler()
by the use of "goto out".

However, since we perform a close, there is no reason to drain data from
the socket. Moreover, before the patch there was no drain and nothing
mentioned the fact that that the drain was added on purpose. Lastly,
sink_forward_io_handler() and sink_forward_oc_io_handler() functions are
strictly identical when in comes to processing logic, and the drain was
only added in sink_forward_oc_io_handler() and not in
sink_forward_io_handler().

As such, it's pretty safe to assume that the drain is not needed here
and was added as accident. So in this patch we remove it in an attempt
to unify sink_forward_io_handler() and sink_forward_oc_io_handler()
functions like it was already the case before.
2024-07-24 17:58:30 +02:00
Aurelien DARRAGON
c81b8ee480 BUG/MEDIUM: sink: properly init applet under sft lock
Since 09d69eacf8 ("MEDIUM: sink: start applets asynchronously") the applet
is no longer initialized under the sft lock while it was the case before.

At first it doesn't seem to be an issue, but if we look closer at
sink_forward_session_init(), we can see that sft->appctx is assigned
while it can be accessed at the same time from sink_init_forward().

Let's restore the old guarantees by performing the .init under the sft
lock.

No backport needed unless 09d69eacf8 is.
2024-07-24 17:58:24 +02:00
Christopher Faulet
06547dcf52 REGTESTS: check/spoe: Re-enable the script performing SPOP health-checks
Thanks to previous patches, it is now possible to re-enable the test on SPOP
health-checks support.
2024-07-24 14:19:10 +02:00
Christopher Faulet
51e18c9aa6 MEDIUM: spoe/tcpcheck: Reintroduce SPOP check as a customized tcp-check
To be able to retrieve accurrate errors when a SPOP health-check is
performed, a customized tcp-check is used. Indeed, it is not possible to
rely on the SPOP multiplexer for now because the check is performed at the
mux connection layer and the error, if any, cannot be retrieved by the
health-check. A L4 success or error is reported.

To fix this issue and restore the previous behavior, a customized tcp-check
is created. The connection is forced to use the PT multiplexer. An hardcoded
message is sent and a customer handler is used to decode the SPOA response.
This way, it is possible to parse the response and return an accurrate
status code.
2024-07-24 14:19:10 +02:00
Christopher Faulet
2f3c4d1b6c MINOR: spoe: export the list of SPOP error reasons
The strings representing the human-readable version for SPOP errors are now
exported. It is now an array of IST to ease manipulation.
2024-07-24 14:19:10 +02:00
Christopher Faulet
f8fed07d3a MINOR: spoe: Add a function to validate a version is supported
spoe_check_vsn() function can now be used to check if a version, converted
to an integer, via spoe_str_to_vsn() for instance, is supported. To do so,
the list of all supported version is now exported.
2024-07-24 14:19:10 +02:00
Frederic Lecaille
735e4aecfc MINOR: quic: Avoid cc priv buffer overflow.
Add two initcall callback with BUG_ON_HOT() to newro and cubic modules to
ensure there is no buffer overflow when accessing the private data of
these congestion control algorithm state structures. This is to ensure
that further modifications about these data structures will not
lead to surprises. At this time there is no possible buffer overflow.
2024-07-24 11:07:19 +02:00
Christopher Faulet
e902db2609 DOC: config: Explicitly state the SPOE streams have a usable parent stream
It is explicitly mentionned in the configuration manual that the parent of a
SPOE stream is the filtered stream. It means variables of the filtered
stream are usable from the SPOE stream.
2024-07-19 16:35:44 +02:00
Christopher Faulet
2e86de0e0f DOC: config: Add info about variable scopes referencing the parent stream
It is now possible for a stream to have a parent and it is also possible to
retrieve variables defined in the parent stream context. To do so, some
extra scopes were introduced. The section 2.8. was updated accordingly.
2024-07-19 16:35:38 +02:00
Christopher Faulet
b643fbb1a6 DOC: config: Add a dedicated section about variables
The variables in the HAProxy configuration are now described in a dedicated
section. Instead of repeating the same description everywhere a variable
name can be used, the section 2.8. is now referenced.
2024-07-19 16:31:13 +02:00
Frederic Lecaille
402ce29e9e BUG/MINOR: quic: Non optimal first datagram.
This bug arrived with this commit:

     b068e758f MINOR: quic: simplify rescheduling for handshake

This commit introduced a bad side effect. Haproxy always replied by an ACK-only
datagram when it received the first client Initial packet. Then it handled
the CRYPTO data insided. And finally, it sent its own CRYPTO data. This broke
the packet coalescing rule whose aim is to optimally build and send as more
as QUIC packets by datagram.

To fix this, simply partially reverts this commit, to make the low level I/O
task return again if some CRYPTO were received. This will delay the
acknowledgement which will be sent with the CRYPTO data from the same
datagram again.

Must be backported to 3.0.
2024-07-19 16:22:00 +02:00
Christopher Faulet
127083a7a2 MEDIUM: spoe: Set the parent stream for SPOE streams
When a SPOE applet is created to send a message to an agent, the parent of
the associated stream is set to the one filtered. And the relationship
between the streams is removed when the applet is released or when the
processing on main stream is finished.

In the mean time, it is possible to get variables of the parent stream from
the SPOE one. It is not a huge change but this will be amazingly useful. For
instance, it is now possible to be sticky on a server using a critera of the
main streem. Here is an example using the client source address:

  listen http
    bind *:80
    tcp-request content set-var(txn.client_src) src
    filter spoe engine {SPOE-NAME} config /{SPOE-CONFIG}
    http-request send-spoe-group {SPOE-NAME} {SPOE-MSG}
    server www 127.0.0.1:8000

  backend spoe-backend
    mode spop
    timeout server 10s

    stick-table type ip size 200k expire 30m
    stick on var(ptxn.client_src)

    server srv1 ...
    server srv2 ...
    server srv3 ...
    server srv4 ...

Of course, the feature is not limited to stick-tables. Everywhere variables
are used, it is now possible to get the value set on the parent stream from
the SPOE stream.
2024-07-18 17:06:12 +02:00
Christopher Faulet
230c1570ac MEDIUM: vars: Be able to retrieve variable of the parent stream, if any
It is now possible to retrieved the value of a variable using the parent
stream or the parent session instead of the current one. It remains
forbidden to set or unset this value. The sample fetch used to store the
result is a local copy. So it may be safely altered by a converter without
changing the value of the original variable.

Note that for now, the parent of a stream is never set. So this part is not
really used. This will change with the SPOE.
2024-07-18 17:06:12 +02:00
Christopher Faulet
1a1afecb8b MINOR: vars: Use a variable description to get variables of a specific scope
Now a variable description is retrieved when a variable is parsed, we can
use it to get the variable value. It is mandatory to be able to know the
parent stream, if any, must be used, instead of the current one.
2024-07-18 17:06:12 +02:00
Christopher Faulet
f93828f229 MEDIUM: vars: Be able to parse parent scopes for variables
Add session/stream scopes related to the parent. To do so, "psess", "ptxn",
"preq" or "pres" must be used instead of tranditionnal scopes (without the
first "p"). the "proc" scope is not concerned by this change because it is
not linked to a stream. When such scopes are used, a specific flags is added
on the variable description during the variable parsing.

For now, theses scopes are parsed and the variable description is updated
accordingly. But at the end, any operation on the variable value fails.
2024-07-18 16:39:39 +02:00
Christopher Faulet
d430edcda3 MINOR: vars: Use a description to set/unset a variable instead of its hash and scope
Now a variable description is retrieved when a variable is parsed, we can
use it to set or unset the variable value. It is mandatory to be able to
know the parent stream, if any, must be used, instead of the current one.
2024-07-18 16:39:38 +02:00
Christopher Faulet
eb2d71614f MINOR: vars: Fill a description instead of hash and scope when a name is parsed
A variable description is now used to parse a variable and extract its name
and its scope. It is mandatory to be able to add some flags on the variable
when it is evaluated (set or get). Among other things, this will be used to
know the parent stream, if any, must be used, instead of the current one.
2024-07-18 16:39:38 +02:00
Christopher Faulet
b020bb73a0 MINOR: stream: Add a pointer to set the parent stream
A pointer to a parent stream was added in the stream structure. For now,
this pointer is never set, but the idea is to have an access to a stream
environment from another one from the moment there is a parent/child
relationship betwee these streams.

Concretely, for now, there is nothing to formalize this relationship.
2024-07-18 16:39:38 +02:00
Christopher Faulet
3cdb3fa5d9 BUG/MINOR: cli: Atomically inc the global request counter between CLI commands
The global request counter is used to set the stream id (s->uniq_id). It is
incremented at different places. And it must be atomically incremented
because it is a global value. However, in the analyer dealing with CLI
command response, this was not the case. It is now fixed.

This patch must be backported to all stable versions.
2024-07-18 16:39:38 +02:00
Christopher Faulet
abaafda485 BUG/MINOR: server: Don't warn fallback IP is used during init-addr resolution
When a fallback IP address is provided in the list of methods to use to
resolve the server address, a warning is emitted if previous methods
failed. The aim is to inform this address will be used for the
server. However, it is valid use-case. It is the expected behavior. There is
no reason to emit a warning. Having a message during HAProxy startup to
inform the fallback IP address will be used is probably a good idea. But it
should be a notice not a warning. Otherwise, checking the configuration
validity will always failed, just like starting HAProxy in zero-warning
mode while the option was set on purpose.

This patch should fix the issue #2627. It must be backported to all stable
versions.
2024-07-18 16:39:38 +02:00
Amaury Denoyelle
ea7ea5198a BUG/MINOR: stick-table: fix crash for src_inc_gpc() without stkcounter
Since 2.5, an array of GPC is provided to replace legacy gpc0/gpc1.
src_inc_gpc is a sample fetch which is used to increment counters in
this array.

A crash occurs if src_inc_gpc is used without any previous track-sc
rule. This is caused by an error in smp_fetch_sc_inc_gpc(). When
temporary stick counter is created via smp_create_src_stkctr(), table
pointer arg value used is not correct : it points to the counter ID
instead of the table argument. To fix this, use the proper sample fetch
second arg.

This can be reproduced with the following config :
  acl mark src_inc_gpc(0,<table>) -m bool
  tcp-request connection accept if mark

This should be backported up to 2.6.
2024-07-18 16:12:36 +02:00
Willy Tarreau
2bd269cf2a DOC: config: improve the http-keep-alive section
Nathan Wehrman suggested this add-on to try to better explain the
interactions between http-keep-alive and other timeouts, and the
impacts on protocols (HTTP/1, HTTP/2 etc).
2024-07-18 14:24:07 +02:00
Valentine Krasnobaeva
83ff4db188 MINOR: cfgparse-global: move no<poller_name> in cfg_kw_list
This commit continues to clean up cfg_parse_global() and to prepare the
refactoring of master-worker mode. Master, after forking a worker, enters in
its wait polling loop to catch signals and to provide master CLI. So, some
poller types could be disabled for master process it as well.
2024-07-18 14:15:59 +02:00
Valentine Krasnobaeva
118ac11cea MINOR: cfgparse-global: move mode's keywords in cfg_kw_list
This commit cleans up cfg_parse_global() and prepares the config parser for
master-worker mode refactoring, where daemon and master-worker fork() calls
will happen very early in init().

So, the config in such case should be read twice:
 - at first: only some keywords in the global section for the mode discovery
   and everything, which is related to master process by opportunity;
 - at second: except the master process, all other keywords would be parsed;
2024-07-18 14:15:52 +02:00