Commit Graph

16164 Commits

Author SHA1 Message Date
Frédéric Lécaille
fee7ba673f MINOR: quic: Delete remaining RX handshake packets
After the handshake has succeeded, we must delete any remaining
Initial or Handshake packets from the RX buffer. This cannot be
done depending on the state the connection (->st quic_conn struct
member value) as the packet are not received/treated in order.
2021-12-07 15:53:56 +01:00
Frédéric Lécaille
7d807c93f4 MINOR: quic: QUIC encryption level RX packets race issue
The tree containing RX packets must be protected from concurrent accesses.
2021-12-07 15:53:56 +01:00
Frédéric Lécaille
d61bc8db59 MINOR: quic: Race issue when consuming RX packets buffer
Add a null byte to the end of the RX buffer to notify the consumer there is no
more data to treat.
Modify quic_rx_packet_pool_purge() which is the function which remove the
RX packet from the buffer.
Also rename this function to quic_rx_pkts_del().
As the RX packets may be accessed by the QUIC connection handler (quic_conn_io_cb())
the function responsible of decrementing their reference counters must not
access other information than these reference counters! It was a very bad idea
to try to purge the RX buffer asap when executing this function.
2021-12-07 15:53:56 +01:00
Frédéric Lécaille
f9cb3a9b0e MINOR: quic: RX buffer full due to wrong CRYPTO data handling
Do not leave in the RX buffer packets with CRYPTO data which were
already received. We do this when parsing CRYPTO frame. If already
received we must not consider such frames as if they were not received
in order! This had as side effect to interrupt the transfer of long streams
(ACK frames not parsed).
2021-12-07 15:53:56 +01:00
Amaury Denoyelle
84ea8dcbc4 MEDIUM: mux-quic: handle when sending buffer is full
Handle the case when the app layer sending buffer is full. A new flag
QC_SF_BLK_MROOM is set in this case and the transfer is interrupted. It
is expected that then the conn-stream layer will subscribe to SEND.

The MROOM flag is reset each time the muxer transfer data from the app
layer to its own buffer. If the app layer has been subscribed on SEND it
is woken up.
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
e257d9e8ec MEDIUM: mux-quic: wake up xprt on data transferred
On qc_send, data are transferred for each stream from their qcs.buf to
the qcs.xprt_buf. Wake up the xprt to warn about new data available for
transmission.
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
a2c58a7c8d MEDIUM: mux-quic: subscribe on xprt if remaining data after send
The streams data are transferred from the qcs.buf to the qcs.xprt_buf
during qc_send. If the xprt_buf is not empty and not all data can be
transferred, subscribe the connection on the xprt for sending.

The mux will be woken up by the xprt when the xprt_buf will be cleared.
This happens on ACK reception.
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
a3f222dc1e MINOR: mux-quic: implement subscribe on stream
Implement the subscription in the mux on the qcs instance.

Subscribe is now used by the h3 layer when receiving an incomplete frame
on the H3 control stream. It is also used when attaching the remote
uni-directional streams on the h3 layer.

In the qc_send, the mux wakes up the qcs for each new transfer executed.
This is done via the method qcs_notify_send().

The xprt wakes up the qcs when receiving data on unidirectional streams.
This is done via the method qcs_notify_recv().
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
c2025c1ec6 MEDIUM: quic: detect the stream FIN
Set the QC_SF_FIN_STREAM on the app layers (h3 / hq-interop) when
reaching the HTX EOM. This is used to warn the mux layer to set the FIN
on the QUIC stream.
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
916f0ac1e7 MEDIUM: mux-quic: implement release mux operation
Implement qc_release. This function is called by the upper layer on
connection close. For the moment, this only happens on client timeout.

This functions is used the free a qcs instance. If all bidirectional
streams are freed, the qcc instance and the connection are purged.
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
deed777766 MAJOR: mux-quic: implement a simplified mux version
Re-implement the QUIC mux. It will reuse the mechanics from the previous
mux without all untested/unsupported features. This should ease the
maintenance.

Note that a lot of features are broken for the moment. They will be
re-implemented on the following commits to have a clean commit history.
2021-12-07 15:44:45 +01:00
Amaury Denoyelle
d1202edadd MINOR: h3: remove duplicated FIN flag position
The FIN flag is already set in h3_snd_buf on HTX EOM reception. The same
action in h3_resp_headers_send is duplicated and thus now removed.
2021-12-07 15:37:53 +01:00
Amaury Denoyelle
e2288c3087 MEDIUM: xprt-quic: finalize app layer initialization after ALPN nego
The app layer is initialized after the handshake completion by the XPRT
stack. Call the finalize operation just after that.

Remove the erroneous call to finalize by the mux in the TPs callback as
the app layer is not yet initialized at this stage.

This should fix the missing H3 settings currently not emitted by
haproxy.
2021-12-07 15:37:53 +01:00
Amaury Denoyelle
e1f3ff0d08 MINOR: h3: add BUG_ON on control receive function
Add BUG_ON statement when handling a non implemented frames on the
control stream. This is required because frames must be removed from the
RX buffer or else it will stall the buffer.
2021-12-07 15:37:53 +01:00
Amaury Denoyelle
942fc79b5f MINOR: quic: fix segfault on CONNECTION_CLOSE parsing
At the moment the reason_phrase member of a
quic_connection_close/quic_connection_close_app structure is not
allocated. Comment the memcpy to it to avoid segfault.
2021-12-07 15:37:53 +01:00
Willy Tarreau
b154422db1 IMPORT: slz: use the correct CRC32 instruction when running in 32-bit mode
Many ARMv8 processors also support Aarch32 and can run armv7 and even
thumb2 code. While armv8 compilers will not emit these instructions,
armv7 compilers that are aware of these processors will do. For
example, using gcc built for an armv7 target and passing it
"-mcpu=cortex-a72" or "-march=armv8-a+crc" will result in the CRC32
instruction to be used.

In this case the current assembly code fails because with the ARM and
Thumb2 instruction sets there is no such "%wX" half-registers. We need
to use "%X" instead as the native 32-bit register when running with a
32-bit instruction set, and use "%wX" when using the 64-bit instruction
set (A64).

This is slz upstream commit fab83248612a1e8ee942963fe916a9cdbf085097
2021-12-06 09:14:20 +01:00
Willy Tarreau
88bc800eae BUILD: tree-wide: avoid warnings caused by redundant checks of obj_types
At many places we use construct such as:

   if (objt_server(blah))
       do_something(objt_server(blah));

At -O2 the compiler manages to simplify the operation and see that the
second one returns the same result as the first one. But at -O1 that's
not always the case, and the compiler is able to emit a second
expression and sees the potential null that results from it, and may
warn about a potential null deref (e.g. with gcc-6.5). There are two
solutions to this:
  - either the result of the first test has to be passed to a local
    variable
  - or the second reference ought to be unchecked using the __objt_*
    variant.

This patch fixes all occurrences at once by taking the second approach
(the least intrusive). For constructs like:

   objt_server(blah) ? objt_server(blah)->name : "no name"

a macro could be useful. It would for example take the object type
(server), the field name (name) and the default value. But there
are probably not enough occurrences across the whole code for this
to really matter.

This should be backported wherever it applies.
2021-12-06 09:11:47 +01:00
Christopher Faulet
3010e00e1c DOC: config: Specify %Ta is only available in HTTP mode
%Ta format can only be used in HTTP mode but it was not specify in the
configuration manual.

This patch should fix the issue #1317.
2021-12-03 10:48:37 +01:00
Christopher Faulet
d322e948c9 DOC: spoe: Clarify use of the event directive in spoe-message section
Only one event is possible for a spoe-message section. If defined several
time, only the last one is considered. The documentation is now explicit on
this point.

This patch is related to the the issue #1351.
2021-12-03 10:18:11 +01:00
Tim Duesterhus
caf5f5d302 BUG/MEDIUM: sample: Fix memory leak in sample_conv_jwt_member_query
The function leaked one full buffer per invocation. Fix this by simply removing
the call to alloc_trash_chunk(), the static chunk from get_trash_chunk() is
sufficient.

This bug was introduced in 0a72f5ee7c, which is
2.5-dev10. This fix needs to be backported to 2.5+.
2021-12-03 09:03:55 +01:00
Christopher Faulet
d6ae912b04 BUILD: bug: Fix error when compiling with -DDEBUG_STRICT_NOCRASH
ha_backtrace_to_stderr() must be declared in CRASH_NOW() macro whe HAProxy
is compiled with DEBUG_STRICT_NOCRASH. Otherwise an error is reported during
compilation:

include/haproxy/bug.h:58:26: error: implicit declaration of function ‘ha_backtrace_to_stderr’ [-Werror=implicit-function-declaration]
   58 | #define CRASH_NOW() do { ha_backtrace_to_stderr(); } while (0)

This patch must be backported as far as 2.4.
2021-12-03 08:58:28 +01:00
Christopher Faulet
af93d2fd70 BUG/MINOR: resolvers: Don't overwrite the error for invalid query domain name
When a response is validated, the query domain name is checked to be sure it
is the same than the one requested. When an error is reported, the wrong
goto label was used. Thus, the error was lost. Instead of
RSLV_RESP_WRONG_NAME, RSLV_RESP_INVALID was reported.

This bug was introduced by the commit c1699f8c1 ("MEDIUM: resolvers: No
longer store query items in a list into the response").

This patch should fix the issue #1473. No backport is needed.
2021-12-02 10:05:04 +01:00
Christopher Faulet
02c893332b BUG/MEDIUM: h1: Properly reset h1m flags when headers parsing is restarted
If H1 headers are not fully received at once, the parsing is restarted a
last time when all headers are finally received. When this happens, the h1m
flags are sanitized to remove all value set during parsing.

But some flags where erroneously preserved. Among others, H1_MF_TE_CHUNKED
flag was not removed, what could lead to parsing error.

To fix the bug and make things easy, a mask has been added with all flags
that must be preserved. It will be more stable. This mask is used to
sanitize h1m flags.

This patch should fix the issue #1469. It must be backported to 2.5.
2021-12-02 09:46:29 +01:00
Emeric Brun
2ad2b1c94c BUG/MAJOR: segfault using multiple log forward sections.
For each new log forward section, the proxy was added to the log forward
proxy list but the ref on the previous log forward section's proxy was
scratched using "init_new_proxy" which performs a memset. After configuration
parsing this list contains only the last section's proxy.

The post processing walk through this list to resolve "ring" names.
Since some section's proxies are missing in this list, the resolving
is not done for those ones and the pointer on the ring is kept to null
causing a segfault at runtime trying to write a log message
into the ring.

This patch shift the "init_new_proxy" before adding the ref on the
previous log forward section's proxy on currently parsed one.

This patch shoud fix github issue #1464

This patch should be backported to 2.3
2021-12-01 15:21:56 +01:00
Christopher Faulet
c1699f8c1b MEDIUM: resolvers: No longer store query items in a list into the response
When the response is parsed, query items are stored in a list, attached to
the parsed response (resolve_response).

First, there is one and only one query sent at a time. Thus, there is no
reason to use a list. There is a test to be sure there is only one query
item in the response. Then, the reference on this query item is only used to
validate the domain name is the one requested. So the query list can be
removed. We only expect one query item, no reason to loop on query records.
In addition, the query domain name is now immediately checked against the
resolution domain name. This way, the query item is only manipulated during
the response parsing.
2021-12-01 15:21:56 +01:00
Christopher Faulet
80b2e34b18 BUG/MEDIUM: resolvers: Detach query item on response error
When a new response is parsed, it is unexpected to have an old query item
still attached to the resolution. And indeed, when the response is parsed
and validated, the query item is detached and used for a last check on its
dname. However, this is only true for a valid response. If an error is
detected, the query is not detached. This leads to undefined behavior (most
probably a crash) on the next response because the first element in the
query list is referencing an old response.

This patch must be backported as far as 2.0.
2021-12-01 11:47:08 +01:00
Christopher Faulet
4ab2679689 BUG/MINOR: server: Don't rely on last default-server to init server SSL context
During post-parsing stage, the SSL context of a server is initialized if SSL
is configured on the server or its default-server. It is required to be able
to enable SSL at runtime. However a regression was introduced, because the
last parsed default-server is used. But it is not necessarily the
default-server line used to configure the server. This may lead to
erroneously initialize the SSL context for a server without SSL parameter or
the skip it while it should be done.

The problem is the default-server used to configure a server is not saved
during configuration parsing. So, the information is lost during the
post-parsing. To fix the bug, the SRV_F_DEFSRV_USE_SSL flag is
introduced. It is used to know when a server was initialized with a
default-server using SSL.

For the record, the commit f63704488e ("MEDIUM: cli/ssl: configure ssl on
server at runtime") has introduced the bug.

This patch must be backported as far as 2.4.
2021-12-01 11:47:08 +01:00
Christopher Faulet
a8ecbc4544 REGTESTS: h1: Add a script to validate H1 splicing support
This script is used to validate the spliciing is properly working for the H1
multiplexer. It relies on the H1 mux stats.
2021-12-01 11:47:08 +01:00
Christopher Faulet
41951ab9d6 MINOR: mux-h1: add stat for total amount of bytes received and sent
Add counters for total amount of bytes received and sent. Bytes received and
sent via kernel splicing are also counted.
2021-12-01 11:47:08 +01:00
Christopher Faulet
3bca28c9fd MINOR: mux-h1: add stat for total count of connections/streams
Add counters for total number of http1 connections/stream since haproxy
startup. Contrary to open_conn/stream, they are never reset to zero.
2021-12-01 11:47:08 +01:00
Christopher Faulet
60fa051e71 MINOR: mux-h1: count open connections/streams on stats
Implement as a gauge h1 counters for currently open connections and
streams. The counters are decremented when closing the stream or the
connection.
2021-12-01 11:47:08 +01:00
Christopher Faulet
563c345f6f MINOR: mux-h1: add counters instance to h1c
Add pointer to counters as a member for h1c structure. This pointer is
initialized on h1_init function. This is useful to quickly access and
manipulate the counters inside every h1 functions.
2021-12-01 11:47:08 +01:00
Christopher Faulet
b4c584eed1 MINOR: mux-h1: register a stats module
Use statistics API to register a new stats module generating counters on h1
module. The counters are attached to frontend/backend instances.
2021-12-01 11:47:08 +01:00
Christopher Faulet
6580f2868e MINOR: mux-h1: Improve H1 traces by adding info about http parsers
Info about the request and the response parsers are now displayed in H1
traces for advanced and complete verbosity only. This should help debugging.

This patch may be backported as far as 2.4.
2021-12-01 11:47:08 +01:00
Christopher Faulet
f5ce320156 BUG/MINOR: mux-h1: Fix splicing for messages with unknown length
Splicing was disabled fo Messages with an unknown length (no C-L or T-E
header) with no valid reason. So now, it is possible to use the kernel
splicing for such messages.

This patch should be backported as far as 2.4.
2021-12-01 11:47:08 +01:00
Christopher Faulet
140f1a5852 BUG/MEDIUM: mux-h1: Fix splicing by properly detecting end of message
Since the 2.4.4, the splicing support in the H1 multiplexer is buggy because
end of the message is not properly detected.

On the 2.4, when the requests is spliced, there is no issue. But when the
response is spliced, the client connection is always closed at the end of the
message. Note the response is still fully sent.

On the 2.5 and higher, when the last requests on a connection is spliced, a
client abort is reported. For other requests there is no issue. In all cases,
the requests are fully sent. When the response is spliced, the server connection
hangs till the server timeout and a server abort is reported. The response is
fully sent with no delay.

The root cause is the EOM block suppression. There is no longer extra block to
be sure to call a last time rcv_buf()/snd_buf() callback functions. At the end,
to fix the issue, we must now detect end of the message in rcv_pipe() and
snd_pipe() callback functions. To do so, we rely on the announced message length
to know when the payload is finished. This works because the chunk-encoded
messages are not spliced.

This patch must be backported as far as 2.4 after an observation period.
2021-12-01 11:46:21 +01:00
David CARLIER
b1e190a885 MEDIUM: pool: Following up on previous pool trimming update.
Apple libmalloc has its own notion of memory arenas as malloc_zone with
rich API having various callbacks for various allocations strategies but
here we just use the defaults.
In trim_all_pools,  we advise to purge each zone as much as possible, called "greedy" mode.
2021-12-01 10:38:31 +01:00
Remi Tricot-Le Breton
bb3e80e181 BUG/MINOR: vars: Fix the set-var and unset-var converters
In commit 3a4bedccc6 the variable logic was changed. Instead of
accessing variables by their name during runtime, the variable tables
are now indexed by a hash of the name. But the set-var and unset-var
converters try to access the correct variable by calculating a hash on
the sample instead of the already calculated variable hash.

It should be backported to 2.5.
2021-12-01 10:32:19 +01:00
Frédéric Lécaille
008386bec4 MINOR: quic: Delete the ODCIDs asap
As soon as the connection ID (the one choosen by the QUIC server) has been used
by the client, we can delete its original destination connection ID from its tree.
2021-11-30 12:01:32 +01:00
Frédéric Lécaille
a7d2c09468 MINOR: quic: Enable the Key Update process
This patch modifies ha_quic_set_encryption_secrets() to store the
secrets received by the TLS stack and prepare the information for the
next key update thanks to quic_tls_key_update().
qc_pkt_decrypt() is modified to check if we must used the next or the
previous key phase information to decrypt a short packet.
The information are rotated if the packet could be decrypted with the
next key phase information. Then new secrets, keys and IVs are updated
calling quic_tls_key_update() to prepare the next key phase.
quic_build_packet_short_header() is also modified to handle the key phase
bit from the current key phase information.
2021-11-30 11:51:12 +01:00
Frédéric Lécaille
a7973a6dce MINOR: quic: Add quic_tls_key_update() function for Key Update
This function derives the next RX and TX keys and IVs from secrets
for the next key update key phase. We also implement quic_tls_rotate_keys()
which rotate the key update key phase information to be able to continue
to decrypt old key phase packets. Most of these information are pointers
to unsigned char.
2021-11-30 11:51:12 +01:00
Frédéric Lécaille
6e351d6c19 MINOR: quic: Optional header protection key for quic_tls_derive_keys()
quic_tls_derive_keys() is responsible to derive the AEAD keys, IVs and$
header protection key from a secret provided by the TLS stack. We want
to make the derivation of the header protection key be optional. This
is required for the Key Update process where there is no update for
the header protection key.
2021-11-30 11:51:12 +01:00
Frédéric Lécaille
40df78f116 MINOR: quic: Add structures to maintain key phase information
When running Key Update process, we must maintain much information
especially when the key phase bit has been toggled by the peer as
it is possible that it is due to late packets. This patch adds
quic_tls_kp new structure to do so. They are used to store
previous and next secrets, keys and IVs associated to the previous
and next RX key phase. We also need the next TX key phase information
to be able to encrypt packets for the next key phase.
2021-11-30 11:51:12 +01:00
Frédéric Lécaille
39484de813 MINOR: quic: Add a function to derive the key update secrets
This is the function used to derive an n+1th secret from the nth one as
described in RFC9001 par. 6.1.
2021-11-30 11:51:12 +01:00
Frédéric Lécaille
fc768ecc88 MINOR: quic: Dynamically allocate the secrete keys
This is done for any encryption level. This is to prepare the Key Update feature.
2021-11-30 11:51:12 +01:00
Frédéric Lécaille
d77c50b6d6 MINOR: quic: Possible crash when inspecting the xprt context
haproxy may crash when running this statement in qc_lstnr_pkt_rcv():
	conn_ctx = qc->conn->xprt_ctx;
because qc->conn may not be initialized. With this patch we ensure
qc->conn is correctly initialized before accessing its ->xprt_ctx
members. We zero the xrpt_ctx structure (ssl_conn_ctx struct), then
initialize its ->conn member with HA_ATOMIC_STORE. Then, ->conn and
->conn->xptr_ctx members of quic_conn struct can be accessed with HA_ATOMIC_LOAD()
2021-11-30 11:50:42 +01:00
Frédéric Lécaille
e2660e61e2 MINOR: quic: Rename qc_prep_hdshk_pkts() to qc_prep_pkts()
qc_prep_hdshk_pkts() does not prepare only handshake packets but
any type of packet.
2021-11-30 11:47:46 +01:00
Frédéric Lécaille
b5b5247b18 MINOR: quic: Immediately close if no transport parameters extension found
If the ClientHello callback does not manage to find a correct QUIC transport
parameters extension, we immediately close the connection with
missing_extension(109) as TLS alert which is turned into 0x16d QUIC connection
error.
2021-11-30 11:47:46 +01:00
Frédéric Lécaille
1fc5e16c4c MINOR: quic: More accurate immediately close.
When sending a CONNECTION_CLOSE frame to immediately close the connection,
do not provide CRYPTO data to the TLS stack. Do not built anything else than a
CONNECTION_CLOSE and do not derive any secret when in immediately close state.
Seize the opportunity of this patch to rename ->err quic_conn struct member
to ->error_code.
2021-11-30 11:47:46 +01:00
Frédéric Lécaille
067a82bba1 MINOR: quic: Set "no_application_protocol" alert
We set this TLS error when no application protocol could be negotiated
via the TLS callback concerned. It is converted as a QUIC CRYPTO_ERROR
error (0x178).
2021-11-30 11:47:46 +01:00