Commit Graph

20240 Commits

Author SHA1 Message Date
Christopher Faulet
535dd920df MINOR: compression: Improve the way Vary header is added
When a message is compressed, A "Vary" header is added with
"accept-encoding" value. However, a new header is always added, regardless
there is already a Vary header or not. In addition, if there is already a
Vary header, there is no check on values to be sure "accept-encoding" value
is not already there. So it is possible to have it twice.

To improve this part, we now test Vary header values and "accept-encoding"
is only added if it was not found. In addition, "accept-encoding" value is
appended to the last Vary header found, if any. Otherwise, a new header is
added.
2023-05-25 11:25:31 +02:00
Willy Tarreau
c8bb9aeb07 [RELEASE] Released version 2.8-dev13
Released version 2.8-dev13 with the following main changes :
    - DOC: add size format section to manual
    - CLEANUP: mux-quic/h3: complete BUG_ON with comments
    - MINOR: quic: remove return val of quic_aead_iv_build()
    - MINOR: quic: use WARN_ON for encrypt failures
    - BUG/MINOR: quic: handle Tx packet allocation failure properly
    - MINOR: quic: fix alignment of oneline show quic
    - MEDIUM: stconn/applet: Allow SF_SL_EOS flag alone
    - MEDIUM: stconn: make the SE_FL_ERR_PENDING to ERROR transition systematic
    - DOC: internal: add a bit of documentation for the stconn closing conditions
    - DOC/MINOR: config: Fix typo in description for `ssl_bc` in configuration.txt
    - BUILD: quic: re-enable chacha20_poly1305 for libressl
    - MINOR: mux-quic: set both EOI EOS for stream fin
    - MINOR: mux-quic: only set EOS on RESET_STREAM recv
    - MINOR: mux-quic: report error on stream-endpoint earlier
    - BUILD: makefile: fix build issue on GNU make < 3.82
    - BUG/MINOR: mux-h2: Check H2_SF_BODY_TUNNEL on H2S flags and not demux frame ones
    - MINOR: mux-h2: Set H2_SF_ES_RCVD flag when decoding the HEADERS frame
    - MINOR: mux-h2: Add a function to propagate termination flags from h2s to SE
    - BUG/MEDIUM: mux-h2: Propagate termination flags when frontend SC is created
    - DEV: add a Lua helper script for SSL keys logging
    - CLEANUP: makefile: don't display a dummy features list without a target
    - BUILD: makefile: do not erase build options for some build options
    - MINOR: quic: Add low level traces (addresses, DCID)
    - BUG/MINOR: quic: Wrong token length check (quic_generate_retry_token())
    - BUG/MINOR: quic: Missing Retry token length on receipt
    - MINOR: quic: Align "show quic" command help information
    - CLEANUP: quic: Indentation fix quic_rx_pkt_retrieve_conn()
    - CLEANUP: quic: Useless tests in qc_rx_pkt_handle()
    - MINOR: quic: Add some counters at QUIC connection level
    - MINOR: quic: Add a counter for sent packets
    - MINOR: hlua: hlua_smp2lua_str() may LJMP
    - MINOR: hlua: hlua_smp2lua() may LJMP
    - MINOR: hlua: hlua_arg2lua() may LJMP
    - DOC: hlua: document hlua_lua2arg() function
    - DOC: hlua: document hlua_lua2smp() function
    - BUG/MINOR: hlua: unsafe hlua_lua2smp() usage
    - BUILD: makefile: commit the tiny FreeBSD makefile stub
    - BUILD: makefile: fix build options when building tools first
    - BUILD: ist: do not put a cast in an array declaration
    - BUILD: ist: use the literal declaration for ist_lc/ist_uc under TCC
    - BUILD: compiler: systematically set USE_OBSOLETE_LINKER with TCC
    - DOC: install: update reference to known supported versions
    - SCRIPTS: publish-release: update the umask to keep group write access
2023-05-24 22:53:55 +02:00
Willy Tarreau
f9b04bd48b SCRIPTS: publish-release: update the umask to keep group write access
This is to avoid the occasional error that arises when a release is
first done by another maintainer.
2023-05-24 22:49:12 +02:00
Willy Tarreau
3098540bba DOC: install: update reference to known supported versions
Gcc 13 is known to work, OpenSSL 3.1 and wolfSSL as well. Add a few
hints about build errors when using QUIC + OpenSSL and warnings about
the dramatic OpenSSL 3.x performance regression.
2023-05-24 22:32:46 +02:00
Willy Tarreau
b298882acc BUILD: compiler: systematically set USE_OBSOLETE_LINKER with TCC
TCC silently ignores the weak and section attributes, which ruins the
initcalls. Technically we're exactly in the same situation as with an
obsolete linker. Let's just automatically set the flag if TCC is
detected, this avoids surprises where the program compiles but does
not start.

No backport is needed.
2023-05-24 21:37:06 +02:00
Willy Tarreau
eced142aa8 BUILD: ist: use the literal declaration for ist_lc/ist_uc under TCC
TCC doesn't knoow about __attribute__((weak)), it silently ignores it.
We could add a "static" modifier there in this case but we already have
an alternate portable mode that is based on a slightly larger literal
for obsolete linkers (and non-ELF systems) which choke on weak. Let's
just add the test for tcc there and use it in this case.

No backport is needed.
2023-05-24 21:33:34 +02:00
Willy Tarreau
4e8720ab78 BUILD: ist: do not put a cast in an array declaration
TCC is upset by the declaration looking like:

  const unsigned char ist_lc[256] __attribute__((weak)) = ((const unsigned char[256]){ ... });

It was written like this because it's expanded from the _IST_LC macro
but it's never used as-is, it's only used from ist_lc, which should be
the one containing the cast so that the macro only contains the list of
bytes that can be used in both places. And this assigns more consistent
roles to the lower and upper case macro/variable now, one is typed and
the other one not. No backport is needed.
2023-05-24 21:27:39 +02:00
Willy Tarreau
6777357a5e BUILD: makefile: fix build options when building tools first
Due to the test on the target introduced by commit 9577a152b ("BUILD:
makefile: do not erase build options for some build options"), if a
tool (e.g. halog) is build first before haproxy after a clean or a
fresh source extraction, the .build_opts file does not exist and
"make" complains since there's no such target. Make sure to define
the empty target for all "else" blocks there. No backport is needed.
2023-05-24 17:23:45 +02:00
Willy Tarreau
af26361837 BUILD: makefile: commit the tiny FreeBSD makefile stub
The idea here is to try to detect the use of "make" instead of "gmake"
on FreeBSD. After having long tried, there's no way to construct a
condition that is common to both makefile languages and could serve as
a differentiator since there's simply no common word between the two
languages. However on FreeBSD (the main used BSD platform), "make" is
configured to look for BSDmakefile before the other ones. It allows us
to intercept it and explain to use gmake with an example of a roughly
converted make command line (we just strip "-J xx,xx" that systematically
gets inserted if "-j" is used). A few tricks are used, such as creating
a dummy target on the fly based on the requested one just to silence the
output, and always match "all" since it's used by default when no target
is specified. .DEFAULTS was initially used but finally dropped thanks to
this.

For example:

  $ make -j$(getconf NPROCESSORS_ONLN) TARGET=freebsd USE_OPENSSL=1
  Please use GNU make instead. It is often called gmake.
  Example:
    gmake  -j 4 TARGET=freebsd USE_OPENSSL=1  all

It will often be sufficient to permit a copy-paste and to try again.
Note that the .gitignore was updated.
2023-05-24 17:17:36 +02:00
Aurelien DARRAGON
1c07da4b48 BUG/MINOR: hlua: unsafe hlua_lua2smp() usage
Fixing hlua_lua2smp() usage in hlua's code since it was assumed that
hlua_lua2smp() makes a standalone smp out of lua data, but it is not
the case.

This is especially true when dealing with lua strings (string is
extracted using lua_tolstring() which returns a pointer to lua string
memory location that may be reclaimed by lua at any time when no longer
used from lua's point of view). Thus, smp generated by hlua_lua2smp() may
only be used from the lua context where the call was initially made, else
it should be explicitly duplicated before exporting it out of lua's
context to ensure safe (standalone) usage.

This should be backported to all stable versions.
2023-05-24 16:48:17 +02:00
Aurelien DARRAGON
a3624cb528 DOC: hlua: document hlua_lua2smp() function
Add some developer notes to hlua_lua2smp() function description since
it lacks some important infos, including a critical usage restriction.
2023-05-24 16:48:17 +02:00
Aurelien DARRAGON
0aaf6c45ca DOC: hlua: document hlua_lua2arg() function
Add some developer notes to hlua_lua2arg() function description since
it lacks some important infos, including an usage restriction.
2023-05-24 16:48:17 +02:00
Aurelien DARRAGON
e5c048a72d MINOR: hlua: hlua_arg2lua() may LJMP
Add LJMP hint to hlua_arg2lua() prototype since it relies on
functions (e.g.: lua_pushlstring()) which may raise lua memory errors.
2023-05-24 16:48:17 +02:00
Aurelien DARRAGON
4121772c50 MINOR: hlua: hlua_smp2lua() may LJMP
Add LJMP hint to hlua_smp2lua() prototype since it relies on
functions (e.g.: lua_pushstring()) which may raise lua memory errors.
2023-05-24 16:48:17 +02:00
Aurelien DARRAGON
742b1a8797 MINOR: hlua: hlua_smp2lua_str() may LJMP
Add LJMP hint to hlua_smp2lua_str() prototype since it relies on
functions (e.g.: lua_pushstring()) which may raise lua memory errors.
2023-05-24 16:48:17 +02:00
Frdric Lcaille
12a815ad19 MINOR: quic: Add a counter for sent packets
Add ->sent_pkt counter to quic_conn struct to count the packet at QUIC connection
level. Then, when the connection is released, the ->sent_pkt counter value
is added to the one for the listener.

Must be backported to 2.7.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
bdd64fd71d MINOR: quic: Add some counters at QUIC connection level
Add some statistical counters to quic_conn struct from quic_counters struct which
are used at listener level to handle them at QUIC connection level. This avoid
calling atomic functions. Furthermore this will be useful soon when a counter will
be added for the total number of packets which have been sent which will be very
often incremented.

Some counters were not added, espcially those which count the number of QUIC errors
by QUIC error types. Indeed such counters would be incremented most of the time
only one time at QUIC connection level.

Implement quic_conn_prx_cntrs_update() which accumulates the QUIC connection level
statistical counters to the listener level statistical counters.

Must be backported to 2.7.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
464281af46 CLEANUP: quic: Useless tests in qc_rx_pkt_handle()
There is no reason to test <qc> nullity at the end of this function because it is
clearly not null, furthermore the trace handle the case where <qc> is null.

Must be backported to 2.7.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
ab3aa0ff22 CLEANUP: quic: Indentation fix quic_rx_pkt_retrieve_conn()
Add missing spaces.

Must be backported to 2.7.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
5fa633e22f MINOR: quic: Align "show quic" command help information
Align the "show quic" help information with all the others command help information.
Furthermore, makes this information match the management documentation.

Must be backported to 2.7.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
35b63964a0 BUG/MINOR: quic: Missing Retry token length on receipt
quic_retry_token_check() must decipher the token sent to and received back from
clients. This token is made of the token format byte, the ODCID prefixed by its one byte
length, the timestamp of its creation, and terminated by an AEAD TAG followed
by the salt used to derive the secret to cipher the token.

So, the length of these data must be between
2 + QUIC_ODCID_MINLEN + sizeof(uint32_t) + QUIC_TLS_TAG_LEN + QUIC_RETRY_TOKEN_SALTLEN
and
2 + QUIC_CID_MAXLEN + sizeof(uint32_t) + QUIC_TLS_TAG_LEN + QUIC_RETRY_TOKEN_SALTLEN.

Must be backported to 2.7 and 2.6.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
6d6ddb2ce5 BUG/MINOR: quic: Wrong token length check (quic_generate_retry_token())
This bug would never occur because the buffer supplied to quic_generate_retry_token()
to build a Retry token is large enough to embed such a token. Anyway, this patch
fixes quic_generate_retry_token() implementation.

There were two errors: this is the ODCID which is added to the token. Furthermore
the timestamp was not taken into an account.

Must be backported to 2.6 and 2.7.
2023-05-24 16:30:11 +02:00
Frdric Lcaille
aaf32f0c83 MINOR: quic: Add low level traces (addresses, DCID)
Add source and destination addresses to QUIC_EV_CONN_RCV trace event. This is
used by datagram/socket level functions (quic_sock.c).

Must be backported to 2.7.
2023-05-24 16:30:11 +02:00
Willy Tarreau
9577a152b5 BUILD: makefile: do not erase build options for some build options
One painfully annoying thing with the build options change detection
is that they get rebuild for about everything except when the build
target is exactly "reg-tests". But in practice every time reg tests
are run we end up having to experience a full rebuild because the
reg-tests script runs "make version" which is sufficient to refresh
the file.

There are two issues here. The first one is that we ought to skip all
targets that do not make use of the build options. This includes all
the tools such as "flags" for example, or utility targets like "tags",
"help" or "version". The second issue is that with most of these extra
targets we do not set the TARGET variable, and that one is used when
creating the build_opts file, so let's preserve the file when TARGET
is not set.

Now it's possible to re-run a make after a make reg-tests without having
to rebuild the whole project.
2023-05-24 16:23:24 +02:00
Willy Tarreau
060769836e CLEANUP: makefile: don't display a dummy features list without a target
"make help" ends with a list of enabled/disabled features for TARGET '',
which makes no sense. Let's only display enabled/disabled features when
a target is set. It also removes visual pollution when users seek help.
2023-05-24 16:23:24 +02:00
Amaury Denoyelle
f1df006ffe DEV: add a Lua helper script for SSL keys logging
This script can be used through a http-request rules to log SSL keys for
traffic on a dedicated frontend. The resulting file can then be injected
into wireshark to decipher the corresponding network capture.
2023-05-24 16:08:23 +02:00
Christopher Faulet
c2f1d0ee5e BUG/MEDIUM: mux-h2: Propagate termination flags when frontend SC is created
We must evaluate if EOS/EOI/ERR_PENDING/ERROR flags must be set on the SE
when the frontend SC is created because the rxbuf is transferred to the
steeam at this stage. It means the call to h2_rcv_buf() may be skipped on
some circumstances.

And indeed, it happens when HAproxy quickly replies, for instance because of
a deny rule. In this case, depending on the scheduling, the abort may block
the receive attempt from the SC. In this case if SE flags were not properly
set earlier, there is no way to terminate the request and the session may be
freezed.

For now, I can't explain why there is no timeout when this happens but it
remains an issue because here we should not rely on timeouts to close the
stream.

This patch relies on following commits:

    * MINOR: mux-h2: Add a function to propagate termination flags from h2s to SE
    * MINOR: mux-h2: Set H2_SF_ES_RCVD flag when decoding the HEADERS frame

The issue was encountered on the 2.8 but it seems the bug exists since the
2.4. But it is probably a good idea to only backport the series to 2.7 only
and wait for a bug report on earlier versions.

This patch should solve the issue #2147.
2023-05-24 16:06:11 +02:00
Christopher Faulet
531dd050ff MINOR: mux-h2: Add a function to propagate termination flags from h2s to SE
The function h2s_propagate_term_flags() was added to check the H2S state and
evaluate when EOI/EOS/ERR_PENDING/ERROR flags must be set on the SE. It is
not the only place where those flags are set. But it centralizes the synchro
between the H2 stream and the SC.

For now, this function is only used at the end of h2_rcv_buf(). But it will
be used to fix a bug.
2023-05-24 16:06:11 +02:00
Christopher Faulet
1a60a66306 MINOR: mux-h2: Set H2_SF_ES_RCVD flag when decoding the HEADERS frame
The flag H2_SF_ES_RCVD is set on the H2 stream when the ES flag is found in
a frame. On HEADERS frame, it was set in function processing the frame. It
is moved in the function decoding the frame. Fundamentally, this changes
nothing. But it will be useful to have this information earlier when a
client H2 stream is created.
2023-05-24 16:06:11 +02:00
Christopher Faulet
78b1eb2b04 BUG/MINOR: mux-h2: Check H2_SF_BODY_TUNNEL on H2S flags and not demux frame ones
In h2c_frt_stream_new(), H2_SF_BODY_TUNNEL flags was tested on demux frame
flags (h2c->dff) instead of the h2s flags.  By chance, it is a noop test
becasue H2_SF_BODY_TUNNEL value, once converted to an int8_t, is 0.

It is a 2.8-specific issue. No backport needed.
2023-05-24 16:06:11 +02:00
Willy Tarreau
1e1c28873c BUILD: makefile: fix build issue on GNU make < 3.82
Thierry Fournier reported a build breakage with the ubiquitous make
3.81, LDFLAGS were ignored. This is caused by the declaration of the
collect_opt_flags macro that is defined with an "=" sign, something
that only appeared in 3.82 and that is not necessary. With it removed,
the build now works fine at least from 3.80 to 4.3.

No backport is needed since this makefile cleanup appeared in 2.8.
2023-05-24 15:51:03 +02:00
Amaury Denoyelle
152beeec34 MINOR: mux-quic: report error on stream-endpoint earlier
A RESET_STREAM is emitted in several occasions :
- protocol error during HTTP/3.0 parsing
- STOP_SENDING reception

In both cases, if a stream-endpoint is attached we must set its ERR
flag. This was correctly done but after some delay as it was only when
the RESET_STREAM was emitted. Change this to set the ERR flag as soon as
one of the upper cases has been encountered. This should help to release
faster streams in error.

This should be backported up to 2.7.
2023-05-24 14:46:52 +02:00
Amaury Denoyelle
37d78997ae MINOR: mux-quic: only set EOS on RESET_STREAM recv
A recent review was done to rationalize ERR/EOS/EOI flags on stream
endpoint. A common definition for both H1/H2/QUIC mux have been written
in the following documentation :
 ./doc/internals/stconn-close.txt

In QUIC it is possible to close each channels of a stream independently
with RESET_STREAM and STOP_SENDING frames. When a RESET_STREAM is
received, it indicates that the peer has ended its transmission in an
abnormal way. However, it is still ready to receive.

Previously, on RESET_STREAM reception, QUIC MUX set the ERR flag on
stream-endpoint. However, according to the QUIC mechanism, it should be
instead EOS but this was impossible due to a BUG_ON() which prevents EOS
without EOI or ERR. This BUG_ON was only present because this case was
never used before the introduction of QUIC. It was removed in a recent
commit which allows us to now properly set EOS alone on RESET_STREAM
reception.

In practice, this change allows to continue to send data even after
RESET_STREAM reception. However, currently browsers always emit it with
a STOP_SENDING as this is used to abort the whole H3 streams. In the end
this will result in a stream-endpoint with EOS and ERR_PENDING/ERR
flags.

This should be backported up to 2.7.
2023-05-24 14:39:17 +02:00
Amaury Denoyelle
8de35925f7 MINOR: mux-quic: set both EOI EOS for stream fin
A recent review was done to rationalize ERR/EOS/EOI flags on stream
endpoint. A common definition for both H1/H2/QUIC mux have been written
in the following documentation :
 ./doc/internals/stconn-close.txt

Always set EOS with EOI flag to conform to this specification. EOI is
set whenever the proper stream end has been encountered : with QUIC it
corresponds to a STREAM frame with FIN bit. At this step, RESET_STREAM
frames are ignored by QUIC MUX as allowed by RFC 9000. This means we can
always set EOS at the same time with EOI.

This should be backported up to 2.7.
2023-05-24 14:23:22 +02:00
Ilya Shipitsin
97c344dae0 BUILD: quic: re-enable chacha20_poly1305 for libressl
this reverts d2be9d4c48

LibreSSL implements EVP_chacha20_poly1305() with EVP_CIPHER for every
released version starting with 3.6.0
2023-05-23 19:20:36 +02:00
Mariam John
6ff043de2c DOC/MINOR: config: Fix typo in description for ssl_bc in configuration.txt
Fix a minor typo in the description of the `ssl_bc` sample fetch method described under
Section `7.3.4. Fetching samples at Layer 5` in configuration.txt. Changed `other` to `to`.
2023-05-23 17:06:06 +02:00
Willy Tarreau
e49e9e64a2 DOC: internal: add a bit of documentation for the stconn closing conditions
The conditions where ERR, EOS and EOI are found are not always
crystal clear, and the fact that there's still a good bunch of
original ones dating from the early days and that seem to test for
non-existing cases doesn't help either.

After auditing the code base and projecting the 3 main muxes' stream
termination conditions, with Christopher and Amaury we could establish
the current flags matrix which indicates both what each combination
means for each mux and when it is set by each of them (or not set and
for what reason).

It should be sufficient to void doubts when adding code or when chasing
a bug.

It *must not* be backported because it is highly specific to the latest
2.8-dev.
2023-05-23 16:18:19 +02:00
Willy Tarreau
b7209d42d9 MEDIUM: stconn: make the SE_FL_ERR_PENDING to ERROR transition systematic
During a code audit of the various situations that promote ERR_PENDING to
ERROR, it appeared that:
  - all muxes use se_fl_set_error() to set it, which chooses either based
    on EOI/EOS presence ;
  - EOI/EOS that arrive late after ERR_PENDING were not systematically
    upgraded to ERROR

This results in confusion about how such ERROR or ERR_PENDING ought to
be handled, which is not quite desirable.

This patch adds a test to se_fl_set() to detect if we're setting EOI or
EOS while ERR_PENDING is present, or the other way around so that any
sequence of EOI/EOS <-> ERR_PENDING results in ERROR being set. This
way there will no longer be possible situations where ERROR is missing
while the other ones are set.
2023-05-23 16:17:04 +02:00
Christopher Faulet
2437377445 MEDIUM: stconn/applet: Allow SF_SL_EOS flag alone
During the refactoring on SC/SE flags, it was stated that SE_FL_EOS flag
should not be set without on of SE_FL_EOI or SE_FL_ERROR flags. In fact, it
is a problem for the QUIC/H3 multiplexer. When a RST_STREAM frame is
received, it means no more data will be received from the peer. And this
happens before the end of the message (RST_STREAM frame received after the
end of the message are ignored). At this stage, it is a problem to report an
error because from the QUIC point of view, it is valid. Data may still be
sent to the peer. If an error is reported, this will stop the data sending
too.

In the same idea, the H1 mulitplexer reports an error when the message is
truncated because of a read0. But only an EOS flag should be reported in
this case, not an error. Fundamentally, it is important to distinguish
errors from shuts for reads because some cases are valid. For instance a H1
client can choose to stop uploading data if it received the server response.

So, relax tests on SE flags by removing BUG_ON_HOT() on SE_FL_EOS flag. For
now, the abort will be handled in the HTTP analyzers.
2023-05-23 15:52:35 +02:00
Amaury Denoyelle
aa39cc9f42 MINOR: quic: fix alignment of oneline show quic
Output of 'show quic' CLI in oneline mode was not correctly done. This
was caused both due to differing qc pointer size and ports length. Force
proper alignment by using maximum sizes as expected and complete with
blanks if needed.

This should be backported up to 2.7.
2023-05-22 14:18:02 +02:00
Amaury Denoyelle
7385ff3f0c BUG/MINOR: quic: handle Tx packet allocation failure properly
qc_prep_app_pkts() is responsible to built several new packets for
sending. It can fail due to memory allocation error. Before this patch,
the Tx buffer was released on error even if some packets were properly
generated.

With this patch, if an error happens on qc_prep_app_pkts(), we still try
to send already built packets if Tx buffer is not empty. The sending
loop is then interrupted and the Tx buffer is released with data
cleared.

This should be backported up to 2.7.
2023-05-22 14:18:02 +02:00
Amaury Denoyelle
f8fbb0b94e MINOR: quic: use WARN_ON for encrypt failures
It is expected that quic_packet_encrypt() and
quic_apply_header_protection() never fails as encryption is done in
place. This allows to remove their return value.

This is useful to simplify error handling on sending path. An error can
only be encountered on the first steps when allocating a new packet or
copying its frame content. After a clear packet is successfully built,
no error is expected on encryption.

However, it's still unclear if our assumption that in-place encryption
function never fail. As such, a WARN_ON() statement is used if an error
is detected at this stage. Currently, it's impossible to properly manage
this without data loss as this will leave partially unencrypted data in
the send buffer. If warning are reported a solution will have to be
implemented.

This should be backported up to 2.7.
2023-05-22 11:20:44 +02:00
Amaury Denoyelle
5eadc27623 MINOR: quic: remove return val of quic_aead_iv_build()
quic_aead_iv_build() should never fail unless we call it with buffers of
different size. This never happens in the code as every input buffers
are of size QUIC_TLS_IV_LEN.

Remove the return value and add a BUG_ON() to prevent future misusage.
This is especially useful to remove one error handling on the sending
patch via quic_packet_encrypt().

This should be backported up to 2.7.
2023-05-22 11:17:18 +02:00
Amaury Denoyelle
8d6d246dbc CLEANUP: mux-quic/h3: complete BUG_ON with comments
Complete each useful BUG_ON statements with a comment to explain its
purpose. Also convert BUG_ON_HOT to BUG_ON as they should not have a
big impact.

This should be backported up to 2.7.
2023-05-22 11:17:18 +02:00
Daniel Epperson
ffdf6a32a7 DOC: add size format section to manual
The manual refers to an HAProxy size format but does not define it.
This patch adds a section to the manual to define the HAProxy size
format.
2023-05-17 17:21:44 +02:00
Christopher Faulet
f48b23f5da [RELEASE] Released version 2.8-dev12
Released version 2.8-dev12 with the following main changes :
    - BUILD: mjson: Fix warning about unused variables
    - MINOR: spoe: Don't stop disabled proxies
    - BUG/MEDIUM: filters: Don't deinit filters for disabled proxies during startup
    - BUG/MINOR: hlua_fcn/queue: fix broken pop_wait()
    - BUG/MINOR: hlua_fcn/queue: fix reference leak
    - CLEANUP: hlua_fcn/queue: make queue:push() easier to read
    - BUG/MINOR: quic: Buggy acknowlegments of acknowlegments function
    - DEBUG: list: add DEBUG_LIST to purposely corrupt list heads after delete
    - MINOR: stats: report the total number of warnings issued
    - MINOR: stats: report the number of times the global maxconn was reached
    - BUG/MINOR: mux-quic: do not prevent shutw on error
    - BUG/MINOR: mux-quic: do not free frame already released by quic-conn
    - BUG/MINOR: mux-quic: no need to subscribe for detach streams
    - MINOR: mux-quic: add traces for stream wake
    - MINOR: mux-quic: do not send STREAM frames if already subscribe
    - MINOR: mux-quic: factorize send subscribing
    - MINOR: mux-quic: simplify return path of qc_send()
    - MEDIUM: quic: streamline error notification
    - MEDIUM: mux-quic: adjust transport layer error handling
    - MINOR: stats: report the listener's protocol along with the address in stats
    - BUG/MEDIUM: mux-fcgi: Never set SE_FL_EOS without SE_FL_EOI or SE_FL_ERROR
    - BUG/MEDIUM: mux-fcgi: Don't request more room if mux is waiting for more data
    - MINOR: stconn: Add a cross-reference between SE descriptor
    - BUG/MINOR: proxy: missing free in free_proxy for redirect rules
    - MINOR: proxy: add http_free_redirect_rule() function
    - BUG/MINOR: http_rules: fix errors paths in http_parse_redirect_rule()
    - CLEANUP: http_act: use http_free_redirect_rule() to clean redirect act
    - MINOR: tree-wide: use free_acl_cond() where relevant
    - CLEANUP: acl: discard prune_acl_cond() function
    - BUG/MINOR: cli: don't complain about empty command on empty lines
    - MINOR: cli: add an option to display the uptime in the CLI's prompt
    - MINOR: master/cli: also implement the timed prompt on the master CLI
    - MINOR: cli: make "show fd" identify QUIC connections and listeners
    - MINOR: httpclient: allow to disable the DNS resolvers of the httpclient
    - BUILD: debug: fix build issue on 32-bit platforms in "debug dev task"
    - MINOR: ncbuf: missing malloc checks in standalone code
    - DOC: lua: fix core.{proxies,frontends,backends} visibility
    - EXAMPLES: fix race condition in lua mailers script
    - BUG/MINOR: errors: handle malloc failure in usermsgs_put()
    - BUG/MINOR: log: fix memory error handling in parse_logsrv()
    - BUG/MINOR: quic: Wrong redispatch for external data on connection socket
    - MINOR: htx: add function to set EOM reliably
    - MINOR: mux-quic: remove dedicated function to handle standalone FIN
    - BUG/MINOR: mux-quic: properly handle buf alloc failure
    - BUG/MINOR: mux-quic: handle properly recv ncbuf alloc failure
    - BUG/MINOR: quic: do not alloc buf count on alloc failure
    - BUG/MINOR: mux-quic: differentiate failure on qc_stream_desc alloc
    - BUG/MINOR: mux-quic: free task on qc_init() app ops failure
    - MEDIUM: session/ssl: return the SSL error string during a SSL handshake error
    - CI: enable monthly Fedora Rawhide clang builds
    - MEDIUM: mworker/cli: does not disconnect the master CLI upon error
    - MINOR: stconn: Remove useless test on sedesc on detach to release the xref
    - MEDIUM: proxy: stop emitting logs for internal proxies when stopping
    - MINOR: ssl: add new sample ssl_c_r_dn
    - BUG/MEDIUM: mux-h2: make sure control frames do not refresh the idle timeout
    - BUILD: ssl: ssl_c_r_dn fetches uses  functiosn only available since 1.1.1
    - BUG/MINOR: mux-quic: handle properly Tx buf exhaustion
    - BUG/MINOR: h3: missing goto on buf alloc failure
    - BUILD: ssl: get0_verified chain is available on libreSSL
    - BUG/MINOR: makefile: use USE_LIBATOMIC instead of USE_ATOMIC
    - MINOR: mux-quic: add trace to stream rcv_buf operation
    - MINOR: mux-quic: properly report end-of-stream on recv
    - MINOR: mux-quic: uninline qc_attach_sc()
    - BUG/MEDIUM: mux-quic: fix EOI for request without payload
    - MINOR: checks: make sure spread-checks is used also at boot time
    - BUG/MINOR: tcp-rules: Don't shortened the inspect-delay when EOI is set
    - REGTESTS: log: Reduce response inspect-delay for last_rule.vtc
    - DOC: config: Clarify conditions to shorten the inspect-delay for TCP rules
    - CLEANUP: server: remove useless tmptrash assigments in srv_update_status()
    - BUG/MINOR: server: memory leak in _srv_update_status_op() on server DOWN
    - CLEANUP: check; Remove some useless assignments to NULL
    - CLEANUP: stats: update the trash chunk where it's used
    - MINOR: clock: measure the total boot time
    - MINOR: stats: report the boot time in "show info"
    - BUG/MINOR: checks: postpone the startup of health checks by the boot time
    - MINOR: clock: provide a function to automatically adjust now_offset
    - BUG/MINOR: clock: automatically adjust the internal clock with the boot time
    - CLEANUP: fcgi-app; Remove useless assignment to NULL
    - REGTESTS: log: Reduce again response inspect-delay for last_rule.vtc
    - CI: drop Fedora m32 pipeline in favour of cross matrix
    - MEDIUM: checks: Stop scheduling healthchecks during stopping stage
    - MEDIUM: resolvers: Stop scheduling resolution during stopping stage
    - BUG/MINOR: hlua: SET_SAFE_LJMP misuse in hlua_event_runner()
    - BUG/MINOR: debug: fix pointer check in debug_parse_cli_task()
2023-05-17 17:10:12 +02:00
Aurelien DARRAGON
b6a24a52a2 BUG/MINOR: debug: fix pointer check in debug_parse_cli_task()
Task pointer check in debug_parse_cli_task() computes the theoric end
address of provided task pointer to check if it is valid or not thanks to
may_access() helper function.

However, relative ending address is calculated by adding task size to 't'
pointer (which is a struct task pointer), thus it will result to incorrect
address since the compiler automatically translates 't + x' to
't + x * sizeof(*t)' internally (with sizeof(*t) != 1 here).

Solving the issue by using 'ptr' (which is the void * raw address) as
starting address to prevent automatic address scaling.

This was revealed by coverity, see GH #2157.

No backport is needed, unless 9867987 ("DEBUG: cli: add "debug dev task"
to show/wake/expire/kill tasks and tasklets") gets backported.
2023-05-17 16:49:17 +02:00
Aurelien DARRAGON
7428adaf0d BUG/MINOR: hlua: SET_SAFE_LJMP misuse in hlua_event_runner()
When hlua_event_runner() pauses the subscription (ie: if the consumer
can't keep up the pace), hlua_traceback() is used to get the current
lua trace (running context) to provide some info to the user.

However, as hlua_traceback() may raise an error (__LJMP) is set, it is
used within a SET_SAFE_LJMP() / RESET_SAFE_LJMP() combination to ensure
lua errors are properly handled and don't result in unexpected behavior.

But the current usage of SET_SAFE_LJMP() within the function is wrong
since hlua_traceback() will run a second time (unprotected) if the
first (protected) attempt fails. This is undefined behavior and could
even lead to crashes.

Hopefully it is very hard to trigger this code path, thus we can consider
this as a minor bug.

Also using this as an opportunity to enhance the message report to make
it more meaningful to the user.

This should fix GH #2159.

It is a 2.8 specific bug, no backport needed unless c84899c636
("MEDIUM: hlua/event_hdl: initial support for event handlers") gets
backported.
2023-05-17 16:48:40 +02:00
Christopher Faulet
06e9c81bd0 MEDIUM: resolvers: Stop scheduling resolution during stopping stage
When the process is stopping, the server resolutions are suspended. However
the task is still periodically woken up for nothing. If there is a huge
number of resolution, it may lead to a noticeable CPU consumption for no
reason.

To avoid this extra CPU cost, we stop to schedule the the resolution tasks
during the stopping stage. Of course, it is only true for server
resolutinos. Dynamic ones, via do-resolve actions, are not concerned. These
ones must still be triggered during stopping stage.

Concretly, during the stopping stage, the resolvers task is no longer
scheduled if there is no running resolutions. In this case, if a do-resolve
action is evaluated, the task is woken up.

This patch should partially solve the issue #2145.
2023-05-17 16:48:33 +02:00
Christopher Faulet
8bca3cc8c7 MEDIUM: checks: Stop scheduling healthchecks during stopping stage
When the process is stopping, the health-checks are suspended. However the
task is still periodically woken up for nothing. If there is a huge number
of health-checks and if they are woken up in same time, it may lead to a
noticeable CPU consumption for no reason.

To avoid this extra CPU cost, we stop to schedule the health-check tasks
when the proxy is disabled or stopped.

This patch should partially solve the issue #2145.
2023-05-17 14:57:10 +02:00