Commit Graph

16005 Commits

Author SHA1 Message Date
Willy Tarreau
bc84657410 DOC: internals: document the IST API
This one was missing. It should be easier to use now. It is obvious that
some functions are missing, and it looks like ist2str() and istpad() are
exactly the same.
2021-11-08 16:50:48 +01:00
William Dauchy
d3141b1d37 DOC: stats: fix location of the text representation
`info_field_names` and `stat_field_names` no longer exist and have been
moved in stats.c
To avoid changing this comment, just mention the name of the new table
`info_fields` and `stat_fields`

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-11-08 13:46:02 +01:00
Willy Tarreau
64755bf856 Revert "DEV: coccinelle: Add rule to use chunk_istcat() instead of chunk_strncat()"
This reverts commit b9656e4837. It's
not needed anymore since 49b0482ed ("CLEANUP: chunk: remove misleading
chunk_strncat() function").
2021-11-08 13:42:03 +01:00
Willy Tarreau
1f38bdb3f6 BUG/MINOR: cache: properly ignore unparsable max-age in quotes
When "max-age" or "s-maxage" receive their values in quotes, the pointer
to the integer to be parsed is advanced by one, but the error pointer
check doesn't consider this advanced offset, so it will not match a
parse error such as max-age="a" and will take the value zero instead.

This probably needs to be backported, though it's unsure it has any
effect in the real world.
2021-11-08 12:09:27 +01:00
Willy Tarreau
49b0482ed4 CLEANUP: chunk: remove misleading chunk_strncat() function
This function claims to perform an strncat()-like operation but it does
not, it always copies the indicated number of bytes, regardless of the
presence of a NUL character (what is currently done by chunk_memcat()).
Let's remove it and explicitly replace it with chunk_memcat().
2021-11-08 12:08:26 +01:00
Tim Duesterhus
8f1d178cd1 CLEANUP: chunk: Remove duplicated chunk_Xcat implementation
Delegate chunk_istcat, chunk_cat and chunk_strncat to the most generic
chunk_memcat.
2021-11-08 12:08:26 +01:00
Tim Duesterhus
9f7ed8a60c CLEANUP: Apply ist.cocci
This is to make use of `chunk_istcat()`.
2021-11-08 12:08:26 +01:00
Tim Duesterhus
b9656e4837 DEV: coccinelle: Add rule to use chunk_istcat() instead of chunk_strncat()
This replaces `chunk_strncat()` with `chunk_istcat()` if the parameters are the
ist's `.ptr` and `.len`.
2021-11-08 12:08:26 +01:00
Tim Duesterhus
755d2419a4 DEV: coccinelle: Add rule to use chunk_istcat() instead of chunk_memcat()
This replaces `chunk_memcat()` with `chunk_istcat()` if the parameters are the
ist's `.ptr` and `.len`.
2021-11-08 12:08:26 +01:00
Tim Duesterhus
2471f5c2b2 CLEANUP: Apply ist.cocci
Make use of the new rules to use `isttrim()`.
2021-11-08 12:08:26 +01:00
Tim Duesterhus
9b80a6ca8f DEV: coccinelle: Add rule to use isttrim() where possible
This replaces `if (i.len > e) i.len = e;` by `isttrim(i, e)`.
2021-11-08 12:08:26 +01:00
Willy Tarreau
e9f4d67b15 OPTIM: halog: skip fields 64 bits at a time when supported
Some architectures like x86_64 and aarch64 support efficient unaligned
64-bit reads. On such architectures, we already know that each string
passed to field_start() has some margin at the end because it's parsed
using fgets2() which looks for the trailing LF using the same method.
Thus let's skip spaces by packs of 8. This increases the parsing speed
by 35%.
2021-11-08 12:08:26 +01:00
Willy Tarreau
fc76bbc0f5 OPTIM: halog: improve field parser speed for modern compilers
Modern compilers were producing producing less efficient code in the
field_start() loop, by not emitting two conditional jumps for a single
test. However by reordering the test we can merge the optimal case and
the default one and get back to good performance so let's simplify the
test. This improves the parsing speed by 5%.
2021-11-08 12:08:26 +01:00
Willy Tarreau
fce4e69144 CLEANUP: halog: remove unused strl2ui()
strl2ui() isn't used anymore in the code, likely because str2ic() is
often used instead. Let's drop it.
2021-11-08 12:08:26 +01:00
Frédéric Lécaille
c4becf5424 MINOR: quic: Fix potential null pointer dereference
Fix compilation warnings about non initialized pointers.

This partially address #1445 github issue.
2021-11-08 11:31:12 +01:00
Amaury Denoyelle
b9ce14e5a2 MINOR: h3: fix potential NULL dereference
Fix potential allocation failure of HTX start-line during H3 request
decoding. In this case, h3_decode_qcs returns -1 as error code.

This addresses in part github issue #1445.
2021-11-08 09:17:24 +01:00
Amaury Denoyelle
7bb54f9906 MINOR: mux-quic: fix gcc11 warning
Fix minor warnings about an unused variable.

This addresses in part github issue #1445.
2021-11-08 08:59:30 +01:00
Amaury Denoyelle
3cae4049b0 MINOR: h3/qpack: fix gcc11 warnings
Fix minor warnings about unused variables and mixed declarations.

This addresses in part github issue #1445.
2021-11-08 08:59:30 +01:00
Willy Tarreau
87e7eafde4 CLEANUP: halog: make the default usage message fit in small screens
The usage message was starting to have long lines, it's preferable that
it still fits well into a default 80-col display so that options are
easy to find. Also cut that into the 3 parts (input filter, modifier,
output format) for improved legibility.
2021-11-08 08:39:28 +01:00
Tim Duesterhus
16cc16dd82 CLEANUP: Re-apply xalloc_size.cocci
Use a consistent size as the parameter for the *alloc family.
2021-11-08 08:05:39 +01:00
Tim Duesterhus
4c8f75fc31 CLEANUP: Apply ist.cocci
Make use of the new rules to use `istend()`.
2021-11-08 08:05:39 +01:00
Tim Duesterhus
958f50454a DEV: coccinelle: Add rule to use istend() where possible
This replaces `i.ptr + i.len` by `istend()`.
2021-11-08 07:58:18 +01:00
Tim Duesterhus
9c523f1042 DEV: coccinelle: Remove unused expression e
Introduced in ef00c533e1.
2021-11-08 07:58:18 +01:00
Willy Tarreau
08d3220de5 [RELEASE] Released version 2.5-dev13
Released version 2.5-dev13 with the following main changes :
    - SCRIPTS: git-show-backports: re-enable file-based filtering
    - MINOR: jwt: Make invalid static JWT algorithms an error in `jwt_verify` converter
    - MINOR: mux-h2: add trace on extended connect usage
    - BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
    - MINOR: stream/mux: implement websocket stream flag
    - MINOR: connection: implement function to update ALPN
    - MINOR: connection: add alternative mux_ops param for conn_install_mux_be
    - MEDIUM: server/backend: implement websocket protocol selection
    - MINOR: server: add ws keyword
    - BUG/MINOR: resolvers: fix sent messages were counted twice
    - BUG/MINOR: resolvers: throw log message if trash not large enough for query
    - MINOR: resolvers/dns: split dns and resolver counters in dns_counter struct
    - MEDIUM: resolvers: rename dns extra counters to resolvers extra counters
    - BUG/MINOR: jwt: Fix jwt_parse_alg incorrectly returning JWS_ALG_NONE
    - DOC: add QUIC instruction in INSTALL
    - CLEANUP: halog: Remove dead stores
    - DEV: coccinelle: Add ha_free.cocci
    - CLEANUP: Apply ha_free.cocci
    - DEV: coccinelle: Add rule to use `istnext()` where possible
    - CLEANUP: Apply ist.cocci
    - REGTESTS: Use `feature cmd` for 2.5+ tests (2)
    - DOC: internals: move some API definitions to an "api" subdirectory
    - MINOR: quic: Allocate listener RX buffers
    - CLEANUP: quic: Remove useless code
    - MINOR: quic: Enhance the listener RX buffering part
    - MINOR: quic: Remove a useless lock for CRYPTO frames
    - MINOR: quic: Use QUIC_LOCK QUIC specific lock label.
    - MINOR: backend: Get client dst address to set the server's one only if needful
    - MINOR: compression: Warn for 'compression offload' in defaults sections
    - MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
    - DOC: configuration: move the default log formats to their own section
    - MINOR: ssl: make the ssl_fc_sni() sample-fetch function always available
    - MEDIUM: log: add the client's SNI to the default HTTPS log format
    - DOC: config: add an example of reasonably complete error-log-format
    - DOC: config: move error-log-format before custom log format
2021-11-06 09:25:57 +01:00
Willy Tarreau
ec5c110e2d DOC: config: move error-log-format before custom log format
All default formats were described before the custom one, except this
one. Better place them all together before the custom log format. This
only swaps and renumbers the sections.
2021-11-06 09:20:07 +01:00
Willy Tarreau
ecc79bbe28 DOC: config: add an example of reasonably complete error-log-format
This commit adds a suggestion of a useful error-log-format that was
tested with success in production.
2021-11-06 09:20:07 +01:00
Willy Tarreau
68574dd492 MEDIUM: log: add the client's SNI to the default HTTPS log format
During a troublehooting it came obvious that the SNI always ought to
be logged on httpslog, as it explains errors caused by selection of
the default certificate (or failure to do so in case of strict-sni).

This expectation was also confirmed on the mailing list.

Since the field may be empty it appeared important not to leave an
empty string in the current format, so it was decided to place the
field before a '/' preceding the SSL version and ciphers, so that
in the worst case a missing field leads to a field looking like
"/TLSv1.2/AES...", though usually a missing element still results
in a "-" in logs.

This will change the log format for users who already deployed the
2.5-dev versions (hence the medium level) but no released version
was using this format yet so there's no harm for stable deployments.
The reg-test was updated to check for "-" there since we don't send
SNI in reg-tests.

Link: https://www.mail-archive.com/haproxy@formilux.org/msg41410.html
Cc: William Lallemand <wlallemand@haproxy.org>
2021-11-06 09:20:07 +01:00
Willy Tarreau
579259d150 MINOR: ssl: make the ssl_fc_sni() sample-fetch function always available
Its definition is enclosed inside an ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
which is defined since OpenSSL 0.9.8. Having it conditioned like this
prevents us from using it by default in a log format, which could cause
an error on an old or exotic library.

Let's just always define it and make the sample fetch fail to return
anything on such libs instead.
2021-11-06 09:20:07 +01:00
Willy Tarreau
2ed7350f4c DOC: configuration: move the default log formats to their own section
I'm always having a very hard time finding the log-format definition of
httplog, because it's not in the httplog description, and looking for
"httplog" doesn't yield the custom log formats section.

It would make more sense to write these log-formats into their respective
sections where they will be easier to find. That's what this commit does.
2021-11-06 09:20:07 +01:00
Willy Tarreau
6f7497616e MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
Commit 3d2093af9 ("MINOR: connection: Add a connection error code sample
fetch") added these convenient sample-fetch functions but it appears that
due to a misunderstanding the redundant "conn" part was kept in their
name, causing confusion, since "fc" already stands for "front connection".

Let's simply call them "fc_err" and "bc_err" to match all other related
ones before they appear in a final release. The VTC they appeared in were
also updated, and the alpha sort in the keywords table updated.

Cc: William Lallemand <wlallemand@haproxy.org>
2021-11-06 09:20:07 +01:00
Christopher Faulet
44d34bfbe7 MINOR: compression: Warn for 'compression offload' in defaults sections
This directive is documented as being ignored if set in a defaults
section. But it is only mentionned in a small note in the configuration
manual. Thus, now, a warning is emitted. To do so, the errors handling in
parse_compression_options() function was slightly changed.

In addition, this directive is now documented apart from the other
compression directives. This way, it is clearly visible that it must not be
used in a defaults section.
2021-11-05 16:36:42 +01:00
Christopher Faulet
34a3eb4c42 MINOR: backend: Get client dst address to set the server's one only if needful
In alloc_dst_address(), the client destination address must only be
retrieved when we are sure to use it. Most of time, this save a syscall to
getsockname(). It is not a bugfix in itself. But it revealed a bug in the
QUIC part. The CO_FL_ADDR_TO_SET flag is not set when the destination
address is create for anew quic client connection.
2021-11-05 15:25:34 +01:00
Frédéric Lécaille
b0006eee09 MINOR: quic: Use QUIC_LOCK QUIC specific lock label.
Very minor modifications without any impact.
2021-11-05 15:20:04 +01:00
Frédéric Lécaille
46ea033be0 MINOR: quic: Remove a useless lock for CRYPTO frames
->frms_rwlock is an old lock supposed to be used when several threads
could handle the same connection. This is no more the case since this
commit:
 "MINOR: quic: Attach the QUIC connection to a thread."
2021-11-05 15:20:04 +01:00
Frédéric Lécaille
324ecdafbb MINOR: quic: Enhance the listener RX buffering part
Add a buffer per QUIC connection. At this time the listener which receives
the UDP datagram is responsible of identifying the underlying QUIC connection
and must copy the QUIC packets to its buffer.
->pkt_list member has been added to quic_conn struct to enlist the packets
in the order they have been copied to the connection buffer so that to be
able to consume this buffer when the packets are freed. This list is locked
thanks to a R/W lock to protect it from concurent accesses.
quic_rx_packet struct does not use a static buffer anymore to store the QUIC
packets contents.
2021-11-05 15:20:04 +01:00
Frédéric Lécaille
c5c69a0ad2 CLEANUP: quic: Remove useless code
Remove old I/O handler implementation (listener and server).
At this time keep a defined but not used function for servers (qc_srv_pkt_rcv()).
2021-11-05 15:20:04 +01:00
Frédéric Lécaille
c1029f6182 MINOR: quic: Allocate listener RX buffers
At this time we allocate an RX buffer by thread.
Also take the opportunity offered by this patch to rename TX related variable
names to distinguish them from the RX part.
2021-11-05 15:20:04 +01:00
Willy Tarreau
a62f184d3d DOC: internals: move some API definitions to an "api" subdirectory
It's not always easy to figure that there are some docs on internal API
stuff, let's move them to their own directory. There's a diagram for
lists that could be placed there but instead would deserve a greppable
description for quick lookups, so it was not moved there.
2021-11-05 11:53:22 +01:00
Tim Duesterhus
41922af957 REGTESTS: Use feature cmd for 2.5+ tests (2)
This patch effectively is identical to 7ba98480cc.
2021-11-05 08:27:32 +01:00
Tim Duesterhus
284fbe1214 CLEANUP: Apply ist.cocci
Make use of the new rules to use `istnext()`.
2021-11-05 07:48:38 +01:00
Tim Duesterhus
ef00c533e1 DEV: coccinelle: Add rule to use istnext() where possible
This matches both `istadv(..., 1)` as well as raw `.ptr++` uses.
2021-11-05 07:48:38 +01:00
Tim Duesterhus
025b93e3a2 CLEANUP: Apply ha_free.cocci
Use `ha_free()` where possible.
2021-11-05 07:48:38 +01:00
Tim Duesterhus
cc17a6e1d3 DEV: coccinelle: Add ha_free.cocci
Taken from 61cfdf4fd8.
2021-11-05 07:48:38 +01:00
Tim Duesterhus
785b84bb8f CLEANUP: halog: Remove dead stores
Found using clang's scan-build.
2021-11-05 07:48:38 +01:00
Amaury Denoyelle
ad3683be36 DOC: add QUIC instruction in INSTALL
Add a new section about QUIC compilation, based on QUICTLS.
2021-11-03 18:32:22 +01:00
Remi Tricot-Le Breton
7266350181 BUG/MINOR: jwt: Fix jwt_parse_alg incorrectly returning JWS_ALG_NONE
jwt_parse_alg would mistakenly return JWT_ALG_NONE for algorithms "",
"n", "no" and "non" because of a strncmp misuse. It now sees them as
unknown algorithms.

No backport needed.

Cc: Tim Duesterhus <tim@bastelstu.be>
2021-11-03 17:19:48 +01:00
Emeric Brun
f8642ee826 MEDIUM: resolvers: rename dns extra counters to resolvers extra counters
This patch renames all dns extra counters and stats functions, types and
enums using the 'resolv' prefix/suffixes.

The dns extra counter domain id used on cli was replaced by "resolvers"
instead of "dns".

The typed extra counter prefix dumping resolvers domain "D." was
also renamed "N." because it points counters on a Nameserver.

This was done to finish the split between "resolver" and "dns" layers
and to avoid further misunderstanding when haproxy will handle dns
load balancing.

This should not be backported.
2021-11-03 17:16:46 +01:00
Emeric Brun
d174f0e59a MINOR: resolvers/dns: split dns and resolver counters in dns_counter struct
This patch add a union and struct into dns_counter struct to split
application specific counters.

The only current existing application is the resolver.c layer but
in futur we could handle different application such as dns load
balancing with others specific counters.

This patch should not be backported.
2021-11-03 17:16:46 +01:00
Emeric Brun
0161d32df2 BUG/MINOR: resolvers: throw log message if trash not large enough for query
Before this patch the sent error counter was increased
for each targeted nameserver as soon as we were unable to build
the query message into the trash buffer. But this counter is here
to count sent errors at dns.c transport layer and this error is not
related to a nameserver.

This patch stops to increase those counters and sent a log message
to signal the trash buffer size is not large enough to build the query.

Note: This case should not happen except if trash size buffer was
customized to a very low value.

The function was also re-worked to return -1 in this error case
as it was specified in comment. This function is currently
called at multiple point in resolver.c but return code
is still not yet handled. So to advert the user of the malfunction
the log message was added.

This patch should be backported on all versions including the
layer split between dns.c and resolver.c (v >= 2.4)
2021-11-03 17:16:46 +01:00
Emeric Brun
c37caab21c BUG/MINOR: resolvers: fix sent messages were counted twice
The sent messages counter was increased at both resolver.c and dns.c
layers.

This patch let the dns.c layer count the sent messages since this
layer handle a retry if transport layer is not ready (EAGAIN on udp
or tcp session ring buffer full).

This patch should be backported on all versions using a split of those
layers for resolving (v >=2.4)
2021-11-03 17:16:46 +01:00