Commit Graph

1922 Commits

Author SHA1 Message Date
Patrick Hemmer
7fccccccea MINOR: acl: add acl() sample fetch
This provides a sample fetch which returns the evaluation result
of the conjunction of named ACLs.
2023-08-01 10:49:06 +02:00
William Lallemand
d52da4ef70 DOC: configuration: rework the custom log format table
Rework the custom log format table to add sample fetch alternatives.

Split the timestamps and the timers from the rest of the tags.
2023-07-28 14:29:08 +02:00
William Lallemand
454c372b60 DOC: configuration: add sample fetches for timing events
Add the alternatives sample fetches for timing events.
2023-07-26 17:45:33 +02:00
William Lallemand
38975c0164 DOC: configuration: describe Td in Timing events
Describe the Td timer in the timing events section.

Must be backported in every stable branches.
2023-07-25 14:58:35 +02:00
William Lallemand
5e63e1636a MEDIUM: sample: implement us and ms variant of utime and ltime
Implement 4 new fetches:

- ms_ltime
- ms_utime

- us_ltime
- us_utime

Which are the same as ltime and utime but with milliseconds and
microseconds input.

The converters also suports the %N conversion specifier like in date(1).

Unfortunately since %N is not supported by strftime, the format string
is parsed twice, once manually to replace %N, and once by strftime.
2023-07-24 17:12:29 +02:00
William Lallemand
739c4e5b1e MINOR: sample: accept_date / request_date return %Ts / %tr timestamp values
Implement %[accept_date] which returns the same as %Ts log-format tag.
Implement %[request_date] which is a timestamp for %tr.

accept_date and request_date take an faculative unit argument which can
be 's', 'ms' or 'us'.

The goal is to be able to convert these 2 timestamps to HAProxy date
format like its done with %T, %tr, %trg etc
2023-07-24 17:12:29 +02:00
William Lallemand
2a46bfe239 MINOR: sample: implement act_conn sample fetch
Implement the act_conn sample fetch which is the same as %ac (actconn)
in the log-format.
2023-07-24 17:12:29 +02:00
William Lallemand
ac87815be9 MINOR: sample: add pid sample
Implement the pid sample fetch.
2023-07-24 17:12:29 +02:00
Willy Tarreau
80cef0c02d [RELEASE] Released version 2.9-dev2
Released version 2.9-dev2 with the following main changes :
    - BUG/MINOR: quic: Possible leak when allocating an encryption level
    - BUG/MINOR: quic: Missing QUIC connection path member initialization
    - BUILD: quic: Compilation fixes for some gcc warnings with -O1
    - DOC: ssl: Fix typo in 'ocsp-update' option
    - DOC: ssl: Add ocsp-update troubleshooting clues and emphasize on crt-list only aspect
    - BUG/MINOR: tcp_sample: bc_{dst,src} return IP not INT
    - MEDIUM: acl/sample: unify sample conv parsing in a single function
    - MINOR: sample: introduce c_pseudo() conv function
    - MEDIUM: sample: add missing ADDR=>? compatibility matrix entries
    - MINOR: sample: fix ipmask sample definition
    - MEDIUM: tree-wide: fetches that may return IPV4+IPV6 now return ADDR
    - MEDIUM: sample: introduce 'same' output type
    - BUG/MINOR: quic: Possible crash in "show quic" dumping packet number spaces
    - BUG/MINOR: cache: A 'max-age=0' cache-control directive can be overriden by a s-maxage
    - BUG/MEDIUM: sink: invalid server list in sink_new_from_logsrv()
    - BUG/MINOR: http_ext: unhandled ERR_ABORT in proxy_http_parse_7239()
    - BUG/MINOR: sink: missing sft free in sink_deinit()
    - BUG/MINOR: ring: size warning incorrectly reported as fatal error
    - BUG/MINOR: ring: maxlen warning reported as alert
    - BUG/MINOR: log: LF upsets maxlen for UDP targets
    - MINOR: sink/api: pass explicit maxlen parameter to sink_write()
    - BUG/MEDIUM: log: improper use of logsrv->maxlen for buffer targets
    - BUG/MINOR: log: fix missing name error message in cfg_parse_log_forward()
    - BUG/MINOR: log: fix multiple error paths in cfg_parse_log_forward()
    - BUG/MINOR: log: free errmsg on error in cfg_parse_log_forward()
    - BUG/MINOR: sink: invalid sft free in sink_deinit()
    - BUG/MINOR: sink: fix errors handling in cfg_post_parse_ring()
    - BUG/MINOR: server: set rid default value in new_server()
    - MINOR: hlua_fcn/mailers: handle timeout mail from mailers section
    - BUG/MINOR: sink/log: properly deinit srv in sink_new_from_logsrv()
    - EXAMPLES: maintain haproxy 2.8 retrocompatibility for lua mailers script
    - BUG/MINOR: hlua_fcn/queue: use atomic load to fetch queue size
    - BUG/MINOR: config: Remove final '\n' in error messages
    - BUG/MINOR: config: Lenient port configuration parsing
    - BUG/MEDIUM: quic: token IV was not computed using a strong secret
    - BUG/MINOR: quic: retry token remove one useless intermediate expand
    - BUG/MEDIUM: quic: missing check of dcid for init pkt including a token
    - BUG/MEDIUM: quic: timestamp shared in token was using internal time clock
    - CLEANUP: quic: remove useless parameter 'key' from quic_packet_encrypt
    - BUG/MINOR: hlua: hlua_yieldk ctx argument should support pointers
    - BUG/MEDIUM: hlua_fcn/queue: bad pop_wait sequencing
    - DOC: config: Fix fc_src description to state the source address is returned
    - BUG/MINOR: sample: Fix wrong overflow detection in add/sub conveters
    - BUG/MINOR: http: Return the right reason for 302
    - MEDIUM: ssl: new sample fetch method to get curve name
    - CI: add naming convention documentation
    - CI: explicitely highlight VTest result section if there's something
    - BUG/MINOR: quic: Unckecked encryption levels availability
    - BUILD: quic: fix warning during compilation using gcc-6.5
    - BUG/MINOR: hlua: add check for lua_newstate
    - BUG/MINOR: h1-htx: Return the right reason for 302 FCGI responses
    - MINOR: lua: Allow reading "proc." scoped vars from LUA core.
    - MINOR: cpuset: add cpu_map_configured() to know if a cpu-map was found
    - BUG/MINOR: config: do not detect NUMA topology when cpu-map is configured
    - BUG/MINOR: cpuset: remove the bogus "proc" from the cpu_map struct
    - BUG/MINOR: init: set process' affinity even in foreground
    - CLEANUP: cpuset: remove the unused proc_t1 field in cpu_map
    - CLEANUP: config: make parse_cpu_set() return documented values
    - BUG/MINOR: server: Don't warn on server resolution failure with init-addr none
    - MINOR: peers: add peers keyword registration
    - MINOR: quic: Stop storing the TX encoded transport parameters
    - MINOR: quic: Dynamic allocation for negotiated Initial TLS cipher context.
    - MINOR: quic: Release asap the negotiated Initial TLS context.
    - MINOR: quic: Add traces to qc_may_build_pkt()
    - MEDIUM: quic: Packet building rework.
    - CLEANUP: quic: Remove a useless TLS related variable from quic_conn_io_cb().
    - MEDIUM: quic: Handshake I/O handler rework.
    - MINOR: quic: Add traces for qc_frm_free()
    - MINOR: quic: add trace about pktns packet/frames releasing
    - BUG/MINOR: quic: Missing parentheses around PTO probe variable.
    - MINOR: quic: Ping from Initial pktns before reaching anti-amplification limit
    - BUG/MINOR: server-state: Ignore empty files
    - BUG/MINOR: server-state: Avoid warning on 'file not found'
    - BUG/MEDIUM: listener: Acquire proxy's lock in relax_listener() if necessary
    - MINOR: quic: QUIC openssl wrapper implementation
    - MINOR: quic: Include QUIC opensssl wrapper header from TLS stacks compatibility header
    - MINOR: quic: Do not enable O-RTT with USE_QUIC_OPENSSL_COMPAT
    - MINOR: quic: Set the QUIC connection as extra data before calling SSL_set_quic_method()
    - MINOR: quic: Do not enable 0RTT with SSL_set_quic_early_data_enabled()
    - MINOR: quic: Add a compilation option for the QUIC OpenSSL wrapper
    - MINOR: quic: Export some KDF functions (QUIC-TLS)
    - MINOR: quic: Make ->set_encryption_secrets() be callable two times
    - MINOR: quic: Initialize TLS contexts for QUIC openssl wrapper
    - MINOR: quic: Call the keylog callback for QUIC openssl wrapper from SSL_CTX_keylog()
    - MINOR: quic: Add a quic_openssl_compat struct to quic_conn struct
    - MINOR: quic: Useless call to SSL_CTX_set_quic_method()
    - MINOR: quic: SSL context initialization with QUIC OpenSSL wrapper.
    - MINOR: quic: Missing encoded transport parameters for QUIC OpenSSL wrapper
    - MINOR: quic: Add "limited-quic" new tuning setting
    - DOC: quic: Add "limited-quic" new tuning setting
    - DOC: install: Document how to build a limited support for QUIC
2023-07-21 20:29:42 +02:00
Frédéric Lécaille
91410302e9 DOC: quic: Add "limited-quic" new tuning setting
Document "limited-quic" new tuning setting which must be used to
enable the QUIC listener bindings when haproxy is compiled against
a TLS/SSL stack without QUIC support.
2023-07-21 19:19:27 +02:00
Mariam John
00b7b49a46 MEDIUM: ssl: new sample fetch method to get curve name
Adds a new sample fetch method to get the curve name used in the
key agreement to enable better observability. In OpenSSLv3, the function
`SSL_get_negotiated_group` returns the NID of the curve and from the NID,
we get the curve name by passing the NID to OBJ_nid2sn. This was not
available in v1.1.1. SSL_get_curve_name(), which returns the curve name
directly was merged into OpenSSL master branch last week but will be available
only in its next release.
2023-07-17 15:45:41 +02:00
Christopher Faulet
46e5876035 DOC: config: Fix fc_src description to state the source address is returned
A typo in the "fc_src" description was fixed. This sample returns the
original source IP address and not the destination one.

This patch should be backported as far as 2.6.
2023-07-17 11:11:39 +02:00
Remi Tricot-Le Breton
a9b1a4edd9 DOC: ssl: Add ocsp-update troubleshooting clues and emphasize on crt-list only aspect
The current limitation of the 'ocsp-update' option and the fact that it
can only be used in crt-lists was puzzling for some people so the doc
was amended to emphasize this specificity. A configuration extract was
added as well.
A few troubleshooting clues were added as well.

Must be backported in 2.8.
2023-07-03 16:04:09 +02:00
Remi Tricot-Le Breton
8352f00261 DOC: ssl: Fix typo in 'ocsp-update' option
This patch fixes a misalignment in the 'ocsp-update' option description
and it splits the example log lines for readability.

Must be backported in 2.8.
2023-07-03 16:03:52 +02:00
Willy Tarreau
fdc57c4021 [RELEASE] Released version 2.9-dev1
Released version 2.9-dev1 with the following main changes :
    - BUG/MINOR: stats: Fix Lua's `get_stats` function
    - MINOR: stats: protect against future stats fields omissions
    - BUG/MINOR: stream: do not use client-fin/server-fin with HTX
    - BUG/MINOR: quic: Possible crash when SSL session init fails
    - CONTRIB: Add vi file extensions to .gitignore
    - BUG/MINOR: spoe: Only skip sending new frame after a receive attempt
    - BUG/MINOR: peers: Improve detection of config errors in peers sections
    - REG-TESTS: stickiness: Delay haproxys start to properly resolv variables
    - DOC: quic: fix misspelled tune.quic.socket-owner
    - DOC: config: fix jwt_verify() example using var()
    - DOC: config: fix rfc7239 converter examples (again)
    - BUG/MINOR: cfgparse-tcp: leak when re-declaring interface from bind line
    - BUG/MINOR: proxy: add missing interface bind free in free_proxy
    - BUG/MINOR: proxy/server: free default-server on deinit
    - BUG/MEDIUM: hlua: Use front SC to detect EOI in HTTP applets' receive functions
    - BUG/MINOR: ssl: log message non thread safe in SSL Hanshake failure
    - BUG/MINOR: quic: Wrong encryption level flags checking
    - BUG/MINOR: quic: Address inversion in "show quic full"
    - BUG/MINOR: server: inherit from netns in srv_settings_cpy()
    - BUG/MINOR: namespace: missing free in netns_sig_stop()
    - BUG/MINOR: quic: Missing initialization (packet number space probing)
    - BUG/MINOR: quic: Possible crash in quic_conn_prx_cntrs_update()
    - BUG/MINOR: quic: Possible endless loop in quic_lstnr_dghdlr()
    - MINOR: quic: Remove pool_zalloc() from qc_new_conn()
    - MINOR: quic: Remove pool_zalloc() from qc_conn_alloc_ssl_ctx()
    - MINOR: quic: Remove pool_zalloc() from quic_dgram_parse()
    - BUG/MINOR: quic: Missing transport parameters initializations
    - BUG/MEDIUM: mworker: increase maxsock with each new worker
    - BUG/MINOR: quic: ticks comparison without ticks API use
    - BUG/MINOR: quic: Missing TLS secret context initialization
    - DOC: Add tune.h2.be.* and tune.h2.fe.* options to table of contents
    - DOC: Add tune.h2.max-frame-size option to table of contents
    - DOC: Attempt to fix dconv parsing error for tune.h2.fe.initial-window-size
    - REGTESTS: h1_host_normalization : Add a barrier to not mix up log messages
    - MEDIUM: mux-h1: Split h1_process_mux() to make code more readable
    - REORG: mux-h1: Rename functions to emit chunk size/crlf in the output buffer
    - MINOR: mux-h1: Add function to append the chunk size to the output buffer
    - MINOR: mux-h1: Add function to prepend the chunk crlf to the output buffer
    - MEDIUM: filters/htx: Don't rely on HTX extra field if payload is filtered
    - MEDIIM: mux-h1: Add splicing support for chunked messages
    - REGTESTS: Add a script to test the kernel splicing with chunked messages
    - CLEANUP: mux-h1: Remove useless __maybe_unused statement
    - BUG/MINOR: http_ext: fix if-none regression in forwardfor option
    - REGTEST: add an extra testcase for ifnone-forwardfor
    - BUG/MINOR: mworker: leak of a socketpair during startup failure
    - BUG/MINOR: quic: Prevent deadlock with CID tree lock
    - MEDIUM: ssl: handle the SSL_ERROR_ZERO_RETURN during the handshake
    - BUG/MINOR: ssl: SSL_ERROR_ZERO_RETURN returns CO_ER_SSL_EMPTY
    - BUILD: mux-h1: silence a harmless fallthrough warning
    - BUG/MEDIUM: quic: error checking buffer large enought to receive the retry tag
    - MINOR: ssl: allow to change the server signature algorithm on server lines
    - MINOR: ssl: allow to change the client-sigalgs on server lines
    - BUG/MINOR: config: fix stick table duplicate name check
    - BUG/MINOR: quic: Missing random bits in Retry packet header
    - BUG/MINOR: quic: Wrong Retry paquet version field endianess
    - BUG/MINOR: quic: Wrong endianess for version field in Retry token
    - IMPORT: slz: implement a synchronous flush() operation
    - MINOR: compression/slz: add support for a pure flush of pending bytes
    - MINOR: quic: Move QUIC TLS encryption level related code (quic_conn_enc_level_init())
    - MINOR: quic: Move QUIC encryption level structure definition
    - MINOR: quic: Implement a packet number space identification function
    - MINOR: quic: Move packet number space related functions
    - MEDIUM: quic: Dynamic allocations of packet number spaces
    - CLEANUP: quic: Remove qc_list_all_rx_pkts() defined but not used
    - MINOR: quic: Add a pool for the QUIC TLS encryption levels
    - MEDIUM: quic: Dynamic allocations of QUIC TLS encryption levels
    - MINOR: quic: Reduce the maximum length of TLS secrets
    - CLEANUP: quic: Remove two useless pools a low QUIC connection level
    - MEDIUM: quic: Handle the RX in one pass
    - MINOR: quic: Remove call to qc_rm_hp_pkts() from I/O callback
    - CLEANUP: quic: Remove server specific about Initial packet number space
    - MEDIUM: quic: Release encryption levels and packet number spaces asap
    - CLEANUP: quic: Remove a useless test about discarded pktns (qc_handle_crypto_frm())
    - MINOR: quic: Move the packet number space status at quic_conn level
    - MINOR: quic: Drop packet with type for discarded packet number space.
    - BUILD: quic: Add a DISGUISE() to please some compiler to qc_prep_hpkts() 1st parameter
    - BUILD: debug: avoid a build warning related to epoll_wait() in debug code
2023-07-02 11:13:42 +02:00
William Lallemand
593c895eed MINOR: ssl: allow to change the client-sigalgs on server lines
This patch introduces the "client-sigalgs" keyword for the server line,
which allows to configure the list of server signature algorithms
negociated during the handshake. Also available as
"ssl-default-server-client-sigalgs" in the global section.
2023-06-29 14:11:46 +02:00
William Lallemand
717f0ad995 MINOR: ssl: allow to change the server signature algorithm on server lines
This patch introduces the "sigalgs" keyword for the server line, which
allows to configure the list of server signature algorithms negociated
during the handshake. Also available as "ssl-default-server-sigalgs" in
the global section.
2023-06-29 13:40:18 +02:00
Tim Duesterhus
bd88998678 DOC: Attempt to fix dconv parsing error for tune.h2.fe.initial-window-size
It appears that dconv dislikes the "see also" part being on the same line as
the regular paragraph. The beginning of the line does not show up in the
rendered version.

Attempt to fix this by inserting an additional newline which is consistent with
other options.
2023-06-20 11:52:47 +02:00
Tim Duesterhus
6a8638ff53 DOC: Add tune.h2.max-frame-size option to table of contents
This option was introduced in a24b35ca18, which
is 2.0+. It should be backported as far as it easily applies.
2023-06-20 11:51:55 +02:00
Tim Duesterhus
6b40cce10a DOC: Add tune.h2.be.* and tune.h2.fe.* options to table of contents
These new options were introduced in commits
9d7abda787 and
ca1027c22f, both of which at in HAProxy 2.8+.
This patch should be backported there.
2023-06-20 11:51:49 +02:00
Aurelien DARRAGON
1bdd490f9f DOC: config: fix rfc7239 converter examples (again)
Complementary fix to ac456ab ("DOC: config: fix rfc7239 converter examples")
since somehow I managed to overlook one example..

This needs to be backported in 2.8 with ac456ab.
2023-06-06 15:15:17 +02:00
Aurelien DARRAGON
b1d94e84e8 DOC: config: fix jwt_verify() example using var()
To prevent bogus matches, var() does not default to string type anymore
since 44c5ff6 ("MEDIUM: vars: make the var() sample fetch function really
return type ANY).

Thanks to the above fix, haproxy now returns an error if var() is used
within an ACL or IF condition and the matching type is not explicitly
set.

However, the documentation was not updated to reflect this change.

This partially fixes GH #2087 and must be backported up to 2.6.
2023-06-06 15:15:17 +02:00
Artur Pydo
a475448161 DOC: quic: fix misspelled tune.quic.socket-owner
Commit 511ddd5 introduced tune.quic.socket-owner parameter related to
QUIC socket behaviour. However it was misspelled in configuration.txt in
'bind' section as tune.quic.conn-owner.
2023-06-06 14:49:40 +02:00
Willy Tarreau
cb6a35fdc1 [RELEASE] Released version 2.9-dev0
Released version 2.9-dev0 with the following main changes :
    - MINOR: version: mention that it's development again
2023-05-31 16:29:19 +02:00
Willy Tarreau
fdd8154ed3 [RELEASE] Released version 2.8.0
Released version 2.8.0 with the following main changes :
    - MINOR: compression: Improve the way Vary header is added
    - BUILD: makefile: search for SSL_INC/wolfssl before SSL_INC
    - MINOR: init: pre-allocate kernel data structures on init
    - DOC: install: add details about WolfSSL
    - BUG/MINOR: ssl_sock: add check for ha_meth
    - BUG/MINOR: thread: add a check for pthread_create
    - BUILD: init: print rlim_cur as regular integer
    - DOC: install: specify the minimum openssl version recommended
    - CLEANUP: mux-quic: remove unneeded fields in qcc
    - MINOR: mux-quic: remove nb_streams from qcc
    - MINOR: quic: fix stats naming for flow control BLOCKED frames
    - BUG/MEDIUM: mux-quic: only set EOI on FIN
    - BUG/MEDIUM: threads: fix a tiny race in thread_isolate()
    - DOC: config: fix rfc7239 converter examples
    - DOC: quic: remove experimental status for QUIC
    - CLEANUP: mux-quic: rename functions for mux_ops
    - CLEANUP: mux-quic: rename internal functions
    - BUG/MINOR: mux-h2: refresh the idle_timer when the mux is empty
    - DOC: config: Fix bind/server/peer documentation in the peers section
    - BUILD: Makefile: use -pthread not -lpthread when threads are enabled
    - CLEANUP: doc: remove 21 totally obsolete docs
    - DOC: install: mention the common strict-aliasing warning on older compilers
    - DOC: install: clarify a few points on the wolfSSL build method
    - MINOR: quic: Add QUIC connection statistical counters values to "show quic"
    - EXAMPLES: update the basic-config-edge file for 2.8
    - MINOR: quic/cli: clarify the "show quic" help message
    - MINOR: version: mention that it's LTS now.
2023-05-31 16:24:38 +02:00
Christopher Faulet
76a98b4905 DOC: config: Fix bind/server/peer documentation in the peers section
Documentation about bind and server directives in the peers section was
retrieved from the proxy part but there are some limitations, especially for
the bind directive. And the same is true for the peer directive. It is
forbidden to have several listening addresses. Multiple addresses or port
range are not allowed.

Here, only the documentation is fixed. The configuration parsing will be
improved later to trigger errors on bad uses.

In addition, it is also specified that unix socket are supported.

This patch partially fixes the issue #2066. It should be backported to all
stable versions.
2023-05-31 12:03:43 +02:00
Amaury Denoyelle
fa86706468 DOC: quic: remove experimental status for QUIC
QUIC support can now be considered production-ready. As such, remove all
statements on the documentation concerning its experimental status.

Do not backport this one.
2023-05-30 14:36:40 +02:00
Aurelien DARRAGON
ac456abc37 DOC: config: fix rfc7239 converter examples
Some rfc7239 converter examples were not working and thus were misleading.
Fixing rfc7239_n2nn and rfc7239_n2np usage examples.

As both converters were introduced in 2.8, no backport needed.
2023-05-30 10:34:05 +02:00
Patrick Hemmer
425d7ad89d MINOR: init: pre-allocate kernel data structures on init
The Linux kernel maintains data structures to track a processes' open file
descriptors, and it expands these structures as necessary when FD usage grows
(at every FD=2^X starting at 64). However when threading is in use, during
expansion the kernel will pause (observed up to 47ms) while it waits for thread
synchronization (see https://bugzilla.kernel.org/show_bug.cgi?id=217366).

This change addresses the issue and avoids the random pauses by opening the
maximum file descriptor during initialization, so that expansion will not occur
while processing traffic.
2023-05-26 09:28:18 +02:00
Willy Tarreau
c8bb9aeb07 [RELEASE] Released version 2.8-dev13
Released version 2.8-dev13 with the following main changes :
    - DOC: add size format section to manual
    - CLEANUP: mux-quic/h3: complete BUG_ON with comments
    - MINOR: quic: remove return val of quic_aead_iv_build()
    - MINOR: quic: use WARN_ON for encrypt failures
    - BUG/MINOR: quic: handle Tx packet allocation failure properly
    - MINOR: quic: fix alignment of oneline show quic
    - MEDIUM: stconn/applet: Allow SF_SL_EOS flag alone
    - MEDIUM: stconn: make the SE_FL_ERR_PENDING to ERROR transition systematic
    - DOC: internal: add a bit of documentation for the stconn closing conditions
    - DOC/MINOR: config: Fix typo in description for `ssl_bc` in configuration.txt
    - BUILD: quic: re-enable chacha20_poly1305 for libressl
    - MINOR: mux-quic: set both EOI EOS for stream fin
    - MINOR: mux-quic: only set EOS on RESET_STREAM recv
    - MINOR: mux-quic: report error on stream-endpoint earlier
    - BUILD: makefile: fix build issue on GNU make < 3.82
    - BUG/MINOR: mux-h2: Check H2_SF_BODY_TUNNEL on H2S flags and not demux frame ones
    - MINOR: mux-h2: Set H2_SF_ES_RCVD flag when decoding the HEADERS frame
    - MINOR: mux-h2: Add a function to propagate termination flags from h2s to SE
    - BUG/MEDIUM: mux-h2: Propagate termination flags when frontend SC is created
    - DEV: add a Lua helper script for SSL keys logging
    - CLEANUP: makefile: don't display a dummy features list without a target
    - BUILD: makefile: do not erase build options for some build options
    - MINOR: quic: Add low level traces (addresses, DCID)
    - BUG/MINOR: quic: Wrong token length check (quic_generate_retry_token())
    - BUG/MINOR: quic: Missing Retry token length on receipt
    - MINOR: quic: Align "show quic" command help information
    - CLEANUP: quic: Indentation fix quic_rx_pkt_retrieve_conn()
    - CLEANUP: quic: Useless tests in qc_rx_pkt_handle()
    - MINOR: quic: Add some counters at QUIC connection level
    - MINOR: quic: Add a counter for sent packets
    - MINOR: hlua: hlua_smp2lua_str() may LJMP
    - MINOR: hlua: hlua_smp2lua() may LJMP
    - MINOR: hlua: hlua_arg2lua() may LJMP
    - DOC: hlua: document hlua_lua2arg() function
    - DOC: hlua: document hlua_lua2smp() function
    - BUG/MINOR: hlua: unsafe hlua_lua2smp() usage
    - BUILD: makefile: commit the tiny FreeBSD makefile stub
    - BUILD: makefile: fix build options when building tools first
    - BUILD: ist: do not put a cast in an array declaration
    - BUILD: ist: use the literal declaration for ist_lc/ist_uc under TCC
    - BUILD: compiler: systematically set USE_OBSOLETE_LINKER with TCC
    - DOC: install: update reference to known supported versions
    - SCRIPTS: publish-release: update the umask to keep group write access
2023-05-24 22:53:55 +02:00
Mariam John
6ff043de2c DOC/MINOR: config: Fix typo in description for ssl_bc in configuration.txt
Fix a minor typo in the description of the `ssl_bc` sample fetch method described under
Section `7.3.4. Fetching samples at Layer 5` in configuration.txt. Changed `other` to `to`.
2023-05-23 17:06:06 +02:00
Daniel Epperson
ffdf6a32a7 DOC: add size format section to manual
The manual refers to an HAProxy size format but does not define it.
This patch adds a section to the manual to define the HAProxy size
format.
2023-05-17 17:21:44 +02:00
Christopher Faulet
f48b23f5da [RELEASE] Released version 2.8-dev12
Released version 2.8-dev12 with the following main changes :
    - BUILD: mjson: Fix warning about unused variables
    - MINOR: spoe: Don't stop disabled proxies
    - BUG/MEDIUM: filters: Don't deinit filters for disabled proxies during startup
    - BUG/MINOR: hlua_fcn/queue: fix broken pop_wait()
    - BUG/MINOR: hlua_fcn/queue: fix reference leak
    - CLEANUP: hlua_fcn/queue: make queue:push() easier to read
    - BUG/MINOR: quic: Buggy acknowlegments of acknowlegments function
    - DEBUG: list: add DEBUG_LIST to purposely corrupt list heads after delete
    - MINOR: stats: report the total number of warnings issued
    - MINOR: stats: report the number of times the global maxconn was reached
    - BUG/MINOR: mux-quic: do not prevent shutw on error
    - BUG/MINOR: mux-quic: do not free frame already released by quic-conn
    - BUG/MINOR: mux-quic: no need to subscribe for detach streams
    - MINOR: mux-quic: add traces for stream wake
    - MINOR: mux-quic: do not send STREAM frames if already subscribe
    - MINOR: mux-quic: factorize send subscribing
    - MINOR: mux-quic: simplify return path of qc_send()
    - MEDIUM: quic: streamline error notification
    - MEDIUM: mux-quic: adjust transport layer error handling
    - MINOR: stats: report the listener's protocol along with the address in stats
    - BUG/MEDIUM: mux-fcgi: Never set SE_FL_EOS without SE_FL_EOI or SE_FL_ERROR
    - BUG/MEDIUM: mux-fcgi: Don't request more room if mux is waiting for more data
    - MINOR: stconn: Add a cross-reference between SE descriptor
    - BUG/MINOR: proxy: missing free in free_proxy for redirect rules
    - MINOR: proxy: add http_free_redirect_rule() function
    - BUG/MINOR: http_rules: fix errors paths in http_parse_redirect_rule()
    - CLEANUP: http_act: use http_free_redirect_rule() to clean redirect act
    - MINOR: tree-wide: use free_acl_cond() where relevant
    - CLEANUP: acl: discard prune_acl_cond() function
    - BUG/MINOR: cli: don't complain about empty command on empty lines
    - MINOR: cli: add an option to display the uptime in the CLI's prompt
    - MINOR: master/cli: also implement the timed prompt on the master CLI
    - MINOR: cli: make "show fd" identify QUIC connections and listeners
    - MINOR: httpclient: allow to disable the DNS resolvers of the httpclient
    - BUILD: debug: fix build issue on 32-bit platforms in "debug dev task"
    - MINOR: ncbuf: missing malloc checks in standalone code
    - DOC: lua: fix core.{proxies,frontends,backends} visibility
    - EXAMPLES: fix race condition in lua mailers script
    - BUG/MINOR: errors: handle malloc failure in usermsgs_put()
    - BUG/MINOR: log: fix memory error handling in parse_logsrv()
    - BUG/MINOR: quic: Wrong redispatch for external data on connection socket
    - MINOR: htx: add function to set EOM reliably
    - MINOR: mux-quic: remove dedicated function to handle standalone FIN
    - BUG/MINOR: mux-quic: properly handle buf alloc failure
    - BUG/MINOR: mux-quic: handle properly recv ncbuf alloc failure
    - BUG/MINOR: quic: do not alloc buf count on alloc failure
    - BUG/MINOR: mux-quic: differentiate failure on qc_stream_desc alloc
    - BUG/MINOR: mux-quic: free task on qc_init() app ops failure
    - MEDIUM: session/ssl: return the SSL error string during a SSL handshake error
    - CI: enable monthly Fedora Rawhide clang builds
    - MEDIUM: mworker/cli: does not disconnect the master CLI upon error
    - MINOR: stconn: Remove useless test on sedesc on detach to release the xref
    - MEDIUM: proxy: stop emitting logs for internal proxies when stopping
    - MINOR: ssl: add new sample ssl_c_r_dn
    - BUG/MEDIUM: mux-h2: make sure control frames do not refresh the idle timeout
    - BUILD: ssl: ssl_c_r_dn fetches uses  functiosn only available since 1.1.1
    - BUG/MINOR: mux-quic: handle properly Tx buf exhaustion
    - BUG/MINOR: h3: missing goto on buf alloc failure
    - BUILD: ssl: get0_verified chain is available on libreSSL
    - BUG/MINOR: makefile: use USE_LIBATOMIC instead of USE_ATOMIC
    - MINOR: mux-quic: add trace to stream rcv_buf operation
    - MINOR: mux-quic: properly report end-of-stream on recv
    - MINOR: mux-quic: uninline qc_attach_sc()
    - BUG/MEDIUM: mux-quic: fix EOI for request without payload
    - MINOR: checks: make sure spread-checks is used also at boot time
    - BUG/MINOR: tcp-rules: Don't shortened the inspect-delay when EOI is set
    - REGTESTS: log: Reduce response inspect-delay for last_rule.vtc
    - DOC: config: Clarify conditions to shorten the inspect-delay for TCP rules
    - CLEANUP: server: remove useless tmptrash assigments in srv_update_status()
    - BUG/MINOR: server: memory leak in _srv_update_status_op() on server DOWN
    - CLEANUP: check; Remove some useless assignments to NULL
    - CLEANUP: stats: update the trash chunk where it's used
    - MINOR: clock: measure the total boot time
    - MINOR: stats: report the boot time in "show info"
    - BUG/MINOR: checks: postpone the startup of health checks by the boot time
    - MINOR: clock: provide a function to automatically adjust now_offset
    - BUG/MINOR: clock: automatically adjust the internal clock with the boot time
    - CLEANUP: fcgi-app; Remove useless assignment to NULL
    - REGTESTS: log: Reduce again response inspect-delay for last_rule.vtc
    - CI: drop Fedora m32 pipeline in favour of cross matrix
    - MEDIUM: checks: Stop scheduling healthchecks during stopping stage
    - MEDIUM: resolvers: Stop scheduling resolution during stopping stage
    - BUG/MINOR: hlua: SET_SAFE_LJMP misuse in hlua_event_runner()
    - BUG/MINOR: debug: fix pointer check in debug_parse_cli_task()
2023-05-17 17:10:12 +02:00
Christopher Faulet
43525abceb DOC: config: Clarify conditions to shorten the inspect-delay for TCP rules
Add a sentence to state when the inspect-delay is shortened for a TCP rule.
2023-05-17 09:21:01 +02:00
Abhijeet Rastogi
df97f472fa MINOR: ssl: add new sample ssl_c_r_dn
This patch addresses #1514, adds the ability to fetch DN of the root
ca that was in the chain when client certificate was verified during SSL
handshake.
2023-05-15 10:48:05 +02:00
William Lallemand
e279f595ad MINOR: httpclient: allow to disable the DNS resolvers of the httpclient
httpclient.resolvers.disabled allow to disable completely the resolvers
of the httpclient, prevents the creation of the "default" resolvers
section, and does not insert the http do-resolve rule in the proxies.
2023-05-11 21:25:37 +02:00
Willy Tarreau
f0e8e79b3b [RELEASE] Released version 2.8-dev11
Released version 2.8-dev11 with the following main changes :
    - BUILD: debug: do not check the isolated_thread variable in non-threaded builds
    - BUILD: quic: fix build warning when threads are disabled
    - CI: more granular failure on generating build matrix
    - CLEANUP: quic: No more used q_buf structure
    - CLEANUP: quic: Rename several <buf> variables in quic_frame.(c|h)
    - CLEANUP: quic: Typo fix for quic_connection_id pool
    - BUG/MINOR: quic: Wrong key update cipher context initialization for encryption
    - BUG/MEDIUM: cache: Don't request more room than the max allowed
    - MEDIUM: stconn: Be sure to always be able to unblock a SC that needs room
    - EXAMPLES: fix IPV6 support for lua mailers script
    - BUILD: ssl: buggy -Werror=dangling-pointer since gcc 13.0
    - DOC: stconn: Update comments about ABRT/SHUT for stconn structure
    - BUG/MEDIUM: stats: Require more room if buffer is almost full
    - DOC: configuration: add info about ssl-engine for 2.6
    - BUG/MINOR: mux-quic: fix transport VS app CONNECTION_CLOSE
    - BUG/MEDIUM: mux-quic: wakeup tasklet to close on error
    - DEV: flags: add a script to decode most flags in the "show sess all" output
    - BUG/MINOR: quic: Possible crash when dumping version information
    - BUG/MINOR: config: make compression work again in defaults section
    - BUG/MEDIUM: stream: Forward shutdowns when unhandled errors are caught
    - MEDIUM: stream: Resync analyzers at the end of process_stream() on change
    - DEV: flags: add missing stream flags to show-sess-to-flags
    - DEV: flags/show-sess-to-flags: only retrieve hex digits from hex fields
    - DEV: flags/show-sess-to-flags: add support for color output
    - CLEANUP: src/listener.c: remove redundant NULL check
2023-05-11 05:33:21 +02:00
Aleksandar Lazic
89fb210214 DOC: configuration: add info about ssl-engine for 2.6
In the announcement of 2.6 is mentioned that the openssl engine
is not enabled by default.

This patch add the information to the configuration.txt.

This is related to GitHub Issue #1752.

Should be back ported to 2.6
2023-05-09 17:31:48 +02:00
Willy Tarreau
d8cbfa5ad5 [RELEASE] Released version 2.8-dev10
Released version 2.8-dev10 with the following main changes :
    - BUG/MINOR: stats: fix typo in `TotalSplicedBytesOut` field name
    - REGTESTS: add success test, "set server" via fqdn
    - MINOR: ssl: disable CRL checks with WolfSSL when no CRL file
    - BUG/MINOR: stream/cli: fix stream age calculation in "show sess"
    - MINOR: debug: clarify "debug dev stream" help message
    - DEBUG: cli: add "debug dev task" to show/wake/expire/kill tasks and tasklets
    - BUG/MINOR: ssl/sample: x509_v_err_str converter output when not found
    - REGTESTS: ssl: simplify X509_V code check in ssl_client_auth.vtc
    - BUILD: cli: fix build on Windows due to isalnum() implemented as a macro
    - MINOR: activity: use a single macro to iterate over all fields
    - MINOR: activity: show the line header inside the SHOW_VAL macro
    - MINOR: activity: iterate over all fields in a main loop for dumping
    - MINOR: activity: allow "show activity" to restart dumping on any line
    - MINOR: activity: allow "show activity" to restart in the middle of a line
    - DEV: haring: automatically disable DEBUG_STRICT
    - DEV: haring: update readme to suggest using the same build options for haring
    - BUG/MINOR: debug: fix incorrect profiling status reporting in show threads
    - MINOR: debug: permit the "debug dev loop" to run under isolation
    - BUG/MEDIUM: mux-h2: Properly handle end of request to expect data from server
    - BUG/MINOR: mux-quic: prevent quic_conn error code to be overwritten
    - MINOR: mux-quic: add trace event for local error
    - MINOR: mux-quic: wake up after recv only if avail data
    - MINOR: mux-quic: adjust local error API
    - MINOR: mux-quic: report local error on stream endpoint asap
    - MINOR: mux-quic: close connection asap on local error
    - BUG/MINOR: debug: do not emit empty lines in thread dumps
    - BUG/MINOR: mux-h2: Also expect data when waiting for a tunnel establishment
    - BUG/MINOR: time: fix NS_TO_TV macro
    - MEDIUM: debug: simplify the thread dump mechanism
    - MINOR: debug: write panic dump to stderr one thread at a time
    - MINOR: debug: make "show threads" properly iterate over all threads
    - CLEANUP: debug: remove the now unused ha_thread_dump_all_to_trash()
    - MINOR: ssl: allow to change the server signature algorithm
    - MINOR: ssl: allow to change the signature algorithm for client authentication
    - MINOR: cli: Use applet API to write output message
    - MINOR: stats: Use the applet API to write data
    - MINOR: peers: Use the applet API to send message
    - MINOR: stconn: Add a field to specify the room needed by the SC to progress
    - MEDIUM: tree-wide: Change sc API to specify required free space to progress
    - BUG/MEDIUM: stconn: Unblock SC from stream if there is enough room to progrees
    - MEDIUM: applet: Check room needed to unblock opposite SC when data was consumed
    - MEDIUM: stconn: Check room needed to unblock SC on fast-forward
    - MEDIUM: stconn: Check room needed to unblock opposite SC when data was sent
    - MINOR: hlua_fcn: fix Server.is_draining() return type
    - MINOR: hlua_fcn: add Server.is_backup()
    - MINOR: hlua_fcn: add Server.is_dynamic()
    - MINOR: hlua_fcn: add Server.tracking()
    - MINOR: hlua_fcn: add Server.get_trackers()
    - MINOR: hlua_fcn: add Server.get_proxy()
    - MINOR: hlua_fcn: add Server.get_pend_conn() and Server.get_cur_sess()
    - MINOR: hlua_fcn: add Proxy.get_srv_act() and Proxy.get_srv_bck()
    - DOC: lua/event: add ServerEvent class header
    - MINOR: server/event_hdl: publish macro helper
    - MINOR: server/event_hdl: add SERVER_STATE event
    - OPTIM: server: publish UP/DOWN events from STATE change
    - MINOR: hlua: expose SERVER_STATE event
    - MINOR: server/event_hdl: add SERVER_ADMIN event
    - MINOR: hlua: expose SERVER_ADMIN event
    - MINOR: checks/event_hdl: SERVER_CHECK event
    - MINOR: hlua/event_hdl: expose SERVER_CHECK event
    - MINOR: mailers/hlua: disable email sending from lua
    - MINOR: hlua: expose proxy mailers
    - EXAMPLES: add lua mailers script to replace tcpcheck mailers
    - BUG/MINOR: hlua: spinning loop in hlua_socket_handler()
    - MINOR: server: fix message report when IDRAIN is set and MAINT is cleared
    - CLEANUP: hlua: hlua_register_task() may longjmp
    - REGTESTS: use lua mailer script for mailers tests
    - MINOR: hlua: declare hlua_{ref,pushref,unref} functions
    - MINOR: hlua: declare hlua_gethlua() function
    - MINOR: hlua: declare hlua_yieldk() function
    - MINOR: hlua_fcn: add Queue class
    - EXAMPLES: mailqueue for lua mailers script
    - MINOR: quic: add format argument for "show quic"
    - MINOR: quic: implement oneline format for "show quic"
    - MINOR: config: allow cpu-map to take commas in lists of ranges
    - CLEANUP: fix a few reported typos in code comments
    - DOC: fix a few reported typos in the config and install doc
2023-05-07 07:31:54 +02:00
Willy Tarreau
2fefab6f90 DOC: fix a few reported typos in the config and install doc
These are only the few relevant changes among those reported here:

  https://github.com/haproxy/haproxy/actions/runs/4856148287/jobs/8655397661
2023-05-07 07:10:55 +02:00
Willy Tarreau
615c301db4 MINOR: config: allow cpu-map to take commas in lists of ranges
The function that cpu-map uses to parse CPU sets, parse_cpu_set(), was
etended in 2.4 with commit a80823543 ("MINOR: cfgparse: support the
comma separator on parse_cpu_set") to support commas between ranges.
But since it was quite late in the development cycle, by then it was
decided not to add a last-minute surprise and not to magically support
commas in cpu-map, hence the "comma_allowed" argument.

Since then we know that it was not the best choice, because the comma
is silently ignored in the cpu-map syntax, causing all sorts of
surprises in field with threads running on a single node for example.
In addition it's quite common to copy-paste a taskset line and put it
directly into the haproxy configuration.

This commit relaxes this rule an finally allows cpu-map to support
commas between ranges. It simply consists in removing the comma_allowed
argument in the parse_cpu_set() function. The doc was updated to
reflect this.
2023-05-05 18:41:52 +02:00
William Lallemand
b6ae2aafde MINOR: ssl: allow to change the signature algorithm for client authentication
This commit introduces the keyword "client-sigalgs" for the bind line,
which does the same as "sigalgs" but for the client authentication.

"ssl-default-bind-client-sigalgs" allows to set the default parameter
for all the bind lines.

This patch should fix issue #2081.
2023-05-05 00:05:46 +02:00
William Lallemand
1d3c822300 MINOR: ssl: allow to change the server signature algorithm
This patch introduces the "sigalgs" keyword for the bind line, which
allows to configure the list of server signature algorithms negociated
during the handshake. Also available as "ssl-default-bind-sigalgs" in
the default section.

This patch was originally written by Bruno Henc.
2023-05-04 22:43:18 +02:00
William Lallemand
117c7fde06 BUG/MINOR: ssl/sample: x509_v_err_str converter output when not found
The x509_v_err_str converter now outputs the numerical value as a string
when the corresponding constant name was not found.

Must be backported as far as 2.7.
2023-05-03 15:19:38 +02:00
Willy Tarreau
9de10ce478 [RELEASE] Released version 2.8-dev9
Released version 2.8-dev9 with the following main changes :
    - MINOR: quic: Move traces at proto level
    - BUG/MINOR: quic: Possible memory leak from TX packets
    - BUG/MINOR: quic: Possible leak during probing retransmissions
    - BUG/MINOR: quic: Useless probing retransmission in draining or killing state
    - BUG/MINOR: quic: Useless I/O handler task wakeups (draining, killing state)
    - CLEANUP: quic: rename frame types with an explicit prefix
    - CLEANUP: quic: rename frame variables
    - CLEANUP: quic: Remove useless parameters passes to qc_purge_tx_buf()
    - CLEANUP: quic: Rename <buf> variable to <token> in quic_generate_retry_token()
    - CLEANUP: quic: Rename <buf> variable into quic_padding_check()
    - CLEANUP: quic: Rename <buf> variable into quic_rx_pkt_parse()
    - CLEANUP: quic: Rename <buf> variable for several low level functions
    - CLEANUP: quic: Make qc_build_pkt() be more readable
    - CLEANUP: quic: Rename quic_get_dgram_dcid() <buf> variable
    - CLEANUP: quic: Rename several <buf> variables at low level
    - CLEANUP: quic: Rename <buf> variable into quic_packet_read_long_header()
    - CLEANUP: quic: Rename <buf> variable into qc_parse_hd_form()
    - CLEANUP: quic: Rename several <buf> variables into quic_sock.c
    - DEBUG: crash using an invalid opcode on x86/x86_64 instead of an invalid access
    - DEBUG: crash using an invalid opcode on aarch64 instead of an invalid access
    - DEV: h2: add a script "mkhdr" to build h2 frames from scratch
    - DEV: h2: support reading frame payload from a file
    - MINOR: acme.sh: add the deploy script for acme.sh in admin directory
    - BUG/MEDIUM: mux-quic: do not emit RESET_STREAM for unknown length
    - BUG/MEDIUM: mux-quic: improve streams fairness to prevent early timeout
    - BUG/MINOR: quic: prevent buggy memcpy for empty STREAM
    - MINOR: mux-quic: do not set buffer for empty STREAM frame
    - MINOR: mux-quic: do not allocate Tx buf for empty STREAM frame
    - MINOR: quic: finalize affinity change as soon as possible
    - BUG/MINOR: quic: fix race on quic_conns list during affinity rebind
    - CI: switch to Fastly CDN to download LibreSSL
    - BUILD: ssl: switch LibreSSL to Fastly CDN
    - BUG/MINOR: clock: fix a few occurrences of 'now' being used in place of 'date'
    - BUG/MINOR: spoe: use "date" not "now" in debug messages
    - BUG/MINOR: activity: show wall-clock date, not internal date in show activity
    - BUG/MINOR: opentracing: use 'date' instead of 'now' in debug output
    - Revert "BUG/MINOR: clock: fix a few occurrences of 'now' being used in place of 'date'"
    - BUG/MINOR: calltrace: fix 'now' being used in place of 'date'
    - BUG/MINOR: trace: show wall-clock date, not internal date in show activity
    - BUG/MINOR: hlua: return wall-clock date, not internal date in core.now()
    - BUG/MEDIUM: spoe: Don't start new applet if there are enough idle ones
    - BUG/MINOR: stconn: Fix SC flags with same value
    - BUG/MINOR: resolvers: Use sc_need_room() to wait more room when dumping stats
    - BUG/MEDIUM: tcpcheck: Don't eval custom expect rule on an empty buffer
    - BUG/MINOR: stats: report the correct start date in "show info"
    - MINOR: time: add conversions to/from nanosecond timestamps
    - MINOR: time: replace calls to tv_ms_elapsed() with a linear subtract
    - MINOR: spoe: switch the timeval-based timestamps to nanosecond timestamps
    - MEDIUM: tree-wide: replace timeval with nanoseconds in tv_accept and tv_request
    - MINOR: stats: use nanoseconds, not timeval to compute uptime
    - MINOR: activity: use nanoseconds, not timeval to compute uptime
    - MINOR: checks: use a nanosecond counters instead of timeval for checks->start
    - MINOR: clock: do not use now.tv_sec anymore
    - MEDIUM: clock: replace timeval "now" with integer "now_ns"
    - MINOR: clock: replace the timeval start_time with start_time_ns
    - MINOR: sample: Add bc_rtt and bc_rttvar
    - MINOR: quic: use real sending rate measurement
    - MINOR: proxy: factorize send rate measurement
2023-04-28 21:52:13 +02:00
Aleksandar Lazic
5529c9985e MINOR: sample: Add bc_rtt and bc_rttvar
This Patch adds fetch samples for backends round trip time.
2023-04-28 16:31:08 +02:00
Willy Tarreau
3b50e5c164 [RELEASE] Released version 2.8-dev8
Released version 2.8-dev8 with the following main changes :
    - BUG/MEDIUM: cli: Set SE_FL_EOI flag for '_getsocks' and 'quit' commands
    - BUG/MEDIUM: cli: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: http-client: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: stats: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: log: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: dns: Kill idle DNS sessions during stopping stage
    - BUG/MINOR: resolvers: Wakeup DNS idle task on stopping
    - BUG/MEDIUM: resolvers: Force the connect timeout for DNS resolutions
    - MINOR: hlua: Stop to check the SC state when executing a hlua cli command
    - BUG/MEDIUM: mux-h1: Report EOI when a TCP connection is upgraded to H2
    - BUG/MEDIUM: mux-h2: Never set SE_FL_EOS without SE_FL_EOI or SE_FL_ERROR
    - MINOR: quic: Trace fix in quic_pto_pktns() (handshaske status)
    - BUG/MINOR: quic: Wrong packet number space probing before confirmed handshake
    - MINOR: quic: Modify qc_try_rm_hp() traces
    - MINOR: quic: Dump more information at proto level when building packets
    - MINOR: quic: Add a trace for packet with an ACK frame
    - MINOR: activity: add a line reporting the average CPU usage to "show activity"
    - BUG/MINOR: stick_table: alert when type len has incorrect characters
    - MINOR: thread: keep a bitmask of enabled groups in thread_set
    - MINOR: fd: optimize fd_claim_tgid() for use in fd_insert()
    - MINOR: fd: add a lock bit with the tgid
    - MINOR: fd: implement fd_migrate_on() to migrate on a non-local thread
    - MINOR: receiver: reserve special values for "shards"
    - MINOR: bind-conf: support a new shards value: "by-group"
    - BUG/MEDIUM: fd: don't wait for tmask to stabilize if we're not in it.
    - MINOR: quic: Add packet loss and maximum cc window to "show quic"
    - BUG/MINOR: quic: Ignored less than 1ms RTTs
    - MINOR: quic: Add connection flags to traces
    - BUG/MEDIUM: quic: Code sanitization about acknowledgements requirements
    - BUG/MINOR: quic: Possible wrapped values used as ACK tree purging limit.
    - BUG/MINOR: quic: SIGFPE in quic_cubic_update()
    - MINOR: quic: Display the packet number space flags in traces
    - MINOR: quic: Remove a useless test about probing in qc_prep_pkts()
    - BUG/MINOR: quic: Wrong Application encryption level selection when probing
    - CI: bump "actions/checkout" to v3 for cross zoo matrix
    - CI: enable monthly test on Fedora Rawhide
    - BUG/MINOR: stream: Fix test on SE_FL_ERROR on the wrong entity
    - BUG/MEDIUM: stream: Report write timeouts before testing the flags
    - BUG/MEDIUM: stconn: Do nothing in sc_conn_recv() when the SC needs more room
    - MINOR: stream: Uninline and export sess_set_term_flags() function
    - MINOR: filters: Review and simplify errors handling
    - REGTESTS: fix the race conditions in log_uri.vtc
    - MINOR: channel: Forwad close to other side on abort
    - MINOR: stream: Introduce stream_abort() to abort on both sides in same time
    - MINOR: stconn: Rename SC_FL_SHUTR_NOW in SC_FL_ABRT_WANTED
    - MINOR: channel/stconn: Replace channel_shutr_now() by sc_schedule_abort()
    - MINOR: stconn: Rename SC_FL_SHUTW_NOW in SC_FL_SHUT_WANTED
    - MINOR: channel/stconn: Replace channel_shutw_now() by sc_schedule_shutdown()
    - MINOR: stconn: Rename SC_FL_SHUTR in SC_FL_ABRT_DONE
    - MINOR: channel/stconn: Replace sc_shutr() by sc_abort()
    - MINOR: stconn: Rename SC_FL_SHUTW in SC_FL_SHUT_DONE
    - MINOR: channel/stconn: Replace sc_shutw() by sc_shutdown()
    - MINOR: tree-wide: Replace several chn_cons() by the corresponding SC
    - MINOR: tree-wide: Replace several chn_prod() by the corresponding SC
    - BUG/MINOR: cli: Don't close when SE_FL_ERR_PENDING is set in cli analyzer
    - MINOR: stconn: Stop to set SE_FL_ERROR on sending path
    - MEDIUM: stconn: Forbid applets with more to deliver if EOI was reached
    - MINOR: stconn: Don't clear SE_FL_ERROR when endpoint is reset
    - MINOR: stconn: Add a flag to ack endpoint errors at SC level
    - MINOR: backend: Set SC_FL_ERROR on connection error
    - MINOR: stream: Set SC_FL_ERROR on channels' buffer allocation error
    - MINOR: tree-wide: Test SC_FL_ERROR with SE_FL_ERROR from upper layer
    - MEDIUM: tree-wide: Stop to set SE_FL_ERROR from upper layer
    - MEDIUM: backend: Stop to use SE flags to detect connection errors
    - MEDIUM: stream: Stop to use SE flags to detect read errors from analyzers
    - MEDIUM: stream: Stop to use SE flags to detect endpoint errors
    - MEDIUM: stconn: Rely on SC flags to handle errors instead of SE flags
    - BUG/MINOR: stconn: Don't set SE_FL_ERROR at the end of sc_conn_send()
    - BUG/MINOR: quic: Do not use ack delay during the handshakes
    - CLEANUP: use "offsetof" where appropriate
    - MINOR: ssl: remove OpenSSL 1.0.2 mention into certificate loading error
    - BUG/MEDIUM: http-ana: Properly switch the request in tunnel mode on upgrade
    - BUG/MEDIUM: log: Properly handle client aborts in syslog applet
    - MINOR: stconn: Add a flag to report EOS at the stream-connector level
    - MINOR: stconn: Propagate EOS from a mux to the attached stream-connector
    - MINOR: stconn: Propagate EOS from an applet to the attached stream-connector
    - MINOR: mux-h2: make the initial window size configurable per side
    - MINOR: mux-h2: make the max number of concurrent streams configurable per side
    - BUG/MINOR: task: allow to use tasklet_wakeup_after with tid -1
    - CLEANUP: quic: remove unused QUIC_LOCK label
    - CLEANUP: quic: remove unused scid_node
    - CLEANUP: quic: remove unused qc param on stateless reset token
    - CLEANUP: quic: rename quic_connection_id vars
    - MINOR: quic: remove uneeded tasklet_wakeup after accept
    - MINOR: quic: adjust Rx packet type parsing
    - MINOR: quic: adjust quic CID derive API
    - MINOR: quic: remove TID ref from quic_conn
    - MEDIUM: quic: use a global CID trees list
    - MINOR: quic: remove TID encoding in CID
    - MEDIUM: quic: handle conn bootstrap/handshake on a random thread
    - MINOR: quic: do not proceed to accept for closing conn
    - MINOR: protocol: define new callback set_affinity
    - MINOR: quic: delay post handshake frames after accept
    - MEDIUM: quic: implement thread affinity rebinding
    - BUG/MINOR: quic: transform qc_set_timer() as a reentrant function
    - MINOR: quic: properly finalize thread rebinding
    - MAJOR: quic: support thread balancing on accept
    - MINOR: listener: remove unneeded local accept flag
    - BUG/MINOR: http-ana: Update analyzers on both sides when switching in TUNNEL mode
    - CLEANUP: backend: Remove useless debug message in assign_server()
    - CLEANUP: cli: Remove useless debug message in cli_io_handler()
    - BUG/MEDIUM: stconn: Propagate error on the SC on sending path
    - MINOR: config: add "no-alpn" support for bind lines
    - REGTESTS: add a new "ssl_alpn" test to test ALPN negotiation
    - DOC: add missing documentation for "no-alpn" on bind lines
    - MINOR: ssl: do not set ALPN callback with the empty string
    - MINOR: ssl_crtlist: dump "no-alpn" on "show crtlist" when "no-alpn" was set
    - MEDIUM: config: set useful ALPN defaults for HTTPS and QUIC
    - BUG/MEDIUM: quic: prevent crash on Retry sending
    - BUG/MINOR: cfgparse: make sure to include openssl-compat
    - MINOR: clock: add now_mono_time_fast() function
    - MINOR: clock: add now_cpu_time_fast() function
    - MEDIUM: hlua: reliable timeout detection
    - MEDIUM: hlua: introduce tune.lua.burst-timeout
    - CLEANUP: hlua: avoid confusion between internal timers and tick based timers
    - MINOR: hlua: hook yield on known lua state
    - MINOR: hlua: safe coroutine.create()
    - BUG/MINOR: quic: Stop removing ACK ranges when building packets
    - MINOR: quic: Do not allocate too much ack ranges
    - BUG/MINOR: quic: Unchecked buffer length when building the token
    - BUG/MINOR: quic: Wrong Retry token generation timestamp computing
    - BUG/MINOR: mux-quic: fix crash with app ops install failure
    - BUG/MINOR: mux-quic: properly handle STREAM frame alloc failure
    - BUG/MINOR: h3: fix crash on h3s alloc failure
    - BUG/MINOR: quic: prevent crash on qc_new_conn() failure
    - BUG/MINOR: quic: consume Rx datagram even on error
    - CLEANUP: errors: fix obsolete function comments
    - CLEANUP: server: fix update_status() function comment
    - MINOR: server/event_hdl: add proxy_uuid to event_hdl_cb_data_server
    - MINOR: hlua/event_hdl: rely on proxy_uuid instead of proxy_name for lookups
    - MINOR: hlua/event_hdl: expose proxy_uuid variable in server events
    - MINOR: hlua/event_hdl: fix return type for hlua_event_hdl_cb_data_push_args
    - MINOR: server/event_hdl: prepare for upcoming refactors
    - BUG/MINOR: event_hdl: don't waste 1 event subtype slot
    - CLEANUP: event_hdl: updating obsolete comment for EVENT_HDL_CB_DATA
    - CLEANUP: event_hdl: fix comment typo about _sync assertion
    - MINOR: event_hdl: dynamically allocated event data members
    - MINOR: event_hdl: provide event->when for advanced handlers
    - MINOR: hlua/event_hdl: timestamp for events
    - DOC: lua: restore 80 char limitation
    - BUG/MINOR: server: incorrect report for tracking servers leaving drain
    - MINOR: server: explicitly commit state change in srv_update_status()
    - BUG/MINOR: server: don't miss proxy stats update on server state transitions
    - BUG/MINOR: server: don't miss server stats update on server state transitions
    - BUG/MINOR: server: don't use date when restoring last_change from state file
    - MINOR: server: central update for server counters on state change
    - MINOR: server: propagate server state change to lb through single function
    - MINOR: server: propagate lb changes through srv_lb_propagate()
    - MINOR: server: change adm_st_chg_cause storage type
    - MINOR: server: srv_append_status refacto
    - MINOR: server: change srv_op_st_chg_cause storage type
    - CLEANUP: server: remove unused variables in srv_update_status()
    - CLEANUP: server: fix srv_set_{running, stopping, stopped} function comment
    - MINOR: server: pass adm and op cause to srv_update_status()
    - MEDIUM: server: split srv_update_status() in two functions
    - MINOR: server/event_hdl: prepare for server event data wrapper
    - MINOR: quic: support migrating the listener as well
    - MINOR: quic_sock: index li->per_thr[] on local thread id, not global one
    - MINOR: listener: support another thread dispatch mode: "fair"
    - MINOR: receiver: add a struct shard_info to store info about each shard
    - MINOR: receiver: add RX_F_MUST_DUP to indicate that an rx must be duped
    - MEDIUM: proto: duplicate receivers marked RX_F_MUST_DUP
    - MINOR: proto: skip socket setup for duped FDs
    - MEDIUM: config: permit to start a bind on multiple groups at once
    - MINOR: listener: make accept_queue index atomic
    - MEDIUM: listener: rework thread assignment to consider all groups
    - MINOR: listener: use a common thr_idx from the reference listener
    - MINOR: listener: resync with the thread index before heavy calculations
    - MINOR: listener: make sure to avoid ABA updates in per-thread index
    - MINOR: listener: always compare the local thread as well
    - MINOR: Make `tasklet_free()` safe to be called with `NULL`
    - CLEANUP: Stop checking the pointer before calling `tasklet_free()`
    - CLEANUP: Stop checking the pointer before calling `pool_free()`
    - CLEANUP: Stop checking the pointer before calling `task_free()`
    - CLEANUP: Stop checking the pointer before calling `ring_free()`
    - BUG/MINOR: cli: clarify error message about stats bind-process
    - CI: cirrus-ci: bump FreeBSD image to 13-1
    - REGTESTS: remove unsupported "stats bind-process" keyword
    - CI: extend spellchecker whitelist, add "clen" as well
    - CLEANUP: assorted typo fixes in the code and comments
    - BUG/MINOR: sock_inet: use SO_REUSEPORT_LB where available
    - BUG/MINOR: tools: check libssl and libcrypto separately
    - BUG/MINOR: config: fix NUMA topology detection on FreeBSD
    - BUILD: sock_inet: forward-declare struct receiver
    - BUILD: proto_tcp: export the correct names for proto_tcpv[46]
    - CLEANUP: protocol: move the l3_addrlen to plug a hole in proto_fam
    - CLEANUP: protocol: move the nb_receivers to plug a hole in protocol
    - REORG: listener: move the bind_conf's thread setup code to listener.c
    - MINOR: proxy: make proxy_type_str() recognize peers sections
    - MEDIUM: peers: call bind_complete_thread_setup() to finish the config
    - MINOR: protocol: add a flags field to store info about protocols
    - MINOR: protocol: move the global reuseport flag to the protocols
    - MINOR: listener: automatically adjust shards based on support for SO_REUSEPORT
    - MINOR: protocol: add a function to check if some features are supported
    - MINOR: sock: add a function to check for SO_REUSEPORT support at runtime
    - MINOR: protocol: perform a live check for SO_REUSEPORT support
    - MINOR: listener: do not restrict CLI to first group anymore
    - MINOR: listener: add a new global tune.listener.default-shards setting
    - MEDIUM: listener: switch the default sharding to by-group
2023-04-23 10:21:37 +02:00
Willy Tarreau
0e875cf291 MEDIUM: listener: switch the default sharding to by-group
Sharding by-group is exactly identical to by-process for a single
group, and will use the same number of file descriptors for more than
one group, while significantly lowering the kernel's locking overhead.

Now that all special listeners (cli, peers) are properly handled, and
that support for SO_REUSEPORT is detected at runtime per protocol, there
should be no more reason for now switching to by-group by default.

That's what this patch does. It does only this and nothing else so that
it's easy to revert, should any issue be raised.

Testing on an AMD EPYC 74F3 featuring 24 cores and 48 threads distributed
into 8 core complexes of 3 cores each, shows that configuring 8 groups
(one per CCX) is sufficient to simply double the forwarded connection
rate from 112k to 214k/s, reducing kernel locking from 71 to 55%.
2023-04-23 10:18:16 +02:00
Willy Tarreau
7310164b2c MINOR: listener: add a new global tune.listener.default-shards setting
This new setting accepts "by-process", "by-group" and "by-thread" and
will dictate how listeners will be sharded by default when nothing is
specified. While the default remains "by-process", "by-group" should be
much more efficient with many threads, while not changing anything for
single-group setups.
2023-04-23 09:46:15 +02:00
Willy Tarreau
c1fbdd6397 MINOR: listener: automatically adjust shards based on support for SO_REUSEPORT
Now if multiple shards are explicitly requested, and the listener's
protocol doesn't support SO_REUSEPORT, sharding is disabled, which will
result in the socket being automatically duped if needed. A warning is
emitted when this happens. If "shards by-group" or "shards by-thread"
are used, these will automatically be turned down to 1 since we want
this to be possible easily using -dR on the command line without having
to djust the config. For "by-thread", a diag warning will be emitted to
help troubleshoot possible performance issues.
2023-04-23 09:46:15 +02:00