Commit Graph

2381 Commits

Author SHA1 Message Date
Ilya Shipitsin
5fa29b8a74 CLEANUP: assorted typo fixes in the code and comments
This is 34th iteration of typo fixes
2022-12-07 09:08:18 +01:00
Amaury Denoyelle
e30f378236 MINOR: quic: activate socket per conn by default
Activate QUIC connection socket to achieve the best performance. The
previous behavior can be reverted by tune.quic.socket-owner
configuration option.

This change is part of quic-conn owned socket implementation.

Contrary to its siblings patches, I suggest to not backport it to 2.7.
This should ensure that stable releases behavior is perserved. If a user
faces issues with QUIC performance on 2.7, he can nonetheless change the
default configuration.
2022-12-02 14:45:43 +01:00
Amaury Denoyelle
511ddd5785 MINOR: quic: define config option for socket per conn
Define global configuration option "tune.quic.socket-owner". This option
can be used to activate or not socket per QUIC connection mode. The
default value is "listener" which disable this feature. It can be
activated with the option "connection".

This change is part of quic-conn owned socket implementation.
It may be backported to 2.7 after a period of observation.
2022-12-02 14:45:43 +01:00
Aurelien DARRAGON
d06b9c8b99 DOC/MINOR: api: add documentation for event_hdl feature
This is an initial work for the dedicated
event handler API internal documentation.

The file is located at doc/internals/api/event_hdl.txt

event_hdl feature has been introduced with:
	MINOR: event_hdl: add event handler base api
2022-12-02 09:40:52 +01:00
Dragan Dosen
a9800a0f58 MEDIUM: 51d: add support for 51Degrees V4 with Hash algorithm
This patch also adds a set of new global options:

- 51degrees-use-performance-graph { on | off }
- 51degrees-use-predictive-graph { on | off }
- 51degrees-drift <number>
- 51degrees-difference <number>
- 51degrees-allow-unmatched { on | off }

To build using the latest 51Degrees V4 engine with Hash algorithm, set
USE_51DEGREES_V4=1.

Other supported build options are 51DEGREES_INC, 51DEGREES_LIB and
51DEGREES_SRC which needs to be set to the directory that contains
headers and C files. For example:

make TARGET=<target> USE_51DEGREES_V4=1 51DEGREES_SRC='51D_REPO_PATH'/src
2022-12-01 16:12:21 +01:00
Willy Tarreau
eaded987ee [RELEASE] Released version 2.8-dev0
Released version 2.8-dev0 with the following main changes :
    - MINOR: version: mention that it's development again
2022-12-01 15:25:34 +01:00
Willy Tarreau
437fd289f2 [RELEASE] Released version 2.7.0
Released version 2.7.0 with the following main changes :
    - MINOR: ssl: forgotten newline in error messages on ca-file
    - BUG/MINOR: ssl: shut the ca-file errors emitted during httpclient init
    - DOC: config: provide some configuration hints for "http-reuse"
    - DOC: config: refer to section about quoting in the "add_item" converter
    - DOC: halog: explain how to use -ac and -ad in the help message
    - DOC: config: clarify the fact that SNI should not be used in HTTP scenarios
    - DOC: config: mention that a single monitor-uri rule is supported
    - DOC: config: explain how default matching method for ACL works
    - DOC: config: clarify the fact that "retries" is not just for connections
    - BUILD: halog: fix missing double-quote at end of help line
    - DOC: config: clarify the -m dir and -m dom pattern matching methods
    - MINOR: activity: report uptime in "show activity"
    - REORG: activity/cli: move the "show activity" handler to activity.c
    - DEV: poll: add support for epoll
    - DEV: tcploop: centralize the polling code into wait_for_fd()
    - DEV: tcploop: add support for POLLRDHUP when supported
    - DEV: tcploop: do not report an error on POLLERR
    - DEV: tcploop: add optional support for epoll
    - SCRIPTS: announce-release: add a link to the data plane API
    - CLEANUP: stick-table: fill alignment holes in the stktable struct
    - MINOR: stick-table: store a per-table hash seed and use it
    - MINOR: stick-table: show the shard number in each entry's "show table" output
    - CLEANUP: ncbuf: remove ncb_blk args by value
    - CLEANUP: ncbuf: inline small functions
    - CLEANUP: ncbuf: use standard BUG_ON with DEBUG_STRICT
    - BUG/MINOR: quic: Endless loop during retransmissions
    - MINOR: mux-h2: add the expire task and its expiration date in "show fd"
    - BUG/MINOR: peers: always initialize the stksess shard value
    - REGTESTS: fix peers-related regtests regarding "show table"
    - BUG/MEDIUM: mux-h1: Close client H1C on EOS when there is no output data
    - MINOR: stick-table: change the API of the function used to calculate the shard
    - CLEANUP: peers: factor out the key len calculation in received updates
    - BUG/MINOR: peers: always update the stksess shard number on incoming updates
    - CLEANUP: assorted typo fixes in the code and comments
    - MINOR: mux-h1: add the expire task and its expiration date in "show fd"
    - MINOR: debug: improve error handling on the memstats command parser
    - BUILD: quic: allow build with USE_QUIC and USE_OPENSSL_WOLFSSL
    - CLEANUP: anon: clarify the help message on "debug dev hash"
    - MINOR: debug: relax access restrictions on "debug dev hash" and "memstats"
    - SCRIPTS: run-regtests: add a version check
    - MINOR: version: mention that it's stable now
2022-12-01 15:16:46 +01:00
Ilya Shipitsin
6f86eaae4f CLEANUP: assorted typo fixes in the code and comments
This is 33rd iteration of typo fixes
2022-11-30 14:02:36 +01:00
Willy Tarreau
16b282f4b0 MINOR: stick-table: show the shard number in each entry's "show table" output
Stick-tables support sharding to multiple peers but there was no way to
know to what shard an entry was going to be sent. Let's display this in
the "show table" output to ease debugging.
2022-11-29 12:00:49 +01:00
Willy Tarreau
f386a2de92 DOC: config: clarify the -m dir and -m dom pattern matching methods
There's regularly some confusion about them (do they match at the
beginning, end ? do they support multiple components etc). Tim
suggested to improve the doc in issue #61, it's never too late, so
let's do it now wih a few examples.
2022-11-25 12:02:25 +01:00
Willy Tarreau
0b4a622b49 DOC: config: clarify the fact that "retries" is not just for connections
In issue #412 it was rightfully reported that the wording in "retries"
still exclusively speaks about connection attempts, while since L7
retries with "retry-on" it's no longer a limitation. Let's update the
text.
2022-11-25 11:06:20 +01:00
Willy Tarreau
4f4fea417b DOC: config: explain how default matching method for ACL works
In issue #698, it's made apparent that the default matching method for
ACL keywords can be confusing when a converter is applied, because
depending on the converters used, users may think that the default
matching method from the sample fetch name might apply to the whole
expression. It's easier to understand that this doesn't make sense
when thinking about converters turning to completely different types
(e.g. hdr_beg(host),do_resolve() returns an IP, thus it's obvious
that _beg makes no sense at all).  This patch states this in the
doc to avoid future confusion.
2022-11-25 10:49:41 +01:00
Willy Tarreau
7fe0c62516 DOC: config: mention that a single monitor-uri rule is supported
It was reported in issue #1059 that when multiple monitor-uri rules are
specified, only the last one is used. While this was done on purpose
since a single URI is used, it was not clearly mentioned in the doc,
possibly leading to confusion or wasted time trying to establish a
working setup. Let's clarify this point.
2022-11-25 10:27:05 +01:00
Willy Tarreau
d26fb57e81 DOC: config: clarify the fact that SNI should not be used in HTTP scenarios
As reported by Tim in issue #1373 some warnings are deserved to explain
why using the frontend SNI for routing or connecting to a server is
usually not correct, especially since it can be tempting and used to
make sense in pure TCP scenarios.
2022-11-25 10:14:12 +01:00
Willy Tarreau
b143d110bf DOC: config: refer to section about quoting in the "add_item" converter
As requested by Nick in issue #1719, let's add a reference to the section
about quoting there, since add_item() will often be used with commas and
it's easy to mess up.
2022-11-25 09:27:15 +01:00
Willy Tarreau
44fce8bd73 DOC: config: provide some configuration hints for "http-reuse"
This adds some configuration hints regarding various workloads that do
not manage to achieve high reuse rates due to too low a global maxconn
or thread groups.

This fixes github issue #1472.
2022-11-25 09:20:50 +01:00
Willy Tarreau
0279df9e82 [RELEASE] Released version 2.7-dev10
Released version 2.7-dev10 with the following main changes :
    - MEDIUM: tcp-act: add parameter rst-ttl to silent-drop
    - BUG/MAJOR: quic: Crash upon retransmission of dgrams with several packets
    - MINOR: cli: print parsed command when not found
    - BUG/MAJOR: quic: Crash after discarding packet number spaces
    - CLEANUP: quic: replace "choosen" with "chosen" all over the code
    - MINOR: cli/pools: store "show pools" results into a temporary array
    - MINOR: cli/pools: add sorting capabilities to "show pools"
    - MINOR: cli/pools: add pool name filtering capability to "show pools"
    - DOC: configuration: fix quic prefix typo
    - MINOR: quic: report error if force-retry without cluster-secret
    - MINOR: global: generate random cluster.secret if not defined
    - BUG/MINOR: resolvers: do not run the timeout task when there's no resolution
    - BUG/MINOR: server/idle: at least use atomic stores when updating max_used_conns
    - MINOR: server/idle: make the next_takeover index per-tgroup
    - BUILD: listener: fix build warning on global_listener_rwlock without threads
    - BUG/MAJOR: sched: protect task during removal from wait queue
    - BUILD: sched: fix build with DEBUG_THREAD with the previous commit
    - DOC: quic: add note on performance issue with listener contention
    - BUG/MINOR: cfgparse-listen: fix ebpt_next_dup pointer dereference on proxy "from" inheritance
    - BUG/MINOR: log: fix parse_log_message rfc5424 size check
    - CLEANUP: arg: remove extra check in make_arg_list arg escaping
    - CLEANUP: tools: extra check in utoa_pad
    - MINOR: h1: Consider empty port as invalid in authority for CONNECT
    - MINOR: http: Considere empty ports as valid default ports
    - BUG/MINOR: http-htx: Normalized absolute URIs with an empty port
    - BUG/MINOR: h1: Replace authority validation to conform RFC3986
    - REG-TESTS: http: Add more tests about authority/host matching
    - BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action
    - BUG/MEDIUM: mux-h1: Don't release H1C on timeout if there is a SC attached
    - BUG/MEDIUM: mux-h1: Subscribe for reads on error on sending path
    - BUILD: http-htx: Silent build error about a possible NULL start-line
    - DOC: configuration.txt: add default_value for table_idle signature
    - BUILD: ssl-sock: Silent error about NULL deref in ssl_sock_bind_verifycbk()
    - BUG/MEDIUM: mux-h1: Remove H1C_F_WAIT_NEXT_REQ flag on a next request
    - BUG/MINOR: mux-h1: Fix handling of 408-Request-Time-Out
    - MINOR: mux-h1: Remove H1C_F_WAIT_NEXT_REQ in functions handling errors
    - MINOR: mux-h1: Avoid useless call to h1_send() if no error is sent
    - DOC: configuration.txt: fix typo in table_idle signature
    - BUILD: stick-tables: fix build breakage in xxhash on older compilers
    - BUILD: compiler: include compiler's definitions before ours
    - BUILD: quic: global.h is needed in cfgparse-quic
    - CLEANUP: tools: do not needlessly include xxhash nor cli from tools.h
    - BUILD: flags: really restrict the cases where flags are exposed
    - BUILD: makefile: minor reordering of objects by build time
    - BUILD: quic: silence two invalid build warnings at -O1 with gcc-6.5
    - BUILD: quic: use openssl-compat.h instead of openssl/ssl.h
    - MEDIUM: ssl: add minimal WolfSSL support with OpenSSL compatibility mode
    - MINOR: sample: make the rand() sample fetch function use the statistical_prng
    - MINOR: auth: silence null dereference warning in check_user()
    - CLEANUP: peers: fix format string for status messages (int signedness)
    - CLEANUP: qpack: fix format string in debugging code (int signedness)
    - CLEANUP: qpack: properly use the QPACK macros not HPACK ones in debug code
    - BUG/MEDIUM: quic: fix datagram dropping on queueing failed
2022-11-24 17:13:05 +01:00
Aurelien DARRAGON
fd766ddfaf DOC: configuration.txt: fix typo in table_idle signature
An extra ',' was mistakenly added in table_idle converter signature
with commit ed36968 ("DOC: configuration.txt: add default_value for
table_idle signature").
2022-11-23 17:21:18 +01:00
Aurelien DARRAGON
ed36968f16 DOC: configuration.txt: add default_value for table_idle signature
table_idle converter takes optional default_value argument.
The documentation correctly describes this usage but <default_value> was
missing in the converter signature.

It should be backported with bbeec37b3 ("MINOR: stick-table:
Add table_expire() and table_idle() new converters")
2022-11-22 19:41:12 +01:00
Christopher Faulet
84cdbe478a BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action
An abosulte URI is marked as normalized if it comes from an H2 client. This
way, we know we can send a relative URI to an H1 server. But, after a
set-uri action, the URI must no longer be considered as normalized.
Otherwise there is no way to send an absolute URI on the server side.

If it is important to update a normalized absolute URI without altering this
property, the host, path and/or query-string must be set separatly.

This patch should fix the issue #1938. It should be backported as far as
2.4.
2022-11-22 17:49:10 +01:00
Amaury Denoyelle
7078fb1f3a DOC: quic: add note on performance issue with listener contention
Complete quic4/quic6 bind lines by a note on performance issues due to
receiver socket contention. Suggest to use sharding to improve the
situation.

This should be backported up to 2.6.
2022-11-22 11:46:29 +01:00
Amaury Denoyelle
28ea31c7cb MINOR: global: generate random cluster.secret if not defined
If no cluster-secret is defined by the user, a random one is silently
generated.

This ensures that at least QUIC Retry tokens are generated if abnormal
conditions are detected. However, it is advisable to specify it in the
configuration for tokens to be valid even after a reload or across LBs
instances in the same cluster.

This should be backported up to 2.6.
2022-11-21 16:41:34 +01:00
Amaury Denoyelle
996ca7d0fa MINOR: quic: report error if force-retry without cluster-secret
QUIC Retry generation relies on global cluster-secret to produce token
valid even after a process restart and across several LBs instances.

Before this patch, Retry is automatically deactivated if no
cluster-secret is provided. This is the case even if a user has
configured a QUIC listener with quic-force-retry. Change this behavior
by now returning an error during configuration parsing. The user must
provide a cluster-secret if quic-force-retry is used.

This shoud be backported up to 2.6.
2022-11-21 16:34:09 +01:00
Amaury Denoyelle
936c135e05 DOC: configuration: fix quic prefix typo
Replace quicv4/quicv6 -> quic4/quic6 as prefix for bind lines of QUIC
listeners.

This should be backported up to 2.6.
2022-11-21 16:14:46 +01:00
Willy Tarreau
7583c36790 MINOR: cli/pools: add pool name filtering capability to "show pools"
Now it becomes possible to match a pool name's prefix, for example:

  $ socat - /tmp/haproxy.sock <<< "show pools match quic byusage"
  Dumping pools usage. Use SIGQUIT to flush them.
    - Pool quic_conn_r (65560 bytes) : 1337 allocated (87653720 bytes), ...
    - Pool quic_crypto (1048 bytes) : 6685 allocated (7005880 bytes), ...
    - Pool quic_conn (4056 bytes) : 1337 allocated (5422872 bytes), ...
    - Pool quic_rxbuf (262168 bytes) : 8 allocated (2097344 bytes), ...
    - Pool quic_connne (184 bytes) : 9359 allocated (1722056 bytes), ...
    - Pool quic_frame (184 bytes) : 7938 allocated (1460592 bytes), ...
    - Pool quic_tx_pac (152 bytes) : 6454 allocated (981008 bytes), ...
    - Pool quic_tls_ke (56 bytes) : 12033 allocated (673848 bytes), ...
    - Pool quic_rx_pac (408 bytes) : 1596 allocated (651168 bytes), ...
    - Pool quic_tls_se (88 bytes) : 6685 allocated (588280 bytes), ...
    - Pool quic_cstrea (88 bytes) : 4011 allocated (352968 bytes), ...
    - Pool quic_tls_iv (24 bytes) : 12033 allocated (288792 bytes), ...
    - Pool quic_dgram (344 bytes) : 732 allocated (251808 bytes), ...
    - Pool quic_arng (56 bytes) : 4011 allocated (224616 bytes), ...
    - Pool quic_conn_c (152 bytes) : 1337 allocated (203224 bytes), ...
  Total: 15 pools, 109578176 bytes allocated, 109578176 used ...

In this case the reported total only concerns the dumped ones.
2022-11-21 10:14:52 +01:00
Willy Tarreau
2fba08faec MINOR: cli/pools: add sorting capabilities to "show pools"
The "show pools" command is used a lot for debugging but didn't get much
love over the years. This patch brings new capabilities:
  - sorting the output by pool names to ese their finding ("byname").
  - sorting the output by reverse item size to spot the biggest ones("bysize")
  - sorting the output by reverse number of allocated bytes ("byusage")

The last one (byusage) also omits displaying the ones with zero allocation.

In addition, an optional max number of output entries may be passed so as
to dump only the N most relevant ones.
2022-11-21 10:14:52 +01:00
Mathias Weiersmueller
d9b7174d99 MEDIUM: tcp-act: add parameter rst-ttl to silent-drop
The silent-drop action was extended with an additional optional parameter,
[rst-ttl <ttl> ], causing HAProxy to send a TCP RST with the specified TTL
towards the client.

With this behaviour, the connection state on your own client-
facing middle-boxes (load balancers, firewalls) will be purged,
but the client will still assume the TCP connection is up because
the TCP RST packet expires before reaching the client.
2022-11-19 04:53:47 +01:00
Willy Tarreau
a0abec8bc0 [RELEASE] Released version 2.7-dev9
Released version 2.7-dev9 with the following main changes :
    - BUILD: quic: QUIC mux build fix for 32-bit build
    - BUILD: scripts: disable tests build on QuicTLS build
    - BUG/MEDIUM: httpclient: segfault when the httpclient parser fails
    - BUILD: ssl_sock: fix null dereference for QUIC build
    - BUILD: quic: Fix build for m68k cross-compilation
    - BUG/MINOR: quic: fix buffer overflow on retry token generation
    - MINOR: quic: add version field on quic_rx_packet
    - MINOR: quic: extend pn_offset field from quic_rx_packet
    - MINOR: quic: define first packet flag
    - MINOR: quic: extract connection retrieval
    - MINOR: quic: split and rename qc_lstnr_pkt_rcv()
    - MINOR: quic: refactor packet drop on reception
    - MINOR: quic: extend Retry token check function
    - BUG/MINOR: log: Preserve message facility when the log target is a ring buffer
    - BUG/MINOR: ring: Properly parse connect timeout
    - BUG/MEDIUM: httpclient/lua: crash when the lua task timeout before the httpclient
    - BUG/MEDIUM: httpclient: check if the httpclient was released in the IO handler
    - REGTESTS: httpclient/lua: test the lua task timeout with the httpclient
    - CI: github: dump the backtrace of coredumps in the alpine container
    - BUILD: Makefile: add "USE_SHM_OPEN" on the linux-musl target
    - DOC: lua: add a note about compression w/ httpclient
    - CLEANUP: mworker/cli: rename the status function to loadstatus
    - MINOR: mworker/cli: does no try to dump the startup-logs w/o USE_SHM_OPEN
    - MINOR: list: fixing typo in MT_LIST_LOCK_ELT
    - DOC/MINOR: list: fixing MT_LIST_LOCK_ELT macro documentation
    - MINOR: list: adding MT_LIST_APPEND_LOCKED macro
    - BUG/MINOR: mux-quic: complete flow-control for uni streams
    - BUG/MEDIUM: compression: handle rewrite errors when updating response headers
    - MINOR: quic: do not crash on unhandled sendto error
    - MINOR: quic: display unknown error sendto counter on stat page
    - MINOR: peers: Support for peer shards
    - MINOR: peers: handle multiple resync requests using shards
    - BUG/MINOR: sink: Only use backend capability for the sink proxies
    - BUG/MINOR: sink: Set default connect/server timeout for implicit ring buffers
    - MINOR: ssl: add the SSL error string when failing to load a certificate
    - MINOR: ssl: add the SSL error string before the chain
    - MEDIUM: ssl: be stricter about chain error
    - BUG/MAJOR: stick-table: don't process store-response rules for applets
    - MINOR: quic: remove unnecessary quic_session_accept()
    - BUG/MINOR: quic: fix subscribe operation
    - BUG/MINOR: log: fixing bug in tcp syslog_io_handler Octet-Counting
    - MINOR: ssl: dump the SSL string error when SSL_CTX_use_PrivateKey() failed.
    - MINOR: quic: add counter for interrupted reception
    - BUG/MINOR: quic: fix race condition on datagram purging
    - CI: add monthly gcc cross compile jobs
    - CLEANUP: assorted typo fixes in the code and comments
    - CLEANUP: ssl: remove dead code in ssl_sock_load_pem_into_ckch()
    - BUG/MINOR: httpclient: fixed memory allocation for the SSL ca_file
    - BUG/MINOR: ssl: Memory leak of DH BIGNUM fields
    - BUG/MINOR: ssl: Memory leak of AUTHORITY_KEYID struct when loading issuer
    - BUG/MINOR: ssl: ocsp structure not freed properly in case of error
    - CI: switch to the "latest" LibreSSL
    - CI: enable QUIC for LibreSSL builds
    - BUG/MEDIUM: ssl: Verify error codes can exceed 63
    - MEDIUM: ssl: {ca,crt}-ignore-err can now use error constant name
    - MINOR: ssl: x509_v_err_str converter transforms an integer to a X509_V_ERR name
    - CLEANUP: cli: rename dynamic error printing state
    - MINOR: cli: define usermsgs print context
    - MINOR: server: clear prefix on stderr logs after add server
    - BUG/MINOR: ssl: bind_conf is uncorrectly accessed when using QUIC
    - BUILD: ssl_utils: fix build on gcc versions before 8
    - BUILD: debug: remove unnecessary quotes in HA_WEAK() calls
    - CI: emit the compiler's version in the build reports
    - IMPORT: xxhash: update xxHash to version 0.8.1
    - IMPORT: slz: declare len to fix debug build when optimal match is enabled
    - IMPORT: slz: mention the potential header in slz_finish()
    - IMPORT: slz: define and use a __fallthrough statement for switch/case
    - BUILD: compiler: add a macro to detect if another one is set and equals 1
    - BUILD: compiler: add a default definition for __has_attribute()
    - BUILD: compiler: define a __fallthrough statement for switch/case
    - BUILD: sample: use __fallthrough in smp_is_rw() and smp_dup()
    - BUILD: quic: use __fallthrough in quic_connect_server()
    - BUILD: ssl/crt-list: use __fallthrough in cli_io_handler_add_crtlist()
    - BUILD: ssl: use __fallthrough in cli_io_handler_commit_{cert,cafile_crlfile}()
    - BUILD: ssl: use __fallthrough in cli_io_handler_tlskeys_files()
    - BUILD: hlua: use __fallthrough in hlua_post_init_state()
    - BUILD: stream: use __fallthrough in stats_dump_full_strm_to_buffer()
    - BUILD: tcpcheck: use __fallthrough in check_proxy_tcpcheck()
    - BUILD: stats: use __fallthrough in stats_dump_proxy_to_buffer()
    - BUILD: peers: use __fallthrough in peer_io_handler()
    - BUILD: hash: use __fallthrough in hash_djb2()
    - BUILD: tools: use __fallthrough in url_decode()
    - BUILD: args: use __fallthrough in make_arg_list()
    - BUILD: acl: use __fallthrough in parse_acl_expr()
    - BUILD: spoe: use __fallthrough in spoe_handle_appctx()
    - BUILD: logs: use __fallthrough in build_log_header()
    - BUILD: check: use __fallthrough in __health_adjust()
    - BUILD: http_act: use __fallthrough in parse_http_del_header()
    - BUILD: h1_htx: use __fallthrough in h1_parse_chunk()
    - BUILD: vars: use __fallthrough in var_accounting_{diff,add}()
    - BUILD: map: use __fallthrough in cli_io_handler_*()
    - BUILD: compression: use __fallthrough in comp_http_payload()
    - BUILD: stconn: use __fallthrough in various shutw() functions
    - BUILD: prometheus: use __fallthrough in promex_dump_metrics() and IO handler()
    - CLEANUP: ssl: remove printf in bind_parse_ignore_err
    - BUG/MINOR: ssl:  crt-ignore-err memory leak with 'all' parameter
    - BUG/MINOR: ssl: Fix potential overflow
    - CLEANUP: stick-table: remove the unused table->exp_next
    - OPTIM: stick-table: avoid atomic ops in stktable_requeue_exp() when possible
    - BUG/MEDIUM: stick-table: fix a race condition when updating the expiration task
    - MEDIUM: http-ana: remove set-cookie2 support
    - BUG/MEDIUM: wdt/clock: properly handle early task hangs
    - MINOR: deinit: add a "quick-exit" option to bypass the deinit step
    - OPTIM: ebtree: make ebmb_insert_prefix() keep a copy the new node's pfx
    - OPTIM: ebtree: make ebmb_insert_prefix() keep a copy the new node's key
    - MINOR: ssl: ssl_sock_load_cert_chain() display error strings
    - MINOR: ssl: reintroduce ERR_GET_LIB(ret) == ERR_LIB_PEM in ssl_sock_load_pem_into_ckch()
    - BUG/MINOR: http-htx: Fix error handling during parsing http replies
    - BUG/MINOR: resolvers: Don't wait periodic resolution on healthcheck failure
    - BUG/MINOR: resolvers: Set port before IP address when processing SRV records
    - BUG/MINOR: mux-fcgi: Be sure to send empty STDING record in case of zero-copy
    - BUG/MEDIUM: mux-fcgi: Avoid value length overflow when it doesn't fit at once
    - BUG/MINOR: ssl: SSL_load_error_strings might not be defined
    - MINOR: pool/debug: create a new pool_alloc_flag() macro
    - MINOR: dynbuf: switch allocation and release to macros to better track users
    - BUG/MINOR: mux-h1: Do not send a last null chunk on body-less answers
    - REG-TESTS: cache: Remove T-E header for 304-Not-Modified responses
    - DOC: config: fix alphabetical ordering of global section
    - MINOR: trace: split the CLI "trace" parser in CLI vs statement
    - MEDIUM: trace: create a new "trace" statement in the "global" section
    - BUG/MEDIUM: ring: fix creation of server in uninitialized ring
    - BUILD: quic: fix dubious 0-byte overflow on qc_release_lost_pkts
    - BUILD: makefile: mark poll and tcploop targets as phony
    - BUILD: makefile: properly pass CC to sub-projects
    - BUILD: makefile: move default verbosity settings to include/make/verbose.mk
    - BUILD: makefile: use $(cmd_MAKE) in quiet mode
    - BUILD: makefile: move the compiler option detection stuff to compiler.mk
    - DEV: poll: make the connect() step an action as well
    - DEV: poll: strip the "do_" prefix from reported function names
    - DEV: poll: indicate the FD's side in front of its value
    - BUG/MINOR: pool/cli: use ullong to report total pool usage in bytes
    - MINOR: mux-h1: Remove usless code inside shutr callback
    - CLEANUP: mux-h1; Rename H1S_F_ERROR flag into H1S_F_ERROR_MASK
    - REORG: mux-h1: Reorg the H1C structure
    - CLEANUP: mux-h1: Rename H1C_F_ST_ERROR and H1C_F_ST_SILENT_SHUT flags
    - MINOR: mux-h1: Add a dedicated enum to deal with H1 connection state
    - MEDIUM: mux-h1: Handle H1C states via its state field instead of H1C_F_ST_*
    - MINOR: mux-h1: Don't handle subscribe for reads in h1_process_demux()
    - CLEANUP: mux-h1: Rename H1C_F_ERR_PENDING into H1C_F_ABRT_PENDING
    - MINOR: mux-h1: Add flag on H1 stream to deal with internal errors
    - MEDIUM: mux-h1: Rely on the H1C to deal with shutdown for reads
    - CLEANUP: mux-h1: Reorder H1 connection flags to avoid holes
    - MEDIUM: mux-h1: Don't report a final error whe a message is aborted
    - MEDIUM: mux-pt: Don't always set a final error on SE on the sending path
    - MEDIUM: mux-h2: Introduce flags to deal with connection read/write errors
    - CLEANUP: mux-h2: Remove unused fields in h2c structures
    - MEDIUM: mux-fcgi: Introduce flags to deal with connection read/write errors
    - MINOR: sconn: Set SE_FL_ERROR only when there is no more data to read
    - MINOR: mux-h1: Rely on a H1S flag to know a WS key was found or not
    - DOC: lua-api: Remove warning about the lua filters
    - BUG/MEDIUM: listener: Fix race condition when updating the global mngmt task
    - CLEANUP: listener: Remove useless task_queue from manage_global_listener_queue
    - BUG/MINOR: mux-h1: Fix error handling when H1S allocation failed on client side
    - DOC: internal: commit notes about polling states and flags
    - DOC: internal: commit notes about polling states and flags on connect()
    - CLEANUP: mux-h1: Don't test h1c in h1_shutw_conn()
    - BUG/MINOR: http_ana/txn: don't re-initialize txn and req var lists
    - BUG/MEDIUM: raw-sock: Don't report connection error if something was received
    - BUG/MINOR: ssl: don't initialize the keylog callback when not required
    - BUILD: Makefile: enable USE_SHM_OPEN by default on freebsd
    - BUG/MEDIUM: peers: messages about unkown tables not correctly ignored
    - MINOR: cfgparse: Always check the section position
    - MEDIUM: thread: Restric nbthread/thread-group(s) to very first global sections
    - BUILD: peers: Remove unused variables
    - MINOR: ncbuf: complete doc for ncb_advance()
    - BUG/MEDIUM: quic: fix unsuccessful handshakes on ncb_advance error
    - BUG/MEDIUM: quic: fix memleak for out-of-order crypto data
    - MINOR: quic: complete traces/debug for handshake
2022-11-18 17:48:49 +01:00
Willy Tarreau
831d613f54 DOC: internal: commit notes about polling states and flags on connect()
Let's keep these notes as references for later use. Polling on connect()
can sometimes return a few unexpected state combinations that such tests
illustrate. They can serve as reminders for special error handling.
2022-11-17 16:49:00 +01:00
Willy Tarreau
c89f665cca DOC: internal: commit notes about polling states and flags
Some detailed observations were made on polling general and POLLHUP
more specifically, they can be useful later.
2022-11-17 16:49:00 +01:00
Christopher Faulet
b6c31b3cda DOC: lua-api: Remove warning about the lua filters
The lua filters api is no longer experimental. There are some filters
depending on this api, thus it is fair to make it stable now.
2022-11-17 14:33:15 +01:00
Willy Tarreau
9fd0542148 MEDIUM: trace: create a new "trace" statement in the "global" section
The exact same commands as those from the CLI may be pre-loaded at boot
time by passing them one per line after the "trace" keyword in the global
section; i.e. just copy-pasting all commands directly there will do the
job. Note that if a ring is mentioned, it needs to be declared before the
global section. Another option is to append another global section after
"ring".

For now the keyword is marked as experimental to discourage its broad
adoption by default. "expose-experimental-directives" needs to be placed
in the global section to expose it.
2022-11-16 17:55:53 +01:00
Willy Tarreau
8e6ad2548c DOC: config: fix alphabetical ordering of global section
the global section keywords were seriously misordered, and it's visible
that some mistakes have induced other ones over time, so it was about
time to fix this. Roughly 20% of the keywords were misplaced.

This commit only reordered the keywords index and their description,
nothing else was changed. It might be backported because it's a real
pain to find certain options there.
2022-11-16 17:55:53 +01:00
Willy Tarreau
e98d385819 MINOR: deinit: add a "quick-exit" option to bypass the deinit step
Once in a while we spot a bug in the deinit code that is complex,
especially when it has to deal with incomplete initializations, and the
ability to bypass this step has regularly been raised. In addition for
fast-reloading setups it could theoretically save some time. Tests have
shown that very large configs can barely save ~100-150ms by skipping the
deinit step. However the ability not to crash if a bug is encountered can
occasionally help.

This patch adds an option to do exactly this. It's obviously not enabled
by default and the documentation discourages from using it, but this might
be useful in the future.
2022-11-15 09:37:09 +01:00
William Lallemand
9fbc84e571 MINOR: ssl: x509_v_err_str converter transforms an integer to a X509_V_ERR name
The x509_v_err_str converter transforms a numerical X509 verify error
to its constant name.
2022-11-10 13:28:37 +01:00
William Lallemand
960fb74cae MEDIUM: ssl: {ca,crt}-ignore-err can now use error constant name
The ca-ignore-err and crt-ignore-err directives are now able to use the
openssl X509_V_ERR constant names instead of the numerical values.

This allow a configuration to survive an OpenSSL upgrade, because the
numerical ID can change between versions. For example
X509_V_ERR_INVALID_CA was 24 in OpenSSL 1 and is 79 in OpenSSL 3.

The list of errors must be updated when a new major OpenSSL version is
released.
2022-11-10 13:28:37 +01:00
Ilya Shipitsin
4a689dad03 CLEANUP: assorted typo fixes in the code and comments
This is 32nd iteration of typo fixes
2022-10-30 17:17:56 +01:00
Frédéric Lécaille
36d1565640 MINOR: peers: Support for peer shards
Add "shards" new keyword for "peers" section to configure the number
of peer shards attached to such secions. This impact all the stick-tables
attached to the section.
Add "shard" new "server" parameter to configure the peers which participate to
all the stick-tables contents distribution. Each peer receive the stick-tables updates
only for keys with this shard value as distribution hash. The "shard" value
is stored in ->shard new server struct member.
cfg_parse_peers() which is the function which is called to parse all
the lines of a "peers" section is modified to parse the "shards" parameter
stored in ->nb_shards new peers struct member.
Add srv_parse_shard() new callback into server.c to pare the "shard"
parameter.
Implement stksess_getkey_hash() to compute the distribution hash for a
stick-table key as the 64-bits xxhash of the key concatenated to the stick-table
name. This function is called by stksess_setkey_shard(), itself
called by the already implemented function which create a new stick-table
key (stksess_new()).
Add ->idlen new stktable struct member to store the stick-table name length
to not have to compute it each time a stick-table key hash is computed.
2022-10-24 10:55:53 +02:00
William Lallemand
a9256194b8 DOC: lua: add a note about compression w/ httpclient
Decompression is not supported by the httpclient.
2022-10-21 11:50:05 +02:00
Willy Tarreau
ea8aebe8c5 [RELEASE] Released version 2.7-dev8
Released version 2.7-dev8 with the following main changes :
    - BUG/MINOR: checks: update pgsql regex on auth packet
    - DOC: config: Fix pgsql-check documentation to make user param mandatory
    - CLEANUP: mux-quic: remove usage of non-standard ull type
    - CLEANUP: quic: remove global var definition in quic_tls header
    - BUG/MINOR: quic: adjust quic_tls prototypes
    - CLEANUP: quic: fix headers
    - CLEANUP: quic: remove unused function prototype
    - CLEANUP: quic: remove duplicated varint code from xprt_quic.h
    - CLEANUP: quic: create a dedicated quic_conn module
    - BUG/MINOR: mux-quic: ignore STOP_SENDING for locally closed stream
    - BUG/MEDIUM: lua: Don't crash in hlua_lua2arg_check on failure
    - BUG/MEDIUM: lua: handle stick table implicit arguments right.
    - BUILD: h1: silence an initiialized warning with gcc-4.7 and -Os
    - MINOR: fd: add a new function to only raise RLIMIT_NOFILE
    - MINOR: init: do not try to shrink existing RLIMIT_NOFIlE
    - BUG/MINOR: http-fetch: Update method after a prefetch in smp_fetch_meth()
    - BUILD: http_fetch: silence an uninitiialized warning with gcc-4/5/6 at -Os
    - BUG/MINOR: hlua: hlua_channel_insert_data() behavior conflicts with documentation
    - MINOR: quic: limit usage of ssl_sock_ctx in favor of quic_conn
    - MINOR: mux-quic: check quic-conn return code on Tx
    - CLEANUP: quic: fix indentation
    - MEDIUM: quic: retrieve frontend destination address
    - CLEANUP: Reapply ist.cocci (2)
    - CLEANUP: Reapply strcmp.cocci
    - CLEANUP: quic/receiver: remove the now unused tx_qring list
    - BUG/MINOR: quic: set IP_PKTINFO socket option for QUIC receivers only
    - MINOR: hlua: some luaL_checktype() calls were not guarded with MAY_LJMP
    - DOC: configuration: missing 'if' in tcp-request content example
    - MINOR: hlua: removing ambiguous lua_pushvalue with 0 index
    - BUG/MAJOR: stick-tables: do not try to index a server name for applets
    - MINOR: plock: support disabling exponential back-off
    - MINOR: freq_ctr: use the thread's local time whenever possible
    - MEDIUM: stick-table: switch the table lock to rwlock
    - MINOR: stick-table: do not take an exclusive lock when downing ref_cnt
    - MINOR: stick-table: move the write lock inside stktable_touch_with_exp()
    - MEDIUM: stick-table: only take the lock when needed in stktable_touch_with_exp()
    - MEDIUM: stick-table: make stksess_kill_if_expired() avoid the exclusive lock
    - MEDIUM: stick-table: return inserted entry in __stktable_store()
    - MEDIUM: stick-table: free newly allocated stkess if it couldn't be inserted
    - MEDIUM: stick-table: switch to rdlock in stktable_lookup() and lookup_key()
    - MEDIUM: stick-table: make stktable_get_entry() look up under a read lock
    - MEDIUM: stick-table: do not take a lock to update t->current anymore.
    - MEDIUM: stick-table: make stktable_set_entry() look up under a read lock
    - MEDIUM: stick-table: requeue the expiration task out of the exclusive lock
    - MINOR: stick-table: split stktable_store() between key and requeue
    - MEDIUM: stick-table: always use atomic ops to requeue the table's task
    - MEDIUM: stick-table: requeue the wakeup task out of the write lock
    - BUG/MINOR: stick-table: fix build with DEBUG_THREAD
    - REORG: mux-fcgi: Extract flags and enums into mux_fcgi-t.h
    - MINOR: flags/mux-fcgi: Decode FCGI connection and stream flags
    - BUG/MEDIUM: mux-h1: Add connection error handling when reading/sending on a pipe
    - BUG/MEDIUM: mux-h1: Handle abort with an incomplete message during parsing
    - BUG/MINOR: server: make sure "show servers state" hides private bits
    - MINOR: checks: use the lighter PRNG for spread checks
    - MEDIUM: checks: spread the checks load over random threads
    - CI: SSL: use proper version generating when "latest" semantic is used
    - CI: SSL: temporarily stick to LibreSSL=3.5.3
    - MINOR: quic: New quic_cstream object implementation
    - MINOR: quic: Extract CRYPTO frame parsing from qc_parse_pkt_frms()
    - MINOR: quic: Use a non-contiguous buffer for RX CRYPTO data
    - BUG/MINOR: quic: Stalled 0RTT connections with big ClientHello TLS message
    - MINOR: quic: Split the secrets key allocation in two parts
    - CLEANUP: quic: remove unused rxbufs member in receiver
    - CLEANUP: quic: improve naming for rxbuf/datagrams handling
    - MINOR: quic: implement datagram cleanup for quic_receiver_buf
    - MINOR: ring: ring_cast_from_area() cast from an allocated area
    - MINOR: buffers: split b_force_xfer() into b_cpy() and b_force_xfer()
    - MINOR: logs: startup-logs can use a shm for logging the reload
    - MINOR: mworker/cli: reload command displays the startup-logs
    - MEDIUM: quic: respect the threads assigned to a bind line
    - DOC: management: update the "reload" command of the master CLI
    - BUILD: ssl_sock: bind_conf uninitialized in ssl_sock_bind_verifycbk()
    - BUG/MEDIUM: httpclient: Don't set EOM flag on an empty HTX message
    - MINOR: httpclient/lua: Don't set req_payload callback if body is empty
    - DOC/CLEANUP: lua-api: some minor corrections
    - DOC: lua-api: updating toolbox link
    - DOC/CLEANUP: lua-api: removing duplicate core.proxies attribute
    - DOC: management: add forgotten "show startup-logs"
    - DOC: management: "show startup-logs" for master CLI
    - CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in matrix.py
    - CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in workflow definition
2022-10-14 20:45:23 +02:00
William Lallemand
5d1e1317c2 DOC: management: "show startup-logs" for master CLI
"show startup-logs" on the master CLI has a slighly different behavior.

No backport needed.
2022-10-14 15:46:19 +02:00
William Lallemand
f76b3b47ea DOC: management: add forgotten "show startup-logs"
The keyword was never documented.

Must be backported in all maintained versions.
2022-10-14 15:43:28 +02:00
Aurelien DARRAGON
b5684c54d1 DOC/CLEANUP: lua-api: removing duplicate core.proxies attribute
core.proxies attribute was found 2 times in the core Class.
Removing the second occurrence, which is purely duplicate and
does not belong here.
2022-10-14 15:18:25 +02:00
Aurelien DARRAGON
846fc7d950 DOC: lua-api: updating toolbox link
Link to lua toolbox was dead (project has been deprecated).
Adding a legacy link to get old toolbox source code as well as
a link to luarocks that seems to have superseded it.
2022-10-14 15:18:25 +02:00
Aurelien DARRAGON
53901f423d DOC/CLEANUP: lua-api: some minor corrections
This is a minor cleanup before 2.7 release to correct some typos
and errors without changing the meaning.
2022-10-14 15:18:25 +02:00
William Lallemand
ef3e5a1b68 DOC: management: update the "reload" command of the master CLI
Update the "reload" command with the new output format.
2022-10-13 18:16:12 +02:00
Aurelien DARRAGON
d49b559a15 DOC: configuration: missing 'if' in tcp-request content example
An example given for tcp-request content rule with lua
was missing 'if' keyword. Using it "as is" makes haproxy unhappy.

The example was introduced with 579d83b05.
So it may be backported as far as 1.6, but it is a really minor typo.
2022-10-12 09:22:05 +02:00
Christopher Faulet
59307b3e4e DOC: config: Fix pgsql-check documentation to make user param mandatory
The username is required in the Start-up message. Thus, since the 2.2, when
this health-check was refactored, the user parameter is mandatory. On prior
versions, when no username is provided, no pgsql check is performed but only
a basic tcpcheck.

This patch should be backported as far as 2.2.
2022-10-03 15:31:23 +02:00
Willy Tarreau
dd4a2a69dc [RELEASE] Released version 2.7-dev7
Released version 2.7-dev7 with the following main changes :
    - BUG/MEDIUM: mux-quic: fix nb_hreq decrement
    - CLEANUP: httpclient: deleted unused variables
    - MINOR: httpclient: enabled the use of SNI presets
    - OPTIM: hpack-huff: reduce the cache footprint of the huffman decoder
    - BUG/MINOR: mux-quic: do not keep detached qcs with empty Tx buffers
    - REORG: mux-quic: extract traces in a dedicated source file
    - REORG: mux-quic: export HTTP related function in a dedicated file
    - MINOR: mux-quic: refactor snd_buf
    - BUG/MEDIUM: mux-quic: properly trim HTX buffer on snd_buf reset
    - BUG/MINOR: mux-h1: Account consumed output data on synchronous connection error
    - BUG/MINOR: log: improper behavior when escaping log data
    - CLEANUP: tools: removing escape_chunk() function
    - MINOR: clock: split local and global date updates
    - MINOR: pollers: only update the local date during busy polling
    - MINOR: clock: do not update the global date too often
    - REGTESTS: 4be_1srv_smtpchk_httpchk_layer47errors: Return valid SMTP replies
    - MINOR: smtpchk: Update expect rule to fully match replies to EHLO commands
    - BUG/MINOR: smtpchk: SMTP Service check should gracefully close SMTP transaction
    - MINOR: list: documenting mt_list_for_each_entry_safe() macro
    - CLEANUP: list: Fix mt_list_for_each_entry_safe indentation
    - BUG/MINOR: hlua: Remove \n in Lua error message built with memprintf
    - MINOR: hlua: Allow argument on lua-lod(-per-thread) directives
    - BUG/MINOR: anon: memory illegal accesses in tools.c with hash_anon and hash_ipanon
    - MEDIUM: mworker/cli: keep the connection of the FD that ask for a reload
    - BUG/MINOR: hlua: fixing ambiguous sizeof in hlua_load_per_thread
    - MINOR: mworker/cli: replace close() by fd_delete()
    - MINOR: mworker: store and shows loading status
    - MINOR: mworker: mworker_cli_proxy_new_listener() returns a bind_conf
    - MINOR: mworker: stores the mcli_reload bind_conf
    - MINOR: mworker/cli: the mcli_reload bind_conf only send the reload status
    - DOC: management: describe the new reload command behavior
    - CLEANUP: list: fix again some style issues in the recent comments
    - BUG/MINOR: stream: Perform errors handling in right order in stream_new()
    - BUG/MEDIUM: stconn: Reset SE descriptor when we fail to create a stream
    - BUG/MEDIUM: resolvers: Remove aborted resolutions from query_ids tree
    - DOC: management: add timeout on the "reload" command
    - BUG/MINOR: ring: fix the size check in ring_make_from_area()
    - BUG/MINOR: config: don't count trailing spaces as empty arg
    - Revert "BUG/MINOR: config: don't count trailing spaces as empty arg"
    - BUG/MINOR: hlua: fixing hlua_http_msg_del_data behavior
    - BUG/MINOR: hlua: fixing hlua_http_msg_insert_data behavior
    - MINOR: cli: Add anonymization on a missed element for 'show sess all'
    - MINOR: cli: remove error message with 'set anon on|off'
    - MINOR: tools: modify hash_ipanon in order to use it in cli
    - MINOR: cli: use hash_ipanon to anonymized address
    - MINOR: cli: Add an anonymization on a missed element in 'show server state'
    - MINOR: config: correct errors about argument number in condition in cfgparse.c
    - MINOR: config: Add other keywords when dump the anonymized configuration file
    - MINOR: config: Add option line when the configuration file is dumped
    - MINOR: cli: correct commentary and replace 'set global-key' name
    - MINOR: tools: Impprove hash_ipanon to support dgram sockets and port offsets
    - MINOR: tools: Impprove hash_ipanon to not hash FD-based addresses
    - BUG/MINOR: hlua: _hlua_http_msg_delete incorrect behavior when offset is used
    - DOC: management: httpclient can resolve server names in URLs
    - BUG/MINOR: hlua: prevent crash when loading numerous arguments using lua-load(per-thread)
    - DOC/CLEANUP: lua-api: removing duplicate date functions doc
    - MINOR: hlua: ambiguous lua_pushvalue with 0 index
    - BUG/MINOR: config: don't count trailing spaces as empty arg (v2)
    - BUG/MEDIUM: config: count line arguments without dereferencing the output
    - BUG/MAJOR: conn-idle: fix hash indexing issues on idle conns
    - BUG/MINOR: config: insufficient syntax check of the global "maxconn" value
    - BUG/MINOR: backend: only enforce turn-around state when not redispatching
2022-10-03 15:20:38 +02:00
Aurelien DARRAGON
28f0a60a75 DOC/CLEANUP: lua-api: removing duplicate date functions doc
As reported by Thierry, core.asctime_date() and core.rfc850() were
both documented multiple times in lua-api doc.

This does not need to be backported.
2022-09-30 15:21:20 +02:00