Commit Graph

2493 Commits

Author SHA1 Message Date
Aurelien DARRAGON
4be36a1337 MINOR: hlua_fcn: add Server.get_trackers()
This function returns an array of servers who are currently tracking
the server.
2023-05-05 16:28:32 +02:00
Aurelien DARRAGON
406511a2df MINOR: hlua_fcn: add Server.tracking()
This function returns the currently tracked server, if any.
2023-05-05 16:28:32 +02:00
Aurelien DARRAGON
7a03dee36f MINOR: hlua_fcn: add Server.is_dynamic()
This function returns true if the current server is dynamic,
meaning that it was instantiated at runtime (ie: from the cli)
2023-05-05 16:28:32 +02:00
Aurelien DARRAGON
c72051d53a MINOR: hlua_fcn: add Server.is_backup()
This function returns true if the current server is a backup server.
2023-05-05 16:28:32 +02:00
William Lallemand
b6ae2aafde MINOR: ssl: allow to change the signature algorithm for client authentication
This commit introduces the keyword "client-sigalgs" for the bind line,
which does the same as "sigalgs" but for the client authentication.

"ssl-default-bind-client-sigalgs" allows to set the default parameter
for all the bind lines.

This patch should fix issue #2081.
2023-05-05 00:05:46 +02:00
William Lallemand
1d3c822300 MINOR: ssl: allow to change the server signature algorithm
This patch introduces the "sigalgs" keyword for the bind line, which
allows to configure the list of server signature algorithms negociated
during the handshake. Also available as "ssl-default-bind-sigalgs" in
the default section.

This patch was originally written by Bruno Henc.
2023-05-04 22:43:18 +02:00
William Lallemand
117c7fde06 BUG/MINOR: ssl/sample: x509_v_err_str converter output when not found
The x509_v_err_str converter now outputs the numerical value as a string
when the corresponding constant name was not found.

Must be backported as far as 2.7.
2023-05-03 15:19:38 +02:00
Willy Tarreau
9de10ce478 [RELEASE] Released version 2.8-dev9
Released version 2.8-dev9 with the following main changes :
    - MINOR: quic: Move traces at proto level
    - BUG/MINOR: quic: Possible memory leak from TX packets
    - BUG/MINOR: quic: Possible leak during probing retransmissions
    - BUG/MINOR: quic: Useless probing retransmission in draining or killing state
    - BUG/MINOR: quic: Useless I/O handler task wakeups (draining, killing state)
    - CLEANUP: quic: rename frame types with an explicit prefix
    - CLEANUP: quic: rename frame variables
    - CLEANUP: quic: Remove useless parameters passes to qc_purge_tx_buf()
    - CLEANUP: quic: Rename <buf> variable to <token> in quic_generate_retry_token()
    - CLEANUP: quic: Rename <buf> variable into quic_padding_check()
    - CLEANUP: quic: Rename <buf> variable into quic_rx_pkt_parse()
    - CLEANUP: quic: Rename <buf> variable for several low level functions
    - CLEANUP: quic: Make qc_build_pkt() be more readable
    - CLEANUP: quic: Rename quic_get_dgram_dcid() <buf> variable
    - CLEANUP: quic: Rename several <buf> variables at low level
    - CLEANUP: quic: Rename <buf> variable into quic_packet_read_long_header()
    - CLEANUP: quic: Rename <buf> variable into qc_parse_hd_form()
    - CLEANUP: quic: Rename several <buf> variables into quic_sock.c
    - DEBUG: crash using an invalid opcode on x86/x86_64 instead of an invalid access
    - DEBUG: crash using an invalid opcode on aarch64 instead of an invalid access
    - DEV: h2: add a script "mkhdr" to build h2 frames from scratch
    - DEV: h2: support reading frame payload from a file
    - MINOR: acme.sh: add the deploy script for acme.sh in admin directory
    - BUG/MEDIUM: mux-quic: do not emit RESET_STREAM for unknown length
    - BUG/MEDIUM: mux-quic: improve streams fairness to prevent early timeout
    - BUG/MINOR: quic: prevent buggy memcpy for empty STREAM
    - MINOR: mux-quic: do not set buffer for empty STREAM frame
    - MINOR: mux-quic: do not allocate Tx buf for empty STREAM frame
    - MINOR: quic: finalize affinity change as soon as possible
    - BUG/MINOR: quic: fix race on quic_conns list during affinity rebind
    - CI: switch to Fastly CDN to download LibreSSL
    - BUILD: ssl: switch LibreSSL to Fastly CDN
    - BUG/MINOR: clock: fix a few occurrences of 'now' being used in place of 'date'
    - BUG/MINOR: spoe: use "date" not "now" in debug messages
    - BUG/MINOR: activity: show wall-clock date, not internal date in show activity
    - BUG/MINOR: opentracing: use 'date' instead of 'now' in debug output
    - Revert "BUG/MINOR: clock: fix a few occurrences of 'now' being used in place of 'date'"
    - BUG/MINOR: calltrace: fix 'now' being used in place of 'date'
    - BUG/MINOR: trace: show wall-clock date, not internal date in show activity
    - BUG/MINOR: hlua: return wall-clock date, not internal date in core.now()
    - BUG/MEDIUM: spoe: Don't start new applet if there are enough idle ones
    - BUG/MINOR: stconn: Fix SC flags with same value
    - BUG/MINOR: resolvers: Use sc_need_room() to wait more room when dumping stats
    - BUG/MEDIUM: tcpcheck: Don't eval custom expect rule on an empty buffer
    - BUG/MINOR: stats: report the correct start date in "show info"
    - MINOR: time: add conversions to/from nanosecond timestamps
    - MINOR: time: replace calls to tv_ms_elapsed() with a linear subtract
    - MINOR: spoe: switch the timeval-based timestamps to nanosecond timestamps
    - MEDIUM: tree-wide: replace timeval with nanoseconds in tv_accept and tv_request
    - MINOR: stats: use nanoseconds, not timeval to compute uptime
    - MINOR: activity: use nanoseconds, not timeval to compute uptime
    - MINOR: checks: use a nanosecond counters instead of timeval for checks->start
    - MINOR: clock: do not use now.tv_sec anymore
    - MEDIUM: clock: replace timeval "now" with integer "now_ns"
    - MINOR: clock: replace the timeval start_time with start_time_ns
    - MINOR: sample: Add bc_rtt and bc_rttvar
    - MINOR: quic: use real sending rate measurement
    - MINOR: proxy: factorize send rate measurement
2023-04-28 21:52:13 +02:00
Aleksandar Lazic
5529c9985e MINOR: sample: Add bc_rtt and bc_rttvar
This Patch adds fetch samples for backends round trip time.
2023-04-28 16:31:08 +02:00
Willy Tarreau
3b50e5c164 [RELEASE] Released version 2.8-dev8
Released version 2.8-dev8 with the following main changes :
    - BUG/MEDIUM: cli: Set SE_FL_EOI flag for '_getsocks' and 'quit' commands
    - BUG/MEDIUM: cli: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: http-client: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: stats: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: log: Eat output data when waiting for appctx shutdown
    - BUG/MEDIUM: dns: Kill idle DNS sessions during stopping stage
    - BUG/MINOR: resolvers: Wakeup DNS idle task on stopping
    - BUG/MEDIUM: resolvers: Force the connect timeout for DNS resolutions
    - MINOR: hlua: Stop to check the SC state when executing a hlua cli command
    - BUG/MEDIUM: mux-h1: Report EOI when a TCP connection is upgraded to H2
    - BUG/MEDIUM: mux-h2: Never set SE_FL_EOS without SE_FL_EOI or SE_FL_ERROR
    - MINOR: quic: Trace fix in quic_pto_pktns() (handshaske status)
    - BUG/MINOR: quic: Wrong packet number space probing before confirmed handshake
    - MINOR: quic: Modify qc_try_rm_hp() traces
    - MINOR: quic: Dump more information at proto level when building packets
    - MINOR: quic: Add a trace for packet with an ACK frame
    - MINOR: activity: add a line reporting the average CPU usage to "show activity"
    - BUG/MINOR: stick_table: alert when type len has incorrect characters
    - MINOR: thread: keep a bitmask of enabled groups in thread_set
    - MINOR: fd: optimize fd_claim_tgid() for use in fd_insert()
    - MINOR: fd: add a lock bit with the tgid
    - MINOR: fd: implement fd_migrate_on() to migrate on a non-local thread
    - MINOR: receiver: reserve special values for "shards"
    - MINOR: bind-conf: support a new shards value: "by-group"
    - BUG/MEDIUM: fd: don't wait for tmask to stabilize if we're not in it.
    - MINOR: quic: Add packet loss and maximum cc window to "show quic"
    - BUG/MINOR: quic: Ignored less than 1ms RTTs
    - MINOR: quic: Add connection flags to traces
    - BUG/MEDIUM: quic: Code sanitization about acknowledgements requirements
    - BUG/MINOR: quic: Possible wrapped values used as ACK tree purging limit.
    - BUG/MINOR: quic: SIGFPE in quic_cubic_update()
    - MINOR: quic: Display the packet number space flags in traces
    - MINOR: quic: Remove a useless test about probing in qc_prep_pkts()
    - BUG/MINOR: quic: Wrong Application encryption level selection when probing
    - CI: bump "actions/checkout" to v3 for cross zoo matrix
    - CI: enable monthly test on Fedora Rawhide
    - BUG/MINOR: stream: Fix test on SE_FL_ERROR on the wrong entity
    - BUG/MEDIUM: stream: Report write timeouts before testing the flags
    - BUG/MEDIUM: stconn: Do nothing in sc_conn_recv() when the SC needs more room
    - MINOR: stream: Uninline and export sess_set_term_flags() function
    - MINOR: filters: Review and simplify errors handling
    - REGTESTS: fix the race conditions in log_uri.vtc
    - MINOR: channel: Forwad close to other side on abort
    - MINOR: stream: Introduce stream_abort() to abort on both sides in same time
    - MINOR: stconn: Rename SC_FL_SHUTR_NOW in SC_FL_ABRT_WANTED
    - MINOR: channel/stconn: Replace channel_shutr_now() by sc_schedule_abort()
    - MINOR: stconn: Rename SC_FL_SHUTW_NOW in SC_FL_SHUT_WANTED
    - MINOR: channel/stconn: Replace channel_shutw_now() by sc_schedule_shutdown()
    - MINOR: stconn: Rename SC_FL_SHUTR in SC_FL_ABRT_DONE
    - MINOR: channel/stconn: Replace sc_shutr() by sc_abort()
    - MINOR: stconn: Rename SC_FL_SHUTW in SC_FL_SHUT_DONE
    - MINOR: channel/stconn: Replace sc_shutw() by sc_shutdown()
    - MINOR: tree-wide: Replace several chn_cons() by the corresponding SC
    - MINOR: tree-wide: Replace several chn_prod() by the corresponding SC
    - BUG/MINOR: cli: Don't close when SE_FL_ERR_PENDING is set in cli analyzer
    - MINOR: stconn: Stop to set SE_FL_ERROR on sending path
    - MEDIUM: stconn: Forbid applets with more to deliver if EOI was reached
    - MINOR: stconn: Don't clear SE_FL_ERROR when endpoint is reset
    - MINOR: stconn: Add a flag to ack endpoint errors at SC level
    - MINOR: backend: Set SC_FL_ERROR on connection error
    - MINOR: stream: Set SC_FL_ERROR on channels' buffer allocation error
    - MINOR: tree-wide: Test SC_FL_ERROR with SE_FL_ERROR from upper layer
    - MEDIUM: tree-wide: Stop to set SE_FL_ERROR from upper layer
    - MEDIUM: backend: Stop to use SE flags to detect connection errors
    - MEDIUM: stream: Stop to use SE flags to detect read errors from analyzers
    - MEDIUM: stream: Stop to use SE flags to detect endpoint errors
    - MEDIUM: stconn: Rely on SC flags to handle errors instead of SE flags
    - BUG/MINOR: stconn: Don't set SE_FL_ERROR at the end of sc_conn_send()
    - BUG/MINOR: quic: Do not use ack delay during the handshakes
    - CLEANUP: use "offsetof" where appropriate
    - MINOR: ssl: remove OpenSSL 1.0.2 mention into certificate loading error
    - BUG/MEDIUM: http-ana: Properly switch the request in tunnel mode on upgrade
    - BUG/MEDIUM: log: Properly handle client aborts in syslog applet
    - MINOR: stconn: Add a flag to report EOS at the stream-connector level
    - MINOR: stconn: Propagate EOS from a mux to the attached stream-connector
    - MINOR: stconn: Propagate EOS from an applet to the attached stream-connector
    - MINOR: mux-h2: make the initial window size configurable per side
    - MINOR: mux-h2: make the max number of concurrent streams configurable per side
    - BUG/MINOR: task: allow to use tasklet_wakeup_after with tid -1
    - CLEANUP: quic: remove unused QUIC_LOCK label
    - CLEANUP: quic: remove unused scid_node
    - CLEANUP: quic: remove unused qc param on stateless reset token
    - CLEANUP: quic: rename quic_connection_id vars
    - MINOR: quic: remove uneeded tasklet_wakeup after accept
    - MINOR: quic: adjust Rx packet type parsing
    - MINOR: quic: adjust quic CID derive API
    - MINOR: quic: remove TID ref from quic_conn
    - MEDIUM: quic: use a global CID trees list
    - MINOR: quic: remove TID encoding in CID
    - MEDIUM: quic: handle conn bootstrap/handshake on a random thread
    - MINOR: quic: do not proceed to accept for closing conn
    - MINOR: protocol: define new callback set_affinity
    - MINOR: quic: delay post handshake frames after accept
    - MEDIUM: quic: implement thread affinity rebinding
    - BUG/MINOR: quic: transform qc_set_timer() as a reentrant function
    - MINOR: quic: properly finalize thread rebinding
    - MAJOR: quic: support thread balancing on accept
    - MINOR: listener: remove unneeded local accept flag
    - BUG/MINOR: http-ana: Update analyzers on both sides when switching in TUNNEL mode
    - CLEANUP: backend: Remove useless debug message in assign_server()
    - CLEANUP: cli: Remove useless debug message in cli_io_handler()
    - BUG/MEDIUM: stconn: Propagate error on the SC on sending path
    - MINOR: config: add "no-alpn" support for bind lines
    - REGTESTS: add a new "ssl_alpn" test to test ALPN negotiation
    - DOC: add missing documentation for "no-alpn" on bind lines
    - MINOR: ssl: do not set ALPN callback with the empty string
    - MINOR: ssl_crtlist: dump "no-alpn" on "show crtlist" when "no-alpn" was set
    - MEDIUM: config: set useful ALPN defaults for HTTPS and QUIC
    - BUG/MEDIUM: quic: prevent crash on Retry sending
    - BUG/MINOR: cfgparse: make sure to include openssl-compat
    - MINOR: clock: add now_mono_time_fast() function
    - MINOR: clock: add now_cpu_time_fast() function
    - MEDIUM: hlua: reliable timeout detection
    - MEDIUM: hlua: introduce tune.lua.burst-timeout
    - CLEANUP: hlua: avoid confusion between internal timers and tick based timers
    - MINOR: hlua: hook yield on known lua state
    - MINOR: hlua: safe coroutine.create()
    - BUG/MINOR: quic: Stop removing ACK ranges when building packets
    - MINOR: quic: Do not allocate too much ack ranges
    - BUG/MINOR: quic: Unchecked buffer length when building the token
    - BUG/MINOR: quic: Wrong Retry token generation timestamp computing
    - BUG/MINOR: mux-quic: fix crash with app ops install failure
    - BUG/MINOR: mux-quic: properly handle STREAM frame alloc failure
    - BUG/MINOR: h3: fix crash on h3s alloc failure
    - BUG/MINOR: quic: prevent crash on qc_new_conn() failure
    - BUG/MINOR: quic: consume Rx datagram even on error
    - CLEANUP: errors: fix obsolete function comments
    - CLEANUP: server: fix update_status() function comment
    - MINOR: server/event_hdl: add proxy_uuid to event_hdl_cb_data_server
    - MINOR: hlua/event_hdl: rely on proxy_uuid instead of proxy_name for lookups
    - MINOR: hlua/event_hdl: expose proxy_uuid variable in server events
    - MINOR: hlua/event_hdl: fix return type for hlua_event_hdl_cb_data_push_args
    - MINOR: server/event_hdl: prepare for upcoming refactors
    - BUG/MINOR: event_hdl: don't waste 1 event subtype slot
    - CLEANUP: event_hdl: updating obsolete comment for EVENT_HDL_CB_DATA
    - CLEANUP: event_hdl: fix comment typo about _sync assertion
    - MINOR: event_hdl: dynamically allocated event data members
    - MINOR: event_hdl: provide event->when for advanced handlers
    - MINOR: hlua/event_hdl: timestamp for events
    - DOC: lua: restore 80 char limitation
    - BUG/MINOR: server: incorrect report for tracking servers leaving drain
    - MINOR: server: explicitly commit state change in srv_update_status()
    - BUG/MINOR: server: don't miss proxy stats update on server state transitions
    - BUG/MINOR: server: don't miss server stats update on server state transitions
    - BUG/MINOR: server: don't use date when restoring last_change from state file
    - MINOR: server: central update for server counters on state change
    - MINOR: server: propagate server state change to lb through single function
    - MINOR: server: propagate lb changes through srv_lb_propagate()
    - MINOR: server: change adm_st_chg_cause storage type
    - MINOR: server: srv_append_status refacto
    - MINOR: server: change srv_op_st_chg_cause storage type
    - CLEANUP: server: remove unused variables in srv_update_status()
    - CLEANUP: server: fix srv_set_{running, stopping, stopped} function comment
    - MINOR: server: pass adm and op cause to srv_update_status()
    - MEDIUM: server: split srv_update_status() in two functions
    - MINOR: server/event_hdl: prepare for server event data wrapper
    - MINOR: quic: support migrating the listener as well
    - MINOR: quic_sock: index li->per_thr[] on local thread id, not global one
    - MINOR: listener: support another thread dispatch mode: "fair"
    - MINOR: receiver: add a struct shard_info to store info about each shard
    - MINOR: receiver: add RX_F_MUST_DUP to indicate that an rx must be duped
    - MEDIUM: proto: duplicate receivers marked RX_F_MUST_DUP
    - MINOR: proto: skip socket setup for duped FDs
    - MEDIUM: config: permit to start a bind on multiple groups at once
    - MINOR: listener: make accept_queue index atomic
    - MEDIUM: listener: rework thread assignment to consider all groups
    - MINOR: listener: use a common thr_idx from the reference listener
    - MINOR: listener: resync with the thread index before heavy calculations
    - MINOR: listener: make sure to avoid ABA updates in per-thread index
    - MINOR: listener: always compare the local thread as well
    - MINOR: Make `tasklet_free()` safe to be called with `NULL`
    - CLEANUP: Stop checking the pointer before calling `tasklet_free()`
    - CLEANUP: Stop checking the pointer before calling `pool_free()`
    - CLEANUP: Stop checking the pointer before calling `task_free()`
    - CLEANUP: Stop checking the pointer before calling `ring_free()`
    - BUG/MINOR: cli: clarify error message about stats bind-process
    - CI: cirrus-ci: bump FreeBSD image to 13-1
    - REGTESTS: remove unsupported "stats bind-process" keyword
    - CI: extend spellchecker whitelist, add "clen" as well
    - CLEANUP: assorted typo fixes in the code and comments
    - BUG/MINOR: sock_inet: use SO_REUSEPORT_LB where available
    - BUG/MINOR: tools: check libssl and libcrypto separately
    - BUG/MINOR: config: fix NUMA topology detection on FreeBSD
    - BUILD: sock_inet: forward-declare struct receiver
    - BUILD: proto_tcp: export the correct names for proto_tcpv[46]
    - CLEANUP: protocol: move the l3_addrlen to plug a hole in proto_fam
    - CLEANUP: protocol: move the nb_receivers to plug a hole in protocol
    - REORG: listener: move the bind_conf's thread setup code to listener.c
    - MINOR: proxy: make proxy_type_str() recognize peers sections
    - MEDIUM: peers: call bind_complete_thread_setup() to finish the config
    - MINOR: protocol: add a flags field to store info about protocols
    - MINOR: protocol: move the global reuseport flag to the protocols
    - MINOR: listener: automatically adjust shards based on support for SO_REUSEPORT
    - MINOR: protocol: add a function to check if some features are supported
    - MINOR: sock: add a function to check for SO_REUSEPORT support at runtime
    - MINOR: protocol: perform a live check for SO_REUSEPORT support
    - MINOR: listener: do not restrict CLI to first group anymore
    - MINOR: listener: add a new global tune.listener.default-shards setting
    - MEDIUM: listener: switch the default sharding to by-group
2023-04-23 10:21:37 +02:00
Willy Tarreau
0e875cf291 MEDIUM: listener: switch the default sharding to by-group
Sharding by-group is exactly identical to by-process for a single
group, and will use the same number of file descriptors for more than
one group, while significantly lowering the kernel's locking overhead.

Now that all special listeners (cli, peers) are properly handled, and
that support for SO_REUSEPORT is detected at runtime per protocol, there
should be no more reason for now switching to by-group by default.

That's what this patch does. It does only this and nothing else so that
it's easy to revert, should any issue be raised.

Testing on an AMD EPYC 74F3 featuring 24 cores and 48 threads distributed
into 8 core complexes of 3 cores each, shows that configuring 8 groups
(one per CCX) is sufficient to simply double the forwarded connection
rate from 112k to 214k/s, reducing kernel locking from 71 to 55%.
2023-04-23 10:18:16 +02:00
Willy Tarreau
7310164b2c MINOR: listener: add a new global tune.listener.default-shards setting
This new setting accepts "by-process", "by-group" and "by-thread" and
will dictate how listeners will be sharded by default when nothing is
specified. While the default remains "by-process", "by-group" should be
much more efficient with many threads, while not changing anything for
single-group setups.
2023-04-23 09:46:15 +02:00
Willy Tarreau
c1fbdd6397 MINOR: listener: automatically adjust shards based on support for SO_REUSEPORT
Now if multiple shards are explicitly requested, and the listener's
protocol doesn't support SO_REUSEPORT, sharding is disabled, which will
result in the socket being automatically duped if needed. A warning is
emitted when this happens. If "shards by-group" or "shards by-thread"
are used, these will automatically be turned down to 1 since we want
this to be possible easily using -dR on the command line without having
to djust the config. For "by-thread", a diag warning will be emitted to
help troubleshoot possible performance issues.
2023-04-23 09:46:15 +02:00
Ilya Shipitsin
ccf8012f28 CLEANUP: assorted typo fixes in the code and comments
This is 36th iteration of typo fixes
2023-04-23 09:44:53 +02:00
Willy Tarreau
09b52d1c3d MEDIUM: config: permit to start a bind on multiple groups at once
Now it's possible for a bind line to span multiple thread groups. When
this happens, the first one will become the reference and will be entirely
set up, and the subsequent ones will be duplicated from this reference,
so that they can be registered in distinct groups. The reference is
always setup and started first so it is always available when the other
ones are started.

The doc was updated to reflect this new possibility with its limitations
and impacts, and the differences with the "shards" option.
2023-04-21 17:41:26 +02:00
Willy Tarreau
84fe1f479b MINOR: listener: support another thread dispatch mode: "fair"
This new algorithm for rebalancing incoming connections to multiple
threads is simpler and instead of considering the threads load, it will
only cycle through all of them, offering a fair share of the traffic to
each thread. It may be well suited for short-lived connections but is
also convenient for very large thread counts where it's not always certain
that the least loaded thread will always be found.
2023-04-21 17:41:26 +02:00
Aurelien DARRAGON
2dac67af7d DOC: lua: restore 80 char limitation
Restore 80 char limitation throughout the file for easier reading
on the cli, and fix some raw formatting issues without altering
html rendering.
2023-04-21 14:36:45 +02:00
Aurelien DARRAGON
096b383e16 MINOR: hlua/event_hdl: timestamp for events
'when' optional argument is provided to lua event handlers.

It is an integer representing the number of seconds elapsed since Epoch
and may be used in conjunction with lua `os.date()` function to provide
a custom format string.
2023-04-21 14:36:45 +02:00
Aurelien DARRAGON
55f84c7cab MINOR: hlua/event_hdl: expose proxy_uuid variable in server events
Adding proxy_uuid to ServerEvent class.
proxy_uuid contains the uuid of the proxy to which the server belongs
2023-04-21 14:36:45 +02:00
Aurelien DARRAGON
58e36e5b14 MEDIUM: hlua: introduce tune.lua.burst-timeout
The "burst" execution timeout applies to any Lua handler.
If the handler fails to finish or yield before timeout is reached,
handler will be aborted to prevent thread contention, to prevent
traffic from not being served for too long, and ultimately to prevent
the process from crashing because of the watchdog kicking in.

Default value is 1000ms.
Combined with forced-yield default value of 10000 lua instructions, it
should be high enough to prevent any existing script breakage, while
still being able to catch slow lua converters or sample fetches
doing thread contention and risking the process stability.

Setting value to 0 completely bypasses this check. (not recommended but
could be required to restore original behavior if this feature breaks
existing setups somehow...)

No backport needed, although it could be used to prevent watchdog crashes
due to poorly coded (slow/cpu consuming) lua sample fetches/converters.
2023-04-19 11:03:31 +02:00
Willy Tarreau
5003ac7fe9 MEDIUM: config: set useful ALPN defaults for HTTPS and QUIC
This commit makes sure that if three is no "alpn", "npn" nor "no-alpn"
setting on a "bind" line which corresponds to an HTTPS or QUIC frontend,
we automatically turn on "h2,http/1.1" as an ALPN default for an HTTP
listener, and "h3" for a QUIC listener. This simplifies the configuration
for end users since they won't have to explicitly configure the ALPN
string to enable H2, considering that at the time of writing, HTTP/1.1
represents less than 7% of the traffic on large infrastructures. The
doc and regtests were updated. For more info, refer to the following
thread:

  https://www.mail-archive.com/haproxy@formilux.org/msg43410.html
2023-04-19 09:52:20 +02:00
Willy Tarreau
74d7cc0891 DOC: add missing documentation for "no-alpn" on bind lines
This is the doc for the no-alpn keyword that was mistakenly left out of
commit 158c18e85 ("MINOR: config: add "no-alpn" support for bind lines").
2023-04-19 09:12:43 +02:00
Willy Tarreau
ca1027c22f MINOR: mux-h2: make the max number of concurrent streams configurable per side
For a long time the maximum number of concurrent streams was set once for
both sides (front and back) while the impacts are different. This commit
allows it to be configured separately for each side. The older settings
remains the fallback choice when other ones are not set.
2023-04-18 15:58:55 +02:00
Willy Tarreau
9d7abda787 MINOR: mux-h2: make the initial window size configurable per side
For a long time the initial window size (per-stream size) was set once
for both directions, frontend and backend, resulting in a tradeoff between
upload speed and download fairness. This commit allows it to be configured
separately for each side. The older settings remains the fallback choice
when other ones are not set.
2023-04-18 15:58:55 +02:00
Willy Tarreau
a07635ead5 MINOR: bind-conf: support a new shards value: "by-group"
Setting "shards by-group" will create one shard per thread group. This
can often be a reasonable tradeoff between a single one that can be
suboptimal on CPUs with many cores, and too many that will eat a lot
of file descriptors. It was shown to provide good results on a 224
thread machine, with a distribution that was even smoother than the
system's since here it can take into account the number of connections
per thread in the group. Depending on how popular it becomes, it could
even become the default setting in a future version.
2023-04-13 17:38:31 +02:00
Willy Tarreau
768b62857e [RELEASE] Released version 2.8-dev7
Released version 2.8-dev7 with the following main changes :
    - BUG/MINOR: stats: Don't replace sc_shutr() by SE_FL_EOS flag yet
    - BUG/MEDIUM: mux-h2: Be able to detect connection error during handshake
    - BUG/MINOR: quic: Missing padding in very short probe packets
    - MINOR: proxy/pool: prevent unnecessary calls to pool_gc()
    - CLEANUP: proxy: remove stop_time related dead code
    - DOC/MINOR: reformat configuration.txt's "quoting and escaping" table
    - MINOR: http_fetch: Add support for empty delim in url_param
    - MINOR: http_fetch: add case insensitive support for smp_fetch_url_param
    - MINOR: http_fetch: Add case-insensitive argument for url_param/urlp_val
    - REGTESTS : Add test support for case insentitive for url_param
    - BUG/MEDIUM: proxy/sktable: prevent watchdog trigger on soft-stop
    - BUG/MINOR: backend: make be_usable_srv() consistent when stopping
    - BUG/MINOR: ssl: Remove dead code in cli_parse_update_ocsp_response
    - BUG/MINOR: ssl: Fix potential leak in cli_parse_update_ocsp_response
    - BUG/MINOR: ssl: ssl-(min|max)-ver parameter not duplicated for bundles in crt-list
    - BUG/MINOR: quic: Wrong use of now_ms timestamps (cubic algo)
    - MINOR: quic: Add recovery related information to "show quic"
    - BUG/MINOR: quic: Wrong use of now_ms timestamps (newreno algo)
    - BUG/MINOR: quic: Missing max_idle_timeout initialization for the connection
    - MINOR: quic: Implement cubic state trace callback
    - MINOR: quic: Adjustments for generic control congestion traces
    - MINOR: quic: Traces adjustments at proto level.
    - MEDIUM: quic: Ack delay implementation
    - BUG/MINOR: quic: Wrong rtt variance computing
    - MINOR: cli: support filtering on FD types in "show fd"
    - MINOR: quic: Add a fake congestion control algorithm named "nocc"
    - CI: run smoke tests on config syntax to check memory related issues
    - CLEANUP: assorted typo fixes in the code and comments
    - CI: exclude doc/{design-thoughts,internals} from spell check
    - BUG/MINOR: quic: Remaining useless statements in cubic slow start callback
    - BUG/MINOR: quic: Cubic congestion control window may wrap
    - MINOR: quic: Add missing traces in cubic algorithm implementation
    - BUG/MAJOR: quic: Congestion algorithms states shared between the connection
    - BUG/MINOR: ssl: Undefined reference when building with OPENSSL_NO_DEPRECATED
    - BUG/MINOR: quic: Remove useless BUG_ON() in newreno and cubic algo implementation
    - MINOR: http-act: emit a warning when a header field name contains forbidden chars
    - DOC: config: strict-sni allows to start without certificate
    - MINOR: quic: Add trace to debug idle timer task issues
    - BUG/MINOR: quic: Unexpected connection closures upon idle timer task execution
    - BUG/MINOR: quic: Wrong idle timer expiration (during 20s)
    - BUILD: quic: 32bits compilation issue in cli_io_handler_dump_quic()
    - BUG/MINOR: quic: Possible wrong PTO computing
    - BUG/MINOR: tcpcheck: Be able to expect an empty response
    - BUG/MEDIUM: stconn: Add a missing return statement in sc_app_shutr()
    - BUG/MINOR: stream: Fix test on channels flags to set clientfin/serverfin touts
    - MINOR: applet: Uninline appctx_free()
    - MEDIUM: applet/trace: Register a new trace source with its events
    - CLEANUP: stconn: Remove remaining debug messages
    - BUG/MEDIUM: channel: Improve reports for shut in co_getblk()
    - BUG/MEDIUM: dns: Properly handle error when a response consumed
    - MINOR: stconn: Remove unecessary test on SE_FL_EOS before receiving data
    - MINOR: stconn/channel: Move CF_READ_DONTWAIT into the SC and rename it
    - MINOR: stconn/channel: Move CF_SEND_DONTWAIT into the SC and rename it
    - MINOR: stconn/channel: Move CF_NEVER_WAIT into the SC and rename it
    - MINOR: stconn/channel: Move CF_EXPECT_MORE into the SC and rename it
    - MINOR: mux-pt: Report end-of-input with the end-of-stream after a read
    - BUG/MINOR: mux-h1: Properly report EOI/ERROR on read0 in h1_rcv_pipe()
    - CLEANUP: mux-h1/mux-pt: Remove useless test on SE_FL_SHR/SE_FL_SHW flags
    - MINOR: mux-h1: Report an error to the SE descriptor on truncated message
    - MINOR: stconn: Always ack EOS at the end of sc_conn_recv()
    - MINOR: stconn/applet: Handle EOI in the applet .wake callback function
    - MINOR: applet: No longer set EOI on the SC
    - MINOR: stconn/applet: Handle EOS in the applet .wake callback function
    - MEDIUM: cache: Use the sedesc to report and detect end of processing
    - MEDIUM: cli: Use the sedesc to report and detect end of processing
    - MINOR: dns: Remove the test on the opposite SC state to send requests
    - MEDIUM: dns: Use the sedesc to report and detect end of processing
    - MEDIUM: spoe: Use the sedesc to report and detect end of processing
    - MEDIUM: hlua/applet: Use the sedesc to report and detect end of processing
    - MEDIUM: log: Use the sedesc to report and detect end of processing
    - MEDIUM: peers: Use the sedesc to report and detect end of processing
    - MINOR: sink: Remove the tests on the opposite SC state to process messages
    - MEDIUM: sink: Use the sedesc to report and detect end of processing
    - MEDIUM: stats: Use the sedesc to report and detect end of processing
    - MEDIUM: promex: Use the sedesc to report and detect end of processing
    - MEDIUM: http_client: Use the sedesc to report and detect end of processing
    - MINOR: stconn/channel: Move CF_EOI into the SC and rename it
    - MEDIUM: tree-wide: Move flags about shut from the channel to the SC
    - MINOR: tree-wide: Simplifiy some tests on SHUT flags by accessing SCs directly
    - MINOR: stconn/applet: Add BUG_ON_HOT() to be sure SE_FL_EOS is never set alone
    - MINOR: server: add SRV_F_DELETED flag
    - BUG/MINOR: server/del: fix srv->next pointer consistency
    - BUG/MINOR: stats: properly handle server stats dumping resumption
    - BUG/MINOR: sink: free forward_px on deinit()
    - BUG/MINOR: log: free log forward proxies on deinit()
    - MINOR: server: always call ssl->destroy_srv when available
    - MINOR: server: correctly free servers on deinit()
    - BUG/MINOR: hlua: hook yield does not behave as expected
    - MINOR: hlua: properly handle hlua_process_task HLUA_E_ETMOUT
    - BUG/MINOR: hlua: enforce proper running context for register_x functions
    - MINOR: hlua: Fix two functions that return nothing useful
    - MEDIUM: hlua: Dynamic list of frontend/backend in Lua
    - MINOR: hlua_fcn: alternative to old proxy and server attributes
    - MEDIUM: hlua_fcn: dynamic server iteration and indexing
    - MEDIUM: hlua_fcn/api: remove some old server and proxy attributes
    - CLEANUP: hlua: fix conflicting comment in hlua_ctx_destroy()
    - MINOR: hlua: add simple hlua reference handling API
    - MINOR: hlua: fix return type for hlua_checkfunction() and hlua_checktable()
    - BUG/MINOR: hlua: fix reference leak in core.register_task()
    - BUG/MINOR: hlua: fix reference leak in hlua_post_init_state()
    - BUG/MINOR: hlua: prevent function and table reference leaks on errors
    - CLEANUP: hlua: use hlua_ref() instead of luaL_ref()
    - CLEANUP: hlua: use hlua_pushref() instead of lua_rawgeti()
    - CLEANUP: hlua: use hlua_unref() instead of luaL_unref()
    - MINOR: hlua: simplify lua locking
    - BUG/MEDIUM: hlua: prevent deadlocks with main lua lock
    - MINOR: hlua_fcn: add server->get_rid() method
    - MINOR: hlua: support for optional arguments to core.register_task()
    - DOC: lua: silence "literal block ends without a blank line" Sphinx warnings
    - DOC: lua: silence "Unexpected indentation" Sphinx warnings
    - BUG/MINOR: event_hdl: fix rid storage type
    - BUG/MINOR: event_hdl: make event_hdl_subscribe thread-safe
    - MINOR: event_hdl: global sublist management clarification
    - BUG/MEDIUM: event_hdl: clean soft-stop handling
    - BUG/MEDIUM: event_hdl: fix async data refcount issue
    - MINOR: event_hdl: normal tasks support for advanced async mode
    - MINOR: event_hdl: add event_hdl_async_equeue_isempty() function
    - MINOR: event_hdl: add event_hdl_async_equeue_size() function
    - MINOR: event_hdl: pause/resume for subscriptions
    - MINOR: proxy: add findserver_unique_id() and findserver_unique_name()
    - MEDIUM: hlua/event_hdl: initial support for event handlers
    - MINOR: hlua/event_hdl: per-server event subscription
    - EXAMPLES: add basic event_hdl lua example script
    - MINOR: http-ana: Add a HTTP_MSGF flag to state the Expect header was checked
    - BUG/MINOR: http-ana: Don't switch message to DATA when waiting for payload
    - BUG/MINOR: quic: Possible crashes in qc_idle_timer_task()
    - MINOR: quic: derive first DCID from client ODCID
    - MINOR: quic: remove ODCID dedicated tree
    - MINOR: quic: remove address concatenation to ODCID
    - BUG/MINOR: mworker: unset more internal variables from program section
    - BUG/MINOR: errors: invalid use of memprintf in startup_logs_init()
    - MINOR: applet: Use unsafe version to get stream from SC in the trace function
    - BUG/MUNOR: http-ana: Use an unsigned integer for http_msg flags
    - MINOR: compression: Make compression offload a flag
    - MINOR: compression: Prepare compression code for request compression
    - MINOR: compression: Store algo and type for both request and response
    - MINOR: compression: Count separately request and response compression
    - MEDIUM: compression: Make it so we can compress requests as well.
    - BUG/MINOR: lua: remove incorrect usage of strncat()
    - CLEANUP: tcpcheck: remove the only occurrence of sprintf() in the code
    - CLEANUP: ocsp: do no use strpcy() to copy a path!
    - CLEANUP: tree-wide: remove strpcy() from constant strings
    - CLEANUP: opentracing: remove the last two occurrences of strncat()
    - BUILD: compiler: fix __equals_1() on older compilers
    - MINOR: compiler: define a __attribute__warning() macro
    - BUILD: bug.h: add a warning in the base API when unsafe functions are used
    - BUG/MEDIUM: listeners: Use the right parameters for strlcpy2().
2023-04-08 17:38:39 +02:00
Olivier Houchard
ead43fe4f2 MEDIUM: compression: Make it so we can compress requests as well.
Add code so that compression can be used for requests as well.
New compression keywords are introduced :
"direction" that specifies what we want to compress. Valid values are
"request", "response", or "both".
"type-req" and "type-res" define content-type to be compressed for
requests and responses, respectively. "type" is kept as an alias for
"type-res" for backward compatibilty.
"algo-req" specifies the compression algorithm to be used for requests.
Only one algorithm can be provided.
"algo-res" provides the list of algorithm that can be used to compress
responses. "algo" is kept as an alias for "algo-res" for backward
compatibility.
2023-04-07 00:49:17 +02:00
Aurelien DARRAGON
223770ddca MINOR: hlua/event_hdl: per-server event subscription
Now that event_hdl api is properly implemented in hlua, we may add the
per-server event subscription in addition to the global event
subscription.

Per-server subscription allows to be notified for events related to
single server. It is useful to track a server UP/DOWN and DEL events.

It works exactly like core.event_sub() except that the subscription
will be performed within the server dedicated subscription list instead
of the global one.
The callback function will only be called for server events affecting
the server from which the subscription was performed.

Regarding the implementation, it is pretty trivial at this point, we add
more doc than code this time.

Usage examples have been added to the (lua) documentation.
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
c84899c636 MEDIUM: hlua/event_hdl: initial support for event handlers
Now that the event handler API is pretty mature, we can expose it in
the lua API.

Introducing the core.event_sub(<event_types>, <cb>) lua function that
takes an array of event types <event_types> as well as a callback
function <cb> as argument.

The function returns a subscription <sub> on success.
Subscription <sub> allows you to manage the subscription from anywhere
in the script.
To this day only the sub->unsub method is implemented.

The following event types are currently supported:
  - "SERVER_ADD": when a server is added
  - "SERVER_DEL": when a server is removed from haproxy
  - "SERVER_DOWN": server states goes from up to down
  - "SERVER_UP": server states goes from down to up

As for the <cb> function: it will be called when one of the registered
event types occur. The function will be called with 3 arguments:
  cb(<event>,<data>,<sub>)

<event>: event type (string) that triggered the function.
(could be any of the types used in <event_types> when registering
the subscription)

<data>: data associated with the event (specific to each event family).

For "SERVER_" family events, server details such as server name/id/proxy
will be provided.
If the server still exists (not yet deleted), a reference to the live
server is provided to spare you from an additionnal lookup if you need
to have direct access to the server from lua.

<sub> refers to the subscription. In case you need to manage it from
within an event handler.
(It refers to the same subscription that the one returned from
core.event_sub())

Subscriptions are per-thread: the thread that will be handling the
event is the one who performed the subscription using
core.event_sub() function.

Each thread treats events sequentially, it means that if you have,
let's say SERVER_UP, then SERVER_DOWN in a short timelapse, then your
cb function will first be called with SERVER_UP, and once you're done
handling the event, your function will be called again with SERVER_DOWN.

This is to ensure event consitency when it comes to logging / triggering
logic from lua.

Your lua cb function may yield if needed, but you're pleased to process
the event as fast as possible to prevent the event queue from growing up

To prevent abuses, if the event queue for the current subscription goes
over 100 unconsumed events, the subscription will pause itself
automatically for as long as it takes for your handler to catch up.
This would lead to events being missed, so a warning will be emitted in
the logs to inform you about that. This is not something you want to let
happen too often, it may indicate that you subscribed to an event that
is occurring too frequently or/and that your callback function is too
slow to keep up the pace and you should review it.

If you want to do some parallel processing because your callback
functions are slow: you might want to create subtasks from lua using
core.register_task() from within your callback function to perform the
heavy job in a dedicated task and allow remaining events to be processed
more quickly.

Please check the lua documentation for more information.
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
b289fd1420 MINOR: event_hdl: normal tasks support for advanced async mode
advanced async mode (EVENT_HDL_ASYNC_TASK) provided full support for
custom tasklets registration.

Due to the similarities between tasks and tasklets, it may be useful
to use the advanced mode with an existing task (not a tasklet).
While the API did not explicitly disallow this usage, things would
get bad if we try to wakeup a task using tasklet_wakeup() for notifying
the task about new events.

To make the API support both custom tasks and tasklets, we use the
TASK_IS_TASKLET() macro to call the proper waking function depending
on the task's type:

  - For tasklets: we use tasklet_wakeup()
  - For tasks: we use task_wakeup()

If 68e692da0 ("MINOR: event_hdl: add event handler base api")
is being backported, then this commit should be backported with it.
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
21f7ebbfcd DOC: lua: silence "Unexpected indentation" Sphinx warnings
When building html documentation from doc/lua-api/index.rst, sphinx
complains about some unexpected indentations:

  "doc/lua-api/index.rst:3221: WARNING: Unexpected indentation"

Silencing them without altering the original output format.
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
d5c80d7ab1 DOC: lua: silence "literal block ends without a blank line" Sphinx warnings
When building html documentation from doc/lua-api/index.rst, sphinx
complains about some literal blocks ending without a blank line:

  "doc/lua-api/index.rst:534: WARNING: Literal block ends without a blank line; unexpected unindent."

Adding the missing blank lines to make sphinx happy
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
b8038996e9 MINOR: hlua: support for optional arguments to core.register_task()
core.register_task(function) may now take up to 4 additional arguments
that will be passed as-is to the task function.
This could be convenient to spawn sub-tasks from existing functions
supporting core.register_task() without the need to use global
variables to pass some context to the newly created task function.

The new prototype is:

  core.register_task(function[, arg1[, arg2[, ...[, arg4]]]])

Implementation remains backward-compatible with existing scripts.
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
94ee6632ee MINOR: hlua_fcn: add server->get_rid() method
Server revision ID was recently added to haproxy with 61e3894
("MINOR: server: add srv->rid (revision id) value")

Let's add it to the hlua server class.
2023-04-05 08:58:17 +02:00
Aurelien DARRAGON
c4b2437037 MEDIUM: hlua_fcn/api: remove some old server and proxy attributes
Since ("MINOR: hlua_fcn: alternative to old proxy and server attributes"):
 - s->name(), s->puid() are superseded by s->get_name() and s->get_puid()
 - px->name(), px->uuid() are superseded by px->get_name() and
   px->get_uuid()

And considering this is now the proper way to retrieve proxy name/uuid
and server name/puid from lua:

We're now removing such legacy attributes, but for retro-compatibility
purposes we will be emulating them and warning the user for some time
before completely dropping their support.

To do this, we first remove old legacy code.
Then we move server and proxy methods out of the metatable to allow
direct elements access without systematically involving the "__index"
metamethod.

This allows us to involve the "__index" metamethod only when the requested
key is missing from the table.

Then we define relevant hlua_proxy_index and hlua_server_index functions
that will be used as the "__index" metamethod to respectively handle
"name, uuid" (proxy) or "name, puid" (server) keys, in which case we
warn the user about the need to use the new getter function instead the
legacy attribute (to prepare for the potential upcoming removal), and we
call the getter function to return the value as if the getter function
was directly called from the script.

Note: Using the legacy variables instead of the getter functions results
in a slight overhead due to the "__index" metamethod indirection, thus
it is recommended to switch to the getter functions right away.

With this commit we're also adding a deprecation notice about legacy
attributes.
2023-04-05 08:58:16 +02:00
Thierry Fournier
b0467730a0 MINOR: hlua_fcn: alternative to old proxy and server attributes
This patch adds new lua methods:
- "Proxy.get_uuid()"
- "Proxy.get_name()"
- "Server.get_puid()"
- "Server.get_name()"

These methods will be equivalent to their old analog Proxy.{uuid,name}
and Server.{puid,name} attributes, but this will be the new preferred
way to fetch such infos as it duplicates memory only when necessary and
thus reduce the overall lua Server/Proxy objects memory footprint.

Legacy attributes (now superseded by the explicit getters) are expected
to be removed some day.

Co-authored-by: Aurelien DARRAGON <adarragon@haproxy.com>
2023-04-05 08:58:16 +02:00
William Lallemand
5c099351d1 DOC: config: strict-sni allows to start without certificate
The strict-sni keyword allows to start without certificate on a bind
line.

Must be backported as far as 2.2.
2023-04-04 16:28:58 +02:00
Ilya Shipitsin
07be66d21b CLEANUP: assorted typo fixes in the code and comments
This is 35th iteration of typo fixes
2023-04-01 18:33:40 +02:00
Willy Tarreau
1cb041a6ee MINOR: cli: support filtering on FD types in "show fd"
Depending on what we're debugging, some FDs can represent pollution in
the "show fd" output. Here we add a set of filters allowing to pick (or
exclude) any combination of listener, frontend conn, backend conn, pipes,
etc. "show fd l" will only list listening connections for example.
2023-03-31 16:35:53 +02:00
Martin DOLEZ
28c5f40ad6 MINOR: http_fetch: Add case-insensitive argument for url_param/urlp_val
This commit adds a new optional argument to smp_fetch_url_param
and smp_fetch_url_param_val that makes the parameter key comparison
case-insensitive.
Now users can retrieve URL parameters regardless of their case,
allowing to match parameters in case insensitive application.
Doc was updated.
2023-03-30 14:11:25 +02:00
Marcos de Oliveira
3b7a351a97 DOC/MINOR: reformat configuration.txt's "quoting and escaping" table
The table in section 2.2 ("Quoting and escaping") was formated in a way
which is not recognized by haproxy-dconv, breaking it, and cutting off
the entire section.
This commit fix that by formatting the table in way which allows the
converter to produce the correct HTML.

Fixes cbonte/haproxy-dconv#35
2023-03-29 07:20:03 +02:00
Willy Tarreau
4c7588dd22 [RELEASE] Released version 2.8-dev6
Released version 2.8-dev6 with the following main changes :
    - BUG/MEDIUM: mux-pt: Set EOS on error on sending path if read0 was received
    - MINOR: ssl: Change the ocsp update log-format
    - MINOR: ssl: Use ocsp update task for "update ssl ocsp-response" command
    - BUG/MINOR: ssl: Fix double free in ocsp update deinit
    - MINOR: ssl: Accept certpath as param in "show ssl ocsp-response" CLI command
    - MINOR: ssl: Add certificate path to 'show ssl ocsp-response' output
    - BUG/MEDIUM: proxy: properly stop backends on soft-stop
    - BUG/MEDIUM: resolvers: Properly stop server resolutions on soft-stop
    - DEBUG: cli/show_fd: Display connection error code
    - DEBUG: ssl-sock/show_fd: Display SSL error code
    - BUG/MEDIUM: mux-h1: Don't block SE_FL_ERROR if EOS is not reported on H1C
    - BUG/MINOR: tcp_sample: fix a bug in fc_dst_port and fc_dst_is_local sample fetches
    - BUG/MINOR: quic: Missing STREAM frame length updates
    - BUG/MEDIUM: connection: Preserve flags when a conn is removed from an idle list
    - BUG/MINOR: mux-h2: make sure the h2c task exists before refreshing it
    - MINOR: buffer: add br_count() to return the number of allocated bufs
    - MINOR: buffer: add br_single() to check if a buffer ring has more than one buf
    - BUG/MEDIUM: mux-h2: only restart sending when mux buffer is decongested
    - BUG/MINOR: mux-h2: set CO_SFL_STREAMER when sending lots of data
    - BUG/MINOR: quic: Missing STREAM frame data pointer updates
    - MINOR: stick-table: add sc-add-gpc() to http-after-response
    - MINOR: doc: missing entries for sc-add-gpc()
    - BUG/MAJOR: qpack: fix possible read out of bounds in static table
    - OPTIM: mux-h1: limit first read size to avoid wrapping
    - MINOR: mux-h2: set CO_SFL_MSG_MORE when sending multiple buffers
    - MINOR: ssl-sock: pass the CO_SFL_MSG_MORE info down the stack
    - MINOR: quic: Stop stressing the acknowledgments process (RX ACK frames)
    - BUG/MINOR: quic: Dysfunctional 01RTT packet number space probing
    - BUG/MEDIUM: stream: do not try to free a failed stream-conn
    - BUG/MEDIUM: mux-h2: do not try to free an unallocated h2s->sd
    - BUG/MEDIUM: mux-h2: erase h2c->wait_event.tasklet on error path
    - BUG/MEDIUM: stconn: don't set the type before allocation succeeds
    - BUG/MINOR: stconn: fix sedesc memory leak on stream allocation failure
    - MINOR: dynbuf: set POOL_F_NO_FAIL on buffer allocation
    - MINOR: pools: preset the allocation failure rate to 1% with -dMfail
    - BUG/MEDIUM: mux-h1: properly destroy a partially allocated h1s
    - BUG/MEDIUM: applet: only set appctx->sedesc on successful allocation
    - BUG/MINOR: quic: wake up MUX on probing only for 01RTT
    - BUG/MINOR: quic: ignore congestion window on probing for MUX wakeup
    - BUILD: thread: implement thread_harmless_end_sig() for threadless builds
    - BUILD: thread: silence a build warning when threads are disabled
    - MINOR: debug: support dumping the libs addresses when running in verbose mode
    - BUG/MINOR: illegal use of the malloc_trim() function if jemalloc is used
    - BUG/MINOR: trace: fix hardcoded level for TRACE_PRINTF
    - BUG/MEDIUM: mux-quic: release data from conn flow-control on qcs reset
    - MINOR: mux-quic: complete traces for qcs emission
    - MINOR: mux-quic: adjust trace level for MAX_DATA/MAX_STREAM_DATA recv
    - MINOR: mux-quic: add flow-control info to minimal trace level
    - MINOR: pools: make sure 'no-memory-trimming' is always used
    - MINOR: pools: intercept malloc_trim() instead of trying to plug holes
    - MEDIUM: pools: move the compat code from trim_all_pools() to malloc_trim()
    - MINOR: pools: export trim_all_pools()
    - MINOR: pattern: use trim_all_pools() instead of a conditional malloc_trim()
    - MINOR: tools: relax dlopen() on malloc/free checks
    - MEDIUM: tools: further relax dlopen() checks too consider grouped symbols
    - BUG/MINOR: pools: restore detection of built-in allocator
    - MINOR: pools: report a replaced memory allocator instead of just malloc_trim()
    - BUG/MINOR: h3: properly handle incomplete remote uni stream type
    - BUG/MINOR: mux-quic: prevent CC status to be erased by shutdown
    - MINOR: mux-quic: interrupt qcc_recv*() operations if CC scheduled
    - MINOR: mux-quic: ensure CONNECTION_CLOSE is scheduled once per conn
    - MINOR: mux-quic: close on qcs allocation failure
    - MINOR: mux-quic: close on frame alloc failure
    - BUG/MINOR: syslog: Request for more data if message was not fully received
    - BUG/MEDIUM: stats: Consume the request except when parsing the POST payload
    - DOC: config: set-var() dconv rendering issues
    - BUG/MEDIUM: mux-h1: Wakeup H1C on shutw if there is no I/O subscription
    - BUG/MINOR: applet/new: fix sedesc freeing logic
    - BUG/MINOR: quic: Missing STREAM frame type updated
    - BUILD: da: extends CFLAGS to support API v3 from 3.1.7 and onwards.
    - BUG/MINOR: ssl: Stop leaking `err` in ssl_sock_load_ocsp()
2023-03-28 13:58:56 +02:00
Aurelien DARRAGON
fedbc17a5e DOC: config: set-var() dconv rendering issues
Since <cond> optional argument support was added to set-var() and friends
in 2.6 with 164726c ("DOC: vars: Add documentation about the set-var
conditions"), dconv is having a hard time rendering related keywords.

Everywhere `[,<cond> ...]` was inserted, html formatting is now broken.

Removing the space between <cond> and '...' allows dconv to properly parse
the token thus restores proper formatting without changing the meaning.

This was discovered when discussing about var() sample fetch doc issues
in GH #2087

This patch should be backported up to 2.6
2023-03-24 09:45:28 +01:00
Willy Tarreau
9ef2742a51 MINOR: debug: support dumping the libs addresses when running in verbose mode
Starting haproxy with -dL helps enumerate the list of libraries in use.
But sometimes in order to go further we'd like to see their address
ranges. This is already supported on the CLI's "show libs" but not on
the command line where it can sometimes help troubleshoot startup issues.
Let's dump them when in verbose mode. This way it doesn't change the
existing behavior for those trying to enumerate libs to produce an archive.
2023-03-22 11:43:15 +01:00
Willy Tarreau
0c4348c982 MINOR: pools: preset the allocation failure rate to 1% with -dMfail
Using -dMfail alone does nothing unless tune.fail-alloc is set, which
renders it pretty useless as-is, and is not intuitive. Let's change
this so that the filure rate is preset to 1% when the option is set on
the command line. This allows to inject failures without having to edit
the configuration.
2023-03-21 09:26:55 +01:00
Aurelien DARRAGON
5b4e16ee2d MINOR: doc: missing entries for sc-add-gpc()
When sc-add-gpc() action was implemented in 5a72d03 ("MINOR:
stick-table: implement the sc-add-gpc() action"), its usage was
only documented for "http-request", but according to the code it
now applies everywhere sc-inc-gpc() is mentioned.

Adding the missing entries in the doc everywhere the action may
be used.

The issue was detected by the haproxy-controller bot and was reported
by Pratik Mohanty and Marko Juraga.

No backport needed, unless 5a72d03 ("MINOR: stick-table: implement the
sc-add-gpc() action") is being backported.
2023-03-17 13:09:09 +01:00
Remi Tricot-Le Breton
7716f27736 MINOR: ssl: Add certificate path to 'show ssl ocsp-response' output
The ocsp-related CLI commands tend to work with OCSP_CERTIDs as well as
certificate paths so the path should also be added to the output of the
"show ssl ocsp-response" command when no certid or path is provided.
2023-03-14 11:07:32 +01:00
Remi Tricot-Le Breton
dafc068f12 MINOR: ssl: Accept certpath as param in "show ssl ocsp-response" CLI command
In order to increase usability, the "show ssl ocsp-response" also takes
a frontend certificate path as parameter. In such a case, it behaves the
same way as "show ssl cert foo.pem.ocsp".
2023-03-14 11:07:32 +01:00
Remi Tricot-Le Breton
c9bfe32b71 MINOR: ssl: Change the ocsp update log-format
The dedicated proxy used for OCSP auto update is renamed OCSP-UPDATE
which should be more explicit than the previous HC_OCSP name. The
reference to the underlying httpclient is simply kept in the
documentation.
The certid is removed from the log line since it is not really
comprehensible and is replaced by the path to the corresponding frontend
certificate.
2023-03-14 11:07:32 +01:00
Willy Tarreau
fc0ad29c29 [RELEASE] Released version 2.8-dev5
Released version 2.8-dev5 with the following main changes :
    - MINOR: ssl: rename confusing ssl_bind_kws
    - BUG/MINOR: config: crt-list keywords mistaken for bind ssl keywords
    - BUG/MEDIUM: http-ana: Detect closed SC on opposite side during body forwarding
    - BUG/MEDIUM: stconn: Don't rearm the read expiration date if EOI was reached
    - MINOR: global: Add an option to disable the data fast-forward
    - MINOR: haproxy: Add an command option to disable data fast-forward
    - REGTESTS: Remove unsupported feature command in http_splicing.vtc
    - BUG/MEDIUM: wdt: fix wrong thread being checked for sleeping
    - BUG/MINOR: sched: properly report long_rq when tasks remain in the queue
    - BUG/MEDIUM: sched: allow a bit more TASK_HEAVY to be processed when needed
    - MINOR: threads: add flags to know if a thread is started and/or running
    - MINOR: h3/hq-interop: handle no data in decode_qcs() with FIN set
    - BUG/MINOR: mux-quic: transfer FIN on empty STREAM frame
    - BUG/MINOR: mworker: prevent incorrect values in uptime
    - MINOR: h3: add traces on decode_qcs callback
    - BUG/MINOR: quic: Possible unexpected counter incrementation on send*() errors
    - MINOR: quic: Add new traces about by connection RX buffer handling
    - MINOR: quic: Move code to wakeup the timer task to avoid anti-amplication deadlock
    - BUG/MINOR: quic: Really cancel the connection timer from qc_set_timer()
    - MINOR: quic: Simplication for qc_set_timer()
    - MINOR: quic: Kill the connections on ICMP (port unreachable) packet receipt
    - MINOR: quic: Add traces to qc_kill_conn()
    - MINOR: quic: Make qc_dgrams_retransmit() return a status.
    - BUG/MINOR: quic: Missing call to task_queue() in qc_idle_timer_do_rearm()
    - MINOR: quic: Add a trace to identify connections which sent Initial packet.
    - MINOR: quic: Add <pto_count> to the traces
    - BUG/MINOR: quic: Do not probe with too little Initial packets
    - BUG/MINOR: quic: Wrong initialization for io_cb_wakeup boolean
    - BUG/MINOR: quic: Do not drop too small datagrams with Initial packets
    - BUG/MINOR: quic: Missing padding for short packets
    - MINOR: quic: adjust request reject when MUX is already freed
    - BUG/MINOR: quic: also send RESET_STREAM if MUX released
    - BUG/MINOR: quic: acknowledge STREAM frame even if MUX is released
    - BUG/MINOR: h3: prevent hypothetical demux failure on int overflow
    - MEDIUM: h3: enforce GOAWAY by resetting higher unhandled stream
    - MINOR: mux-quic: define qc_shutdown()
    - MINOR: mux-quic: define qc_process()
    - MINOR: mux-quic: implement client-fin timeout
    - MEDIUM: mux-quic: properly implement soft-stop
    - MINOR: quic: mark quic-conn as jobs on socket allocation
    - MEDIUM: quic: trigger fast connection closing on process stopping
    - MINOR: mux-h2/traces: do not log h2s pointer for dummy streams
    - MINOR: mux-h2/traces: add a missing TRACE_LEAVE() in h2s_frt_handle_headers()
    - BUG/MEDIUM: quic: Missing TX buffer draining from qc_send_ppkts()
    - DEBUG: stream: Add a BUG_ON to never exit process_stream with an expired task
    - DOC: config: Fix description of options about HTTP connection modes
    - MINOR: proxy: Only consider backend httpclose option for server connections
    - BUG/MINOR: haproxy: Fix option to disable the fast-forward
    - DOC: config: Add the missing tune.fail-alloc option from global listing
    - MINOR: cfgcond: Implement strstr condition expression
    - MINOR: cfgcond: Implement enabled condition expression
    - REGTESTS: Skip http_splicing.vtc script if fast-forward is disabled
    - REGTESTS: Fix ssl_errors.vtc script to wait for connections close
    - BUG/MINOR: mworker: stop doing strtok directly from the env
    - BUG/MEDIUM: mworker: prevent inconsistent reload when upgrading from old versions
    - BUG/MEDIUM: mworker: don't register mworker_accept_wrapper() when master FD is wrong
    - MINOR: startup: HAPROXY_STARTUP_VERSION contains the version used to start
    - BUG/MINOR: cache: Cache response even if request has "no-cache" directive
    - BUG/MINOR: cache: Check cache entry is complete in case of Vary
    - MINOR: compiler: add a TOSTR() macro to turn a value into a string
    - BUG/MINOR: lua/httpclient: missing free in hlua_httpclient_send()
    - BUG/MEDIUM: httpclient/lua: fix a race between lua GC and hlua_ctx_destroy
    - MEDIUM: channel: Remove CF_READ_NOEXP flag
    - MAJOR: channel: Remove flags to report READ or WRITE errors
    - DEBUG: stream/trace: Add sedesc flags in trace messages
    - MINOR: channel/stconn: Move rto/wto from the channel to the stconn
    - MEDIUM: channel/stconn: Move rex/wex timer from the channel to the sedesc
    - MEDIUM: stconn: Don't requeue the stream's task after I/O
    - MEDIUM: stconn: Replace read and write timeouts by a unique I/O timeout
    - MEDIUM: stconn: Add two date to track successful reads and blocked sends
    - MINOR: applet/stconn: Add a SE flag to specify an endpoint does not expect data
    - MAJOR: stream: Use SE descriptor date to detect read/write timeouts
    - MINOR: stream: Dump the task expiration date in trace messages
    - MINOR: stream: Report rex/wex value using the sedesc date in trace messages
    - MINOR: stream: Use relative expiration date in trace messages
    - MINOR: stconn: Always report READ/WRITE event on shutr/shutw
    - CLEANUP: stconn: Remove old read and write expiration dates
    - MINOR: stconn: Set half-close timeout using proxy settings
    - MINOR: stconn: Remove half-closed timeout
    - REGTESTS: cache: Use rxresphdrs to only get headers for 304 responses
    - MINOR: stconn: Add functions to set/clear SE_FL_EXP_NO_DATA flag from endpoint
    - BUG/MINOR: proto_ux: report correct error when bind_listener fails
    - BUG/MINOR: protocol: fix minor memory leak in protocol_bind_all()
    - MINOR: proto_uxst: add resume method
    - MINOR: listener/api: add lli hint to listener functions
    - MINOR: listener: add relax_listener() function
    - MINOR: listener: workaround for closing a tiny race between resume_listener() and stopping
    - MINOR: listener: make sure we don't pause/resume bypassed listeners
    - BUG/MEDIUM: listener: fix pause_listener() suspend return value handling
    - BUG/MINOR: listener: fix resume_listener() resume return value handling
    - BUG/MEDIUM: resume from LI_ASSIGNED in default_resume_listener()
    - MINOR: listener: pause_listener() becomes suspend_listener()
    - BUG/MEDIUM: listener/proxy: fix listeners notify for proxy resume
    - BUG/MINOR: sock_unix: match finalname with tempname in sock_unix_addrcmp()
    - MEDIUM: proto_ux: properly suspend named UNIX listeners
    - MINOR: proto_ux: ability to dump ABNS names in error messages
    - MINOR: haproxy: always protocol unbind on startup error path
    - BUILD: quic: 32-bits compilation issue with %zu in quic_rx_pkts_del()
    - BUG/MINOR: ring: do not realign ring contents on resize
    - MEDIUM: ring: make the offset relative to the head/tail instead of absolute
    - CLEANUP: ring: remove the now unused ring's offset
    - MINOR: config: add HAPROXY_BRANCH environment variable
    - BUILD: thead: Fix several 32 bits compilation issues with uint64_t variables
    - BUG/MEDIUM: fd: avoid infinite loops in fd_add_to_fd_list and fd_rm_from_fd_list
    - BUG/MEDIUM: h1-htx: Never copy more than the max data allowed during parsing
    - BUG/MINOR: stream: Remove BUG_ON about the task expiration in process_stream()
    - MINOR: stream: Handle stream's timeouts in a dedicated function
    - MEDIUM: stream: Eventually handle stream timeouts when exiting process_stream()
    - MINOR: stconn: Report a send activity when endpoint is willing to consume data
    - BUG/MEDIUM: stconn: Report a blocked send if some output data are not consumed
    - MEDIUM: mux-h1: Don't expect data from server as long as request is unfinished
    - MEDIUM: mux-h2: Don't expect data from server as long as request is unfinished
    - MEDIUM: mux-quic: Don't expect data from server as long as request is unfinished
    - DOC: config: Clarify the meaning of 'hold' in the 'resolvers' section
    - DOC: config: Replace TABs by spaces
    - BUG/MINOR: fd: used the update list from the fd's group instead of tgid
    - BUG/MEDIUM: fd: make fd_delete() support being called from a different group
    - CLEANUP: listener: only store conn counts for local threads
    - MINOR: tinfo: make thread_set functions return nth group/mask instead of first
    - MEDIUM: quic: improve fatal error handling on send
    - MINOR: quic: consider EBADF as critical on send()
    - BUG/MEDIUM: connection: Clear flags when a conn is removed from an idle list
    - BUG/MINOR: mux-h1: Don't report an error on an early response close
    - BUG/MINOR: http-check: Don't set HTX_SL_F_BODYLESS flag with a log-format body
    - BUG/MINOR: http-check: Skip C-L header for empty body when it's not mandatory
    - BUG/MINOR: http-fetch: recognize IPv6 addresses in square brackets in req.hdr_ip()
    - REGTEST: added tests covering smp_fetch_hdr_ip()
    - MINOR: quic: simplify return path in send functions
    - MINOR: quic: implement qc_notify_send()
    - MINOR: quic: purge txbuf before preparing new packets
    - MEDIUM: quic: implement poller subscribe on sendto error
    - MINOR: quic: notify on send ready
    - BUG/MINOR: http-ana: Don't increment conn_retries counter before the L7 retry
    - BUG/MINOR: http-ana: Do a L7 retry on read error if there is no response
    - BUG/MEDIUM: http-ana: Don't close request side when waiting for response
    - BUG/MINOR: mxu-h1: Report a parsing error on abort with pending data
    - MINOR: ssl: Destroy ocsp update http_client during cleanup
    - MINOR: ssl: Reinsert ocsp update entries later in case of unknown error
    - MINOR: ssl: Add ocsp update success/failure counters
    - MINOR: ssl: Store specific ocsp update errors in response and update ctx
    - MINOR: ssl: Add certificate's path to certificate_ocsp structure
    - MINOR: ssl: Add 'show ssl ocsp-updates' CLI command
    - MINOR: ssl: Add sample fetches related to OCSP update
    - MINOR: ssl: Use dedicated proxy and log-format for OCSP update
    - MINOR: ssl: Reorder struct certificate_ocsp members
    - MINOR: ssl: Increment OCSP update replay delay in case of failure
    - MINOR: ssl: Add way to dump ocsp response in base64
    - MINOR: ssl: Add global options to modify ocsp update min/max delay
    - REGTESTS: ssl: Fix ocsp update crt-lists
    - REGTESTS: ssl: Add test for new ocsp update cli commands
    - MINOR: ssl: Add ocsp-update information to "show ssl crt-list"
    - BUG/MINOR: ssl: Fix ocsp-update when using "add ssl crt-list"
    - MINOR: ssl: Replace now.tv_sec with date.tv_sec in ocsp update task
    - BUG/MINOR: ssl: Use 'date' instead of 'now' in ocsp stapling callback
    - BUG/MEDIUM: quic: properly handle duplicated STREAM frames
    - BUG/MINOR: cli: fix CLI handler "set anon global-key" call
    - MINOR: http_ext: adding some documentation, forgot to inline function
    - BUG/MINOR: quic: Do not send too small datagrams (with Initial packets)
    - MINOR: quic: Add a BUG_ON_HOT() call for too small datagrams
    - BUG/MINOR: quic: Ensure to be able to build datagrams to be retransmitted
    - BUG/MINOR: quic: v2 Initial packets decryption failed
    - MINOR: quic: Add traces about QUIC TLS key update
    - BUG/MINOR: quic: Remove force_ack for Initial,Handshake packets
    - BUG/MINOR: quic: Ensure not to retransmit packets with no ack-eliciting frames
    - BUG/MINOR: quic: Do not resend already acked frames
    - BUG/MINOR: quic: Missing detections of amplification limit reached
    - MINOR: quic: Send PING frames when probing Initial packet number space
    - BUG/MEDIUM: quic: do not crash when handling STREAM on released MUX
    - BUG/MAJOR: fd/thread: fix race between updates and closing FD
    - BUG/MEDIUM: dns: ensure ring offset is properly reajusted to head
    - BUG/MINOR: mux-quic: properly init STREAM frame as not duplicated
    - MINOR: quic: Do not accept wrong active_connection_id_limit values
    - MINOR: quic: Store the next connection IDs sequence number in the connection
    - MINOR: quic: Typo fix for ACK_ECN frame
    - MINOR: quic: RETIRE_CONNECTION_ID frame handling (RX)
    - MINOR: quic: Useless TLS context allocations in qc_do_rm_hp()
    - MINOR: quic: Add spin bit support
    - MINOR: quic: Add transport parameters to "show quic"
    - BUG/MEDIUM: sink/forwarder: ensure ring offset is properly readjusted to head
    - BUG/MINOR: dns: fix ring offset calculation on first read
    - BUG/MINOR: dns: fix ring offset calculation in dns_resolve_send()
    - MINOR: jwt: Add support for RSA-PSS signatures (PS256 algorithm)
    - MINOR: h3: add traces on h3_init_uni_stream() error paths
    - MINOR: quic: create a global list dedicated for closing QUIC conns
    - MINOR: quic: handle new closing list in show quic
    - MEDIUM: quic: release closing connections on stopping
    - BUG/MINOR: quic: Wrong RETIRE_CONNECTION_ID sequence number check
    - MINOR: fd/cli: report the polling mask in "show fd"
    - CLEANUP: sock: always perform last connection updates before wakeup
    - MINOR: quic: Do not stress the peer during retransmissions of lost packets
    - BUG/MINOR: init: properly detect NUMA bindings on large systems
    - BUG/MINOR: thread: report thread and group counts in the correct order
    - BUG/MAJOR: fd/threads: close a race on closing connections after takeover
    - MINOR: debug: add random delay injection with "debug dev delay-inj"
    - BUG/MINOR: mworker: use MASTER_MAXCONN as default maxconn value
    - BUG/MINOR: quic: Missing listener accept queue tasklet wakeups
    - MINOR: quic_sock: un-statify quic_conn_sock_fd_iocb()
    - DOC: config: fix typo "dependeing" in bind thread description
    - DOC/CLEANUP: fix typos
2023-03-10 16:28:37 +01:00