Commit Graph

2710 Commits

Author SHA1 Message Date
Thayne McCombs
001bddf48c DOC: configuration: clarify http-request wait-for-body
Make it more explicit what happens in the various scenarios that cause
HAProxy to stop waiting when "http-request wait-for-body" is used.

Also fix a couple of grammatical errors.

Fixes: #2410
Signed-Off-By: Thayne McCombs <astrothayne@gmail.com>
2024-01-29 07:48:30 +01:00
Willy Tarreau
535b247bf6 [RELEASE] Released version 3.0-dev2
Released version 3.0-dev2 with the following main changes :
    - MINOR: ot: logsrv struct becomes logger
    - MINOR: ssl: Update ssl_fc_curve/ssl_bc_curve to use SSL_get0_group_name
    - CLEANUP: ssl: fix indentation in smp_fetch_ssl_fc_ec()
    - DEV: patchbot: produce a verdict for too long commit messages
    - CLEANUP: ssl: fix indentation in smp_fetch_ssl_fc_ec() (part 2)
    - CLEANUP: quic: Double quic_dgram_parse() prototype declaration.
    - BUG/MINOR: map: list-based matching potential ordering regression
    - REGTESTS: add a test to ensure map-ordering is preserved
    - DOC: config: fix typo about map_*_key converters
    - DOC: configuration: corrected description of keyword tune.ssl.ocsp-update.mindelay
    - MINOR: map: mapfile ordering also matters for tree-based match types
    - DEV: phash: add a trivial perfect hash generator for integers
    - OPTIM: http: simplify http_get_status_idx() using a hash
    - CLEANUP: http: avoid duplicating literals in find_http_meth()
    - MINOR: http: add infrastructure to choose status codes for err / fail
    - MEDIUM: http_act: check status codes against the bit fields for err/fail
    - MEDIUM: http: add the ability to redefine http-err-codes and http-fail-codes
    - CI: codespell: ignore some words in URLs
    - CI: codespell: add more words to whitelist
    - CLEANUP: fix spelling of "occured" in src/h3.c
    - BUILD: quic: missing include for quic_tp
    - BUG/MINOR: mux-quic: do not prevent non-STREAM sending on flow control
    - MEDIUM: ssl: allow multiple fallback certificate to allow ECDSA/RSA selection
    - MEDIUM: ssl: generate '*' SNI filters for default certificates
    - MEDIUM: ssl: does not use default_ctx for 'generate-certificate' option
    - REORG: ssl: move 'generate-certificates' code to ssl_gencert.c
    - DOC: configuration: update configuration on how to have multiple default certs
    - MEDIUM: ssl: implements 'default-crt' keyword for bind Lines
    - CI: github: update wolfSSL to 5.6.6
    - DOC: INSTALL: require at least WolfSSL 5.6.6
    - DEV: h2: add support for multiple flags in mkhdr
    - DEV: h2: support hex-encoded data sequences in mkhdr
    - BUG/MINOR: mux-h2: also count streams for refused ones
    - BUG/MEDIUM: quic: keylog callback not called (USE_OPENSSL_COMPAT)
    - MINOR: vars: fix indentation in var_clear_buffer()
    - DOC: configuration: fix set-dst in actions keywords matrix
    - BUG/MEDIUM: mux-h2: refine connection vs stream error on headers
    - MINOR: mux-h2/traces: add a missing trace on connection WU with negative inc
    - MINOR: mux-h2: add a counter of "glitches" on a connection
    - MINOR: connection: add a new mux_ctl to report number of connection glitches
    - MINOR: mux-h2: implement MUX_CTL_GET_GLITCHES
    - MINOR: connection: add sample fetches to report per-connection glitches
    - BUILD: stick-table: fix build error on 32-bit platforms
    - MINOR: quic: Transport parameters encoding without version_information
    - MINOR: quic: Enable early data at SSL session level (aws-lc)
    - MINOR: ssl_sock: Early data disabled during SSL_CTX switching (aws-lc)
    - MINOR: quic: Correctly wait for the completion of handshakes with early data (aws-lc)
    - BUG/MEDIUM: cli: some err/warn msg dumps add LR into CSV output on stat's CLI
    - BUG/MINOR: jwt: fix jwt_verify crash on 32-bit archs
    - BUILD: quic: fix build error when using the compatibility layer
    - BUILD: quic: Fix build error when building QUIC against wolfssl.
    - BUILD: quic: Fix build error when building QUIC against libressl.
    - BUG/MINOR: hlua: fix uninitialized var in hlua_core_get_var()
    - CLEANUP: hlua: fix indent, remove extra return in hlua_core_get_var()
    - BUG/MEDIUM: cache: Fix crash when deleting secondary entry
    - BUG/MINOR: quic: newreno QUIC congestion control algorithm no more available
    - CLEANUP: quic: Remove unused CUBIC_BETA_SCALE_FACTOR_SHIFT macro.
    - MINOR: quic: Stop hardcoding a scale shifting value (CUBIC_BETA_SCALE_FACTOR_SHIFT)
    - MINOR: quic: extract qc_stream_buf free in a dedicated function
    - BUG/MEDIUM: quic: remove unsent data from qc_stream_desc buf
    - CLEANUP: fix spelling of "elemt"
    - CI: extend spell check white list
    - CI: enable spell check on git push
    - BUILD: makefile: also define cmd_CXX to pretty-print C++ build commands
    - BUILD/MEDIUM: deviceatlas: addon build rework.
    - DOC: deviceatlas: update to be in line with the v3 api.
    - BUILD/MEDIUM: deviceatlas: updating the addon part.
    - BUILD: deviceatlas: remove unneeded depenency on libcurl / libzip
    - BUILD: deviceatlas: fix empty "-I" left on CFLAGS
    - Revert "CI: enable spell check on git push"
2024-01-26 20:11:39 +01:00
David Carlier
f960cde786 DOC: deviceatlas: update to be in line with the v3 api.
Reflecting here all the changes, no longer need to cater with
the legacy v2 neither.
2024-01-26 18:54:27 +01:00
Willy Tarreau
6e5aa16145 MINOR: connection: add sample fetches to report per-connection glitches
Now with fc_glitches and bc_glitches we can retrieve the number of
detected glitches on a front or back connection. On the backend it
can indicate a bug in a server that may induce frequent reconnections
hence CPU usage in TLS reconnections, and on the frontend it may
indicate an abusive client that may be trying to attack the stack
or to fingerprint it. Small non-zero values are definitely expected
and can be caused by network glitches for example, as well as rare
bugs in the other component (or maybe even in haproxy). These should
never be considered as alarming as long as they remain low (i.e.
much less than one per request). A reg-test is provided.
2024-01-18 17:21:44 +01:00
Aurelien DARRAGON
62ef7966f0 DOC: configuration: fix set-dst in actions keywords matrix
Since d54e8f8107 ("DOC: config: reorganize actions into their own section")
dconv-generated shortlink for "set-dst" in actions keywords matrix is
broken.

This is due to trailing "<expr>" which should not be specified in the
matrix, but only in the actual keyword prototype and description.

This should be backported in 2.9 with d54e8f8107.
2024-01-18 16:36:47 +01:00
William Lallemand
97832ab823 MEDIUM: ssl: implements 'default-crt' keyword for bind Lines
The 'default-crt' bind keyword allows to specify multiples
default/fallback certificates, allowing one to have an RSA as well as an
ECDSA default.
2024-01-12 17:40:42 +01:00
William Lallemand
47bae78147 DOC: configuration: update configuration on how to have multiple default certs
HAProxy now allows to configure default certificates with SNI filters or
multi-cert bundle.
2024-01-12 17:40:42 +01:00
Willy Tarreau
4cc25f26f9 MEDIUM: http: add the ability to redefine http-err-codes and http-fail-codes
The new global keywords "http-err-codes" and "http-fail-codes" allow to
redefine which HTTP status codes indicate a client-induced error or a
server error, as tracked by stick-table counters. This is only done
globally, though everything was done so that it could easily be extended
to a per-proxy mechanism if there was a real need for this (but it would
eat quite more RAM then).

A simple reg-test was added (http-err-fail.vtc).
2024-01-11 15:10:08 +01:00
Miroslav Zagorac
f6ab0446fb DOC: configuration: corrected description of keyword tune.ssl.ocsp-update.mindelay
Deleted the text paragraph in the description of keyword
tune.ssl.ocsp-update.mindelay, which was added in the commit 5843237
"MINOR: ssl: Add global options to modify ocsp update min/max delay",
because it was a copy of the description of tune.ssl.ssl-ctx-cache-size.
2024-01-11 09:02:45 +01:00
Aurelien DARRAGON
c3a94a3750 DOC: config: fix typo about map_*_key converters
Fix a doc typo that was introduced with ca4758378 ("MINOR: map: add
map_*_key converters to provide the matching key").

No backport needed unless ca4758378 is.
2024-01-10 18:02:54 +01:00
Willy Tarreau
2b930aa7c3 [RELEASE] Released version 3.0-dev1
Released version 3.0-dev1 with the following main changes :
    - MINOR: channel: Use dedicated functions to deal with STREAMER flags
    - MEDIUM: applet: Handle channel's STREAMER flags on applets size
    - MINOR: applets: Use channel's field to compute amount of data received
    - MEDIUM: cache: Save body size of cached objects and track it on delivery
    - MEDIUM: cache: Add support for endp-to-endp fast-forwarding
    - MINOR: cache: Add global option to enable/disable zero-copy forwarding
    - MINOR: pattern: Use reference name as filename to read patterns from a file
    - MEDIUM: pattern: Add support for virtual and optional files for patterns
    - DOC: config: Add section about name format for maps and ACLs
    - DOC: management/lua: Update commands about map and acl
    - MINOR: promex: Add support for specialized front/back/li/srv metric names
    - MINOR: promex: Export active/backup metrics per-server
    - BUG/MINOR: ssl: Double free of OCSP Certificate ID
    - MINOR: ssl/cli: Add ha_(warning|alert) msgs to CLI ckch callback
    - BUG/MINOR: ssl: Wrong OCSP CID after modifying an SSL certficate
    - BUG/MINOR: lua: Wrong OCSP CID after modifying an SSL certficate (LUA)
    - DOC: configuration: typo req.ssl_hello_type
    - MINOR: hq-interop: add fastfwd support
    - CLEANUP: mux_quic: rename ffwd function with prefix qmux_strm_
    - MINOR: mux-quic: add traces for 0-copy/fast-forward
    - BUG/MINOR: mworker/cli: fix set severity-output support
    - CLEANUP: mworker/cli: add comments about pcli_find_and_exec_kw()
    - BUG/MEDIUM: quic: Possible buffer overflow when building TLS records
    - BUILD: ssl: update types in wolfssl cert selection callback
    - MINOR: ssl: activate the certificate selection callback for WolfSSL
    - CI: github: switch to wolfssl git-c4b77ad for new PR
    - BUG/MEDIUM: map/acl: pat_ref_{set,delete}_by_id regressions
    - BUG/MINOR: ext-check: cannot use without preserve-env
    - CLEANUP: mux-quic: remove unused prototype
    - MINOR: mux-quic: clean up qcs Rx buffer allocation API
    - MINOR: mux-quic: clean up qcs Tx buffer allocation API
    - CLEANUP: mux-quic: clean up app ops callback definitions
    - MINOR: mux-quic: factorize QC_SF_UNKNOWN_PL_LENGTH set
    - MINOR: h3: complete traces for sending
    - MINOR: h3: adjust zero-copy sending related code
    - MINOR: hq-interop: use zero-copy to transfer single HTX data block
    - BUG/MEDIUM: quic: QUIC CID removed from tree without locking
    - BUG/MEDIUM: stconn: Block zero-copy forwarding if EOS/ERROR on consumer side
    - BUG/MEDIUM: mux-h1: Cound data from input buf during zero-copy forwarding
    - BUG/MEDIUM: mux-h1: Explicitly skip request's C-L header if not set originally
    - CLEANUP: mux-h1: Fix a trace message about C-L header addition
    - BUG/MEDIUM: mux-h2: Report too large HEADERS frame only when rxbuf is empty
    - BUG/MEDIUM: mux-quic: report early error on stream
    - DOC: config: add arguments to sample fetch methods in the table
    - DOC: config: also add arguments to the converters in the table
    - BUG/MINOR: resolvers: default resolvers fails when network not configured
    - SCRIPTS: mk-patch-list: produce a list of patches
    - DEV: patchbot: add the AI-based bot to pre-select candidate patches to backport
    - BUG/MEDIUM: mux-h2: Switch pending error to error if demux buffer is empty
    - BUG/MEDIUM: mux-h2: Only Report H2C error on read error if demux buffer is empty
    - BUG/MEDIUM: mux-h2: Don't report error on SE if error is only pending on H2C
    - BUG/MEDIUM: mux-h2: Don't report error on SE for closed H2 streams
    - DOC: config: Update documentation about local haproxy response
    - DEV: patchbot: use checked buttons as reference instead of internal table
    - DEV: patchbot: allow to show/hide backported patches
    - MINOR: h3: remove quic_conn only reference
    - BUG/MINOR: server: Use the configured address family for the initial resolution
    - MINOR: mux-quic: remove qcc_shutdown() from qcc_release()
    - MINOR: mux-quic: use qcc_release in case of init failure
    - MINOR: mux-quic: adjust error code in init failure
    - MINOR: h3: add traces for connection init stage
    - BUG/MINOR: h3: properly handle alloc failure on finalize
    - MINOR: h3: use INTERNAL_ERROR code for init failure
    - BUG/MAJOR: stconn: Disable zero-copy forwarding if consumer is shut or in error
    - MINOR: stats: store the parent proxy in stats ctx (http)
    - BUG/MEDIUM: stats: unhandled switching rules with TCP frontend
    - MEDIUM: proxy: set PR_O_HTTP_UPG on implicit upgrades
    - MINOR: proxy: monitor-uri works with tcp->http upgrades
    - OPTIM: server: eb lookup for server_find_by_name()
    - OPTIM: server: ebtree lookups for findserver_unique_* functions
    - MINOR: server/event_hdl: add server_inetaddr struct to facilitate event data usage
    - MINOR: server/event_hdl: update _srv_event_hdl_prepare_inetaddr prototype
    - BUG/MINOR: server/event_hdl: propagate map port info through inetaddr event
    - MINOR: server: ensure connection cleanup on server addr changes
    - CLEANUP: server/event_hdl: remove purge_conn hint in INETADDR event
    - MEDIUM: server: merge srv_update_addr() and srv_update_addr_port() logic
    - CLEANUP: server: remove unused server_parse_addr_change_request() function
    - CLEANUP: resolvers: remove duplicate func prototype
    - MINOR: resolvers: add unique numeric id to nameservers
    - MEDIUM: server: make server_set_inetaddr() updater serializable
    - MINOR: server/event_hdl: expose updater info through INETADDR event
    - MINOR: server: add dns hint in server_inetaddr_updater struct
    - MEDIUM: server/dns: clear RMAINT when addr resolves again
    - BUG/MINOR: server/dns: use server_set_inetaddr() to unset srv addr from DNS
    - BUG/MEDIUM: server/dns: perform svc_port updates atomically from SRV records
    - MEDIUM: peers: use server as stream target
    - CLEANUP: peers: remove unused sock_init_arg struct member
    - CLEANUP: peers: remove unused "proto" and "xprt" struct members
    - MINOR: peers: rely on srv->addr and remove peer->addr
    - DOC: config: add context hint for server keywords
    - MINOR: stktable: add table_process_entry helper function
    - MINOR: stktable: use {show,set,clear} table with ptr
    - MINOR: map: add map_*_key converters to provide the matching key
    - DOC: fix typo for fastfwd QUIC option
    - BUG/MINOR: mux-quic: always report error to SC on RESET_STREAM emission
    - MEDIUM: mux-quic: add BUG_ON if sending on locally closed QCS
    - BUG/MINOR: mux-quic: disable fast-fwd if connection on error
    - BUG/MINOR: quic: Wrong keylog callback setting.
    - BUG/MINOR: quic: Missing call to TLS message callbacks
    - MINOR: h3: check connection error during sending
    - BUG/MINOR: h3: close connection on header list too big
    - BUG/MINOR: h3: close connection on sending alloc errors
    - BUG/MINOR: h3: disable fast-forward on buffer alloc failure
    - Revert "MINOR: mux-quic: Disable zero-copy forwarding for send by default"
    - MINOR: stktable: stktable_data_ptr() cannot fail in table_process_entry()
    - CLEANUP: assorted typo fixes in the code and comments
    - CI: use semantic version compare for determing "latest" OpenSSL
    - CLEANUP: server: remove ambiguous check in srv_update_addr_port()
    - CLEANUP: resolvers: remove unused RSLV_UPD_OBSOLETE_IP flag
    - CLEANUP: resolvers: remove some more unused RSLV_UDP flags
    - MEDIUM: server: simplify snr_set_srv_down() to prevent confusions
    - MINOR: backend: export get_server_*() functions
    - MINOR: tcpcheck: export proxy_parse_tcpcheck()
    - MEDIUM: udp: allow to retrieve the frontend destination address
    - MINOR: global: export a way to list build options
    - MINOR: debug: add features and build options to "show dev"
    - BUG/MINOR: server: fix server_find_by_name() usage during parsing
    - REGTESTS: check attach-srv out of order declaration
    - CLEANUP: quic: Remaining useless code into server part
    - BUILD: quic: Missing quic_ssl.h header protection
    - BUG/MEDIUM: h3: fix incorrect snd_buf return value
    - MINOR: h3: do not consider missing buf room as error on trailers
    - BUG/MEDIUM: stconn: Forward shutdown on write timeout only if it is forwardable
    - BUG/MEDIUM: stconn: Set fsb date if zero-copy forwarding is blocked during nego
    - BUG/MEDIUM: spoe: Never create new spoe applet if there is no server up
    - MINOR: mux-h2: support limiting the total number of H2 streams per connection
    - CLEANUP: mux-h2: remove the printfs from previous commit on h2 streams limit.
    - DEV: h2: add the ability to emit literals in mkhdr
    - DEV: h2: add the preface as well in supported output types
    - DEV: h2: support passing raw data for a frame
    - IMPORT: ebtree: implement and use flsnz_long() to count bits
    - IMPORT: ebtree: switch the sizes and offsets to size_t and ssize_t
    - IMPORT: ebtree: rework the fls macros to better deal with arch-specific ones
    - IMPORT: ebtree: make string_equal_bits turn back to unsigned char
    - IMPORT: ebtree: use unsigned ints for flznz()
    - IMPORT: ebtree: make string_equal_bits() return an unsigned
2024-01-06 14:09:35 +01:00
Willy Tarreau
983ac4397d MINOR: mux-h2: support limiting the total number of H2 streams per connection
This patch introduces a new setting: tune.h2.fe.max-total-streams. It
sets the HTTP/2 maximum number of total streams processed per incoming
connection. Once this limit is reached, HAProxy will send a graceful GOAWAY
frame informing the client that it will close the connection after all
pending streams have been closed. In practice, clients tend to close as fast
as possible when receiving this, and to establish a new connection for next
requests. Doing this is sometimes useful and desired in situations where
clients stay connected for a very long time and cause some imbalance inside a
farm. For example, in some highly dynamic environments, it is possible that
new load balancers are instantiated on the fly to adapt to a load increase,
and that once the load goes down they should be stopped without breaking
established connections. By setting a limit here, the connections will have
a limited lifetime and will be frequently renewed, with some possibly being
established to other nodes, so that existing resources are quickly released.

The default value is zero, which enforces no limit beyond those implied by
the protocol (2^30 ~= 1.07 billion). Values around 1000 were found to
already cause frequent enough connection renewal without causing any
perceptible latency to most clients. One notable exception here is h2load
which reports errors for all requests that were expected to be sent over
a given connection after it receives a GOAWAY. This is an already known
limitation: https://github.com/nghttp2/nghttp2/issues/981

The patch was made in two parts inside h2_frt_handle_headers():
  - the first one, at the end of the function, which verifies if the
    configured limit was reached and if it's needed to emit a GOAWAY ;

  - the second, just before decoding the stream frame, which verifies if
    a previously configured limit was ignored by the client, and closes
    the connection if this happens. Indeed, one reason for a connection
    to stay alive for too long definitely comes from a stupid bot that
    periodically fetches the same resource, scans lots of URLs or tries
    to brute-force something. These ones are more likely to just ignore
    the last stream ID advertised in GOAWAY than a regular browser, or
    a well-behaving client such as curl which respects it. So in order
    to make sure we can close the connection we need to enforce the
    advertised limit.

Note that a regular client will not face a problem with that because in
the worst case it will have max_concurrent_streams in flight and this
limit is taken into account when calculating the advertised last
acceptable stream ID.

Just a note: it may also be possible to move the first part above to
h2s_frt_stream_new() instead so that it's not processed for trailers,
though it doesn't seem to be more interesting, first because it has
two return points.

This is something that may be backported to 2.9 and 2.8 to offer more
control to those dealing with dynamic infrastructures, especially since
for now we cannot force a connection to be cleanly closed using rules
(e.g. github issues #946, #2146).
2024-01-05 18:49:11 +01:00
Ilya Shipitsin
8705e45964 CLEANUP: assorted typo fixes in the code and comments
This is 38th iteration of typo fixes
2024-01-02 10:19:48 +01:00
Amaury Denoyelle
b7274e69ef Revert "MINOR: mux-quic: Disable zero-copy forwarding for send by default"
This reverts commit 18f2ccd244.

Found issues related to QUIC fast-forward were resolved (see github
issue #2372). Reenable it by default. If any issue arises, it can be
disabled using the global statement :
  tune.quit.zero-copy-fwd-send off

This can be backported to 2.9, but only after a sensible period of
observation.
2023-12-22 16:30:37 +01:00
Amaury Denoyelle
c62487fd08 DOC: fix typo for fastfwd QUIC option
Replace prefix 'tune.quit.' by 'tune.quic.'.

This should be backported up to 2.9.
2023-12-21 15:42:08 +01:00
Aurelien DARRAGON
ca47583787 MINOR: map: add map_*_key converters to provide the matching key
All map_*_ converters now have an additional output type: key. Such
converters will return the matched entry's key (as found in the map file)
as a string instead of the value.

Consider this example map file:
 |example.com value1
 |haproxy value2

With the above map file:

str(test.example.com/url),map_dom_key(file.map) will return "example.com"
str(running haproxy),map_sub_key(file.map) will return "haproxy"

This should address GH #1446.
2023-12-21 14:22:27 +01:00
Aurelien DARRAGON
3d4e1e682b DOC: config: add context hint for server keywords
Add a small list of contexts where each server keyword is expected to be
employed.

This should NOT be backported.
2023-12-21 14:22:27 +01:00
Christopher Faulet
0a203c1d3f DOC: config: Update documentation about local haproxy response
Documentation about 'L' state in the termination state was outdated. Today,
not only the request may be intercepted, but also the response.
Documentation about 'L' must be more generic.

However, documentation about possible 2-letter termination states was also
extended to add 'LC' and 'LH' in the list. And 'LR' was adapted too.

This patch should fix the issue #2384. It may be backported to every stable
versions. Note that on 2.8 and lowers, we talk about session and not stream.
2023-12-19 10:53:22 +01:00
Willy Tarreau
c5bde03a0a DOC: config: also add arguments to the converters in the table
Now that dconv supports linking to keywords with arguments from tables,
let's mention the arguments in the summary table of converters.
2023-12-15 11:18:27 +01:00
Willy Tarreau
0d261dd13a DOC: config: add arguments to sample fetch methods in the table
Now that dconv supports linking to keywords with arguments from tables,
let's mention the arguments in the summary tables of sample fetch methods.
2023-12-15 11:18:08 +01:00
William Lallemand
906525b16b DOC: configuration: typo req.ssl_hello_type
rep_ssl_hello_type was renamed in res.ssl_hello_type a long time ago.

This patch fixes a typo where an example was renamed
"rep.ssl_hello_type" instead of "res.ssl_hello_type"

fixes issue #2377 and #2379.

Must be backported in all maintained versions.
2023-12-07 15:00:58 +01:00
Christopher Faulet
f792a25904 DOC: management/lua: Update commands about map and acl
Because maps and list of ACLs are no longer necessarily referenced by
filenames, CLI commands to manipulate them were updated accordingly. Instead
of "filename" we talk about "name" now.

The same is performed in the LUA documentation.
2023-12-06 10:24:41 +01:00
Christopher Faulet
27a4a4872c DOC: config: Add section about name format for maps and ACLs
Maps and list of ACLs can now reference something else than regular files
and can have prefix to set the type of the list (file, virutal file or
optional file). So, the configuration manual was updated accordingly.

The section 2.7. about name format for maps and ACLs was added (the former
2.7. sections with some examples was moved to 2.8.) and references to map or
ACLs files were updated.
2023-12-06 10:24:41 +01:00
Christopher Faulet
533121a56e MINOR: cache: Add global option to enable/disable zero-copy forwarding
tune.cache.zero-copy-forwarding parameter can now be used to enable or
disable the zero-copy fast-forwarding for the cache applet only. It is
enabled ('on') by default. It can be disabled by setting the parameter to
'off'.
2023-12-06 10:24:41 +01:00
Willy Tarreau
eb67d63456 [RELEASE] Released version 3.0-dev0
Released version 3.0-dev0 with the following main changes :
    - exact copy of 2.9.0
2023-12-05 16:19:35 +01:00
Willy Tarreau
fddb8c13b6 [RELEASE] Released version 2.9.0
Released version 2.9.0 with the following main changes :
    - DOC: config: add missing colon to "bytes_out" sample fetch keyword (2)
    - BUG/MINOR: cfgparse-listen: fix warning being reported as an alert
    - DOC: config: add matrix entry for "max-session-srv-conns"
    - DOC: config: fix monitor-fail typo
    - DOC: config: add context hint for proxy keywords
    - DEBUG: stream: Report lra/fsb values for front end back SC in stream dump
    - REGTESTS: sample: Test the behavior of consecutive delimiters for the field converter
    - BUG/MINOR: sample: Make the `word` converter compatible with `-m found`
    - DOC: Clarify the differences between field() and word()
    - BUG/MINOR: server/event_hdl: properly handle AF_UNSPEC for INETADDR event
    - BUILD: http_htx: silence uninitialized warning on some gcc versions
    - MINOR: acme.sh: don't use '*' in the filename for wildcard domain
    - MINOR: global: Use a dedicated bitfield to customize zero-copy fast-forwarding
    - MINOR: mux-pt: Add global option to enable/disable zero-copy forwarding
    - MINOR: mux-h1: Add global option to enable/disable zero-copy forwarding
    - MINOR: mux-h2: Add global option to enable/disable zero-copy forwarding
    - MINOR: mux-quic: Add global option to enable/disable zero-copy forwarding
    - MINOR: mux-quic: Disable zero-copy forwarding for send by default
    - DOC: config: update the reminder on the HTTP model and add some terminology
    - DOC: config: add a few more differences between HTTP/1 and 2+
    - DOC: config: clarify session vs stream
    - DOC: config: fix typo abandonned -> abandoned
    - DOC: management: fix two latest typos (optionally, exception)
    - BUG/MEDIUM: peers: fix partial message decoding
    - DOC: management: update stream vs session
2023-12-05 16:15:30 +01:00
Willy Tarreau
ba168dcf18 DOC: management: update stream vs session
Indicate for some commands such as "show sess" that we now dump streams
and not sessions.
2023-12-05 09:31:23 +01:00
Willy Tarreau
89807a76d6 DOC: management: fix two latest typos (optionally, exception)
No backport needed, these were introduced by latest commits 3dd55fa13
("MINOR: mworker/cli: implement hard-reload over the master CLI") and
cef29d370 ("MINOR: trace: define simple -dt argument").
2023-12-05 04:04:50 +01:00
Willy Tarreau
a583c3e940 DOC: config: fix typo abandonned -> abandoned
No need to backport, it was introduced by recent commit fafa34e5f ("DOC:
config: update the reminder on the HTTP model and add some terminology").
2023-12-05 04:02:25 +01:00
Willy Tarreau
200f05ecd2 DOC: config: clarify session vs stream
Rename "session" to "stream" where relevant (termination states, queue,
and so on).
2023-12-04 18:58:32 +01:00
Willy Tarreau
f39fe4377e DOC: config: add a few more differences between HTTP/1 and 2+
Mention the lack of reason phrase and the existence of pseudo-headers for
the request and the response.
2023-12-04 18:58:32 +01:00
Willy Tarreau
fafa34e5f5 DOC: config: update the reminder on the HTTP model and add some terminology
It was really necessary to try to clear the confusion between sessions
and streams, so let's first lift a little bit the HTTP model part to
better consider new protocols, and explain what a stream is and how this
differs from the earlier sessions.
2023-12-04 18:58:32 +01:00
Christopher Faulet
18f2ccd244 MINOR: mux-quic: Disable zero-copy forwarding for send by default
There is at least an bug for now in this part and it is still unstable. Thus
it is better to disable it for now by default. It can be enable by setting
tune.quic.zero-copy-fwd-send to 'on'.
2023-12-04 15:36:02 +01:00
Christopher Faulet
5c959336fd MINOR: mux-quic: Add global option to enable/disable zero-copy forwarding
tune.quic.zero-copy-fwd-send can now be used to enable or disable the
zero-copy fast-forwarding for the QUIC mux only, for sends. For now, there
is no option to disable it for receives because it is not supported yet.

It is enabled ('on') by default.
2023-12-04 15:33:52 +01:00
Christopher Faulet
6da0429e75 MINOR: mux-h2: Add global option to enable/disable zero-copy forwarding
tune.h2.zero-copy-fwd-send can now be used to enable or disable the
zero-copy fast-forwarding for the H2 mux only, for sends. For now, there is
no option to disable it for receives because it is not supported yet.

It is enabled ('on') by default.
2023-12-04 15:33:34 +01:00
Christopher Faulet
f5e73024e9 MINOR: mux-h1: Add global option to enable/disable zero-copy forwarding
tune.h1.zero-copy-fwd-recv and tune.h1.zero-copy-fwd-send can now be used to
enable or disable the zero-copy fast-forwarding for the H1 mux only, for
receives or sends. Unlike the PT mux, there are 2 options here because
client and server sides can use difference muxes.

Both are enabled ('on') by default.
2023-12-04 15:33:07 +01:00
Christopher Faulet
eccef69137 MINOR: mux-pt: Add global option to enable/disable zero-copy forwarding
tune.pt.zero-copy-forwarding parameter can now be used to enable or disable
the zero-copy fast-forwarding for the PT mux only. It is enabled ('on') by
default. It can be disabled by setting the parameter to 'off'. In this case,
this disables receive and send side.
2023-12-04 15:32:32 +01:00
Christopher Faulet
7732323cf3 MINOR: global: Use a dedicated bitfield to customize zero-copy fast-forwarding
Zero-copy fast-forwading feature is a quite new and is a bit sensitive.
There is an option to disable it globally. However, all protocols have not
the same maturity. For instance, for the PT multiplexer, there is nothing
really new. The zero-copy fast-forwading is only another name for the kernel
splicing. However, for the QUIC/H3, it is pretty new, not really optimized
and it will evolved. And soon, the support will be added for the cache
applet.

In this context, it is usefull to be able to enable/disable zero-copy
fast-forwading per-protocol and applet. And when it is applicable, on sends
or receives separately. So, instead of having one flag to disable it
globally, there is now a dedicated bitfield, global.tune.no_zero_copy_fwd.
2023-12-04 15:31:47 +01:00
Tim Duesterhus
ed3bc0bfdd DOC: Clarify the differences between field() and word()
word() mentions that delimiters at the start and end are ignored, but
it does not mention that consecutive delimiters are merged.

May be backported as far as the patch applies.
2023-12-01 14:35:47 +01:00
Aurelien DARRAGON
54627f9916 DOC: config: add context hint for proxy keywords
Add a small list of contexts where each proxy keyword is expected to be
employed. (Similar to the defaults/frontend/backend/listen compatibility
grid).
2023-12-01 09:09:45 +01:00
Aurelien DARRAGON
f4bf84f33f DOC: config: fix monitor-fail typo
monitor-fail doesn't exist, but it was mentionned in the documentation.
Fixing it with "monitor fail" instead.
2023-12-01 09:09:45 +01:00
Aurelien DARRAGON
8b2152181c DOC: config: add matrix entry for "max-session-srv-conns"
Following 4039329 ("DOC: config: specify supported sections for
"max-session-srv-conns"), "max-session-srv-conns" was also missing
from the proxy keyword matrix.
2023-12-01 09:09:45 +01:00
Tim Duesterhus
49293dfb65 DOC: config: add missing colon to "bytes_out" sample fetch keyword (2)
This reapplies 1eb049dc67, as the change was
accidentally reverted in 5ef48e063e.
2023-11-30 20:33:28 +01:00
Willy Tarreau
7ed737d5a7 [RELEASE] Released version 2.9-dev12
Released version 2.9-dev12 with the following main changes :
    - BUG/MINOR: global: Fix tune.disable-(fast-forward/zero-copy-forwarding) options
    - DOC: config: removing "log-balance" references
    - MINOR: server/event_hdl: add SERVER_INETADDR event
    - MINOR: tools: use const for read only pointers in ip{cmp,cpy}
    - MINOR: server/ip: centralize server ip updates
    - MINOR: backend: remove invalid mode test for "hash-balance-factor"
    - Revert "MINOR: cfgparse-listen: warn when use-server rules is used in wrong mode"
    - MINOR: proxy: add free_logformat_list() helper function
    - MINOR: proxy: add free_server_rules() helper function
    - MINOR: log/backend: prevent "use-server" rules use with LOG mode
    - MINOR: log/balance: set lbprm tot_weight on server on queue/dequeue
    - DOC: config: specify supported sections for "max-session-srv-conns"
    - DOC: config: fix timeout check inheritance restrictions
    - REGTESTS: connection: disable http_reuse_be_transparent.vtc if !TPROXY
    - DOC: lua: add sticktable class reference from Proxy.stktable
    - DOC: lua: fix Proxy.get_mode() output
    - DOC: lua: add "syslog" to Proxy.get_mode() output
    - MEDIUM: ssl: implement rsa/ecdsa selection with WolfSSL
    - MINOR: ssl: replace 'trash.area' by 'servername' in ssl_sock_switchctx_cbk()
    - MINOR: ssl: move certificate selection in a dedicate function
    - MEDIUM: ssl: use ssl_sock_chose_sni_ctx() in the clienthello callback
    - MINOR: mworker/cli: implement hard-reload over the master CLI
    - BUG/MEDIUM: mux-h1: Properly ignore trailers when a content-length is announced
    - MINOR: task/profiling: do not record task_drop_running() as a caller
    - OPTIM: pattern: save memory and time using ebst instead of ebis
    - BUILD: map: fix build warning
    - MINOR: trace: define simple -dt argument
    - MINOR: trace: parse level in a function
    - MINOR: trace: parse verbosity in a function
    - MINOR: trace: support -dt optional format
    - OPTIM: mux-h2/zero-copy: don't allocate more buffers per connections than streams
    - BUG/MINOR: quic: fix CONNECTION_CLOSE_APP encoding
    - BUG/MEDIUM: stconn: Don't perform zero-copy FF if opposite SC is blocked
    - BUG/MEDIUM: mux-h2: Remove H2_SF_NOTIFIED flag for H2S blocked on fast-forward
    - CLEANUP: quic: Remove dead definitions/declarations
    - REORG: quic: Move some QUIC CLI code to its C file
    - REORG: quic: Add a new module to handle QUIC connection IDs
    - REORG: quic: QUIC connection types header cleaning
    - BUILD: quic: Missing RX header inclusions
    - REORG: quic: Move CRYPTO data buffer defintions to QUIC TLS module
    - REORG: quic: Move QUIC CRYPTO stream definitions/declarations to QUIC TLS
    - REORG: quic: Move several inlined functions from quic_conn.h
    - REORG: quic: Move QUIC SSL BIO method related functions to quic_ssl.c
    - REORG: quic: Move the QUIC DCID parser to quic_sock.c
    - REORG: quic: Rename some functions used upon ACK receipt
    - REORG: quic: Move QUIC path definitions/declarations to quic_cc module
    - REORG: quic: Move qc_handle_conn_migration() to quic_conn.c
    - REORG: quic: Move quic_build_post_handshake_frames() to quic_conn module
    - REORG: quic: Move qc_may_probe_ipktns() to quic_tls.h
    - REORG: quic: Move qc_pkt_long() to quic_rx.h
    - REORG: quic: Rename some (quic|qc)_conn* objects to quic_conn_closed
    - REORG: quic: Move NEW_CONNECTION_ID frame builder to quic_cid
    - REORG: quic: Move ncbuf related function from quic_rx to quic_conn
    - REORG: quic: Add a new module for QUIC retry
    - BUILD: quic: Several compiler warns fixes after retry module creation
    - REORG: quic: Move qc_notify_send() to quic_conn
    - REORG: quic: Add a new module for retransmissions
    - REORG: quic: Remove qc_pkt_insert() implementation
    - REORG: quic: Move quic_increment_curr_handshake() to quic_sock
    - BUG/MINOR: cache: Remove incomplete entries from the cache when stream is closed
    - MEDIUM: cli: allow custom pattern for payload
    - CLEANUP: mworker/cli: use a label to return errors
    - MINOR: mworker/cli: implements the customized payload pattern for master CLI
    - DOC: management: add documentation about customized payload pattern
    - BUG/MEDIUM: server/event_hdl: memory overrun in _srv_event_hdl_prepare_inetaddr()
    - MINOR: event_hdl: add global tunables
    - BUG/MAJOR: server/addr: fix a race during server addr:svc_port updates
    - MEDIUM: log/balance: support FQDN for UDP log servers
    - BUG/MINOR: compression: possible NULL dereferences in comp_prepare_compress_request()
    - BUG/MEDIUM: master/cli: Properly pin the master CLI on thread 1 / group 1
    - BUG/MEDIUM: mux-quic: Stop zero-copy FF during nego if input is not empty
    - CLEANUP: log: Fix %rc comment in sess_build_logline()
    - BUG/MINOR: h3: fix TRAILERS encoding
    - BUG/MINOR: h3: always reject PUSH_PROMISE
    - MINOR: h3: use correct error code for missing SETTINGS
    - MINOR: http-fetch: Add a sample to retrieve the server status code
    - DOC: config: Improve 'status' sample documentation
    - MINOR: http-fetch: Add a sample to get the transaction status code
    - MEDIUM: http-ana: Set termination state before returning haproxy response
    - MINOR: stream: Expose session terminate state via a new sample fetch
    - MINOR: stream: add a sample fetch to get the number of connection retries
    - MINOR: stream: Expose the stream's uniq_id via a new sample fetch
    - MINOR: muxes: Rename mux_ctl_type values to use MUX_CTL_ prefix
    - MINOR: muxes: Add a callback function to send commands to mux streams
    - MINOR: muxes: Implement ->sctl() callback for muxes and return the stream id
    - MINOR: Add sample fetches to get the frontend and backend stream ID
    - BUG/MEDIUM: cli: Don't look for payload pattern on empty commands
    - DOC: config: Add argument for tune.lua.maxmem
    - DOC: config: fix mention of request slot in http-response capture
    - DOC: config: fix remaining mention of @reverse for attach-srv action
    - DOC: config: fix missing characters in set-spoe-group action
    - DOC: config: reorganize actions into their own section
    - BUG/MINOR: acme.sh: update the deploy script
    - MINOR: rhttp: mark reverse HTTP as experimental
    - CLEANUP: quic_cid: remove unused listener arg
    - BUG/MINOR: quic_tp: fix preferred_address decoding
    - MINOR: quic_tp: use in_addr/in6_addr for preferred_address
    - MINOR: acme.sh: use the master CLI for hot update
    - DOC: config: move the cache-use and cache-store actions to the proper section
    - DOC: config: fix alphabetical ordering of converter keywords
    - DOC: config: add missing colon to "bytes_out" sample fetch keyword
    - DOC: config: add an index of converter keywords
    - DOC: config: add an index of sample fetch keywords
    - BUG/MINOR: config: Stopped parsing upon unmatched environment variables
    - DEBUG: unstatify a few functions that are often present in backtraces
    - BUILD: server: shut a bogus gcc warning on certain ubuntu
2023-11-30 18:07:06 +01:00
Willy Tarreau
fa0a304f35 DOC: config: add an index of sample fetch keywords
Now we're adding a table for each section, it allows to more easily
spot the list of available sample fetch functions and their types.
For now the arguments are not mentioned in the table because they'd
break indexing but they can be added back later.
2023-11-30 16:28:56 +01:00
Willy Tarreau
5ef48e063e DOC: config: add an index of converter keywords
The goal here is to have a centralized list of converters and
their in/out types.
2023-11-30 16:28:56 +01:00
Willy Tarreau
1eb049dc67 DOC: config: add missing colon to "bytes_out" sample fetch keyword
The colon was missing between the keyword and the type, breaking
rendering and indexing.
2023-11-30 16:28:56 +01:00
Willy Tarreau
9930c084ea DOC: config: fix alphabetical ordering of converter keywords
- rfc7239_* were misplaced and incorrectly ordered
- table_gpt was placed before some table_gpc*
- capture-req/res were misplaced
- htonl was misplaced
- upper/url_* were misplaced
- x509_v_err_str was misplaced

Let's fix these since poor ordering complicates their finding.
2023-11-30 16:28:56 +01:00
Willy Tarreau
0d58f19c26 DOC: config: move the cache-use and cache-store actions to the proper section
Actions were grouped by previous commit d54e8f810 ("DOC: config: reorganize
actions into their own section") but cache-use and cache-store were still
making reference to the cache section. This moves the text back to their
respective keywords in the actions section and leaves the example and an
explanation of how to use the keywords in the cache section.
2023-11-30 16:28:56 +01:00
Amaury Denoyelle
86e5c607d1 MINOR: rhttp: mark reverse HTTP as experimental
Mark the reverse HTTP feature as experimental. This will allow to adjust
if needed the configuration mechanism with future developments without
maintaining retro-compatibility.

Concretely, each config directives linked to it now requires to specify
first global expose-experimental-directives before. This is the case for
the following directives :
- rhttp@ prefix uses in bind and server lines
- nbconn bind keyword
- attach-srv tcp rule

Each documentation section refering to these keywords are updated to
highlight this new requirement.

Note that this commit has duplicated on several places the code from the
global function check_kw_experimental(). This is because the latter only
work with cfg_keyword type. This is not adapted with bind_kw or
action_kw types. This should be improve in a future patch.
2023-11-30 15:04:27 +01:00