Deleted the text paragraph in the description of keyword
tune.ssl.ocsp-update.mindelay, which was added in the commit 5843237
"MINOR: ssl: Add global options to modify ocsp update min/max delay",
because it was a copy of the description of tune.ssl.ssl-ctx-cache-size.
Fix a doc typo that was introduced with ca4758378 ("MINOR: map: add
map_*_key converters to provide the matching key").
No backport needed unless ca4758378 is.
Released version 3.0-dev1 with the following main changes :
- MINOR: channel: Use dedicated functions to deal with STREAMER flags
- MEDIUM: applet: Handle channel's STREAMER flags on applets size
- MINOR: applets: Use channel's field to compute amount of data received
- MEDIUM: cache: Save body size of cached objects and track it on delivery
- MEDIUM: cache: Add support for endp-to-endp fast-forwarding
- MINOR: cache: Add global option to enable/disable zero-copy forwarding
- MINOR: pattern: Use reference name as filename to read patterns from a file
- MEDIUM: pattern: Add support for virtual and optional files for patterns
- DOC: config: Add section about name format for maps and ACLs
- DOC: management/lua: Update commands about map and acl
- MINOR: promex: Add support for specialized front/back/li/srv metric names
- MINOR: promex: Export active/backup metrics per-server
- BUG/MINOR: ssl: Double free of OCSP Certificate ID
- MINOR: ssl/cli: Add ha_(warning|alert) msgs to CLI ckch callback
- BUG/MINOR: ssl: Wrong OCSP CID after modifying an SSL certficate
- BUG/MINOR: lua: Wrong OCSP CID after modifying an SSL certficate (LUA)
- DOC: configuration: typo req.ssl_hello_type
- MINOR: hq-interop: add fastfwd support
- CLEANUP: mux_quic: rename ffwd function with prefix qmux_strm_
- MINOR: mux-quic: add traces for 0-copy/fast-forward
- BUG/MINOR: mworker/cli: fix set severity-output support
- CLEANUP: mworker/cli: add comments about pcli_find_and_exec_kw()
- BUG/MEDIUM: quic: Possible buffer overflow when building TLS records
- BUILD: ssl: update types in wolfssl cert selection callback
- MINOR: ssl: activate the certificate selection callback for WolfSSL
- CI: github: switch to wolfssl git-c4b77ad for new PR
- BUG/MEDIUM: map/acl: pat_ref_{set,delete}_by_id regressions
- BUG/MINOR: ext-check: cannot use without preserve-env
- CLEANUP: mux-quic: remove unused prototype
- MINOR: mux-quic: clean up qcs Rx buffer allocation API
- MINOR: mux-quic: clean up qcs Tx buffer allocation API
- CLEANUP: mux-quic: clean up app ops callback definitions
- MINOR: mux-quic: factorize QC_SF_UNKNOWN_PL_LENGTH set
- MINOR: h3: complete traces for sending
- MINOR: h3: adjust zero-copy sending related code
- MINOR: hq-interop: use zero-copy to transfer single HTX data block
- BUG/MEDIUM: quic: QUIC CID removed from tree without locking
- BUG/MEDIUM: stconn: Block zero-copy forwarding if EOS/ERROR on consumer side
- BUG/MEDIUM: mux-h1: Cound data from input buf during zero-copy forwarding
- BUG/MEDIUM: mux-h1: Explicitly skip request's C-L header if not set originally
- CLEANUP: mux-h1: Fix a trace message about C-L header addition
- BUG/MEDIUM: mux-h2: Report too large HEADERS frame only when rxbuf is empty
- BUG/MEDIUM: mux-quic: report early error on stream
- DOC: config: add arguments to sample fetch methods in the table
- DOC: config: also add arguments to the converters in the table
- BUG/MINOR: resolvers: default resolvers fails when network not configured
- SCRIPTS: mk-patch-list: produce a list of patches
- DEV: patchbot: add the AI-based bot to pre-select candidate patches to backport
- BUG/MEDIUM: mux-h2: Switch pending error to error if demux buffer is empty
- BUG/MEDIUM: mux-h2: Only Report H2C error on read error if demux buffer is empty
- BUG/MEDIUM: mux-h2: Don't report error on SE if error is only pending on H2C
- BUG/MEDIUM: mux-h2: Don't report error on SE for closed H2 streams
- DOC: config: Update documentation about local haproxy response
- DEV: patchbot: use checked buttons as reference instead of internal table
- DEV: patchbot: allow to show/hide backported patches
- MINOR: h3: remove quic_conn only reference
- BUG/MINOR: server: Use the configured address family for the initial resolution
- MINOR: mux-quic: remove qcc_shutdown() from qcc_release()
- MINOR: mux-quic: use qcc_release in case of init failure
- MINOR: mux-quic: adjust error code in init failure
- MINOR: h3: add traces for connection init stage
- BUG/MINOR: h3: properly handle alloc failure on finalize
- MINOR: h3: use INTERNAL_ERROR code for init failure
- BUG/MAJOR: stconn: Disable zero-copy forwarding if consumer is shut or in error
- MINOR: stats: store the parent proxy in stats ctx (http)
- BUG/MEDIUM: stats: unhandled switching rules with TCP frontend
- MEDIUM: proxy: set PR_O_HTTP_UPG on implicit upgrades
- MINOR: proxy: monitor-uri works with tcp->http upgrades
- OPTIM: server: eb lookup for server_find_by_name()
- OPTIM: server: ebtree lookups for findserver_unique_* functions
- MINOR: server/event_hdl: add server_inetaddr struct to facilitate event data usage
- MINOR: server/event_hdl: update _srv_event_hdl_prepare_inetaddr prototype
- BUG/MINOR: server/event_hdl: propagate map port info through inetaddr event
- MINOR: server: ensure connection cleanup on server addr changes
- CLEANUP: server/event_hdl: remove purge_conn hint in INETADDR event
- MEDIUM: server: merge srv_update_addr() and srv_update_addr_port() logic
- CLEANUP: server: remove unused server_parse_addr_change_request() function
- CLEANUP: resolvers: remove duplicate func prototype
- MINOR: resolvers: add unique numeric id to nameservers
- MEDIUM: server: make server_set_inetaddr() updater serializable
- MINOR: server/event_hdl: expose updater info through INETADDR event
- MINOR: server: add dns hint in server_inetaddr_updater struct
- MEDIUM: server/dns: clear RMAINT when addr resolves again
- BUG/MINOR: server/dns: use server_set_inetaddr() to unset srv addr from DNS
- BUG/MEDIUM: server/dns: perform svc_port updates atomically from SRV records
- MEDIUM: peers: use server as stream target
- CLEANUP: peers: remove unused sock_init_arg struct member
- CLEANUP: peers: remove unused "proto" and "xprt" struct members
- MINOR: peers: rely on srv->addr and remove peer->addr
- DOC: config: add context hint for server keywords
- MINOR: stktable: add table_process_entry helper function
- MINOR: stktable: use {show,set,clear} table with ptr
- MINOR: map: add map_*_key converters to provide the matching key
- DOC: fix typo for fastfwd QUIC option
- BUG/MINOR: mux-quic: always report error to SC on RESET_STREAM emission
- MEDIUM: mux-quic: add BUG_ON if sending on locally closed QCS
- BUG/MINOR: mux-quic: disable fast-fwd if connection on error
- BUG/MINOR: quic: Wrong keylog callback setting.
- BUG/MINOR: quic: Missing call to TLS message callbacks
- MINOR: h3: check connection error during sending
- BUG/MINOR: h3: close connection on header list too big
- BUG/MINOR: h3: close connection on sending alloc errors
- BUG/MINOR: h3: disable fast-forward on buffer alloc failure
- Revert "MINOR: mux-quic: Disable zero-copy forwarding for send by default"
- MINOR: stktable: stktable_data_ptr() cannot fail in table_process_entry()
- CLEANUP: assorted typo fixes in the code and comments
- CI: use semantic version compare for determing "latest" OpenSSL
- CLEANUP: server: remove ambiguous check in srv_update_addr_port()
- CLEANUP: resolvers: remove unused RSLV_UPD_OBSOLETE_IP flag
- CLEANUP: resolvers: remove some more unused RSLV_UDP flags
- MEDIUM: server: simplify snr_set_srv_down() to prevent confusions
- MINOR: backend: export get_server_*() functions
- MINOR: tcpcheck: export proxy_parse_tcpcheck()
- MEDIUM: udp: allow to retrieve the frontend destination address
- MINOR: global: export a way to list build options
- MINOR: debug: add features and build options to "show dev"
- BUG/MINOR: server: fix server_find_by_name() usage during parsing
- REGTESTS: check attach-srv out of order declaration
- CLEANUP: quic: Remaining useless code into server part
- BUILD: quic: Missing quic_ssl.h header protection
- BUG/MEDIUM: h3: fix incorrect snd_buf return value
- MINOR: h3: do not consider missing buf room as error on trailers
- BUG/MEDIUM: stconn: Forward shutdown on write timeout only if it is forwardable
- BUG/MEDIUM: stconn: Set fsb date if zero-copy forwarding is blocked during nego
- BUG/MEDIUM: spoe: Never create new spoe applet if there is no server up
- MINOR: mux-h2: support limiting the total number of H2 streams per connection
- CLEANUP: mux-h2: remove the printfs from previous commit on h2 streams limit.
- DEV: h2: add the ability to emit literals in mkhdr
- DEV: h2: add the preface as well in supported output types
- DEV: h2: support passing raw data for a frame
- IMPORT: ebtree: implement and use flsnz_long() to count bits
- IMPORT: ebtree: switch the sizes and offsets to size_t and ssize_t
- IMPORT: ebtree: rework the fls macros to better deal with arch-specific ones
- IMPORT: ebtree: make string_equal_bits turn back to unsigned char
- IMPORT: ebtree: use unsigned ints for flznz()
- IMPORT: ebtree: make string_equal_bits() return an unsigned
This patch introduces a new setting: tune.h2.fe.max-total-streams. It
sets the HTTP/2 maximum number of total streams processed per incoming
connection. Once this limit is reached, HAProxy will send a graceful GOAWAY
frame informing the client that it will close the connection after all
pending streams have been closed. In practice, clients tend to close as fast
as possible when receiving this, and to establish a new connection for next
requests. Doing this is sometimes useful and desired in situations where
clients stay connected for a very long time and cause some imbalance inside a
farm. For example, in some highly dynamic environments, it is possible that
new load balancers are instantiated on the fly to adapt to a load increase,
and that once the load goes down they should be stopped without breaking
established connections. By setting a limit here, the connections will have
a limited lifetime and will be frequently renewed, with some possibly being
established to other nodes, so that existing resources are quickly released.
The default value is zero, which enforces no limit beyond those implied by
the protocol (2^30 ~= 1.07 billion). Values around 1000 were found to
already cause frequent enough connection renewal without causing any
perceptible latency to most clients. One notable exception here is h2load
which reports errors for all requests that were expected to be sent over
a given connection after it receives a GOAWAY. This is an already known
limitation: https://github.com/nghttp2/nghttp2/issues/981
The patch was made in two parts inside h2_frt_handle_headers():
- the first one, at the end of the function, which verifies if the
configured limit was reached and if it's needed to emit a GOAWAY ;
- the second, just before decoding the stream frame, which verifies if
a previously configured limit was ignored by the client, and closes
the connection if this happens. Indeed, one reason for a connection
to stay alive for too long definitely comes from a stupid bot that
periodically fetches the same resource, scans lots of URLs or tries
to brute-force something. These ones are more likely to just ignore
the last stream ID advertised in GOAWAY than a regular browser, or
a well-behaving client such as curl which respects it. So in order
to make sure we can close the connection we need to enforce the
advertised limit.
Note that a regular client will not face a problem with that because in
the worst case it will have max_concurrent_streams in flight and this
limit is taken into account when calculating the advertised last
acceptable stream ID.
Just a note: it may also be possible to move the first part above to
h2s_frt_stream_new() instead so that it's not processed for trailers,
though it doesn't seem to be more interesting, first because it has
two return points.
This is something that may be backported to 2.9 and 2.8 to offer more
control to those dealing with dynamic infrastructures, especially since
for now we cannot force a connection to be cleanly closed using rules
(e.g. github issues #946, #2146).
This reverts commit 18f2ccd244.
Found issues related to QUIC fast-forward were resolved (see github
issue #2372). Reenable it by default. If any issue arises, it can be
disabled using the global statement :
tune.quit.zero-copy-fwd-send off
This can be backported to 2.9, but only after a sensible period of
observation.
All map_*_ converters now have an additional output type: key. Such
converters will return the matched entry's key (as found in the map file)
as a string instead of the value.
Consider this example map file:
|example.com value1
|haproxy value2
With the above map file:
str(test.example.com/url),map_dom_key(file.map) will return "example.com"
str(running haproxy),map_sub_key(file.map) will return "haproxy"
This should address GH #1446.
Documentation about 'L' state in the termination state was outdated. Today,
not only the request may be intercepted, but also the response.
Documentation about 'L' must be more generic.
However, documentation about possible 2-letter termination states was also
extended to add 'LC' and 'LH' in the list. And 'LR' was adapted too.
This patch should fix the issue #2384. It may be backported to every stable
versions. Note that on 2.8 and lowers, we talk about session and not stream.
rep_ssl_hello_type was renamed in res.ssl_hello_type a long time ago.
This patch fixes a typo where an example was renamed
"rep.ssl_hello_type" instead of "res.ssl_hello_type"
fixes issue #2377 and #2379.
Must be backported in all maintained versions.
Because maps and list of ACLs are no longer necessarily referenced by
filenames, CLI commands to manipulate them were updated accordingly. Instead
of "filename" we talk about "name" now.
The same is performed in the LUA documentation.
Maps and list of ACLs can now reference something else than regular files
and can have prefix to set the type of the list (file, virutal file or
optional file). So, the configuration manual was updated accordingly.
The section 2.7. about name format for maps and ACLs was added (the former
2.7. sections with some examples was moved to 2.8.) and references to map or
ACLs files were updated.
tune.cache.zero-copy-forwarding parameter can now be used to enable or
disable the zero-copy fast-forwarding for the cache applet only. It is
enabled ('on') by default. It can be disabled by setting the parameter to
'off'.
Released version 2.9.0 with the following main changes :
- DOC: config: add missing colon to "bytes_out" sample fetch keyword (2)
- BUG/MINOR: cfgparse-listen: fix warning being reported as an alert
- DOC: config: add matrix entry for "max-session-srv-conns"
- DOC: config: fix monitor-fail typo
- DOC: config: add context hint for proxy keywords
- DEBUG: stream: Report lra/fsb values for front end back SC in stream dump
- REGTESTS: sample: Test the behavior of consecutive delimiters for the field converter
- BUG/MINOR: sample: Make the `word` converter compatible with `-m found`
- DOC: Clarify the differences between field() and word()
- BUG/MINOR: server/event_hdl: properly handle AF_UNSPEC for INETADDR event
- BUILD: http_htx: silence uninitialized warning on some gcc versions
- MINOR: acme.sh: don't use '*' in the filename for wildcard domain
- MINOR: global: Use a dedicated bitfield to customize zero-copy fast-forwarding
- MINOR: mux-pt: Add global option to enable/disable zero-copy forwarding
- MINOR: mux-h1: Add global option to enable/disable zero-copy forwarding
- MINOR: mux-h2: Add global option to enable/disable zero-copy forwarding
- MINOR: mux-quic: Add global option to enable/disable zero-copy forwarding
- MINOR: mux-quic: Disable zero-copy forwarding for send by default
- DOC: config: update the reminder on the HTTP model and add some terminology
- DOC: config: add a few more differences between HTTP/1 and 2+
- DOC: config: clarify session vs stream
- DOC: config: fix typo abandonned -> abandoned
- DOC: management: fix two latest typos (optionally, exception)
- BUG/MEDIUM: peers: fix partial message decoding
- DOC: management: update stream vs session
No backport needed, these were introduced by latest commits 3dd55fa13
("MINOR: mworker/cli: implement hard-reload over the master CLI") and
cef29d370 ("MINOR: trace: define simple -dt argument").
It was really necessary to try to clear the confusion between sessions
and streams, so let's first lift a little bit the HTTP model part to
better consider new protocols, and explain what a stream is and how this
differs from the earlier sessions.
There is at least an bug for now in this part and it is still unstable. Thus
it is better to disable it for now by default. It can be enable by setting
tune.quic.zero-copy-fwd-send to 'on'.
tune.quic.zero-copy-fwd-send can now be used to enable or disable the
zero-copy fast-forwarding for the QUIC mux only, for sends. For now, there
is no option to disable it for receives because it is not supported yet.
It is enabled ('on') by default.
tune.h2.zero-copy-fwd-send can now be used to enable or disable the
zero-copy fast-forwarding for the H2 mux only, for sends. For now, there is
no option to disable it for receives because it is not supported yet.
It is enabled ('on') by default.
tune.h1.zero-copy-fwd-recv and tune.h1.zero-copy-fwd-send can now be used to
enable or disable the zero-copy fast-forwarding for the H1 mux only, for
receives or sends. Unlike the PT mux, there are 2 options here because
client and server sides can use difference muxes.
Both are enabled ('on') by default.
tune.pt.zero-copy-forwarding parameter can now be used to enable or disable
the zero-copy fast-forwarding for the PT mux only. It is enabled ('on') by
default. It can be disabled by setting the parameter to 'off'. In this case,
this disables receive and send side.
Zero-copy fast-forwading feature is a quite new and is a bit sensitive.
There is an option to disable it globally. However, all protocols have not
the same maturity. For instance, for the PT multiplexer, there is nothing
really new. The zero-copy fast-forwading is only another name for the kernel
splicing. However, for the QUIC/H3, it is pretty new, not really optimized
and it will evolved. And soon, the support will be added for the cache
applet.
In this context, it is usefull to be able to enable/disable zero-copy
fast-forwading per-protocol and applet. And when it is applicable, on sends
or receives separately. So, instead of having one flag to disable it
globally, there is now a dedicated bitfield, global.tune.no_zero_copy_fwd.
word() mentions that delimiters at the start and end are ignored, but
it does not mention that consecutive delimiters are merged.
May be backported as far as the patch applies.
Add a small list of contexts where each proxy keyword is expected to be
employed. (Similar to the defaults/frontend/backend/listen compatibility
grid).
Following 4039329 ("DOC: config: specify supported sections for
"max-session-srv-conns"), "max-session-srv-conns" was also missing
from the proxy keyword matrix.
Released version 2.9-dev12 with the following main changes :
- BUG/MINOR: global: Fix tune.disable-(fast-forward/zero-copy-forwarding) options
- DOC: config: removing "log-balance" references
- MINOR: server/event_hdl: add SERVER_INETADDR event
- MINOR: tools: use const for read only pointers in ip{cmp,cpy}
- MINOR: server/ip: centralize server ip updates
- MINOR: backend: remove invalid mode test for "hash-balance-factor"
- Revert "MINOR: cfgparse-listen: warn when use-server rules is used in wrong mode"
- MINOR: proxy: add free_logformat_list() helper function
- MINOR: proxy: add free_server_rules() helper function
- MINOR: log/backend: prevent "use-server" rules use with LOG mode
- MINOR: log/balance: set lbprm tot_weight on server on queue/dequeue
- DOC: config: specify supported sections for "max-session-srv-conns"
- DOC: config: fix timeout check inheritance restrictions
- REGTESTS: connection: disable http_reuse_be_transparent.vtc if !TPROXY
- DOC: lua: add sticktable class reference from Proxy.stktable
- DOC: lua: fix Proxy.get_mode() output
- DOC: lua: add "syslog" to Proxy.get_mode() output
- MEDIUM: ssl: implement rsa/ecdsa selection with WolfSSL
- MINOR: ssl: replace 'trash.area' by 'servername' in ssl_sock_switchctx_cbk()
- MINOR: ssl: move certificate selection in a dedicate function
- MEDIUM: ssl: use ssl_sock_chose_sni_ctx() in the clienthello callback
- MINOR: mworker/cli: implement hard-reload over the master CLI
- BUG/MEDIUM: mux-h1: Properly ignore trailers when a content-length is announced
- MINOR: task/profiling: do not record task_drop_running() as a caller
- OPTIM: pattern: save memory and time using ebst instead of ebis
- BUILD: map: fix build warning
- MINOR: trace: define simple -dt argument
- MINOR: trace: parse level in a function
- MINOR: trace: parse verbosity in a function
- MINOR: trace: support -dt optional format
- OPTIM: mux-h2/zero-copy: don't allocate more buffers per connections than streams
- BUG/MINOR: quic: fix CONNECTION_CLOSE_APP encoding
- BUG/MEDIUM: stconn: Don't perform zero-copy FF if opposite SC is blocked
- BUG/MEDIUM: mux-h2: Remove H2_SF_NOTIFIED flag for H2S blocked on fast-forward
- CLEANUP: quic: Remove dead definitions/declarations
- REORG: quic: Move some QUIC CLI code to its C file
- REORG: quic: Add a new module to handle QUIC connection IDs
- REORG: quic: QUIC connection types header cleaning
- BUILD: quic: Missing RX header inclusions
- REORG: quic: Move CRYPTO data buffer defintions to QUIC TLS module
- REORG: quic: Move QUIC CRYPTO stream definitions/declarations to QUIC TLS
- REORG: quic: Move several inlined functions from quic_conn.h
- REORG: quic: Move QUIC SSL BIO method related functions to quic_ssl.c
- REORG: quic: Move the QUIC DCID parser to quic_sock.c
- REORG: quic: Rename some functions used upon ACK receipt
- REORG: quic: Move QUIC path definitions/declarations to quic_cc module
- REORG: quic: Move qc_handle_conn_migration() to quic_conn.c
- REORG: quic: Move quic_build_post_handshake_frames() to quic_conn module
- REORG: quic: Move qc_may_probe_ipktns() to quic_tls.h
- REORG: quic: Move qc_pkt_long() to quic_rx.h
- REORG: quic: Rename some (quic|qc)_conn* objects to quic_conn_closed
- REORG: quic: Move NEW_CONNECTION_ID frame builder to quic_cid
- REORG: quic: Move ncbuf related function from quic_rx to quic_conn
- REORG: quic: Add a new module for QUIC retry
- BUILD: quic: Several compiler warns fixes after retry module creation
- REORG: quic: Move qc_notify_send() to quic_conn
- REORG: quic: Add a new module for retransmissions
- REORG: quic: Remove qc_pkt_insert() implementation
- REORG: quic: Move quic_increment_curr_handshake() to quic_sock
- BUG/MINOR: cache: Remove incomplete entries from the cache when stream is closed
- MEDIUM: cli: allow custom pattern for payload
- CLEANUP: mworker/cli: use a label to return errors
- MINOR: mworker/cli: implements the customized payload pattern for master CLI
- DOC: management: add documentation about customized payload pattern
- BUG/MEDIUM: server/event_hdl: memory overrun in _srv_event_hdl_prepare_inetaddr()
- MINOR: event_hdl: add global tunables
- BUG/MAJOR: server/addr: fix a race during server addr:svc_port updates
- MEDIUM: log/balance: support FQDN for UDP log servers
- BUG/MINOR: compression: possible NULL dereferences in comp_prepare_compress_request()
- BUG/MEDIUM: master/cli: Properly pin the master CLI on thread 1 / group 1
- BUG/MEDIUM: mux-quic: Stop zero-copy FF during nego if input is not empty
- CLEANUP: log: Fix %rc comment in sess_build_logline()
- BUG/MINOR: h3: fix TRAILERS encoding
- BUG/MINOR: h3: always reject PUSH_PROMISE
- MINOR: h3: use correct error code for missing SETTINGS
- MINOR: http-fetch: Add a sample to retrieve the server status code
- DOC: config: Improve 'status' sample documentation
- MINOR: http-fetch: Add a sample to get the transaction status code
- MEDIUM: http-ana: Set termination state before returning haproxy response
- MINOR: stream: Expose session terminate state via a new sample fetch
- MINOR: stream: add a sample fetch to get the number of connection retries
- MINOR: stream: Expose the stream's uniq_id via a new sample fetch
- MINOR: muxes: Rename mux_ctl_type values to use MUX_CTL_ prefix
- MINOR: muxes: Add a callback function to send commands to mux streams
- MINOR: muxes: Implement ->sctl() callback for muxes and return the stream id
- MINOR: Add sample fetches to get the frontend and backend stream ID
- BUG/MEDIUM: cli: Don't look for payload pattern on empty commands
- DOC: config: Add argument for tune.lua.maxmem
- DOC: config: fix mention of request slot in http-response capture
- DOC: config: fix remaining mention of @reverse for attach-srv action
- DOC: config: fix missing characters in set-spoe-group action
- DOC: config: reorganize actions into their own section
- BUG/MINOR: acme.sh: update the deploy script
- MINOR: rhttp: mark reverse HTTP as experimental
- CLEANUP: quic_cid: remove unused listener arg
- BUG/MINOR: quic_tp: fix preferred_address decoding
- MINOR: quic_tp: use in_addr/in6_addr for preferred_address
- MINOR: acme.sh: use the master CLI for hot update
- DOC: config: move the cache-use and cache-store actions to the proper section
- DOC: config: fix alphabetical ordering of converter keywords
- DOC: config: add missing colon to "bytes_out" sample fetch keyword
- DOC: config: add an index of converter keywords
- DOC: config: add an index of sample fetch keywords
- BUG/MINOR: config: Stopped parsing upon unmatched environment variables
- DEBUG: unstatify a few functions that are often present in backtraces
- BUILD: server: shut a bogus gcc warning on certain ubuntu
Now we're adding a table for each section, it allows to more easily
spot the list of available sample fetch functions and their types.
For now the arguments are not mentioned in the table because they'd
break indexing but they can be added back later.
- rfc7239_* were misplaced and incorrectly ordered
- table_gpt was placed before some table_gpc*
- capture-req/res were misplaced
- htonl was misplaced
- upper/url_* were misplaced
- x509_v_err_str was misplaced
Let's fix these since poor ordering complicates their finding.
Actions were grouped by previous commit d54e8f810 ("DOC: config: reorganize
actions into their own section") but cache-use and cache-store were still
making reference to the cache section. This moves the text back to their
respective keywords in the actions section and leaves the example and an
explanation of how to use the keywords in the cache section.
Mark the reverse HTTP feature as experimental. This will allow to adjust
if needed the configuration mechanism with future developments without
maintaining retro-compatibility.
Concretely, each config directives linked to it now requires to specify
first global expose-experimental-directives before. This is the case for
the following directives :
- rhttp@ prefix uses in bind and server lines
- nbconn bind keyword
- attach-srv tcp rule
Each documentation section refering to these keywords are updated to
highlight this new requirement.
Note that this commit has duplicated on several places the code from the
global function check_kw_experimental(). This is because the latter only
work with cfg_keyword type. This is not adapted with bind_kw or
action_kw types. This should be improve in a future patch.
The split of the rulesets from their respective actions has long been
overdue so it's time to do it because it has become extremely difficult
to add simple actions in the documentation, as well as it's hard to find
them.
This commit creates two new sections "4.3 Actions keywords matrix" and
"4.4 Alphabetically sorted actions reference" that enumerates all known
actions, with a check indicating for which rule sets they're valid. This
removes all the repetition, occurrences of "see http-request blah for
details" and significantly reduces the number of keywords listed in the
proxies section. This removes 2245 lines from the proxies section in
exchange of 1608 in these new sections.
"fc.id" and "bc.id" sample fetches can now be used to get, respectively, the
frontend or the backend stream ID. They rely on ->sctl() callback function
on the mux attached to the corresponding SC.
It means these sample fetches work only for connection, not applets, and
from the time a multiplexer is installed.
"txn.conn_retries" can now be used to get the number of connection
retries. This value is only stable once the connection is fully
established. For HTTP sessions, L7-retries must also be passed.
It is now possible to retrieve the session terminate state, using
"txn.sess_term_state". The sample fetch returns the 2-character session
termation state.
Of course, the result of this sample fetch is volatile. It is subject to
change. It is also most of time useless because no termation state is set
except at the end. It should only be useful in http-after-response rule
sets. It may also be used to customize the logs using a log-format
directive.
This patch should fix the issue #2221.
It was possible get the status code in the HTTP response and the one
received from the server. Thanks to 'txn.status', it is now possible to get
the transaction status code. It is equivalent to '%ST' in log-format.
Most of time, it is the same than 'status', except if the status code of the
HTTP reply does not match the one used to interrupt the transaction. For
instance, an error file use mapped on 400 containing a 404.
We clearly state the 'status' sample returns the status code the client will
receive, if no change happens on the HTTP response. This should avoid
ambiguities with the 'server-status' sample fetch.
The code returned by the "status" sample fetch is the one in the HTTP
response at the moment the sample is evaluated. It may be the status code in
the server response or the one of the HAProxy reply in case of error, deny,
redirect...
However, it could be handy to retrieve the status code returned by the
server, when a HTTP response was really received from it. It is the purpose
of the "server_status" sample fetch. The server status code itself is stored
in the HTTP txn.
The local variable "event_hdl_async_max_notif_at_once" which was
introduced with the event_hdl API was left as is but with a TODO note
telling that we should make it a global tunable.
Well, we're doing this now. To prepare for upcoming tunables related to
event_hdl API, we add a dedicated struct named event_hdl_tune which is
globally exposed through the event_hdl header file so that it may be used
from everywhere. The struct is automatically initialized in
event_hdl_init() according to defaults.h.
"event_hdl_async_max_notif_at_once" now becomes
"event_hdl_tune.max_events_at_once" with it's dedicated
configuation keyword: "tune.events.max-events-at-once".
We're also taking this opportunity to raise the default value from 10
to 100 since it's seems quite reasonnable given existing async event_hdl
users.
The documentation was updated accordingly.
Add an optional argument for "-dt". This argument is interpreted as a
list of several trace statement separated by comma. For each statement,
a specific trace name can be specifed, or none to act on all sources.
Using double-colon separator, it is possible to add specifications on
the wanted level and verbosity.
Add '-dt' haproxy process argument. This will automatically activate all
trace sources on stderr with the error level. This could be useful to
troubleshoot issues such as protocol violations.
The mworker mode never had a proper 'hard-stop' (-st) for the reload,
this is a mode which was commonly used with the daemon mode, but it was
never implemented in mworker mode.
This patch fixes the problem by implementing a "hard-reload" command
over the master CLI. It does the same as the "reload" command, but
instead of waiting for the connections to stop in the previous process,
it immediately quits the previous process after binding.
Following previous commit: in this patch we add the "syslog" output as
possible return value for Proxy.get_mode() function since log backend
may now be enumerated from lua with 9a74a6c ("MAJOR: log: introduce log
backends")
Proxy.get_mode() function internally relies on proxy_mode_str() to return
the proxy mode. The current function description is exhaustive about the
possible outputs for the function. I can't tell if it's relevant or not
but it's subject to changes. Here it is the case, the documentation
indicates that "health" mode may be returned, which cannot happen
since 77e0daef9 ("MEDIUM: proxy: remove obsolete "mode health"").
This should be backported up to 2.4
In 6e0425b718 ("DOC: config: Add documentation about TCP/HTTP rules in
defaults section") an error was made: the restriction note about the
setting not being inherited from anonymous default section was added
by mistake in the "timeout check" documentation. But it is wrong,
"timeout check" behaves like other "timeout" directives for proxy
sections.
This should be backported up to 2.6.
There was no info about supported sections for "max-session-srv-conns"
proxy directive. A quick look at the code tells us that it may be used
in proxies with the FE capability set.
"log-balance" keyword was removed by b61147f ("MEDIUM: log/balance: merge
tcp/http algo with log ones") but it was still documented.
Removing "log-balance" references in the documentation where needed.
Released version 2.9-dev11 with the following main changes :
- BUG/MINOR: startup: set GTUNE_SOCKET_TRANSFER correctly
- BUG/MINOR: sock: mark abns sockets as non-suspendable and always unbind them
- BUILD: cache: fix build error on older compilers
- BUG/MAJOR: quic: complete thread migration before tcp-rules
- BUG/MEDIUM: quic: Possible crash for connections to be killed
- MINOR: quic: remove unneeded QUIC specific stopping function
- MINOR: acl: define explicit HTTP_3.0
- DEBUG: connection/flags: update flags for reverse HTTP
- BUILD: log: silence a build warning when threads are disabled
- MINOR: quic: Add traces to debug frames handling during retransmissions
- BUG/MEDIUM: quic: Possible crash during retransmissions and heavy load
- BUG/MINOR: quic: Possible leak of TX packets under heavy load
- BUG/MINOR: quic: Possible RX packet memory leak under heavy load
- BUG/MINOR: server: do not leak default-server in defaults sections
- DEBUG: tinfo: store the pthread ID and the stack pointer in tinfo
- MINOR: debug: start to create a new struct post_mortem
- MINOR: debug: add OS/hardware info to the post_mortem struct
- MINOR: debug: report in port_mortem whether a container was detected
- MINOR: debug: report in post_mortem if the container techno used is docker
- MINOR: debug: detect CPU model and store it in post_mortem
- MINOR: debug: report any detected hypervisor in post_mortem
- MINOR: debug: collect some boot-time info related to the process
- MINOR: debug: copy the thread info into the post_mortem struct
- MINOR: debug: dump the mapping of the libs into post_mortem
- MINOR: debug: add the ability to enter components in the post_mortem struct
- MINOR: init: add info about the main program to the post_mortem struct
- DOC: management: document "show dev"
- CLEANUP: assorted typo fixes in the code and comments
- CI: limit codespell checks to main repo, not forks
- DOC: 51d: updated 51Degrees repo URL for v3.2.10
- DOC: install: update the list of openssl versions
- MINOR: ext-check: add an option to preserve environment variables
- BUG/MEDIUM: mux-h1: Don't set CO_SFL_MSG_MORE flag on last fast-forward send
- MINOR: rhttp: rename proto_reverse_connect
- MINOR: rhttp: large renaming to use rhttp prefix
- MINOR: rhttp: add count of active conns per thread
- MEDIUM: rhttp: support multi-thread active connect
- MINOR: listener: allow thread kw for rhttp bind
- DOC: rhttp: replace maxconn by nbconn
- MINOR: log/balance: rename "log-sticky" to "sticky"
- MEDIUM: mux-quic: Add consumer-side fast-forwarding support
- MAJOR: h3: Implement zero-copy support to send DATA frame
After giving it some thought, it could pretty well happen that other
protocols benefit from the sticky algorithm that some used to emulate
using a "stick-on int(0)" or things like this previously. So better
rename it to "sticky" right now instead of having to keep that "log-"
prefix forever. It's still limited to logs, of course, only the algo
is renamed in the config.
Usage of existing "maxconn" for rhttp listeners configuration was
replaced recently by a new dedicating "nbconn" keyword. Update the
documentation part to reflect this.
No need to backport.
Thanks to previous commit, a reverse HTTP listener is able to distribute
actively opened connections accross its threads. To be able to exploit
this, allow "thread" keyword for such a listener.
An extra check is added to explicitely forbids a reverse bind to span
multiple thread groups. Without this, multiple listeners instances will
be created, each with its owned "nbconn" value. This may surprise users
so for now, better to deactivate this possibility.
In Github issue #2128, @jvincze84 explained the complexity of using
external checks in some advanced setups due to the systematic purge of
environment variables, and expressed the desire to preserve the
existing environment. During the discussion an agreement was found
around having an option to "external-check" to do that and that
solution was tested and confirmed to work by user @nyxi.
This patch just cleans this up, implements the option as
"preserve-env" and documents it. The default behavior does not change,
the environment is still purged, unless "preserve-env" is passed. The
choice of not using "import-env" instead was made so that we could
later use it to name specific variables that have to be imported
instead of keeping the whole environment.
The patch is simple enough that it could be backported if needed (and
was in fact tested on 2.6 first).
Released version 2.9-dev10 with the following main changes :
- CLEANUP: Re-apply xalloc_size.cocci (3)
- BUG/MEDIUM: stconn: Report send activity during mux-to-mux fast-forward
- BUG/MEDIUM: stconn: Don't report rcv/snd expiration date if SC cannot epxire
- MINOR: stconn: Don't queue stream task in past in sc_notify()
- BUG/MEDIUM: Don't apply a max value on room_needed in sc_need_room()
- BUG/MINOR: stconn: Sanitize report for read activity
- CLEANUP: htx: Properly indent htx_reserve_max_data() function
- DOC: stconn: Improve comments about lra and fsb usage
- BUG/MEDIUM: quic: fix actconn on quic_conn alloc failure
- BUG/MEDIUM: quic: fix sslconns on quic_conn alloc failure
- BUG/MEDIUM: mux-h1: Be sure xprt support splicing to use it during fast-forward
- MINOR: proto_reverse_connect: use connect timeout
- BUG/MINOR: mux-h1: Release empty ibuf during data fast-forwarding
- BUG/MINOR: stick-table/cli: Check for invalid ipv4 key
- MEDIUM: stktable/cli: simplify entry key handling
- MINOR: stktable/cli: support v6tov4 and v4tov6 conversions
- BUG/MINOR: mux-h1: Properly handle http-request and http-keep-alive timeouts
- BUG/MEDIUM: freq-ctr: Don't report overshoot for long inactivity period
- BUG/MEDIUM: pool: fix releasable pool calculation when overloaded
- BUG/MINOR: pool: check one other random bucket on alloc conflict
- BUG/MEDIUM: pool: try once to allocate from another bucket if empty
- MEDIUM: stconn/muxes: Loop on data fast-forwarding to forward at least a buffer
- MINOR: stconn/mux-h2: Use a iobuf flag to report EOI to consumer side during FF
- MEDIUM: quic: Heavy task mode during handshake
- MEDIUM: quic: Heavy task mode with non contiguously bufferized CRYPTO data
- MINOR: quic: release the TLS context asap from quic_conn_release()
- MINOR: quic: Add idle timer task pointer to traces
- BUG/MINOR: quic: idle timer task requeued in the past
- CLEANUP: quic: Indentation fix in qc_do_build_pkt()
- MINOR: quic: Avoid zeroing frame structures
- BUG/MEDIUM: quic: Too short Initial packet sent (enc. level allocation failed)
- BUG/MEDIUM: quic: Avoid trying to send ACK frames from an empty ack ranges tree
- BUG/MEDIUM: quic: Possible crashes when sending too short Initial packets
- BUG/MEDIUM: quic: Avoid some crashes upon TX packet allocation failures
- BUG/MEDIUM: quic: Possible crashes during secrets allocations (heavy load)
- BUG/MEDIUM: stconn: Don't update stream expiration date if already expired
- MINOR: errors: ha_alert() and ha_warning() uses warn_exec_path()
- MINOR: errors: does not check MODE_STARTING for log emission
- MEDIUM: errors: move the MODE_QUIET test in print_message()
- DOC: management: -q is quiet all the time
- MEDIUM: mworker: -W is mandatory when using -S
- BUG/MEDIUM: mux-h1: Exit early if fast-forward is not supported by opposite SC
- MEDIUM: quic: adjust address validation
- MINOR: quic: reduce half open counters scope
- MEDIUM: quic: limit handshake per listener
- MEDIUM: quic: define an accept queue limit
- BUG/MINOR: quic: fix retry token check inconsistency
- MINOR: task/debug: explicitly support passing a null caller to wakeup functions
- MINOR: task/debug: make task_queue() and task_schedule() possible callers
- OPTIM: mux-h2: don't allocate more buffers per connections than streams
- BUG/MINOR: quic: remove dead code in error path
- MEDIUM: quic: respect closing state even on soft-stop
- MEDIUM: quic: release conn socket before using quic_cc_conn
- DOC: config: use the word 'backend' instead of 'proxy' in 'track' description
- BUG/MEDIUM: applet: Remove appctx from buffer wait list on release
- MINOR: tools: make str2sa_range() directly return type hints
- BUG/MEDIUM: server: invalid address (post)parsing checks
- BUG/MINOR: sink: don't learn srv port from srv addr
- CLEANUP: sink: bad indent in sink_new_from_logger()
- CLEANUP: sink: useless leftover in sink_add_srv()
- BUG/MINOR: quic: Useless use of non-contiguous buffer for in order CRYPTO data
- MINOR: server: always initialize pp_tlvs for default servers
- BUG/MEDIUM: proxy: always initialize the default settings after init
- MEDIUM: startup: 'haproxy -c' is quiet when valid
- BUG/MINOR: sample: Fix bytes converter if offset is bigger than sample length
- BUG/MINOR: log: keep the ref in dup_logger()
- BUG/MINOR: quic: fix crash on qc_new_conn alloc failure
- BUG/MINOR: quic: fix decrement of half_open counter on qc alloc failure
- BUG/MEDIUM: quic: fix FD for quic_cc_conn
- DOC: config: Fix name for tune.disable-zero-copy-forwarding global param
- REGTESTS: startup: -conf-OK requires -V with current VTest
- BUG/MEDIUM: quic: Non initialized CRYPTO data stream deferencing
- MINOR: quic: Add a max window parameter to congestion control algorithms
- MINOR: quic: Maximum congestion control window for each algo
- DOC: quic: Wrong syntax for "quic-cc-algo" keyword.
- DOC: quic: Maximum congestion control window configuration
- BUG/MINOR: quic: maximum window limits do not match the doc
- BUG/MEDIUM: connection: report connection errors even when no mux is installed
- BUG/MINOR: stconn: Handle abortonclose if backend connection was already set up
- MINOR: connection: Add a CTL flag to notify mux it should wait for reads again
- MEDIUM: mux-h1: Handle MUX_SUBS_RECV flag in h1_ctl() and susbscribe for reads
- BUG/MEDIUM: stream: Properly handle abortonclose when set on backend only
- MINOR: stconn: Use SC to detect frontend connections in sc_conn_recv()
- REGTESTS: http: Improve script testing abortonclose option
- MINOR: activity: report profiling duration and age in "show profiling"
- BUG/MEDIUM: mworker: set the master variable earlier
- BUG/MEDIUM: stream: Don't call mux .ctl() callback if not implemented
- MINOR: connection: update rhttp flags usage
- BUG/MINOR: mux_h2: reject passive reverse conn if error on add to idle
- MINOR: server: force add to idle on reverse
- MINOR: shctx: Set last_append to NULL when reserving block in hot list
- MEDIUM: shctx: Move list between hot and avail list in O(1)
- MEDIUM: shctx: Simplify shctx_row_reserve_hot loop
- MINOR: shctx: Remove explicit 'from' param from shctx_row_data_append
- MEDIUM: cache: Use dedicated cache tree lock alongside shctx lock
- MINOR: cache: Remove expired entry delete in "show cache" command
- MINOR: cache: Add option to avoid removing expired entries in lookup function
- MEDIUM: cache: Use rdlock on cache in cache_use
- MEDIUM: shctx: Remove 'hot' list from shared_context
- MINOR: cache: Use dedicated trash for "show cache" cli command
- MEDIUM: cache: Switch shctx spinlock to rwlock and restrict its scope
- MEDIUM: cache: Add refcount on cache_entry
- MEDIUM: shctx: Descend shctx_lock calls into the shctx_row_reserve_hot
- MINOR: shctx: Add new reserve_finish callback call to shctx_row_reserve_hot
- MAJOR: cache: Delay cache entry delete in reserve_hot function
- MINOR: shctx: Remove redundant arg from free_block callback
- MINOR: shctx: Remove 'use_shared_mem' variable
- DOC: cache: Specify when function expects a cache lock
- BUG/MEDIUM: stconn: Update fsb date on partial sends
- MINOR: htx: Use a macro for overhead induced by HTX
- MINOR: channel: Add functions to get info on buffers and deal with HTX streams
- BUG/MINOR: stconn: Fix streamer detection for HTX streams
- BUG/MINOR: stconn: Use HTX-aware channel's functions to get info on buffer
- BUG/MINOR: stconn/applet: Report send activity only if there was output data
- BUG/MINOR: stconn: Report read activity on non-indep streams for partial sends
- BUG/MINOR: shctx: Remove old HA_SPIN_INIT
- REGTESTS: try to activate again the seamless reload test with the master CLI
- MINOR: proxy: Add "handshake" new timeout (frontend side)
- MEDIUM: quic: Add support for "handshake" timeout setting.
- MINOR: quic: Dump the expiration date of the idle timer task
- BUG/MINOR: quic: Malformed CONNECTION_CLOSE frame
- MEDIUM: session: handshake timeout (TCP)
- DOC: proxy: Add "handshake" timeout documentation.
- MINOR: quic: Rename "handshake" timeout to "client-hs"
- CLEANUP: haproxy: remove old comment from 1.1 from the file header
- BUG/MEDIUM: mux-h2: fail earlier on malloc in takeover()
- BUG/MEDIUM: mux-h1: fail earlier on malloc in takeover()
- BUG/MEDIUM: mux-fcgi: fail earlier on malloc in takeover()
- MINOR: rhttp: remove the unused outgoing connect() function
- MINOR: backend: without ->connect(), allow to pick another thread's connection
- BUG/MINOR: stream/cli: report correct stream age in "show sess"
- MINOR: stream/cli: add an optional "older" filter for "show sess"
- MINOR: stream/cli: add another filter "susp" to "show sess"
- MINOR: stktable: add stktable_deinit function
- BUG/MINOR: proxy/stktable: missing frees on proxy cleanup
- CLEANUP: backend: removing unused LB param
- MEDIUM: lbprm: store algo params on 32bits
- MEDIUM: log/balance: merge tcp/http algo with log ones
- Revert "MINOR: proxy: report a warning for max_ka_queue in proxy_cfg_ensure_no_http()"
- Revert "MINOR: tcp_rules: tcp-{request,response} requires TCP or HTTP mode"
- Revert "MINOR: stktable: "stick" requires TCP or HTTP mode"
- Revert "MINOR: cfgparse-listen: "http-send-name-header" requires TCP or HTTP mode"
- Revert "MINOR: cfgparse-listen: "dynamic-cookie-key" requires TCP or HTTP mode"
- Revert "MINOR: cfgparse-listen: "http-reuse" requires TCP or HTTP mode"
- Revert "MINOR: fcgi-app: "use-fcgi-app" requires TCP or HTTP mode"
- Revert "MINOR: http_htx/errors: prevent the use of some keywords when not in tcp/http mode"
- Revert "MINOR: flt_http_comp: "compression" requires TCP or HTTP mode"
- Revert "MINOR: filter: "filter" requires TCP or HTTP mode"
- MINOR: log/backend: ensure log exclusive params are not used in other modes
- MINOR: log/backend: prevent tcp-{request,response} use with LOG mode
- MINOR: log/backend: prevent stick table and stick rules with LOG mode
- MINOR: log/backend: prevent "http-send-name-header" use with LOG mode
- MINOR: log/backend: prevent "dynamic-cookie-key" use with LOG mode
- REGTESTS: http: add a test to validate chunked responses delivery
"log-balance" directive was recently introduced to configure the
balancing algorithm to use when in a log backend. However, it is
confusing and it causes issues when used in default section.
In this patch, we take another approach: first we remove the
"log-balance" directive, and instead we rely on existing "balance"
directive to configure log load balancing in log backend.
Some algorithms such as roundrobin can be used as-is in a log backend,
and for log-only algorithms, they are implemented as "log-$name" inside
the "backend" directive.
The documentation was updated accordingly.
This one reports streams considered as "suspicious", i.e. those with
no expiration dates or dates in the past, or those without a front
endpoint. More criteria could be added in the future.
It's often needed to be able to refine "show sess" when debugging, and
very often a first glance at old streams is performed, but that's a
difficult task in large dumps, and it takes lots of resources to dump
everything.
This commit adds "older <age>" to "show sess" in order to specify the
minimum age of streams that will be dumped. This should simplify the
identification of blocked ones.
Document the "handshake" timeout new setting available one frontend side.
This should at least be helpful for QUIC client connections to prevent
an attacker from refreshing plenty of connections without completing
the handshake step, leading haproxy to consume memory for nothing.
"disable-" prefix was missing. This param was correctly named in the list of
supported keywords in the global section, but not in the keyword
description.
No backport needed.
User @nwehrman reported in issue #2328 that the used of "proxy" instead
of "backend" in the argument of the "track" server keyword is confusing.
Admittedly, all other places in the doc use "backend/server" instead of
"proxy/server", so let's update it for the sake of consistency.
QUIC connections are pushed manually into a dedicated listener queue
when they are ready to be accepted. This happens after handshake
finalization or on 0-RTT packet reception. Listener is then woken up to
dequeue them with listener_accept().
This patch comptabilizes the number of connections currently stored in
the accept queue. If reaching a certain limit, INITIAL packets are
dropped on reception to prevent further QUIC connections allocation.
This should help to preserve system resources.
This limit is automatically derived from the listener backlog. Half of
its value is reserved for handshakes and the other half for accept
queues. By default, backlog is equal to maxconn which guarantee that
there can't be no more than maxconn connections in handshake or waiting
to be accepted.
Implement a limit per listener for concurrent number of QUIC
connections. When reached, INITIAL packets for new connections are
automatically dropped until the number of handshakes is reduced.
The limit value is automatically based on listener backlog, which itself
defaults to maxconn.
This feature is important to ensure CPU and memory resources are not
consume if too many handshakes attempt are started in parallel.
Special care is taken if a connection is released before handshake
completion. In this case, counter must be decremented. This forces to
ensure that member <qc.state> is set early in qc_new_conn() before any
quic_conn_release() invocation.
The documentation about -q seems wrong, it does not output messages
after the startup, it disables all messages. It was always quiet with
the stdio_quiet() function.
Must be backported in all stable versions.
Released version 2.9-dev9 with the following main changes :
- DOC: internal: filters: fix reference to entities.pdf
- BUG/MINOR: ssl: load correctly @system-ca when ca-base is define
- MINOR: lua: Add flags to configure logging behaviour
- MINOR: lua: change tune.lua.log.stderr default from 'on' to 'auto'
- BUG/MINOR: backend: fix wrong BUG_ON for avail conn
- BUG/MAJOR: backend: fix idle conn crash under low FD
- MINOR: backend: refactor insertion in avail conns tree
- DEBUG: mux-h2/flags: fix list of h2c flags used by the flags decoder
- BUG/MEDIUM: server/log: "mode log" after server keyword causes crash
- MINOR: connection: add conn_pr_mode_to_proto_mode() helper func
- BUG/MEDIUM: server: "proto" not working for dynamic servers
- MINOR: server: add helper function to detach server from proxy list
- DEBUG: add a tainted flag when ha_panic() is called
- DEBUG: lua: add tainted flags for stuck Lua contexts
- DEBUG: pools: detect that malloc_trim() is in progress
- BUG/MINOR: quic: do not consider idle timeout on CLOSING state
- MINOR: frontend: implement a dedicated actconn increment function
- BUG/MINOR: ssl: use a thread-safe sslconns increment
- MEDIUM: quic: count quic_conn instance for maxconn
- MEDIUM: quic: count quic_conn for global sslconns
- BUG/MINOR: ssl: suboptimal certificate selection with TLSv1.3 and dual ECDSA/RSA
- REGTESTS: ssl: update the filters test for TLSv1.3 and sigalgs
- BUG/MINOR: mux-quic: fix early close if unset client timeout
- BUG/MEDIUM: ssl: segfault when cipher is NULL
- BUG/MINOR: tcpcheck: Report hexstring instead of binary one on check failure
- MEDIUM: systemd: be more verbose about the reload
- MINOR: sample: Add fetcher for getting all cookie names
- BUG/MINOR: proto_reverse_connect: support SNI on active connect
- MINOR: proxy/stktable: add resolve_stick_rule helper function
- BUG/MINOR: stktable: missing free in parse_stick_table()
- BUG/MINOR: cfgparse/stktable: fix error message on stktable_init() failure
- MINOR: stktable: stktable_init() sets err_msg on error
- MINOR: stktable: check if a type should be used as-is
- MEDIUM: stktable/peers: "write-to" local table on peer updates
- CI: github: update wolfSSL to 5.6.4
- DOC: install: update the wolfSSL required version
- MINOR: server: Add parser support for set-proxy-v2-tlv-fmt
- MINOR: connection: Send out generic, user-defined server TLVs
- BUG/MEDIUM: pattern: don't trim pools under lock in pat_ref_purge_range()
- MINOR: mux-h2: always use h2_send() in h2_done_ff(), not h2_process()
- OPTIM: mux-h2: call h2_send() directly from h2_snd_buf()
- BUG/MINOR: server: remove some incorrect free() calls on null elements
To follow-up the implementation of the new set-proxy-v2-tlv-fmt
keyword in the server, the connection is updated to use the previously
allocated TLVs. If no value was specified, we send out an empty TLV.
As the feature is fully working with this commit, documentation and a
test for the server and default-server are added as well.
In this patch, we add the possibility to declare on a table definition
("table" in peer section, or "stick-table" in proxy section) that we
want the remote/peer updates on that table to be pushed on a local
haproxy table in addition to the source table.
Consider this example:
|peers mypeers
| peer local 127.0.0.1:3334
| peer clust 127.0.0.1:3333
| table t1.local type string size 10m store server_id,server_key expire 30s
| table t1.clust type string size 10m store server_id,server_key write-to mypeers/t1.local expire 30s
With this setup, we consider haproxy uses t1.local as cache/local table
for read and write operations, and that t1.clust is a remote table
containing datas processed from t1.local and similar tables from other
haproxy peers in a cluster setup. The t1.clust table will be used to
refresh the local/cache one via the "write-to" statement.
What will happen, is that every time haproxy will see entry updates for
the t1.clust table: it will overwrite t1.local table with fresh data and
will update the entry expiration timer. If t1.local entry doesn't exist
yet (key doesn't exist), it will automatically create it. Note that only
types that cannot be used for arithmetic ops will be handled, and this
to prevent processed values from the remote table from interfering with
computations based on values from the local table. (ie: prevent
cumulative counters from growing indefinitely).
"write-to" will only push supported types if they both exist in the source
and the target table. Be careful with server_id and server_key storage
because they are often declared implicitly when referencing a table in
sticking rules but it is required to declare them explicitly for them to
be pushed between a remote and a local table through "write-to" option.
Also note that the "write-to" target table should have the same type as
the source one, and that the key length should be strictly equal,
otherwise haproxy will raise an error due to the tables being
incompatibles. A table that is already being written to cannot be used
as a source table for a "write-to" target.
Thanks to this patch, it will now be possible to use sticking rules in
peer cluster context by using a local table as a local cache which
will be automatically refreshed by one or multiple remote table(s).
This commit depends on:
- "MINOR: stktable: stktable_init() sets err_msg on error"
- "MINOR: stktable: check if a type should be used as-is"
This new fetcher can be used to extract the list of cookie names from
Cookie request header or from Set-Cookie response header depending on
the stream direction. There is an optional argument that can be used
as the delimiter (which is assumed to be the first character of the
argument) between cookie names. The default delimiter is comma (,).
Note that we will treat the Cookie request header as a semi-colon
separated list of cookies and each Set-Cookie response header as
a single cookie and extract the cookie names accordingly.
After making it configurable in previous commit "MINOR: lua: Add flags
to configure logging behaviour", this patch changes the default value
of tune.lua.log.stderr from 'on' (unconditionally forward LUA logs to
stderr) to 'auto' (only forward LUA logs to stderr if logging via a
standard logger is disabled, or none is configured for the current context)
Since this is a change in behaviour, it shouldn't be backported
Until now, messages printed from LUA log functions were sent both to
the any logger configured for the current proxy, and additionally to
stderr (in most cases)
This introduces two flags to configure LUA log handling:
- tune.lua.log.loggers to use standard loggers or not
- tune.lua.log.stderr to use stderr, or not, or only conditionally
This addresses github feature request #2316
This can be backported to 2.8 as it doesn't change previous behaviour.
Released version 2.9-dev8 with the following main changes :
- MINOR: ssl: add an explicit error when 'ciphersuites' are not supported
- BUILD: ssl: enable 'ciphersuites' for WolfSSL
- BUILD: ssl: add 'ssl_c_r_dn' fetch for WolfSSL
- BUILD: ssl: add 'secure_memcmp' converter for WolfSSL and awslc
- BUILD: ssl: enable keylog for awslc
- CLEANUP: ssl: remove compat functions for openssl < 1.0.0
- BUILD: ssl: enable keylog for WolfSSL
- REGTESTS: pki: add a pki for SSL tests
- REGTESTS: ssl: update common.pem with the new pki
- REGTESTS: ssl: disable ssl_dh.vtc for WolfSSL
- REGTESTS: wolfssl: temporarly disable some failing reg-tests
- CI: ssl: add wolfssl to build-ssl.sh
- CI: ssl: add git id support for wolfssl download
- CI: github: add a wolfssl entry to the CI
- CI: github: update wolfssl to git revision d83f2fa
- CI: github: add awslc 1.16.0 to the push CI
- BUG/MINOR: quic: Avoid crashing with unsupported cryptographic algos
- REORG: quic: cleanup traces definition
- BUG/MINOR: quic: reject packet with no frame
- BUG/MEDIUM: mux-quic: fix RESET_STREAM on send-only stream
- BUG/MINOR: mux-quic: support initial 0 max-stream-data
- BUG/MINOR: h3: strengthen host/authority header parsing
- CLEANUP: connection: drop an uneeded leftover cast
- BUG/MAJOR: connection: make sure to always remove a connection from the tree
- BUG/MINOR: quic: fix qc.cids access on quic-conn fail alloc
- BUG/MINOR: quic: fix free on quic-conn fail alloc
- BUG/MINOR: mux-quic: fix free on qcs-new fail alloc
- BUG/MEDIUM: quic-conn: free unsent frames on retransmit to prevent crash
- MEDIUM: tree-wide: logsrv struct becomes logger
- MEDIUM: log: introduce log target
- DOC: config: log <address> becomes log <target> in "log" related doc
- MEDIUM: sink/log: stop relying on AF_UNSPEC for rings
- MINOR: log: support explicit log target as argument in __do_send_log()
- MINOR: log: remove the logger dependency in do_send_log()
- MEDIUM: log/sink: simplify log header handling
- MEDIUM: sink: inherit from caller fmt in ring_write() when rings didn't set one
- MINOR: sink: add sink_new_from_srv() function
- MAJOR: log: introduce log backends
- MINOR: log/balance: support for the "sticky" lb algorithm
- MINOR: log/balance: support for the "random" lb algorithm
- MINOR: lbprm: support for the "none" hash-type function
- MINOR: lbprm: compute the hash avalanche in gen_hash()
- MINOR: sample: add sample_process_cnv() function
- MEDIUM: log/balance: support for the "hash" lb algorithm
- REGTEST: add a test for log-backend used as a log target
- MINOR: server: introduce "log-bufsize" kw
- BUG/MEDIUM: stconn: Report a send activity everytime data were sent
- BUG/MEDIUM: applet: Report a send activity everytime data were sent
- BUG/MINOR: mux-h1: Send a 400-bad-request on shutdown before the first request
- MINOR: support for http-response set-timeout
- BUG/MINOR: mux-h2: make up other blocked streams upon removal from list
- DEBUG: pool: store the memprof bin on alloc() and update it on free()
- BUG/MEDIUM: quic_conn: let the scheduler kill the task when needed
- CLEANUP: hlua: Remove dead-code on error path in hlua_socket_new()
- BUG/MEDIUM: mux-h1: do not forget TLR/EOT even when no data is sent
- BUG/MINOR: htpp-ana/stats: Specify that HTX redirect messages have a C-L header
- BUG/MEDIUM: mux-h2: Don't report an error on shutr if a shutw is pending
- MEDIUM: stconn/channel: Move pipes used for the splicing in the SE descriptors
- MINOR: stconn: Start to introduce mux-to-mux fast-forwarding notion
- MINOR: stconn: Extend iobuf to handle a buffer in addition to a pipe
- MINOR: connection: Add new mux callbacks to perform data fast-forwarding
- MINOR: stconn: Temporarily remove kernel splicing support
- MINOR: mux-pt: Temporarily remove splicing support
- MINOR: mux-h1: Temporarily remove splicing support
- MINOR: connection: Remove mux callbacks about splicing
- MEDIUM: stconn: Add mux-to-mux fast-forward support
- MINOR: mux-h1: Use HTX extra field only for responses with known length
- MEDIUM: mux-h1: Properly handle state transitions of chunked outgoing messages
- MEDIUM: raw-sock: Specifiy amount of data to send via snd_pipe callback
- MINOR: mux-h1: Add function to add size of a chunk to an outgoind message
- MEDIUM: mux-h1: Simplify zero-copy on sending path
- MEDIUM: mux-h1: Simplify payload formatting based on HTX blocks on sending path
- MEDIUM: mux-h1: Add fast-forwarding support
- MINOR: h2: Set the BODYLESS_RESP flag on the HTX start-line if necessary
- MEDIUM: mux-h2: Add consumer-side fast-forwarding support
- MEDIUM: channel: don't look at iobuf to report an empty channel
- MINOR: tree-wide: Only rely on co_data() to check channel emptyness
- REGTESTS: Reenable HTTP tests about splicing
- CLEAN: mux-h1: Remove useless __maybe_unused attribute on h1_make_chunk()
- MEDIUM: mux-pt: Add fast-forwarding support
- MINOR: global: Add an option to disable the zero-copy forwarding
- BUILD: mux-h1: Fix build without kernel splicing support
- REORG: stconn/muxes: Rename init step in fast-forwarding
- MINOR: dgram: allow to set rcv/sndbuf for dgram sockets as well
- BUG/MINOR: mux-h2: fix http-request and http-keep-alive timeouts again
- BUG/MINOR: trace: fix trace parser error reporting
- BUG/MEDIUM: peers: Be sure to always refresh recconnect timer in sync task
- BUG/MEDIUM: peers: Fix synchro for huge number of tables
- MINOR: cfgparse: forbid mixing reverse and standard listeners
- MINOR: listener: add nbconn kw for reverse connect
- MINOR: server: convert @reverse to rev@ standard format
- MINOR: cfgparse: rename "rev@" prefix to "rhttp@"
- REGTESTS: remove maxconn from rhttp bind line
- MINOR: listener: forbid most keywords for reverse HTTP bind
- MINOR: sample: Added support for Arrays in sample_conv_json_query in sample.c
- MINOR: mux-h2/traces: explicitly show the error/refused stream states
- MINOR: mux-h2/traces: clarify the "rejected H2 request" event
- BUG/MINOR: mux-h2: commit the current stream ID even on reject
- BUG/MINOR: mux-h2: update tracked counters with req cnt/req err
Method now returns the content of Json Arrays, if it is specified in
Json Path as String. The start and end character is a square bracket. Any
complex object in the array is returned as Json, so that you might get Arrays
of Array or objects. Only recommended for Arrays of simple types (e.g.,
String or int) which will be returned as CSV String. Also updated
documentation and fixed issue with parenthesis and other changes from
comments.
This patch was discussed in issue #2281.
Signed-off-by: William Lallemand <wlallemand@haproxy.com>
Previously, maxconn keyword was reused for a specific usage on reverse
HTTP binds to specify the number of active connect to proceed. To avoid
confusion, introduce a new dedicated keyword 'nbconn' which is specific
to reverse HTTP bind.
This new keyword is forbidden for non-reverse listener. A fatal error is
emitted during config parsing if this rule is not respected. It's safe
because it's also forbidden to mix standard and reverse addresses on the
same bind line.
Internally, nbconn value will be reassigned to 'maxconn' member of
bind_conf structure. This ensures that listener layer will automatically
reenable the preconnect task each time a connection is closed.
tune.rcvbuf.client and tune.rcvbuf.server are not suitable for shared
dgram sockets because they're per connection so their units are not the
same. However, QUIC's listener and log servers are not connected and
take per-thread or per-process traffic where a socket log buffer might
be too small, causing undesirable packet losses and retransmits in the
case of QUIC. This essentially manifests in listener mode with new
connections taking a lot of time to set up under heavy traffic due to
the small queues causing delays. Let's add a few new settings allowing
to set these shared socket sizes on the frontend and backend side (which
reminds that these are per-front/back and not per client/server hence
not per connection).
