Commit Graph

2192 Commits

Author SHA1 Message Date
Thierry Fournier
55c40ea177 DOC: fix misspelled keyword "resolve_retries" in resolvers
"resolve_retries" was spelled "resolve_retires".
2021-12-21 08:27:52 +01:00
Remi Tricot-Le Breton
cc750efbc5 MINOR: ssl: Remove empty lines from "show ssl ocsp-response" output
There were empty lines in the output of the CLI's "show ssl
ocsp-response" command (after the certificate ID and between two
certificates). This patch removes them since an empty line should mark
the end of the output.

Must be backported in 2.5.
2021-12-20 12:02:17 +01:00
Remi Tricot-Le Breton
164726c9b8 DOC: vars: Add documentation about the set-var conditions
The set-var converter as well as the http and tcp set-var actions can
now be given multiple conditions that need to all be true for the
variable's contents to actually be changed. Those conditions can concern
the variable as well as the input contents and can also work by
comparing the variable and the input values.
2021-12-16 17:31:57 +01:00
Amaury Denoyelle
b09f4477f4 CLEANUP: cfgparse: modify preprocessor guards around numa detection code
numa_detect_topology() is always define now if USE_CPU_AFFINITY is
activated. For the moment, only on Linux an actual implementation is
provided. For other platforms, it always return 0.

This change has been made to easily add implementation of NUMA detection
for other platforms. The phrasing of the documentation has also been
edited to removed the mention of Linux-only on numa-cpu-mapping
configuration option.
2021-12-15 11:05:51 +01:00
William Lallemand
740629e296 MINOR: cli: "show version" displays the current process version
This patch implements a simple "show version" command which returns
the version of the current process.

It's available from the master and the worker processes, so it is easy
to check if the master and the workers have the same version.

This is a minor patch that really improve compatibility checks
for scripts.

Could be backported in haproxy version as far as 2.0.
2021-12-14 15:40:06 +01:00
Lukas Tribus
2b94973e0f DOC: config: fix error-log-format example
In commit 6f7497616 ("MEDIUM: connection: rename fc_conn_err and
bc_conn_err to fc_err and bc_err"), fc_conn_err became fc_err, so
update this example.
2021-12-09 03:57:17 +01:00
Lukas Tribus
de16008c29 DOC: config: retry-on list is space-delimited
We are using comma-delimited list for init-addr for example, let's
document that this is space-delimited to avoid the guessing game.
2021-12-09 03:57:17 +01:00
Christopher Faulet
ba8f06304e MINOR: http-rules: Add capture action to http-after-response ruleset
It is now possible to perform captures on the response when
http-after-response rules are evaluated. It may be handy to capture headers
from responses generated by HAProxy.

This patch is trivial, it may be backported if necessary.
2021-12-07 19:04:33 +01:00
Christopher Faulet
3010e00e1c DOC: config: Specify %Ta is only available in HTTP mode
%Ta format can only be used in HTTP mode but it was not specify in the
configuration manual.

This patch should fix the issue #1317.
2021-12-03 10:48:37 +01:00
Christopher Faulet
d322e948c9 DOC: spoe: Clarify use of the event directive in spoe-message section
Only one event is possible for a spoe-message section. If defined several
time, only the last one is considered. The documentation is now explicit on
this point.

This patch is related to the the issue #1351.
2021-12-03 10:18:11 +01:00
William Lallemand
2be557f7cb MEDIUM: mworker: seamless reload use the internal sockpairs
With the master worker, the seamless reload was still requiring an
external stats socket to the previous process, which is a pain to
configure.

This patch implements a way to use the internal socketpair between the
master and the workers to transfer the sockets during the reload.
This way, the master will always try to transfer the socket, even
without any configuration.

The master will still reload with the -x argument, followed by the
sockpair@ syntax. ( ex -x sockpair@4 ). Which use the FD of internal CLI
to the worker.
2021-11-24 19:00:39 +01:00
Willy Tarreau
73dec76e85 [RELEASE] Released version 2.6-dev0
Released version 2.6-dev0 with the following main changes :
    - MINOR: version: it's development again
2021-11-23 15:50:11 +01:00
Willy Tarreau
f2e0833f16 [RELEASE] Released version 2.5.0
Released version 2.5.0 with the following main changes :
    - BUILD: SSL: add quictls build to scripts/build-ssl.sh
    - BUILD: SSL: add QUICTLS to build matrix
    - CLEANUP: sock: Wrap `accept4_broken = 1` into additional parenthesis
    - BUILD: cli: clear a maybe-unused  warning on some older compilers
    - BUG/MEDIUM: cli: make sure we can report a warning from a bind keyword
    - BUG/MINOR: ssl: make SSL counters atomic
    - CLEANUP: assorted typo fixes in the code and comments
    - BUG/MINOR: ssl: free correctly the sni in the backend SSL cache
    - MINOR: version: mention that it's stable now
2021-11-23 15:40:21 +01:00
Ilya Shipitsin
a4d09e7ffd CLEANUP: assorted typo fixes in the code and comments
This is 28th iteration of typo fixes
2021-11-22 19:08:12 +01:00
Willy Tarreau
a99cdfb531 [RELEASE] Released version 2.5-dev15
Released version 2.5-dev15 with the following main changes :
    - BUG/MINOR: stick-table/cli: Check for invalid ipv6 key
    - CLEANUP: peers: Remove useless test on peer variable in peer_trace()
    - DOC: log: Add comments to specify when session's listener is defined or not
    - BUG/MEDIUM: mux-h1: Handle delayed silent shut in h1_process() to release H1C
    - REGTESTS: ssl_crt-list_filters: feature cmd incorrectly set
    - DOC: internals: document the list API
    - BUG/MINOR: h3: ignore unknown frame types
    - MINOR: quic: redirect app_ops snd_buf through mux
    - MEDIUM: quic: inspect ALPN to install app_ops
    - MINOR: quic: support hq-interop
    - MEDIUM: quic: send version negotiation packet on unknown version
    - BUG/MEDIUM: mworker: cleanup the listeners when reexecuting
    - DOC: internals: document the scheduler API
    - BUG/MINOR: quic: fix version negotiation packet generation
    - CLEANUP: ssl: fix wrong #else commentary
    - MINOR: config: support default values for environment variables
    - SCRIPTS: run-regtests: reduce the number of processes needed to check options
    - SCRIPT: run-regtests: avoid several calls to grep to test for features
    - SCRIPT: run-regtests: avoid calling awk to compute the version
    - REGTEST: set retries count to zero for all tests that expect at 503
    - REGTESTS: make tcp-check_min-recv fail fast
    - REGTESTS: extend the default I/O timeouts and make them overridable
    - BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3
    - BUG/MEDIUM: ssl: abort with the correct SSL error when SNI not found
    - REGTESTS: ssl: test the TLS resumption
    - BUILD: makefile: stop opening sub-shells for each and every command
    - BUILD: makefile: reorder objects by build time
    - BUG/MEDIUM: mux-h2: always process a pending shut read
    - MINOR: quic_sock: missing CO_FL_ADDR_TO_SET flag
    - MINOR: quic: Possible wrong connection identification
    - MINOR: quic: Correctly pad UDP datagrams
    - MINOR: quic: Support transport parameters draft TLS extension
    - MINOR: quic: Anti-amplification implementation
    - MINOR: quic: Wrong Initial packet connection initialization
    - MINOR: quic: Wrong ACK range building
    - MINOR: quic: Update some QUIC protocol errors
    - MINOR: quic: Send CONNECTION_CLOSE frame upon TLS alert
    - MINOR: quic: Wrong largest acked packet number parsing
    - MINOR: quic: Add minimalistic support for stream flow control frames
    - MINOR: quic: Wrong value for version negotiation packet 'Unused' field
    - MINOR: quic: Support draft-29 QUIC version
    - BUG/MINOR: quic: fix segfault on trace for version negotiation
    - BUG/MINOR: hq-interop: fix potential NULL dereference
    - BUILD: quic: fix potential NULL dereference on xprt_quic
    - DOC: lua: documentation about the httpclient API
    - BUG/MEDIUM: cache/cli: make "show cache" thread-safe
    - BUG/MEDIUM: shctx: leave the block allocator when enough blocks are found
    - BUG/MINOR: shctx: do not look for available blocks when the first one is enough
    - MINOR: shctx: add a few BUG_ON() for consistency checks
2021-11-19 19:30:04 +01:00
William Lallemand
00a15025be DOC: lua: documentation about the httpclient API
The patch adds the documentation about the httpclient lua
implementation.
2021-11-19 16:06:23 +01:00
Willy Tarreau
ec347b1239 MINOR: config: support default values for environment variables
Sometimes it is really useful to be able to specify a default value for
an optional environment variable, like the ${name-value} construct in
shell. In fact we're really missing this for a number of settings in
reg tests, starting with timeouts.

This commit simply adds support for the common syntax above. Other
common forms like '+' to replace existing variables, or ':-' and ':+'
to act on empty variables, were not implemented at this stage, as they
are less commonly needed.
2021-11-18 17:54:49 +01:00
Willy Tarreau
6232d11626 DOC: internals: document the scheduler API
Another non-trivial part that is often needed. Exported functions
and flags available to applications were documented as well as some
restrictions and falltraps.
2021-11-18 11:27:30 +01:00
Willy Tarreau
946ebbb0bd DOC: internals: document the list API
This one is aging and not always trivial. Let's indicate what the
macros do and what traps not to fall into.
2021-11-17 15:30:53 +01:00
Willy Tarreau
d83f6e6224 [RELEASE] Released version 2.5-dev14
Released version 2.5-dev14 with the following main changes :
    - DEV: coccinelle: Remove unused `expression e`
    - DEV: coccinelle: Add rule to use `istend()` where possible
    - CLEANUP: Apply ist.cocci
    - CLEANUP: Re-apply xalloc_size.cocci
    - CLEANUP: halog: make the default usage message fit in small screens
    - MINOR: h3/qpack: fix gcc11 warnings
    - MINOR: mux-quic: fix gcc11 warning
    - MINOR: h3: fix potential NULL dereference
    - MINOR: quic: Fix potential null pointer dereference
    - CLEANUP: halog: remove unused strl2ui()
    - OPTIM: halog: improve field parser speed for modern compilers
    - OPTIM: halog: skip fields 64 bits at a time when supported
    - DEV: coccinelle: Add rule to use `isttrim()` where possible
    - CLEANUP: Apply ist.cocci
    - DEV: coccinelle: Add rule to use `chunk_istcat()` instead of `chunk_memcat()`
    - DEV: coccinelle: Add rule to use `chunk_istcat()` instead of `chunk_strncat()`
    - CLEANUP: Apply ist.cocci
    - CLEANUP: chunk: Remove duplicated chunk_Xcat implementation
    - CLEANUP: chunk: remove misleading chunk_strncat() function
    - BUG/MINOR: cache: properly ignore unparsable max-age in quotes
    - Revert "DEV: coccinelle: Add rule to use `chunk_istcat()` instead of `chunk_strncat()`"
    - DOC: stats: fix location of the text representation
    - DOC: internals: document the IST API
    - BUG/MINOR: httpclient/lua: rcv freeze when no request payload
    - BUG/MEDIUM: httpclient: channel_add_input() must use htx->data
    - MINOR: promex: backend aggregated server check status
    - DOC: config: Fix typo in ssl_fc_unique_id description
    - BUG/MINOR: http-ana: Apply stop to the current section for http-response rules
    - Revert "BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on back"
    - DOC: config: Be more explicit in "allow" actions description
    - DOC: lua: Be explicit with the Reply object limits
    - MINOR: mux-h1: Slightly Improve H1 traces
    - BUG/MEDIUM: conn-stream: Don't reset CS flags on close
    - CLEANUP: mworker: remove any relative PID reference
    - MEDIUM: mworker: reexec in waitpid mode after successful loading
    - MINOR: mworker: clarify starting/failure messages
    - MINOR: mworker: only increment the number of reload in wait mode
    - MINOR: mworker: implement a reload failure counter
    - MINOR: mworker: ReloadFailed shown depending on failedreload
    - MINOR: mworker: change the way we set PROC_O_LEAVING
    - BUG/MINOR: mworker: doesn't launch the program postparser
    - DOC: management: edit the "show proc" example to show the current output
    - BUG/MEDIUM: httpclient/cli: free of unallocated hc->req.uri
    - REGTESTS: httpclient/lua: add greater body values
    - BUG/MINOR: mux-h2: Fix H2_CF_DEM_SHORT_READ value
    - BUG/MINOR: pools: don't mark ourselves as harmless in DEBUG_UAF mode
    - BUG/MEDIUM: connection: make cs_shutr/cs_shutw//cs_close() idempotent
    - BUILD: makefile: simplify detection of libatomic
2021-11-14 16:04:57 +01:00
William Lallemand
a68a00fe28 DOC: management: edit the "show proc" example to show the current output
The "show proc" output changed and it's time to update the example.

The output does not display the relative PID anymore since the nbproc
keyword disappeared, and it displays the number of failed reloads since
the last successful one.
2021-11-10 15:53:01 +01:00
Christopher Faulet
7855b19734 DOC: lua: Be explicit with the Reply object limits
In HTTP, when a lua action is evaluated, a reply object can be used to send
a response to the client and interrupt the transaction. This reply object is
converted into HTX and is limited to the response channel buffer. Its size,
once converted, cannot exceed the buffer size. There is no streaming at this
stage. However, this limitation was not documented.

Note that, for now, there is no easy way to know if the reply will fit or
not int the response channel buffer. Thus the reply must be reasonably
small. Otherwise a 500-Internal-Error message is returned.

This patch is related to the issue #1447. It may be backported as far as
2.2.
2021-11-09 18:49:46 +01:00
Christopher Faulet
2702560d7b DOC: config: Be more explicit in "allow" actions description
TCP/HTTP allow actions stop rules evaluation of the current section
only. Only the http-response description was accurate on this
point. Thus, the documentation is now explicit on this point for all
other concerned rulesets.

This patch may be backported, to all supported versions for tcp-request
and http-request documentation, and as far as 2.2 for http-after-response
documentation.
2021-11-09 18:05:33 +01:00
Christopher Faulet
15ae22c02c DOC: config: Fix typo in ssl_fc_unique_id description
In ssl_fc_unique_id decription, threre is a reference to the wrong sample
fetch. ssl_bc_unique_id is used instead of ssl_fc_unique_id.

This patch should fix the issue #1449. It may be backported to all
supportted versions.
2021-11-09 14:23:37 +01:00
Willy Tarreau
bc84657410 DOC: internals: document the IST API
This one was missing. It should be easier to use now. It is obvious that
some functions are missing, and it looks like ist2str() and istpad() are
exactly the same.
2021-11-08 16:50:48 +01:00
Willy Tarreau
08d3220de5 [RELEASE] Released version 2.5-dev13
Released version 2.5-dev13 with the following main changes :
    - SCRIPTS: git-show-backports: re-enable file-based filtering
    - MINOR: jwt: Make invalid static JWT algorithms an error in `jwt_verify` converter
    - MINOR: mux-h2: add trace on extended connect usage
    - BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
    - MINOR: stream/mux: implement websocket stream flag
    - MINOR: connection: implement function to update ALPN
    - MINOR: connection: add alternative mux_ops param for conn_install_mux_be
    - MEDIUM: server/backend: implement websocket protocol selection
    - MINOR: server: add ws keyword
    - BUG/MINOR: resolvers: fix sent messages were counted twice
    - BUG/MINOR: resolvers: throw log message if trash not large enough for query
    - MINOR: resolvers/dns: split dns and resolver counters in dns_counter struct
    - MEDIUM: resolvers: rename dns extra counters to resolvers extra counters
    - BUG/MINOR: jwt: Fix jwt_parse_alg incorrectly returning JWS_ALG_NONE
    - DOC: add QUIC instruction in INSTALL
    - CLEANUP: halog: Remove dead stores
    - DEV: coccinelle: Add ha_free.cocci
    - CLEANUP: Apply ha_free.cocci
    - DEV: coccinelle: Add rule to use `istnext()` where possible
    - CLEANUP: Apply ist.cocci
    - REGTESTS: Use `feature cmd` for 2.5+ tests (2)
    - DOC: internals: move some API definitions to an "api" subdirectory
    - MINOR: quic: Allocate listener RX buffers
    - CLEANUP: quic: Remove useless code
    - MINOR: quic: Enhance the listener RX buffering part
    - MINOR: quic: Remove a useless lock for CRYPTO frames
    - MINOR: quic: Use QUIC_LOCK QUIC specific lock label.
    - MINOR: backend: Get client dst address to set the server's one only if needful
    - MINOR: compression: Warn for 'compression offload' in defaults sections
    - MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
    - DOC: configuration: move the default log formats to their own section
    - MINOR: ssl: make the ssl_fc_sni() sample-fetch function always available
    - MEDIUM: log: add the client's SNI to the default HTTPS log format
    - DOC: config: add an example of reasonably complete error-log-format
    - DOC: config: move error-log-format before custom log format
2021-11-06 09:25:57 +01:00
Willy Tarreau
ec5c110e2d DOC: config: move error-log-format before custom log format
All default formats were described before the custom one, except this
one. Better place them all together before the custom log format. This
only swaps and renumbers the sections.
2021-11-06 09:20:07 +01:00
Willy Tarreau
ecc79bbe28 DOC: config: add an example of reasonably complete error-log-format
This commit adds a suggestion of a useful error-log-format that was
tested with success in production.
2021-11-06 09:20:07 +01:00
Willy Tarreau
68574dd492 MEDIUM: log: add the client's SNI to the default HTTPS log format
During a troublehooting it came obvious that the SNI always ought to
be logged on httpslog, as it explains errors caused by selection of
the default certificate (or failure to do so in case of strict-sni).

This expectation was also confirmed on the mailing list.

Since the field may be empty it appeared important not to leave an
empty string in the current format, so it was decided to place the
field before a '/' preceding the SSL version and ciphers, so that
in the worst case a missing field leads to a field looking like
"/TLSv1.2/AES...", though usually a missing element still results
in a "-" in logs.

This will change the log format for users who already deployed the
2.5-dev versions (hence the medium level) but no released version
was using this format yet so there's no harm for stable deployments.
The reg-test was updated to check for "-" there since we don't send
SNI in reg-tests.

Link: https://www.mail-archive.com/haproxy@formilux.org/msg41410.html
Cc: William Lallemand <wlallemand@haproxy.org>
2021-11-06 09:20:07 +01:00
Willy Tarreau
2ed7350f4c DOC: configuration: move the default log formats to their own section
I'm always having a very hard time finding the log-format definition of
httplog, because it's not in the httplog description, and looking for
"httplog" doesn't yield the custom log formats section.

It would make more sense to write these log-formats into their respective
sections where they will be easier to find. That's what this commit does.
2021-11-06 09:20:07 +01:00
Willy Tarreau
6f7497616e MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
Commit 3d2093af9 ("MINOR: connection: Add a connection error code sample
fetch") added these convenient sample-fetch functions but it appears that
due to a misunderstanding the redundant "conn" part was kept in their
name, causing confusion, since "fc" already stands for "front connection".

Let's simply call them "fc_err" and "bc_err" to match all other related
ones before they appear in a final release. The VTC they appeared in were
also updated, and the alpha sort in the keywords table updated.

Cc: William Lallemand <wlallemand@haproxy.org>
2021-11-06 09:20:07 +01:00
Christopher Faulet
44d34bfbe7 MINOR: compression: Warn for 'compression offload' in defaults sections
This directive is documented as being ignored if set in a defaults
section. But it is only mentionned in a small note in the configuration
manual. Thus, now, a warning is emitted. To do so, the errors handling in
parse_compression_options() function was slightly changed.

In addition, this directive is now documented apart from the other
compression directives. This way, it is clearly visible that it must not be
used in a defaults section.
2021-11-05 16:36:42 +01:00
Willy Tarreau
a62f184d3d DOC: internals: move some API definitions to an "api" subdirectory
It's not always easy to figure that there are some docs on internal API
stuff, let's move them to their own directory. There's a diagram for
lists that could be placed there but instead would deserve a greppable
description for quick lookups, so it was not moved there.
2021-11-05 11:53:22 +01:00
Amaury Denoyelle
f9d5957cd9 MINOR: server: add ws keyword
Implement parsing for the server keyword 'ws'. This is used to configure
the mode of selection for websocket protocol. The configuration
documentation has been updated.

A new regtest has been created to test the proper behavior of the
keyword.
2021-11-03 16:24:48 +01:00
Willy Tarreau
35dc13f224 [RELEASE] Released version 2.5-dev12
Released version 2.5-dev12 with the following main changes :
    - MINOR: httpclient: support payload within a buffer
    - MINOR: httpclient/lua: support more HTTP methods
    - MINOR: httpclient/lua: return an error when it can't generate the request
    - CLEANUP: lua: Remove any ambiguities about lua txn execution context flags
    - BUG/MEDIUM: lua: fix invalid return types in hlua_http_msg_get_body
    - CLEANUP: connection: No longer export make_proxy_line_v1/v2 functions
    - CLEANUP: tools: Use const address for get_net_port() and get_host_port()
    - CLEANUP: lua: Use a const address to retrieve info about a connection
    - MINOR: connection: Add function to get src/dst without updating the connection
    - MINOR: session: Add src and dst addresses to the session
    - MINOR: stream-int: Add src and dst addresses to the stream-interface
    - MINOR: frontend: Rely on client src and dst addresses at stream level
    - MINOR: log: Rely on client addresses at the appropriate level to log messages
    - MINOR: session: Rely on client source address at session level to log error
    - MINOR: http-ana: Rely on addresses at stream level to set xff and xot headers
    - MINOR: http-fetch: Rely on addresses at stream level in HTTP sample fetches
    - MINOR: mux-fcgi: Rely on client addresses at stream level to set default params
    - MEDIUM: tcp-sample: Rely on addresses at the appropriate level in tcp samples
    - MEDIUM: connection: Rely on addresses at stream level to make proxy line
    - MEDIUM: backend: Rely on addresses at stream level to init server connection
    - MEDIUM: connection: Assign session addresses when PROXY line is received
    - MEDIUM: connection: Assign session addresses when NetScaler CIP proto is parsed
    - MEDIUM: tcp-act: Set addresses at the apprioriate level in set-(src/dst) actions
    - MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
    - DOC: config: Fix alphabetical order of fc_* samples
    - MINOR: tcp-sample: Add samples to get original info about client connection
    - REGTESTS: Add script to test client src/dst manipulation at different levels
    - MINOR: stream: Use backend stream-interface dst address instead of target_addr
    - BUILD: log: Fix compilation without SSL support
    - DEBUG: protocol: yell loudly during registration of invalid sock_domain
    - MINOR: protocols: add a new protocol type selector
    - MINOR: protocols: make use of the protocol type to select the protocol
    - MINOR: protocols: replace protocol_by_family() with protocol_lookup()
    - MINOR: halog: Add -qry parameter allowing to preserve the query string in -uX
    - CLEANUP: jwt: Remove the use of a trash buffer in jwt_jwsverify_hmac()
    - CLEANUP: jwt: Remove the use of a trash buffer in jwt_jwsverify_rsa_ecdsa()
    - DEV: coccinelle: Add realloc_leak.cocci
    - CLEANUP: hlua: Remove obsolete branch in `hlua_alloc()`
    - BUILD: atomic: prefer __atomic_compare_exchange_n() for __ha_cas_dw()
    - BUILD: atomic: fix build on mac/arm64
    - MINOR: atomic: remove the memcpy() call and dependency on string.h
    - MINOR: httpclient: request streaming with a callback
    - MINOR: httpclient/lua: handle the streaming into the lua applet
    - REGTESTS: lua: test httpclient with body streaming
    - DOC: halog: Move the `-qry` parameter into the correct section in help text
    - MINOR: halog: Rename -qry to -query
    - CLEANUP: halog: Use consistent indentation in help()
    - BUG/MINOR: halog: Add missing newlines in die() messages
    - MINOR: halog: Add support for extracting captures using -hdr
    - DOC: Typo fixed "it" should be "is"
    - BUG/MINOR: mux-h1: Save shutdown mode if the shutdown is delayed
    - BUG/MEDIUM: mux-h1: Perform a connection shutdown when the h1c is released
    - BUG/MEDIUM: resolvers: Don't recursively perform requester unlink
    - BUG/MEDIUM: http-ana: Drain request data waiting the tarpit timeout expiration
    - BUG/MINOR: http: Authorization value can have multiple spaces after the scheme
    - BUG/MINOR: http: http_auth_bearer fetch does not work on custom header name
    - BUG/MINOR: httpclient/lua: misplaced luaL_buffinit()
    - BUILD/MINOR: cpuset freebsd build fix
    - BUG/MINOR: httpclient: use a placeholder value for Host header
    - BUG/MEDIUM: stream-int: Block reads if channel cannot receive more data
    - BUG/MEDIUM: resolvers: Track api calls with a counter to free resolutions
    - MINOR: stream: Improve dump of bogus streams
    - DOC/peers: some grammar fixes for peers 2.1 spec
    - MEDIUM: vars: make the var() sample fetch function really return type ANY
    - MINOR: vars: add "set-var" for "tcp-request connection" rules.
2021-11-02 18:05:41 +01:00
Jaroslaw Rzeszótko
c8637032a7 MINOR: vars: add "set-var" for "tcp-request connection" rules.
Session struct is already allocated when "tcp-request connection" rules
are evaluated so session-scoped variables turned out easy to support.

This resolves github issue #1408.
2021-11-02 17:58:35 +01:00
John Roesler
7f31fec896 DOC/peers: some grammar fixes for peers 2.1 spec
These are a few minor grammar fixes for the peers protocol 2.1 documentation.
2021-11-02 17:28:43 +01:00
Anubhav
e09efaa7cb DOC: Typo fixed "it" should be "is"
This patch was proposed in GitHub PR #1415.

Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
2021-10-29 08:22:15 +02:00
Christopher Faulet
888cd700f4 MINOR: tcp-sample: Add samples to get original info about client connection
Because source and destination address of the client connection are now
updated at the appropriated level (connection, session or stream), original
info about the client connection are preserved.  src/src_port/src_is_local
and dst/dst_port/dst_is_local return current info about the client
connection. It is the info at the highest available level. Most of time, the
stream. Any tcp/http rules may alter this info.

To get original info, "fc_" prefix must be added. For instance
"fc_src". Here, only "tcp-request connection" rules may alter source and
destination address/port.
2021-10-27 11:35:59 +02:00
Christopher Faulet
7bd21921d1 DOC: config: Fix alphabetical order of fc_* samples
fc_* samples were not properly ordered. This patch may be backported as far
as 1.8.
2021-10-27 11:35:59 +02:00
Christopher Faulet
1e83b70409 MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
This patch was reverted because it was inconsitent to change connection
addresses at stream level. Especially in HTTP because all requests was
affected by this change and not only the current one. In HTTP/2, it was
worse. Several streams was able to change the connection addresses at the
same time.

It is no longer an issue, thanks to recent changes. With multi-level client
source and destination addresses, it is possible to limit the change to the
current request. Thus this patch can be reintroduced.

If it possible to set source IP/Port from "tcp-request connection",
"tcp-request session" and "http-request" rules but not from "tcp-request
content" rules. There is no reason for this limitation and it may be a
problem for anyone wanting to call a lua fetch to dynamically set source
IP/Port from a TCP proxy. Indeed, to call a lua fetch, we must have a
stream. And there is no stream when "tcp-request connection/session" rules
are evaluated.

Thanks to this patch, "set-src" and "set-src-port" action are now supported
by "tcp_request content" rules.

This patch is related to the issue #1303.
2021-10-27 11:35:59 +02:00
Willy Tarreau
b4d0cd02c1 [RELEASE] Released version 2.5-dev11
Released version 2.5-dev11 with the following main changes :
    - DEV: coccinelle: Add strcmp.cocci
    - CLEANUP: Apply strcmp.cocci
    - CI: Add `permissions` to GitHub Actions
    - CI: Clean up formatting in GitHub Action definitions
    - MINOR: add ::1 to predefined LOCALHOST acl
    - CLEANUP: assorted typo fixes in the code and comments
    - CLEANUP: Consistently `unsigned int` for bitfields
    - MEDIUM: resolvers: lower-case labels when converting from/to DNS names
    - MEDIUM: resolvers: replace bogus resolv_hostname_cmp() with memcmp()
    - MINOR: jwt: Empty the certificate tree during deinit
    - MINOR: jwt: jwt_verify returns negative values in case of error
    - MINOR: jwt: Do not rely on enum order anymore
    - BUG/MEDIUM: stream: Keep FLT_END analyzers if a stream detects a channel error
    - MINOR: httpclient/cli: access should be only done from expert mode
    - DOC: management: doc about the CLI httpclient
    - BUG/MEDIUM: tcpcheck: Properly catch early HTTP parsing errors
    - BUG/MAJOR: dns: tcp session can remain attached to a list after a free
    - BUG/MAJOR: dns: attempt to lock globaly for msg waiter list instead of use barrier
    - CLEANUP: dns: always detach the appctx from the dns session on release
    - DEBUG: dns: add a few more BUG_ON at sensitive places
    - BUG/MAJOR: resolvers: add other missing references during resolution removal
    - CLEANUP: resolvers: do not export resolv_purge_resolution_answer_records()
    - BUILD: resolvers: avoid a possible warning on null-deref
    - BUG/MEDIUM: resolvers: always check a valid item in query_list
    - CLEANUP: always initialize the answer_list
    - CLEANUP: resolvers: simplify resolv_link_resolution() regarding requesters
    - CLEANUP: resolvers: replace all LIST_DELETE with LIST_DEL_INIT
    - MEDIUM: resolvers: use a kill list to preserve the list consistency
    - MEDIUM: resolvers: remove the last occurrences of the "safe" argument
    - BUG/MEDIUM: checks: fix the starting thread for external checks
    - MEDIUM: resolvers: replace the answer_list with a (flat) tree
    - MEDIUM: resolvers: hash the records before inserting them into the tree
    - BUG/MAJOR: buf: fix varint API post- vs pre- increment
    - OPTIM: resolvers: move the eb32 node before the data in the answer_item
    - MINOR: list: add new macro LIST_INLIST_ATOMIC()
    - OPTIM: dns: use an atomic check for the list membership
    - BUG/MINOR: task: do not set TASK_F_USR1 for no reason
    - BUG/MINOR: mux-h2: do not prevent from sending a final GOAWAY frame
    - MINOR: connection: add a new CO_FL_WANT_DRAIN flag to force drain on close
    - MINOR: mux-h2: perform a full cycle shutdown+drain on close
    - CLEANUP: resolvers: get rid of single-iteration loop in resolv_get_ip_from_response()
    - MINOR: quic: Increase the size of handshake RX UDP datagrams
    - BUG/MEDIUM: lua: fix memory leaks with realloc() on non-glibc systems
    - MINOR: memprof: report the delta between alloc and free on realloc()
    - MINOR: memprof: add one pointer size to the size of allocations
    - BUILD: fix compilation on NetBSD
    - MINOR: backend: add traces for idle connections reuse
    - BUG/MINOR: backend: fix improper insert in avail tree for always reuse
    - MINOR: backend: improve perf with tcp proxies skipping idle conns
    - MINOR: connection: remove unneeded memset 0 for idle conns
2021-10-22 19:40:44 +02:00
William Lallemand
b175c23812 DOC: management: doc about the CLI httpclient
Add the doc about the CLI httpclient.
2021-10-19 15:02:42 +02:00
Remi Tricot-Le Breton
1c891bcc90 MINOR: jwt: jwt_verify returns negative values in case of error
In order for all the error return values to be distributed on the same
side (instead of surrounding the success error code), the return values
for errors other than a simple verification failure are switched to
negative values. This way the result of the jwt_verify converter can be
compared strictly to 1 as well relative to 0 (any <= 0 return value is
an error).
The documentation was also modified to discourage conversion of the
return value into a boolean (which would definitely not work).
2021-10-18 16:02:29 +02:00
Ilya Shipitsin
bd6b4be721 CLEANUP: assorted typo fixes in the code and comments
This is 27th iteration of typo fixes
2021-10-18 07:26:19 +02:00
Björn Jacke
20d0f50b00 MINOR: add ::1 to predefined LOCALHOST acl
The "LOCALHOST" ACL currently matches only 127.0.0.1/8. This adds the
IPv6 "::1" address to the supported patterns.
2021-10-18 07:21:28 +02:00
Willy Tarreau
f2b1b4dd14 [RELEASE] Released version 2.5-dev10
Released version 2.5-dev10 with the following main changes :
    - MINOR: initcall: Rename __GLOBL and __GLOBL1.
    - MINOR: rules: add a new function new_act_rule() to allocate act_rules
    - MINOR: rules: add a file name and line number to act_rules
    - MINOR: stream: report the current rule in "show sess all" when known
    - MINOR: stream: report the current filter in "show sess all" when known
    - CLEANUP: stream: Properly indent current_rule line in "show sess all"
    - BUG/MINOR: lua: Fix lua error handling in `hlua_config_prepend_path()`
    - CI: github: switch to OpenSSL 3.0.0
    - REGTESTS: ssl: Fix references to removed option in test description
    - MINOR: ssl: Add ssllib_name_startswith precondition
    - REGTESTS: ssl: Fix ssl_errors test for OpenSSL v3
    - REGTESTS: ssl: Reenable ssl_errors test for OpenSSL only
    - REGTESTS: ssl: Use mostly TLSv1.2 in ssl_errors test
    - MEDIUM: mux-quic: rationalize tx buffers between qcc/qcs
    - MEDIUM: h3: properly manage tx buffers for large data
    - MINOR: mux-quic: standardize h3 settings sending
    - CLEANUP: h3: remove dead code
    - MINOR: mux-quic: implement standard method to detect if qcc is dead
    - MEDIUM: mux-quic: defer stream shut if remaining tx data
    - MINOR: mux: remove last occurences of qcc ring buffer
    - MINOR: quic: handle CONNECTION_CLOSE frame
    - REGTESTS: ssl: re-enable set_ssl_cert_bundle.vtc
    - MINOR: ssl: add ssl_fc_is_resumed to "option httpslog"
    - MINOR: http: Add http_auth_bearer sample fetch
    - MINOR: jwt: Parse JWT alg field
    - MINOR: jwt: JWT tokenizing helper function
    - MINOR: jwt: Insert public certificates into dedicated JWT tree
    - MINOR: jwt: jwt_header_query and jwt_payload_query converters
    - MEDIUM: jwt: Add jwt_verify converter to verify JWT integrity
    - REGTESTS: jwt: Add tests for the jwt_verify converter
    - BUILD: jwt: fix declaration of EVP_KEY in jwt-h.h
    - MINOR: proto_tcp: use chunk_appendf() to ouput socket setup errors
    - MINOR: proto_tcp: also report the attempted MSS values in error message
    - MINOR: inet: report the faulty interface name in "bind" errors
    - MINOR: protocol: report the file and line number for binding/listening errors
    - MINOR: protocol: uniformize protocol errors
    - MINOR: resolvers: fix the resolv_str_to_dn_label() API about trailing zero
    - BUG/MEDIUM: resolver: make sure to always use the correct hostname length
    - BUG/MINOR: resolvers: do not reject host names of length 255 in SRV records
    - MINOR: resolvers: fix the resolv_dn_label_to_str() API about trailing zero
    - MEDIUM: listeners: split the thread mask between receiver and bind_conf
    - MINOR: listeners: add clone_listener() to duplicate listeners at boot time
    - MEDIUM: listener: add the "shards" bind keyword
    - BUG/MEDIUM: resolvers: use correct storage for the target address
    - MINOR: resolvers: merge address and target into a union "data"
    - BUG/MEDIUM: resolvers: fix truncated TLD consecutive to the API fix
    - BUG/MEDIUM: jwt: fix base64 decoding error detection
    - BUG/MINOR: jwt: use CRYPTO_memcmp() to compare HMACs
    - DOC: jwt: fix a typo in the jwt_verify() keyword description
    - BUG/MEDIUM: sample/jwt: fix another instance of base64 error detection
    - BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on back
    - BUG/MINOR: sample: Fix 'fix_tag_value' sample when waiting for more data
    - DOC: config: Move 'tcp-response content' at the right place
    - BUG/MINOR: proxy: Use .disabled field as a bitfield as documented
    - MINOR: proxy: Introduce proxy flags to replace disabled bitfield
    - MINOR: sample/arg: Be able to resolve args found in defaults sections
    - MEDIUM: proxy: Warn about ambiguous use of named defaults sections
    - MINOR: proxy: Be able to reference the defaults section used by a proxy
    - MINOR: proxy: Add PR_FL_READY flag on fully configured and usable proxies
    - MINOR: config: Finish configuration for referenced default proxies
    - MINOR: config: No longer remove previous anonymous defaults section
    - MINOR: tcpcheck: Support 2-steps args resolution in defaults sections
    - MEDIUM: rules/acl: Parse TCP/HTTP rules and acls defined in defaults sections
    - MEDIUM: tcp-rules: Eval TCP rules defined in defaults sections
    - MEDIUM: http-ana: Eval HTTP rules defined in defaults sections
    - BUG/MEDIUM: sample: Cumulate frontend and backend sample validity flags
    - REGTESTS: Add scripts to test support of TCP/HTTP rules in defaults sections
    - DOC: config: Add documentation about TCP/HTTP rules in defaults section
    - DOC: config: Rework and uniformize how TCP/HTTP rules are documented
    - BUG/MINOR: proxy: Release ACLs and TCP/HTTP rules of default proxies
    - BUG/MEDIUM: cpuset: fix cpuset size for FreeBSD
    - BUG/MINOR: sample: fix backend direction flags consecutive to last fix
    - BUG/MINOR: listener: fix incorrect return on out-of-memory
    - BUG/MINOR: listener: add an error check for unallocatable trash
    - CLEANUP: listeners: remove unreachable code in clone_listener()
2021-10-16 15:24:22 +02:00
Christopher Faulet
71d1892190 DOC: config: Rework and uniformize how TCP/HTTP rules are documented
Now all these rules are documented using the same structure. First there is
a general description with the list of all supported actions. Then all
actions are described in details. Thus, it is easy to have a quick list of
all supported actions and this avoids to have a huge description with all
info about these actions. In addition, when it is possible, we make a
reference to already documented parts.
2021-10-15 14:21:32 +02:00
Christopher Faulet
6e0425b718 DOC: config: Add documentation about TCP/HTTP rules in defaults section
Documentation of each directive that can now be used in defaults section was
updated to explain how it works. A special mark was added to specify when a
keyword is supported by defaults sections with a name but not anonymous
ones. In this case an exclamation mark is added.
2021-10-15 14:13:14 +02:00
Christopher Faulet
b40542000d MEDIUM: proxy: Warn about ambiguous use of named defaults sections
It is now possible to designate the defaults section to use by adding a name
of the corresponding defaults section and referencing it in the desired
proxy section. However, this introduces an ambiguity. This named defaults
section may still be implicitly used by other proxies if it is the last one
defined. In this case for instance:

  default common
    ...

  default frt from common
    ...

  default bck from common
    ...

  frontend fe from frt
    ...

  backend be from bck
    ...

  listen stats
    ...

Here, it is not really obvious the last section will use the 'bck' defaults
section. And it is probably not the expected behaviour. To help users to
properly configure their haproxy, a warning is now emitted if a defaults
section is explicitly AND implicitly used. The configuration manual was
updated accordingly.

Because this patch adds a warning, it should probably not be backported to
2.4. However, if is is backported, it depends on commit "MINOR: proxy:
Introduce proxy flags to replace disabled bitfield".
2021-10-15 14:12:19 +02:00