Commit Graph

1624 Commits

Author SHA1 Message Date
Willy Tarreau
c8cac04bd5 MEDIUM: listener: deprecate "process" in favor of "thread" on bind lines
The "process" directive on "bind" lines becomes quite confusing considering
that the only allowed value is 1 for the process, and that threads are
optional and come after the mandatory "1/".

Let's introduce a new "thread" directive to directly configure thread
numbers, and mark "process" as deprecated. Now "process" will emit a
warning and will suggest how to be replaced with "thread" instead.
The doc was updated accordingly (mostly a copy-paste of the previous
description which was already up to date).

This is marked as MEDIUM as it will impact users having "zero-warning"
and "process" specified.
2021-09-21 14:35:42 +02:00
Willy Tarreau
4b3a9fefab [RELEASE] Released version 2.5-dev7
Released version 2.5-dev7 with the following main changes :
    - BUG/MINOR: config: reject configs using HTTP with bufsize >= 256 MB
    - CLEANUP: htx: remove comments about "must be < 256 MB"
    - BUG/MAJOR: htx: fix missing header name length check in htx_add_header/trailer
    - Revert "BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive"
    - MINOR: proxy: add a global "grace" directive to postpone soft-stop
    - MINOR: vars: rename vars_init() to vars_init_head()
    - CLEANUP: vars: rename sample_clear_stream() to var_unset()
    - REORG: vars: remerge sample_store{,_stream}() into var_set()
    - MEDIUM: vars: make the ifexist variant of set-var only apply to the proc scope
    - MINOR: vars: add a VF_CREATEONLY flag for creation
    - MINOR: vars: support storing empty sample data with a variable
    - MINOR: vars: store flags into variables and add VF_PERMANENT
    - MEDIUM: vars: make var_clear() only reset VF_PERMANENT variables
    - MEDIUM: vars: pre-create parsed SCOPE_PROC variables as permanent ones
    - MINOR: vars: preset a random seed to hash variables names
    - MEDIUM: vars: replace the global name index with a hash
    - CLEANUP: vars: remove the now unused var_names array
    - MINOR: vars: centralize the lock/unlock into static inlines
    - OPTIM: vars: only takes the variables lock on shared entries
    - OPTIM: vars: remove internal bookkeeping for vars_global_size
    - OPTIM: vars: do not keep variables usage stats if no limit is set
    - BUILD: fix dragonfly build again on __read_mostly
    - CI: Github Actions: temporarily disable Opentracing
    - BUG/MEDIUM: mux-h1: Remove "Upgrade:" header for requests with payload
    - MINOR: htx: Skip headers with no value when adding a header list to a message
    - CLEANUP: mux-h1: Remove condition rejecting upgrade requests with payload
    - BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is reached
    - BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
    - BUG/MINOR: filters: Set right FLT_END analyser depending on channel
    - CLEANUP: Add haproxy/xxhash.h to avoid modifying import/xxhash.h
    - CLEANUP: ebmbtree: Replace always-taken elseif by else
    - CLEANUP: Move XXH3 macro from haproxy/compat.h to haproxy/xxhash.h
    - BUILD: opentracing: exclude the use of haproxy variables for the OpenTracing context
    - BUG/MINOR: opentracing: enable the use of http headers without a set value
    - CLEANUP: opentracing: use the haproxy function to generate uuid
    - MINOR: opentracing: change the scope of the variable 'ot.uuid' from 'sess' to 'txn'
    - CI: Github Actions: re-enable Opentracing
    - CLEANUP: opentracing: simplify the condition on the empty header
    - BUG/MEDIUM lua: Add missing call to RESET_SAFE_LJMP in hlua_filter_new()
2021-09-12 11:36:38 +02:00
Willy Tarreau
10080716bf MINOR: proxy: add a global "grace" directive to postpone soft-stop
In ticket #1348 some users expressed some concerns regarding the removal
of the "grace" directive from the proxies. Their use case very closely
mimmicks the original intent of the grace keyword, which is, let haproxy
accept traffic for some time when stopping, while indicating an external
LB that it's stopping.

This is implemented here by starting a task whose expiration triggers
the soft-stop for real. The global "stopping" variable is immediately
set however. For example, this below will be sufficient to instantly
notify an external check on port 9999 that the service is going down,
while other services remain active for 10s:

    global
      grace 10s

    frontend ext-check
      bind :9999
      monitor-uri /ext-check
      monitor fail if { stopping }
2021-09-07 17:34:29 +02:00
Willy Tarreau
f653e83064 [RELEASE] Released version 2.5-dev6
Released version 2.5-dev6 with the following main changes :
    - BUG/MINOR threads: Use get_(local|gm)time instead of (local|gm)time
    - BUG/MINOR: tools: Fix loop condition in dump_text()
    - BUILD: ssl: next round of build warnings on LIBRESSL_VERSION_NUMBER
    - BUILD: ssl: fix two remaining occurrences of #if USE_OPENSSL
    - BUILD: tools: properly guard __GLIBC__ with defined()
    - BUILD: globally enable -Wundef
    - MINOR: log: Remove log-error-via-logformat option
    - MINOR: log: Add new "error-log-format" option
    - BUG/MAJOR: queue: better protect a pendconn being picked from the proxy
    - CLEANUP: Add missing include guard to signal.h
    - MINOR: ssl: Add new ssl_bc_hsk_err sample fetch
    - MINOR: connection: Add a connection error code sample fetch for backend side
    - REGTESTS: ssl: Add tests for bc_conn_err and ssl_bc_hsk_err sample fetches
    - MINOR: http-rules: add a new "ignore-empty" option to redirects.
    - CI: Github Actions: temporarily disable BoringSSL builds
    - BUG/MINOR: vars: fix set-var/unset-var exclusivity in the keyword parser
    - BUG/MINOR: vars: improve accuracy of the rules used to check expression validity
    - MINOR: sample: add missing ARGC_ entries
    - BUG/MINOR: vars: properly set the argument parsing context in the expression
    - DOC: configuration: remove wrong tcp-request examples in tcp-response
    - MEDIUM: vars: add a new "set-var-fmt" action
    - BUG/MEDIUM: vars: run over the correct list in release_store_rules()
    - BUG/MINOR: vars: truncate the variable name in error reports about scope.
    - BUG/MINOR: vars: do not talk about global section in CLI errors for set-var
    - CLEANUP: vars: name the temporary proxy "CFG" instead of "CLI" for global vars
    - MINOR: log: make log-format expressions completely usable outside of req/resp
    - MINOR: vars: add a "set-var-fmt" directive to the global section
    - MEDIUM: vars: also support format strings in CLI's "set var" command
    - CLEANUP: vars: factor out common code from vars_get_by_{desc,name}
    - MINOR: vars: make vars_get_by_* support an optional default value
    - MINOR: vars: make the vars() sample fetch function support a default value
    - BUILD: ot: add argument for default value to vars_get_by_name()
2021-09-03 15:19:56 +02:00
Willy Tarreau
54496a6a5b MINOR: vars: make the vars() sample fetch function support a default value
It is quite common to see in configurations constructions like the
following one:

    http-request set-var(txn.bodylen) 0
    http-request set-var(txn.bodylen) req.hdr(content-length)
    ...
    http-request set-header orig-len %[var(txn.bodylen)]

The set-var() rules are almost always duplicated when manipulating
integers or any other value that is mandatory along operations. This is
a problem because it makes the configurations complicated to maintain
and slower than needed. And it becomes even more complicated when several
conditions may set the same variable because the risk of forgetting to
initialize it or to accidentally reset it is high.

This patch extends the var() sample fetch function to take an optional
argument which contains a default value to be returned if the variable
was not set. This way it becomes much simpler to use the variable, just
set it where needed, and read it with a fall back to the default value:

    http-request set-var(txn.bodylen) req.hdr(content-length)
    ...
    http-request set-header orig-len %[var(txn.bodylen,0)]

The default value is always passed as a string, thus it will experience
a cast to the output type. It doesn't seem userful to complicate the
configuration to pass an explicit type at this point.

The vars.vtc regtest was updated accordingly.
2021-09-03 12:08:54 +02:00
Willy Tarreau
753d4db5f3 MINOR: vars: add a "set-var-fmt" directive to the global section
Just like the set-var-fmt action for tcp/http rules, the set-var-fmt
directive in global sections allows to pre-set process-wide variables
using a format string instead of a sample expression. This is often
more convenient when it is required to concatenate multiple fields,
or when emitting just one word.
2021-09-03 11:01:48 +02:00
Willy Tarreau
9a621ae76d MEDIUM: vars: add a new "set-var-fmt" action
The set-var() action is convenient because it preserves the input type
but it's a pain to deal with when trying to concatenate values. The
most recurring example is when it's needed to build a variable composed
of the source address and the source port. Usually it ends up like this:

    tcp-request session set-var(sess.port) src_port
    tcp-request session set-var(sess.addr) src,concat(":",sess.port)

This is even worse when trying to aggregate multiple fields from stick-table
data for example. Due to this a lot of users instead abuse headers from HTTP
rules:

    http-request set-header(x-addr) %[src]:%[src_port]

But this requires some careful cleanups to make sure they won't leak, and
it's significantly more expensive to deal with. And generally speaking it's
not clean. Plus it must be performed for each and every request, which is
expensive for this common case of ip+port that doesn't change for the whole
session.

This patch addresses this limitation by implementing a new "set-var-fmt"
action which performs the same work as "set-var" but takes a format string
in argument instead of an expression. This way it becomes pretty simple to
just write:

    tcp-request session set-var-fmt(sess.addr) %[src]:%[src_port]

It is usable in all rulesets that already support the "set-var" action.
It is not yet implemented for the global "set-var" directive (which already
takes a string) and the CLI's "set var" command, which would definitely
benefit from it but currently uses its own parser and engine, thus it
must be reworked.

The doc and regtests were updated.
2021-09-02 21:22:22 +02:00
Willy Tarreau
e7267120d5 DOC: configuration: remove wrong tcp-request examples in tcp-response
There is a massive abuse of copy-paste in the doc that is visible in
the examples and arguments declaration. Let's at least remove irrelevant
examples for now.
2021-09-02 20:51:21 +02:00
Willy Tarreau
bc1223be79 MINOR: http-rules: add a new "ignore-empty" option to redirects.
Sometimes it is convenient to remap large sets of URIs to new ones (e.g.
after a site migration for example). This can be achieved using
"http-request redirect" combined with maps, but one difficulty there is
that non-matching entries will return an empty response. In order to
avoid this, duplicating the operation as an ACL condition ending in
"-m found" is possible but it becomes complex and error-prone while it's
known that an empty URL is not valid in a location header.

This patch addresses this by improving the redirect rules to be able to
simply ignore the rule and skip to the next one if the result of the
evaluation of the "location" expression is empty. However in order not
to break existing setups, it requires a new "ignore-empty" keyword.

There used to be an ACT_FLAG_FINAL on redirect rules that's used during
the parsing to emit a warning if followed by another rule, so here we
only set it if the option is not there. The http_apply_redirect_rule()
function now returns a 3rd value to mention that it did nothing and
that this was not an error, so that callers can just ignore the rule.
The regular "redirect" rules were not modified however since this does
not apply there.

The map_redirect VTC was completed with such a test and updated to 2.5
and an example was added into the documentation.
2021-09-02 17:06:18 +02:00
Remi Tricot-Le Breton
942c167229 MINOR: connection: Add a connection error code sample fetch for backend side
The bc_conn_err and bc_conn_err_str sample fetches give the status of
the connection on the backend side. The error codes and error messages
are the same than the ones that can be raised by the fc_conn_err fetch.
2021-09-01 22:55:54 +02:00
Remi Tricot-Le Breton
163cdeba37 MINOR: ssl: Add new ssl_bc_hsk_err sample fetch
This new sample fetch along the ssl_bc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the backend's perspective).
2021-09-01 22:55:39 +02:00
Remi Tricot-Le Breton
fe21fe76bd MINOR: log: Add new "error-log-format" option
This option can be used to define a specific log format that will be
used in case of error, timeout, connection failure on a frontend... It
will be used for any log line concerned by the log-separate-errors
option. It will also replace the format of specific error messages
decribed in section 8.2.6.
If no "error-log-format" is defined, the legacy error messages are still
emitted and the other error logs keep using the regular log-format.
2021-08-31 12:13:08 +02:00
Remi Tricot-Le Breton
3d6350e108 MINOR: log: Remove log-error-via-logformat option
This option will be replaced by a "error-log-format" that enables to use
a dedicated log-format for connection error messages instead of the
regular log-format (in which most of the fields would be invalid in such
a case).
The "log-error-via-logformat" mechanism will then be replaced by a test
on the presence of such an error log format or not. If a format is
defined, it is used for connection error messages, otherwise the legacy
error log format is used.
2021-08-31 12:13:06 +02:00
Willy Tarreau
446344ccef [RELEASE] Released version 2.5-dev5
Released version 2.5-dev5 with the following main changes :
    - MINOR: httpclient: initialize the proxy
    - MINOR: httpclient: implement a simple HTTP Client API
    - MINOR: httpclient/cli: implement a simple client over the CLI
    - MINOR: httpclient/cli: change the User-Agent to "HAProxy"
    - MEDIUM: ssl: Keep a reference to the client's certificate for use in logs
    - BUG/MEDIUM: h2: match absolute-path not path-absolute for :path
    - BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
    - MINOR: server: check if srv is NULL in free_server()
    - MINOR: proxy: check if p is NULL in free_proxy()
    - BUG/MEDIUM: cfgparse: do not allocate IDs to automatic internal proxies
    - BUG/MINOR: http_client: make sure to preset the proxy's default settings
    - REGTESTS: http_upgrade: fix incorrect expectation on TCP->H1->H2
    - REGTESTS: abortonclose: after retries, 503 is expected, not close
    - REGTESTS: server: fix agent-check syntax and expectation
    - BUG/MINOR: httpclient: fix uninitialized sl variable
    - BUG/MINOR: httpclient/cli: change the appctx test in the callbacks
    - BUG/MINOR: httpclient: check if hdr_num is not 0
    - MINOR: httpclient: cleanup the include files
    - MINOR: hlua: take the global Lua lock inside a global function
    - MINOR: tools: add FreeBSD support to get_exec_path()
    - BUG/MINOR: systemd: ExecStartPre must use -Ws
    - MINOR: systemd: remove the ExecStartPre line in the unit file
    - MINOR: ssl: add an openssl version string parser
    - MINOR: cfgcond: implements openssl_version_atleast and openssl_version_before
    - CLEANUP: ssl: remove useless check on p in openssl_version_parser()
    - BUG/MINOR: stick-table: fix the sc-set-gpt* parser when using expressions
    - BUG/MINOR: httpclient: remove deinit of the httpclient
    - BUG/MEDIUM: base64: check output boundaries within base64{dec,urldec}
    - MINOR: httpclient: set verify none on the https server
    - MINOR: httpclient: add the server to the proxy
    - BUG/MINOR: httpclient: fix Host header
    - BUILD: httpclient: fix build without OpenSSL
    - CI: github-actions: remove obsolete options
    - CLEANUP: assorted typo fixes in the code and comments
    - MINOR: proc: setting the process to produce a core dump on FreeBSD.
    - BUILD: adopt script/build-ssl.sh for OpenSSL-3.0.0beta2
    - MINOR: server: return the next srv instance on free_server
    - BUG/MINOR: stats: use refcount to protect dynamic server on dump
    - MEDIUM: server: extend refcount for all servers
    - MINOR: server: define non purgeable server flag
    - MINOR: server: mark referenced servers as non purgeable
    - MINOR: server: mark servers referenced by LUA script as non purgeable
    - MEDIUM: server: allow to remove servers at runtime except non purgeable
    - BUG/MINOR: base64: base64urldec() ignores padding in output size check
    - REGTEST: add missing lua requirements on server removal test
    - REGTEST: fix haproxy required version for server removal test
    - BUG/MINOR: proxy: don't dump servers of internal proxies
    - REGTESTS: Use `feature cmd` for 2.5+ tests
    - REGTESTS: Remove REQUIRE_VERSION=1.5 from all tests
    - BUG/MINOR: resolvers: mark servers with name-resolution as non purgeable
    - MINOR: compiler: implement an ONLY_ONCE() macro
    - BUG/MINOR: lua: use strlcpy2() not strncpy() to copy sample keywords
    - MEDIUM: ssl: Capture more info from Client Hello
    - MINOR: sample: Expose SSL captures using new fetchers
    - MINOR: sample: Add be2dec converter
    - MINOR: sample: Add be2hex converter
    - MEDIUM: config: Deprecate tune.ssl.capture-cipherlist-size
    - BUG/MINOR: time: fix idle time computation for long sleeps
    - MINOR: time: add report_idle() to report process-wide idle time
    - BUG/MINOR: ebtree: remove dependency on incorrect macro for bits per long
    - BUILD: activity: use #ifdef not #if on USE_MEMORY_PROFILING
    - BUILD/MINOR: defaults: eliminate warning on MAXHOSTNAMELEN with -Wundef
    - BUILD/MINOR: ssl: avoid a build warning on LIBRESSL_VERSION with -Wundef
    - IMPORT: slz: silence a build warning with -Wundef
    - BUILD/MINOR: regex: avoid a build warning on USE_PCRE2 with -Wundef
2021-08-28 13:46:11 +02:00
Marcin Deranek
310a260e4a MEDIUM: config: Deprecate tune.ssl.capture-cipherlist-size
Deprecate tune.ssl.capture-cipherlist-size in favor of
tune.ssl.capture-buffer-size which better describes the purpose of the
setting.
2021-08-26 19:52:04 +02:00
Marcin Deranek
da0264a968 MINOR: sample: Add be2hex converter
Add be2hex converter to convert big-endian binary data into hex string
with optional string separators.
2021-08-26 19:48:34 +02:00
Marcin Deranek
40ca09c7bb MINOR: sample: Add be2dec converter
Add be2dec converter which allows to build JA3 compatible TLS
fingerprints by converting big-endian binary data into string
separated unsigned integers eg.

http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
    %[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
    %[ssl_fc_extlist_bin(1),be2dec(-,2)],\
    %[ssl_fc_eclist_bin(1),be2dec(-,2)],\
    %[ssl_fc_ecformats_bin,be2dec(-,1)]
2021-08-26 19:48:34 +02:00
Marcin Deranek
959a48c116 MINOR: sample: Expose SSL captures using new fetchers
To be able to provide JA3 compatible TLS Fingerprints we need to expose
all Client Hello captured data using fetchers. Patch provides new
and modifies existing fetchers to add ability to filter out GREASE values:
- ssl_fc_cipherlist_*
- ssl_fc_ecformats_bin
- ssl_fc_eclist_bin
- ssl_fc_extlist_bin
- ssl_fc_protocol_hello_id
2021-08-26 19:48:34 +02:00
Marcin Deranek
769fd2e447 MEDIUM: ssl: Capture more info from Client Hello
When we set tune.ssl.capture-cipherlist-size to a non-zero value
we are able to capture cipherlist supported by the client. To be able to
provide JA3 compatible TLS fingerprinting we need to capture more
information from Client Hello message:
- SSL Version
- SSL Extensions
- Elliptic Curves
- Elliptic Curve Point Formats
This patch allows HAProxy to capture such information and store it for
later use.
2021-08-26 19:48:33 +02:00
Willy Tarreau
08d0f23138 [RELEASE] Released version 2.5-dev4
Released version 2.5-dev4 with the following main changes :
    - MINOR: log: rename 'dontloglegacyconnerr' to 'log-error-via-logformat'
    - MINOR: doc: rename conn_status in `option httsplog`
    - MINOR: proxy: disabled takes a stopping and a disabled state
    - MINOR: stats: shows proxy in a stopped state
    - BUG/MINOR: server: fix race on error path of 'add server' CLI if track
    - CLEANUP: thread: fix fantaisist indentation of thread_harmless_till_end()
    - MINOR: threads: make thread_release() not wait for other ones to complete
    - MEDIUM: threads: add a stronger thread_isolate_full() call
    - MEDIUM: servers: make the server deletion code run under full thread isolation
    - BUG/MINOR: server: remove srv from px list on CLI 'add server' error
    - MINOR: activity/fd: remove the dead_fd counter
    - MAJOR: fd: get rid of the DWCAS when setting the running_mask
    - CLEANUP: fd: remove the now unused fd_set_running()
    - CLEANUP: fd: remove the now unneeded fd_mig_lock
    - BUG/MINOR: server: update last_change on maint->ready transitions too
    - MINOR: spoe: Add a pointer on the filter config in the spoe_agent structure
    - BUG/MEDIUM: spoe: Create a SPOE applet if necessary when the last one is released
    - BUG/MEDIUM: spoe: Fix policy to close applets when SPOE connections are queued
    - MINOR: server: unmark deprecated on enable health/agent cli
    - MEDIUM: task: implement tasklet kill
    - MINOR: server: initialize fields for dynamic server check
    - MINOR: check: allocate default check ruleset for every backends
    - MINOR: check: export check init functions
    - MINOR: check: do not increment global maxsock at runtime
    - MINOR: server: implement a refcount for dynamic servers
    - MEDIUM: check: implement check deletion for dynamic servers
    - MINOR: check: enable safe keywords for dynamic servers
    - MEDIUM: server: implement check for dynamic servers
    - MEDIUM: server: implement agent check for dynamic servers
    - REGTESTS: server: add dynamic check server test
    - MINOR: doc: specify ulimit-n usage for dynamic servers
    - REGTESTS: server: fix dynamic server with checks test
    - CI: travis-ci: temporarily disable arm64 builds
    - BUG/MINOR: check: test if server is not null in purge
    - MINOR: global: define MODE_STOPPING
    - BUG/MINOR: server: do not use refcount in free_server in stopping mode
    - ADMIN: dyncookie: implement a simple dynamic cookie calculator
    - BUG/MINOR: check: do not reset check flags on purge
    - BUG/MINOR: check: fix leak on add dynamic server with agent-check error
    - BUG/MEDIUM: check: fix leak on agent-check purge
    - BUG/MEDIUM: server: support both check/agent-check on a dynamic instance
    - BUG/MINOR: buffer: fix buffer_dump() formatting
    - MINOR: channel: remove an htx block from a channel
    - BUG/MINOR: tcpcheck: Properly detect pending HTTP data in output buffer
    - BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
    - MINOR: lua: Add a flag on lua context to know the yield capability at run time
    - BUG/MINOR: lua: Yield in channel functions only if lua context can yield
    - BUG/MINOR: lua: Don't yield in channel.append() and channel.set()
    - MINOR: filters/lua: Release filters before the lua context
    - MINOR: lua: Add a function to get a reference on a table in the stack
    - MEDIUM: lua: Process buffer data using an offset and a length
    - MEDIUM: lua: Improve/revisit the lua api to manipulate channels
    - DOC: Improve the lua documentation
    - MEDIUM: filters/lua: Add support for dummy filters written in lua
    - MINOR: lua: Add a function to get a filter attached to a channel class
    - MINOR: lua: Add flags on the lua TXN to know the execution context
    - MEDIUM: filters/lua: Be prepared to filter TCP payloads
    - MEDIUM: filters/lua: Support declaration of some filter callback functions in lua
    - MEDIUM: filters/lua: Add HTTPMessage class to help HTTP filtering
    - MINOR: filters/lua: Add request and response HTTP messages in the lua TXN
    - MINOR: filters/lua: Support the HTTP filtering from filters written in lua
    - DOC: config: Fix 'http-response send-spoe-group' documentation
    - BUG/MINOR: lua: Properly check negative offset in Channel/HttpMessage functions
    - BUG/MINOR: lua: Properly catch alloc errors when parsing lua filter directives
    - BUG/MEDIUM: cfgcheck: verify existing log-forward listeners during config check
    - MINOR: cli: delare the CLI frontend as an internal proxy
    - MINOR: proxy: disable warnings for internal proxies
    - BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag is set
    - BUG/MINOR: lua/filters: Return right code when txn:done() is called
    - DOC: lua-api: Add documentation about lua filters
    - CI: Remove obsolete USE_SLZ=1 CI job
    - CLEANUP: assorted typo fixes in the code and comments
    - CI: github actions: relax OpenSSL-3.0.0 version comparision
    - BUILD: tools: get the absolute path of the current binary on NetBSD.
    - DOC: Minor typo fix - 'question mark' -> 'exclamation mark'
    - DOC/MINOR: fix typo in management document
    - MINOR: http: add a new function http_validate_scheme() to validate a scheme
    - BUG/MAJOR: h2: verify early that non-http/https schemes match the valid syntax
    - BUG/MAJOR: h2: verify that :path starts with a '/' before concatenating it
    - BUG/MAJOR: h2: enforce stricter syntax checks on the :method pseudo-header
    - BUG/MEDIUM: h2: give :authority precedence over Host
    - REGTESTS: add a test to prevent h2 desync attacks
2021-08-17 14:08:55 +02:00
Kunal Gangakhedkar
d0bacde843 DOC: Minor typo fix - 'question mark' -> 'exclamation mark'
Signed-off-by: Kunal Gangakhedkar <kunal.gangakhedkar@gmail.com>
2021-08-17 09:57:54 +02:00
Ilya Shipitsin
01881087fc CLEANUP: assorted typo fixes in the code and comments
This is 25th iteration of typo fixes
2021-08-16 12:37:59 +02:00
Christopher Faulet
24e7f354e9 DOC: config: Fix 'http-response send-spoe-group' documentation
Arguments were missing in the rule heading. This patch may be backported as
far as 2.0.
2021-08-12 09:32:08 +02:00
Amaury Denoyelle
414a612bb3 MINOR: doc: specify ulimit-n usage for dynamic servers
Complete the documentation of the dynamic servers to warn about a
possible fd resource exhaustion when using a large number of them.
2021-08-06 11:22:01 +02:00
William Lallemand
fdc3faf654 MINOR: doc: rename conn_status in option httsplog
Rename the conn_status field by the real name of the sample fetch in the
`option httpslog` documentation.
2021-08-02 10:57:49 +02:00
William Lallemand
56f1f75715 MINOR: log: rename 'dontloglegacyconnerr' to 'log-error-via-logformat'
Rename the 'dontloglegacyconnerr' option to 'log-error-via-logformat'
which is much more self-explanatory and readable.

Note: only legacy keywords don't use hyphens, it is recommended to
separate words with them in new keywords.
2021-08-02 10:42:42 +02:00
Willy Tarreau
8441deb1e2 [RELEASE] Released version 2.5-dev3
Released version 2.5-dev3 with the following main changes :
    - BUG/MINOR: arg: free all args on make_arg_list()'s error path
    - BUG/MINOR: cfgcond: revisit the condition freeing mechanism to avoid a leak
    - MEDIUM: proxy: remove long-broken 'option http_proxy'
    - CLEANUP: http_ana: Remove now unused label from http_process_request()
    - MINOR: deinit: always deinit the init_mutex on failed initialization
    - BUG/MEDIUM: cfgcond: limit recursion level in the condition expression parser
    - BUG/MEDIUM: mworker: do not register an exit handler if exit is expected
    - BUG/MINOR: mworker: do not export HAPROXY_MWORKER_REEXEC across programs
    - BUILD/MINOR: memprof fix macOs build.
    - BUG/MEDIUM: ssl_sample: fix segfault for srv samples on invalid request
    - BUG/MINOR: stats: Add missing agent stats on servers
    - BUG/MINOR: check: fix the condition to validate a port-less server
    - BUILD: threads: fix pthread_mutex_unlock when !USE_THREAD
    - BUG/MINOR: resolvers: Use a null-terminated string to lookup in servers tree
    - MINOR: ssl: use __objt_* variant when retrieving counters
    - BUG/MINOR: systemd: must check the configuration using -Ws
    - BUG/MINOR: mux-h1: Obey dontlognull option for empty requests
    - BUG/MINOR: mux-h2: Obey dontlognull option during the preface
    - BUG/MINOR: mux-h1: Be sure to swap H1C to splice mode when rcv_pipe() is called
    - BUG/MEDIUM: mux-h2: Handle remaining read0 cases on partial frames
    - MINOR: proxy: rename PR_CAP_LUA to PR_CAP_INT
    - MINOR: mworker: the mworker CLI proxy is internal
    - MINOR: stats: don't output internal proxies (PR_CAP_INT)
    - CLEANUP: mworker: use the proxy helper functions in mworker_cli_proxy_create()
    - CLEANUP: mworker: PR_CAP already initialized with alloc_new_proxy()
    - BUG/MINOR: connection: Add missing error labels to conn_err_code_str
    - MINOR: connection: Add a connection error code sample fetch
    - MINOR: ssl: Enable error fetches in case of handshake error
    - MINOR: ssl: Add new ssl_fc_hsk_err sample fetch
    - MINOR: ssl: Define a default https log format
    - MEDIUM: connection: Add option to disable legacy error log
    - REGTESTS: ssl: Add tests for the connection and SSL error fetches
    - REGTESTS: ssl: ssl_errors.vtc does not work with old openssl version
    - BUG/MEDIUM: connection: close a rare race between idle conn close and takeover
    - BUG/MEDIUM: pollers: clear the sleeping bit after waking up, not before
    - BUG/MINOR: select: fix excess number of dead/skip reported
    - BUG/MINOR: poll: fix abnormally high skip_fd counter
    - BUG/MINOR: pollers: always program an update for migrated FDs
    - BUG/MINOR: fd: protect fd state harder against a concurrent takeover
    - DOC: internals: document the FD takeover process
    - MINOR: fd: update flags only once in fd_update_events()
    - MINOR: poll/epoll: move detection of RDHUP support earlier
    - REORG: fd: uninline fd_update_events()
    - MEDIUM: fd: rely more on fd_update_events() to detect changes
    - BUG/MINOR: freq_ctr: use stricter barriers between updates and readings
    - MEDIUM: atomic: simplify the atomic load/store/exchange operations
    - MEDIUM: atomic: relax the load/store barriers on x86_64
    - BUILD: opentracing: fixed build when using pkg-config utility
2021-08-01 18:19:51 +02:00
Remi Tricot-Le Breton
4a6328f066 MEDIUM: connection: Add option to disable legacy error log
In case of connection failure, a dedicated error message is output,
following the format described in section "Error log format" of the
documentation. These messages cannot be configured through a log-format
option.
This patch adds a new option, "dontloglegacyconnerr", that disables
those error logs when set, and "replaces" them by a regular log line
that follows the configured log-format (thanks to a call to sess_log in
session_kill_embryonic).
The new fc_conn_err sample fetch allows to add the legacy error log
information into a regular log format.
This new option is unset by default so the logging logic will remain the
same until this new option is used.
2021-07-29 15:40:45 +02:00
Remi Tricot-Le Breton
98b930d043 MINOR: ssl: Define a default https log format
This patch adds a new httpslog option and a new HTTP over SSL log-format
that expands the default HTTP format and adds SSL specific information.
2021-07-29 15:40:45 +02:00
Remi Tricot-Le Breton
7c6898ee49 MINOR: ssl: Add new ssl_fc_hsk_err sample fetch
This new sample fetch along the ssl_fc_hsk_err_str fetch contain the
last SSL error of the error stack that occurred during the SSL
handshake (from the frontend's perspective). The errors happening during
the client's certificate verification will still be given by the
ssl_c_err and ssl_c_ca_err fetches. This new fetch will only hold errors
retrieved by the OpenSSL ERR_get_error function.
2021-07-29 15:40:45 +02:00
Remi Tricot-Le Breton
3d2093af9b MINOR: connection: Add a connection error code sample fetch
The fc_conn_err and fc_conn_err_str sample fetches give information
about the problem that made the connection fail. This information would
previously only have been given by the error log messages meaning that
thanks to these fetches, the error log can now be included in a custom
log format. The log strings were all found in the conn_err_code_str
function.
2021-07-29 15:40:45 +02:00
Willy Tarreau
252412316e MEDIUM: proxy: remove long-broken 'option http_proxy'
This option had always been broken in HTX, which means that the first
breakage appeared in 1.9, that it was broken by default in 2.0 and that
no workaround existed starting with 2.1. The way this option works is
praticularly unfit to the rest of the configuration and to the internal
architecture. It had some uses when it was introduced 14 years ago but
nowadays it's possible to do much better and more reliable using a
set of "http-request set-dst" and "http-request set-uri" rules, which
additionally are compatible with DNS resolution (via do-resolve) and
are not exclusive to normal load balancing. The "option-http_proxy"
example config file was updated to reflect this.

The option is still parsed so that an error message gives hints about
what to look for.
2021-07-18 19:35:32 +02:00
Willy Tarreau
bccc91d33e [RELEASE] Released version 2.5-dev2
Released version 2.5-dev2 with the following main changes :
    - BUILD/MEDIUM: tcp: set-mark support for OpenBSD
    - DOC: config: use CREATE USER for mysql-check
    - BUG/MINOR: stick-table: fix several printf sign errors dumping tables
    - BUG/MINOR: peers: fix data_type bit computation more than 32 data_types
    - MINOR: stick-table: make skttable_data_cast to use only std types
    - MEDIUM: stick-table: handle arrays of standard types into stick-tables
    - MEDIUM: peers: handle arrays of std types in peers protocol
    - DOC: stick-table: add missing documentation about gpt0 stored type
    - MEDIUM: stick-table: add the new array of gpt data_type
    - MEDIUM: stick-table: make the use of 'gpt' excluding the use of 'gpt0'
    - MEDIUM: stick-table: add the new arrays of gpc and gpc_rate
    - MEDIUM: stick-table: make the use of 'gpc' excluding the use of 'gpc0/1''
    - BUG/MEDIUM: sock: make sure to never miss early connection failures
    - BUG/MINOR: cli: fix server name output in "show fd"
    - Revert "MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules"
    - MEDIUM: stats: include disabled proxies that hold active sessions to stats
    - BUILD: stick-table: shut up invalid "uninitialized" warning in gcc 8.3
    - MINOR: http: implement http_get_scheme
    - MEDIUM: http: implement scheme-based normalization
    - MEDIUM: h1-htx: apply scheme-based normalization on h1 requests
    - MEDIUM: h2: apply scheme-based normalization on h2 requests
    - REGTESTS: add http scheme-based normalization test
    - BUILD: http_htx: fix ci compilation error with isdigit for Windows
    - MINOR: http: implement http uri parser
    - MINOR: http: use http uri parser for scheme
    - MINOR: http: use http uri parser for authority
    - REORG: http_ana: split conditions for monitor-uri in wait for request
    - MINOR: http: use http uri parser for path
    - BUG/MEDIUM: http_ana: fix crash for http_proxy mode during uri rewrite
    - MINOR: mux_h2: define config to disable h2 websocket support
    - CLEANUP: applet: remove unused thread_mask
    - BUG/MINOR: ssl: Default-server configuration ignored by server
    - BUILD: add detection of missing important CFLAGS
    - BUILD: lua: silence a build warning with TCC
    - MINOR: srv: extract tracking server config function
    - MINOR: srv: do not allow to track a dynamic server
    - MEDIUM: server: support track keyword for dynamic servers
    - REGTESTS: test track support for dynamic servers
    - MINOR: init: verify that there is a single word on "-cc"
    - MINOR: init: make -cc support environment variables expansion
    - MINOR: arg: add a free_args() function to free an args array
    - CLEANUP: config: use free_args() to release args array in cfg_eval_condition()
    - CLEANUP: hlua: use free_args() to release args arrays
    - REORG: config: move the condition preprocessing code to its own file
    - MINOR: cfgcond: start to split the condition parser to introduce terms
    - MEDIUM: cfgcond: report invalid trailing chars after expressions
    - MINOR: cfgcond: remerge all arguments into a single line
    - MINOR: cfgcond: support negating conditional expressions
    - MINOR: cfgcond: make the conditional term parser automatically allocate nodes
    - MINOR: cfgcond: insert an expression between the condition and the term
    - MINOR: cfgcond: support terms made of parenthesis around expressions
    - REGTEST: make check_condition.vtc fail as soon as possible
    - REGTESTS: add more complex check conditions to check_conditions.vtc
    - BUG/MEDIUM: init: restore behavior of command-line "-m" for memory limitation
2021-07-17 12:35:11 +02:00
Willy Tarreau
316ea7ede5 MINOR: cfgcond: support terms made of parenthesis around expressions
Now it's possible to form a term using parenthesis around an expression.
This will soon allow to build more complex expressions. For now they're
still pretty limited but parenthesis do work.
2021-07-16 19:18:41 +02:00
Willy Tarreau
ca81887599 MINOR: cfgcond: insert an expression between the condition and the term
Now evaluating a condition will rely on an expression (or an empty string),
and this expression will support ORing a sub-expression with another
optional expression. The sub-expressions ANDs a term with another optional
sub-expression. With this alone precedence between && and || is respected,
and the following expression:

     A && B && C || D || E && F || G

will naturally evaluate as:

     (A && B && C) || D || (E && F) || G
2021-07-16 19:18:41 +02:00
Willy Tarreau
ca56d3d28b MINOR: cfgcond: support negating conditional expressions
Now preceeding a config condition term with "!" will simply negate it.
Example:

   .if !feature(OPENSSL)
       .alert "SSL support is mandatory"
   .endif
2021-07-16 19:18:41 +02:00
Amaury Denoyelle
befeae88e8 MINOR: mux_h2: define config to disable h2 websocket support
Define a new global config statement named
"h2-workaround-bogus-websocket-clients".

This statement will disable the automatic announce of h2 websocket
support as specified in the RFC8441. This can be use to overcome clients
which fail to implement the relatively fresh RFC8441. Clients will in
his case automatically downgrade to http/1.1 for the websocket tunnel
if the haproxy configuration allows it.

This feature is relatively simple and can be backported up to 2.4, which
saw the introduction of h2 websocket support.
2021-07-12 10:41:45 +02:00
Christopher Faulet
23048875a4 Revert "MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules"
This reverts commit 19bbbe0562.

For now, set-src/set-src-port actions are directly performed on the client
connection. Using these actions at the stream level is really a problem with
HTTP connection (See #90) because all requests are affected by this change
and not only the current request. And it is worse with the H2, because
several requests can set their source address into the same connection at
the same time.

It is already an issue when these actions are called from "http-request"
rules. It is safer to wait a bit before adding the support to "tcp-request
content" rules. The solution is to be able to set src/dst address on the
stream and not on the connection when the action if performed from the L7
level..

Reverting the above commit means the issue #1303 is no longer fixed.

This patch must be backported in all branches containing the above commit
(as far as 2.0 for now).
2021-07-06 11:44:04 +02:00
Emeric Brun
726783db18 MEDIUM: stick-table: make the use of 'gpc' excluding the use of 'gpc0/1''
This patch makes the use of 'gpc' excluding the use of the legacy
types 'gpc0' and 'gpc1" on the same table.

It also makes the use of 'gpc_rate' excluding the use of the legacy
types 'gpc0_rate' and 'gpc1_rate" on the same table.

The 'gpc0' and 'gpc1' related fetches and actions will apply
to the first two elements of the 'gpc' array if stored in table.

The 'gpc0_rate' and 'gpc1_rate' related fetches and actions will apply
to the first two elements of the 'gpc_rate' array if stored in table.
2021-07-06 07:24:42 +02:00
Emeric Brun
4d7ada8f9e MEDIUM: stick-table: add the new arrays of gpc and gpc_rate
This patch adds the definition of two new array data_types:
'gpc': This is an array of 32bits General Purpose Counters.
'gpc_rate': This is an array on increment rates of General Purpose Counters.

Like for all arrays, they are limited to 100 elements.

This patch also adds actions and fetches to handle
elements of those arrays.

Note: As documented, those new actions and fetches won't
apply to the legacy 'gpc0', 'gpc1', 'gpc0_rate' nor 'gpc1_rate'.
2021-07-06 07:24:42 +02:00
Emeric Brun
f7ab0bfb62 MEDIUM: stick-table: make the use of 'gpt' excluding the use of 'gpt0'
This patch makes the use of 'gpt' excluding the use of the legacy
type 'gpt0' on the same table.

It also makes the 'gpt0' related fetches and actions applying
to the first element of the 'gpt' array if stored in table.
2021-07-06 07:24:42 +02:00
Emeric Brun
877b0b5a7b MEDIUM: stick-table: add the new array of gpt data_type
This patch adds the definition of a new array data_type
'gpt'. This is an array of 32bits General Purpose Tags.

Like for all arrays, it is limited to 100 elements.

This patch also adds actions and fetches to handle
elements of this array.

Note: As documented, those new actions and fetches won't
apply to the legacy 'gpt0' data type.
2021-07-06 07:24:42 +02:00
Emeric Brun
1a6b7254de DOC: stick-table: add missing documentation about gpt0 stored type
The store type 'gpt0' was present in code but was not documented.

The patch fix this and should be backported since 'gpt0' is supported.
[wt: ~1.6-dev4 hence all stable]
2021-07-06 07:24:42 +02:00
Daniel Black
d3e7dc498b DOC: config: use CREATE USER for mysql-check
CREATE USER has been the standard way of creating users since
MySQL-5.0 (2005).

The current syntax of INSERT INTO mysql.user won't actually work
on MariaDB-10.4+.

Because haproxy doesn't use any resources the MySQL executable comment
syntax provides resource contraints to make it more palatable
to risk adverse users.

/*!50701 is a syntax recognised by MySQL and MariaDB 5.7.1+ when
resource contraints where added.

/*M!100201 is a MariaDB executable comment syntax recognised for MariaDB
for the 10.2.1 where the MAX_STATEMENT_TIME was added.

This patch may be backported as far as 2.0.
2021-07-05 10:53:18 +02:00
David Carlier
bae4cb2790 BUILD/MEDIUM: tcp: set-mark support for OpenBSD
set-mark support for this platform, for routing table purpose.
Follow-up from f7f53afcf9, this time for OpenBSD.
2021-07-05 10:53:18 +02:00
Willy Tarreau
96a2f50b51 [RELEASE] Released version 2.5-dev1
Released version 2.5-dev1 with the following main changes :
    - CLEANUP: ssl: Move ssl_store related code to ssl_ckch.c
    - MINOR: ssl: Allow duplicated entries in the cafile_tree
    - MEDIUM: ssl: Chain ckch instances in ca-file entries
    - MINOR: ssl: Add reference to default ckch instance in bind_conf
    - MINOR: ssl: Add helper functions to create/delete cafile entries
    - MEDIUM: ssl: Add a way to load a ca-file content from memory
    - MINOR: ssl: Add helper function to add cafile entries
    - MINOR: ssl: Ckch instance rebuild and cleanup factorization in CLI handler
    - MEDIUM: ssl: Add "set+commit ssl ca-file" CLI commands
    - REGTESTS: ssl: Add new ca-file update tests
    - MINOR: ssl: Add "abort ssl ca-file" CLI command
    - MINOR: ssl: Add a cafile_entry type field
    - MINOR: ssl: Refactorize the "show certificate details" code
    - MEDIUM: ssl: Add "show ssl ca-file" CLI command
    - MEDIUM: ssl: Add "new ssl ca-file" CLI command
    - MINOR: ssl: Add "del ssl ca-file" CLI command
    - REGTESTS: ssl: Add "new/del ssl ca-file" tests
    - DOC: ssl: Add documentation about CA file hot update commands
    - DOC: internals: update the SSL architecture schema
    - MINOR: ssl: Chain instances in ca-file entries
    - MEDIUM: ssl: Add "set+commit ssl crl-file" CLI commands
    - MEDIUM: ssl: Add "new+del crl-file" CLI commands
    - MINOR: ssl: Add "abort ssl crl-file" CLI command
    - MEDIUM: ssl: Add "show ssl crl-file" CLI command
    - REGTESTS: ssl: Add "new/del ssl crl-file" tests
    - REGTESTS: ssl: Add "set/commit ssl crl-file" test
    - DOC: ssl: Add documentation about CRL file hot update commands
    - BUILD/MINOR: ssl: Fix compilation with SSL enabled
    - BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
    - CI: introduce scripts/build-vtest.sh for installing VTest
    - CLEANUP: ssl: Fix coverity issues found in CA file hot update code
    - CI: github actions: add OpenTracing builds
    - BUG/MEDIUM: ebtree: Invalid read when looking for dup entry
    - BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
    - BUILD/MINOR: opentracing: fixed build when using clang
    - BUG/MEDIUM: filters: Exec pre/post analysers only one time per filter
    - BUG/MINOR: http-comp: Preserve HTTP_MSGF_COMPRESSIONG flag on the response
    - MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
    - CLEANUP: pattern: remove export of non-existent function pattern_delete()
    - MINOR: h1-htx: Update h1 parsing functions to return result as a size_t
    - MEDIUM: h1-htx: Adapt H1 data parsing to copy wrapping data in one call
    - MINOR: mux-h1/mux-fcgi: Don't needlessly loop on data parsing
    - MINOR: h1-htx: Move HTTP chunks parsing into a dedicated function
    - MEDIUM: h1-htx: Split function to parse a chunk and the loop on the buffer
    - MEDIUM: h1-htx: Add a function to parse contiguous small chunks
    - MINOR: h1-htx: Use a correlation table to speed-up small chunks parsing
    - MINOR: buf: Add function to realign a buffer with a specific head position
    - MINOR: muxes/h1-htx: Realign input buffer using b_slow_realign_ofs()
    - CLEANUP: mux-h1: Rename functions parsing input buf and filling output buf
    - Revert "MEDIUM: http-ana: Deal with L7 retries in HTTP analysers"
    - BUG/MINOR: http-ana: Send the right error if max retries is reached on L7 retry
    - BUG/MINOR: http-ana: Handle L7 retries on refused early data before K/A aborts
    - MINOR: http-ana: Perform L7 retries because of status codes in response analyser
    - MINOR: cfgparse: Fail when encountering extra arguments in macro
    - DOC: intro: Fix typo in starter guide
    - BUG/MINOR: server: Missing calloc return value check in srv_parse_source
    - BUG/MINOR: peers: Missing calloc return value check in peers_register_table
    - BUG/MINOR: ssl: Missing calloc return value check in ssl_init_single_engine
    - BUG/MINOR: http: Missing calloc return value check in parse_http_req_capture
    - BUG/MINOR: proxy: Missing calloc return value check in proxy_parse_declare
    - BUG/MINOR: proxy: Missing calloc return value check in proxy_defproxy_cpy
    - BUG/MINOR: http: Missing calloc return value check while parsing tcp-request/tcp-response
    - BUG/MINOR: http: Missing calloc return value check while parsing tcp-request rule
    - BUG/MINOR: compression: Missing calloc return value check in comp_append_type/algo
    - BUG/MINOR: worker: Missing calloc return value check in mworker_env_to_proc_list
    - BUG/MINOR: http: Missing calloc return value check while parsing redirect rule
    - BUG/MINOR: http: Missing calloc return value check in make_arg_list
    - BUG/MINOR: proxy: Missing calloc return value check in chash_init_server_tree
    - CLEANUP: http-ana: Remove useless if statement about L7 retries
    - BUG/MAJOR: stream-int: Release SI endpoint on server side ASAP on retry
    - MINOR: backend: Don't release SI endpoint anymore in connect_server()
    - BUG/MINOR: vars: Be sure to have a session to get checks variables
    - DOC/MINOR: move uuid in the configuration to the right alphabetical order
    - CLEANUP: mux-fcgi: Don't needlessly store result of data/trailers parsing
    - BUILD: fix compilation for OpenSSL-3.0.0-alpha17
    - MINOR: http-ana: Use -1 status for client aborts during queuing and connect
    - REGTESTS: Fix http_abortonclose.vtc to support -1 status for some client aborts
    - CLEANUP: backend: fix incorrect comments on locking conditions for lb functions
    - CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests
    - CI: github actions: add OpenSSL-3.0.0 builds
    - CI: github actions: -Wno-deprecated-declarations with OpenSSL 3.0.0
    - MINOR: errors: allow empty va_args for diag variadic macro
    - REORG: errors: split errors reporting function from log.c
    - CLEANUP: server: fix cosmetic of error message on sni parsing
    - MEDIUM: errors: implement user messages buffer
    - MINOR: log: do not discard stderr when starting is over
    - MEDIUM: errors: implement parsing context type
    - MINOR: errors: use user messages context in print_message
    - MINOR: log: display exec path on first warning
    - MINOR: errors: specify prefix "config" for parsing output
    - MINOR: log: define server user message format
    - REORG: server: use parsing ctx for server parsing
    - REORG: config: use parsing ctx for server config check
    - MINOR: server: use parsing ctx for server init addr
    - MINOR: server: use ha_alert in server parsing functions
    - DOC: use the req.ssl_sni in examples
    - CLEANUP: cfgparse: Remove duplication of `MAX_LINE_ARGS + 1`
    - CLEANUP: tools: Make errptr const in `parse_line()`
    - MINOR: haproxy: Add `-cc` argument
    - BUG: errors: remove printf positional args for user messages context
    - CI: Make matrix.py executable and add shebang
    - BUILD: make tune.ssl.keylog available again
    - BUG/MINOR: ssl: OCSP stapling does not work if expire too far in the future
    - Revert "BUG/MINOR: opentracing: initialization after establishing daemon mode"
    - BUG/MEDIUM: opentracing: initialization before establishing daemon and/or chroot mode
    - SCRIPTS: opentracing: enable parallel builds in build-ot.sh
    - BUG/MEDIUM: compression: Fix loop skipping unused blocks to get the next block
    - BUG/MEDIUM: compression: Properly get the next block to iterate on payload
    - BUG/MEDIUM: compression: Add a flag to know the filter is still processing data
    - MINOR: ssl: Keep the actual key length in the certificate_ocsp structure
    - MINOR: ssl: Add new "show ssl ocsp-response" CLI command
    - MINOR: ssl: Add the OCSP entry key when displaying the details of a certificate
    - MINOR: ssl: Add the "show ssl cert foo.pem.ocsp" CLI command
    - REGTESTS: ssl: Add "show ssl ocsp-response" test
    - BUG/MINOR: server: explicitly set "none" init-addr for dynamic servers
    - BUG/MINOR: pools: fix a possible memory leak in the lockless pool_flush()
    - BUG/MINOR: pools: make DEBUG_UAF always write to the to-be-freed location
    - MINOR: pools: do not maintain the lock during pool_flush()
    - MINOR: pools: call malloc_trim() under thread isolation
    - MEDIUM: pools: use a single pool_gc() function for locked and lockless
    - BUG/MAJOR: pools: fix possible race with free() in the lockless variant
    - CLEANUP: pools: remove now unused seq and pool_free_list
    - MEDIUM: pools: remove the locked pools implementation
    - BUILD: ssl: Fix compilation with BoringSSL
    - BUG/MEDIUM: errors: include missing obj_type file
    - REGTESTS: ssl: show_ssl_ocspresponce.vtc is broken with BoringSSL
    - BUG/MAJOR: htx: Fix htx_defrag() when an HTX block is expanded
    - BUG/MINOR: mux-fcgi: Expose SERVER_SOFTWARE parameter by default
    - BUG/MINOR: h1-htx: Fix a signess bug with char data type when parsing chunk size
    - CLEANUP: l7-retries: do not test the buffer before calling b_alloc()
    - BUG/MINOR: resolvers: answser item list was randomly purged or errors
    - MEDIUM: resolvers: add a ref on server to the used A/AAAA answer item
    - MEDIUM: resolvers: add a ref between servers and srv request or used SRV record
    - BUG/MINOR: server-state: load SRV resolution only if params match the config
    - MINOR: config: remove support for deprecated option "tune.chksize"
    - MINOR: config: completely remove support for "no option http-use-htx"
    - MINOR: log: remove the long-deprecated early log-format tags
    - MINOR: http: remove the long deprecated "set-cookie()" sample fetch function
    - MINOR: config: reject long-deprecated "option forceclose"
    - MINOR: config: remove deprecated option "http-tunnel"
    - MEDIUM: proxy: remove the deprecated "grace" keyword
    - MAJOR: config: remove parsing of the global "nbproc" directive
    - BUILD: init: remove initialization of multi-process thread mappings
    - BUILD: log: remove unused fmt_directive()
    - REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests
    - REGTESTS: Remove REQUIRE_VERSION=1.7 from all tests
    - CI: github actions: enable alpine/musl builds
    - BUG/MAJOR: resolvers: segfault using server template without SRV RECORDs
    - DOC: lua: Add a warning about buffers modification in HTTP
    - MINOR: ssl: Use OpenSSL's ASN1_TIME convertor when available
    - BUG/MINOR: stick-table: insert srv in used_name tree even with fixed id
    - BUG/MEDIUM: server: extend thread-isolate over much of CLI 'add server'
    - BUG/MEDIUM: server: clear dynamic srv on delete from proxy id/name trees
    - BUG/MEDIUM: server: do not forget to generate the dynamic servers ids
    - BUG/MINOR: server: do not keep an invalid dynamic server in px ids tree
    - BUG/MEDIUM: server: do not auto insert a dynamic server in px addr_node
    - BUG/MEDIUM: shctx: use at least thread-based locking on USE_PRIVATE_CACHE
    - BUG/MINOR: ssl: use atomic ops to update global shctx stats
    - BUG/MINOR: mworker: fix typo in chroot error message
    - CLEANUP: global: remove unused definition of stopping_task[]
    - MEDIUM: init: remove the loop over processes during init
    - MINOR: mworker: remove the initialization loop over processes
    - CLEANUP: global: remove the nbproc field from the global structure
    - CLEANUP: global: remove pid_bit and all_proc_mask
    - MEDIUM: global: remove dead code from nbproc/bind_proc removal
    - MEDIUM: config: simplify cpu-map handling
    - MEDIUM: cpu-set: make the proc a single bit field and not an array
    - CLEANUP: global: remove unused definition of MAX_PROCS
    - MEDIUM: global: remove the relative_pid from global and mworker
    - DOC: update references to process numbers in cpu-map and bind-process
    - MEDIUM: config: warn about "bind-process" deprecation
    - CLEANUP: shctx: remove the different inter-process locking techniques
    - BUG/MAJOR: queue: set SF_ASSIGNED when setting strm->target on dequeue
    - MINOR: backend: only skip LB when there are actual connections
    - BUG/MINOR: mux-h1: do not skip the error response on bad requests
    - MINOR: connection: add helper conn_append_debug_info()
    - MINOR: mux-h2/trace: report a few connection-level info during h2_init()
    - CLEANUP: mux-h2/traces: better align user messages
    - BUG/MINOR: stats: make "show stat typed desc" work again
    - MINOR: mux-h2: obey http-ignore-probes during the preface
    - BUG/MINOR: mux-h2/traces: bring back the lost "rcvd H2 REQ" trace
    - BUG/MINOR: mux-h2/traces: bring back the lost "sent H2 REQ/RES" traces
    - CLEANUP: assorted typo fixes in the code and comments
    - CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'
    - REGTESTS: Replace REQUIRE_VERSION=2.5 with 'haproxy -cc'
    - REGTESTS: Replace REQUIRE_OPTIONS with 'haproxy -cc' for 2.5+ tests
    - REGTESTS: Replace REQUIRE_BINARIES with 'command -v'
    - REGTESTS: Remove support for REQUIRE_BINARIES
    - CI: ssl: enable parallel builds for OpenSSL on Linux
    - CI: ssl: do not needlessly build the OpenSSL docs
    - CI: ssl: keep the old method for ancient OpenSSL versions
    - CLEANUP: server: a separate function for initializing the per_thr field
    - BUG/MINOR: server: Forbid to set fqdn on the CLI if SRV resolution is enabled
    - BUG/MEDIUM: server/cli: Fix ABBA deadlock when fqdn is set from the CLI
    - MINOR: resolvers: Clean server in a dedicated function when removing a SRV item
    - MINOR: resolvers: Remove server from named_servers tree when removing a SRV item
    - BUG/MEDIUM: resolvers: Add a task on servers to check SRV resolution status
    - BUG/MINOR: backend: restore the SF_SRV_REUSED flag original purpose
    - BUG/MINOR: backend: do not set sni on connection reuse
    - BUG/MINOR: resolvers: Use resolver's lock in resolv_srvrq_expire_task()
    - BUG/MINOR: server/cli: Fix locking in function processing "set server" command
    - BUG/MINOR: cache: Correctly handle existing-but-empty 'accept-encoding' header
    - MINOR: ssl: fix typo in usage for 'new ssl ca-file'
    - MINOR: ssl: always initialize random generator
    - MINOR: ssl: check allocation in ssl_sock_init_srv
    - MINOR: ssl: check allocation in parse ciphers/ciphersuites/verifyhost
    - MINOR: ssl: check allocation in parse npn/sni
    - MINOR: server: disable CLI 'set server ssl' for dynamic servers
    - MINOR: ssl: render file-access optional on server crt loading
    - MINOR: ssl: split parse functions for alpn/check-alpn
    - MINOR: ssl: support ca-file arg for dynamic servers
    - MINOR: ssl: support crt arg for dynamic servers
    - MINOR: ssl: support crl arg for dynamic servers
    - MINOR: ssl: enable a series of ssl keywords for dynamic servers
    - MINOR: ssl: support ssl keyword for dynamic servers
    - REGTESTS: server: test ssl support for dynamic servers
    - MINOR: queue: update the stream's pend_pos before queuing it
    - CLEANUP: Prevent channel-t.h from being detected as C++ by GitHub
    - BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check
    - REGTESTS: fix maxconn update with agent-check
    - MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn
    - MINOR: queue: update proxy->served once out of the loop
    - MEDIUM: queue: refine the locking in process_srv_queue()
    - MINOR: lb/api: remove the locked argument from take_conn/drop_conn
    - MINOR: queue: create a new structure type "queue"
    - MINOR: proxy: replace the pendconns-related stuff with a struct queue
    - MINOR: server: replace the pendconns-related stuff with a struct queue
    - MEDIUM: queue: use a dedicated lock for the queues
    - MEDIUM: queue: simplify again the process_srv_queue() API
    - MINOR: queue: factor out the proxy/server queuing code
    - MINOR: queue: use atomic-ops to update the queue's index
    - MEDIUM: queue: determine in process_srv_queue() if the proxy is usable
    - MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()
    - MEDIUM: queue: unlock as soon as possible
    - MINOR: queue: make pendconn_first() take the lock by itself
    - CLEANUP: backend: remove impossible case of round-robin + consistent hash
    - MINOR: tcp-act: Add set-src/set-src-port for "tcp-request content" rules
    - DOC: config: Add missing actions in "tcp-request session" documentation
    - CLEANUP: dns: Remove a forgotten debug message
    - DOC: Replace issue templates by issue forms
    - Revert "MINOR: queue: make pendconn_first() take the lock by itself"
    - Revert "MEDIUM: queue: unlock as soon as possible"
    - Revert "MEDIUM: queue: move the queue lock manipulation to pendconn_process_next_strm()"
    - Revert "MEDIUM: queue: determine in process_srv_queue() if the proxy is usable"
    - Revert "MINOR: queue: use atomic-ops to update the queue's index"
    - Revert "MINOR: queue: factor out the proxy/server queuing code"
    - Revert "MEDIUM: queue: simplify again the process_srv_queue() API"
    - Revert "MEDIUM: queue: use a dedicated lock for the queues"
    - Revert "MEDIUM: queue: refine the locking in process_srv_queue()"
    - Revert "MINOR: queue: update proxy->served once out of the loop"
    - Revert "MEDIUM: queue: make pendconn_process_next_strm() only return the pendconn"
    - MEDIUM: queue: update px->served and lb's take_conn once per loop
    - MEDIUM: queue: use a dedicated lock for the queues (v2)
    - MEDIUM: queue: simplify again the process_srv_queue() API (v2)
    - MEDIUM: queue: determine in process_srv_queue() if the proxy is usable (v2)
    - MINOR: queue: factor out the proxy/server queuing code (v2)
    - MINOR: queue: use atomic-ops to update the queue's index (v2)
    - MEDIUM: queue: take the proxy lock only during the px queue accesses
    - MEDIUM: queue: use a trylock on the server's queue
    - MINOR: queue: add queue_init() to initialize a queue
    - MINOR: queue: add a pointer to the server and the proxy in the queue
    - MINOR: queue: store a pointer to the queue into the pendconn
    - MINOR: queue: remove the px/srv fields from pendconn
    - MINOR: queue: simplify pendconn_unlink() regarding srv vs px
    - BUG: backend: stop looking for queued connections once there's no more
    - BUG/MINOR: queue/debug: use the correct lock labels on the queue lock
    - BUG/MINOR: resolvers: Always attach server on matching record on resolution
    - BUG/MINOR: resolvers: Reset server IP when no ip is found in the response
    - MINOR: resolvers: Reset server IP on error in resolv_get_ip_from_response()
    - BUG/MINOR: checks: return correct error code for srv_parse_agent_check
    - BUILD: Makefile: fix linkage for Haiku.
    - BUG/MINOR: tcpcheck: Fix numbering of implicit HTTP send/expect rules
    - MINOR: http-act/tcp-act: Add "set-log-level" for tcp content rules
    - MINOR: http-act/tcp-act: Add "set-nice" for tcp content rules
    - MINOR: http-act/tcp-act: Add "set-mark" and "set-tos" for tcp content rules
    - CLEANUP: tcp-act: Sort action lists
    - BUILD/MEDIUM: tcp: set-mark setting support for FreeBSD.
    - BUILD: tcp-act: avoid warning when set-mark / set-tos are not supported
    - BUG/MINOR: mqtt: Fix parser for string with more than 127 characters
    - BUG/MINOR: mqtt: Support empty client ID in CONNECT message
    - BUG/MEDIUM: resolvers: Make 1st server of a template take part to SRV resolution
    - CLEANUP: peers: re-write intdecode function comment.
2021-06-30 16:16:14 +02:00
David Carlier
f7f53afcf9 BUILD/MEDIUM: tcp: set-mark setting support for FreeBSD.
This platform has a similar socket option from Linux's SO_MARK,
marking a socket with an id for packet filter purpose, DTrace
monitoring and so on.
2021-06-28 07:03:35 +02:00
Christopher Faulet
469c06c30e MINOR: http-act/tcp-act: Add "set-mark" and "set-tos" for tcp content rules
It is now possible to set the Netfilter MARK and the TOS field value in all
packets sent to the client from any tcp-request rulesets or the "tcp-response
content" one. To do so, the parsing of "set-mark" and "set-tos" actions are
moved in tcp_act.c and the actions evaluation is handled in dedicated functions.

This patch may be backported as far as 2.2 if necessary.
2021-06-25 16:11:58 +02:00
Christopher Faulet
1da374af2f MINOR: http-act/tcp-act: Add "set-nice" for tcp content rules
It is now possible to set the "nice" factor of the current stream from a
"tcp-request content" or "tcp-response content" ruleset. To do so, the
action parsing is moved in stream.c and the action evaluation is handled in
a dedicated function.

This patch may be backported as far as 2.2 if necessary.
2021-06-25 16:11:53 +02:00
Christopher Faulet
551a641cff MINOR: http-act/tcp-act: Add "set-log-level" for tcp content rules
It is now possible to set the stream log level from a "tcp-request content"
or "tcp-response content" ruleset. To do so, the action parsing is moved in
stream.c and the action evaluation is handled in a dedicated function.

This patch should fix issue #1306. It may be backported as far as 2.2 if
necessary.
2021-06-25 16:11:46 +02:00