Commit Graph

2976 Commits

Author SHA1 Message Date
Willy Tarreau da1620b317 DOC: configuration: explain quotes and spaces in conditional blocks
Conditional blocks inherit the same tokenizer and argument parser as
the rest of the configuration, but are also silently concatenated
around groups of spaces and tabs. This can lead to subtle failures
for configs containing spaces around commas and parenthesis, where
a string comparison might silently fail for example. Let's better
document this particular case.

Thanks to Valentine for analysing and reporting the problem.

This can be backported to 2.4.
2024-11-20 09:04:53 +01:00
Willy Tarreau 962d5e038f DOC: configuration: explain the rules regarding spaces in arguments
Spaces around commas or parenthesis in expressions are generally part
of the value due to the long history of supporting unquoted arguments.
But this tends to come as a surprise to new users and sometimes creates
subtly invalid configurations. Let's add some text covering this.

This can be backported to 2.4.
2024-11-20 08:42:02 +01:00
Willy Tarreau 12fcd65468 MINOR: tasklet: support an optional set of wakeup flags to tasklet_wakeup_on()
tasklet_wakeup_on() and its derivates (tasklet_wakeup_after() and
tasklet_wakeup()) do not support passing a wakeup cause like
task_wakeup(). This is essentially due to an API limitation cause by
the fact that for a very long time the only reason for waking up was
to process pending I/O. But with the growing complexity of mux tasks,
it is becoming important to be able to skip certain heavy processing
when not strictly needed.

One possibility is to permit the caller of tasklet_wakeup() to pass
flags like task_wakeup(). Instead of going with a complex naming scheme,
let's simply make the flags optional and be zero when not specified. This
means that tasklet_wakeup_on() now takes either 2 or 3 args, and that the
third one is the optional flags to be passed to the callee. Eligible flags
are essentially the non-persistent ones (TASK_F_UEVT* and TASK_WOKEN_*)
which are cleared when the tasklet is executed. This way the handler
will find them in its <state> argument and will be able to distinguish
various causes for the call.
2024-11-19 20:13:41 +01:00
Willy Tarreau 8dc68f3c75 DOC: sched: document the missing TASK_F_UEVT* flags
These are user-defined one-shot events that are application-specific
and reset upon wakeup and were not documented. No backport is needed
since these were added to 3.1.
2024-11-19 20:13:41 +01:00
Willy Tarreau e5ca72cb6f DOC: sched: add missing scheduler API documentation for tasklet_wakeup_after()
This was added to 2.6 but the doc was forgotten. Let's add it. It's not
needed to backport this since it's only used for new developments.
2024-11-19 20:13:41 +01:00
Aurelien DARRAGON 501827ebe0 DOC: lua: fix yield-dependent methods expected contexts
Contrary to what the doc states, it is not expected (nor relevant) to
use yield-dependent methods such as core.yield() or core.(m)sleep() from
contexts that don't support yielding. Such contexts include body, init,
fetches and converters.

Thus the doc got it wrong since the beginning, because such methods were
never supported from the above contexts, yet it was listed in the list
of compatible contexts (probably the result of a copy-paste), which is
error-prone because it could either cause a Lua runtime error to be
thrown, or be ignored in some other cases.

It should be backported to all stable versions.
2024-11-19 19:36:02 +01:00
Amaury Denoyelle 24cea66e07 MEDIUM: quic: define cubic-pacing congestion algorithm
Define a new QUIC congestion algorithm token 'cubic-pacing' for
quic-cc-algo bind keyword. This is identical to default cubic
implementation, except that pacing is used for STREAM frames emission.

This algorithm supports an extra argument to specify a burst size. This
is stored into a new bind_conf member named quic_pacing_burst which can
be reuse to initialize quic path.

Pacing support is still considered experimental. As such, 'cubic-pacing'
can only be used with expose-experimental-directives set.
2024-11-19 16:20:58 +01:00
Amaury Denoyelle 6dfc8fbf1d MINOR: quic: extend quic-cc-algo optional parameters
Modify quic-cc-algo for better extensability of optional parameters
parsing. This will be useful to support a new parameter for maximum
allowed pacing burst size.

Take this opportunity to refine quic-cc-algo documentation. Optional
parameters are now presented as a list which would be soon extended.
2024-11-19 16:20:52 +01:00
Amaury Denoyelle 8540886f00 DOC: quic: rename max-window-size as with default prefix
Rename 'tune.quic.frontend.max-window-size' with the prefix 'default-'.
This highlights the fact that it is not a hard limit, as it can be
overriden if specifying an optional window size via quic-cc-algo on a
bind line.

No need to backport as this keyword was added on the current dev
version.
2024-11-19 16:16:48 +01:00
Christopher Faulet e68c6852ad DOC: config: Move fs.* and bs.* in section about L5 samples
These sample fetch functions were added in the wrong section. Move them in
the section about sample fetch functions at L5 layer.
2024-11-19 15:29:41 +01:00
Christopher Faulet 4ccc3f4048 DOC: config: Move wait_end in section about internal samples
wait_end is an internal sample fetch functions and not a L6 one. So move it
in the corresponding section.
2024-11-19 15:29:40 +01:00
Christopher Faulet e9021a4ca1 DOC: config: Slightly improve the %Tr documentation
Specify -1 can also be reported for %Tr delay when the response is invalid.
2024-11-19 15:29:40 +01:00
Christopher Faulet 62dc8750a9 MINOR: http: Add support for HTTP 414/431 status codes
414-Uri-Too-Long and 431-Request-Header-Fields-Too-Large are now part of
supported status codes that can be define as error files. The hash table
defined in http_get_status_idx() was updated accordingly.
2024-11-19 15:29:40 +01:00
Christopher Faulet 18de419f96 DOC: config: Fix a typo in "1.3.1. The Request line"
At the beginning of the last paragraph of this section, HTTP/3 was used
instead of HTTP/2. It is not fixed.
2024-11-19 15:29:40 +01:00
Christopher Faulet 3af2d91b3b DOC: config: A a space before ':' for {bs,fs}.aborted and {bs,fs}.rst_code
A space was missing before the ':' for the sample fetch functions above. It
was an issue for the text to HTML conversion script. So, let's fix it.
2024-11-19 15:29:40 +01:00
Christopher Faulet fa43ca2ed0 MINOR: stream: Add an option to "show sess" command to dump the captured URI
"show sess" command now supports a list of options that can be set after all
other possible arguments (<id>, all...). For now, "show-uri" is the only
supported option. With this options, the captured URI, if non-null, is added
to the dump of a stream, complete or now. The URI may be anonymized if
necessary.

This patch should fix the issue #663.
2024-11-19 15:29:40 +01:00
Christopher Faulet e9bc5937c9 MINOR: agent-check: Be able to set absolute weight via an agent
Historically, an agent-check program is only able to set a proportial weight
to the initial server's weight. However, it could be handy to also set an
absolute value. It is the purpose of this patch.

Instead of changing the current way to set a server's weight, a new
agent-check command is introduced. The string "weight:", followed by an
positive interger or a positive interger percentage, can now be used. If the
value ends with the '%' sign, then the new weight will be proportional to
the initially weight of the server. Otherwise, the value is considered as an
absolute weight and must be between 0 and 256.

This patch should fix the issue #360.
2024-11-19 15:29:40 +01:00
Christopher Faulet 1be7140ade MINOR: http-ana: Add support for "set-cookie-fmt" option to redirect rules
It is now possible to use a log-format string to define the "Set-Cookie"
header value of a response generated by a redirect rule. There is no special
check on the result format and it is not possible during the configuration
parsing. It is proably not a big deal because already existing "set-cookie"
and "clear-cookie" options don't perform any check.

Here is an example:

  http-request redirect location https://someurl.com/ set-cookie haproxy="%[var(txn.var)]"

This patch should fix the issue #1784.
2024-11-19 15:20:02 +01:00
Christopher Faulet b2877db47c MINOR: http-ana: Add option to keep query-string on a localtion-based redirect
On prefix-based redirect, there is an option to drop the query-string of the
location. Here it is the opposite. an option is added to preserve the
query-string of the original URI for a localtion-based redirect.

By setting "keep-query" option, for a location-based redirect only, the
query-string of the original URI is appended to the location. If there is no
query-string, nothing is added (no empty '?'). If there is already a
non-empty query-string on the localtion, the original one is appended with
'&' separator.

This patch should fix issue #2728.
2024-11-19 15:20:02 +01:00
Willy Tarreau e72b525832 MINOR: cfgparse: parse tune.bufsize.small as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "4k". Let's make use of parse_size_err() on it so that
units are supported. This requires to turn it to uint as well, which
was verified to be OK.
2024-11-18 19:07:05 +01:00
Willy Tarreau a344d37fad MINOR: cfgparse: parse tune.bufsize as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, preventing from starting when set
e.g. to "64k". Let's make use of parse_size_err() on it so that units are
supported. This requires to turn it to uint as well, and to explicitly
limit its range to INT_MAX - 2*sizeof(void*), which was previously
partially handled as part of the sign check.
2024-11-18 19:06:25 +01:00
Willy Tarreau 2f0c6ff3a5 MINOR: cfgparse: parse tune.recv_enough as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on it so that
units are supported. This requires to turn it to uint as well, and
since it's sometimes compared to an int, we limit its range to
0..INT_MAX.
2024-11-18 19:01:28 +01:00
Willy Tarreau a90a7d4d60 MINOR: cfgparse: parse tune.pipesize as a size
Till now this value was parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on it so that
units are supported. This requires to turn it to uint as well, which
was verified to be OK.
2024-11-18 18:51:31 +01:00
Willy Tarreau f9f28b7584 MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{frontend,backend} as sizes
Till now these values were parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on them so that
units are supported. This requires to turn them to uint as well, which
is OK.
2024-11-18 18:50:02 +01:00
Willy Tarreau a923c72357 MINOR: cfgparse: parse tune.{rcvbuf,sndbuf}.{client,server} as sizes
Till now these values were parsed as raw integer using atol() and would
silently ignore any trailing suffix, causing unexpected behaviors when
set, e.g. to "512k". Let's make use of parse_size_err() on them so that
units are supported. This requires to turn them to uint as well, which
is OK.
2024-11-18 18:49:01 +01:00
Willy Tarreau 45f9e95f22 MINOR: sample: extend the "when" converter to support an ACL
Sometimes conditions to decide of an anomaly are not as easy to define
as just an error or a success. One example use case would be to monitor
the transfer time and fix a threshold.

An idea suggested by Tristan would be to make permit the "when"
converter to refer to a more variable or dynamic condition.

Here we make this possible by making "when" rely on a named ACL. The
ACL then needs to be specified in either the proxy or the defaults
section. Since it is evaluated inline, it may even refer to information
available at the end (at log time) such as the data transfer time. If
the ACL evalutates to true, the converter passes the data.

Example: log "dbg={-}" when fine, or "dbg={... debug info ...}" on slow
transfers:

  acl slow_xfer res.timer.data ge 10000   # more than 10s is slow
  log-format "$HAPROXY_HTTP_LOG_FMT                                \
              fsdbg={%[fs.debug_str,when(acl,slow_xfer)]}          \
              bsdbg={%[bs.debug_str,when(acl,slow_xfer)]}"
2024-11-18 16:11:55 +01:00
Willy Tarreau 9539f2b097 [RELEASE] Released version 3.1-dev13
Released version 3.1-dev13 with the following main changes :
    - MEDIUM: mworker: depreciate the 'program' section
    - BUILD: ot: use a cebtree instead of a list for variable names
    - MINOR: startup: replace HAPROXY_LOAD_SUCCESS with global load_status
    - BUG/MINOR: startup: set HAPROXY_CFGFILES in read_cfg
    - BUG/MINOR: cli: don't show sockpairs in HAPROXY_CLI and HAPROXY_MASTER_CLI
    - BUG/MEDIUM: stconn: Don't forward shut for SC in connecting state
    - BUG/MEDIUM: resolvers: Insert a non-executed resulution in front of the wait list
    - MINOR: debug: explicitly permit the counter condition to be empty
    - MINOR: debug: add a new counter type for glitches
    - MINOR: mux-h2: count glitches when they're reported
    - BUG/MINOR: deinit: release uri_auth admin rules
    - MINOR: uri_auth: add stats_uri_auth_free helper
    - MEDIUM: uri_auth: implement clean uri_auth cleaning
    - MINOR: mux-quic/h3: count glitches when they're reported
    - BUG/MEDIUM: mux-h2: Don't send RST_STREAM frame for streams with no ID
    - BUG/MINOR: Don't report early srv aborts on request forwarding in DONE state
    - MINOR: promex: Expose the global node and description in process metrics
    - MINOR: promex: Add global and proxies description as labels to all metrics
    - OPTIM: pattern: only apply LRU cache for large enough lists
    - BUG/MEDIUM: checks: make sure to always apply offsets to now_ms in expiration
    - BUG/MINOR: debug: do not set task expiration to TICK_ETERNITY
    - BUG/MEDIUM: mailers: make sure to always apply offsets to now_ms in expiration
    - BUG/MINOR: mux_quic: make sure to always apply offsets to now_ms in expiration
    - BUG/MINOR: peers: make sure to always apply offsets to now_ms in expiration
    - BUG/MEDIUM: clock: make sure now_ms cannot be TICK_ETERNITY
    - MINOR: debug/cli: replace "debug dev counters" with "debug counters"
    - DOC: config: add tune.h2.{be,fe}.rxbuf to the global keywords index
    - MINOR: chunk: add a BUG_ON upon the next init_trash_buffer()
2024-11-15 18:42:29 +01:00
Willy Tarreau 5f37af7a8e DOC: config: add tune.h2.{be,fe}.rxbuf to the global keywords index
These two keywords were missing from the index, let's add them.
2024-11-15 16:32:37 +01:00
Willy Tarreau 4420939fcd MINOR: debug/cli: replace "debug dev counters" with "debug counters"
"debug dev" commands are not meant to be used by end-users, and are
purposely not documented. Yet due to their usefulness in troubleshooting
sessions, users are increasingly invited by developers to use some of
them.

"debug dev counters" is one of them. Better move it to "debug counters"
and document it so that users can check them even if the output can look
cryptic at times. This, combined with DEBUG_GLITCHES, can be convenient
to observe suspcious activity. The doc however precises that the format
may change between versions and that new entries/types might appear
within a stable branch.
2024-11-15 16:26:01 +01:00
William Lallemand 581c8a27d9 MEDIUM: mworker: depreciate the 'program' section
The program section is unreliable and should not be used, more reliable
alternatives exist outside HAProxy. Let's depreciate the section so we
could remove it completely in 3.3.
2024-11-08 17:06:58 +01:00
Willy Tarreau 0434e87348 [RELEASE] Released version 3.1-dev12
Released version 3.1-dev12 with the following main changes :
    - MINOR: startup: tune.renice.{startup,runtime} allow to change priorities
    - BUG/MEDIUM: promex: Fix dump of extra counters
    - BUILD: import/mt_list: support building with TCC
    - BUILD: compiler: define __builtin_prefetch() for tcc
    - CLEANUP: quic: Remove the useless directive "tune.quic.backend.max-idle-timeou"
    - DOC: config: document connection error 44 (reverse connect failure)
    - CLEANUP: connection: properly name the CO_ER_SSL_FATAL enum entry
    - DEBUG: cli: support closing "hard" using close() in addition to fd_delete()
    - MINOR: connection: add more connection error codes to cover common errno
    - MINOR: rawsock: set connection error codes when returning from recv/send/splice
    - MINOR: connection: add new sample fetch functions fc_err_name and bc_err_name
    - MINOR: quic: Help diagnosing malformed probing packets
    - BUG/MINOR: quic: fix malformed probing packet building
    - MINOR: listener: Remove useless checks on the receiver protocol existence
    - MINOR: http-conv: Remove unreachable goto statement in sample_conv_q_preferred
    - MINOR: http: don't %-encode the payload when not relevant
    - MINOR: quic: simplify qc_parse_pkt_frms() return path
    - MINOR: quic: use dynamically allocated frame on parsing
    - MINOR: quic: extend return value of CRYPTO parsing
    - BUG/MINOR: quic: repeat packet parsing to deal with fragmented CRYPTO
    - BUG/MINOR: mworker: do 'program' postparser checks in read_cfg_in_discovery_mode
    - EXAMPLES: add "traces.cfg" with traces examples
    - BUG/MEDIUM: quic: do not consider ACK on released stream as error
    - CLEANUP: stats: fix misleading comment on top of stat_idx_info
    - MINOR: wdt: move the local timers to a struct
    - MINOR: debug: add a function to dump a stuck thread
    - DEBUG: wdt: better detect apparently locked up threads and warn about them
    - DEBUG: cli: make it possible for "debug dev loop" to trigger warnings
    - DEBUG: wdt: make the blocked traffic warning delay configurable
    - DEBUG: wdt: add a stats counter "BlockedTrafficWarnings" in show info
    - DEBUG: wdt: set the default blocked task delay to 100 ms
    - MINOR: debug: move the "recover now" warn message after the optional notes
    - MINOR: event_hdl: add event_hdl_sub_list_empty() helper func
    - MINOR: pattern: add _pat_ref_new() helper func
    - OPTIM: pattern: use malloc() to initialize new pat_ref struct
    - MINOR: pattern: add pat_ref_free() helper func
    - CLEANUP: guid: remove global tree export
    - BUG/MINOR: guid/server: ensure thread-safety on GUID insert/delete
    - DOC: management: explain the change of behavior of the program section
    - BUG/MEDIUM: mux-h2: try to wait for the peer to read the GOAWAY
    - BUG/MEDIUM: quic: prevent crash due to CRYPTO parsing error
2024-11-08 15:46:54 +01:00
William Lallemand 75b302d123 DOC: management: explain the change of behavior of the program section
The program section does not work exactly the same way with the
master-worker rework of HAProxy 3.1. Let's explain it in the program
documentation.
2024-11-08 12:00:26 +01:00
Willy Tarreau 5f4fe20116 DEBUG: wdt: set the default blocked task delay to 100 ms
The warn-blocked-traffic-after can be significantly lowered. In any
case, in order to be usable it must be well below the limit to have a
chance to emit exploitable traces before the watchdog finally fires.
Even configured at 1ms it looks very difficult to trigger it on a
laptop doing SSL and compression, so applying a 100-fold factor to
cover for large configs and small machines sounds sane for 3.1. In any
case, even at 100ms, the service degradation becomes quite visible.
2024-11-06 18:35:42 +01:00
Willy Tarreau 6127e5a4e9 DEBUG: wdt: make the blocked traffic warning delay configurable
The new global "warn-blocked-traffic-after" allows one to configure
after how much time a warning should be emitted when traffic is blocked.
2024-11-06 18:35:42 +01:00
Willy Tarreau 601b34fe7b MINOR: connection: add new sample fetch functions fc_err_name and bc_err_name
These functions return a symbolic error code such as ECONNRESET to keep
logs compact while making them human-readable. It's a good alternative
to the numeric code in that it's more expressive, and a good one to the
full message since it's shorter and more precise (some codes even match
errno names).

The doc was updated so that the symbolic names appear in the table. It
could be useful to backport this feature to help with troubleshooting
some issues, though backporting the doc might possibly be more annoying
in case users have local patches already, so maybe the table update does
not need to be backported in this case.
2024-11-05 18:57:43 +01:00
Willy Tarreau 00c383ff65 MINOR: connection: add more connection error codes to cover common errno
While we get reports of connection setup errors in fc_err/bc_err, we
don't have the equivalent for the recv/send/splice syscalls. Let's
add provisions for new codes that cover the common errno values that
recv/send/splice can return, i.e. ECONNREFUSED, ENOMEM, EBADF, EFAULT,
EINVAL, ENOTCONN, ENOTSOCK, ENOBUFS, EPIPE. We also add a special case
for when the poller reported the error itself. It's worth noting that
EBADF/EFAULT/EINVAL will generally indicate serious bugs in the code
and should not be reported.

The only thing is that it's quite hard to forcefully (and reliably)
trigger these errors in automated tests as the timing is critical.
Using iptables to manually reset established connections in the
middle of large transfers at least permits to see some ECONNRESET
and/or EPIPE, but the other ones are harder to trigger.
2024-11-05 18:57:43 +01:00
Willy Tarreau abed9e0426 DOC: config: document connection error 44 (reverse connect failure)
It was missing from commit ac1164de7c ("MINOR: connection: define error
for reverse connect"), and can be backported to 3.0 and 2.9.
2024-11-05 18:57:42 +01:00
William Lallemand e75a019fba MINOR: startup: tune.renice.{startup,runtime} allow to change priorities
This commit introduces the tune.renice.startup and tune.renice.runtime
global keywords that allows to change the priority with setpriority().

tune.renice.startup is parsed and applied in the worker or the standalone
process for configuration parsing. If this keyword is used alone, the
nice value is changed to the previous one after configuration parsing.

tune.renice.runtime is applied after configuration parsing, so in the
worker or a standalone process. Combined with tune.renice.startup it
allows to have a different nice value during configuration parsing and
during runtime.

The feature was discussed in github issue #1919.

Example:

   global
        tune.renice.startup 15
        tune.renice.runtime 0
2024-11-04 17:48:58 +01:00
Willy Tarreau 2092199353 [RELEASE] Released version 3.1-dev11
Released version 3.1-dev11 with the following main changes :
    - BUG/MINOR: httpclient: return NULL when no proxy available during httpclient_new()
    - BUG/MEDIUM: mworker/httpclient: initialization skipped by accident in mworker mode
    - BUG/MINOR: resolvers/mworker: missing default resolvers in mworker mode
    - MINOR: mworker/ocsp: skip ocsp-update proxy init in master
    - BUG/MEDIUM: stconn: Wait iobuf is empty to shut SE down during a check send
    - MINOR: mux-h1: Show the SD iobuf in trace messages on stream send events
    - MINOR: mux-h1: Add a trace on shutdown when keep-alive is not possible
    - BUG/MINOR: http-ana: Don't report a server abort if response payload is invalid
    - BUG/MEDIUM: stconn: Check FF data of SC to perform a shutdown in sc_notify()
    - BUG/MAJOR: filters/htx: Add a flag to state the payload is altered by a filter
    - REGTESTS: Never reuse server connection in http-messaging/truncated.vtc
    - BUG/MINOR: quic: avoid leaking post handshake frames
    - MINOR: quic: send new tokens (NEW_TOKEN) even for 1RTT sessions
    - BUG/MEDIUM: quic: avoid freezing 0RTT connections
    - DOC: config: fix rfc7239 forwarded typo in desc
    - MINOR: http_ext: implement rfc7239_{nn,np} converters
    - CLEANUP: http_ext: remove useless BUG_ON() in http_handle_xot_header()
    - BUG/MINOR: sample: free err2 in smp_resolve_args for type ARGT_REG
    - MINOR: arg: add an argument type for identifier
    - BUILD: buffers: keep b_getblk_nc() and b_peek_varint() in buf.h
    - CLEANUP: buffers: simplify b_get_varint()
    - OPTIM: buffers: avoid a useless wrapping check for ofs == 0
    - MINOR: debug: make mark_tainted() return the previous value
    - MINOR: chunk: drop the global thread_dump_buffer
    - MINOR: debug: split ha_thread_dump() in two parts
    - MINOR: debug: slightly change the thread_dump_pointer signification
    - MINOR: debug: make ha_thread_dump_done() take the pointer to be used
    - MINOR: debug: replace ha_thread_dump() with its two components
    - MEDIUM: debug: on panic, make the target thread automatically allocate its buf
    - BUILD: mux-h2/traces: fix build on 32-bit due to size of the DATA frame
    - CI: prepare Coverity build for Ubuntu 24
    - CI: bump development builds explicitely to Ubuntu 24.04
    - CI: modernize macos builds to macos-15
    - BUG/MINOR: mworker: fix mworker-max-reloads parser
    - MINOR: mux-quic: simplify sending of empty STREAM FIN
    - BUG/MINOR: mux-quic: do not close STREAM with empty FIN if no data sent
    - CLEANUP: debug: make the BUG_ON() macros check the condition in the outer one
    - MEDIUM: debug: add match counters for BUG_ON/WARN_ON/CHECK_IF
    - MINOR: debug: add a new debug macro COUNT_IF()
    - MINOR: debug: add "debug dev counters" to list code counters
    - BUG/MEDIUM: stats-html: Never dump more data than expected during 0-copy FF
    - BUG/MEDIUM: mux-h2: Remove H2S from send list if data are sent via 0-copy FF
    - BUG/MINOR: stconn: Pretend the SE have more data to deliver on abortonclose
    - CLEANUP: stream: remove outdated comments
    - DEBUG: stream: Add debug counters to track some client/server aborts
    - DEBUG: mux-h1: Add debug counters to track some errors
    - MINOR: mux-h1: Add support of the debug string for logs
    - MINOR: stream: maintain per-stream counters of the number of passes on code
    - MINOR: filters: add per-filter call counters
    - MINOR: sample: add the "when" converter to condition some expressions
    - BUG/MEDIUM: connection/http-reuse: fix address collision on unhandled address families
    - BUILD: spoe: fix build warning on older gcc around sub-struct initialization
    - Revert "OPTIM: mux-h2: make h2_send() report more accurate wake up conditions"
    - DEBUG: mux-h1: Add debug counters to track errors with in/out pending data
    - BUG/MINOR: mux-h1: Fix conditions on pipe in some COUNT_IF()
    - MINOR: activity/memprofile: show per-DSO stats
    - BUG/MINOR: mworker/cli: show master startup logs in recovery mode
    - MINOR: mworker: stop MASTER proxy listener on worker mcli sockpair
    - MINOR: error: simplify startup_logs_init_shm
    - BUG/MINOR: mworker: show worker warnings in startup logs
    - CLEANUP: mworker: clean mworker_reexec
    - MINOR: mworker/cli: split mworker_cli_proxy_create
    - BUG/MINOR: server: fix dynamic server leak with check on failed init
    - BUG/MEDIUM: server: fix race on servers_list during server deletion
    - BUG/MEDIUM: stconn: Report blocked send if sends are blocked by an error
    - BUG/MINOR: http-ana: Fix wrong client abort reports during responses forwarding
    - BUG/MINOR: stconn: Don't disable 0-copy FF if EOS was reported on consumer side
    - MINOR: mworker/cli: add 'debug' to 'show proc'
    - MINOR: mworker/cli: remove comment line for program when useless
    - MINOR: mworker/cli: 'show proc debug' for old workers
    - BUILD: debug: silence a build warning with threads disabled
    - CLEANUP: mux-h2: remove the unused "full" variable in h2_frt_transfer_data()
    - MINOR: pools: export the pools variable
    - MINOR: debug: place a magic pattern at the beginning of post_mortem
    - MINOR: debug: place the post_mortem struct in its own section.
    - MINOR: debug: store important pointers in post_mortem
    - MINOR: debug: do not limit backtraces to stuck threads
    - MINOR: cli: remove non-printable characters from 'debug dev fd'
    - MINOR: cli: add an 'echo' command
    - MINOR: debug: also add a pointer to struct global to post_mortem
    - CLEANUP: mworker: make mworker_create_master_cli more readable
    - BUG/MEIDUM: mworker: fix fd leak from master to worker
    - BUG/MINOR: mworker/cli: fix mworker_cli_global_proxy_new_listener
    - MINOR: tools: add strnlen2() helper
    - CLEANUP: log: use strnlen2() in _lf_text_len() to compute string length
    - DOC: design: add notes about more detailed error reporting for logs
    - MINOR: debug: also add fdtab and acitvity to struct post_mortem
    - MINOR: debug: remove the redundant process.thread_info array from post_mortem
    - DEV: gdb: add a number of gdb scripts to navigate in core dumps
    - BUG/MINOR: trace: stop rewriting argv with -dt
    - MEDIUM: protocol: make abns a custom unix socket address family
    - MEDIUM: protocol: rely on AF_CUST_ABNS family to recognize ABNS sockets
    - CLEANUP: tools: rely on address family to detect ABNS sockets
    - MINOR: protocol: create abnsz socket address family
    - MINOR: sock: restore effective UNIX family in sock_get_old_sockets()
    - MEDIUM: sock: also restore effective unix family in get_{src,dst}()
    - MEDIUM: sock_unix: use per-family addrcmp function
    - MEDIUM: socket: add zero-terminated ABNS alternative
    - BUG/MINOR: ssl/cli: 'set ssl cert' does not check the transaction name correctly
    - BUG/MINOR: mworker: mworker_reexec: unset MODE_STARTING before free startup logs ring
    - BUG/MINOR: errors: startup_logs_free: set global startup_logs ptr to NULL
    - BUG/MINOR: errors: print_message: don't allocate startup logs ring
    - BUG/MINOR: startup: don't fork worker if started with -c -W
    - BUG/MINOR: startup: dump libs only in worker if started with -W -dL
    - BUG/MINOR: startup: dump keywords only in worker if started with -W -dKAll
    - BUG/MINOR: startup: don't dump polling info for master in verbose mode
    - CI: switch QUIC Interop on AWS-LC to common docker image
    - CI: switch QUIC Interop on LibreSSL to common docker image
    - CI: enable chacha20 test on LibreSSL QUIC Interop
    - DOC: config: add missing glitch_{cnt,rate} data types
    - DOC: config: add missing glitch_{cnt,rate} sample definitions
    - CI: LibreSSL QUIC Interop: fix docker context
    - DEBUG: mux-h1: Add H1C expiration dates in trace messages
    - BUG/MEDIUM: mux-h1: Fix how timeouts are applied on H1 connections
    - BUG/MINOR: http-ana: Report internal error if an action yields on a final eval
    - MINOR: stream: Save last evaluated rule on invalid yield
    - MINOR: quic: complete trace in qc_may_build_pkt()
    - MINOR: quic: move qc_send_mux() prototype into quic_tx.h
    - MINOR: stream: Replace last_rule_file/line fields by a more generic field
    - MINOR: stream: Save the last filter evaluated interrupting the processing
    - MINOR: stream: Save the entity waiting to continue its processing
    - MINOR: stream: Use an enum to identify last and waiting entities for streams
    - MINOR: stream: Add http-buffer-request option in the waiting entities
    - DOC: config: Add documentation about last_entity sample fetch
    - DOC: config: Add documentation about waiting_entity sample fetch
2024-11-01 10:17:02 +01:00
Christopher Faulet 1cd8173687 DOC: config: Add documentation about waiting_entity sample fetch
The commit adds the documentation for the waiting_entity sample fetch.
2024-10-31 20:47:59 +01:00
Christopher Faulet 6034080c49 DOC: config: Add documentation about last_entity sample fetch
The commit adds the documentation for the last_entity sample fetch.
2024-10-31 20:25:07 +01:00
Aurelien DARRAGON 0686fd8cfc DOC: config: add missing glitch_{cnt,rate} sample definitions
Following previous commit, when glitch_cnt and glitch_rate data types were
implemented in c9c6b683f ("MEDIUM: stick-tables: add a new stored type for
glitch_cnt and glitch_rate"), newly exposed samples such as
table_glitch_cnt(), table_glitch_rate, src_glitch_cnt() and
src_glitch_rate() were documented but their definitions was missing in
supported keywords list.

It should be backported in 3.0 with c9c6b683f
2024-10-30 17:47:30 +01:00
Aurelien DARRAGON 9a6fc2d474 DOC: config: add missing glitch_{cnt,rate} data types
When glitch_cnt and glitch_rate data types were implemented in
c9c6b683f ("MEDIUM: stick-tables: add a new stored type for glitch_cnt and
glitch_rate"), the data types list for "stick-table" keyword documentation
was overlooked.

This was reported by Nick Ramirez.

It should be backported in 3.0 with c9c6b683f.
2024-10-30 17:47:24 +01:00
Tristan 18582ede05 MEDIUM: socket: add zero-terminated ABNS alternative
When an abstract unix socket is bound by HAProxy (using "abns@" prefix),
NUL bytes are appended at the end of its path until sun_path is filled
(for a total of 108 characters).

Here we add an alternative to pass only the non-NUL length of that path
to connect/bind calls, such that the effective path of the socket's name
is as humanly written. This may be useful to interconnect with existing
softwares that implement abstract sockets with this logic instead of the
default haproxy one.

This is achieved by implementing the "abnsz" socket prefix (instead of
"abns"), which stands for "zero-terminated ABNS". "abnsz" prefix may be
used anywhere "abns" is. Internally, haproxy uses the custom socket
family (AF_CUST_ABNS vs AF_CUST_ABNSZ) to differentiate default abns
sockets from zero-terminated ones.

Documentation was updated and regtest was added.

Fixes GH issues #977 and #2479

Co-authored-by: Aurelien DARRAGON <adarragon@haproxy.com>
2024-10-29 12:15:24 +01:00
Willy Tarreau 20ffa35f66 DOC: design: add notes about more detailed error reporting for logs
These are the notes of a day long code analysis session (CFA+WTA)
aimed at figuring what's missing during most code troubleshooting
sessions.  The goal is to provide good indications about what rules/
filters were still active when the processing ended (timeout, error
etc), what subscribers are still active (indicating waiting for an
event), and what shut/abort events were met at the various levels
of each side's stack, in each direction.
2024-10-28 17:14:48 +01:00
William Lallemand dc1c0a169c MINOR: cli: add an 'echo' command
Add an echo command to write text over the CLI output.
2024-10-24 17:20:57 +02:00
William Lallemand 84640aaa2a MINOR: mworker/cli: add 'debug' to 'show proc'
This patch adds a 'debug' parameter to the 'show proc' command of the
master CLI. It allows to show debug details about the processes.

Example:

echo 'show proc debug' | socat /tmp/master.sock -
\#<PID>          <type>          <reloads>       <uptime>        <version>      		<ipc_fd[0]>     <ipc_fd[1]>
391999          master          0 [failed: 0]   0d00h00m02s     3.1-dev10-b9095a-63		5               6
\# workers
392001          worker          0               0d00h00m02s     3.1-dev10-b9095a-63		3               -1
\# programs
2024-10-24 14:23:27 +02:00
Willy Tarreau b74fb1325e MINOR: sample: add the "when" converter to condition some expressions
Sometimes it would be desirable to include some debugging output only
under certain conditions, but the end of the transfer is too late to
apply some rules.

Here we take the approach of making a converter ("when") that takes a
condition among an arbitrary list, and decides whether or not to let
the input sample pass through or not based on the condition. This
allows for example to log debugging information only when an error
was encountered during the processing (sort of an extension of
dontlog-normal). The conditions are quite limited (stopping, error,
normal, toapplet, forwarded, processed) and can be negated. The
converter can also be chained to use more complex conditions.

A suggested example will be:

    # log "dbg={-}" when fine, or "dbg={... debug info ...}" on error:
    log-format "$HAPROXY_HTTP_LOG_FMT dbg={%[bs.debug_str,when(!normal)]}"
2024-10-22 20:13:00 +02:00
Aurelien DARRAGON d28d016f43 MINOR: http_ext: implement rfc7239_{nn,np} converters
"option forwarded" provides a convenient way to automatically insert
rfc7239 forwarded header to requests sent to servers.

On the other hand, manually crafting the header is quite complicated due
to specific formatting rules that must be followed as per rfc7239.
However, sometimes it may be necessary to craft the header manually, for
instance if it has to be conditional or based on parameters that "option
forwarded" doesn't provide. To ease this task, in this patch we implement
rfc7239_nn and rfc7239_np which are respectively meant to craft nodename:
nodeport values, specifically intended to manually build rfc7239 'for'
and 'by' header fields while ensuring rfc7239 compliancy.

Example:
  # build RFC-compliant 7239 header:
  http-request set-var-fmt(txn.forwarded) "for=\"%[ipv6(::1),rfc7239_nn]:%[str(8888),rfc7239_np]\";host=\"haproxy.org\";proto=http"
  # check RFC-compliancy:
  http-request set-var(txn.test) "var(txn.forwarded),debug(ok,stderr),rfc7239_is_valid,debug(ok,stderr)"
  #  stderr output:
  #    [debug] ok: type=str <for="[::1]:_8888";host="haproxy.org";proto=http>
  #    [debug] ok: type=bool <1>

See documentation for more info and examples.
2024-10-17 17:24:58 +02:00
Aurelien DARRAGON 45cbbdc845 DOC: config: fix rfc7239 forwarded typo in desc
replace specicy with specify in rfc7239 forwarded option description.
Multiple occurences were found.

May be backported in 2.8.
2024-10-17 17:24:51 +02:00