Commit Graph

18708 Commits

Author SHA1 Message Date
Christopher Faulet
4cfc038cb1 BUG/MINOR: stream: Perform errors handling in right order in stream_new()
The frontend SC is attached before the backend one is allocated. Thus an
allocation error on backend SC must be handled before an error on the
frontend SC.

This patch must be backported to 2.6.
2022-09-27 10:56:51 +02:00
Willy Tarreau
56ac2cbf58 CLEANUP: list: fix again some style issues in the recent comments
While reading the recent changes around mt_list_for_each_entry_safe() I
noticed a spurious "q" at the beginning of a line introduced by commit
455843721 ("CLEANUP: list: Fix mt_list_for_each_entry_safe indentation")
and that visually confusing multi-line comments missing the trailing '\'
character were introduced by previous commit 60cffbaca ("MINOR: list:
documenting mt_list_for_each_entry_safe() macro"), which at first glance
made the macro look broken. In addition, multi-line comments must end
with a "*/" on its own line to instantly spot where it ends without
having to read the whole line, like this:

    /* we know from the above that foo is always valid
     * here so it's safe to end the string:
     */
    *(unsigned char *)foo = 0;

Not like this:

    /* we know from the above that foo is always valid
     * here so it's safe to end the string: */
    *(unsigned char *)foo = 0;

Finally, macro's main comment mentionned the wrong macro name and types,
and was randomly indented.
2022-09-27 08:04:08 +02:00
William Lallemand
70c5ad4425 DOC: management: describe the new reload command behavior
The master CLI command now allows to do a synchronous reload with a
status.
2022-09-24 16:47:04 +02:00
William Lallemand
0a0512f76d MINOR: mworker/cli: the mcli_reload bind_conf only send the reload status
Upon a reload with the master CLI, the FD of the master CLI session is
received by the internal socketpair listener.

This session is used to display the status of the reload and then will
close.
2022-09-24 16:35:23 +02:00
William Lallemand
56f73b21a5 MINOR: mworker: stores the mcli_reload bind_conf
Stores the mcli_reload bind_conf in order to identify it later.
2022-09-24 15:56:25 +02:00
William Lallemand
21623b5949 MINOR: mworker: mworker_cli_proxy_new_listener() returns a bind_conf
mworker_cli_proxy_new_listener() now returns a bind_conf * or NULL upon
failure.
2022-09-24 15:51:27 +02:00
William Lallemand
68192b2cdf MINOR: mworker: store and shows loading status
The environment variable HAPROXY_LOAD_SUCCESS stores "1" if it
successfully load the configuration and started, "0" otherwise.

The "_loadstatus" master CLI command displays either
"Loading failure!\n" or "Loading success.\n"
2022-09-24 15:44:42 +02:00
William Lallemand
479cb3ed3a MINOR: mworker/cli: replace close() by fd_delete()
Replace the close() call in cli_parse_reload() by a fd_delete() since
the FD is one present in the fdtab.
2022-09-23 10:28:56 +02:00
Aurelien DARRAGON
b12d169ea3 BUG/MINOR: hlua: fixing ambiguous sizeof in hlua_load_per_thread
As pointed out by chipitsine in GH #1879, coverity complains
about a sizeof with char ** type where it should be char *.

This was introduced in 'MINOR: hlua: Allow argument on
lua-lod(-per-thread) directives' (ae6b568)

Luckily this had no effect so far because on most platforms
sizeof(char **) == sizeof(char *), but this can not be safely
assumed for portability reasons.

The fix simply changes the argument to sizeof so that it refers to
'*per_thread_load[len]' instead of 'per_thread_load[len]'.

No backport needed.
2022-09-23 09:50:11 +02:00
William Lallemand
ec059c249e MEDIUM: mworker/cli: keep the connection of the FD that ask for a reload
When using the "reload" command over the master CLI, all connections to
the master CLI were cut, this was unfortunate because it could have been
used to implement a synchronous reload command.

This patch implements an architecture to keep the connection alive after
the reload.

The master CLI is now equipped with a listener which uses a socketpair,
the 2 FDs of this socketpair are stored in the mworker_proc of the
master, which the master keeps via the environment variable.

ipc_fd[1] is used as a listener for the master CLI. During the "reload"
command, the CLI will send the FD of the current session over ipc_fd[0],
then the reload is achieved, so the master won't handle the recv of the
FD. Once reloaded, ipc_fd[1] receives the FD of the session, so the
connection is preserved. Of course it is a new context, so everything
like the "prompt mode" are lost.

Only the FD which performs the reload is kept.
2022-09-22 18:16:19 +02:00
Erwan Le Goas
d2605cf0e5 BUG/MINOR: anon: memory illegal accesses in tools.c with hash_anon and hash_ipanon
chipitsine reported in github issue #1872 that in function hash_anon and
hash_ipanon, index_hash can be equal to NB_L_HASH_WORD and can reach an
inexisting line table, the table is initialized hash_word[NB_L_HASH_WORD][20];
so hash_word[NB_L_HASH_WORD] doesn't exist.

No backport needed, except if anonymization mechanism is backported.
2022-09-22 15:44:13 +02:00
Thierry Fournier
ae6b56800f MINOR: hlua: Allow argument on lua-lod(-per-thread) directives
Allow per-lua file argument which makes multiples configuration
easier to handle

This patch fixes issue #1609.
2022-09-22 15:24:29 +02:00
Thierry Fournier
70e38e91b4 BUG/MINOR: hlua: Remove \n in Lua error message built with memprintf
Because memprintf return an error to the caller and not on screen.
the function which perform display of message on the right output
is in charge of adding \n if it is necessary.

This patch may be backported.
2022-09-21 16:02:40 +02:00
Christopher Faulet
4558437211 CLEANUP: list: Fix mt_list_for_each_entry_safe indentation
It makes the macro easier to read.
2022-09-21 16:02:40 +02:00
Aurelien DARRAGON
60cffbaca5 MINOR: list: documenting mt_list_for_each_entry_safe() macro
- Adding some comments in mt_list_for_each_entry_safe() macro to make it
  somehow understandable.
  The macro is performing critical stuff but was not documented at all.
  Moreover, nested loops with conditional tricks are used,
  making it even harder to understand the steps performed in it.

- Updating mt_list_for_each_entry_safe usage example.

- Added a "FIXME:" comment in a specific condition that seems to
  never be reached even when deeply stress-testing mt_lists
  (using test_list binary provided in the repository).
2022-09-21 16:02:40 +02:00
wrightlaw
9a8d8a3fd0 BUG/MINOR: smtpchk: SMTP Service check should gracefully close SMTP transaction
At present option smtpchk closes the TCP connection abruptly on completion of service checking,
even if successful. This can result in a very high volume of errors in backend SMTP server logs.
This patch ensures an SMTP QUIT is sent and a positive 2xx response is received from the SMTP
server prior to disconnection.

This patch depends on the following one:

 * MINOR: smtpchk: Update expect rule to fully match replies to EHLO commands

This patch should fix the issue #1812. It may be backported as far as 2.2
with the commit above On the 2.2, proxy_parse_smtpchk_opt() function is
located in src/check.c

[cf: I updated reg-tests script accordingly]
2022-09-21 16:01:42 +02:00
Christopher Faulet
2ec1ffaed0 MINOR: smtpchk: Update expect rule to fully match replies to EHLO commands
The response to EHLO command is a multiline reply. However the corresponding
expect rule only match on the first line. For now, it is not an issue. But
to be able to send the QUIT command and gracefully close the connection, we
must be sure to consume the full EHLO reply first.

To do so, the regex has been updated to match all 2xx lines at a time.
2022-09-21 15:11:26 +02:00
Christopher Faulet
330af2d7ed REGTESTS: 4be_1srv_smtpchk_httpchk_layer47errors: Return valid SMTP replies
The s1 server is acting like a SMTP server. But it sends two CRLF at the end of
each line, while only one CRLF must be returned. It only works becaue both CRLF
are received at the same time.
2022-09-21 15:11:26 +02:00
Willy Tarreau
4eaf85f5d9 MINOR: clock: do not update the global date too often
Tests with forced wakeups on a 24c/48t machine showed that we're caping
at 7.3M loops/s, which means 6.6 microseconds of loop delay without
having anything to do.

This is caused by two factors:
  - the load and update of the now_offset variable
  - the update of the global_now variable

What is happening is that threads are not running within the one-
microsecond time precision provided by gettimeofday(), so each thread
waking up sees a slightly different date and causes undesired updates
to global_now. But worse, these undesired updates mean that we then
have to adjust the now_offset to match that, and adds significant noise
to this variable, which then needs to be updated upon each call.

By only allowing sightly less precision we can completely eliminate
that contention. Here we're ignoring the 5 lowest bits of the usec
part, meaning that the global_now variable may be off by up to 31 us
(16 on avg). The variable is only used to correct the time drift some
threads might be observing in environments where CPU clocks are not
synchronized, and it's used by freq counters. In both cases we don't
need that level of precision and even one millisecond would be pretty
fine. We're just 30 times better at almost no cost since the global_now
and now_offset variables now only need to be updated 30000 times a
second in the worst case, which is unnoticeable.

After this change, the wakeup rate jumped from 7.3M/s to 66M/s, meaning
that the loop delay went from 6.6us to 0.73us, that's a 9x improvement
when under load! With real tasks we're seeing a boost from 28M to 52M
wakeups/s. The clock_update_global_date() function now only takes
1.6%, it's good enough so that we don't need to go further.
2022-09-21 09:06:28 +02:00
Willy Tarreau
58b73f9fa8 MINOR: pollers: only update the local date during busy polling
This patch modifies epoll, kqueue and evports (the 3 pollers that support
busy polling) to only update the local date in the inner polling loop,
the global one being done when leaving the loop. Testing with epoll on
a 24c/48t machine showed a boost from 53M to 352M loops/s, indicating
that the loop was spending 85% of its time updating the global date or
causing side effects (which was confirmed with perf top showing 67% in
clock_update_global_date() alone).
2022-09-21 09:06:28 +02:00
Willy Tarreau
a700420671 MINOR: clock: split local and global date updates
Pollers that support busy polling spend a lot of time (and cause
contention) updating the global date when they're looping over themselves
while it serves no purpose: what's needed is only an update on the local
date to know when to stop looping.

This patch splits clock_pudate_date() into a pair of local and global
update functions, so that pollers can be easily improved.
2022-09-21 09:06:28 +02:00
Aurelien DARRAGON
ae1e14d65b CLEANUP: tools: removing escape_chunk() function
Func is not used anymore. See e3bde807d.
2022-09-20 16:25:30 +02:00
Aurelien DARRAGON
c5bff8e550 BUG/MINOR: log: improper behavior when escaping log data
Patrick Hemmer reported an improper log behavior when using
log-format to escape log data (+E option):
Some bytes were truncated from the output:

- escape_string() function now takes an extra parameter that
  allow the caller to specify input string stop pointer in
  case the input string is not guaranteed to be zero-terminated.
- Minors checks were added into lf_text_len() to make sure dst
  string will not overflow.
- lf_text_len() now makes proper use of escape_string() function.

This should be backported as far as 1.8.
2022-09-20 16:25:30 +02:00
Christopher Faulet
b0b8e9bbd2 BUG/MINOR: mux-h1: Account consumed output data on synchronous connection error
The commit 372b38f935 ("BUG/MEDIUM: mux-h1: Handle connection error after a
synchronous send") introduced a bug. In h1_snd_buf(), consumed data are not
properly accounted if a connection error is detected. Indeed, data are
consumed when the output buffer is filled. But, on connection error, we exit
from the loop without incremented total variable accordingly.

When this happens, this leaves the channel buffer in an inconsistent
state. The buffer may be empty with some output at the channel level.
Because an error is reported, it is harmless. But it is safer to fix this
bug now to avoid any regression in future.

This patch must be backported as far as 2.2.
2022-09-20 16:23:45 +02:00
Amaury Denoyelle
0ed617ac2f BUG/MEDIUM: mux-quic: properly trim HTX buffer on snd_buf reset
MUX QUIC snd_buf operation whill return early if a qcs instance is
resetted. In this case, HTX is left untouched and the callback returns
the whole bufer size. This lead to an undefined behavior as the stream
layer is notified about a transfer but does not see its HTX buffer
emptied. In the end, the transfer may stall which will lead to a leak on
session.

To fix this, HTX buffer is now resetted when snd_buf is short-circuited.
This should fix the issue as now the stream layer can continue the
transfer until its completion.

This patch has already been tested by Tristan and is reported to solve
the github issue #1801.

This should be backported up to 2.6.
2022-09-20 15:35:33 +02:00
Amaury Denoyelle
9534e59bb9 MINOR: mux-quic: refactor snd_buf
Factorize common code between h3 and hq-interop snd_buf operation. This
is inserted in MUX QUIC snd_buf own callback.

The h3/hq-interop API has been adjusted to directly receive a HTX
message instead of a plain buf. This led to extracting part of MUX QUIC
snd_buf in qmux_http module.

This should be backported up to 2.6.
2022-09-20 15:35:29 +02:00
Amaury Denoyelle
d80fbcaca2 REORG: mux-quic: export HTTP related function in a dedicated file
Extract function dealing with HTX outside of MUX QUIC. For the moment,
only rcv_buf stream operation is concerned.

The main objective is to be able to support both TCP and HTTP proxy mode
with a common base and add specialized modules on top of it.

This should be backported up to 2.6.
2022-09-20 15:35:23 +02:00
Amaury Denoyelle
36d50bff22 REORG: mux-quic: extract traces in a dedicated source file
QUIC MUX implements several APIs to interface with stream, quic-conn and
app-ops layers. It is planified to better separate this roles, possibly
by using several files.

The first step is to extract QUIC MUX traces in a dedicated source
files. This will allow to reuse traces in multiple files.

The main objective is to be
able to support both TCP and HTTP proxy mode with a common base and add
specialized modules on top of it.

This should be backported up to 2.6.
2022-09-20 15:35:09 +02:00
Amaury Denoyelle
3dc4e5a5b9 BUG/MINOR: mux-quic: do not keep detached qcs with empty Tx buffers
A qcs instance free may be postponed in stream detach operation if the
stream is not locally closed. This condition is there to achieve
transfering data still present in Tx buffer. Once all data have been
emitted to quic-conn layer, qcs instance can be released.

However, the stream is only closed locally if HTX EOM has been seen or
it has been resetted. In case the transfer finished without EOM, a
detached qcs won't be freed even if there is no more activity on it.

This bug was not reproduced but was found on code analysis. Its precise
impact is unknown but it should not cause any leak as all qcs instances
are freed with its parent qcc connection : this should eventually happen
on MUX timeout or QUIC idle timeout.

To adjust this, condition to mark a stream as locally closed has been
extended. On qcc_streams_sent_done() notification, if its Tx buffer has
been fully transmitted, it will be closed if either FIN STREAM was set
or the stream is detached.

This must be backported up to 2.6.
2022-09-20 10:46:59 +02:00
Willy Tarreau
9f4f6b038c OPTIM: hpack-huff: reduce the cache footprint of the huffman decoder
Some tables are currently used to decode bit blocks and lengths. We do
see such lookups in perf top. We have 4 512-byte tables and one 64-byte
one. Looking closer, the second half of the table (length) has so few
variations that most of the time it will be computed in a single "if",
and never more than 3. This alone allows to cut the tables in half. In
addition, one table (bits 15-11) is only 32-element long, while another
one (bits 11-4) starts at 0x60, so we can merge the two as they do not
overlap, and further save size. We're now down to 4 256-entries tables.

This is visible in h3 and h2 where the max request rate is slightly higher
(e.g. +1.6% for h2). The huff_dec() function got slightly larger but the
overall code size shrunk:

  $ nm --size haproxy-before | grep huff_dec
  000000000000029e T huff_dec
  $ nm --size haproxy-after | grep huff_dec
  0000000000000345 T huff_dec
  $ size haproxy-before haproxy-after
     text    data     bss     dec     hex filename
  7591126  569268 2761348 10921742         a6a70e haproxy-before
  7591082  568180 2761348 10920610         a6a2a2 haproxy-after
2022-09-20 07:41:58 +02:00
Miroslav Zagorac
cbfee3a9f6 MINOR: httpclient: enabled the use of SNI presets
This commit allows setting SNI outside http_client.c code.
2022-09-19 14:39:28 +02:00
Miroslav Zagorac
133e2a23d0 CLEANUP: httpclient: deleted unused variables
The locally defined static variables 'httpclient_srv_raw' and
'httpclient_srv_ssl' are not used anywhere in the source code,
except that they are set in the httpclient_precheck() function.
2022-09-19 14:39:28 +02:00
Amaury Denoyelle
afb7b9d8e5 BUG/MEDIUM: mux-quic: fix nb_hreq decrement
nb_hreq is a counter on qcc for active HTTP requests. It is incremented
for each qcs where a full HTTP request was received. It is decremented
when the stream is closed locally :
- on HTTP response fully transmitted
- on stream reset

A bug will occur if a stream is resetted without having processed a full
HTTP request. nb_hreq will be decremented whereas it was not
incremented. This will lead to a crash when building with
DEBUG_STRICT=2. If BUG_ON_HOT are not active, nb_hreq counter will wrap
which may break the timeout logic for the connection.

This bug was triggered on haproxy.org. It can be reproduced by
simulating the reception of a STOP_SENDING frame instead of a STREAM one
by patching qc_handle_strm_frm() :

+       if (quic_stream_is_bidi(strm_frm->id))
+               qcc_recv_stop_sending(qc->qcc, strm_frm->id, 0);
+       //ret = qcc_recv(qc->qcc, strm_frm->id, strm_frm->len,
+       //               strm_frm->offset.key, strm_frm->fin,
+       //               (char *)strm_frm->data);

To fix this bug, a qcs is now flagged with a new QC_SF_HREQ_RECV. This
is set when the full HTTP request is received. When the stream is closed
locally, nb_hreq will be decremented only if this flag was set.

This must be backported up to 2.6.
2022-09-19 12:12:21 +02:00
Willy Tarreau
4b10a5c439 [RELEASE] Released version 2.7-dev6
Released version 2.7-dev6 with the following main changes :
    - MINOR: Revert part of clarifying samples support per os commit
    - BUILD: makefile: enable crypt(3) for NetBSD
    - BUG/MINOR: quic: Retransmitted frames marked as acknowledged
    - BUG/MINOR: quic: Possible crash with "tls-ticket-keys" on QUIC bind lines
    - MINOR: http-check: Remove support for headers/body in "option httpchk" version
    - BUG/MINOR: h1: Support headers case adjustment for TCP proxies
    - BUG/MINOR: quic: Possible crash when verifying certificates
    - BUILD: quic: add some ifdef around the SSL_ERROR_* for libressl
    - BUILD: ssl: fix ssl_sock_switchtx_cbk when no client_hello_cb
    - BUILD: quic: temporarly ignore chacha20_poly1305 for libressl
    - BUILD: quic: enable early data only with >= openssl 1.1.1
    - BUILD: ssl: fix the ifdef mess in ssl_sock_initial_ctx
    - BUILD: quic: fix the #ifdef in ssl_quic_initial_ctx()
    - MINOR: quic: add QUIC support when no client_hello_cb
    - MINOR: quic: Add traces about sent or resent TX frames
    - MINOR: quic: No TRACE_LEAVE() in retrieve_qc_conn_from_cid()
    - BUG/MINOR: quic: Wrong connection ID to thread ID association
    - BUG/MINOR: task: always reset a new tasklet's call date
    - BUG/MINOR: task: make task_instant_wakeup() work on a task not a tasklet
    - MINOR: task: permanently enable latency measurement on tasklets
    - CLEANUP: task: rename ->call_date to ->wake_date
    - BUG/MINOR: sched: properly account for the CPU time of dying tasks
    - MINOR: sched: store the current profile entry in the thread context
    - BUG/MINOR: stream/sched: take into account CPU profiling for the last call
    - MINOR: tasks: do not keep cpu and latency times in struct task
    - MINOR: tools: add generic pointer hashing functions
    - CLEANUP: activity: make memprof use the generic ptr_hash() function
    - CLEANUP: activity: make taskprof use ptr_hash()
    - MINOR: debug: add struct ha_caller to describe a calling location
    - CLEANUP: debug: use struct ha_caller for memstat
    - DEBUG: task: define a series of wakeup types for tasks and tasklets
    - DEBUG: task: use struct ha_caller instead of arrays of file:line
    - DEBUG: applet: instrument appctx_wakeup() to log the caller's location
    - DEBUG: task: simplify the caller recording in DEBUG_TASK
    - CLEANUP: task: move tid and wake_date into the common part
    - CLEANUP: sched: remove duplicate code in run_tasks_from_list()
    - CLEANUP: activity: make the number of sched activity entries more configurable
    - DEBUG: resolvers: unstatify process_resolvers() to make it appear in profiling
    - DEBUG: quic: export the few task handlers that often appear in task dumps
    - MEDIUM: tasks/activity: combine the called function with the caller
    - MINOR: tasks/activity: improve the caller-callee activity hash
    - MINOR: activity/cli: support aggregating task profiling outputs
    - MINOR: activity/cli: support sorting task profiling by total CPU time
    - BUG/MINOR: signals/poller: set the poller timeout to 0 when there are signals
    - BUG/MINOR: quic: Speed up the handshake completion only one time
    - BUG/MINOR: quic: Trace fix about packet number space information.
    - BUG/MINOR: h3: Crash when h3 trace verbosity is "minimal"
    - MINOR: h3: Add the quic_conn object to h3 traces
    - MINOR: h3: Missing connection argument for a TRACE_LEAVE() argument
    - MINOR: h3: Send the h3 settings with others streams (requests)
    - MINOR: dev/udp: Apply the corruption to both directions
    - BUILD: udp-perturb: Add a make target for udp-perturb tool
    - BUG/MINOR: signals/poller: ensure wakeup from signals
    - CI: cirrus-ci: bump FreeBSD image to 13-1
    - DEV: flags: fix usage message to reflect available options
    - DEV: flags: add missing CO_FL_FDLESS connection flag
    - MINOR: flags: add a new file to host flag dumping macros
    - MINOR: flags: implement a macro used to dump enums inside masks
    - MINOR: flags/channel: use flag dumping for channel flags and analysers
    - MINOR: flags/connection: use flag dumping for connection flags
    - MINOR: flags/stconn: use flag dumping for stconn and sedesc flags
    - MINOR: flags/stream: use flag dumping for stream error type
    - MINOR: flags/stream: use flag dumping for stream flags
    - MINOR: flags/task: use flag dumping for task state
    - MINOR: flags/http_ana: use flag dumping for txn flags
    - DEV: flags: remove the now unused SHOW_FLAG() definition
    - DEV: flags: remove the now useless intermediary functions
    - MINOR: flags/htx: use flag dumping to show htx and start-line flags
    - MINOR: flags/http_ana: use flag dumping to show http msg states
    - BUG/MEDIUM: proxy: ensure pause_proxy() and resume_proxy() own PROXY_LOCK
    - MINOR: listener: small API change
    - MINOR: proxy/listener: support for additional PAUSED state
    - BUG/MINOR: stats: fixing stat shows disabled frontend status as 'OPEN'
    - BUILD: flags: fix build warning in some macros used by show_flags
    - BUILD: flags: fix the fallback macros for missing stdio
    - CLEANUP: pollers: remove dead code in the polling loop
    - BUG/MINOR: mux-h1: Increment open_streams counter when H1 stream is created
    - REGTESTS: healthcheckmail: Relax matching on the healthcheck log message
    - CLEANUP: listener: function comment typo in stop_listener()
    - BUG/MINOR: listener: null pointer dereference suspected by coverity
    - MINOR: flags/fd: decode FD flags states
    - REORG: mux-h2: extract flags and enums into mux_h2-t.h
    - MINOR: flags/mux-h2: decode H2C and H2S flags
    - REGTESTS: log: test the log-forward feature
    - BUG/MEDIUM: sink: bad init sequence on tcp sink from a ring.
    - REGTESTS: ssl/log: test the log-forward with SSL
    - MEDIUM: httpclient: httpclient_create_proxy() creates a proxy for httpclient
    - MEDIUM: httpclient: allow to use another proxy
    - DOC: fix TOC in starter guide for subsection 3.3.8. Statistics
    - MINOR: httpclient: export httpclient_create_proxy()
    - MEDIUM: quic: separate path for rx and tx with set_encryption_secrets
    - BUG/MEDIUM: mux-quic: fix crash on early app-ops release
    - REORG: mux-h1: extract flags and enums into mux_h1-t.h
    - MINOR: flags/mux-h1: decode H1C and H1S flags
    - CLEANUP: mux-quic: remove stconn usage in h3/hq
    - BUG/MINOR: mux-quic: do not remotely close stream too early
    - CLEANUP: exclude udp-perturb with .gitignore
    - BUG/MEDIUM: server: segv when adding server with hostname from CLI
    - CLEANUP: quic,ssl: fix tiny typos in C comments
    - BUG/MEDIUM: captures: free() an error capture out of the proxy lock
    - BUILD: fd: fix a build warning on the DWCAS
    - MINOR: anon: add new macros and functions to anonymize contents
    - MINOR: anon: store the anonymizing key in the global structure
    - MINOR: anon: store the anonymizing key in the CLI's appctx
    - MINOR: cli: anonymize commands 'show sess' and 'show sess all'
    - MINOR: cli: anonymize 'show servers state' and 'show servers conn'
    - MINOR: config: add command-line -dC to dump the configuration file
    - SCRIPTS: announce-release: update some URLs to https
2022-09-17 12:24:53 +02:00
Willy Tarreau
68b3e135e3 SCRIPTS: announce-release: update some URLs to https
Some components like Discourse were already redirecting to https. Other
ones like docs and git are covered by the certificate, and finally
switching the advertised scheme for www should increase the ratio of
H2 and H3 in the stats (resp 8.9 and 1.9%) and possibly help spot new
issues.
2022-09-17 12:17:07 +02:00
Erwan Le Goas
b0c0501516 MINOR: config: add command-line -dC to dump the configuration file
This commit adds a new command line option -dC to dump the configuration
file. An optional key may be appended to -dC in order to produce an
anonymized dump using this key. The anonymizing process uses the same
algorithm as the CLI so that the same key will produce the same hashes
for the same identifiers. This way an admin may share an anonymized
extract of a configuration to match against live dumps. Note that key 0
will not anonymize the output. However, in any case, the configuration
is dumped after tokenizing, thus comments are lost.
2022-09-17 11:27:09 +02:00
Erwan Le Goas
acfdf7600b MINOR: cli: anonymize 'show servers state' and 'show servers conn'
Modify proxy.c in order to anonymize the following confidential data on
commands 'show servers state' and 'show servers conn':
  - proxy name
  - server name
  - server address
2022-09-17 11:27:09 +02:00
Erwan Le Goas
57e35f4b87 MINOR: cli: anonymize commands 'show sess' and 'show sess all'
Modify stream.c in order to hash the following confidential data if the
anonymized mode is enabled:
  - configuration elements such as frontend/backend/server names
  - IP addresses
2022-09-17 11:27:09 +02:00
Erwan Le Goas
54966dffda MINOR: anon: store the anonymizing key in the CLI's appctx
In order to allow users to dump internal states using a specific key
without changing the global one, we're introducing a key in the CLI's
appctx. This key is preloaded from the global one when "set anon on"
is used (and if none exists, a random one is assigned). And the key
can optionally be assigned manually for the whole CLI session.

A "show anon" command was also added to show the anon state, and the
current key if the users has sufficient permissions. In addition, a
"debug dev hash" command was added to test the feature.
2022-09-17 11:27:09 +02:00
Erwan Le Goas
fad9da83da MINOR: anon: store the anonymizing key in the global structure
Add a uint32_t key in global to hash words with it. A new CLI command
'set global-key <key>' was added to change the global anonymizing key.
The global may also be set in the configuration using the global
"anonkey" directive. For now this key is not used.
2022-09-17 11:24:53 +02:00
Erwan Le Goas
9c76637fff MINOR: anon: add new macros and functions to anonymize contents
These macros and functions will be used to anonymize strings by producing
a short hash. This will allow to match config elements against dump elements
without revealing the original data. This will later be used to anonymize
configuration parts and CLI commands output. For now only string, identifiers
and addresses are supported, but the model is easily extensible.
2022-09-17 11:24:53 +02:00
Willy Tarreau
85af760704 BUILD: fd: fix a build warning on the DWCAS
Ilya reported in issue #1816 a build warning on armhf (promoted to error
here since -Werror):

  src/fd.c: In function fd_rm_from_fd_list:
  src/fd.c:209:87: error: passing argument 3 of __ha_cas_dw discards volatile qualifier from pointer target type [-Werror=discarded-array-qualifiers]
    209 |    unlikely(!_HA_ATOMIC_DWCAS(((long *)&fdtab[fd].update), (uint32_t *)&cur_list.u32, &next_list.u32))
        |                                                                                       ^~~~~~~~~~~~~~

This happens only on such an architecture because the DWCAS requires the
pointer not the value, and gcc seems to be needlessly picky about reading
a const from a volatile! This may safely be backported to older versions.
2022-09-17 11:20:44 +02:00
Willy Tarreau
da9f258759 BUG/MEDIUM: captures: free() an error capture out of the proxy lock
Ed Hein reported in github issue #1856 some occasional watchdog panics
in 2.4.18 showing extreme contention on the proxy's lock while the libc
was in malloc()/free(). One cause of this problem is that we call free()
under the proxy's lock in proxy_capture_error(), which makes no sense
since if we can free the object under the lock after it's been detached,
we can also free it after releasing the lock (since it's not referenced
anymore).

This should be backported to all relevant versions, likely all
supported ones.
2022-09-17 11:07:19 +02:00
cui fliter
a94bedc0de CLEANUP: quic,ssl: fix tiny typos in C comments
This fixes 4 tiny and harmless typos in mux_quic.c, quic_tls.c and
ssl_sock.c. Originally sent via GitHub PR #1843.

Signed-off-by: cui fliter <imcusg@gmail.com>
[Tim: Rephrased the commit message]
[wt: further complete the commit message]
2022-09-17 10:59:59 +02:00
Aurelien DARRAGON
8d0ff28406 BUG/MEDIUM: server: segv when adding server with hostname from CLI
When calling 'add server' with a hostname from the cli (runtime),
str2sa_range() does not resolve hostname because it is purposely
called without PA_O_RESOLVE flag.

This leads to 'srv->addr_node.key' being NULL. According to Willy it
is fine behavior, as long as we handle it properly, and is already
handled like this in srv_set_addr_desc().

This patch fixes GH #1865 by adding an extra check before inserting
'srv->addr_node' into 'be->used_server_addr'. Insertion and removal
will be skipped if 'addr_node.key' is NULL.

It must be backported to 2.6 and 2.5 only.
2022-09-17 06:30:59 +02:00
Amaury Denoyelle
e3e3121227 CLEANUP: exclude udp-perturb with .gitignore
udp-perturb is a tool which can be used as a UDP gateway. It can be used
to reorder, remove or corrupt datagrams. It is compiled in dev/
directory and added to .gitignore to not clutter git status output.
2022-09-16 15:47:04 +02:00
Amaury Denoyelle
d1310f8d32 BUG/MINOR: mux-quic: do not remotely close stream too early
A stream is considered as remotely closed once we have received all the
data with the FIN bit set.

The condition to close the stream was wrong. In particular, if we
receive an empty STREAM frame with FIN bit set, this would have close
the stream even if we do not have yet received all the data. The
condition is now adjusted to ensure that Rx buffer contains all the data
up to the stream final size.

In most cases, this bug is harmless. However, if compiled with
DEBUG_STRICT=2, a BUG_ON_HOT crash would have been triggered if close is
done too early. This was most notably the case sometimes on interop test
suite with quinn or kwik clients. This can also be artificially
reproduced by simulating reception of an empty STREAM frame with FIN bit
set in qc_handle_strm_frm() :

+       if (strm_frm->fin) {
+               qcc_recv(qc->qcc, strm_frm->id, 0,
+                        strm_frm->len, strm_frm->fin,
+                        (char *)strm_frm->data);
+       }
        ret = qcc_recv(qc->qcc, strm_frm->id, strm_frm->len,
                       strm_frm->offset.key, strm_frm->fin,
                       (char *)strm_frm->data);

This must be backported up to 2.6.
2022-09-16 14:17:27 +02:00
Amaury Denoyelle
8d4ac48d3d CLEANUP: mux-quic: remove stconn usage in h3/hq
Small cleanup on snd_buf for application protocol layer.
* do not export h3_snd_buf
* replace stconn by a qcs argument. This is better as h3/hq-interop only
  uses the qcs instance.

This should be backported up to 2.6.
2022-09-16 13:53:30 +02:00
Christopher Faulet
7c4b2ec09d MINOR: flags/mux-h1: decode H1C and H1S flags
The new functions h1c_show_flags() and h1s_show_flags() decode the flags
state into a string, and are used by dev/flags:

$ /dev/flags/flags h1c 0x2200
h1c->flags = H1C_F_ST_READY | H1C_F_ST_ATTACHED

./dev/flags/flags h1s 0x190
h1s->flags = H1S_F_BODYLESS_RESP | H1S_F_NOT_FIRST | H1S_F_WANT_KAL
2022-09-15 11:01:59 +02:00
Christopher Faulet
18ad15f5c4 REORG: mux-h1: extract flags and enums into mux_h1-t.h
The same was performed for the H2 multiplexer. H1C and H1S flags are moved
in a dedicated header file. It will be mainly used to be able to decode
mux-h1 flags from the flags utility.

In this patch, we only move the flags to mux_h1-t.h.
2022-09-15 11:01:59 +02:00