Commit Graph

18406 Commits

Author SHA1 Message Date
Willy Tarreau
3d4cdb198c MEDIUM: tasks/activity: combine the called function with the caller
Now instead of getting aggregate stats per called function, we have
them per function AND per call place. The "byaddr" sort considers
the function pointer first, then the call count, so that dominant
callers of a given callee are instantly spotted. This allows to get
sorted outputs like this:

Tasks activity:
  function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
  h1_io_cb                   17357952   40.91s    2.357us   4.849m    16.76us <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  sc_conn_io_cb              10357182   6.297s    607.0ns   27.93m    161.8us <- sc_app_chk_rcv_conn@src/stconn.c:762 tasklet_wakeup
  process_stream              9891131   1.809m    10.97us   53.61m    325.2us <- sc_notify@src/stconn.c:1209 task_wakeup
  process_stream              9823934   1.887m    11.52us   48.31m    295.1us <- stream_new@src/stream.c:563 task_wakeup
  sc_conn_io_cb               9347863   16.59s    1.774us   6.143m    39.43us <- h1_wake_stream_for_recv@src/mux_h1.c:2600 tasklet_wakeup
  h1_io_cb                     501344   1.848s    3.686us   6.544m    783.2us <- conn_subscribe@src/connection.c:732 tasklet_wakeup
  sc_conn_io_cb                239717   492.3ms   2.053us   3.213m    804.3us <- qcs_notify_send@src/mux_quic.c:529 tasklet_wakeup
  h2_io_cb                     173019   4.204s    24.30us   40.95s    236.7us <- h2_snd_buf@src/mux_h2.c:6712 tasklet_wakeup
  h2_io_cb                     149487   424.3ms   2.838us   14.63s    97.87us <- h2c_restart_reading@src/mux_h2.c:856 tasklet_wakeup
  other                        101893   4.626s    45.40us   14.84s    145.7us
  quic_lstnr_dghdlr             94389   614.0ms   6.504us   30.54s    323.6us <- quic_lstnr_dgram_dispatch@src/quic_sock.c:255 tasklet_wakeup
  quic_conn_app_io_cb           92205   3.735s    40.51us   390.9ms   4.239us <- qc_lstnr_pkt_rcv@src/xprt_quic.c:6184 tasklet_wakeup_after
  qc_io_cb                      50355   19.01s    377.5us   10.65s    211.4us <- qc_treat_acked_tx_frm@src/xprt_quic.c:1695 tasklet_wakeup
  h1_io_cb                      44427   155.0ms   3.489us   21.50s    484.0us <- h1_takeover@src/mux_h1.c:4085 tasklet_wakeup
  qc_io_cb                       9018   4.924s    546.0us   3.084s    342.0us <- qc_stream_desc_ack@src/quic_stream.c:128 tasklet_wakeup
  h1_timeout_task                3236   1.172ms   362.0ns   1.119s    345.9us <- h1_release@src/mux_h1.c:1087 task_wakeup
  h1_io_cb                       2804   7.974ms   2.843us   1.980s    706.0us <- sock_conn_iocb@src/sock.c:849 tasklet_wakeup
  sc_conn_io_cb                  2804   33.44ms   11.92us   2.597s    926.2us <- h1_wake_stream_for_send@src/mux_h1.c:2610 tasklet_wakeup
  qc_io_cb                       2623   2.669s    1.017ms   1.347s    513.5us <- h3_snd_buf@src/h3.c:1084 tasklet_wakeup
  qc_process_timer                662   526.4us   795.0ns   1.081s    1.633ms <- wake_expired_tasks@src/task.c:344 task_wakeup
  quic_conn_app_io_cb             648   12.62ms   19.47us   225.7ms   348.2us <- qc_process_timer@src/xprt_quic.c:4635 tasklet_wakeup
  accept_queue_process            286   1.571ms   5.494us   72.55ms   253.7us <- listener_accept@src/listener.c:1099 tasklet_wakeup
  process_resolvers               176   157.8us   896.0ns   7.835ms   44.52us <- wake_expired_tasks@src/task.c:429 task_drop_running
  qc_io_cb                        167   10.71ms   64.12us   32.47ms   194.4us <- qc_process_timer@src/xprt_quic.c:4602 tasklet_wakeup
  sc_conn_io_cb                   123   80.05us   650.0ns   50.35ms   409.4us <- qcs_notify_recv@src/mux_quic.c:519 tasklet_wakeup
  h2_timeout_task                  32   30.69us   958.0ns   9.038ms   282.4us <- h2_release@src/mux_h2.c:1191 task_wakeup
  task_run_applet                  24   33.79ms   1.408ms   5.838ms   243.3us <- sc_applet_create@src/stconn.c:489 appctx_wakeup
  accept_queue_process             17   56.34us   3.314us   7.505ms   441.5us <- accept_queue_process@src/listener.c:165 tasklet_wakeup
  srv_cleanup_toremove_conns       16   1.133ms   70.81us   5.685ms   355.3us <- srv_cleanup_idle_conns@src/server.c:5948 task_wakeup
  srv_cleanup_idle_conns           16   74.57us   4.660us   2.797ms   174.8us <- wake_expired_tasks@src/task.c:429 task_drop_running
  quic_conn_app_io_cb              12   786.9us   65.58us   2.042ms   170.1us <- qc_process_timer@src/xprt_quic.c:4589 tasklet_wakeup
  sc_conn_io_cb                     9   20.55us   2.283us   2.475ms   275.0us <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  h2_io_cb                          8   34.12us   4.265us   1.784ms   223.0us <- h2_do_shutw@src/mux_h2.c:4656 tasklet_wakeup
  task_run_applet                   4   6.615ms   1.654ms   2.306us   576.0ns <- sc_app_chk_snd_applet@src/stconn.c:996 appctx_wakeup
  quic_conn_io_cb                   4   4.278ms   1.069ms   6.469us   1.617us <- qc_lstnr_pkt_rcv@src/xprt_quic.c:6184 tasklet_wakeup_after
  qc_io_cb                          2   20.81us   10.40us   4.943us   2.471us <- qc_init@src/mux_quic.c:2057 tasklet_wakeup
  quic_conn_app_io_cb               2   752.9us   376.4us   63.97us   31.99us <- qc_xprt_start@src/xprt_quic.c:7122 tasklet_wakeup
  quic_accept_run                   2   13.84us   6.920us   172.8us   86.42us <- quic_accept_push_qc@src/quic_sock.c:458 tasklet_wakeup
  qc_idle_timer_task                2   295.0us   147.5us   8.761us   4.380us <- wake_expired_tasks@src/task.c:344 task_wakeup
  qc_io_cb                          1   867.1us   867.1us   812.8us   812.8us <- qcs_consume@src/mux_quic.c:800 tasklet_wakeup

... and calls sorted by address like this:

Tasks activity:
  function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
  task_run_applet                  23   32.73ms   1.423ms   5.837ms   253.8us <- sc_applet_create@src/stconn.c:489 appctx_wakeup
  task_run_applet                   4   6.615ms   1.654ms   2.306us   576.0ns <- sc_app_chk_snd_applet@src/stconn.c:996 appctx_wakeup
  accept_queue_process            285   1.566ms   5.495us   72.49ms   254.3us <- listener_accept@src/listener.c:1099 tasklet_wakeup
  accept_queue_process             17   56.34us   3.314us   7.505ms   441.5us <- accept_queue_process@src/listener.c:165 tasklet_wakeup
  sc_conn_io_cb              10357182   6.297s    607.0ns   27.93m    161.8us <- sc_app_chk_rcv_conn@src/stconn.c:762 tasklet_wakeup
  sc_conn_io_cb               9347863   16.59s    1.774us   6.143m    39.43us <- h1_wake_stream_for_recv@src/mux_h1.c:2600 tasklet_wakeup
  sc_conn_io_cb                239717   492.3ms   2.053us   3.213m    804.3us <- qcs_notify_send@src/mux_quic.c:529 tasklet_wakeup
  sc_conn_io_cb                  2804   33.44ms   11.92us   2.597s    926.2us <- h1_wake_stream_for_send@src/mux_h1.c:2610 tasklet_wakeup
  sc_conn_io_cb                   123   80.05us   650.0ns   50.35ms   409.4us <- qcs_notify_recv@src/mux_quic.c:519 tasklet_wakeup
  sc_conn_io_cb                     9   20.55us   2.283us   2.475ms   275.0us <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  process_resolvers               159   145.9us   917.0ns   7.823ms   49.20us <- wake_expired_tasks@src/task.c:429 task_drop_running
  srv_cleanup_idle_conns           16   74.57us   4.660us   2.797ms   174.8us <- wake_expired_tasks@src/task.c:429 task_drop_running
  srv_cleanup_toremove_conns       16   1.133ms   70.81us   5.685ms   355.3us <- srv_cleanup_idle_conns@src/server.c:5948 task_wakeup
  process_stream              9891130   1.809m    10.97us   53.61m    325.2us <- sc_notify@src/stconn.c:1209 task_wakeup
  process_stream              9823933   1.887m    11.52us   48.31m    295.1us <- stream_new@src/stream.c:563 task_wakeup
  h1_io_cb                   17357952   40.91s    2.357us   4.849m    16.76us <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  h1_io_cb                     501344   1.848s    3.686us   6.544m    783.2us <- conn_subscribe@src/connection.c:732 tasklet_wakeup
  h1_io_cb                      44427   155.0ms   3.489us   21.50s    484.0us <- h1_takeover@src/mux_h1.c:4085 tasklet_wakeup
  h1_io_cb                       2804   7.974ms   2.843us   1.980s    706.0us <- sock_conn_iocb@src/sock.c:849 tasklet_wakeup
  h1_timeout_task                3236   1.172ms   362.0ns   1.119s    345.9us <- h1_release@src/mux_h1.c:1087 task_wakeup
  h2_timeout_task                  32   30.69us   958.0ns   9.038ms   282.4us <- h2_release@src/mux_h2.c:1191 task_wakeup
  h2_io_cb                     173019   4.204s    24.30us   40.95s    236.7us <- h2_snd_buf@src/mux_h2.c:6712 tasklet_wakeup
  h2_io_cb                     149487   424.3ms   2.838us   14.63s    97.87us <- h2c_restart_reading@src/mux_h2.c:856 tasklet_wakeup
  h2_io_cb                          8   34.12us   4.265us   1.784ms   223.0us <- h2_do_shutw@src/mux_h2.c:4656 tasklet_wakeup
  qc_io_cb                      50355   19.01s    377.5us   10.65s    211.4us <- qc_treat_acked_tx_frm@src/xprt_quic.c:1695 tasklet_wakeup
  qc_io_cb                       9018   4.924s    546.0us   3.084s    342.0us <- qc_stream_desc_ack@src/quic_stream.c:128 tasklet_wakeup
  qc_io_cb                       2623   2.669s    1.017ms   1.347s    513.5us <- h3_snd_buf@src/h3.c:1084 tasklet_wakeup
  qc_io_cb                        167   10.71ms   64.12us   32.47ms   194.4us <- qc_process_timer@src/xprt_quic.c:4602 tasklet_wakeup
  qc_io_cb                          2   20.81us   10.40us   4.943us   2.471us <- qc_init@src/mux_quic.c:2057 tasklet_wakeup
  qc_io_cb                          1   867.1us   867.1us   812.8us   812.8us <- qcs_consume@src/mux_quic.c:800 tasklet_wakeup
  qc_idle_timer_task                2   295.0us   147.5us   8.761us   4.380us <- wake_expired_tasks@src/task.c:344 task_wakeup
  quic_conn_io_cb                   4   4.278ms   1.069ms   6.469us   1.617us <- qc_lstnr_pkt_rcv@src/xprt_quic.c:6184 tasklet_wakeup_after
  quic_conn_app_io_cb           92205   3.735s    40.51us   390.9ms   4.239us <- qc_lstnr_pkt_rcv@src/xprt_quic.c:6184 tasklet_wakeup_after
  quic_conn_app_io_cb             648   12.62ms   19.47us   225.7ms   348.2us <- qc_process_timer@src/xprt_quic.c:4635 tasklet_wakeup
  quic_conn_app_io_cb              12   786.9us   65.58us   2.042ms   170.1us <- qc_process_timer@src/xprt_quic.c:4589 tasklet_wakeup
  quic_conn_app_io_cb               2   752.9us   376.4us   63.97us   31.99us <- qc_xprt_start@src/xprt_quic.c:7122 tasklet_wakeup
  quic_lstnr_dghdlr             94389   614.0ms   6.504us   30.54s    323.6us <- quic_lstnr_dgram_dispatch@src/quic_sock.c:255 tasklet_wakeup
  qc_process_timer                662   526.4us   795.0ns   1.081s    1.633ms <- wake_expired_tasks@src/task.c:344 task_wakeup
  quic_accept_run                   2   13.84us   6.920us   172.8us   86.42us <- quic_accept_push_qc@src/quic_sock.c:458 tasklet_wakeup
  other                        101892   4.626s    45.40us   14.84s    145.7us

It already becomes visible that some tasks have different very costs
depending where they're called (e.g. process_stream). The method used
to wake them up is also shown. Applets are handled specially and shown
as appctx_wakeup.
2022-09-08 16:21:22 +02:00
Willy Tarreau
41e701e2c1 DEBUG: quic: export the few task handlers that often appear in task dumps
The following task/tasklet handlers often appear in "show profiling tasks"
but were not resolved since static:

 qc_io_cb, quic_conn_app_io_cb, process_timer,
 quic_accept_run, qc_idle_timer_task

This commit simply exports them so they can be resolved now. "process_timer"
which was a bit too generic and renamed to qc_process_timer.
2022-09-08 16:13:38 +02:00
Willy Tarreau
0fbc16cfb9 DEBUG: resolvers: unstatify process_resolvers() to make it appear in profiling
The function appears like this in "show profiling tasks", so let's export
it:

  function       calls  cpu_tot  cpu_avg  lat_tot  lat_avg
  main+0x1463f0     92  77.28us  839.0ns  2.018ms  21.93us <- wake_expired_tasks@src/task.c:429 task_drop_running
2022-09-08 16:13:38 +02:00
Willy Tarreau
a3423873fe CLEANUP: activity: make the number of sched activity entries more configurable
This removes all the hard-coded 8-bit and 256 entries to use a pair of
macros instead so that we can more easily experiment with larger table
sizes if needed.
2022-09-08 14:55:09 +02:00
Willy Tarreau
a9a2384612 CLEANUP: sched: remove duplicate code in run_tasks_from_list()
Now that ->wake_date is common to tasks and tasklets, we don't need
anymore to carry a duplicate control block to read and update it for
tasks and tasklets. And given that this code was present early in the
if/else fork between tasks and tasklets, taking it out of the block
allows to move the task part into a more visible "else" branch that
also allows to factor the epilogue that resets th_ctx->current and
updates profile_entry->cpu_time, which also used to be duplicated.

Overall, doing just that saved 253 bytes in the function, or ~1/6,
which is not bad considering that it's on a hot path. And the code
got much ore readable.
2022-09-08 14:30:38 +02:00
Willy Tarreau
e0e6d81460 CLEANUP: task: move tid and wake_date into the common part
There used to be one tid for tasklets and a thread_mask for tasks. Since
2.7, both tasks and tasklets now use a tid (albeit with a very slight
semantic difference for the negative value), to in order to limit code
duplication and to ease debugging it makes sense to move tid into the
common part. One limitation is that it will leave a hole in the structure,
but we now have the wake_date that is always present and can move there as
well to plug the hole.

This results in something overall pretty clean (and cleaner than before),
with the low-level stuff (state,tid,process,context) appearing first, then
the caller stuff (caller,wake_date,calls,debug) next, and finally the
type-specific stuff (rq/wq/expire/nice).
2022-09-08 14:30:38 +02:00
Willy Tarreau
2830d282e5 DEBUG: task: simplify the caller recording in DEBUG_TASK
Instead of storing an index that's swapped at every call, let's use the
two pointers as a shifting history. Now we have a permanent "caller"
field that records the last caller, and an optional prev_caller in the
debug section enabled by DEBUG_TASK that keeps a copy of the previous
caller one. This way, not only it's much easier to follow what's
happening during debugging, but it saves 8 bytes in the struct task in
debug mode and still keeps it under 2 cache lines in nominal mode, and
this will finally be usable everywhere and later in profiling.

The caller_idx was also used as a hint that the entry was freed, in order
to detect wakeup-after-free. This was changed by setting caller to -1
instead and preserving its value in caller[1].

Finally, the operations were made atomic. That's not critical but since
it's used for debugging and race conditions represent a significant part
of the issues in multi-threaded mode, it seems wise to at least eliminate
some possible factors of faulty analysis.
2022-09-08 14:30:38 +02:00
Willy Tarreau
8d71abf0cd DEBUG: applet: instrument appctx_wakeup() to log the caller's location
appctx_wakeup() relies on task_wakeup(), but since it calls it from a
function, the calling place is always appctx_wakeup() itself, which is
not very useful.

Let's turn it to a macro so that we can log the location of the caller
instead. As an example, the cli_io_handler() which used to be seen as
this:

  (gdb) p *appctx->t.debug.caller[0]
  $10 = {
    func = 0x9ffb78 <__func__.37996> "appctx_wakeup",
    file = 0x9b336a "include/haproxy/applet.h",
    line = 110,
    what = 1 '\001',
    arg8 = 0 '\000',
    arg32 = 0
  }

Now shows the more useful:

  (gdb) p *appctx->t.debug.caller[0]
  $6 = {
    func = 0x9ffe80 <__func__.38641> "sc_app_chk_snd_applet",
    file = 0xa00320 "src/stconn.c",
    line = 996,
    what = 6 '\006',
    arg8 = 0 '\000',
    arg32 = 0
  }
2022-09-08 14:30:38 +02:00
Willy Tarreau
e08af9a0f4 DEBUG: task: use struct ha_caller instead of arrays of file:line
This reduces the task struct by 8 bytes, reduces the code size a little
bit by simplifying the calling convention (one argument dropped), and
as a bonus provides the function name in the caller.
2022-09-08 14:30:38 +02:00
Willy Tarreau
d2b2ad902b DEBUG: task: define a series of wakeup types for tasks and tasklets
The WAKEUP_* values will be used to report how a task/tasklet was woken
up, and task_wakeup_type_str() wlil report the associated function name.
2022-09-08 14:30:16 +02:00
Willy Tarreau
d96d214b4c CLEANUP: debug: use struct ha_caller for memstat
The memstats code currently defines its own file/function/line number,
type and extra pointer. We don't need to keep them separate and we can
easily replace them all with just a struct ha_caller. Note that the
extra pointer could be converted to a pool ID stored into arg8 or
arg32 and be dropped as well, but this would first require to define
IDs for pools (which we currently do not have).
2022-09-08 14:19:15 +02:00
Willy Tarreau
7f2f1f294c MINOR: debug: add struct ha_caller to describe a calling location
The purpose of this structure is to assemble all constant parts of a
generic calling point for a specific event. These ones are created by
the compiler as a static const element outside of the code path, so
they cost nothing in terms of CPU, and a pointer to that descriptor
can be passed to the place that needs it. This is very similar to what
is being done for the mem_stat stuff.

This will be useful to simplify and improve DEBUG_TASK.
2022-09-08 14:19:15 +02:00
Willy Tarreau
4c1bc01f31 CLEANUP: activity: make taskprof use ptr_hash()
There's no more point using a different hash function here, xxh64 is
of course better distributed but we really don't care so let's unify
the code.
2022-09-08 14:19:15 +02:00
Willy Tarreau
245d32fe8f CLEANUP: activity: make memprof use the generic ptr_hash() function
There's no need to keep a local version of that function anymore.
2022-09-08 14:19:15 +02:00
Willy Tarreau
4a3907617f MINOR: tools: add generic pointer hashing functions
There are a few places where it's convenient to hash a pointer to compute
a statistics bucket. Here we're basically reusing the hash that was used
by memory profiling with a minor update that the multiplier was corrected
to be prime and stand by its promise to have equal numbers of 1 and 0,
and that 32-bit platforms won't lose range anymore.

A two-pointer variant was also added.
2022-09-08 14:19:15 +02:00
Willy Tarreau
6a28a30efa MINOR: tasks: do not keep cpu and latency times in struct task
It was a mistake to put these two fields in the struct task. This
was added in 1.9 via commit 9efd7456e ("MEDIUM: tasks: collect per-task
CPU time and latency"). These fields are used solely by streams in
order to report the measurements via the lat_ns* and cpu_ns* sample
fetch functions when task profiling is enabled. For the rest of the
tasks, this is pure CPU waste when profiling is enabled, and memory
waste 100% of the time, as the point where these latencies and usages
are measured is in the profiling array.

Let's move the fields to the stream instead, and have process_stream()
retrieve the relevant info from the thread's context.

The struct task is now back to 120 bytes, i.e. almost two cache lines,
with 32 bit still available.
2022-09-08 14:19:15 +02:00
Willy Tarreau
beee600491 BUG/MINOR: stream/sched: take into account CPU profiling for the last call
When task profiling is enabled, the reported CPU time for short requests
and responses (e.g. redirect) is always zero in the logs, because
process_stream() is only called once and the CPU time is measured after
it returns. This is particuarly annoying when dealing with denies and in
general anything that deals with parasitic traffic because it can be
difficult to figure where the CPU is spent.

The solution taken in this patch consists in having process_stream()
update the cpu time itself before logging and quitting. It's very simple.
It will not take into account the time taken to produce the log nor
freeing the stream, but that's marginal compared to always logging zero.
The task's wake_date is also reset so that the scheduler doesn't have to
perform these operations again. This is dependent on the following patch:

   MINOR: sched: store the current profile entry in the thread context

It should be backported to 2.6 as it does help for troubleshooting.
2022-09-08 14:19:15 +02:00
Willy Tarreau
1efddfa6bf MINOR: sched: store the current profile entry in the thread context
The profile entry that corresponds to the current task/tasklet being
profiled is now stored into the thread's context. This will allow it
to be accessed from the tasks themselves. This is needed for an upcoming
fix.
2022-09-08 14:19:15 +02:00
Willy Tarreau
62b5b96bcc BUG/MINOR: sched: properly account for the CPU time of dying tasks
When task profiling is enabled, the scheduler can measure and report
the cumulated time spent in each task and their respective latencies. But
this was wrong for tasks with few wakeups as well as for self-waking ones,
because the call date needed to measure how long it takes to process the
task is retrieved in the task itself (->wake_date was turned to the call
date), and we could face two conditions:
  - a new wakeup while the task is executing would reset the ->wake_date
    field before returning and make abnormally low values being reported;
    that was likely the case for taskèrun_applet for self-waking applets;

  - when the task dies, NULL is returned and the call date couldn't be
    retrieved, so that CPU time was not being accounted for. This was
    particularly visible with process_stream() which is usually called
    only twice per request, and whose time was systematically halved.

The cleanest solution here is to keep in mind that the scheduler already
uses quite a bit of local context in th_ctx, and place the intermediary
values there so that they cannot vanish. The wake_date has to be reset
immediately once read, and only its copy is used along the function. Note
that this must be done both for tasks and tasklet, and that until recently
tasklets were also able to report wrong values due to their sole dependency
on TH_FL_TASK_PROFILING between tests.

One nice benefit for future improvements is that such information will now
be available from the task without having to be stored into the task itself
anymore.

Since the tasklet part was computed on wrapping 32-bit arithmetics and
the task one was on 64-bit, the values were now consistently moved to
32-bit as it's already largely sufficient (4s spent in a task is more
than twice what the watchdog would tolerate). Some further cleanups might
be necessary, but the patch aimed at staying minimal.

Task profiling output after 1 million HTTP request previously looked like
this:

  Tasks activity:
    function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
    h1_io_cb                    2012338   4.850s    2.410us   12.91s    6.417us
    process_stream              2000136   9.594s    4.796us   34.26s    17.13us
    sc_conn_io_cb               2000135   1.973s    986.0ns   30.24s    15.12us
    h1_timeout_task                 137      -         -      2.649ms   19.34us
    accept_queue_process             49   152.3us   3.107us   321.7yr   6.564yr
    main+0x146430                     7   5.250us   750.0ns   25.92us   3.702us
    srv_cleanup_idle_conns            1   559.0ns   559.0ns   918.0ns   918.0ns
    task_run_applet                   1      -         -      2.162us   2.162us

  Now it looks like this:
  Tasks activity:
    function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
    h1_io_cb                    2014194   4.794s    2.380us   13.75s    6.826us
    process_stream              2000151   20.01s    10.00us   36.04s    18.02us
    sc_conn_io_cb               2000148   2.167s    1.083us   32.27s    16.13us
    h1_timeout_task                 198   54.24us   273.0ns   3.487ms   17.61us
    accept_queue_process             52   158.3us   3.044us   409.9us   7.882us
    main+0x1466e0                    18   16.77us   931.0ns   63.98us   3.554us
    srv_cleanup_toremove_conns        8   282.1us   35.26us   546.8us   68.35us
    srv_cleanup_idle_conns            3   149.2us   49.73us   8.131us   2.710us
    task_run_applet                   3   268.1us   89.38us   11.61us   3.871us

Note the two-fold difference on process_stream().

This feature is essentially used for debugging so it has extremely limited
impact. However it's used quite a bit more in bug reports and it would be
desirable that at least 2.6 gets this fix backported. It depends on at least
these two previous patches which will then also have to be backported:

     MINOR: task: permanently enable latency measurement on tasklets
     CLEANUP: task: rename ->call_date to ->wake_date
2022-09-08 14:19:15 +02:00
Willy Tarreau
04e50b3d32 CLEANUP: task: rename ->call_date to ->wake_date
This field is misnamed because its real and important content is the
date the task was woken up, not the date it was called. It temporarily
holds the call date during execution but this remains confusing. In
fact before the latency measurements were possible it was indeed a call
date. Thus is will now be called wake_date.

This change is necessary because a subsequent fix will require the
introduction of the real call date in the thread ctx.
2022-09-08 14:19:15 +02:00
Willy Tarreau
768c2c5678 MINOR: task: permanently enable latency measurement on tasklets
When tasklet latency measurement was enabled in 2.4 with commit b2285de04
("MINOR: tasks: also compute the tasklet latency when DEBUG_TASK is set"),
the feature was conditionned on DEBUG_TASK because the field would add 8
bytes to the struct tasklet.

This approach was not a very good idea because the struct ends on an int
anyway thus it does finish with a 32-bit hole regardless of the presence
of this field. What is true however is that adding it turned a 64-byte
struct to 72-byte when caller debugging is enabled.

This patch revisits this with a minor change. Now only the lowest 32
bits of the call date are stored, so they always fit in the remaining
hole, and this allows to remove the dependency on DEBUG_TASK. With
debugging off, we're now seeing a 48-byte struct, and with debugging
on it's exactly 64 bytes, thus still exactly one cache line. 32 bits
allow a latency of 4 seconds on a tasklet, which already indicates a
completely dead process, so there's no point storing the upper bits at
all. And even in the event it would happen once in a while, the lost
upper bits do not really add any value to the debug reports. Also, now
one tasklet wakeup every 4 billion will not be sampled due to the test
on the value itself. Similarly we just don't care, it's statistics and
the measurements are not 9-digit accurate anyway.
2022-09-08 14:19:15 +02:00
Willy Tarreau
0fae3a0360 BUG/MINOR: task: make task_instant_wakeup() work on a task not a tasklet
There's a subtle (harmless) bug in task_instant_wakeup(). As it uses
some tasklet code instead of some task code, the debug part also acts
on the tasklet equivalent, and the call_date is only set when DEBUG_TASK
is set instead of inconditionally like with tasks. As such, without this
debugging macro, call dates are not updated for tasks woken this way.

There isn't any impact yet because this function was introduced in 2.6 to
solve certain classes of issues and is not used yet, and in the worst case
it would only affect the reported latency time.

This may be backported to 2.6 in case a future fix would depend on it but
currently will not fix existing code.
2022-09-08 14:19:15 +02:00
Willy Tarreau
f27acd961e BUG/MINOR: task: always reset a new tasklet's call date
The tasklet's call date was not reset, so if profiling was enabled while
some tasklets were in the run queue, their initial random value could be
used to preload a bogus initial latency value into the task profiling bin.
Let's just zero the initial value.

This should be backported to 2.4 as it was brought with initial commit
b2285de04 ("MINOR: tasks: also compute the tasklet latency when DEBUG_TASK
is set"). The impact is very low though.
2022-09-08 14:19:15 +02:00
Frédéric Lécaille
3122c75fd1 BUG/MINOR: quic: Wrong connection ID to thread ID association
To work, quic_pin_cid_to_tid() must set cid[0] to a value with <target_id>
as <global.nbthread> modulo. For each integer n, (n - (n % m)) + d has always
d as modulo m (with d < m).

So, this statement seemed correct:

     cid[0] = cid[0] - (cid[0] % global.nbthread) + target_tid;

except when n wraps or when another modulo is applied to the addition result.
Here, for 8bit modulo arithmetic, if m does not divides 256, this cannot
works for values which wraps when we increment them by d.
For instance n=255 m=3 and d=1 the formula result is 0 (should be d).

To fix this, we first limit c[0] to 255 - <target_id> to prevent c[0] from wrapping.

Thank you to @esb for having reported this issue in GH #1855.

Must be backported to 2.6
2022-09-07 15:59:43 +02:00
Frédéric Lécaille
614742b79c MINOR: quic: No TRACE_LEAVE() in retrieve_qc_conn_from_cid()
This macro was confused with TRACE_ENTER().

Should be backported to 2.6.
2022-09-07 15:59:43 +02:00
Frédéric Lécaille
449804e27d MINOR: quic: Add traces about sent or resent TX frames
Very useful to help in debugging issues, especially during retransmissions.

Should be backported to 2.6
2022-09-07 15:59:29 +02:00
William Lallemand
70a6e637b4 MINOR: quic: add QUIC support when no client_hello_cb
Add QUIC support to the ssl_sock_switchctx_cbk() variant used only when
no client_hello_cb is available.

This could be used with libreSSL implementation of QUIC for example.
It also works with quictls when HAVE_SSL_CLIENT_HELLO_CB is removed from
openss-compat.h
2022-09-07 11:33:28 +02:00
William Lallemand
373ce73695 BUILD: quic: fix the #ifdef in ssl_quic_initial_ctx()
As done on with ssl_sock_initial_ctx(), cleanup the ifdef for the
client_hello_cb and the no anti replay.
2022-09-07 11:11:59 +02:00
William Lallemand
4b7938d160 BUILD: ssl: fix the ifdef mess in ssl_sock_initial_ctx
ssl_sock_initial_ctx uses the wrong #ifdef to check the availability of
the client_hello_cb.

Cleanup the #ifdef, add comments and indentation.
2022-09-07 10:54:17 +02:00
William Lallemand
e6ec626ac5 BUILD: quic: enable early data only with >= openssl 1.1.1
Disable the early data in the QUIC code when not built with openssl >=
1.1.1.

LibreSSL 3.6.0 is impacted.
2022-09-07 09:33:46 +02:00
William Lallemand
d2be9d4c48 BUILD: quic: temporarly ignore chacha20_poly1305 for libressl
LibreSSL does not implement EVP_chacha20_poly1305() with EVP_CIPHER but
uses the EVP_AEAD API instead:

https://man.openbsd.org/EVP_AEAD_CTX_init

This patch disables this cipher for libreSSL for now.
2022-09-07 09:33:46 +02:00
William Lallemand
844009d77a BUILD: ssl: fix ssl_sock_switchtx_cbk when no client_hello_cb
When building HAProxy with USE_QUIC and libressl 3.6.0, the
ssl_sock_switchtx_cbk symbol is not found because libressl does not
implement the client_hello_cb.

A ssl_sock_switchtx_cbk version for the servername callback is available
but wasn't exported correctly.
2022-09-07 09:33:46 +02:00
William Lallemand
6d74e179ee BUILD: quic: add some ifdef around the SSL_ERROR_* for libressl
SSL_ERROR_WANT_ASYNC, SSL_ERROR_WANT_ASYNC_JOB and
SSL_ERROR_WANT_CLIENT_HELLO_CB does not seems supported by libressl.
2022-09-07 09:33:46 +02:00
Frédéric Lécaille
2be0ac55e1 BUG/MINOR: quic: Possible crash when verifying certificates
This verification is done by ssl_sock_bind_verifycbk() which is set at different
locations in the ssl_sock.c code . About QUIC connections, there are a lot of chances
the connection object is not initialized when entering this function. What must
be accessed is the SSL object to retrieve the connection or quic_conn objects,
then the bind_conf object of the listener. If the connection object is not found,
we try to find the quic_conn object.

Modify ssl_sock_dump_errors() interface which takes a connection object as parameter
to also passed a quic_conn object as parameter. Again this function try first
to access the connection object if not NULL or the quic_conn object if not.

There is a remaining thing to do for QUIC: store the certificate verification error
code as it is currently stored in the connection object. This error code is at least
used by the "bc_err" and "fc_err" sample fetches.

There are chances this bug is in relation with GH #1851. Thank you to @tasavis
for the report.

Must be merged into 2.6.
2022-09-06 20:42:02 +02:00
Christopher Faulet
a9e934bbd1 BUG/MINOR: h1: Support headers case adjustment for TCP proxies
On frontend side, "h1-case-adjust-bogus-client" option is now supported in
TCP mode. It is important to be able to adjust the case of response headers
when a connection is routed to an HTTP backend. In this case, the client
connection is upgraded to H1.

On backend side, "h1-case-adjust-bogus-server" option is now also supported
in TCP mode to be able to perform HTTP health-checks with a case adjustment
of the request headers.

This patch should be backported as far as 2.0.
2022-09-06 18:23:14 +02:00
Christopher Faulet
4b5f3029bc MINOR: http-check: Remove support for headers/body in "option httpchk" version
This trick is deprecated since the health-check refactoring, It is now
invalid. It means the following line will trigger an error during the
configuration parsing:

  option httpchk OPTIONS * HTTP/1.1\r\nHost:\ www

It must be replaced by:

  option httpchk OPTIONS * HTTP/1.1
  http-check send hdr Host www
2022-09-06 18:23:14 +02:00
Frédéric Lécaille
6aec1f380e BUG/MINOR: quic: Possible crash with "tls-ticket-keys" on QUIC bind lines
ssl_tlsext_ticket_key_cb() is called when "tls-ticket-keys" option is used on a
"bind" line. It needs to have an access to the TLS ticket keys which have been
stored into the listener bind_conf struct. The fix consists in nitializing the
<ref> variable (references to TLS secret keys) the correct way when this callback
is called for a QUIC connection. The bind_conf struct is store into the quic_conn
object (QUIC connection).

This issue may be in relation with GH #1851. Thank you for @tasavis for the report.

Must be backported to 2.6.
2022-09-06 17:56:53 +02:00
Frédéric Lécaille
025945f12c BUG/MINOR: quic: Retransmitted frames marked as acknowledged
Obviously, frames which are duplicated from others must not be retransmitted if
the original frame they were duplicated from was already acknowledged.
This should have been detected by qc_build_frms() which skips such frames,
except if the QUIC xprt does really bad things which are not supported by
the upper layer. This will have to be checked with Amaury.

To prevent the retransmision of these frames which leads to crashes as reported by
hpn0t0ad this gdb backtrace in GH #1809 where the frame builder tries to copy a huge
number of bytes to the packet buffer:

Thread 7 (Thread 0x7fddf373a700 (LWP 13)):
 #0  __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:520
No locals.
 #1  0x000055b17435705e in quic_build_stream_frame (buf=0x7fddf372ef78, end=<optimized out>, frm=0x7fdde08d3470, conn=<optimized out>) at src/quic_frame.c:515
        to_copy = 18446697703428890384
        stream = 0x7fdde08d3490
        wrap = <optimized out>

which matches this part of quic_frame.c code:

    wrap = (const unsigned char *)b_wrap(stream->buf);
    if (stream->data + stream->len > wrap) {
        size_t to_copy = wrap - stream->data;
        memcpy(*buf, stream->data, to_copy);
        *buf += to_copy;

we release as soon as possible the impacted frames as there is really no need
to retransmit such frames.

Thank you to @hpn0t0ad for having provided us with useful traces in github
issue #1809.

Must be backported in 2.6.
2022-09-06 14:23:52 +02:00
Brad Smith
2f105b8a45 BUILD: makefile: enable crypt(3) for NetBSD
Allow NetBSD to support encrypted passwords in Userlists.
2022-09-03 06:11:08 +02:00
Brad Smith
ef9d594839 MINOR: Revert part of clarifying samples support per os commit
Commit 5c83e3a156 made some adjustments
to clarify which TCP_INFO information is supported by each respective
OS.

There was a comment like so..

Note that fc_rtt and fc_rttvar are supported on any OS that has TCP_INFO,
not just linux/freebsd/netbsd, so we continue to expose them unconditionally.

But the diff didn't do so in a consistent manner.
2022-09-03 06:11:08 +02:00
Willy Tarreau
3bb2b5db50 [RELEASE] Released version 2.7-dev5
Released version 2.7-dev5 with the following main changes :
    - BUG/MINOR: mux-quic: Fix memleak on QUIC stream buffer for unacknowledged data
    - BUG/MEDIUM: cpu-map: fix thread 1's affinity affecting all threads
    - MINOR: cpu-map: remove obsolete diag warning about combined ranges
    - BUG/MAJOR: mworker: fix infinite loop on master with no proxies.
    - REGTESTS: launch http_reuse_always in mworker mode
    - BUG/MINOR: quix: Memleak for non in flight TX packets
    - BUG/MINOR: quic: Wrong list_for_each_entry() use when building packets from qc_do_build_pkt()
    - BUG/MINOR: quic: Safer QUIC frame builders
    - MINOR: quic: Replace MT_LISTs by LISTs for RX packets.
    - BUG/MEDIUM: applet: fix incorrect check for abnormal return condition from handler
    - BUG/MINOR: applet: make the call_rate only count the no-progress calls
    - MEDIUM: peers: limit the number of updates sent at once
    - BUILD: tcp_sample: fix build of get_tcp_info() on OpenBSD
    - BUG/MINOR: resolvers: return the correct value in resolvers_finalize_config()
    - BUG/MINOR: mworker: does not create the "default" resolvers in wait mode
    - BUG/MINOR: tcpcheck: Disable QUICKACK only if data should be sent after connect
    - REGTESTS: Fix prometheus script to perform HTTP health-checks
    - MINOR: resolvers: shut the warning when "default" resolvers is implicit
    - Revert "BUG/MINOR: quix: Memleak for non in flight TX packets"
    - BUG/MINOR: quic: Leak in qc_release_lost_pkts() for non in flight TX packets
    - BUG/MINOR: quic: Stalled connections (missing I/O handler wakeup)
    - CLEANUP: quic: No more use ->rx_list MT_LIST entry point (quic_rx_packet)
    - CLEANUP: quic: Remove a useless check in qc_lstnr_pkt_rcv()
    - MINOR: quic: Remove useless traces about references to TX packets
    - Revert "MINOR: quic: Remove useless traces about references to TX packets"
    - DOC: configuration: do-resolve doesn't work with a port in the string
    - MINOR: sample: add the host_only and port_only converters
    - BUG/MINOR: httpclient: fix resolution with port
    - DOC: configuration.txt: do-resolve must use host_only to remove its port.
    - BUG/MINOR: quic: Null packet dereferencing from qc_dup_pkt_frms() trace
    - BUG/MINOR: quic: Frames added to packets even if not built.
    - BUG/MEDIUM: spoe: Properly update streams waiting for a ACK in async mode
    - BUG/MEDIUM: peers: Add connect and server timeut to peers proxy
    - BUG/MEDIUM: peers: Don't use resync timer when local resync is in progress
    - BUG/MEDIUM: peers: Don't start resync on reload if local peer is not up-to-date
    - BUG/MINOR: hlua: Rely on CF_EOI to detect end of message in HTTP applets
    - BUG/MEDIUM: mux-h1: do not refrain from signaling errors after end of input
    - BUG/MINOR: epoll: do not actively poll for Rx after an error
    - MINOR: raw-sock: don't try to send if an error was already reported
    - BUG/MINOR: quic: Missing header protection AES cipher context initialisations (draft-v2)
    - MINOR: quic: Add a trace to distinguish the datagram from the packets inside
    - BUG/MINOR: ssl: fix deinit of the ca-file tree
    - BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free()
    - BUG/MINOR: tcpcheck: Disable QUICKACK for default tcp-check (with no rule)
    - BUG/MEDIUM: ssl: Fix a UAF when old ckch instances are released
    - BUG/MINOR: ssl: revert two wrong fixes with ckhi_link
    - BUG/MINOR: dev/udp: properly preset the rx address size
    - BUILD: debug: make sure debug macros are never empty
    - MINOR: quic: Move traces about RX/TX bytes from QUIC_EV_CONN_PRSAFRM event
    - BUG/MINOR: quic: TX frames memleak
    - BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free() v2
    - MINOR: sink/ring: rotate non-empty file-backed contents only
    - BUG/MINOR: regex: Properly handle PCRE2 lib compiled without JIT support
    - REGTESTS: http_request_buffer: Add a barrier to not mix up log messages
    - BUG/MEDIUM: mux-h1: always use RST to kill idle connections in pools
    - MINOR: backend: always satisfy the first req reuse rule with l7 retries
    - BUG/MINOR: quic: Do not ack when probing
    - MINOR: quic: Add TX frames addresses to traces to several trace events
    - MINOR: quic: Trace typo fix in qc_release_frm()
    - BUG/MINOR: quic: Frames leak during retransmissions
    - BUG/MINOR: h2: properly set the direction flag on HTX response
    - BUG/MEDIUM: httpclient: always detach the caller before self-killing
    - BUG/MINOR: httpclient: only ask for more room on failed writes
    - BUG/MINOR: httpclient: keep-alive was accidentely disabled
    - MEDIUM: httpclient: enable ALPN support on outgoing https connections
    - BUG/MINOR: mux-h2: fix the "show fd" dest buffer for the subscriber
    - BUG/MINOR: mux-h1: fix the "show fd" dest buffer for the subscriber
    - BUG/MINOR: mux-fcgi: fix the "show fd" dest buffer for the subscriber
    - DEBUG: stream: minor rearrangement of a few fields in struct stream.
    - MINOR: debug: report applet pointer and handler in crashes when known
    - MINOR: mux-h2: extract the stream dump function out of h2_show_fd()
    - MINOR: mux-h2: extract the connection dump function out of h2_show_fd()
    - MINOR: muxes: add a "show_sd" helper to complete "show sess" dumps
    - MINOR: mux-h2: provide a "show_sd" helper to output stream debugging info
    - MINOR: mux-h2: insert line breaks in "show sess all" output for legibility
    - MINOR: mux-quic: provide a "show_sd" helper to output stream debugging info
    - MINOR: mux-h1: split "show_fd" into connection and stream
    - MINOR: mux-h1: provide a "show_sd" helper to output stream debugging info
    - BUG/MINOR: http-act: initialize http fmt head earlier
2022-09-02 19:36:50 +02:00
Willy Tarreau
6a03a0d86d BUG/MINOR: http-act: initialize http fmt head earlier
In github issue #1850, Christian Ruppert reported a case of crash in
2.6 when failing to parse some http rules. This started to happen
with 2.6 commit dd7e6c6 ("BUG/MINOR: http-rules: completely free
incorrect TCP rules on error") but has some of its roots in 2.2
commit 2eb539687 ("MINOR: http-rules: Add release functions for
existing HTTP actions").

The cause is that when the release function is set for HTTP actions,
the rule->arg.http.fmt list head is not yet initialized, hence is
NULL, thus the release function crashes when it tries to iterate over
it. In fact this code was initially not written with the perspective
of releasing such elements upon error, so the arg list initialization
happened after error checking.

This patch just moves the list initialization just after setting the
release pointer and that's OK.

This patch must be backported to 2.6 since the problem is visible
there. It could be backported to 2.5 but the issue is not triggered
there without the first mentioned patch above that landed in 2.6, so
it will not bring any obvious benefit.
2022-09-02 19:24:12 +02:00
Willy Tarreau
e6f389d1a5 MINOR: mux-h1: provide a "show_sd" helper to output stream debugging info
With this, it now becomes possible to see the state of each H1 stream from
"show sess all". Example (added lines highlighted with '>'):

  0x7fc9b40460a0: [02/Sep/2022:16:29:31.267228] id=49 proto=tcpv4 source=127.0.0.1:53548
    flags=0xc4a, conn_retries=0, conn_exp=<NEVER> conn_et=0x000 srv_conn=0x2dc4b20, pend_pos=(nil) waiting=0 epoch=0
    frontend=decrypt (id=2 mode=http), listener=? (id=3) addr=127.0.0.1:8001
    backend=decrypt (id=2 mode=http) addr=127.0.0.1:25168
    server=httpterm (id=1) addr=127.0.0.1:8000
    task=0x7fc9b4046490 (state=0x00 nice=0 calls=4 rate=0 exp=3s tid=7(1/7) age=2s)
    txn=0x7fc9b4046650 flags=0x3000 meth=1 status=200 req.st=MSG_DONE rsp.st=MSG_DATA req.f=0x4c rsp.f=0x0d
    scf=0x7fc9b4046030 flags=0x00000080 state=EST endp=CONN,0x7fc9b4041f00,0x02804001 sub=1
>       h1s=0x7fc9b4041f00 h1s.flg=0x104010 .sd.flg=0x2804001 .req.state=MSG_DONE .res.state=MSG_DATA
>       .meth=GET status=200 .sd.flg=0x02804001 .sc.flg=0x00000080 .sc.app=0x7fc9b40460a0
>       .subs=0x7fc9b4046040(ev=1 tl=0x7fc9b4046540 tl.calls=9 tl.ctx=0x7fc9b4046030 tl.fct=sc_conn_io_cb)
>       h1c=0x7fc9b402b3f0 h1c.flg=0x302200 .sub=1 .ibuf=0@(nil)+0/0 .obuf=0@(nil)+0/0
        co0=0x7fc9bc02e740 ctrl=tcpv4 xprt=RAW mux=H1 data=STRM target=LISTENER:0x2dc3c40
        flags=0x00000300 fd=79 fd.state=421 updt=0 fd.tmask=0x80
    scb=0x7fc9b4046590 flags=0x00000011 state=EST endp=CONN,0x7fc9b4048660,0x02840001 sub=0
>       h1s=0x7fc9b4048660 h1s.flg=0x4010 .sd.flg=0x2840001 .req.state=MSG_DONE .res.state=MSG_DATA
>       .meth=GET status=200 .sd.flg=0x02840001 .sc.flg=0x00000011 .sc.app=0x7fc9b40460a0 .subs=(nil)
>       h1c=0x7fc9b4048490 h1c.flg=0x80002200 .sub=0 .ibuf=0@(nil)+0/0 .obuf=0@(nil)+0/0
        co1=0x7fc9b4048270 ctrl=tcpv4 xprt=RAW mux=H1 data=STRM target=SERVER:0x2dc4b20
        flags=0x00000300 fd=131 fd.state=10122 updt=0 fd.tmask=0x80
    req=0x7fc9b40460c0 (f=0x49c40080 an=0x8000 pipe=0 tofwd=0 total=56)
        an_exp=<NEVER> rex=<NEVER> wex=<NEVER>
        buf=0x7fc9b40460c8 data=(nil) o=0 p=0 i=0 size=0
        htx=0xdd90a0 flags=0x0 size=0 data=0 used=0 wrap=NO extra=0
    res=0x7fc9b4046120 (f=0x80070202 an=0x4000000 pipe=0 tofwd=-1 total=603840788)
        an_exp=<NEVER> rex=<NEVER> wex=<NEVER>
        buf=0x7fc9b4046128 data=(nil) o=0 p=0 i=0 size=0
        htx=0xdd90a0 flags=0x0 size=0 data=0 used=0 wrap=NO extra=0
2022-09-02 16:43:25 +02:00
Willy Tarreau
7079c0fbbd MINOR: mux-h1: split "show_fd" into connection and stream
We now have two functions, one for dumping connections and the other
one for dumping the streams. This will permit to use it from show_sd.
A few optional line breaks were inserted where relevant to keep lines
homogenous when a prefix is passed.
2022-09-02 16:43:25 +02:00
Willy Tarreau
b4a4feee87 MINOR: mux-quic: provide a "show_sd" helper to output stream debugging info
It's very limited but at least provides the very basic info about QCS and
QCC when issuing "show sess all":

  scf=0x7fa9642394a0 flags=0x00000080 state=EST endp=CONN,0x7fa9642351f0,0x02001001 sub=3
>     qcs=0x7fa9642351f0 .flg=0x5 .id=396 .st=HCR .ctx=0x7fa9642353f0, .err=0
>     qcc=0x7fa96405ce20 .flg=0 .nbsc=100 .nbhreq=100, .task=0x7fa964054260
      co0=0x7fa96405cd50 ctrl=quic4 xprt=QUIC mux=QUIC data=STRM target=LISTENER:0x328c530
      flags=0x00200300 fd=-1 fd.state=00 updt=0 fd.tmask=0x0

It will need to be improved but it's better than nothing already. This
should be backported to 2.6 if the other dumps are backported.
2022-09-02 16:43:25 +02:00
Willy Tarreau
7051f73efe MINOR: mux-h2: insert line breaks in "show sess all" output for legibility
h2s and h2c were extremely long in the "show sess all" output, around 300
chars each. This adds a few line breaks to improve legibility, there are
now 3 lines for each, which are around the same length as the other ones
while keeping a natural arrangement. E.g (lines highlighted with '>'):

  0x7faad8144f80: [02/Sep/2022:15:49:40.171620] id=105283 proto=tcpv4 source=127.0.0.1:42942
    flags=0x100c4a, conn_retries=0, conn_exp=<NEVER> conn_et=0x000 srv_conn=0x1f44b20, pend_pos=(nil) waiting=0 epoch=0
    frontend=decrypt (id=2 mode=http), listener=? (id=3) addr=127.0.0.1:8001
    backend=decrypt (id=2 mode=http) addr=127.0.0.1:18144
    server=httpterm (id=1) addr=127.0.0.1:8000
    task=0x7faad812b7c0 (state=0x00 nice=0 calls=2 rate=0 exp=4s tid=7(1/7) age=0s)
    txn=0x7faad81453e0 flags=0x43000 meth=1 status=200 req.st=MSG_DONE rsp.st=MSG_DATA req.f=0x4c rsp.f=0x0d
    scf=0x7faad81625d0 flags=0x00000080 state=EST endp=CONN,0x7faad811d380,0x02805001 sub=1
>       h2s=0x7faad811d380 h2s.id=2113 .st=HCR .flg=0x207001 .rxbuf=0@(nil)+0/0
>       .sc=0x7faad81625d0(.flg=0x00000080 .app=0x7faad8144f80) .sd=0x7faad8119dc0(.flg=0x02805001)
>       .subs=0x7faad81625e0(ev=1 tl=0x7faad86d6500 tl.calls=4 tl.ctx=0x7faad81625d0 tl.fct=sc_conn_io_cb)
>       h2c=0x7faad802c640 h2c.st0=FRH .err=0 .maxid=2157 .lastid=-1 .flg=0x0600 .nbst=1 .nbsc=1
>       .fctl_cnt=0 .send_cnt=0 .tree_cnt=1 .orph_cnt=0 .sub=1 .dsi=2157 .dbuf=0@(nil)+0/0
>       .msi=-1 .mbuf=[6..6|32],h=[0@(nil)+0/0],t=[0@(nil)+0/0]
        co0=0x7faae402efc0 ctrl=tcpv4 xprt=RAW mux=H2 data=STRM target=LISTENER:0x1f43c40
        flags=0x00000300 fd=95 fd.state=121 updt=0 fd.tmask=0x80
    scb=0x7faad8145370 flags=0x00000011 state=EST endp=CONN,0x7faad8115630,0x02840001 sub=1
        co1=0x7faad86c0730 ctrl=tcpv4 xprt=RAW mux=H1 data=STRM target=SERVER:0x1f44b20
        flags=0x00000300 fd=1656 fd.state=10121 updt=0 fd.tmask=0x80
    req=0x7faad8144fa0 (f=0x49c40000 an=0x8000 pipe=0 tofwd=0 total=110)
        an_exp=<NEVER> rex=<NEVER> wex=<NEVER>
        buf=0x7faad8144fa8 data=(nil) o=0 p=0 i=0 size=0
        htx=0xdd90a0 flags=0x0 size=0 data=0 used=0 wrap=NO extra=0
    res=0x7faad8145000 (f=0x80040202 an=0x4000000 pipe=0 tofwd=-1 total=60365)
        an_exp=<NEVER> rex=<NEVER> wex=<NEVER>
        buf=0x7faad8145008 data=(nil) o=0 p=0 i=0 size=0
        htx=0xdd90a0 flags=0x0 size=0 data=0 used=0 wrap=NO extra=0
2022-09-02 16:43:03 +02:00
Willy Tarreau
bf4ec6f4a0 MINOR: mux-h2: provide a "show_sd" helper to output stream debugging info
With this, it now becomes possible to see the state of each H2 stream from
"show sess all". Lines are still too long and need to be split, but that's
for another patch.
2022-09-02 15:48:50 +02:00
Willy Tarreau
ce57777660 MINOR: muxes: add a "show_sd" helper to complete "show sess" dumps
This helper will be called for muxes that provide it and will be used
to let the mux provide extra information about the stream attached to
a stream descriptor. A line prefix is passed in argument so that the
mux is free to break long lines without breaking indent. No prefix
means no line breaks should be produced (e.g. for short dumps).
2022-09-02 15:48:50 +02:00
Willy Tarreau
4e97bcc76b MINOR: mux-h2: extract the connection dump function out of h2_show_fd()
The function will be reusable to dump connections, so let's extract it.
2022-09-02 15:48:10 +02:00
Willy Tarreau
90bffa2ce3 MINOR: mux-h2: extract the stream dump function out of h2_show_fd()
The function will be reusable to dump streams, so let's extract it.
Note that due to "last_h2s" being originally printed as a prefix for
the stream dump, now the pointer is displayed by the caller instead.
2022-09-02 15:48:10 +02:00