Commit Graph

15426 Commits

Author SHA1 Message Date
Amaury Denoyelle
0f456d5029 BUG/MINOR: server: allow 'enable health' only if check configured
Test that checks have been configured on the server before enabling via
the 'enable health' CLI. This mirrors the 'enable agent' command.

Without this, a user can use the command on the server without checks.
This leaves the server in an undefined state. Notably, the stat page
reports the server in check transition.

This condition was left on the following reorg commit.
  2c04eda8b5
  REORG: cli: move "{enable|disable} health" to server.c

This should be backported up to 1.8.
2021-09-21 11:50:22 +02:00
Tim Duesterhus
4f065262e9 CLEANUP: Remove unreachable break from parse_time_err()
The `return` already leaves the function.
2021-09-20 18:37:32 +02:00
Tim Duesterhus
75e2f8dcdd CLEANUP: Include check.h in flt_spoe.c
This is required for the prototype of spoe_prepare_healthcheck_request().
2021-09-20 18:37:32 +02:00
William Lallemand
79a3478c24 MINOR: httpclient: add the EOH when no headers where provided
httpclient_req_gen() now adds the end of headers block when no header
was provided, which avoid adding it manually.
2021-09-20 16:24:54 +02:00
Dragan Dosen
a8018eb470 BUG/MINOR: flt-trace: fix an infinite loop when random-parsing is set
The issue is introduced with the commit c41d8bd65 ("CLEANUP: flt-trace:
Remove unused random-parsing option").

This must be backported everywhere the above commit is.
2021-09-20 16:06:58 +02:00
Tim Duesterhus
ecf55968a1 DEV: coccinelle: Add xalloc_cast.cocci
This remove's C++ style casts from the return value of malloc/calloc.

see 403fd722ac
2021-09-17 17:22:05 +02:00
Tim Duesterhus
ec4a8754da CLEANUP: Apply xalloc_size.cocci
This fixes a few locations with a hardcoded type within `sizeof()`.
2021-09-17 17:22:05 +02:00
Tim Duesterhus
02fa646a37 DEV: coccinelle: Add bug_on.cocci
This replaces an if + ABORT_NOW() by BUG_ON(). It might change behavior,
because BUG_ON will result in a no-op if not enabled.
2021-09-17 17:22:05 +02:00
Tim Duesterhus
63ee0e4c01 DEV: coccinelle: Add xalloc_size.cocci
This commits the Coccinelle patch to clean up sizeof handling for malloc/calloc.
2021-09-17 17:22:05 +02:00
Tim Duesterhus
16554245e2 CLEANUP: Apply bug_on.cocci
The changes look safe to me, even if `DEBUG_STRICT` is not enabled.
2021-09-17 17:22:05 +02:00
Tim Duesterhus
c1af0bae69 DEV: coccinelle: Add ist.cocci
This commits the Coccinelle patch to clean up ist handling.
2021-09-17 17:22:05 +02:00
Tim Duesterhus
b113b5ca24 CLEANUP: Apply ist.cocci
This cleans up ist handling.
2021-09-17 17:22:05 +02:00
Willy Tarreau
81a76f4827 REORG: threads: move ha_get_pthread_id() to tinfo.h
This solely manipulates the thread_info struct, it ought to be in
tinfo.h, not in thread.h.
2021-09-17 16:08:34 +02:00
Willy Tarreau
e61244631a MINOR: applet: remove the thread mask from appctx_new()
appctx_new() is exclusively called with tid_bit and it only uses the
mask to pass it to the accompanying task. There is no point requiring
the caller to know about a mask there, nor is there any point in
creating an applet outside of the context of its own thread anyway.
Let's drop this and pass tid_bit to task_new() directly.
2021-09-17 16:08:34 +02:00
Willy Tarreau
87063a7da1 BUILD: fd: remove unused variable totlen in fd_write_frag_line()
Ilya reports in GH #1392 that clang 13 complains about totlen being
calculated and not used in fd_write_frag_line(), which is true. It's
a leftover of some older code.
2021-09-17 12:00:27 +02:00
Willy Tarreau
b5d1141305 BUILD: proto_uxst: do not set unused flag
Similarly to previous patch for sockpair, UNIX sockets set the
CONNECT_HAS_DATA flag without using it later, we can drop it.
2021-09-17 11:59:15 +02:00
Willy Tarreau
0ce77ac204 BUILD: sockpair: do not set unused flag
Ilya reports in GH #1392 that clang 13 complains about a flag being added
to the "flags" parameter without being used later. That's generic code
that was shared from TCP but we can indeed drop this flag since it's used
for TFO which we don't have in socketpairs.
2021-09-17 11:56:25 +02:00
Willy Tarreau
f2dda52e78 BUG/MINOR: cli/payload: do not search for args inside payload
The CLI's payload parser is over-complicated and as such contains more
bugs than needed. One of them is that it uses strstr() to find the
ending tag, ignoring spaces before it, while the argument locator
creates a new arg on each space, without checking if the end of the
word appears past the previously found end. This results in "<<" being
considered as the start of a new argument if preceeded by more than
one space, and the payload being damaged with a \0 inserted at the
first space or tab.

Let's make an easily backportable fix for now. This fix makes sure that
the trailing zero from the first line is properly kept after '<<' and
that the end tag is looked for only as an isolated argument and nothing
else. This also gets rid of the unsuitable strstr() call and now makes
sure that strcspn() will not return elements that are found in the
payload.

For the long term the loop must be rewritten to get rid of those
unsuitable strcspn() and strstr() calls which work past each other, and
the cli_parse_request() function should be split into a tokenizer and
an executor that are used from the caller instead of letting the caller
play games with what it finds there.

This should be backported wherever CLI payload is supported, i.e. 2.0+.
2021-09-17 11:50:09 +02:00
Amaury Denoyelle
7a8aff2688 BUILD: ist: prevent gcc11 maybe-uninitialized warning on istalloc
A new warning is reported by gcc11 when using a pointer to uninitialized
memory block for a function with a const pointer argument. The warning
is triggered for istalloc, used by http_client.c / proxy.c / tcpcheck.c.

This warning is reported because the uninitialized memory block
allocated by malloc should not be passed to a const argument as in ist2.
See https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Warning-Options.html#index-Wmaybe-uninitialized

This should be backported up to 2.2.
2021-09-17 09:57:27 +02:00
Amaury Denoyelle
4837293ca0 BUG/MINOR: connection: prevent null deref on mux cleanup task allocation
Move the code to allocate/free the mux cleanup task outside of the polling
loop. A new thread_alloc/free handler is registered for this in
connection.c.

This has the benefit to clean up the polling loop code. And as another
benefit, if the task allocation fails, the handler can report an error
to exit the haproxy process. This prevents a potential null pointer
dereferencing.

This should fix the github issue #1389.

This must be backported up to 2.4.
2021-09-16 17:45:52 +02:00
William Lallemand
ed8bfadd8d DOC: management: certificate files must be sanitized before injection
A lot of people encounter problems when trying to inject a certificate
file which contains extra informations or empty lines.

This patch adds a paragraph and a sanitizing example.

Must be backported as far as 2.1.
2021-09-16 17:36:18 +02:00
Christopher Faulet
8a0e5f822b BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
When the LDAP response is parsed, the message length is not properly
decoded. While it works for LDAP servers encoding it on 1 byte, it does not
work for those using a multi-bytes encoding. Among others, Active Directory
servers seems to encode messages or elements length on 4 bytes.

In this patch, we only handle length of BindResponse messages encoded on 1,
2 or 4 bytes. In theory, it may be encoded on any bytes number less than 127
bytes. But it is useless to make this part too complex. It should be ok this
way.

This patch should fix the issue #1390. It should be backported to all stable
versions. While it should be easy to backport it as far as 2.2, the patch
will have to be totally rewritten for lower versions.
2021-09-16 17:24:50 +02:00
Willy Tarreau
c2afb860f2 MINOR: pools: use mallinfo2() when available instead of mallinfo()
Ilya reported in issue #1391 a build warning on Fedora about mallinfo()
being deprecated in favor of mallinfo2() since glibc-2.33. Let's add
support for it. This should be backported where the following commit is
also backported: 157e39303 ("MINOR: pools: automatically disable
malloc_trim() with external allocators").
2021-09-16 09:20:16 +02:00
Willy Tarreau
a096155e08 DOC: update Tim's address in .mailmap
Let's keep the ASCII-only version that's already used in most commits
and presents well everywhere.
2021-09-16 09:14:14 +02:00
Christopher Faulet
ab7389dc3c BUG/MAJOR: mux-h1: Don't eval input data if an error was reported
If an error was already reported on the H1 connection, pending input data
must not be (re)evaluated in h1_process(). Otherwise an unexpected internal
error will be reported, in addition of the first one. And on some
conditions, this may generate an infinite loop because the mux tries to send
an internal error but it fails to do so thus it loops to retry.

This patch should fix the issue #1356. It must be backported to 2.4.
2021-09-16 08:31:46 +02:00
Christopher Faulet
51324b8720 CLEANUP: acl: Remove unused variable when releasing an acl expression
The "unresolved" variable is unused since commit 9fa0df5 ("BUG/MINOR: acl:
Fix freeing of expr->smp in prune_acl_expr").

This patch should fix the issue #1359.
2021-09-16 08:31:46 +02:00
Tim Duesterhus
8f1669b10f CLEANUP: Remove prototype for non-existent thread_get_default_count()
This is the only location of `thread_get_default_count` within the codebase.
2021-09-15 11:07:18 +02:00
Tim Duesterhus
992007ec78 CLEANUP: tree-wide: fix prototypes for functions taking no arguments.
"f(void)" is the correct and preferred form for a function taking no
argument, while some places use the older "f()". These were reported
by clang's -Wmissing-prototypes, for example:

  src/cpuset.c:111:5: warning: no previous prototype for function 'ha_cpuset_size' [-Wmissing-prototypes]
  int ha_cpuset_size()
  include/haproxy/cpuset.h:42:5: note: this declaration is not a prototype; add 'void' to make it a prototype for a zero-parameter function
  int ha_cpuset_size();
      ^
                     void

This aggregate patch fixes this for the following functions:

   ha_backtrace_to_stderr(), ha_cpuset_size(), ha_panic(), ha_random64(),
   ha_thread_dump_all_to_trash(), get_exec_path(), check_config_validity(),
   mworker_child_nb(), mworker_cli_proxy_(create|stop)(),
   mworker_cleantasks(), mworker_cleanlisteners(), mworker_ext_launch_all(),
   mworker_reload(), mworker_(env|proc_list)_to_(proc_list|env)(),
   mworker_(un|)block_signals(), proxy_adjust_all_maxconn(),
   proxy_destroy_all_defaults(), get_tainted(),
   pool_total_(allocated|used)(), thread_isolate(_full|)(),
   thread(_sync|)_release(), thread_harmless_till_end(),
   thread_cpu_mask_forced(), dequeue_all_listeners(), next_timer_expiry(),
   wake_expired_tasks(), process_runnable_tasks(), init_acl(),
   init_buffer(), (de|)init_log_buffers(), (de|)init_pollers(),
   fork_poller(), pool_destroy_all(), pool_evict_from_local_caches(),
   pool_total_failures(), dump_pools_to_trash(), cfg_run_diagnostics(),
   tv_init_(process|thread)_date(), __signal_process_queue(),
   deinit_signals(), haproxy_unblock_signals()
2021-09-15 11:07:18 +02:00
Tim Duesterhus
062a54dd47 DOC: Add .mailmap
Ensure that Tim's last name is consistent no matter how the patch is generated
and applied, preventing Tim from showing up as two different persons in the
shortlog in releases.
2021-09-15 11:07:18 +02:00
Willy Tarreau
845b560f6a MINOR: pools: report it when malloc_trim() is enabled
Since we can detect it at runtime now, it could help to have it mentioned
in haproxy -vv.
2021-09-15 10:41:24 +02:00
Willy Tarreau
157e393039 MINOR: pools: automatically disable malloc_trim() with external allocators
Pierre Cheynier reported some occasional crashes in malloc_trim() on a
recent glibc when running with jemalloc(). While in theory there should
not be any link between the two, it remains plausible that something
allocated early with one is tentatively freed with the other and that
attempts to trim end up badly. There's no point calling the glibc specific
malloc_trim() with external allocators anyway. However these ones are often
enabled at link time or even at run time with LD_PRELOAD, so we cannot rely
on build options for this.

This patch implements runtime detection for the allocator in use by checking
with mallinfo() that a malloc() call is properly accounted for in glibc's
malloc. It only enables malloc_trim() in this case, and ignores it for
other cases. It's fine to proceed like this because mallinfo() is provided
by a wider range of glibcs than malloc_trim().

This could be backported to 2.4 and 2.3. If so, it will also need previous
patch "CLEANUP: pools: factor all malloc_trim() calls into trim_all_pools()".
2021-09-15 10:40:39 +02:00
Willy Tarreau
ea3323f62c CLEANUP: pools: factor all malloc_trim() calls into trim_all_pools()
The code was slightly cleaned up by removing repeated occurrences of ifdefs
and moving that into a single trim_all_pools() function.
2021-09-15 10:38:21 +02:00
Willy Tarreau
c5d0fc9b9f BUILD: sample: fix format warning on 32-bit archs in sample_conv_be2dec_check()
The sizeof() was printed as a long but it's just an unsigned on some
32-bit platforms, hence the format warning. No backport is needed, as
this arrived in 2.5 with commit  40ca09c7b ("MINOR: sample: Add be2dec
converter").
2021-09-15 10:32:12 +02:00
Willy Tarreau
4f5485bfad BUG/MINOR: compat: make sure __WORDSIZE is always defined
-Wundef triggered on a MIPS-based musl build on __WORDSIZE that's used
in ultoa_o() and some Lua initialization. The former will fail to convert
integers larger to 1 billion to proper string in this case. Let's make
sure this macro is defined and fall back to values determined from
__SIZEOF_LONG__ otherwise. A cleaner long-term approach would consist
in removing all remaining occurrences of this macro.

This can be backported to all versions.
2021-09-15 10:32:12 +02:00
Willy Tarreau
8ab9419394 BUILD: threads: fix -Wundef for _POSIX_PRIORITY_SCHEDULING on libmusl
Building with an old musl-based toolchain reported this warning:

  include/haproxy/thread.h: In function 'ha_thread_relax':
  include/haproxy/thread.h:256:5: warning: "_POSIX_PRIORITY_SCHEDULING" is not defined [-Wundef]
   #if _POSIX_PRIORITY_SCHEDULING
       ^

There were indeed two "#if" insteadd of #ifdef" for this macro, let's
fix them.
2021-09-15 10:32:12 +02:00
Willy Tarreau
80d3daad50 BUILD: halog: fix a -Wundef warning on non-glibc systems
Dmitry reported this warning on FreeBSD since the introduction of -Wundef:

  admin/halog/fgets2.c:38:30: warning: '__GLIBC__' is not defined, evaluates to 0 [-Wundef]
  #if defined(__x86_64__) &&  (__GLIBC__ > 2 || (__GLIBC__ == 2 && __GLIBC_MINOR__ >= 15))
                               ^
A defined() was missing.
2021-09-13 09:32:01 +02:00
Willy Tarreau
8ac6597cbe BUILD: compiler: fixed a missing test on defined(__GNUC__)
This one could theoretically trigger -Wundef on non-gcc compatible
compilers if DEBUG_USE_ABORT is not set.
2021-09-13 09:30:47 +02:00
Willy Tarreau
4b3a9fefab [RELEASE] Released version 2.5-dev7
Released version 2.5-dev7 with the following main changes :
    - BUG/MINOR: config: reject configs using HTTP with bufsize >= 256 MB
    - CLEANUP: htx: remove comments about "must be < 256 MB"
    - BUG/MAJOR: htx: fix missing header name length check in htx_add_header/trailer
    - Revert "BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive"
    - MINOR: proxy: add a global "grace" directive to postpone soft-stop
    - MINOR: vars: rename vars_init() to vars_init_head()
    - CLEANUP: vars: rename sample_clear_stream() to var_unset()
    - REORG: vars: remerge sample_store{,_stream}() into var_set()
    - MEDIUM: vars: make the ifexist variant of set-var only apply to the proc scope
    - MINOR: vars: add a VF_CREATEONLY flag for creation
    - MINOR: vars: support storing empty sample data with a variable
    - MINOR: vars: store flags into variables and add VF_PERMANENT
    - MEDIUM: vars: make var_clear() only reset VF_PERMANENT variables
    - MEDIUM: vars: pre-create parsed SCOPE_PROC variables as permanent ones
    - MINOR: vars: preset a random seed to hash variables names
    - MEDIUM: vars: replace the global name index with a hash
    - CLEANUP: vars: remove the now unused var_names array
    - MINOR: vars: centralize the lock/unlock into static inlines
    - OPTIM: vars: only takes the variables lock on shared entries
    - OPTIM: vars: remove internal bookkeeping for vars_global_size
    - OPTIM: vars: do not keep variables usage stats if no limit is set
    - BUILD: fix dragonfly build again on __read_mostly
    - CI: Github Actions: temporarily disable Opentracing
    - BUG/MEDIUM: mux-h1: Remove "Upgrade:" header for requests with payload
    - MINOR: htx: Skip headers with no value when adding a header list to a message
    - CLEANUP: mux-h1: Remove condition rejecting upgrade requests with payload
    - BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is reached
    - BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
    - BUG/MINOR: filters: Set right FLT_END analyser depending on channel
    - CLEANUP: Add haproxy/xxhash.h to avoid modifying import/xxhash.h
    - CLEANUP: ebmbtree: Replace always-taken elseif by else
    - CLEANUP: Move XXH3 macro from haproxy/compat.h to haproxy/xxhash.h
    - BUILD: opentracing: exclude the use of haproxy variables for the OpenTracing context
    - BUG/MINOR: opentracing: enable the use of http headers without a set value
    - CLEANUP: opentracing: use the haproxy function to generate uuid
    - MINOR: opentracing: change the scope of the variable 'ot.uuid' from 'sess' to 'txn'
    - CI: Github Actions: re-enable Opentracing
    - CLEANUP: opentracing: simplify the condition on the empty header
    - BUG/MEDIUM lua: Add missing call to RESET_SAFE_LJMP in hlua_filter_new()
2021-09-12 11:36:38 +02:00
Tim Duesterhus
2281738256 BUG/MEDIUM lua: Add missing call to RESET_SAFE_LJMP in hlua_filter_new()
In one case before exiting leaving the function the panic handler was not
reset.

Introduced in 69c581a092, which is 2.5+.
No backport required.
2021-09-12 08:21:07 +02:00
Miroslav Zagorac
e956c9cdac CLEANUP: opentracing: simplify the condition on the empty header
The second patch from the last series of patches has been redesigned
here, the ist() function is used to set an empty string instead of
working directly with the string pointer.

I thank Tim Düsterhus for his advice.
2021-09-12 08:17:02 +02:00
Willy Tarreau
ed17ca7006 CI: Github Actions: re-enable Opentracing
Miroslav already fixed the build of OpenTracing so we can re-enable it
in the CI. For now variables are disabled but this will change soon.
2021-09-12 07:08:14 +02:00
Miroslav Zagorac
560c7b874a MINOR: opentracing: change the scope of the variable 'ot.uuid' from 'sess' to 'txn'
At the suggestion of Willy Tarreau, the scope of the 'ot.uuid' variable was
changed from 'sess' to 'txn', so it is now limited to the transaction only.
2021-09-12 07:08:14 +02:00
Miroslav Zagorac
a75f3404c8 CLEANUP: opentracing: use the haproxy function to generate uuid
To avoid duplicate source code, the original haproxy function is used to
generate the OpenTracing runtime context UUID.

Also, the structure flt_ot_runtime_context is simplified because the
detailed definition of UUID is removed from it (struct flt_ot_uuid),
ie the UUID is left only in the form of a string.
2021-09-12 07:08:14 +02:00
Miroslav Zagorac
71cb1ab6b9 BUG/MINOR: opentracing: enable the use of http headers without a set value
In case we transfer some data that does not have a set value via the HTTP
header, adding that header in the text map was done incorrectly.

This simple patch allows the use of HTTP headers without a set value.
2021-09-12 07:08:14 +02:00
Miroslav Zagorac
4cb2c83f46 BUILD: opentracing: exclude the use of haproxy variables for the OpenTracing context
Due to a recent change in the handling of haproxy variables, their use for
OpenTracing context transfer has been excluded from the compilation process.

The use of variables can be re-enabled if the newly defined variable
OT_USE_VARS is set to 1 when calling the 'make' utility.  However,
this should not be used for now as the compilation will end in error.

This change prevents the use of haproxy variables to convey the OpenTracing
context.  This means that the 'use-vars' parameter cannot be used in the
OpenTracing filter configuration for 'inject' and 'extract' operations.

An example configuration that uses this feature is in the test/ctx
directory, while the script to run that test is test/run-ctx.sh.

Then, the 'sess.ot.uuid' variable is no longer set when initializing the
OpenTracing session.  This means that this variable can still be used in
the OpenTracing configuration, but its contents will be empty.
2021-09-12 07:07:13 +02:00
Tim Duesterhus
cf6f574872 CLEANUP: Move XXH3 macro from haproxy/compat.h to haproxy/xxhash.h
This moves all the xxhash functionality into a single location.

see d5fc8fcb86
2021-09-11 20:37:50 +02:00
Tim Düsterhus
a8bfb4d135 CLEANUP: ebmbtree: Replace always-taken elseif by else
`diff` is guaranteed to be less than 0, because the `if` handles the `>= 0`
case.

Found using GitHub's CodeQL scan in HAProxy's codebase.
2021-09-11 20:15:28 +02:00
Tim Duesterhus
d5fc8fcb86 CLEANUP: Add haproxy/xxhash.h to avoid modifying import/xxhash.h
This solves setting XXH_INLINE_ALL in a cleaner way, because the imported
header is not modified, easing future updates.

see 6f7cc11e6d
2021-09-11 19:58:45 +02:00
Christopher Faulet
949b6ca961 BUG/MINOR: filters: Set right FLT_END analyser depending on channel
A bug was introduced by the commit 26eb5ea35 ("BUG/MINOR: filters: Always
set FLT_END analyser when CF_FLT_ANALYZE flag is set"). Depending on the
channel evaluated, the rigth FLT_END analyser must be set. AN_REQ_FLT_END
for the request channel and AN_RES_FLT_END for the response one.

Ths patch must be backported everywhere the above commit was backported.
2021-09-10 10:35:53 +02:00
Christopher Faulet
2d56500826 BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
When an error is returned to the client, via a call to
http_reply_and_close(), the request channel is flushed and shut down and
HTTP analysis on both direction is finished. So it is safer to centralize
reset of channels analysers at this place. It is especially important when a
filter is attached to the stream when a client abort is detected. Because,
otherwise, the stream remains blocked because request analysers are not
reset.

This bug was hidden for a while. But since the fix 6fcd2d328 ("BUG/MINOR:
stream: Don't release a stream if FLT_END is still registered"), it is
possible to trigger it.

This patch must be backported everywhere the above commit was backported.
2021-09-10 10:35:53 +02:00