Commit Graph

13337 Commits

Author SHA1 Message Date
Ilya Shipitsin
69c10f1d50 CI: Github Actions: enable prometheus exporter 2020-11-21 05:40:27 +01:00
William Lallemand
77e1c6fb0a BUG/MEDIUM: ssl/crt-list: fix error when no file found
When a file from a crt-list was not found, this one was ignored silently
letting HAProxy starts without it.

This bug was introduced by 47da821 ("MEDIUM: ssl: emulates the
multi-cert bundles in the crtlist").

This commit adds a found variable which is checked once we tried every
bundle combination so we can exits with an error if none were found.

Must be backported in 2.3.
2020-11-20 18:38:56 +01:00
William Lallemand
7340457158 BUG/MINOR: ssl/crt-list: load bundle in crt-list only if activated
Don't try to load a bundle from a crt-list if the bundle support was
disabled with ssl-load-extra-files.

Must be backported to 2.3.
2020-11-20 18:38:56 +01:00
William Lallemand
06ce84a100 BUG/MEDIUM: ssl: error when no certificate are found
When a non-existing file was specified in the configuration, haproxy
does not exits with an error which is not normal.

This bug was introduced by dfa93be ("MEDIUM: ssl: emulate multi-cert
bundles loading in standard loading") which does nothing if the stat
failed.

This patch introduce a "found" variable which is checked at the end of
the function so we exit with an error if no find were found.

Must be backported to 2.3.
2020-11-20 18:38:56 +01:00
William Lallemand
86c2dd60f1 BUG/MEDIUM: ssl/crt-list: bundle support broken in crt-list
In issue #970 it was reported that the bundle loading does not work
anymore with crt-list.

This bug was introduced by 47da821 ("MEDIUM: ssl: emulates the
multi-cert bundles in the crtlist") which incorrectly uses "path"
instead of "crt_path" in the name resolution.

Must be backported to 2.3.
2020-11-20 18:38:51 +01:00
Christopher Faulet
aab1b67383 BUG/MEDIUM: http-ana: Don't eval http-after-response ruleset on empty messages
It is not possible on response comming from a server, but an errorfile may be
empty. In this case, the http-after-response ruleset must not be evaluated
because it is totally unexpected to manipulate headers on an empty HTX message.

This patch must be backported everywhere the http-after-response rules are
supported, i.e as far as 2.2.
2020-11-20 09:43:31 +01:00
Ilya Shipitsin
bdec3ba796 BUILD: ssl: use SSL_MODE_ASYNC macro instead of OPENSSL_VERSION 2020-11-19 19:59:32 +01:00
William Lallemand
f69cd68737 BUG/MINOR: ssl: segv on startup when AKID but no keyid
In bug #959 it was reported that haproxy segfault on startup when trying
to load a certifcate which use the X509v3 AKID extension but without the
keyid field.

This field is not mandatory and could be replaced by the serial or the
DirName.

For example:

   X509v3 extensions:
       X509v3 Basic Constraints:
           CA:FALSE
       X509v3 Subject Key Identifier:
           42:7D:5F:6C:3E:0D:B7:2C:FD:6A:8A:32:C6:C6:B9:90:05:D1:B2:9B
       X509v3 Authority Key Identifier:
           DirName:/O=HAProxy Technologies/CN=HAProxy Test Intermediate CA
           serial:F2:AB:C1:41:9F:AB:45:8E:86:23:AD:C5:54:ED:DF:FA

This bug was introduced by 70df7b ("MINOR: ssl: add "issuers-chain-path" directive").

This patch must be backported as far as 2.2.
2020-11-19 16:24:13 +01:00
Willy Tarreau
02ec3fe669 DOC: coding-style: update a few rules about pointers
It's really annoying to see that in 2020 we're still facing bugs caused
by dangling pointers in the code that result from poorly written rules
about how these pointers are supposed to be handled, set and reset. Let's
add a few supposedly obvious (but apparently not) rules about how pointers
have to be used through out the code in hope to make such bad practices
disappear (or at least have something to point the authors to after
reviewing their code).
2020-11-18 19:59:38 +01:00
William Lallemand
a1ef754a6d REGTEST: server/cli_set_ssl.vtc requires OpenSSL
Add OpenSSL as a requirement for this test.
2020-11-18 17:41:28 +01:00
William Dauchy
f63704488e MEDIUM: cli/ssl: configure ssl on server at runtime
in the context of a progressive backend migration, we want to be able to
activate SSL on outgoing connections to the server at runtime without
reloading.
This patch adds a `set server ssl` command; in order to allow that:

- add `srv_use_ssl` to `show servers state` command for compatibility,
  also update associated parsing
- when using default-server ssl setting, and `no-ssl` on server line,
  init SSL ctx without activating it
- when triggering ssl API, de/activate SSL connections as requested
- clean ongoing connections as it is done for addr/port changes, without
  checking prior server state

example config:

backend be_foo
  default-server ssl
  server srv0 127.0.0.1:6011 weight 1 no-ssl

show servers state:

  5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - -1

where srv0 can switch to ssl later during the runtime:

  set server be_foo/srv0 ssl on

  5 be_foo 1 srv0 127.0.0.1 2 0 1 1 15 1 0 4 0 0 0 0 - 6011 - 1

Also update existing tests and create a new one.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2020-11-18 17:22:28 +01:00
William Dauchy
fc52f524b0 MINOR: ssl: create common ssl_ctx init
a common init for ssl_ctx will be later usable in other functions in
order to support hot enable of ssl during runtime.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2020-11-18 17:22:28 +01:00
Amaury Denoyelle
034c162b9b MEDIUM: stats: add counters for failed handshake
Report on ssl stats the total number of handshakes terminated in a
failure.
2020-11-18 16:10:42 +01:00
Amaury Denoyelle
f70b7db825 MINOR: ssl: remove client hello counters
Remove the ssl client hello received counter. This counter is not
meaningful and was only implemented on the fronted.
2020-11-18 16:10:42 +01:00
William Lallemand
0217b7b24b DOC: add missing 3.10 in the summary
3.10. Log forwarding was missing in the summary.
2020-11-18 10:43:36 +01:00
Ilya Shipitsin
ddc202a024 CI: travis-ci: arm64 are not allowed to fail anymore 2020-11-17 12:17:41 +01:00
Ilya Shipitsin
6d54113406 CI: travis-ci: remove amd64, osx builds
they are migrated to Github Actions
2020-11-17 12:17:22 +01:00
Tim Duesterhus
cdaa38ea6e CI: Make the h2spec workflow more consistent with the VTest workflow
This PR aims to make the workflow more consistent, by reusing the same wording
for the step names and the same commands to make it look like the vtest
workflow as much as possible.

It was renamed to compliance.yml to match the human readable name better. This
also allows to extend the workflow with other compliance tools later on, nicely
grouping those jobs together in a single file.

No functional changes have been made.
2020-11-17 11:55:48 +01:00
Tim Duesterhus
fa1747dc7d CI: Stop hijacking the hosts file
vtest/VTest#24 is merged now. This step is no longer required.
2020-11-17 11:55:07 +01:00
William Dauchy
a2a46ee572 REGTESTS: converter: add url_dec test
while looking at `url_dec` implementation I realised there was not yet a
simple test to avoid future regressions.
This one is testing simple case, including the "+" behaviour depending
on the argument passed to `url_dec`

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2020-11-17 11:53:25 +01:00
Willy Tarreau
1bc898398b REGTESTS: mark the abns test as broken again
There is apparently a race in this test that would require relying on
haproxy's output to make it reliably work, but currently vtest doesn't
have this option. Let's mark it broken again to avoid polluting the CI.

This is discussed in github issue #950.
2020-11-17 11:45:10 +01:00
Christopher Faulet
4f33760fe0 REGTESTS: Add a script to test the random forwarding with several filters
Three filters are used. The compression filter is enclose by two trace filters,
both with the random forwarding enabled. An HTTP test is performed but also a
TCP test using an HTTP tunnel.
2020-11-17 11:34:36 +01:00
Christopher Faulet
47d9a4e870 MINOR: flt-trace: Use a bitfield for the trace options
Instead of using a integer for each option, we now use a bitfield. Each option
is represented as a flag now.
2020-11-17 11:34:36 +01:00
Christopher Faulet
96a577acae MINOR: flt-trace: Add an option to inhibits trace messages
The 'quiet' option may be set to inibits the trace messages. The trace filter is
a bit verbose. This option may be used to not display the messages.
2020-11-17 11:34:36 +01:00
Christopher Faulet
c41d8bd65a CLEANUP: flt-trace: Remove unused random-parsing option
This option was only used by the legacy HTTP mode. In HTX, it is not used. So it
can be removed.
2020-11-17 11:34:30 +01:00
Christopher Faulet
63c69a9b4e BUG/MINOR: http-ana: Don't wait for the body of CONNECT requests
CONNECT requests are bodyless messages but with no EOM blocks. Thus, conditions
to stop waiting for the message payload are not suited to this kind of
messages. Indeed, the message finishes on an EOH block. But the tunnel mode at
the stream level is only set in HTTP_XFER_BODY analyser. So, the stream is
blocked, waiting for a body that does not exist till a timeout expires.

To fix this bug, we just stop waiting for a body for CONNECT requests. Another
solution is to rely on HTX_SL_F_BODYLESS/HTTP_MSGF_BODYLESS flags. But this one
is less intrusive.

This message must be backported as far as 2.0. For the 2.0, only the HTX part
must be fixed.
2020-11-17 10:03:12 +01:00
Christopher Faulet
22fca1f2c8 BUG/MEDIUM: filters: Forward all filtered data at the end of http filtering
When http filtering ends, if there are some filtered data not forwarded yet, we
forward them, in flt_http_end(). Most of time, this doesn't happen, except when
a tunnel is established using a CONNECT. In this case, there is not EOM on the
request and there is no body. Thus the headers are never forwarded, blocking the
stream.

This patch must be backported as far as 2.0. Prior versions don't suffer of this
bug because there is no HTX support. On the 2.0, the change is only applicable
on HTX streams. A special test must be performed to make sure.
2020-11-17 09:59:35 +01:00
Tim Duesterhus
afe36e457f REGTESTS: Add sample_fetches/cook.vtc
Add a reg-test verifying the fix in dea7c209f8.

Some parts of the configuration used in the were taken from the initial bug
report from Maciej.

Should be backported together with dea7c209f8
(all stable versions).

Co-authored-by: Maciej Zdeb <maciej@zdeb.pl>
2020-11-13 19:46:15 +01:00
Christopher Faulet
c300747dec REGTEST: make ssl_client_samples and ssl_server_samples require to 2.2
Some missing sample fetches was backported to 2.2 making these tests compatible
with the 2.2.
2020-11-13 17:12:30 +01:00
Eric Salama
9139ec34ed MINOR: cfgparse: tighten the scope of newnameserver variable, free it on error.
This should fix issue GH #931.

Also remove a misleading comment.

This commit can be backported as far as 1.9
2020-11-13 16:26:10 +01:00
Christopher Faulet
fc633b6eff CLEANUP: config: Return ERR_NONE from config callbacks instead of 0
Return ERR_NONE instead of 0 on success for all config callbacks that should
return ERR_* codes. There is no change because ERR_NONE is a macro equals to
0. But this makes the return value more explicit.
2020-11-13 16:26:10 +01:00
Christopher Faulet
5214099233 MINOR: config/mux-h2: Return ERR_ flags from init_h2() instead of a status
post-check function callbacks must return ERR_* flags. Thus, init_h2() is fixed
to return ERR_NONE on success or (ERR_ALERT|ERR_FATAL) on error.

This patch may be backported as far as 2.2.
2020-11-13 16:26:10 +01:00
Christopher Faulet
83fefbcdff MINOR: init: Fix the prototype for per-thread free callbacks
Functions registered to release memory per-thread have no return value. But the
registering function and the function pointer in per_thread_free_fct structure
specify it should return an integer. This patch fixes it.

This patch may be backported as far as 2.0.
2020-11-13 16:26:10 +01:00
Christopher Faulet
c751b4508d BUG/MINOR: tcpcheck: Don't warn on unused rules if check option is after
When tcp-check or http-check rules are used, if the corresponding check option
(option tcp-check and option httpchk) is declared after the ruleset, a warning
is emitted about an unused check ruleset while there is no problem in reality.

This patch must be backported as far as 2.2.
2020-11-13 16:26:10 +01:00
Christopher Faulet
c7ba91039a MINOR: spoe: Don't close connection in sync mode on processing timeout
In sync mode, if an applet receives a ack while the processing delay has already
expired, there is not frame waiting for this ack. But there is no reason to
close the connection in this case. The ack may be ignored and the connection may
be reused to process another frame. The only reason to trigger an error and
close the connection is when the wrong ack is received while there is still a
frame waiting for its ack. In sync mode, this should never happen.

This patch may be backported in all versions supporting the SPOE.
2020-11-13 16:26:10 +01:00
Christopher Faulet
cf181c76e3 BUG/MAJOR: spoe: Be sure to remove all references on a released spoe applet
When a SPOE applet is used to send a frame, a reference on this applet is saved
in the spoe context of the offladed stream. But, if the applet is released
before receving the corresponding ack, we must be sure to remove this
reference. This was performed for fragmented frames only. But it must also be
performed for a spoe contexts in the applet waiting_queue and in the thread
waiting_queue (used in async mode).

This bug leads to a memory corruption when an offloaded stream try to update the
state of a released applet because it still have a reference on it. There are
many ways to trigger this bug. The easiest is probably during reloads. On the
old process, all applets are woken up to be released ASAP.

Many thanks to Maciej Zdeb to report the bug and to work on it for 2
months. Without his help, it would have been much more difficult to fix the
bug. It is always a huge pleasure to see how some users are enthousiast and
helpful. Thanks again Maciej !

This patch must be backported to all versions where the spoe is supported (>=
1.7).
2020-11-13 16:26:10 +01:00
Christopher Faulet
3005d28eb8 BUG/MINOR: http-htx: Handle warnings when parsing http-error and http-errors
First of all, this patch is tagged as a bug. But in fact, it only fixes a bug in
the 2.2. On the 2.3 and above, it only add the ability to display warnings, when
an http-error directive is parsed from a proxy section and when an errorfile
directive is parsed from a http-errors section.

But on the 2.2, it make sure to display the warning emitted on a content-length
mismatch when an errorfile is parsed. The following is only applicable to the
2.2.

commit "BUG/MINOR: http-htx: Just warn if payload of an errorfile doesn't match
the C-L" (which is only present in 2.2, 2.1 and 2.0 trees, i.e see commit
7bf3d81d3cf4b9f4587 in 2.2 tree), is changing the behavior of `http_str_to_htx`
function. It may now emit warnings. And, it is the caller responsibility to
display it.

But the warning is missing when an 'http-error' directive is parsed from
a proxy section. It is also missing when an 'errorfile' directive is
parsed from a http-errors section.

This bug only exists on the 2.2. On earlier versions, these directives
are not supported and on later ones, an error is triggered instead of a
warning.

Thanks to William Dauchy that spotted the bug.

This patch must be backported as far as 2.2.
2020-11-13 16:26:10 +01:00
Amaury Denoyelle
90eb93f792 MINOR: check: report error on incompatible connect proto
Report an error when using an explicit proto for a connect rule with
non-compatible mode in regards with the selected check type (tcp-check
vs http-check).
2020-11-13 16:26:10 +01:00
Amaury Denoyelle
7c14890183 MINOR: check: report error on incompatible proto
If the check mux has been explicitly defined but is incompatible with
the selected check type (tcp-check vs http-check), report a warning and
prevent haproxy startup.
2020-11-13 16:26:10 +01:00
Amaury Denoyelle
0519bd4d04 BUG/MEDIUM: check: reuse srv proto only if using same mode
Only reuse the mux from server if the check is using the same mode.
For example, this prevents a tcp-check on a h2 server to select the h2
multiplexer instead of passthrough.

This bug was introduced by the following commit :
BUG/MEDIUM: checks: Use the mux protocol specified on the server line
It must be backported up to 2.2.

Fixes github issue #945.
2020-11-13 16:26:10 +01:00
Christopher Faulet
97fc8da264 BUG/MINOR: http-fetch: Fix calls w/o parentheses of the cookie sample fetches
req.cook, req.cook_val, req.cook_cnt and and their response counterparts may be
called without cookie name. In this case, empty parentheses may be used, or no
parentheses at all. In both, the result must be the same. But only the first one
works. The second one always returns a failure. This patch fixes this bug.

Note that on old versions (< 2.2), both cases fail.

This patch must be backported in all stable versions.
2020-11-13 16:26:10 +01:00
Maciej Zdeb
dea7c209f8 BUG/MINOR: http-fetch: Extract cookie value even when no cookie name
HTTP sample fetches dealing with the cookies (req/res.cook,
req/res.cook_val and req/res.cook_cnt) must be prepared to be called
without cookie name. For the first two, the first cookie value is
returned, regardless its name. For the last one, all cookies are counted.

To do so, http_extract_cookie_value() may now be called with no cookie
name (cookie_name_l set to 0). In this case, the matching on the cookie
name is ignored and the first value found is returned.

Note this patch also fixes matching on cookie values in ACLs.

This should be backported in all stable versions.
2020-11-13 16:26:10 +01:00
Willy Tarreau
1dfd4f106f BUG/MEDIUM: peers: fix decoding of multi-byte length in stick-table messages
There is a bug in peer_recv_msg() due to an incorrect cast when trying
to decode the varint length of a stick-table message, causing lengths
comprised between 128 and 255 to consume one extra byte, ending in
protocol errors.  The root cause of this is that peer_recv_msg() tries
hard to reimplement all the parsing and control that is already done in
intdecode() just to measure the length before calling it. And it got it
wrong.

Let's just get rid of this unneeded code duplication and solely rely on
intdecode() instead. The bug was introduced in 2.0 as part of a cleanup
pass on this code with commit 95203f218 ("MINOR: peers: Move high level
receive code to reduce the size of I/O handler."), so this patch must
be backported to 2.0.

Thanks to Yves Lafon for reporting the problem.
2020-11-13 15:21:50 +01:00
Frédéric Lécaille
ea875e62e6 BUG/MINOR: peers: Missing TX cache entries reset.
The TX part of a cache for a dictionary is made of an reserved array of ebtree nodes
which are pointers to dictionary entries. So when we flush the TX part of such a
cache, we must not only remove these nodes to dictionary entries from their ebtree.
We must also reset their values. Furthermore, the LRU key and the last lookup
result must also be reset.
2020-11-13 06:04:18 +01:00
Frédéric Lécaille
f9e51beec1 BUG/MINOR: peers: Do not ignore a protocol error for dictionary entries.
If we could not decode the ID of a dictionary entry from a peer update message,
we must inform the remote peer about such an error as this is done for
any other decoding error.
2020-11-13 06:04:08 +01:00
Frédéric Lécaille
d865935f32 MINOR: peers: Add traces to peer_treat_updatemsg().
Add minimalistic traces for peers with only one event to diagnose potential
issues when decode peer update messages.
2020-11-12 17:38:49 +01:00
Amaury Denoyelle
7f8f6cb926 BUG/MEDIUM: stats: prevent crash if counters not alloc with dummy one
Define a per-thread counters allocated with the greatest size of any
stat module counters. This variable is named trash_counters.

When using a proxy without allocated counters, return the trash counters
from EXTRA_COUNTERS_GET instead of a dangling pointer to prevent
segfault.

This is useful for all the proxies used internally and not
belonging to the global proxy list. As these objects does not appears on
the stat report, it does not matter to use the dummy counters.

For this fix to be functional, the extra counters are explicitly
initialized to NULL on proxy/server/listener init functions.

Most notably, the crash has already been detected with the following
vtc:
- reg-tests/lua/txn_get_priv.vtc
- reg-tests/peers/tls_basic_sync.vtc
- reg-tests/peers/tls_basic_sync_wo_stkt_backend.vtc
There is probably other parts that may be impacted (SPOE for example).

This bug was introduced in the current release and do not need to be
backported. The faulty commits are
"MINOR: ssl: count client hello for stats" and
"MINOR: ssl: add counters for ssl sessions".
2020-11-12 15:16:05 +01:00
Amaury Denoyelle
a2a6899bee BUG/MINOR: stats: free dynamically stats fields/lines on shutdown
Register a new function on POST DEINIT to free stats fields/lines for
each domain.

This patch does not fix a critical bug but may be backported to 2.3.
2020-11-12 15:16:05 +01:00
Remi Tricot-Le Breton
cc9bf2e5fe MEDIUM: cache: Change caching conditions
Do not cache responses that do not have an explicit expiration time
(s-maxage or max-age Cache-Control directives or Expires header) or a
validator (ETag or Last-Modified headers) anymore, as suggested in
RFC 7234#3.
The TX_FLAG_IGNORE flag is used instead of the TX_FLAG_CACHEABLE so as
not to change the behavior of the checkcache option.
2020-11-12 11:22:05 +01:00
Thierry Fournier
91dc0c0d8f BUG/MINOR: lua: set buffer size during map lookups
This size is used by some pattern matching to determine if there
is sufficient room in the buffer to add final \0 if necessary.
If the size is not set, the conditions use uninitialized value.

Note: it seems this bug can't cause a crash.

Should be backported until 2.2 (at least)
2020-11-11 10:43:21 +01:00