Commit Graph

137 Commits

Author SHA1 Message Date
William Lallemand
9496e7e888 REGTESTS: ssl: update the filters test for TLSv1.3 and sigalgs
Signature algorithms allows us to select the right certificates when
using TLSv1.3. This patch update the ssl_crt-list_filters.vtc regtest to
do more precise testing with TLSv1.3 in addition to TLSv1.2.

This allow us to test correctly bug #2300.

It could be backported to 2.8 with the previous fix for certificate
selection.
2023-10-26 19:23:04 +02:00
William Lallemand
fde517ba66 REGTESTS: wolfssl: temporarly disable some failing reg-tests
Temporarly disable the last failing reg-tests with WolfSSL in order to
be able to setup a CI.
2023-10-09 23:05:18 +02:00
William Lallemand
c24cc33c0f REGTESTS: ssl: disable ssl_dh.vtc for WolfSSL
Skip the ssl_dh reg-tests which is not working for WolfSSL.
2023-10-09 22:11:37 +02:00
William Lallemand
1105524cbe REGTESTS: ssl: update common.pem with the new pki
Update the SSL reg-test in order to use the new pki.
2023-10-09 21:58:21 +02:00
William Lallemand
61b6a4da6c REGTESTS: ssl: skip generate-certificates test w/ wolfSSL
WolfSSL does not seem to work correctly with the generate-certificates
features. This patch disables it temporarly.

    ssl-max-ver TLSv1.2 seems to be a problem in the reg-test and
    wolfSSL but without it it's not able to generate correctly the cert:

    ***  h1    debug|00000004:clear-lst.accept(0007)=0028 from [127.0.0.1:35956] ALPN=<none>
    ***  h1    debug|00000004:clear-lst.clireq[0028:ffffffff]: GET / HTTP/1.1
    ***  h1    debug|00000004:clear-lst.clihdr[0028:ffffffff]: x-sni: unknown-sni.com
    ***  h1    debug|00000004:clear-lst.clihdr[0028:ffffffff]: host: 127.0.0.1
    ***  h1    debug|fd[0x29] OpenSSL error[0x13d] : need the private key
    ***  h1    debug|<134>Sep 20 15:42:58 haproxy[165743]: unix:1 [20/Sep/2023:15:42:58.042] ssl-lst/1: SSL handshake failure (need the private key)
    **** dT    1.072
    ***  h1    debug|fd[0x2a] OpenSSL error[0x13d] : need the private key
    ***  h1    debug|<134>Sep 20 15:42:59 haproxy[165743]: unix:1 [20/Sep/2023:15:42:59.044] ssl-lst/1: SSL handshake failure (need the private key)
    **** dT    2.075
    ***  h1    debug|fd[0x29] OpenSSL error[0x13d] : need the private key
    ***  h1    debug|<134>Sep 20 15:43:00 haproxy[165743]: unix:1 [20/Sep/2023:15:43:00.046] ssl-lst/1: SSL handshake failure (need the private key)
    **** dT    3.079
    ***  h1    debug|fd[0x29] OpenSSL error[0x13d] : need the private key
    ***  h1    debug|<134>Sep 20 15:43:01 haproxy[165743]: unix:1 [20/Sep/2023:15:43:01.050] ssl-lst/1: SSL handshake failure (need the private key)
    **** dT    3.080
    ***  h1    debug|00000004:default_backend.clicls[0028:0023]
    ***  h1    debug|00000004:default_backend.closed[0028:0023]
    ***  h1    debug|<134>Sep 20 15:43:01 haproxy[165743]: 127.0.0.1:35956 [20/Sep/2023:15:42:58.042] clear-lst default_backend/s1 0/0/-1/-1/+3009 503 +217 - - SC-- 3/1/0/0/3 0/0 "GET / HTTP/1.1" 0/-/-/-/0 -/-/-
    **** c3    rxhdr|HTTP/1.1 503 Service Unavailable\r
    **** c3    rxhdr|content-length: 107\r
    **** c3    rxhdr|cache-control: no-cache\r
    **** c3    rxhdr|content-type: text/html\r
    **** c3    rxhdr|\r
2023-09-20 16:02:16 +02:00
William Lallemand
64a4b44a44 REGTESTS: ssl: skip OCSP test w/ WolfSSL
The OCSP test does not seem to be working correctly with wolfSSL.

i2d_OCSP_CERTID(data->ocsp_cid, NULL); always returns 0.

Skip it for now.
2023-09-20 15:23:32 +02:00
William Lallemand
15e591b6e0 MINOR: ssl: add support for 'curves' keyword on server lines
This patch implements the 'curves' keyword on server lines as well as
the 'ssl-default-server-curves' keyword in the global section.

It also add the keyword on the server line in the ssl_curves reg-test.

These keywords allow the configuration of the curves list for a server.
2023-09-07 23:29:10 +02:00
Andrew Hopkins
88988bb06c REGTESTS: ssl: skip ssl_dh test with AWS-LC
skip ssl_dh test when HAProxy is built with AWS-LC which does not support FFDH ciphersuites.
2023-09-04 18:21:01 +02:00
William Lallemand
e7d9082315 BUG/MINOR: ssl/cli: can't find ".crt" files when replacing a certificate
Bug was introduced by commit 26654 ("MINOR: ssl: add "crt" in the
cert_exts array").

When looking for a .crt directly in the cert_exts array, the
ssl_sock_load_pem_into_ckch() function will be called with a argument
which does not have its ".crt" extensions anymore.

If "ssl-load-extra-del-ext" is used this is not a problem since we try
to add the ".crt" when doing the lookup in the tree.

However when using directly a ".crt" without this option it will failed
looking for the file in the tree.

The fix removes the "crt" entry from the array since it does not seem to
be really useful without a rework of all the lookups.

Should fix issue #2265

Must be backported as far as 2.6.
2023-08-28 18:20:39 +02:00
Mariam John
00b7b49a46 MEDIUM: ssl: new sample fetch method to get curve name
Adds a new sample fetch method to get the curve name used in the
key agreement to enable better observability. In OpenSSLv3, the function
`SSL_get_negotiated_group` returns the NID of the curve and from the NID,
we get the curve name by passing the NID to OBJ_nid2sn. This was not
available in v1.1.1. SSL_get_curve_name(), which returns the curve name
directly was merged into OpenSSL master branch last week but will be available
only in its next release.
2023-07-17 15:45:41 +02:00
William Lallemand
6e0c39d7ac BUILD: ssl: ssl_c_r_dn fetches uses functiosn only available since 1.1.1
Fix the openssl build with older openssl version by disabling the new
ssl_c_r_dn fetch.

This also disable the ssl_client_samples.vtc file for OpenSSL version
older than 1.1.1
2023-05-15 12:07:52 +02:00
Abhijeet Rastogi
df97f472fa MINOR: ssl: add new sample ssl_c_r_dn
This patch addresses #1514, adds the ability to fetch DN of the root
ca that was in the chain when client certificate was verified during SSL
handshake.
2023-05-15 10:48:05 +02:00
William Lallemand
a415e895a9 REGTESTS: ssl: simplify X509_V code check in ssl_client_auth.vtc
simplify the X509_V code check in ssl_client_auth.vtc
2023-05-03 15:54:49 +02:00
Willy Tarreau
5003ac7fe9 MEDIUM: config: set useful ALPN defaults for HTTPS and QUIC
This commit makes sure that if three is no "alpn", "npn" nor "no-alpn"
setting on a "bind" line which corresponds to an HTTPS or QUIC frontend,
we automatically turn on "h2,http/1.1" as an ALPN default for an HTTP
listener, and "h3" for a QUIC listener. This simplifies the configuration
for end users since they won't have to explicitly configure the ALPN
string to enable H2, considering that at the time of writing, HTTP/1.1
represents less than 7% of the traffic on large infrastructures. The
doc and regtests were updated. For more info, refer to the following
thread:

  https://www.mail-archive.com/haproxy@formilux.org/msg43410.html
2023-04-19 09:52:20 +02:00
Willy Tarreau
2e3c04a7c7 REGTESTS: add a new "ssl_alpn" test to test ALPN negotiation
This teg-test verifies that different ALPN values on the "server" line
will negotiate the expected protocol depending on the ALPN "bind" line.
2023-04-19 08:38:06 +02:00
Remi Tricot-Le Breton
dafc068f12 MINOR: ssl: Accept certpath as param in "show ssl ocsp-response" CLI command
In order to increase usability, the "show ssl ocsp-response" also takes
a frontend certificate path as parameter. In such a case, it behaves the
same way as "show ssl cert foo.pem.ocsp".
2023-03-14 11:07:32 +01:00
Remi Tricot-Le Breton
a6c0a59e9a MINOR: ssl: Use ocsp update task for "update ssl ocsp-response" command
Instead of having a dedicated httpclient instance and its own code
decorrelated from the actual auto update one, the "update ssl
ocsp-response" will now use the update task in order to perform updates.

Since the cli command allows to update responses that were never
included in the auto update tree, a new flag was added to the
certificate_ocsp structure so that the said entry can be inserted into
the tree "by hand" and it won't be reinserted back into the tree after
the update process is performed. The 'update_once' flag "stole" a bit
from the 'fail_count' counter since it is the one less likely to reach
UINT_MAX among the ocsp counters of the certificate_ocsp structure.

This new logic required that every certificate_ocsp entry contained all
the ocsp-related information at all time since entries that are not
supposed to be configured automatically can still be updated through the
cli. The logic of the ssl_sock_load_ocsp was changed accordingly.
2023-03-14 11:07:32 +01:00
Remi Tricot-Le Breton
86d1e0b163 BUG/MINOR: ssl: Fix ocsp-update when using "add ssl crt-list"
When adding a new certificate through the CLI and appending it to a
crt-list with the 'ocsp-update' option set, the new certificate would
not be added to the OCSP response update list.
The only thing that was missing was the copy of the ocsp_update mode
from the ssl_bind_conf into the ckch_store's object.
An extra wakeup of the update task also needed to happen in case the
newly inserted entry needs to be updated before the next wakeup of the
task.

This patch does not need to be backported.
2023-03-02 15:57:56 +01:00
Remi Tricot-Le Breton
5ab54c61b0 REGTESTS: ssl: Add test for new ocsp update cli commands
Add tests for the "show ssl ocsp-updates" cli command as well as the new
'base64' parameter that can be passed to the "show ssl ocsp-response"
command.
2023-03-02 15:57:55 +01:00
Remi Tricot-Le Breton
780504ae4d REGTESTS: ssl: Fix ocsp update crt-lists
The options were after the filters which does not work well and now
raises a warning. It did not break the regtest because the crt-lists
were not actually used by clients.
2023-03-02 15:37:23 +01:00
Christopher Faulet
4ad6ee94ab REGTESTS: Fix ssl_errors.vtc script to wait for connections close
In this scripts, several clients perform a requests and exit because an SSL
error is expected and thus no response is sent. However, we must explicitly
wait for the connection close, via an "expect_close" statement.  Otherwise,
depending on the timing, HAProxy may detect the client abort before any
connection attempt on the server side and no SSL error is reported, making
the script to fail.
2023-02-21 11:44:55 +01:00
Remi Tricot-Le Breton
522841c47b REGTEST: ssl: Add test for 'update ssl ocsp-response' CLI command
This patch adds tests for the newly added 'update ssl ocsp-response' CLI
command.
2023-01-12 13:13:45 +01:00
Remi Tricot-Le Breton
2f275fb338 REGTESTS: ssl: Add tests for ocsp auto update mechanism
Tests a subpart of the ocsp auto update feature. It will mainly focus on
the 'auto' mode since the 'on' one relies strongly on timers way too
long to be used in a regtest context.
2022-12-21 11:21:07 +01:00
William Lallemand
8f5699bda1 REGTESTS: ssl: enable the ssl_reuse.vtc test for WolfSSL
Not working yet but it is needed to debug session resumption with
wolfSSL.

Could be backported in 2.7.
2022-12-20 15:28:37 +01:00
William Lallemand
9fbc84e571 MINOR: ssl: x509_v_err_str converter transforms an integer to a X509_V_ERR name
The x509_v_err_str converter transforms a numerical X509 verify error
to its constant name.
2022-11-10 13:28:37 +01:00
William Lallemand
960fb74cae MEDIUM: ssl: {ca,crt}-ignore-err can now use error constant name
The ca-ignore-err and crt-ignore-err directives are now able to use the
openssl X509_V_ERR constant names instead of the numerical values.

This allow a configuration to survive an OpenSSL upgrade, because the
numerical ID can change between versions. For example
X509_V_ERR_INVALID_CA was 24 in OpenSSL 1 and is 79 in OpenSSL 3.

The list of errors must be updated when a new major OpenSSL version is
released.
2022-11-10 13:28:37 +01:00
William Lallemand
23bc0b20bd REGTESTS: ssl/log: test the log-forward with SSL
Test the log-forward section with an SSL server and an SSL bind.

Must be backported as far as 2.3.
2022-09-13 17:03:30 +02:00
William Lallemand
62c0b99e3b MINOR: ssl/cli: implement "add ssl ca-file"
In ticket #1805 an user is impacted by the limitation of size of the CLI
buffer when updating a ca-file.

This patch allows a user to append new certificates to a ca-file instead
of trying to put them all with "set ssl ca-file"

The implementation use a new function ssl_store_dup_cafile_entry() which
duplicates a cafile_entry and its X509_STORE.

ssl_store_load_ca_from_buf() was modified to take an apped parameter so
we could share the function for "set" and "add".
2022-08-19 19:58:53 +02:00
Ilya Shipitsin
b6189bc268 REGTESTS: ssl: fix grep invocation to use extended regex in ssl_generate_certificate.vtc
in 2f2a2884b7 grep should have use regex flag -E, but flag
was lost by mistake
2022-08-06 23:24:13 +02:00
Ilya Shipitsin
2f2a2884b7 REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
on Ubuntu-22.04 openssl-3.0.5 is shipped which has changed ec curve
description to "Server Temp Key: ECDH, secp384r1, 384 bits"
2022-08-06 17:46:10 +02:00
Ilya Shipitsin
0865160b93 REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
on Ubuntu-22.04 openssl-3.0.5 is shipped which has changed ec curve
description to "Server Temp Key: ECDH, prime256v1, 256 bits"
2022-08-06 17:45:55 +02:00
William Lallemand
ae6547f65f REGTESTS: ssl: add the same cert for client/server
Add the same certificate in server and bind line so we can try to catch
problems like in issue #1748 when updating over the CLI.
2022-06-20 18:06:43 +02:00
William Lallemand
cb6c5f4683 BUG/MEDIUM: ssl/cli: crash when crt inserted into a crt-list
The crash occures when the same certificate which is used on both a
server line and a bind line is inserted in a crt-list over the CLI.

This is quite uncommon as using the same file for a client and a server
certificate does not make sense in a lot of environments.

This patch fixes the issue by skipping the insertion of the SNI when no
bind_conf is available in the ckch_inst.

Change the reg-test to reproduce this corner case.

Should fix issue #1748.

Must be backported as far as 2.2. (it was previously in ssl_sock.c)
2022-06-20 17:27:49 +02:00
Remi Tricot-Le Breton
85c0ff5987 REGTESTS: ssl: Update error messages that changed with OpenSSLv3.1.0-dev
Some error messages changed with OpenSSL 3.1.0-dev, making the
ssl_errors.vtc wrongly fail.
2022-04-21 14:54:47 +02:00
Remi Tricot-Le Breton
1d6338ea96 MEDIUM: ssl: Disable DHE ciphers by default
DHE ciphers do not present a security risk if the key is big enough but
they are slow and mostly obsoleted by ECDHE. This patch removes any
default DH parameters. This will effectively disable all DHE ciphers
unless a global ssl-dh-param-file is defined, or
tune.ssl.default-dh-param is set, or a frontend has DH parameters
included in its PEM certificate. In this latter case, only the frontends
that have DH parameters will have DHE ciphers enabled.
Adding explicitely a DHE ciphers in a "bind" line will not be enough to
actually enable DHE. We would still need to know which DH parameters to
use so one of the three conditions described above must be met.

This request was described in GitHub issue #1604.
2022-04-20 17:30:55 +02:00
Amaury Denoyelle
76e8b70e43 MEDIUM: server: remove experimental-mode for dynamic servers
Dynamic servers feature is now judged to be stable enough. Remove the
experimental-mode requirement for "add/del server" commands. This should
facilitate dynamic servers adoption.
2022-03-11 14:28:28 +01:00
Willy Tarreau
42f2a511d3 REGTESTS: server: close an occasional race on dynamic_server_ssl.vtc
Sometimes when sending commands to shut down a server, haproxy complains
that some connections remain, this is because the server-side connection
might not always be completely released at the moment the client leaves
and the operation is emitted. While shutting down server sessions work,
it seems cleaner to just use "option httpclose" which releases the server
earlier and avoids the race.

This can be backported to 2.5.
2022-02-16 14:42:13 +01:00
Remi Tricot-Le Breton
17b7384f82 REGTESTS: ssl: Add tests for DH related options
This new test checks that the DH-related mechanism works, be it through
specific DH parameters included in a bind line's certificate or by using
the ssl-dh-param-file or tune.ssl.default-dh-param global options.
2022-02-14 10:07:14 +01:00
Remi Tricot-Le Breton
3f269bb370 REGTESTS: ssl: Add test for "generate-certificates" SSL option
The 'generate-certificates' bind line option that allows to create
server certificates on-the-fly for newly used SNIs was not tested yet.
2022-02-09 12:10:32 +01:00
Remi Tricot-Le Breton
eb561cefd4 REGTESTS: ssl: Add test for "curves" and "ecdhe" SSL options
The "curves" and the older "ecdhe" SSL options that can be used to
define a subset of curves than can be used in an SSL handshake were not
tested in a regtest yet.
2022-02-09 11:15:44 +01:00
Ilya Shipitsin
8de3eff8f7 REGTESTS: ssl: skip show_ssl_ocspresponse.vtc when BoringSSL is used
OCSP stapling implementation is not compatible with BoringSSL, test
is broken in BoringSSL
2022-02-02 17:48:02 +01:00
Remi Tricot-Le Breton
aab8d255bc REGTESTS: ssl: Fix ssl_errors regtest with OpenSSL 1.0.2
This test was broken with OpenSSL 1.0.2 after commit a996763619
(BUG/MINOR: ssl: Store client SNI in SSL context in case of ClientHello
error) because it expected the default TLS version to be 1.3 in some
cases (when it can't be the case with OpenSSL 1.0.2).
2022-01-11 20:02:37 +01:00
Remi Tricot-Le Breton
a996763619 BUG/MINOR: ssl: Store client SNI in SSL context in case of ClientHello error
If an error is raised during the ClientHello callback on the server side
(ssl_sock_switchctx_cbk), the servername callback won't be called and
the client's SNI will not be saved in the SSL context. But since we use
the SSL_get_servername function to return this SNI in the ssl_fc_sni
sample fetch, that means that in case of error, such as an SNI mismatch
with a frontend having the strict-sni option enabled, the sample fetch
would not work (making strict-sni related errors hard to debug).

This patch fixes that by storing the SNI as an ex_data in the SSL
context in case the ClientHello callback returns an error. This way the
sample fetch can fallback to getting the SNI this way. It will still
first call the SSL_get_servername function first since it is the proper
way of getting a client's SNI when the handshake succeeded.

In order to avoid memory allocations are runtime into this highly used
runtime function, a new memory pool was created to store those client
SNIs. Its entry size is set to 256 bytes since SNIs can't be longer than
255 characters.

This fixes GitHub #1484.

It can be backported in 2.5.
2022-01-10 16:31:22 +01:00
Ilya Shipitsin
37d3e38130 CLEANUP: assorted typo fixes in the code and comments
This is 30th iteration of typo fixes
2022-01-07 14:42:54 +01:00
William Lallemand
acd546b07c REGTESTS: ssl: update of a crt with server deletion
This test verifies that a certificate is in a "Unused" state once every
server which uses it are dynamically removed.
2021-12-30 16:57:16 +01:00
William Lallemand
0387632ac0 REGTESTS: ssl: fix ssl_default_server.vtc
Patch 2c776f1 ("BUG/MEDIUM: ssl: initialize correctly ssl w/
default-server") added tests that are not relevant anymore and broke the
reg-test. revert them.
2021-12-29 18:20:19 +01:00
William Lallemand
2c776f1c30 BUG/MEDIUM: ssl: initialize correctly ssl w/ default-server
This bug was introduced by d817dc73 ("MEDIUM: ssl: Load client
certificates in a ckch for backend servers") in which the creation of
the SSL_CTX for a server was moved to the configuration parser when
using a "crt" keyword instead of being done in ssl_sock_prepare_srv_ctx().

The patch 0498fa40 ("BUG/MINOR: ssl: Default-server configuration ignored by
server") made it worse by setting the same SSL_CTX for every servers
using a default-server. Resulting in any SSL option on a server applied
to every server in its backend.

This patch fixes the issue by reintroducing a string which store the
path of certificate inside the server structure, and loading the
certificate in ssl_sock_prepare_srv_ctx() again.

This is a quick fix to backport, a cleaner way can be achieve by always
creating the SSL_CTX in ssl_sock_prepare_srv_ctx() and splitting
properly the ssl_sock_load_srv_cert() function.

This patch fixes issue #1488.

Must be backported as far as 2.4.
2021-12-29 14:42:16 +01:00
Ilya Shipitsin
fc1126610b REGTESTS: ssl: use X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY for cert check
LibreSSL-3.4.2 introduced cert revocation check behaviour change, for some
checks now X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY (20) is returned.

https://github.com/libressl-portable/portable/issues/697

let us modify vtc accordingly
2021-12-10 16:16:02 +01:00
William Lallemand
e15ab93244 REGTESTS: ssl: test the TLS resumption
This test is able to check if the TLS resumption is working correctly
with TLSv1.2, TLSv1.3, with tickets and session cache.
2021-11-19 04:07:07 +01:00
Willy Tarreau
f673923629 REGTESTS: extend the default I/O timeouts and make them overridable
With the CI occasionally slowing down, we're starting to see again some
spurious failures despite the long 1-second timeouts. This reports false
positives that are disturbing and doesn't provide as much value as this
could. However at this delay it already becomes a pain for developers
to wait for the tests to complete.

This commit adds support for the new environment variable
HAPROXY_TEST_TIMEOUT that will allow anyone to modify the connect,
client and server timeouts. It was set to 5 seconds by default, which
should be plenty for quite some time in the CI. All relevant values
that were 200ms or above were replaced by this one. A few larger
values were left as they are special. One test for the set-timeout
action that used to rely on a fixed 1-sec value was extended to a
fixed 5-sec, as the timeout is normally not reached, but it needs
to be known to compare the old and new values.
2021-11-18 17:57:11 +01:00