Commit Graph

11699 Commits

Author SHA1 Message Date
William Lallemand 0a9b9414f0 MINOR: ssl/cli: 'del ssl crt-list' delete an entry
Delete an entry in a crt-list, this is done by iterating over the
ckch_inst in the crtlist_entry. For each ckch_inst the bind_conf lock is
held during the deletion of the sni_ctx in the SNI trees. Everything
is free'd.

If there is several entries with the same certificate, a line number
must be provided to chose with entry delete.
2020-04-06 19:33:28 +02:00
William Lallemand 58a522227b BUG/MINOR: ssl/cli: fix spaces in 'show ssl crt-list'
Fix a inconsistency in the spaces which were not printed everywhere if
there was no SSL options but some filters.
2020-04-02 18:15:30 +02:00
William Lallemand accac23aff DOC: management: add the new crt-list CLI commands
Add the new crt-lists command in the management manual:
"new ssl cert", "add ssl crt-list", "dump ssl crt-list", "show ssl
crt-list".
2020-04-02 17:50:02 +02:00
Ilya Shipitsin 6fb0f2148f CLEANUP: assorted typo fixes in the code and comments
This is sixth iteration of typo fixes
2020-04-02 16:25:45 +02:00
William Lallemand 4fd9433fb0 REGTEST: ssl/cli: add other cases of 'add ssl crt-list'
Add some certificates in the crt-list with no filters and/or no options
so we can test more cases.
2020-04-02 15:40:19 +02:00
William Lallemand a690fed5be BUG/MINOR: ssl/cli: initialize fcount int crtlist_entry
Initialize fcount to 0 when 'add ssl crt-list' does not contain any
filters. This bug can lead to trying to read some filters even if they
doesn't exist.
2020-04-02 15:40:19 +02:00
Ilya Shipitsin b7e43f0cab CLEANUP: src/fd.c: mask setsockopt with DISGUISE
we do not care of what is returned, let static analyzers to
know that
2020-04-02 15:10:17 +02:00
Frdric Lcaille 876ed55d9b BUG/MINOR: protocol_buffer: Wrong maximum shifting.
This patch fixes a bad stop condition when decoding a protocol buffer variable integer
whose maximum lenghts are 10, shifting a uint64_t value by more than 63.

Thank you to Ilya for having reported this issue.

Must be backported to 2.1 and 2.0.
2020-04-02 15:09:46 +02:00
Willy Tarreau 5dfc5d5cd0 BUG/CRITICAL: hpack: never index a header into the headroom after wrapping
The HPACK header table is implemented as a wrapping list inside a contigous
area. Headers names and values are stored from right to left while indexes
are stored from left to right. When there's no more room to store a new one,
we wrap to the right again, or possibly defragment it if needed. The condition
do use the right part (called tailroom) or the left part (called headroom)
depends on the location of the last inserted header. After wrapping happens,
the code forces to stick to tailroom by pretending there's no more headroom,
so that the size fit test always fails. The problem is that nothing prevents
from storing a header with an empty name and empty value, resulting in a
total size of zero bytes, which satisfies the condition to use the headroom.
Doing this in a wrapped buffer results in changing the "front" header index
and causing miscalculations on the available size and the addresses of the
next headers. This may even allow to overwrite some parts of the index,
opening the possibility to perform arbitrary writes into a 32-bit relative
address space.

This patch fixes the issue by making sure the headroom is considered only
when the buffer does not wrap, instead of relying on the zero size. This
must be backported to all versions supporting H2, which is as far as 1.8.

Many thanks to Felix Wilhelm of Google Project Zero for responsibly
reporting this problem with a reproducer and a detailed analysis.
CVE-2020-11100 was assigned to this issue.
2020-04-02 08:45:54 +02:00
William Lallemand 785325141d REGTEST: ssl: pollute the crt-list file
Pollute localhost.crt-list with extra spaces, empty lines and comments
so the parser of the crt-list could be tested in a better way.
2020-04-01 20:10:53 +02:00
William Lallemand fdb6db4850 REGTEST: ssl/cli: tests options and filters w/ add ssl crt-list
Now that the 'add ssl crt-list' command supports filters and options,
add some in the vtc file to test them.
2020-04-01 20:10:53 +02:00
William Lallemand 557823f847 MINOR: ssl: add a comment above the ssl_bind_conf keywords
Add a warning above the ssl_bind_conf keywords list so developers check
if their keywords are relevant for the list.
2020-04-01 20:10:53 +02:00
William Lallemand c7c7a6b39f MINOR: ssl/cli: support filters and options in add ssl crt-list
Add the support for filters and SSL options in the CLI command
"add ssl crt-list".

The feature was implemented by applying the same parser as the crt-list
file to the payload.

The new options are passed to the command as a payload with the same
format that is suppported by the crt-list file itself, so you can easily
copy a line from a file and push it via the CLI.

Example:
  printf "add ssl crt-list localhost.crt-list <<\necdsa.pem [verify none allow-0rtt] localhost !www.test1.com\n\n" | socat /tmp/sock1 -
2020-04-01 20:10:53 +02:00
William Lallemand 97b0810f4c MINOR: ssl: split the line parsing of the crt-list
In order to reuse the crt-list line parsing in "add ssl crt-list",
the line parsing was extracted from crtlist_parse_file() to a new
function crtlist_parse_line().

This function fills a crtlist_entry structure with a bind_ssl_conf and
the list of filters.
2020-04-01 20:10:53 +02:00
Olivier Houchard b17b884870 BUG/MEDIUM: dns: Consider the fact that dns answers are case-insensitive
We can't expect the DNS answer to always match the case we used for the
request, so we can't just use memcmp() to compare the DNS answer with what
we are expected.
Instead, introduce dns_hostname_cmp(), which compares each string in a
case-insensitive way.
This should fix github issue #566.

This should be backported to 2.1, 2.0, 1.9 and 1.8.
2020-04-01 18:35:05 +02:00
Willy Tarreau 5e8017d53c REGTEST: make the unique-id test depend on version 2.0
Regtest unique-id.vtc was added by commit 5fcec84c58 ("REGTEST: Add
unique-id reg-test") but it relies on the "uuid" sample fetch which
is only available in version 2.0 and above. Let's reflect that in
the REQUIRE_VERSION tag.
2020-04-01 16:08:43 +02:00
Olivier Houchard 4a0e7fe4f7 MINOR: connections: Don't mark conn flags 0x00000001 and 0x00000002 as unused.
Remove the comments saying 0x00000001 and 0x00000002 are unused, they are
now used by CO_FL_SAFE_LIST and CO_FL_IDLE_LIST.
2020-03-31 23:04:20 +02:00
Miroslav Zagorac d80f5c0d0c DOC: internals: Fix spelling errors in filters.txt 2020-03-31 17:24:07 +02:00
Daniel Corbett b428517fee BUG/MINOR: stats: Fix color of draining servers on stats page
This patch fixes #53 where it was noticed that when an active
server is set to DRAIN it no longer has the color blue reflected
within the stats page. This patch addresses that and adds the
color back to drain. It's to be noted that backup servers are
configured to have an orange color when they are draining.

Should be backported as far as 1.7.
2020-03-31 17:21:51 +02:00
Ilya Shipitsin ce7b00f926 CLEANUP: assorted typo fixes in the code and comments
This is fifth iteration of typo fixes
2020-03-31 17:09:35 +02:00
Willy Tarreau 1d52c7b52b REGTEST: make the PROXY TLV validation depend on version 2.2
Regtest proxy_protocol_tlv_validation was added by commit 488ee7fb6e
("BUG/MAJOR: proxy_protocol: Properly validate TLV lengths") but it
relies on a trick involving http-after-response to append a header
after a 400-badreq response, which is not possible in earlier versions,
so make it depend on 2.2.
2020-03-31 16:37:58 +02:00
William Lallemand 4781fad407 REGTEST: ssl/cli: change test type to devel
Change the type of test from slow to devel for add_ssl-crt-list.vtc and
set_ssl_cert.vtc.
2020-03-31 14:52:22 +02:00
William Lallemand c2e3b72adf BUG/MINOR: ssl: entry->ckch_inst not initialized
The head of the list entry->ckch_inst was not initialized when opening a
directory or reading a crt-list.
2020-03-31 14:40:51 +02:00
William Lallemand 2be4a2e02d REGTEST: ssl/cli: test the 'add ssl crt-list' command
Test the 'add ssl crt-list' feature by inserting the ecdsa.pem
certificate and verifying with curl and strict-sni that it works.
2020-03-31 12:32:18 +02:00
William Lallemand e67c80be7f MEDIUM: ssl/cli: 'add ssl crt-list' command
The new 'add ssl crt-list' command allows to add a new crt-list entry in
a crt-list (or a directory since they are handled the same way).

The principle is basicaly the same as the "commit ssl cert" command with
the exception that it iterates on every bind_conf that uses the crt-list
and generates a ckch instance (ckch_inst) for each of them.

The IO handler yield every 10 bind_confs so HAProxy does not get stuck in
a too much time consuming generation if it needs to generate too much
SSL_CTX'.

This command does not handle the SNI filters and the SSL configuration
yet.

Example:

  $ echo "new ssl cert foo.net.pem" | socat /tmp/sock1 -
  New empty certificate store 'foo.net.pem'!

  $ echo -e -n "set ssl cert foo.net.pem <<\n$(cat foo.net.pem)\n\n" | socat /tmp/sock1 -
  Transaction created for certificate foo.net.pem!

  $ echo "commit ssl cert foo.net.pem" | socat /tmp/sock1 -
  Committing foo.net.pem
  Success!

  $ echo "add ssl crt-list one.crt-list foo.net.pem" | socat /tmp/sock1 -
  Inserting certificate 'foo.net.pem' in crt-list 'one.crt-list'......
  Success!

  $ echo "show ssl crt-list one.crt-list" | socat /tmp/sock1 -
  # one.crt-list
  0x55d17d7be360 kikyo.pem.rsa [ssl-min-ver TLSv1.0 ssl-max-ver TLSv1.3]
  0x55d17d82cb10 foo.net.pem
2020-03-31 12:32:18 +02:00
William Lallemand 90afe90681 MINOR: ssl/cli: update pointer to store in 'commit ssl cert'
The crtlist_entry structure use a pointer to the store as key.
That's a problem with the dynamic update of a certificate over the CLI,
because it allocates a new ckch_store. So updating the pointers is
needed. To achieve that, a linked list of the crtlist_entry was added in
the ckch_store, so it's easy to iterate on this list to update the
pointers. Another solution would have been to rework the system so we
don't allocate a new ckch_store, but it requires a rework of the ckch
code.
2020-03-31 12:32:17 +02:00
William Lallemand fa8cf0c476 MINOR: ssl: store a ptr to crtlist in crtlist_entry
Store a pointer to crtlist in crtlist_entry so we can re-insert a
crtlist_entry in its crtlist ebpt after updating its key.
2020-03-31 12:32:17 +02:00
William Lallemand 23d61c00b9 MINOR: ssl: add a list of crtlist_entry in ckch_store
When updating a ckch_store we may want to update its pointer in the
crtlist_entry which use it. To do this, we need the list of the entries
using the store.
2020-03-31 12:32:17 +02:00
William Lallemand 09bd5a0787 MINOR: ssl: use crtlist_free() upon error in directory loading
Replace the manual cleaninp which is done in crtlist_load_cert_dir() by
a call to the crtlist_free() function.
2020-03-31 12:32:17 +02:00
William Lallemand 4c68bba5c1 REORG: ssl: move some functions above crtlist_load_cert_dir()
Move some function above crtlist_load_cert_dir() so
crtlist_load_cert_dir() is at the right place, and crtlist_free() can be
used inside.
2020-03-31 12:32:17 +02:00
William Lallemand 493983128b BUG/MINOR: ssl: ckch_inst wrongly inserted in crtlist_entry
The instances were wrongly inserted in the crtlist entries, all
instances of a crt-list were inserted in the last crt-list entry.
Which was kind of handy to free all instances upon error.

Now that it's done correctly, the error path was changed, it must
iterate on the entries and find the ckch_insts which were generated for
this bind_conf. To avoid wasting time, it stops the iteration once it
found the first unsuccessful generation.
2020-03-31 12:32:17 +02:00
William Lallemand ad3c37b760 REORG: ssl: move SETCERT enum to ssl_sock.h
Move the SETCERT enum at the right place to cleanup ssl_sock.c.
2020-03-31 12:32:17 +02:00
William Lallemand 79d31ec0d4 MINOR: ssl: add a list of bind_conf in struct crtlist
In order to be able to add new certificate in a crt-list, we need the
list of bind_conf that uses this crt-list so we can create a ckch_inst
for each of them.
2020-03-31 12:32:17 +02:00
William Lallemand 638f6ad033 MINOR: cli: add a general purpose pointer in the CLI struct
This patch adds a p2 generic pointer which is inialized to zero before
calling the parser.
2020-03-31 12:32:17 +02:00
Olivier Houchard 079cb9af22 MEDIUM: connections: Revamp the way idle connections are killed
The original algorithm always killed half the idle connections. This doesn't
take into account the way the load can change. Instead, we now kill half
of the exceeding connections (exceeding connection being the number of
used + idle connections past the last maximum used connections reached).
That way if we reach a peak, we will kill much less, and it'll slowly go back
down when there's less usage.
2020-03-30 00:30:07 +02:00
Olivier Houchard cf612a0457 MINOR: servers: Add a counter for the number of currently used connections.
Add a counter to know the current number of used connections, as well as the
max, this will be used later to refine the algorithm used to kill idle
connections, based on current usage.
2020-03-30 00:30:01 +02:00
Jerome Magnin 824186bb08 MEDIUM: stream: support use-server rules with dynamic names
With server-template was introduced the possibility to scale the
number of servers in a backend without needing a configuration change
and associated reload. On the other hand it became impractical to
write use-server rules for these servers as they would only accept
existing server labels as argument. This patch allows the use of
log-format notation to describe targets of a use-server rules, such
as in the example below:

  listen test
    bind *:1234
    use-server %[hdr(srv)] if { hdr(srv) -m found }
    use-server s1 if { path / }
    server s1 127.0.0.1:18080
    server s2 127.0.0.1:18081

If a use-server rule is applied because it was conditionned by an
ACL returning true, but the target of the use-server rule cannot be
resolved, no other use-server rule is evaluated and we fall back to
load balancing.

This feature was requested on the ML, and bumped with issue #563.
2020-03-29 09:55:10 +02:00
Jerome Magnin eb421b2fe0 MINOR: listener: add so_name sample fetch
Add a sample fetch for the name of a bind. This can be useful to
take decisions when PROXY protocol is used and we can't rely on dst,
such as the sample config below.

  defaults
    mode http
  listen bar
    bind 127.0.0.1:1111
    server s1 127.0.1.1:1234 send-proxy

  listen foo
    bind 127.0.1.1:1234 name foo accept-proxy
    http-request return status 200 hdr dst %[dst] if { dst 127.0.1.1 }
2020-03-29 05:47:29 +02:00
Emmanuel Hocdet 1673977892 MINOR: ssl: skip self issued CA in cert chain for ssl_ctx
First: self issued CA, aka root CA, is the enchor for chain validation,
no need to send it, client must have it. HAProxy can skip it in ssl_ctx.
Second: the main motivation to skip root CA in ssl_ctx is to be able to
provide it in the chain without drawback. Use case is to provide issuer
for ocsp without the need for .issuer and be able to share it in
issuers-chain-path. This concerns all certificates without intermediate
certificates. It's useless for BoringSSL, .issuer is ignored because ocsp
bits doesn't need it.
2020-03-26 12:53:53 +01:00
Baptiste Assmann 37950c8d27 BUG/MEDIUM: dns: improper parsing of aditional records
13a9232ebc introduced parsing of
Additionnal DNS response section to pick up IP address when available.
That said, this introduced a side effect for other query types (A and
AAAA) leading to consider those responses invalid when parsing the
Additional section.
This patch avoids this situation by ensuring the Additional section is
parsed only for SRV queries.
2020-03-26 12:43:36 +01:00
Baptiste Assmann 17ab79f07d CLEANUP: remove obsolete comments
This patch removes some old comments introduced by
13a9232ebc.
Those comments are related to issues already fixed.
2020-03-26 12:43:36 +01:00
Olivier Houchard c3500c3ccd MINOR: build: Fix build in mux_h1
We want to check if the input buffer contains data, not the connection.
This should unbreak the build.
2020-03-25 17:06:16 +01:00
Olivier Houchard 69664419d2 BUG/MEDIUM: mux_h1: Process a new request if we already received it.
In h1_detach(), if our input buffer isn't empty, don't just subscribe(), we
may hold a new request, and there's nothing left to read. Instead, call
h1_process() directly, so that a new stream is created.
Failure to do so means if we received the new request to early, the
connecetion will just hang, as it happens when using svn.
2020-03-25 12:38:40 +01:00
Ilya Shipitsin 3e128fe973 CI: github actions: add weekly h2spec test
ML link: https://www.mail-archive.com/haproxy@formilux.org/msg36753.html

this commit adds scheduled run of h2spec tool to test http2 and HPACK
compliance.

h2spec might be found at: https://github.com/summerwind/h2spec
2020-03-24 21:04:25 +01:00
Frédéric Lécaille 87eacbb12f BUG/MINOR: peers: Use after free of "peers" section.
When a "peers" section has not any local peer, it is removed of the list
of "peers" sections by check_config_validity(). But a stick-table which
refers to a "peers" section stores a pointer to this peers section.
These pointer must be reset to NULL value for each stick-table refering to
such a "peers" section to prevent stktable_init() to start the peers frontend
attached to the peers section dereferencing the invalid pointer.

Furthemore this patch stops the peers frontend as this is done for other
configurations invalidated by check_config_validity().

Thank you to Olivier D for having reported this issue with such a
simple configuration file which made haproxy crash when started with
-c option for configuration file validation.

  defaults
    mode http

  peers mypeers
    peer toto 127.0.0.1:1024

  backend test
    stick-table type ip size 10k expire 1h store http_req_rate(1h) peers mypeers

Must be backported to 2.1 and 2.0.
2020-03-24 20:49:38 +01:00
William Lallemand 3ef2d56530 BUG/MINOR: peers: avoid an infinite loop with peers_fe is NULL
Fix an infinite loop which was added in an attempt to fix #558.
If the peers_fe is NULL, it will loop forever.

Must be backported with a2cfd7e as far as 1.8.
2020-03-24 16:45:53 +01:00
William Lallemand a2cfd7e356 BUG/MINOR: peers: init bind_proc to 1 if it wasn't initialized
Tim reported that in master-worker mode, if a stick-table is declared
but not used in the configuration, its associated peers listener won't
bind.

This problem is due the fact that the master-worker and the daemon mode,
depend on the bind_proc field of the peers proxy to disable the listener.
Unfortunately the bind_proc is left to 0 if no stick-table were used in
the configuration, stopping the listener on all processes.

This fixes sets the bind_proc to the first process if it wasn't
initialized.

Should fix bug #558. Should be backported as far as 1.8.
2020-03-24 16:18:15 +01:00
Emmanuel Hocdet 4fed93eb72 MINOR: ssl: rework add cert chain to CTX to be libssl independent
SSL_CTX_set1_chain is used for openssl >= 1.0.2 and a loop with
SSL_CTX_add_extra_chain_cert for openssl < 1.0.2.
SSL_CTX_add_extra_chain_cert exist with openssl >= 1.0.2 and can be
used for all openssl version (is new name is SSL_CTX_add0_chain_cert).
This patch use SSL_CTX_add_extra_chain_cert to remove any #ifdef for
compatibilty. In addition sk_X509_num()/sk_X509_value() replace
sk_X509_shift() to extract CA from chain, as it is used in others part
of the code.
2020-03-24 14:46:01 +01:00
Emmanuel Hocdet ef87e0a3da CLEANUP: ssl: rename ssl_get_issuer_chain to ssl_get0_issuer_chain
Rename ssl_get_issuer_chain to ssl_get0_issuer_chain to be consistent
with openssl >= 1.0.2 API.
2020-03-23 15:35:39 +01:00
Emmanuel Hocdet f4f14eacd3 BUG/MINOR: ssl: memory leak when find_chain is NULL
This bug was introduced by 85888573 "BUG/MEDIUM: ssl: chain must be
initialized with sk_X509_new_null()". No need to set find_chain with
sk_X509_new_null(), use find_chain conditionally to fix issue #516.

This bug was referenced by issue #559.

[wla: fix some alignment/indentation issue]
2020-03-23 13:10:10 +01:00