Commit Graph

12845 Commits

Author SHA1 Message Date
Brad Smith
382001b46b BUILD: Add a DragonFlyBSD target
Add a target for DragonFlyBSD 4.3 and above.
2020-10-08 20:54:18 +02:00
Willy Tarreau
465dc7d63f DOC: fix a confusing typo on a regsub example
Sébastien reported a confusing example in the doc about regsub when used
with quotes. Nested quotes are already not trivial to grasp, but when
typos are there and result in something valid, it's even worse. The closing
quote ought to have been inside the brackets. However haproxy will not make
any difference because the single quotes delimit a word and the delimited
word remains the same. Let's just not add yet another level of confusion.
2020-10-08 18:05:56 +02:00
Emeric Brun
da46c1ca60 DOC: re-work log forward bind statement documentation.
This patch re-work the documentation about the bind statement
of log forward section.
2020-10-08 08:54:04 +02:00
Emeric Brun
b0c331f71f BUG/MINOR: proxy/log: frontend/backend and log forward names must differ
This patch disallow to use same name for a log forward section
and a frontend/backend section.
2020-10-08 08:53:26 +02:00
Emeric Brun
cbb7bf7dd1 MEDIUM: log: syslog TCP support on log forward section.
This patch re-introduce the "bind" statement on log forward
sections to handle syslog TCP listeners as defined in
rfc-6587.

As complement it introduce "maxconn", "backlog" and "timeout
client" statements to parameter those listeners.
2020-10-07 17:17:27 +02:00
Emeric Brun
6d75616951 MINOR: channel: new getword and getchar functions on channel.
This patch adds two new functions to get a char
or a word from a channel.
2020-10-07 17:17:27 +02:00
Emeric Brun
2897644ae5 MINOR: stats: inc req counter on listeners.
This patch enables count of requests for listeners
if listener's counters are enabled.
2020-10-07 17:17:27 +02:00
Emeric Brun
c47ba59d1e BUG/MEDIUM: log: old processes with log foward section don't die on soft stop.
Old processes didn't die if a log foward section is declared and
a soft stop is requested.

This patch fix this issue and should be backpored in banches including
the log forward feature.
2020-10-07 17:17:27 +02:00
Emeric Brun
a39ecbdac1 BUG/MINOR: proxy: inc req counter on new syslog messages.
Increase req counter instead of conn counter on
new syslog messages.

This should be backported on branches including the
syslog forward feature.
2020-10-07 17:17:27 +02:00
Christopher Faulet
9589aa0fe5 CLEANUP: sock-unix: Remove an unreachable goto clause
Coverity reported dead code in sock_unix_bind_receiver() function. A goto clause
is unreachable because of the preceeding if/else block.

This patch should fix the issue #865. No backport needed.
2020-10-07 14:37:03 +02:00
Christopher Faulet
7b06d3adaa MINOR: mux-h1: Don't wakeup the H1C when output buffer become available
There is no reason to wake up the H1 connection when a new output buffer is
retrieved after an allocation failure because only the H1 stream will fill it.
2020-10-07 14:07:29 +02:00
Christopher Faulet
e9da975aab BUG/MINOR: mux-h1: Always set the session on frontend h1 stream
The session is always defined for a frontend connection. When a new client
connection is established, the session is set for the first H1 stream. But on
keep-alived connections, it is not set for the followings H1 streams while it is
possible.

This patch is tagged as a bug because it fixes an inconsistency in the H1
streams creation. But it does not fixed a known bug.

This patch must be backported as far as 2.0.
2020-10-07 14:07:29 +02:00
Christopher Faulet
69f2cb8df3 BUG/MINOR: mux-h1: Be sure to only set CO_RFL_READ_ONCE for the first read
The condition to set CO_RFL_READ_ONCE flag is not really accurate. We must check
the request state on frontend connection only and, in the opposite, the response
state on backend connection only. Only the parsed side must be considered, not
the opposite one.

This patch must be backported to 2.2.
2020-10-07 14:07:29 +02:00
Christopher Faulet
58feb49ed2 CLEANUP: ssl: Release cached SSL sessions on deinit
On deinit, when the server SSL ctx is released, we must take care to release the
cached SSL sessions stored in the array <ssl_ctx.reused_sess>. There are
global.nbthread entries in this array, each one may have a pointer on a cached
session.

This patch should fix the issue #802. No backport needed.
2020-10-07 14:07:29 +02:00
Tim Duesterhus
d7c6e6a71d CLEANUP: cache: Fix leak of cconf->c.name during config check
During the config check, the post parsing is not performed. Thus, cache filters
are not fully initialized and their cache name are never released. To be able to
release them, a flag is now set when a cache filter is fully initialized. On
deinit, if the flag is not set, it means the cache name must be freed.

The patch should fix #849. No backport needed.

[Cf: Tim is the patch author, but I added the commit message]
2020-10-07 14:07:29 +02:00
Christopher Faulet
a10000305f BUG/MINOR: proto_tcp: Report warning messages when listeners are bound
When a TCP listener is bound, in the tcp_bind_listener() function, a warning
message may be reported and should be displayed on verbose mode. But the warning
message is actually lost if the socket is successfully bound because we don't
fill the <errmsg> variable in this case.

This patch should fix the issue #863. No backport is needed.
2020-10-07 14:07:16 +02:00
Brad Smith
3f1977c934 DOC: update INSTALL with supported OpenBSD / FreeBSD versions
Update INSTALL with the OpenBSD / FreeBSD versions that are supported.
2020-10-07 08:10:25 +02:00
Frédéric Lécaille
e7e2b21d27 BUG/MINOR: peers: Inconsistency when dumping peer status codes.
A peer connection status must be considered as valid only if there is an applet
which has been instantiated for the connection to the peer. So, ->statuscode
should be considered as the last known peer connection status from the last
connection to this peer if any. To reflect this, "statuscode" field of peer dump
is renamed to "last_statuscode".
This patch also add "active"/"inactive" field after the peer location type
("remote" or "local") if an applet has been instantiated for this peer connection
or not.

Thank you to Emeric for having noticed this issue.

Must be backported in >=1.9 version.
2020-10-07 07:27:01 +02:00
William Dauchy
57dd6f1c5a DOC: ssl: fix typo about ocsp files
one of the last I saw in this section while working on github issue #872

might be backported in all still supported versions

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-10-06 15:47:50 +02:00
Amaury Denoyelle
27373f7f75 MINOR: stats: remove for loop declaration
Remove variable declaration inside a for-loop. This was introduced by my
patches serie of the implementation of dynamic stats. This is not
supported by older gcc, notably on the freebsd environment of the ci.
2020-10-05 17:55:40 +02:00
Amaury Denoyelle
fbd0bc98fe MINOR: dns/stats: integrate dns counters in stats
Use the new stats module API to integrate the dns counters in the
standard stats. This is done in order to avoid code duplication, keep
the code related to cli out of dns and use the full possibility of the
stats function, allowing to print dns stats in csv or json format.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
0b70a8a314 MINOR: stats: add config "stats show modules"
By default, hide the extra statistics on the html page. Define a new
flag STAT_SHMODULES which is activated if the config "stats show
modules" is set.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
e3f576c29e MINOR: stats: display extra proxy stats on the html page
Integrate the additional proxy stats on the html stats page. For each
module, a new column is displayed with the individual stats available as
a tooltip.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
d3700a7fda MINOR: stats: support clear counters for dynamic stats
Add a boolean 'clearable' on stats module structure. If set, it forces
all the counters to be reset on 'clear counters' cli command. If not,
the counters are reset only when 'clear counters all' is used.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
ee63d4bd67 MEDIUM: stats: integrate static proxies stats in new stats
This is executed on startup with the registered statistics module. The
existing statistics have been merged in a list containing all
statistics for each domain. This is useful to print all available
statistics in a generic way.

Allocate extra counters for all proxies/servers/listeners instances.
These counters are allocated with the counters from the stats modules
registered on startup.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
730c727ea3 MEDIUM: stats: add abstract type to store counters
Implement a small API to easily add extra counters inside a structure
instance. This will be used to implement dynamic statistics linked on
every type of object as needed.

The counters are stored in a dynamic array inside the relevant objects.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
58d395e0d6 MEDIUM: stats: define an API to register stat modules
A stat module can be registered to quickly add new statistics on
haproxy. It must be attached to one of the available stats domain. The
register must be done using INITCALL on STG_REGISTER.

The stat module has a name which should be unique for each new module in
a domain. It also contains a statistics list with their name/desc and a
pointer to a function used to fill the stats from the module counters.

The module also provides the initial counters values used on
automatically allocated counters. The offset for these counters
are stored in the module structure.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
50660a894d MEDIUM: stats: add delimiter for static proxy stats on csv
Use the character '-' to mark the end of static statistics on proxy
domain. After this marker, the order of the fields is not guaranteed and
should be parsed with care.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
72b16e5173 MINOR: stats: define additional flag px cap on domain
This flag can be used to determine on what type of proxy object the
statistics should be relevant. It will be useful when adding dynamic
statistics. Currently, this flag is not used.
2020-10-05 12:02:14 +02:00
Amaury Denoyelle
072f97eddf MINOR: stats: define the concept of domain for statistics
The domain option will be used to have statistics attached to other
objects than proxies/listeners/servers. At the moment, only the PROXY
domain is available.

Add an argument 'domain' on the 'show stats' cli command to specify the
domain. Only 'domain proxy' is available now. If not specified, proxy
will be considered the default domain.

For HTML output, only proxy statistics will be displayed.
2020-10-05 12:02:14 +02:00
Christopher Faulet
f98d821b94 MINOR: hlua: Display debug messages on stderr only in debug mode
Debug Messages emitted in lua using core.Debug() or core.log() are now only
displayed on stderr if HAProxy is started in debug mode (-d parameter on the
command line). There is no change for other message levels.

This patch should fix the issue #879. It may be backported to all stable
versions.
2020-10-05 11:11:36 +02:00
Amaury Denoyelle
98b81cb393 REORG: stats: extract proxies dump loop in a function
Create a dedicated function to loop on proxies and dump them. This will
be clearer when other object will be dump as well.

This patch is needed to extend stat support to components other than
proxies objects.
2020-10-05 10:54:35 +02:00
Amaury Denoyelle
f34017bb74 REORG: stats: extract proxy json dump
Create a dedicated function to dump a proxy as a json content. This
patch will be needed when other types of objects will be available for
json dump.

This patch is needed to extend stat support to components other than
proxies objects.
2020-10-05 10:53:50 +02:00
Amaury Denoyelle
da5b6d1cd9 MINOR: stats: hide px/sv/li fields in applet struct
Use an opaque pointer to store proxy instance. Regroup server/listener
as a single opaque pointer. This has the benefit to render the structure
more evolutive to support statistics on other types of objects in the
future.

This patch is needed to extend stat support for components other than
proxies objects.

The prometheus module has been adapted for these changes.
2020-10-05 10:48:58 +02:00
Amaury Denoyelle
97323c9ed4 MINOR: stats: add stats size as a parameter for csv/json dump
Render the stats size parametric in csv/json dump functions. This is
needed for the future patch which provides dynamic stats. For now the
static value ST_F_TOTAL_FIELDS is provided.

Remove unused parameter px on stats_dump_one_line.

This patch is needed to extend stat support to components other than
proxies objects.
2020-10-05 09:06:10 +02:00
Amaury Denoyelle
3ca927e68f REORG: stats: export some functions
Un-mark stats_dump_one_line and stats_putchk as static and export them
in the header file. These functions will be reusable by other components to
print their statistics.

This patch is needed to extend stat support to components other than
proxies objects.
2020-10-05 09:06:10 +02:00
Amaury Denoyelle
a53ce4cc01 BUG/MINOR: stats: fix validity of the json schema
The json schema seems to be invalid when checking using the validator
from https://www.jsonschemavalidator.net/. Correct it using the
following specification :
http://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.9.1

The impact of the bug it not well known as I am not sure of how useful
the json schema is for users. It is probably not used at all or else
this bug would have been reported.

This should be backported up to 1.8.
2020-10-05 09:06:06 +02:00
Amaury Denoyelle
cd3de50779 MINOR: counters: fix a typo in comment
Wrong copy/paste comment, replace listeners/frontends by
servers/backends

This may be backported up to 1.7.
2020-10-05 09:05:57 +02:00
William Lallemand
51f784bcf9 CLEANUP: ssl: "bundle" is not an OpenSSL wording
There is a confusion between the HAProxy bundle and OpenSSL. OpenSSL
does not have "bundles" but multiple certificates in the same store.

Fix a commentary in the crt-list code.
2020-10-02 18:11:47 +02:00
Willy Tarreau
fac0f645df BUG/MEDIUM: queue: make pendconn_cond_unlink() really thread-safe
A crash reported in github issue #880 looks impossible unless
pendconn_cond_unlink() occasionally sees a null leaf_p when attempting
to remove an entry, which seems to be confirmed by the reporter. What
seems to be happening is that depending on compiler optimizations,
this pointer can appear as null while pointers are moved if one of
the node's parents is removed from or inserted into the tree. There's
no explicit null of the pointer during these operations but those
pointers are rewritten in multiple steps and nothing prevents this
situation from happening, and there are no particular barrier nor
atomic ops around this.

This test was used to avoid unnecessary locking, for already deleted
entries, but looking at the code it appears that pendconn_free() already
resets s->pend_pos that's used as <p> there, and that the other call
reasons are after an error where the connection will be dropped as
well. So we don't save anything by doing this test, and make it
unsafe. The older code used to check for list emptiness there and
not inside pendconn_unlink(), which explains why the code has stayed
there. Let's just remove this now.

Thanks to @jaroslawr for reporting this issue in great details and for
testing the proposed fix.

This should be backpored to 1.8, where the test on LIST_ISEMPTY should
be moved to pendconn_unlink() instead (inside the lock, just like 2.0+).
2020-10-02 18:10:26 +02:00
William Lallemand
f9ff3ece83 DOC: ssl: new "cert bundle" behavior
Update the documentation with the new bundle behavior which does not use
the same OpenSSL certificate store anymore but loads the PEM separately
as multiple "crt" were specified.

It should fix issue #872.
2020-10-02 18:02:16 +02:00
Christopher Faulet
f7177271f3 BUG/MINOR: tcpcheck: Set socks4 and send-proxy flags before the connect call
Since the health-check refactoring in the 2.2, the checks through a socks4 proxy
are broken. To fix this bug, CO_FL_SOCKS4 flag must be set on the connection
before calling the connect() callback function because this flags is checked to
use the right destination address. The same is done for the CO_FL_SEND_PROXY
flag for a consistency purpose.

A reg-test has been added to test the "check-via-socks4" directive.

This patch must be backported to 2.2.
2020-10-02 17:14:34 +02:00
Christopher Faulet
2079a4ad36 MEDIUM: tcp-rules: Warn if a track-sc* content rule doesn't depend on content
The warning is only emitted for HTTP frontend. Idea is to encourage the usage of
"tcp-request session" rules to track counters that does not depend on the
request content. The documentation has been updated accordingly.

The warning is important because since the multiplexers were added in the
processing chain, the HTTP parsing is performed at a lower level. Thus parsing
errors are detected in the multiplexers, before the stream creation. In HTTP/2,
the error is reported by the multiplexer itself and the stream is never
created. This difference has a certain number of consequences, one of which is
that HTTP request counting in stick tables only works for valid H2 request, and
HTTP error tracking in stick tables never considers invalid H2 requests but only
invalid H1 ones. And the aim is to do the same with the mux-h1. This change will
not be done for the 2.3, but the 2.4. At the end, H1 and H2 parsing errors will
be caught by the multiplexers, at the session level. Thus, tracking counters at
the content level should be reserved for rules using a key based on the request
content or those using ACLs based on the request content.

To be clear, a warning will be emitted for the following rules :

  tcp-request content track-sc0 src
  tcp-request content track-sc0 src if ! { src 10.0.0.0/24 }
  tcp-request content track-sc0 src if { ssl_fc }

But not for the following ones :

  tcp-request content track-sc0 req.hdr(host)
  tcp-request content track-sc0 src if { req.hdr(host) -m found }
2020-10-02 15:50:26 +02:00
Christopher Faulet
7ea509e15f DOC: tcp-rules: Refresh details about L7 matching for tcp-request content rules
Because the parsing of HTTP message is now performed in the HTTP multiplexers,
the content is immediatly available when "tcp-request content" rules are
evaluated for an HTTP frontend. So, it is a good idea to make the documentation
explicit on this point. In addition, because in all cases, the parsing is
already performed, there is no reason to still use "tcp-request content" rules
based on L7 matching, although it is still valid. The recommended way is to use
"http-request" rules instead. Again, it is a good idea to update the
documentation on this point.
2020-10-02 15:50:26 +02:00
Eric Salama
7cea6065ac BUG/MINOR: Fix several leaks of 'log_tag' in init().
We use chunk_initstr() to store the program name as the default log-tag.

If we use the log-tag directive in the config file, this chunk will be
destroyed and replaced. chunk_initstr() sets the chunk size to 0 so we
will free the chunk itself, but not its content.

This happens for a global section and also for a proxy.

We fix this by using chunk_initlen() instead of chunk_initstr().
We also check that the memory allocation was successfull, otherwise we quit.

This fixes github issue #850.
It can be backported as far as 1.9, with minor adjustments to includes.
2020-10-02 15:50:26 +02:00
William Dauchy
1d0206e71f MINOR: ssl: remove uneeded check in crtlist_parse_file
this condition is never true as we either break or goto error, so those
two lines could be removed in the current state of the code.

this is fixing github issue #862

Signed-off-by: William Dauchy <w.dauchy@criteo.com>
2020-10-02 15:43:01 +02:00
Tim Duesterhus
b9f6accc9e MINOR: ssl: Add error if a crt-list might be truncated
Similar to warning during the parsing of the regular configuration file
that was added in 2fd5bdb439 this patch adds
a warning to the parsing of a crt-list if the file does not end in a
newline (and thus might have been truncated).

The logic essentially just was copied over. It might be good to refactor
this in the future, allowing easy re-use within all line-based config
parsers.

see https://github.com/haproxy/haproxy/issues/860#issuecomment-693422936
see 0354b658f0

This should be backported as a warning to 2.2.
2020-10-02 12:29:03 +02:00
Tim Duesterhus
6d07fae3c0 CLEANUP: ssl: Use structured format for error line report during crt-list parsing
This reuses the known `parsing [%s:%d]:` from regular config file error
reporting.
2020-10-02 12:29:03 +02:00
Brad Smith
7c503bb459 BUILD: makefile: Enable closefrom() support on Solaris
Solaris 9 (released 2002) added support for closefrom().

I bumped the version in the comment to 10 as the default feature
flags already has event ports enabled which were introduced in
Solaris 10.
2020-10-02 08:32:33 +02:00
Willy Tarreau
fe2cc41151 BUILD: tools: fix minor build issue on isspace()
Previous commit fa41cb679 ("MINOR: tools: support for word expansion
of environment in parse_line") introduced two new isspace() on a char
and broke the build on systems using an array disguised in a macro
instead of a function (like cygwin). Just use the usual cast.
2020-10-01 18:05:48 +02:00