Commit Graph

598 Commits

Author SHA1 Message Date
Miroslav Zagorac
70230c6484 CONTRIB: opentracing: add the OpenTracing filter
This commit adds the OpenTracing filter (hereinafter we will use the
abbreviated name 'the OT filter') to the contrib tree.

The OT filter adds native support for using distributed tracing in HAProxy.
This is enabled by sending an OpenTracing compliant request to one of the
supported tracers; such as Datadog, Jaeger, Lightstep and Zipkin tracers.
Please note: tracers are not listed by any preference, but alphabetically.

The OT filter is a standard HAProxy filter, so what applies to others also
applies to this one (of course, by that I mean what is described in the
documentation, more precisely in the doc/internals/filters.txt file).

The OT filter activation is done explicitly by specifying it in the HAProxy
configuration.  If this is not done, the OT filter in no way participates
in the work of HAProxy.

As for the impact on HAProxy speed, this is documented with several tests
located in the test directory, and the result is found in the README-speed-*
files.  In short, the speed of operation depends on the way it is used and
the complexity of the configuration, from an almost immeasurable impact to
a significant deceleration (5x and more).  I think that in some normal use
the speed of HAProxy with the filter on will be quite satisfactory with a
slowdown of less than 4%.

The OT filter allows intensive use of ACLs, which can be defined anywhere in
the configuration.  Thus, it is possible to use the filter only for those
connections that are of interest to us.

More detailed documentation related to the operation, configuration and use
of the filter can be found in the contrib/opentracing directory.

To make the OpenTracing filter easier to configure and compile, several
entries have been added to the Makefile.  When running the make utility,
it is possible to use several new arguments:

  USE_OT=1     : enable the OpenTracing filter
  OT_DEBUG=1   : compile the OpenTracing filter in debug mode
  OT_INC=path  : force the include path to libopentracing-c-wrapper
  OT_LIB=path  : force the lib path to libopentracing-c-wrapper
  OT_RUNPATH=1 : add libopentracing-c-wrapper RUNPATH to haproxy executable

If USE_OT is set, then an additional Makefile from the contrib/opentracing
directory is included in the compilation process.
2020-12-16 15:49:53 +01:00
Willy Tarreau
da867d8d68 BUILD: Makefile: have "make clean" destroy .o/.a/.s in contrib subdirs as well
Now that we sometimes link some contrib subparts directly into the
haproxy binary, it's becoming a real problem that they're not cleaned
on make clean.  Some of the tools there are useful as .so or pure
binaries and we don't want to remove them, but anything intermediary
susceptible to be linked into haproxy should be clenaed. This is what
this patch does for 3 levels of subdirs into contrib/, without touching
the rest. It should be sufficient for the vast majority of use cases.
2020-12-16 14:19:44 +01:00
Tim Duesterhus
c8d19702f4 BUILD: Show the value of DEBUG= in haproxy -vv
Previously this was not visible after building.
2020-11-21 18:27:33 +01:00
Tim Duesterhus
81e948e051 BUILD: Make DEBUG part of .build_opts
This forces a recompilation if the value of DEBUG= changes.
2020-11-21 18:27:33 +01:00
Matthieu Guegan
496374e592 BUILD: makefile: enable crypt(3) for OpenBSD
Allow OpenBSD to support encrypted passwords in Userlists.

OpenBSD's crypt(3) function is provided directly by libc and does not
require -lcrypt.
Signed-off-by: Matthieu Guegan <matthieu.guegan@deindeal.ch>
2020-11-21 05:45:05 +01:00
Baptiste Assmann
e279ca6bbe MINOR: sample: Add converts to parses MQTT messages
This patch implements a couple of converters to validate and extract data from a
MQTT (Message Queuing Telemetry Transport) message. The validation consists of a
few checks as well as "packet size" validation. The extraction can get any field
from the variable header and the payload.

This is limited to CONNECT and CONNACK packet types only. All other messages are
considered as invalid. It is not a problem for now because only the first packet
on each side can be parsed (CONNECT for the client and CONNACK for the server).

MQTT 3.1.1 and 5.0 are supported.

Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
2020-11-05 19:27:03 +01:00
Baptiste Assmann
e138dda1e0 MINOR: sample: Add converters to parse FIX messages
This patch implements a couple of converters to validate and extract tag value
from a FIX (Financial Information eXchange) message. The validation consists in
a few checks such as mandatory fields and checksum computation. The extraction
can get any tag value based on a tag string or tag id.

This patch requires the istend() function. Thus it depends on "MINOR: ist: Add
istend() function to return a pointer to the end of the string".

Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
2020-11-05 19:26:30 +01:00
Willy Tarreau
2404e37838 BUILD: makefile: usual reorder of objects for faster builds
Reordered the objets by reverse build times made the total build time
go down from 17.7s to 17.2s at -O2 using make -j8 on my PC, and from
~3.2 to ~2.7s on the build farm.
2020-11-05 16:46:24 +01:00
Willy Tarreau
5c643f37d0 BUILD: makefile: add entries to build common debugging tools
A few tools in contrib/ such as halog, flags, poll and tcploop are
occasionally useful at least to developers, and some of them such as
halog or flags can occasionally break due to some changes in the include
files. As reported in issue #907, their inability to inherit the global
build options also causes some warnings related to some specificities
of the main include files. Let's just add entries in the main makefile
to build them.
2020-10-22 05:17:08 +02:00
Brad Smith
ad5afbafea BUILD: makefile: Enable getaddrinfo() on OS/X
Enable getaddrinfo() on OS/X.
2020-10-10 10:09:29 +02:00
Brad Smith
0fdfe4179e BUILD: makefile: Update feature flags for NetBSD
This updates the feature flags for NetBSD.

NetBSD 8 adds support for accept4().

Enable getaddrinfo().
2020-10-09 09:53:56 +02:00
Willy Tarreau
4a539f343a BUG/MINOR: makefile: fix a tiny typo in the target list
Previous commit 382001b46 ("BUILD: Add a DragonFlyBSD target") introduced
a tiny typo in the target list ("iopenbs" vs "openbsd"). This will have to
be backported if that patch is backported.
2020-10-09 05:58:40 +02:00
Brad Smith
382001b46b BUILD: Add a DragonFlyBSD target
Add a target for DragonFlyBSD 4.3 and above.
2020-10-08 20:54:18 +02:00
Brad Smith
7c503bb459 BUILD: makefile: Enable closefrom() support on Solaris
Solaris 9 (released 2002) added support for closefrom().

I bumped the version in the comment to 10 as the default feature
flags already has event ports enabled which were introduced in
Solaris 10.
2020-10-02 08:32:33 +02:00
Willy Tarreau
ac32b4b98c BUILD: makefile: add an EXTRAVERSION variable to ease local naming
Sometimes it's desirable to append local version naming to packages,
and currently it can only be done using SUBVERS which is already set
by default to the git commit ID and patch count since last known tag,
making the addition a bit complicated.

Let's just add a new EXTRAVERSION field that is empty by default, and
systematically appended verbatim to the version string everywhere. This
way it becomes trivial to append some local strings, such as:

   make TARGET=foo EXTRAVERSION=+$(quilt applied|wc -l)
   -> 2.3-dev5-5018aa-15+1

or :

   make TARGET=foo EXTRAVERSION=-$(date +%F)
   -> 2.3-dev5-5018aa-15-20200110

Let's be careful not to add double quotes (used as the string delimiter)
nor spaces (which can confuse version parsers on the output). The extra
version is also used to name a tarball. It's always pre-initialized to an
empty string so that it's not accidently inherited from the environment.
It's not reported in "make version" to avoid fooling tools (it would be
pointless anyway).

As a side effect it also becomes possible to force VERSION and SUBVERS
to an empty string and use EXTRAVERSION alone to force a specific version
(could possibly be useful when bisecting from patch queues outside of Git
for example).
2020-10-01 04:13:36 +02:00
Brad Smith
5018aacae5 BUILD: makefile: Fix building with closefrom() support enabled
I noticed the USE_CLOSEFROM define was not being passed along like the rest
during the build.

Looking around I see this was broken with the following two commits and related
series..

BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
http://git.haproxy.org/?p=haproxy.git;a=commit;h=05fd82da76d1bbc8d65d63ab246bda7cbcf8481a

BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
http://git.haproxy.org/?p=haproxy.git;a=commit;h=824cd00d3bda8f7f6d4c30baf77ba6c19ab47811

Looks like this should be back ported to 2.0, 2.1 and 2.2.
2020-09-30 16:06:51 +02:00
Brad Smith
9d85eb02d5 BUILD: makefile: Update feature flags for FreeBSD
This updates the feature flags for FreeBSD.

FreeBSD 10 adds support for accept4().

Enable getaddrinfo().

From the FreeBSD port / package.
2020-09-29 11:11:04 +02:00
Brad Smith
90c9d78447 BUILD: makefile: Update feature flags for OpenBSD
Update the OpenBSD target features being enabled.

I updated the list of features after noticing
"BUILD: makefile: disable threads by default on OpenBSD".

The Makefile utilizing gcc(1) by default resulted in utilizing
our legacy and obsolete compiler (GCC 4.2.1) instead of the
proper system compiler (Clang), which does support TLS. With
"BUILD: makefile: change default value of CC from gcc to cc"
that is resolved.
2020-09-27 21:48:18 +02:00
Brad Smith
94636436df BUILD: makefile: change default value of CC from gcc to cc
Change the default value of CC from gcc to cc to be more appropriate
for modern day mix of compilers. On GCC based OS's cc -> gcc. On Clang
based OS's cc -> clang. FreeBSD / OpenBSD have switched to Clang and
this corrects building with the proper compiler on OS's using Clang
as the default compiler. This especially matters for the necessity for
TLS on OpenBSD. I would expect this affects OpenMandriva and other
Linux OS's using Clang as well.
2020-09-15 07:32:06 +02:00
Ilya Shipitsin
4a034f2212 BUILD: introduce possibility to define ABORT_NOW() conditionally
code analysis tools recognize abort() better, so let us introduce
such possibility
2020-09-12 13:11:27 +02:00
Willy Tarreau
0d06df6448 MINOR: sock: introduce sock_inet and sock_unix
These files will regroup everything specific to AF_INET, AF_INET6 and
AF_UNIX socket definitions and address management. Some code there might
be agnostic to the socket type and could later move to af_xxxx.c but for
now we only support regular sockets so no need to go too far.

The files are quite poor at this step, they only contain the address
comparison function for each address family.
2020-08-28 18:51:36 +02:00
Willy Tarreau
18b7df7a2b REORG: sock: start to move some generic socket code to sock.c
The new file sock.c will contain generic code for standard sockets
relying on file descriptors. We currently have way too much duplication
between proto_uxst, proto_tcp, proto_sockpair and proto_udp.

For now only get_src, get_dst and sock_create_server_socket were moved,
and are used where appropriate.
2020-08-28 18:51:36 +02:00
Willy Tarreau
1318034317 REORG: unix: move UNIX bind/server keywords from proto_uxst.c to cfgparse-unix.c
Let's finish the cleanup and get rid of all bind and server keywords
parsers from proto_uxst.c. They're now moved to cfgparse-unix.c. Now
proto_uxst.c is clean and only contains code related to binding and
connecting.
2020-08-28 18:51:36 +02:00
Willy Tarreau
de70ca5dfd REORG: tcp: move TCP bind/server keywords from proto_tcp.c to cfgparse-tcp.c
Let's continue the cleanup and get rid of all bind and server keywords
parsers from proto_tcp.c. They're now moved to cfgparse-tcp.c, just as
was done for ssl before 2.2 release. Nothing has changed beyond this.
Now proto_tcp.c is clean and only contains code related to binding and
connecting.
2020-08-28 18:51:36 +02:00
Willy Tarreau
8987e7a8c9 REORG: tcp: move TCP sample fetches from proto_tcp.c to tcp_sample.c
Let's continue the cleanup and get rid of all sample fetch functions
from proto_tcp.c. They're now moved to tcp_sample.c, just as was done
for ssl before 2.2 release. Nothing has changed beyond this.
2020-08-28 18:51:36 +02:00
Willy Tarreau
aeae66cf22 REORG: tcp: move TCP actions from proto_tcp.c to tcp_act.c
The file proto_tcp.c has become a real mess because it still contains
tons of definitions that have nothing to do with the TCP protocol setup.
This commit moves the ruleset actions "set-src-port", "set-dst-port",
"set-src", "set-dst", and "silent-drop" to a new file "tcp_act.c".
Nothing has changed beyond this.
2020-08-28 18:51:36 +02:00
Willy Tarreau
b36dcc66ed BUILD: makefile: don't disable -Wstringop-overflow anymore
This basically reverts commit c4e6460f6 ("MINOR: build: Disable
-Wstringop-overflow.") which is no more needed after previous one.
2020-08-11 10:31:18 +02:00
Ilya Shipitsin
f21023e1cf BUILD: Makefile: require SSL_LIB, SSL_INC to be explicitly set
The SSL_INC and SSL_LIB variables were not initialized in the Makefile,
so they could be accidently inherited from the environment. We require
that any makefile variable is explicitly set on the command line so they
must be initialized.

Note that the Travis scripts used to rely only on these variables to be
exported, so it was adjusted as well.
2020-08-05 11:37:32 +02:00
Emeric Brun
3835c0dcb5 MEDIUM: udp: adds minimal proto udp support for message listeners.
This patch introduce proto_udp.c targeting a further support of
log forwarding feature.

This code was originally produced by Frederic Lecaille working on
QUIC support and only minimal requirements for syslog support
have been merged.
2020-07-15 17:50:12 +02:00
Willy Tarreau
4c442b08b3 CLEANUP: makefile: update the outdated list of DEBUG_xxx options
A few options didn't exist anymore (FSM, HASH) and quite a few ones were
added since last update (MEM_STATS, DONT_SHARE_POOLS, NO_LOCKLESS_POOLS,
NO_LOCAL_POOLS, FAIL_ALLOC, STRICT_NOCRASH, HPACK.
2020-07-04 12:43:46 +02:00
Willy Tarreau
45e83e8c56 BUILD: makefile: disable threads by default on OpenBSD
As reported by Ilya in issue #725, building with threads on OpenBSD
is broken with gcc:

  include/haproxy/tinfo.h:30: error: thread-local storage not supported for this target

Better stay safe and disable it. Clang seems to support (or emulate)
thread-local, at least it builds. Those willing to experiment can
easily pass USE_THREAD=1.
2020-07-03 18:56:33 +02:00
Willy Tarreau
739879a23b REORG: buffer: rename buffer.c to dynbuf.c
The include part was renamed by commit 2741c8c but I somehow missed
the renaming of the C file, whose name didn't match the H file anymore.
2020-06-29 09:26:59 +02:00
Willy Tarreau
38e8a1c7b8 MINOR: debug: add a new DEBUG_FD build option
When DEBUG_FD is set at build time, we'll keep a counter of per-FD events
in the fdtab. This counter is reported in "show fd" even for closed FDs if
not zero. The purpose is to help spot situations where an apparently closed
FD continues to be reported in loops, or where some events are dismissed.
2020-06-23 10:04:54 +02:00
Tim Duesterhus
588b3148d9 BUILD: Re-enable -Wimplicit-fallthrough
Getting rid of this warning is cleaner solved using a 'fall through' comment,
because it clarifies intent to a human reader.

This patch adjust a few places that cause -Wimplicit-fallthrough to trigger:

- Fix typos in the comment.
- Remove redundant 'no break' that trips up gcc from comment.
- Move the comment out of the block when the 'case' is completely surrounded
  by braces.
- Add comments where I could determine that the fall through was intentional.

Changes tested on

    gcc (Debian 9.3.0-13) 9.3.0
    Copyright (C) 2019 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

using

    make -j4 all TARGET=linux-glibc USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1
2020-06-11 16:49:37 +02:00
Tim Duesterhus
4989de2811 BUILD: Remove nowarn for warnings that do not trigger
Tested with

    make -j4 all TARGET=linux-glibc USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1

against

    gcc (Debian 9.3.0-13) 9.3.0
    Copyright (C) 2019 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
2020-06-11 15:08:27 +02:00
Willy Tarreau
7c18b54106 REORG: dgram: rename proto_udp to dgram
The set of files proto_udp.{c,h} were misleadingly named, as they do not
provide anything related to the UDP protocol but to datagram handling
instead, since currently all UDP processing is hard-coded where it's used
(dns, logs). They are to UDP what connection.{c,h} are to proto_tcp. This
was causing confusion about how to insert UDP socket management code,
so let's rename them right now to dgram.{c,h} which more accurately
matches what's inside since every function and type is already prefixed
with "dgram_".
2020-06-11 10:18:59 +02:00
Willy Tarreau
920214e8c4 BUILD: reorder objects in the Makefile for faster builds
Splitting large files and changing includes has changed the per-file
build time. After a careful reordering based on build time, we're now
down to 5.8s at -O0 on the PC at -j8 and 2.4-2.6s on the farm at -j120.
Some room for at least one file name was left on each line to ease
future additions.
2020-06-11 10:18:59 +02:00
Willy Tarreau
bcc6733fab REORG: check: extract the external checks from check.{c,h}
The health check code is ugly enough, let's take the external checks
out of it to simplify the code and shrink the file a little bit.
2020-06-11 10:18:58 +02:00
Willy Tarreau
51cd5956ee REORG: check: move tcpchecks away from check.c
Checks.c remains one of the largest file of the project and it contains
too many things. The tcpchecks code represents half of this file, and
both parts are relatively isolated, so let's move it away into its own
file. We now have tcpcheck.c, tcpcheck{,-t}.h.

Doing so required to export quite a number of functions because check.c
has almost everything made static, which really doesn't help to split!
2020-06-11 10:18:58 +02:00
Willy Tarreau
4aa573da6f REORG: include: move checks.h to haproxy/check{,-t}.h
All includes that were not absolutely necessary were removed because
checks.h happens to very often be part of dependency loops. A warning
was added about this in check-t.h. The fields, enums and structs were
a bit tidied because it's particularly tedious to find anything there.
It would make sense to split this in two or more files (at least
extract tcp-checks).

The file was renamed to the singular because it was one of the rare
exceptions to have an "s" appended to its name compared to the struct
name.
2020-06-11 10:18:58 +02:00
Willy Tarreau
48fbcae07c REORG: tools: split common/standard.h into haproxy/tools{,-t}.h
And also rename standard.c to tools.c. The original split between
tools.h and standard.h dates from version 1.3-dev and was mostly an
accident. This patch moves the files back to what they were expected
to be, and takes care of not changing anything else. However this
time tools.h was split between functions and types, because it contains
a small number of commonly used macros and structures (e.g. name_desc)
which in turn cause the massive list of includes of tools.h to conflict
with the callers.

They remain the ugliest files of the whole project and definitely need
to be cleaned and split apart. A few types are defined there only for
functions provided there, and some parts are even OS-specific and should
move somewhere else, such as the symbol resolution code.
2020-06-11 10:18:57 +02:00
Willy Tarreau
d0ef439699 REORG: include: move common/memory.h to haproxy/pool.h
Now the file is ready to be stored into its final destination. A few
minor reorderings were performed to keep the file properly organized,
making the various sections more visible (cache & lockless).

In addition and to stay consistent, memory.c was renamed to pool.c.
2020-06-11 10:18:57 +02:00
Willy Tarreau
3f567e4949 REORG: include: split hathreads into haproxy/thread.h and haproxy/thread-t.h
This splits the hathreads.h file into types+macros and functions. Given
that most users of this file used to include it only to get the definition
of THREAD_LOCAL and MAXTHREADS, the bare minimum was placed into thread-t.h
(i.e. types and macros).

All the thread management was left to haproxy/thread.h. It's worth noting
the drop of the trailing "s" in the name, to remove the permanent confusion
that arises between this one and the system implementation (no "s") and the
makefile's option (no "s").

For consistency, src/hathreads.c was also renamed thread.c.

A number of files were updated to only include thread-t which is the one
they really needed.

Some future improvements are possible like replacing empty inlined
functions with macros for the thread-less case, as building at -O0 disables
inlining and causes these ones to be emitted. But this really is cosmetic.
2020-06-11 10:18:56 +02:00
Willy Tarreau
fc80e30217 REORG: ebtree: clean up remains of the ebtree/ directory
The only leftovers were the unused compiler.h file and the LICENSE file
which is already mentioned in each and every ebtree file header.

A few build paths were updated in the contrib/ directory not to mention
this directory anymore, and all its occurrences were dropped from the
main makefile. From now on no other include path but include/ will be
needed anymore to build any file.
2020-06-11 09:31:11 +02:00
Willy Tarreau
ff0e8a44a4 REORG: ebtree: move the C files from ebtree/ to src/
As part of the include files cleanup, we're going to kill the ebtree
directory. For this we need to host its C files in a different location
and src/ is the right one.
2020-06-11 09:31:11 +02:00
William Lallemand
6a66a5ec9b REORG: ssl: move utility functions to src/ssl_utils.c
These functions are mainly used to extract information from
certificates.
2020-05-15 14:11:54 +02:00
William Lallemand
15e169447d REORG: ssl: move sample fetches to src/ssl_sample.c
Move all SSL sample fetches to src/ssl_sample.c.
2020-05-15 14:11:54 +02:00
William Lallemand
dad3105157 REORG: ssl: move ssl configuration to cfgparse-ssl.c
Move all the configuration parsing of the ssl keywords in cfgparse-ssl.c
2020-05-15 14:11:54 +02:00
William Lallemand
03c331c80a REORG: ssl: move the ckch_store related functions to src/ssl_ckch.c
Move the cert_key_and_chain functions:

int ssl_sock_load_files_into_ckch(const char *path, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_pem_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch , char **err);
void ssl_sock_free_cert_key_and_chain_contents(struct cert_key_and_chain *ckch);

int ssl_sock_load_key_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch , char **err);
int ssl_sock_load_ocsp_response_from_file(const char *ocsp_path, char *buf, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_sctl_from_file(const char *sctl_path, char *buf, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_issuer_file_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch, char **err);

And the utility ckch_store functions:

void ckch_store_free(struct ckch_store *store)
struct ckch_store *ckch_store_new(const char *filename, int nmemb)
struct ckch_store *ckchs_dup(const struct ckch_store *src)
ckch_store *ckchs_lookup(char *path)
ckch_store *ckchs_load_cert_file(char *path, int multi, char **err)
2020-05-15 14:11:54 +02:00
William Lallemand
6e9556b635 REORG: ssl: move crtlist functions to src/ssl_crtlist.c
Move the crtlist functions to src/ssl_crtlist.c and their definitions to
proto/ssl_crtlist.h.

The following functions were moved:

/* crt-list entry functions */
void ssl_sock_free_ssl_conf(struct ssl_bind_conf *conf);
char **crtlist_dup_filters(char **args, int fcount);
void crtlist_free_filters(char **args);
void crtlist_entry_free(struct crtlist_entry *entry);
struct crtlist_entry *crtlist_entry_new();

/* crt-list functions */
void crtlist_free(struct crtlist *crtlist);
struct crtlist *crtlist_new(const char *filename, int unique);

/* file loading */
int crtlist_parse_line(char *line, char **crt_path, struct crtlist_entry *entry, const char *file, int linenum, char **err);
int crtlist_parse_file(char *file, struct bind_conf *bind_conf, struct proxy *curproxy, struct crtlist **crtlist, char **err);
int crtlist_load_cert_dir(char *path, struct bind_conf *bind_conf, struct crtlist **crtlist, char **err);
2020-05-15 14:11:54 +02:00
Willy Tarreau
39b2fda915 BUILD: Makefile: add linux-musl to TARGET
Other users are using musl, namely on Docker. It builds fine with
linux-glibc-legacy but not linux-glibc, which needs to first disable
USE_BACKTRACE. Better add a valid entry for it instead of hacking
around another libc.
2020-04-16 15:17:13 +02:00
Willy Tarreau
48e8603a9c BUILD: makefile: fix expression again to detect ARM platform
I messed up the fix in 67b095e ("BUILD: makefile: fix regex syntax in
ARM platform detection"), I tried it by hand in the shell without "-v"
but left it in the expression. It works on ARM because it only finds
lines starting with '#' but on other platforms it insists for -latomic.
2020-03-18 08:21:57 +01:00
Willy Tarreau
67b095e797 BUILD: makefile: fix regex syntax in ARM platform detection
Commit d93e6ec ("BUILD: on ARM, must be linked to libatomic.") broke the
build due to a missing backslash in front of the '#':

  Makefile:331: *** invalid syntax in conditional.  Stop.

Let's address this and make sure we only pick relevant lines (and not
possibly empty lines).
2020-03-16 09:38:00 +01:00
David Carlier
d93e6ec4cc BUILD: on ARM, must be linked to libatomic.
For load/store operations, needs to be linked to.
tested on raspberry.
2020-03-16 08:33:17 +01:00
Willy Tarreau
4b61e694b4 BUILD: Makefile: the compiler-specific flags should all be in SPEC_CFLAGS
We used to have -Wall -Wextra -Werror in COPTS which are flags fed by
the various USE_* options, and all other warnings in SPEC_CFLAGS. This
makes it impossible to remove these -W* entries (typically -Wextra).

Let's move these 3 flags into SPEC_CFLAGS where they should have been.
Now it's possible to override SPEC_CFLAGS to match any compiler's
specificities, or to clear all warnings at once, or to replace them
all with "-w" to silence warnings.
2020-03-10 09:42:30 +01:00
Willy Tarreau
06c63aec95 CLEANUP: remove support for USE_MY_SPLICE
The splice() syscall has been supported in glibc since version 2.5 issued
in 2006 and is present on supported systems so there's no need for having
our own arch-specific syscall definitions anymore.
2020-03-10 07:23:41 +01:00
Willy Tarreau
3858b122a6 CLEANUP: remove support for USE_MY_EPOLL
This was made to support epoll on patched 2.4 kernels, and on early 2.6
using alternative libcs thanks to the arch-specific syscall definitions.
All the features we support have been around since 2.6.2 and present in
glibc since 2.3.2, neither of which are found in field anymore. Let's
simply drop this and use epoll normally.
2020-03-10 07:08:10 +01:00
Willy Tarreau
618ac6ea52 CLEANUP: drop support for USE_MY_ACCEPT4
The accept4() syscall has been present for a while now, there is no more
reason for maintaining our own arch-specific syscall implementation for
systems lacking it in libc but having it in the kernel.
2020-03-10 07:02:46 +01:00
Willy Tarreau
c3e926bf3b CLEANUP: remove support for Linux i686 vsyscalls
This was introduced 10 years ago to squeeze a few CPU cycles per syscall
on 32-bit x86 machines and was already quite old by then, requiring to
explicitly enable support for this in the kernel. We don't even know if
it still builds, let alone if it works at all on recent kernels! Let's
completely drop this now.
2020-03-10 06:55:52 +01:00
Ilya Shipitsin
2a950d02a9 DOC: assorted typo fixes in the documentation and Makefile
This is another round of cleanups in various docs and comments in the
Makefile.
2020-03-06 10:49:55 +01:00
Willy Tarreau
6f6d96de19 BUILD: makefile: do not modify the build options during make reg-tests
I'm quite fed up with having to rebuild everything from scratch after each
and every "make reg-tests", especially during bisects. The only reason for
this is that there are no build options passed to make for reg-tests, which
modifies the .build_opts file again, resulting in a change upon next build.
Let's just keep this file out of the dependency check for make reg-tests.
2020-03-04 19:29:12 +01:00
Willy Tarreau
c0bbdc196d BUILD: Makefile: include librt before libpthread
Statically building on for i386/x86_64 on linux+glibc 2.18 fails in rt with
undefined references to pthread_attr_init and a few others. Let's just swap
the two libs in order to fix this.
2020-03-04 12:02:27 +01:00
Willy Tarreau
f5b4e064dc MEDIUM: debug: add support for dumping backtraces of stuck threads
When a panic() occurs due to a stuck thread, we'll try to dump a
backtrace of this thread if the config directive USE_BACKTRACE is
set (which is the case on linux+glibc). For this we use the
backtrace() call provided by glibc and iterate the pointers through
resolve_sym_name(). In order to minimize the output (which is limited
to one buffer), we only do this for stuck threads, and we start the
dump above ha_panic()/ha_thread_dump_all_to_trash(), and stop when
meeting known points such as main/run_tasks_from_list/run_poll_loop.

If enabled without USE_DL, the dump will be complete with no details
except that pointers will all be given relative to main, which is
still better than nothing.

The new USE_BACKTRACE config option is enabled by default on glibc since
it has been present for ages. When it is set, the export-dynamic linker
option is enabled so that all non-static symbols are properly resolved.
2020-03-03 18:40:03 +01:00
Willy Tarreau
d1c847abbc BUILD: makefile: re-enable strict aliasing
For a very long time we've used to build without strict aliasing due to
very few places in the stick-tables code mostly, that initially we didn't
know how to deal with. The problem of doing this is that it encourages
to write possibly incorrect code such as the few SSL sample fetch functions
that were recently fixed.

All places causing aliasing errors on x86_64, i586, armv8, armv7 and
mips were fixed so it's about time to re-enable the warning hoping to
catch such errors early in the development cycle. As a bonus, this
removed about 5kB of code.
2020-02-25 10:24:51 +01:00
Willy Tarreau
03e7853581 BUILD: remove obsolete support for -mregparm / USE_REGPARM
This used to be a minor optimization on ix86 where registers are scarce
and the calling convention not very efficient, but this platform is not
relevant enough anymore to warrant all this dirt in the code for the sake
of saving 1 or 2% of performance. Modern platforms don't use this at all
since their calling convention already defaults to using several registers
so better get rid of this once for all.
2020-02-25 07:41:47 +01:00
Christian Lachner
c13223022c MINOR: build: add aix72-gcc build TARGET and power{8,9} CPUs
As haproxy wont build on AIX 7.2 using the old "aix52" TARGET a new
TARGET was introduced which adds two special CFLAGS to prevent the
loading of AIXs xmem.h and var.h. This is done by defining the
corresponding include-guards _H_XMEM and _H_VAR. Without excluding
those headers-files the build fails because of redefinition errors:

1)
  CC      src/mux_fcgi.o
In file included from /usr/include/sys/uio.h:90,
                 from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8.3.0/include-fixed/sys/socket.h:104,
                 from include/common/compat.h:32,
                 from include/common/cfgparse.h:25,
                 from src/mux_fcgi.c:13:
src/mux_fcgi.c:204:13: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
  struct ist rem_addr;
             ^~~~~~~~

2)
  CC      src/cfgparse-listen.o
In file included from include/types/arg.h:31,
                 from include/types/acl.h:29,
                 from include/types/proxy.h:41,
                 from include/proto/log.h:34,
                 from include/common/cfgparse.h:30,
                 from src/mux_h2.c:13:
include/types/vars.h:30:8: error: redefinition of 'struct var'
 struct var {
        ^~~

Futhermore, to enable multithreading via USE_THREAD, the atomic
library was added to the LDFLAGS. Finally, two new CPUs were added
to simplify the usage of power8 and power9 optimizations.

This TARGET was only tested on GCC 8.3 and may or may not work on
IBM's native C-compiler (XLC).

Should be backported to 2.1.
2020-02-12 15:37:13 +01:00
Tim Duesterhus
541fe1ec52 MINOR: lua: Add HLUA_PREPEND_C?PATH build option
This complements the lua-prepend-path configuration option to allow
distro maintainers to add a default path for HAProxy specific Lua
libraries.
2020-01-24 09:22:03 +01:00
Willy Tarreau
ef915dc9ca BUILD: reorder the objects in the makefile
After a number of reorganization, addition of fcgi and the removal of
the legacy mode, some late files ended up being slow to build and were
slowing down the parallel build. Let's reorder them based on the build
time. Full build went down from 8.3-9.2s to 6.8s.
2019-11-25 19:47:23 +01:00
Willy Tarreau
c1b16734c0 BUILD: do not disable -Wformat-truncation anymore
There were very few entries to fix and this warning, while often
wrong, can actually spot future issues. If it can help developers
adjust their code in the future to make it more robust, it's not
necessarily that bad. Let's re-enable it and see how it goes.
2019-10-29 10:54:24 +01:00
David Carlier
5e4c8e2a67 BUILD/MEDIUM: threads: enable cpu_affinity on osx
Enable it but on a per thread basis only using Darwin native API.
2019-10-17 07:20:58 +02:00
David Carlier
7fa6fdf86a BUILD/SMALL: threads: enable threads on osx
Enable multi thread on Darwin platform too.
2019-10-17 07:18:00 +02:00
Willy Tarreau
c77c2d4f9c BUILD: makefile: work around yet another GCC fantasy (-Wstring-plus-int)
According to issue #294 some gcc versions suspect that developers are
having trouble dealing with string offsets and now emit another new
childish warning when mapping indexes to characters. Instead of annoying
developers each time it happens and ask them to modify their valid code,
let's just get rid of this absurd warning.
2019-09-24 09:43:34 +02:00
Christopher Faulet
99eff65f4f MEDIUM: mux-fcgi: Add the FCGI multiplexer
This multiplexer is only available on the backend side. It may handle
multiplexed connections if the FCGI application supports it. A FCGI application
must be configured on the backend to be used. If not redefined during the
request processing by the FCGI filter, this mux handles all mandatory
parameters.

There is a limitation on the way the requests are processed. The parameters must
be encoded into a uniq PARAMS record. It means, once encoded, all HTTP headers
and FCGI parameters must small enough to be store in a buffer. Otherwise, an
internal processing error is returned.
2019-09-17 10:18:54 +02:00
Christopher Faulet
78fbb9f991 MEDIUM: fcgi-app: Add FCGI application and filter
The FCGI application handles all the configuration parameters used to format
requests sent to an application. The configuration of an application is grouped
in a dedicated section (fcgi-app <name>) and referenced in a backend to be used
(use-fcgi-app <name>). To be valid, a FCGI application must at least define a
document root. But it is also possible to set the default index, a regex to
split the script name and the path-info from the request URI, parameters to set
or unset...  In addition, this patch also adds a FCGI filter, responsible for
all processing on a stream.
2019-09-17 10:18:54 +02:00
Christopher Faulet
63bbf284a1 MINOR: fcgi: Add code related to FCGI protocol
This code is independant and is only responsible to encode and decode part of
the FCGI protocol.
2019-09-17 10:18:54 +02:00
Christopher Faulet
4f0f88a9d0 MEDIUM: mux-h1/h1-htx: move HTX convertion of H1 messages in dedicated file
To avoid code duplication in the futur mux FCGI, functions parsing H1 messages
and converting them into HTX have been moved in the file h1_htx.c. Some
specific parts remain in the mux H1. But most of the parsing is now generic.
2019-09-17 10:18:54 +02:00
David Carlier
6c00eba63b BUILD/MINOR: auth: enabling for osx
macOS supports this but as part of libc.
Little typo fix while here.
2019-09-08 12:20:13 +02:00
Lukas Tribus
cc1eb1619f MINOR: build: add linux-glibc-legacy build TARGET
As discussed in issue #128, introduce a new build TARGET
linux-glibc-legacy to allow the build on old, legacy OS.

Should be backported to 2.0.
2019-09-01 17:28:10 +02:00
Willy Tarreau
172945fbad MINOR: ring: add a new mechanism for retrieving/storing ring data in buffers
Our circular buffers are well suited for being used as ring buffers for
not-so-structured data. The machanism here consists in making room in a
buffer before inserting a new record which is prefixed by its size, and
looking up next record based on the previous one's offset and size. We
can have up to 255 consumers watching for data (dump in progress, tail)
which guarantee that entrees are not recycled while they're being dumped.
The complete representation is described in the header file. For now only
ring_new(), ring_resize() and ring_free() are created.
2019-08-27 17:14:19 +02:00
Willy Tarreau
4151c753fc MINOR: trace: start to create a new trace subsystem
The principle of this subsystem will be to support taking live traces
at various places in the code with conditional triggers, filters, and
ability to lock on some elements. The traces will support typed events
and will be sent into sinks made of ring buffers, file descriptors or
remote servers.
2019-08-22 20:21:00 +02:00
Willy Tarreau
67b5a161b4 MINOR: sink: create definitions a minimal code for event sinks
The principle will be to be able to dispatch events to various destinations
called "sinks". This is already done in part in logs where log servers can
be either a UDP socket or a file descriptor. This will be needed with the
new trace subsystem where we may also want to add ring buffers. And it turns
out that all such destinations make sense at all places. Logs may need to be
sent to a TCP server via a ring buffer, or consulted from the CLI. Trace
events may need to be sent to stdout/stderr as well as to remote log servers.

This patch creates a new structure "sink" aiming at addressing these similar
needs. The goal is to merge together what is common to all of them, such as
the output format, the dropped events count, etc, and also keep separately
the target identification (network address, file descriptor). Provisions
were made to have a "waiter" on the sink. For a TCP log server it will be
the task to wake up after writing to the log buffer. For a ring buffer, it
could be the list of watchers on the CLI running a "tail" operation and
waiting for new events. A lock was also placed in the struct since many
operations will require some locking, including the FD ones. The output
formats covers those in use by logs and two extra ones prepending the ISO
time in front of the message (convenient for stdio/buffer).

For now only the generic infrastructure is present, no type-specific
output is implemented. There's the sink_write() function which prepares
and formats a message to be sent, trying hard to avoid copies and only
using pointer manipulation, where the type-specific code just has to be
added. Dropped messages are already counted (for now 100% drop). The
message is put into an iovec array as it will be trivial to use with
file descriptors and sockets.
2019-08-22 20:21:00 +02:00
Willy Tarreau
9eebd8a978 REORG: trace: rename trace.c to calltrace.c and mention it's not thread-safe
The function call tracing code is a quite old and was never ported to
support threads. It's not even sure whether it still works well, but
at least its presence creates confusion for future work so let's rename
it to calltrace.c and add a comment about its lack of thread-safety.
2019-08-22 20:21:00 +02:00
Christopher Faulet
fc9cfe4006 REORG: proto_htx: Move HTX analyzers & co to http_ana.{c,h} files
The old module proto_http does not exist anymore. All code dedicated to the HTTP
analysis is now grouped in the file proto_htx.c. So, to finish the polishing
after removing the legacy HTTP code, proto_htx.{c,h} files have been moved in
http_ana.{c,h} files.

In addition, all HTX analyzers and related functions prefixed with "htx_" have
been renamed to start with "http_" instead.
2019-07-19 09:24:12 +02:00
Christopher Faulet
711ed6ae4a MAJOR: http: Remove the HTTP legacy code
First of all, all legacy HTTP analyzers and all functions exclusively used by
them were removed. So the most of the functions in proto_http.{c,h} were
removed. Only functions to deal with the HTTP transaction have been kept. Then,
http_msg and hdr_idx modules were entirely removed. And finally the structure
http_msg was lightened of all its useless information about the legacy HTTP. The
structure hdr_ctx was also removed because unused now, just like unused states
in the enum h1_state. Note that the memory pool "hdr_idx" was removed and
"http_txn" is now smaller.
2019-07-19 09:24:12 +02:00
Willy Tarreau
3c55efb7dd BUILD: makefile: do not rely on shell substitutions to determine git version
Solaris's default shell doesn't support substitutions at the beginning or
end of variables, which are still used to determine the version based on
git. Since we added --abbrev=0 we don't need the last one. And using cut
it's trivial to replace the first one, actually simplifying the whole
expression.

This may be backported to all stable branches.
2019-06-22 08:28:32 +02:00
Willy Tarreau
e8d0719924 BUILD: makefile: adjust the sed expression of "make help" for solaris
Solaris's sed doesn't take the 'p' argument on the 's' command so
nothing is printed. Just passing ';p' fixes this without affecting
other implementations. Additionally, optional characters cannot be
matched using a question mark, which is always searched verbatim, so
the leading '#' wasn't stripped. Using \{0,1\} works fine everywhere
so let's use this instead.
2019-06-22 08:13:24 +02:00
Willy Tarreau
30a6f6402e BUILD: makefile: use :space: instead of digits to count commits
The 'tr' command on Solaris doesn't conform to POSIX and requires
brackets around ranges. So the sequence '0-9' is understood as the
3 characters '0', '-', and '9'. This causes tagged versions (those
with no commit after the last commit) to be numberred as an empty
string, resulting in an error being reported while computing the
version number.

All implementations support '[:space:]' to delete heading spaces,
so let's use this instead.

This may be backported to all stable versions.
2019-06-22 07:58:28 +02:00
Willy Tarreau
5fb76036a3 BUILD: makefile: do not use echo -n
On certain UNIXes (Solaris at least), echo -n displays "-n". Let's
simply re-arrange the "if" block in the help message not to use it.
2019-06-16 19:26:18 +02:00
Willy Tarreau
f8cc19a1fe BUILD: makefile: enable USE_RT on Solaris
It doesn't build on Ultra5 under Solaris 10 without this due to
clock_gettime() not being found.
2019-06-16 19:18:53 +02:00
Willy Tarreau
c31e5a5126 BUILD: makefile: make the obsolete target detection compatible with make-3.80
Older versions of GNU make do not support "else ifneq", let's split
this in two lines.
2019-06-16 17:53:39 +02:00
Willy Tarreau
364d6f529c BUILD: makefile: enable getaddrinfo on the linux-glibc target
getaddrinfo() has been available since glibc 2.3.3 or so and is generally
enabled by distro packagers. The main reason for not enabling it on Linux
in the past is that it was known broken on some libc alternatives. It's
the right moment to enable it by default with glibc.
2019-06-15 18:03:48 +02:00
Willy Tarreau
a26181e74b BUILD: makefile: enable TFO on linux platforms
TCP Fast Open is supported on all supported Linux kernels and on all
kernels shipped in supported distros, except the older 2.6.32 that
comes with RHEL6. However the option is harmless, will not prevent
from building and smoothly falls back even if forcefully enabled, so
it makes sense to enable it by default. It's still possible to pass
"USE_TFO=" to force it disabled if really desired.
2019-06-15 18:03:48 +02:00
Willy Tarreau
20e6cedc43 BUILD: makefile: enable linux namespaces by default on linux
Oldest kernel found on a supported Linux distro (2.6.32 + backports on
RHEL6) supports network namespaces, so we have no reason not to enable
them by default on the linux-glibc target.
2019-06-15 18:03:48 +02:00
Willy Tarreau
99e49e93f8 BUILD: makefile: detect and reject recently removed linux targets
We've just removed old linux targets "linux22", "linux24", "linux24e",
"linux26" and "linux2628" and it's likely that many build scripts and
packages will still reference these. So let's have the makefile detect
these and reject with instructions instead of silently building with
incorrect options.
2019-06-15 18:03:48 +02:00
Willy Tarreau
61cf0dc9a0 BUILD: makefile: rename "linux2628" to "linux-glibc" and remove older targets
The linux targets have become more than confusing over time. We used to
have "linux2628" to match the features available in kernels 2.6.28 and
above, without consideration for the libc, and due to many new features
appearing later in kernels, some other options were added that are not
enabled by default in linux2628, so this target doesn't make any sense
anymore. The older ones (linux 2.2, linux 2.4, ...) do not make sense
either since these versions are not supported anymore. Let's clean things
up by creating a new "linux-glibc" target that matches what is available
by default on Linux kernels and glibc present on supported distros at the
time of release. Other libc implementation may use a custom or generic
target or be added later if needed.

All the older linux targets were removed.
2019-06-15 17:41:15 +02:00
Willy Tarreau
648fa9deb4 BUILD: makefile: further clarify the "help" output and list targets
When a target is not set we now also list the known ones. A minor
alignment issue in the output was also addressed.
2019-06-15 17:40:40 +02:00
Willy Tarreau
50b1aaeffc BUILD: makefile: clarify the "help" output and list options
The list of enable and disabled build options now appears separately
at the end of "make help". This is convenient to know what is enabled
by default on a given target. For example :

  $ make help TARGET=linux2628
  Enabled features for TARGET 'linux2628' (disable with 'USE_xxx=') :
    EPOLL NETFILTER POLL THREAD TPROXY LINUX_TPROXY LINUX_SPLICE LIBCRYPT
    CRYPT_H FUTEX ACCEPT4 CPU_AFFINITY DL RT PRCTL THREAD_DUMP

  Disabled features for TARGET 'linux2628' (enable with 'USE_xxx=1') :
    KQUEUE MY_EPOLL MY_SPLICE PCRE PCRE_JIT PCRE2 PCRE2_JIT PRIVATE_CACHE
    PTHREAD_PSHARED REGPARM STATIC_PCRE STATIC_PCRE2 VSYSCALL GETADDRINFO
    OPENSSL LUA MY_ACCEPT4 ZLIB SLZ TFO NS DEVICEATLAS 51DEGREES WURFL
    SYSTEMD OBSOLETE_LINKER EVPORTS
2019-06-14 16:18:03 +02:00
Olivier Houchard
fe50bfb82c MEDIUM: connections: Introduce a handshake pseudo-XPRT.
Add a new XPRT that is used when using non-SSL handshakes, such as proxy
protocol or Netscaler, instead of taking care of it in conn_fd_handler().
This XPRT is installed when any of those is used, and it removes itself once
the handshake is done.
This should allow us to remove the distinction between CO_FL_SOCK* and
CO_FL_XPRT*.
2019-06-05 18:03:38 +02:00
Frdric Lcaille
4a3fef834c MINOR: dict: Add dictionary new data structure.
This patch adds minimalistic definitions to implement dictionary new data structure
which is an ebtree of ebpt_node structs with strings as keys. Note that this has nothing
to see with real dictionary data structure (maps of keys in association with values).
2019-06-05 08:33:35 +02:00
Willy Tarreau
3844747536 CLEANUP: raw_sock: remove support for very old linux splice bug workaround
We've been dealing with a workaround for a bug in splice that used to
affect version 2.6.25 to 2.6.27.12 and which was fixed 10 years ago
in kernel versions which are not supported anymore. Given that people
who would use a kernel in such a range would face much more serious
stability and security issues, it's about time to get rid of this
workaround and of the ASSUME_SPLICE_WORKS build option used to disable
it.
2019-05-22 20:02:15 +02:00
Willy Tarreau
e5733234f6 CLEANUP: build: rename some build macros to use the USE_* ones
We still have quite a number of build macros which are mapped 1:1 to a
USE_something setting in the makefile but which have a different name.
This patch cleans this up by renaming them to use the USE_something
one, allowing to clean up the makefile and make it more obvious when
reading the code what build option needs to be added.

The following renames were done :

 ENABLE_POLL -> USE_POLL
 ENABLE_EPOLL -> USE_EPOLL
 ENABLE_KQUEUE -> USE_KQUEUE
 ENABLE_EVPORTS -> USE_EVPORTS
 TPROXY -> USE_TPROXY
 NETFILTER -> USE_NETFILTER
 NEED_CRYPT_H -> USE_CRYPT_H
 CONFIG_HAP_CRYPT -> USE_LIBCRYPT
 CONFIG_HAP_NS -> DUSE_NS
 CONFIG_HAP_LINUX_SPLICE -> USE_LINUX_SPLICE
 CONFIG_HAP_LINUX_TPROXY -> USE_LINUX_TPROXY
 CONFIG_HAP_LINUX_VSYSCALL -> USE_LINUX_VSYSCALL
2019-05-22 19:47:57 +02:00
Willy Tarreau
2bfefdbaef MAJOR: watchdog: implement a thread lockup detection mechanism
Since threads were introduced, we've naturally had a number of bugs
related to locking issues. In addition we've also got some issues
with corrupted lists in certain rare cases not necessarily involving
threads. Not only these events cause a lot of trouble to the production
as it is very hard to detect that the process is stuck in a loop and
doesn't deliver the service anymore, but it's often difficult (or too
late) to collect more debugging information.

The patch presented here implements a lockup detection mechanism, also
known as "watchdog". The principle is that (on systems supporting it),
each thread will have its own CPU timer which progresses as the thread
consumes CPU cycles, and when a deadline is met, a signal is delivered
(SIGALRM here since it doesn't interrupt gdb by default).

The thread handling this signal (which is not necessarily the one which
triggered the timer) figures the thread ID from the signal arguments and
checks if it's really stuck by looking at the time spent since last exit
from poll() and by checking that the thread's scheduler is still alive
(so that even when dealing with configuration issues resulting in insane
amount of tasks being called in turn, it is not possible to accidently
trigger it). Checking the scheduler's activity will usually result in a
second chance, thus doubling the detecting time.

In order not to incorrectly flag a thread as being the cause of the
lockup, the thread_harmless_mask is checked : a thread could very well
be spinning on itself waiting for all other threads to join (typically
what happens when issuing "show sess"). In this case, once all threads
but one (or two) have joined, all the innocent ones are marked harmless
and will not trigger the timer. Only the ones not reacting will.

The deadline is set to one second, which already appears impossible to
reach, especially since it's 1 second of CPU usage, not elapsed time
with the CPU being preempted by other threads/processes/hypervisor. In
practice due to the scheduler's health verification it takes up to two
seconds to decide to panic.

Once all conditions are met, the goal is to crash from the offending
thread. So if it's the current one, we call ha_panic() otherwise the
signal is bounced to the offending thread which deals with it. This
will result in all threads being woken up in turn to dump their context,
the whole state is emitted on stderr in hope that it can be logged, and
the process aborts, leaving a chance for a core to be dumped and for a
service manager to restart it.

An alternative mechanism could be implemented for systems unable to
wake up a thread once its CPU clock reaches a deadline (e.g. FreeBSD).
Instead of waking the timer each and every deadline, it is possible to
use a standard timer which is reset each time we leave poll(). Since
the signal handler rechecks the CPU consumption this will also work.
However a totally idle process may trigger it from time to time which
may or may not confuse some debugging sessions. The same is true for
alarm() which could be another option for systems not having such a
broad choice of timers (but it seems that in this case they will not
have per-thread CPU measurements available either).

The feature is currently implemented only when threads are enabled in
order to keep the code clean, since the main purpose is to detect and
address inter-thread deadlocks. But if it proves useful for other
situations this condition might be relaxed.
2019-05-22 11:50:48 +02:00
Emmanuel Hocdet
0ba4f483d2 MAJOR: polling: add event ports support (Solaris)
Event ports are kqueue/epoll polling class for Solaris. Code is based
on https://github.com/joyent/haproxy-1.8/tree/joyent/dev-v1.8.8.
Event ports are available only on SunOS systems derived from
Solaris 10 and later (including illumos systems).
2019-05-21 15:16:45 +02:00
Emmanuel Hocdet
3c0edfa1ff BUILD: makefile: remove -fomit-frame-pointer optimisation (solaris)
-fomit-frame-pointer is commonly avoided because tools like dtrace
needs frame-pointer. Remove it from Makefile and let builder's env
do the job.

This patch could be backported to 1.9.
2019-05-21 15:16:25 +02:00
Emmanuel Hocdet
7093c193f6 BUILD: makefile: use USE_OBSOLETE_LINKER for solaris
USE_OBSOLETE_LINKER is needed to build on SunOS systems.

This patch must be backported to 1.9.
2019-05-21 15:16:10 +02:00
Willy Tarreau
6bdf3e9b11 MINOR: debug/cli: add some debugging commands for developers
When haproxy is built with DEBUG_DEV, the following commands are added
to the CLI :

  debug dev close <fd>        : close this file descriptor
  debug dev delay [ms]        : sleep this long
  debug dev exec  [cmd] ...   : show this command's output
  debug dev exit  [code]      : immediately exit the process
  debug dev hex   <addr> [len]: dump a memory area
  debug dev log   [msg] ...   : send this msg to global logs
  debug dev loop  [ms]        : loop this long
  debug dev panic             : immediately trigger a panic
  debug dev tkill [thr] [sig] : send signal to thread

These are essentially aimed at helping developers trigger certain
conditions and are expected to be complemented over time.
2019-05-20 16:59:30 +02:00
Willy Tarreau
c7091d89ae MEDIUM: debug/threads: implement an advanced thread dump system
The current "show threads" command was too limited as it was not possible
to dump other threads' detailed states (e.g. their tasks). This patch
goes further by using thread signals so that each thread can dump its
own state in turn into a shared buffer provided by the caller. Threads
are synchronized using a mechanism very similar to the rendez-vous point
and using this method, each thread can safely dump any of its contents
and the caller can finally report the aggregated ones from the buffer.

It is important to keep in mind that the list of signal-safe functions
is limited, so we take care of only using chunk_printf() to write to a
pre-allocated buffer.

This mechanism is enabled by USE_THREAD_DUMP and is enabled by default
on Linux 2.6.28+. On other platforms it falls back to the previous
solution using the loop and the less precise dump.
2019-05-17 17:16:20 +02:00
Willy Tarreau
4e2b646d60 MINOR: cli/debug: add a thread dump function
The new function ha_thread_dump() will dump debugging info about all known
threads. The current thread will contain a bit more info. The long-term goal
is to make it possible to use it in signal handlers to improve the accuracy
of some dumps.

The function dumps its output into the trash so as it was trivial to add,
a new "show threads" command appeared on the CLI.
2019-05-16 18:06:45 +02:00
Frdric Lcaille
dc1a3bd999 REGTEST: replace LEVEL option by a more human readable one.
This patch replaces LEVEL variable by REGTESTS_TYPES variable which is more
mnemonic and human readable. It is uses as a filter to run the reg tests scripts
where a commented #REGTEST_TYPE may be defined to designate their types.
Running the following command:

    $ REGTESTS_TYPES=slow,default

will start all the reg tests where REGTEST_TYPE is defines as 'slow' or 'default'.
Note that 'default' is also the default value of REGTEST_TYPE when not specified
dedicated to run all the current h*.vtc files. When REGTESTS_TYPES is not specified
there is no filter at all. All the tests are run.

This patches also defines REGTEST_TYPE with 'slow' value for all the s*.vtc files,
'bug' value for al the b*.vtc files, 'broken' value for all the k*.vtc files.
2019-04-23 15:14:52 +02:00
Willy Tarreau
1426198efb BUILD: add USE_WURFL to the list of known build options
Since the removal of WURFL we've improved the build system to report
known build options, let's reference this option there as well.
2019-04-23 11:00:23 +02:00
Willy Tarreau
b3cc9f2887 Revert "CLEANUP: wurfl: remove dead, broken and unmaintained code"
This reverts commit 8e5e1e7bf0.

The following patches will fix this code and may be backported.
2019-04-23 10:34:43 +02:00
Willy Tarreau
636848aa86 MINOR: init: add a "set-dumpable" global directive to enable core dumps
It's always a pain to get a core dump when enabling user/group setting
(which disables the dumpable flag on Linux), when using a chroot and/or
when haproxy is started by a service management tool which requires
complex operations to just raise the core dump limit.

This patch introduces a new "set-dumpable" global directive to work
around these troubles by doing the following :

  - remove file size limits     (equivalent of ulimit -f unlimited)
  - remove core size limits     (equivalent of ulimit -c unlimited)
  - mark the process dumpable again (equivalent of suid_dumpable=1)

Some of these will depend on the operating system. This way it becomes
much easier to retrieve a core file. Temporarily moving the chroot to
a user-writable place generally enough.
2019-04-16 14:31:23 +02:00
William Lallemand
9a1ee7ac31 MEDIUM: mworker-prog: implement program for master-worker
This patch implements the external binary support in the master worker.

To configure an external process, you need to use the program section,
for example:

	program dataplane-api
		command ./dataplane_api

Those processes are launched at the same time as the workers.

During a reload of HAProxy, those processes are dealing with the same
sequence as a worker:

  - the master is re-executed
  - the master sends a USR1 signal to the program
  - the master launches a new instance of the program

During a stop, or restart, a SIGTERM is sent to the program.
2019-04-01 14:45:37 +02:00
William Lallemand
48dfbbdea9 REORG: mworker: move serializing functions to mworker.c
Move the 2 following functions to mworker.c:

void mworker_proc_list_to_env()
void mworker_env_to_proc_list()
2019-04-01 14:45:37 +02:00
Willy Tarreau
13d9b0231a BUILD: Makefile: disable shared cache on AIX 5.1
AIX 5.1 is missing the following builtins used for atomic locking of the
shared inter-process cache :

   .__sync_val_compare_and_swap_4
   .__sync_lock_test_and_set_4
   .__sync_sub_and_fetch_4

Let's simply use the private cache by default since nobody cares on
such old systems. No test was made on a more recent version.
2019-04-01 07:46:07 +02:00
Willy Tarreau
0aed6acac5 BUILD: define unsetenv on AIX 5.1
This version doesn't have unsetenv(), so let's map it to my_unsetenv() instead.
This wasn't tested on more recent versions.
2019-04-01 07:45:49 +02:00
Willy Tarreau
891d65a672 BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
Not tested on later versions, but at least there _LINUX_SOURCE_COMPAT
must be defined to access the CMSG_SPACE() and CMSG_LEN() macros.
2019-04-01 07:45:37 +02:00
Willy Tarreau
6f4fd1b183 BUILD: makefile: fix build of IPv6 header on aix51
ip6_hdr is called ip6hdr there and is only defined when STEVENS_API is
defined.
2019-04-01 07:45:22 +02:00
Willy Tarreau
7b5654f54a BUILD: re-implement an initcall variant without using executable sections
The current initcall implementation relies on dedicated sections (one
section per init stage) to store the initcall descriptors. Then upon
startup, these sections are scanned from beginning to end and all items
found there are called in sequence.

On platforms like AIX or Cygwin it seems difficult to figure the
beginning and end of sections as the linker doesn't seem to provide
the corresponding symbols. In order to replace this, this patch
simply implements an array of single linked (one per init stage)
which are fed using constructors for each register call. These
constructors are declared static, with a name depending on their
line number in the file, in order to avoid name clashes. The final
effect is the same, except that the method is slightly more expensive
in that it explicitly produces code to register these initcalls :

$ size  haproxy.sections haproxy.constructor
   text    data     bss     dec     hex filename
4060312  249176 1457652 5767140  57ffe4 haproxy.sections
4062862  260408 1457652 5780922  5835ba haproxy.constructor

This mechanism is enabled as an alternative to the default one when
build option USE_OBSOLETE_LINKER is set. This option is currently
enabled by default only on AIX and Cygwin, and may be attempted for
any target which fails to build complaining about missing symbols
__start_init_* and/or __stop_init_*.

Once confirmed as a reliable fix, this will likely have to be backported
to 1.9 where AIX and Cygwin do not build anymore.
2019-04-01 07:43:07 +02:00
Willy Tarreau
57c99ec18e BUILD: makefile: work around another bug in make 3.80
GNU make 3.80 has an issue with calls to functions inside an if block,
which is just what we recently introduced to simplify the targets
declaration. The fix is easy, it simply consists in assigning the
command to a variable inside the if block and evaluating this command
after the block. This also makes the code slightly more readable so we
can keep compatibility with 3.80 for now.

No backport is needed.
2019-03-29 21:00:01 +01:00
Willy Tarreau
509a009c5d BUILD: makefile: work around an old bug in GNU make-3.80
GNU make-3.80 fails on the .build_opts target, expecting the closing
brace before the first semi-colon in the shell command, it probably
uses a more limited parser for dependencies. Actually it appears it's
enough to place this command in a variable and reference the variable
there. Since it doesn't affect later versions (and the resulting string
is always empty anyway), let's apply the minor change to continue to
comply with the announced dependencies.

This could be backported as far as 1.6.
2019-03-29 20:59:59 +01:00
Willy Tarreau
824cd00d3b BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
Many of these variables are already passed verbatim. Let's now pass
all of them, this will require less changes in the future. A number
of older variables have different names for the makefile and the code
and should be adjusted to further simplify this. A few remain though,
mainly the ones which imply another one (e.g. USE_STATIC_PCRE implies
USE_PCRE).
2019-03-27 14:47:55 +01:00
Willy Tarreau
7728ed3565 BUILD: report the whole feature set with their status in haproxy -vv
It's not convenient not to know the status of default options, and
requires the user to know what option is enabled by default in each
target. With this patch, a new "Features list" line is added to the
output of "haproxy -vv" to report the whole list of known features
with their respective status. They're prefixed with a "+" when enabled
or a "-" when disabled. The "USE_" prefix is removed for clarity.
2019-03-27 14:32:58 +01:00
Willy Tarreau
39f1992d7b BUILD: Makefile: clean up the target declarations
The target declarations were historically made of a series of if/else but
this is pointless and only makes the list unreadable given the number of
entries, especially the long tail of "endif". Just use a series of
"if/endif" for each target instead, and take this opportuity to clean up
the comments.
2019-03-27 14:32:58 +01:00
Willy Tarreau
f6bf8e9ead BUILD: Makefile: shorten default settings declaration
By using a "default_opts" function we can enumerate at once all the
settings we want to enable by default for each platform instead of
individually assigning each variable. Doing this removed 46 lines
in the makefile.
2019-03-27 14:32:58 +01:00
Willy Tarreau
05fd82da76 BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
Now we iterate over all known variables and report in the BUILD_OPTIONS
string all those which differ from the target's defaults. This means that
if a target sets a variable by default (e.g. USE_THREAD in linux2628) and
the user disables it on the command line, the BUILD_OPTIONS string will
now properly report "USE_THREAD=".
2019-03-27 14:32:55 +01:00
Willy Tarreau
09fe566936 BUILD: Makefile: consider a variable's origin and not its value for the options list
Right now it's annoying not to be able to enumerate disabled options that
are set by default for a given target. The reason is that we rely on the
fact that the variable is neither cleared nor set to "implicit" in order
to list it.

Here we modify the ignore_implicit function to check the variable's origin
instead of its value. We consider as modified any variable whose origin is
"environment" or "command". Other ones are "undefined" (variable not set)
and "file" (variable set in the Makefile). For now this doesn't change
anything since variables are only dumped when not empty. However if a
variable was forced to "implicit" on the command line, it would now appear.
2019-03-27 14:30:16 +01:00
Willy Tarreau
c3643517f3 BUILD: Makefile: remove outdated support for dlmalloc
dlmalloc has remained unused for quite a while now, in part because it
is not thread-safe and in part because it has been superseded by the
much better and faster jemalloc. So let's simplify the makefile and
remove entries related to this library.
2019-03-27 14:30:15 +01:00
Willy Tarreau
9fc5cc609d BUILD: remove 10-years old error message for obsolete option USE_TCPSPLICE
The USE_TCPSPLICE option was removed in 1.4-dev3 10 years ago, and the
error message remained to warn the user. Let's get rid of it now.
2019-03-27 14:29:29 +01:00
Willy Tarreau
5c0ac3a51e BUILD: Makefile: remove 11-years old workarounds for deprecated options
Build options "REGEX=" and "DEFINE=-DTPROXY" have been deprecated by
commit 9f2b730 in 1.3.15 and have been emitting warnings for over 11
years. It's about time to get rid of them.
2019-03-27 14:29:29 +01:00
Willy Tarreau
c6c37b2d46 BUILD: Makefile: resolve LEVEL before calling run-regtests
Calling "make reg-tests V=1" shows --LEVEL "$LEVEL" which is not quite
useful. Let's use "$(LEVEL)" instead of "$$LEVEL" so that make resolves
the variable before launching the command. This way the reported command
is usable from the shell.
2019-03-15 17:29:53 +01:00
Willy Tarreau
f22bd9055b BUILD: Makefile: allow the reg-tests target to be verbose
When debugging reg-tests, it's quite annoying not to be able to figure
the syntax to call the scripts. Let's replace the '@' with '$(Q)' as for
other commands so that launching them with "V=1" is enough to reveal the
command line.
2019-03-15 17:28:36 +01:00
Willy Tarreau
8e5e1e7bf0 CLEANUP: wurfl: remove dead, broken and unmaintained code
Since the "wurfl" device detection engine was merged slightly more than
two years ago (2016-11-04), it never received a single fix nor update.
For almost two years it didn't receive even the minimal review or changes
needed to be compatible with threads, and it's remained build-broken for
about the last 9 months, consecutive to the last buffer API changes,
without anyone ever noticing! When asked on the list, nobody confirmed
using it :

   https://www.mail-archive.com/haproxy@formilux.org/msg32516.html

And obviously nobody even cared to verify that it did still build. So we
are left with this broken code with no user and no maintainer. It might
even suffer from remotely exploitable vulnerabilities without anyone
being able to check if it presents any risk. It's a pain to update each
time there is an API change because it doesn't build as it depends on
external libraries that are not publicly accessible, leading to careful
blind changes. It slows down the whole project. This situation is not
acceptable at all.

It's time to cure the problem where it is. This patch removes all this
dead, non-buildable, non-working code. If anyone ever decides to use it,
which I seriously doubt based on history, it could be reintegrated, but
this time the following guarantees will be required :
  - someone has to step up as a maintainer and have his name listed in
    the MAINTAINERS file (I should have been more careful last time).
    This person will take the sole blame for all issues and will be
    responsible for fixing the bugs and incompatibilities affecting
    this code, and for making it evolve to follow regular internal API
    updates.

  - support building on a standard distro with automated tools (i.e. no
    more "click on this site, register your e-mail and download an
    archive then figure how to place this into your build system").
    Dummy libs are OK though as long as they allow the mainline code to
    build and start.

  - multi-threaded support must be fixed. I mean seriously, not worked
    around with a check saying "please disable threads, we've been busy
    fishing for the last two years".

This may be backported to 1.9 given that the code has never worked there
either, thus at least we're certain nobody will miss it.
2019-03-05 13:46:12 +01:00
Olivier Houchard
2292edf67c MINOR: fd: Use closefrom() as my_closefrom() if supported.
Add a new option, USE_CLOSEFROM. If set, it is assumed the system provides
a closefrom() function, so use it.
It is only implicitely used on FreeBSD for now, it should work on
OpenBSD/NetBSD/DragonflyBSD/Solaris too, but as I have no such system to
test it, I'd rather leave it disabled by default. Users can add USE_CLOSEFROM
explicitely on their make command line to activate it.
2019-02-25 16:51:03 +01:00
Ben51Degrees
4ddf59d070 MEDIUM: 51d: Enabled multi threaded operation in the 51Degrees module.
The existing threading flag in the 51Degrees API
(FIFTYONEDEGREES_NO_THREADING) has now been mapped to the HAProxy
threading flag (USE_THREAD), and the 51Degrees module code has been made
thread safe.
In Pattern, the cache is now locked with a spin lock from hathreads.h
using a new lable 'OTHER_LOCK'. The workset pool is now created with the
same size as the number of threads to avoid any time waiting on a
worket.
In Hash Trie, the global device offsets structure is only used in single
threaded operation. Multi threaded operation creates a new offsets
structure in each thread.
2019-02-08 21:29:23 +01:00
Frédéric Lécaille
4336123b5c REGTEST: Switch to vtest.
This patch replace the usage of the formerly varnish cache reg
testing program, name varnishtest by the new standalone one: vtest.
2019-01-14 14:21:13 +01:00
Willy Tarreau
ada5d09142 BUILD: makefile: add an EXTRA_OBJS variable to help build optional code
This variable will be useful to build experimental autonomous code like
new muxes without having to patch the makefile.
2019-01-10 10:01:03 +01:00
Willy Tarreau
a1065a1a4f DOC: regtest: make it clearer what the purpose of the "broken" series is
The purpose of the "broken" series of reg tests is to integrate scripts
which are known for triggering bugs that are not fixed at the time the
script is merged. These ones are not useful to validate non-regression
after merging a change, but have an important value to help fix the bug
they trigger. This patch updates the description in the Makefile to make
this clearer.
2019-01-08 09:59:24 +01:00
Willy Tarreau
909b9d852b BUILD: add a new file "version.c" to carry version updates
While testing fixes, it's sometimes confusing to rebuild only one C file
(e.g. a mux) and not to have the correct commit ID reported in "haproxy -v"
nor on the stats page.

This patch adds a new "version.c" file which is always rebuilt. It's
very small and contains only 3 variables derived from the various
version strings. These variables are used instead of the macros at the
few places showing the version. This way the output version of the
running code is always correct for the parts that were rebuilt.
2019-01-04 18:20:32 +01:00
Frédéric Lécaille
a3b4cbff7b REGTEST: Make reg-tests target support argument.
With this patch we can provide a list of argument to reg-tests target.
Useful to run reg tests for a list of VTC files like that:

    $ VARNISHTEST_PROGRAM=<...> make reg-tests reg-tests/checks/*.vtc
2018-12-20 10:37:32 +01:00
Frederic Lecaille
d4f36e3eaa REGTEST: Reg testing improvements.
Add a new target to the Makefile named "reg-tests-help" to have an idea
about how to run the reg tests from haproxy Makefile.
Handle list of levels and lists of level range passed to make with LEVEL variable.
New supported syntax:
    LEVEL=1,4     make reg-tests
    LEVEL=1-2,5-6 make reg-tests
Add two new levels 5 and 6. 5 is for broken script, 6 for experimental scripts.

Signed-off-by: Frdric Lcaille <flecaille@haproxy.com>
2018-12-14 06:50:45 +01:00
William Lallemand
f1b60f78e0 BUILD: Makefile: Implements the help target
Use the header of the Makefile to output the help target.
2018-12-13 14:06:57 +01:00
Willy Tarreau
c5a4fd5c30 REORG: http: create http_msg.c to place there some legacy HTTP parts
Lots of HTTP code still uses struct http_msg. Not only this code is
still huge, but it's part of the legacy interface. Let's move most
of these functions to a separate file http_msg.c to make it more
visible which file relies on what. It's mostly symmetrical with
what is present in http_htx.c.

The function http_transform_header_str() which used to rely on two
function pointers to look up a header was simplified to rely on
two variants http_legacy_replace_{,full_}header(), making both
sides of the function much simpler.

No code was changed beyond these moves.
2018-12-11 17:15:13 +01:00
PiBa-NL
7250404b71 REGTEST/MINOR: script: add run-regtests.sh script
Some tests require a minimal haproxy version or compilation options to be
able to run successfully. This script allows to add 'requirements' to tests
to check so they will automatically be skipped if a requirement is not met.
The script supports several parameters to slightly modify its behavior
including the directories to search for tests.

Also some features are not available for certain OS's these can also
be 'excluded', this should allow for the complete set of test cases to be
run on any OS against any haproxy release without 'expected failures'.

The test .vtc files will need to be modified to include their 'requirements'
by listing including text options as shown below:
    #EXCLUDE_TARGETS=dos,freebsd,windows
    #REQUIRE_OPTIONS=ZLIB,OPENSSL,LUA
    #REQUIRE_VERSION=0.0
    #REQUIRE_VERSION_BELOW=99.9,
When excluding a OS by its TARGET, please do make a comment why the test
can not succeed on that TARGET.
2018-11-29 04:54:45 +01:00
Olivier Houchard
73bce43812 BUILD: Makefile: Disable -Wcast-function-type if it exists.
Disable -Wcast-function-type for recent gcc, if we're casting a function
type to another one, it is assumed we know what we're doing.
2018-11-28 04:40:15 +01:00
Willy Tarreau
db86333901 REGTEST: add the option to test only a specific set of files
It currently is quite difficult to re-reun a specific test after an
error occurs. This patch adds a REG_TEST_FILES variable to the makefile,
which will be used to override the find operation. This helps focusing
on a specific file, which is essential during bisect to figure what
commit introduced a specific regression. Multiple files may be tested,
the return code will indicate the number of failed tests.
2018-11-23 08:34:03 +01:00
Willy Tarreau
609aad9e73 REORG: time/activity: move activity measurements to activity.{c,h}
At the moment the situation with activity measurement is quite tricky
because the struct activity is defined in global.h and declared in
haproxy.c, with operations made in time.h and relying on freq_ctr
which are defined in freq_ctr.h which itself includes time.h. It's
barely possible to touch any of these files without breaking all the
circular dependency.

Let's move all this stuff to activity.{c,h} and be done with it. The
measurement of active and stolen time is now done in a dedicated
function called just after tv_before_poll() instead of mixing the two,
which used to be a lazy (but convenient) decision.

No code was changed, stuff was just moved around.
2018-11-22 11:48:41 +01:00
Willy Tarreau
1611965e7d BUILD: Makefile: switch to quiet mode by default for CC/LD/AR
These commands are now replaced with a prefix and the target name only
in quiet mode, which is much more readable and allows better detection
of build warnings than the default verbose mode. Using V=1 switches back
to the detailed output.
2018-11-19 08:18:49 +01:00
Willy Tarreau
6db8266409 BUILD: Makefile: add "$(Q)" to clean, tags and cscope targets
These ones didn't have the quiet mode yet.
2018-11-19 08:18:49 +01:00
Willy Tarreau
4ab5d030c7 BUILD: Makefile: add the quiet mode to a few more targets
The various install-* and *-tar targets are now launched with $(Q). The
install argument "-v" was added to install commands to see what is copied
where.
2018-11-19 08:18:49 +01:00
Willy Tarreau
b42233a19d BUILD: Makefile: make "V=1" show some of the commands that are executed
By default some commands are hidden but sometimes it would be nice to
see what is executed. Make the usual "V=1" unhide these commands.
2018-11-19 08:18:49 +01:00
Willy Tarreau
2401835899 BUILD: reorder the objects in the makefile
This is the annual reordering of the make file consisting in sorting
the files by reverse build time. This has sped up the parallel build
at -O2 from 10.5 sec down to 7.9.
2018-11-19 08:18:49 +01:00
Willy Tarreau
9dbfa051a0 BUILD: update the list of supported targets and compilers in makefile and readme
The list of suggested targets reported in the default make command was not
up to date. The equivalent versions were updated in the README as well as
the supported compiler versions.
2018-11-19 08:18:49 +01:00
Willy Tarreau
3a1f5fda10 REORG: config: extract the proxy parser into cfgparse-listen.c
This was the largest function of the whole file, taking a rough second
to build alone. Let's move it to a distinct file along with a few
dependencies. Doing so saved about 2 seconds on the total build time.
2018-11-19 06:47:09 +01:00
Willy Tarreau
36b9e222bb REORG: config: extract the global section parser into cfgparse-global
The config parser is the largest file to build and its build dominates
the total project's build time. Let's start to split it into multiple
smaller pieces by extracting the "global" section parser into a new
file called "cfgparse-global.c". This removes 1/4th of the file's build
time.
2018-11-19 06:41:57 +01:00
Christopher Faulet
47596d3787 MINOR: http_htx: Add functions to manipulate HTX messages in http_htx.c
This file will host all functions to manipulate HTTP messages using the HTX
representation. Functions in this file will be able to be called from anywhere
and are mainly related to the HTTP semantics.
2018-11-18 22:08:53 +01:00
Christopher Faulet
a3d2a16fad MEDIUM: htx: Add API to deal with the internal representation of HTTP messages
The internal representation of an HTTP message, called HTX, is a structured
representation, unlike the old one which is a raw representation of
messages. Idea is to have a version-agnostic representation of the HTTP
messages, which can be easily used by to handle HTTP/1, HTTP/2 and hopefully
QUIC messages, and communication from one of them to another.

In this patch, we add types to define the internal representation itself and the
main functions to manipulate them.
2018-11-18 22:08:53 +01:00
Christopher Faulet
51dbc94d48 MEDIUM: mux-h1: Add dummy mux to handle HTTP/1.1 connections
For now, it is just an other kind of passthrough multiplexer, but with internal
buffers to be prepared to parse incoming messages and to format outgoing
ones. There is also a task attached to it to handle timeouts. However, because
it does not handle any timeout for now, this task is unused. And finally,
because it handles internal buffers, it also handles retries on recv/send. To
use this multiplexer, you must use the option "http-use-htx" both on the
frontend and the backend.

It does not support keep-alive and will freeze connections after the first
request/response.
2018-11-18 22:02:11 +01:00
Christopher Faulet
f4eb75d177 MINOR: htx: Add proto_htx.c file
This file is empty for now. But it will be used to add new versions of the HTTP
analyzers based on the internal representation of HTTP messages (not implemented
yet but called HTX).
2018-11-18 21:45:48 +01:00
Willy Tarreau
7520e4ff57 MINOR: namespaces: don't build namespace.c if disabled
When namespaces are disabled, support is still reported because the file
is built with almost nothing in it but built anyway. Instead of extending
the scope of the numerous ifdefs in this file, better avoid building it
when namespaces are diabled. In this case we define my_socketat() as an
inline function mapping directly to socket(). The struct netns_entry
still needs to be defined because it's used by various other functions
in the code.
2018-11-12 19:15:15 +01:00
Joseph Herlant
4e8683c015 CLEANUP: fix typos in the comments of the Makefile
This is not user-visible issues, just a cleanup of comments.
2018-11-12 08:51:54 +01:00
Willy Tarreau
943e7ec025 MEDIUM: auth/threads: make use of crypt_r() on systems supporting it
On systems where crypt_r() is available, prefer it over a locked crypt().
This improves performance especially on very slow crypto algorithms.
2018-10-29 19:17:39 +01:00
Willy Tarreau
23cd43e2d6 BUILD: Makefile: add the new ERR variable to force -Werror
Instead of having to fiddle with the CFLAGS, let's have ERR=1 to enable
-Werror.
2018-10-22 06:22:46 +02:00
Willy Tarreau
58e90cbb9e BUILD: Makefile: add USE_RT to pass -lrt for clock_gettime() and friends
Some code will require clock_gettime() which needs -lrt on most Linux
distros (those with glibc < 2.17). For this reason, this patch introduces
USE_RT to enable -lrt, which is implicitly set for all Linux flavors,
since it's harmless to link with it on more recent ones. Those who know
they can safely get rid of -lrt can remove it using "USE_RT=".
2018-10-18 16:39:03 +02:00
Olivier Houchard
c4e6460f66 MINOR: build: Disable -Wstringop-overflow.
Disable -Wstringop-overflow, as it gives annoying false positives
with gcc 8.
2018-10-16 19:28:23 +02:00
Willy Tarreau
0d7a2ae4f5 BUILD: Makefile: silence an option conflict warning with clang
clang complains that -fno-strict-overflow is not used when -fwrapv is
used, which breaks the build when -Werror is used. Let's introduce a
cc-opt-alt function to emit the former only then the latter is not
supported (since it implies the former).
2018-10-16 18:13:11 +02:00
Willy Tarreau
f11ca5e7a4 BUILD: Makefile: speed up compiler options detection
Commits b78016649 and d3a7f4035 brought the ability to detect the build
options and warnings that the compiler supports. However, they're detected
using "$(CC) -c", which is 50% slower than "$(CC) -E" for the same result,
just because it starts the assembler at the end. Given that we're starting
to check for a number of warnings, this detection alone starts to become
visible, taking a bit more than 300 ms on the build time. Let's switch to
-E instead to shrink this incompressible time by roughly 100 ms.
2018-10-03 09:55:53 +02:00
Willy Tarreau
a8b12c6bb7 BUILD: Makefile: add a "make opts" target to simply show the build options
We're often missing an easy way to map input variables to output ones.
The "opts" build target will simply show the input variables and the ones
passed to the compiler and linker. This way it's easier to quickly see
what a given build script or package will use, or the detected warnings
supported by the compiler.
2018-10-03 09:55:51 +02:00
Willy Tarreau
61c112aa5b REORG: http: move HTTP rules parsing to http_rules.c
These ones are mostly called from cfgparse.c for the parsing and do
not depend on the HTTP representation. The functions's prototypes
were moved to proto/http_rules.h, making this file work exactly like
tcp_rules. Ideally we should stop calling these functions directly
from cfgparse and register keywords, but there are a few cases where
that wouldn't work (stats http-request) so it's probably not worth
trying to go this far.
2018-10-02 18:28:05 +02:00
Willy Tarreau
79e57336b5 REORG: http: move the code to different files
The current proto_http.c file is huge and contains different processing
domains making it very difficult to work on an alternative representation.
This commit moves some parts to other files :

  - ACL registration code => http_acl.c
    This code only creates some ACL mappings and doesn't know anything
    about HTTP nor about the representation. This code could even have
    moved to acl.c but it was not worth polluting it again.

  - HTTP sample conversion => http_conv.c
    This code doesn't depend on the internal representation but definitely
    manipulates some HTTP elements, such as dates. It also has access to
    captures.

  - HTTP sample fetching => http_fetch.c
    This code does depend entirely on the internal representation but is
    totally independent on the analysers. Placing it into a different
    file will ease the transition to the new representation and the
    creation of a wrapper if required. An include file was created due
    to CHECK_HTTP_MESSAGE_FIRST() being used at various places.

  - HTTP action registration => http_act.c
    This code doesn't directly interact with the messages nor the
    transaction but it does so via some exported http functions like
    http_replace_req_line() or http_set_status() so it will be easier
    to change only this after the conversion.

  - a few very generic parts were found and moved to http.{c,h} as
    relevant.

It is worth noting that the functions moved to these new files are not
referenced anywhere outside of the files and are only called as registered
callbacks, so these files do not even require associated include files.
2018-10-02 18:26:59 +02:00
Fabrice Fontaine
7b4c8c3b7f BUILD: Allow configuration of pcre-config path
Add PCRE_CONFIG and PCRE2_CONFIG variables to allow the user to
configure path of pcre-config or pcre2-config instead of using the one
in his path.
This is particulary useful when cross-compiling.

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
2018-09-30 16:23:04 +02:00
Willy Tarreau
f7db9305aa BUILD: build with -Wextra and sort out certain warnings
We're not far from being able to build with -Wextra -Werror. The
following warnings had to be disabled to enable a clean build at
-Wextra on x86_64 using gcc 4.7, 5.5, 6.4 and 7.3 :

   sign-compare, unused-parameter, old-style-declaration,
   ignored-qualifiers, clobbered, missing-field-initializers,
   implicit-fallthrough

The following extra warnings could be added without side effects :

   type-limits, shift-negative-value, shift-overflow=2 duplicated-cond,
   null-dereference

As a result, -Wextra was enabled by default, hoping it will help catch
issues over the long term. If new undesired warnings pop up, it's easy
to disable them using the nowarn call.
2018-09-20 11:43:19 +02:00
William Lallemand
2fe7dd0b2e MEDIUM: protocol: sockpair protocol
This protocol is based on the uxst one, but it uses socketpair and FD
passing insteads of a connect()/accept().

The "sockpair@" prefix has been implemented for both bind and server
keywords.

When HAProxy wants to connect through a sockpair@, it creates 2 new
sockets using the socketpair() syscall and pass one of the socket
through the FD specified on the server line.

On the bind side, haproxy will receive the FD, and will use it like it
was the FD of an accept() syscall.

This protocol was designed for internal communication within HAProxy
between the master and the workers, but it's possible to use it
externaly with a wrapper and pass the FD through environment variabls.
2018-09-12 07:20:17 +02:00
Willy Tarreau
35b51c6e5b REORG: http: move the HTTP semantics definitions to http.h/http.c
It's a bit painful to have to deal with HTTP semantics for each protocol
version (H1 and H2), and working on the version-agnostic code further
emphasizes the problem.

This patch creates http.h and http.c which are agnostic to the version
in use, and which borrow a few parts from proto_http and from h1. For
example the once thought h1-specific h1_char_classes array is in fact
dictated by RFC7231 and is used to parse HTTP headers. A few changes
were made to a few files which were including proto_http.h while they
only needed http.h.

Certain string definitions pre-dated the introduction of indirect
strings (ist) so some were used to simplify the definition of the known
HTTP methods. The current lookup code saves 2 kB of a heavily used table
and is faster than the previous table based lookup (typ. 14 ns vs 16
before).
2018-09-11 10:30:25 +02:00
Frdric Lcaille
9931634e71 REGTEST/MINOR: Add a new class of regression testing files.
Add LEVEL #4 regression testing files which is dedicated to
VTC files in relation with bugs they help to reproduce.
At the date of this commit, all VTC files are LEVEL 4 VTC files.
2018-08-23 15:47:10 +02:00
Frdric Lcaille
26ac8a6c4e REGTEST/MINOR: Add levels to reg-tests target.
With this patch we can provide LEVEL environment variable when
running reg-tests Makefile targe (reg testing) to set the execution
level of the reg-tests make target to run.

LEVEL default value is 1.

LEVEL=1 is to run all h*.vtc files which are the most important
reg testing files (to test haproxy core, HTTP compliance etc).

LEVEL=2 is to run all s*.vtc files which are a bit slow tests,
for instance tests requiring external programs (curl, socat etc).

LEVEL=3 is to run all l*.vtc files which are test files with again
more slow or with little interest.
2018-06-25 22:04:58 +02:00
Frdric Lcaille
bca3a980e9 REGTEST/MINOR: Set HAPROXY_PROGRAM default value.
With this patch, we set HAPROXY_PROGRAM environment variable
default value to the haproxy executable of the current working directory.
So, if the current directory is the haproxy sources directory,
the reg-tests Makefile target may be run with this shorter command:

  $ VARNISTEST_PROGRAM=<...> make reg-tests

in place of

  $ VARNISTEST_PROGRAM=<...> HAPROXY_PROGRAM=<...> make reg-tests
2018-06-25 22:04:42 +02:00
Frdric Lcaille
153b2b68bf MINOR: tests: First regression testing file.
Add a makefile target 'reg-tests' to run all regression testing file
found in 'reg-tests' directory.
Add reg-tests/lua/h00000.vtc first regression testing file for a LUA
fixed by f874a83 commit.
2018-06-19 10:14:59 +02:00
Willy Tarreau
cde05c85ef BUILD/BUG: enable -fno-strict-overflow by default
Some time ago, integer overflows detection stopped working in the timer
code on recent compliers and were addressed by commit 73bdb32 ("BUG/MAJOR:
Use -fwrapv."). By then it was thought that -fno-strict-overflow was not
needed as implied, but it resulted from a misinterpretation of the doc,
as this one is still needed to disable pointer overflow optimization that
is automatically enabled at -O2/-O3/-Os.

Unfortunately the compiler happily removes overflow checks without the
slightest warning so it's not trivial to guess the extent of this issue
without comparing the emitted asm code. By checking the emitted assembly
code with and without the option, it was found that the only affected
location was the reported one, in ssl_sock_parse_clienthello(), where
the test can never fail on any system where the highest userland pointer
is at least 64kB away from wrapping (ie all 32/64 bit OS in field), so
there it is harmless.

This patch must be backported to all maintained versions.

Special thanks to Ilya Shipitsin for reporting this issue.
2018-03-20 17:17:27 +01:00
Willy Tarreau
b684e7a52c BUILD/MINOR: fix Lua build on Mac OS X (again)
Previous commit (13113d6 "MINOR/BUILD: fix Lua build on Mac OS X")
contains a typo, it uses "-export-dynamic" instead of "-export_dynamic"
(dash instead of underscore), despite what the commit message suggests,
and it obviously doesn't work. Thanks to Kirill A. Korinsky for reporting
it.

This patch should be backported on each version from 1.6 like the
aforementionned one above.
2018-03-05 15:39:39 +01:00
Thierry Fournier
13113d6abb MINOR/BUILD: fix Lua build on Mac OS X
Change gcc option syntax for Mac. -Wl,--export-dynamic is not
supported, use -Wl,-export_dynamic.

Thanks to Kirill A. Korinsky for the report.

This patch should be backported on each version from 1.6
2018-03-05 14:19:34 +01:00
David Carlier
903ddfd9c8 BUILD/MINOR: Makefile : enabling USE_CPU_AFFINITY
FreeBSD can handle cpuset matters just fine, we can hence enable it
by default as linux2628 TARGET.
2017-11-29 14:31:10 +01:00
Willy Tarreau
d73efb4349 BUILD: Makefile: reorder object files by size
We've added many files since last version, it was about time to reorder
the makefile to improve parallel builds by having the slower files built
first. This allows to consistently stay below 4 seconds when using a
20-core build farm.
2017-11-26 19:15:59 +01:00
Willy Tarreau
158fa75811 MINOR: pools: implement DEBUG_UAF to detect use after free
This code has been used successfully a few times in the past to detect
that a pool was used after being freed. Its main goal is to allocate a
full page for each object so that they are always released individually
and unmapped from memory. This way if any part of the code reference the
object after is was freed and before it is reallocated, a segv occurs at
the exact offending location. It does a few extra things such as writing
to the memory area before freeing to detect double-frees and free of
read-only areas, and placing the data at the end of the page instead of
the beginning so that out of bounds accesses are easier to spot. The
amount of memory used with this is huge (about 10 times the regular
usage) but it can be useful sometimes.
2017-11-22 19:43:57 +01:00
Willy Tarreau
f24ea8e45e MEDIUM: h2: add a function to emit an HTTP/1 request from a headers list
The current H2 to H1 protocol conversion presents some issues which will
require to perform some processing on certain headers before writing them
so it's not possible to convert HPACK to H1 on the fly.

Here we introduce a function which performs half of what hpack_decode_header()
used to do, which is to take a list of headers on input and emit the
corresponding request in HTTP/1.1 format. The code is the same and functions
were renamed to be prefixed with "h2" instead of "hpack", though it ends
up being simpler as the various HPACK-specific cases could be fused into
a single one (ie: add header).

Moving this part here makes a lot of sense as now this code is specific to
what is documented in HTTP/2 RFC 7540 and will be able to deal with special
cases related to H2 to H1 conversion enumerated in section 8.1.

Various error codes which were previously assigned to HPACK were never
used (aside being negative) and were all replaced by -1 with a comment
indicating what error was detected. The code could be further factored
thanks to this but this commit focuses on compatibility first.

This code is not yet used but builds fine.
2017-11-21 21:13:33 +01:00
Tim Duesterhus
d6942c8297 MEDIUM: mworker: Add systemd Type=notify support
This patch adds support for `Type=notify` to the systemd unit.

Supporting `Type=notify` improves both starting as well as reloading
of the unit, because systemd will be let known when the action completed.

See this quote from `systemd.service(5)`:
> Note however that reloading a daemon by sending a signal (as with the
> example line above) is usually not a good choice, because this is an
> asynchronous operation and hence not suitable to order reloads of
> multiple services against each other. It is strongly recommended to
> set ExecReload= to a command that not only triggers a configuration
> reload of the daemon, but also synchronously waits for it to complete.

By making systemd aware of a reload in progress it is able to wait until
the reload actually succeeded.

This patch introduces both a new `USE_SYSTEMD` build option which controls
including the sd-daemon library as well as a `-Ws` runtime option which
runs haproxy in master-worker mode with systemd support.

When haproxy is running in master-worker mode with systemd support it will
send status messages to systemd using `sd_notify(3)` in the following cases:

- The master process forked off the worker processes (READY=1)
- The master process entered the `mworker_reload()` function (RELOADING=1)
- The master process received the SIGUSR1 or SIGTERM signal (STOPPING=1)

Change the unit file to specify `Type=notify` and replace master-worker
mode (`-W`) with master-worker mode with systemd support (`-Ws`).

Future evolutions of this feature could include making use of the `STATUS`
feature of `sd_notify()` to send information about the number of active
connections to systemd. This would require bidirectional communication
between the master and the workers and thus is left for future work.
2017-11-20 18:39:41 +01:00
Christopher Faulet
f5d79ac5a8 BUILD: enable USE_THREAD for Solaris build. 2017-11-07 11:10:35 +01:00
Willy Tarreau
ca30839a85 MINOR: ebtree: implement the scope-aware functions for eb32
A new kind of tree nodes is currently being developed in ebtree v7,
consisting in storing a scope in each node indicating a visibility
mask so that certain nodes are not reported on certain lookups. The
initial goal was to make this usable with a multi-thread scheduler.

Since the ebtree v7 code is completely different from v6, this patch
instead copies the minimally required functions from eb32 and ebtree
and calls them "eb32sc_*". At the moment the scope is not implemented,
it's only passed in arguments.
2017-11-06 11:20:11 +01:00
David Carlier
7567c4002f BUILD: enable USE_THREAD for OpenBSD build. 2017-11-03 15:41:06 +01:00
William Lallemand
b620e987d0 BUILD: shctx: allow to be built without openssl
The shctx functions does not depend of openssl anymore, allows to build
them without openssl.
2017-11-02 16:58:25 +01:00
William Lallemand
41db46035e MEDIUM: cache: configuration parsing and initialization
Parse a configuration section "cache" and a http-{response,request}
actions.

Example:

    listen frt
        mode http
        http-response cache-store foobar
        http-request cache-use foobar

    cache foobar
        total-max-size 4   # size in megabytes
2017-10-31 21:17:19 +01:00
Willy Tarreau
62f5269d05 MINOR: h2: create a very minimalistic h2 mux
This one currently does nothing and rejects every connection. It
registers ALPN token "h2".
2017-10-31 18:03:24 +01:00
Willy Tarreau
1be4f3d8af MEDIUM: hpack: implement basic hpack encoding
For now it only supports literals and a bit of static header table
references for the 9 most common header field names (date, server,
content-type, content-length, last-modified, accept-ranges, etag,
cache-control, location).

A previous incarnation of this commit used to strip the forbidden H2
header names (connection, proxy-connection, upgrade, transfer-encoding,
keep-alive) but this is no longer the case as this filtering is irrelevant
to HPACK encoding and is specific to H2, so this will have to be done by
the caller.

It's quite not optimal but works fine enough to prepare some valid and
partially compressed responses during development.
2017-10-31 18:03:24 +01:00
Willy Tarreau
679790baae MINOR: hpack: implement the decoder
The decoder is now fully functional. It makes use of the dynamic header
table. Dynamic header table size updates are currently ignored, as our
initially advertised value is the highest we support. Strictly speaking,
the impact is that a client referencing a header field after such an
update wouldn't observe an error instead of the connection being dropped
if it was implemented.

Decoded header fields are copied into a target buffer in HTTP/1 format
using HTTP/1.1 as the version. The Host header field is automatically
appended if a ":authority" header field is present.

All decoded header fields can be displayed if the file is compiled with
DEBUG_HPACK.
2017-10-31 18:03:24 +01:00
Willy Tarreau
ce04094c4a MINOR: hpack: implement the header tables management
This code deals with header insertion, retrieval and eviction, as well
as with dynamic header table defragmentation. It is functional for use
as a decoder and was heavily tested in this context. There's still some
room for optimization (eg: the defragmentation code currently does it
in place using a memcpy).

Also for now the dynamic header table is allocated using malloc() while
a pool needs to be created instead.

This code was mostly imported from https://github.com/wtarreau/http2-exp
with "hpack_" prepended in front of most names to avoid risks of conflicts.
Some small cleanups and renamings were applied during the import. This
version must be considered more recent.

Some HPACK error codes were placed here (HPACK_ERR_*), not exactly because
they're needed by the decoder but they'll be needed by all callers. Maybe
a different location should be found.
2017-10-31 18:03:24 +01:00
Willy Tarreau
a004ade512 MINOR: hpack: implement the HPACK Huffman table decoder
The code was borrowed from the HPACK experimental implementations
available here :

    https://github.com/wtarreau/http2-exp

It contains the Huffman table as specified in RFC7541 Appendix B, and a
set of reverse tables used to decode a Huffman byte stream, and produced
by contrib/h2/gen-rht. The encoder is not finalized, it doesn't emit the
byte stream but this is not needed for now.
2017-10-31 18:03:24 +01:00
Willy Tarreau
53a4766e40 MEDIUM: connection: start to introduce a mux layer between xprt and data
For HTTP/2 and QUIC, we'll need to deal with multiplexed streams inside
a connection. After quite a long brainstorming, it appears that the
connection interface to the existing streams is appropriate just like
the connection interface to the lower layers. In fact we need to have
the mux layer in the middle of the connection, between the transport
and the data layer.

A mux can exist on two directions/sides. On the inbound direction, it
instanciates new streams from incoming connections, while on the outbound
direction it muxes streams into outgoing connections. The difference is
visible on the mux->init() call : in one case, an upper context is already
known (outgoing connection), and in the other case, the upper context is
not yet known (incoming connection) and will have to be allocated by the
mux. The session doesn't have to create the new streams anymore, as this
is performed by the mux itself.

This patch introduces this and creates a pass-through mux called
"mux_pt" which is used for all new connections and which only
calls the data layer's recv,send,wake() calls. One incoming stream
is immediately created when init() is called on the inbound direction.
There should not be any visible impact.

Note that the connection's mux is purposely not set until the session
is completed so that we don't accidently run with the wrong mux. This
must not cause any issue as the xprt_done_cb function is always called
prior to using mux's recv/send functions.
2017-10-31 18:03:23 +01:00
Christopher Faulet
a1ae7e81cd MAJOR: threads: Offically enable the threads support in HAProxy
Now, USE_THREAD option is implicitly enabled when HAProxy is compiled, for
targets linux2628 and freebsd. To enable it for other targets, you can set
"USE_THREAD=1" explicitly on the command line. And to disable it explicitly, you
must set "USE_THREAD=" on the command line.

Now, to be clear. This does not means it is bug free, far from that. But it
seems stable enough to be tested. You can try to experiment it and to report
bugs of course by setting nbthread parameter. By leaving it to 1 (or not using
it at all), it should be as safe as an HAProxy compiled without threads.

Between the commit "MINOR: threads: Prepare makefile to link with pthread" and
this one, the feature was in development and really unstable. It could be hard
to track a bug using a bisect for all these commits.
2017-10-31 13:58:33 +01:00
Christopher Faulet
1a2b56ea8e MEDIUM: threads: Add hathreads header file
This file contains all functions and macros used to deal with concurrency in
HAProxy. It contains all high-level function to do atomic operation
(HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic
operation. So HAProxy can be compiled with the thread support iff these builtins
are available.

It also contains wrappers around plocks to use spin or read/write locks. These
wrappers are used to abstract the internal representation of the locking system
and to add information to help debugging, when compiled with suitable
options.

To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or
DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we
keep info on where a lock was acquired the last time (function name, file and
line). There are also the thread id and a flag to know if it is still locked or
not. This will be useful to debug deadlocks.
2017-10-31 13:58:23 +01:00
Emeric Brun
5f271850bd MINOR: threads: Prepare makefile to link with pthread
USE_THREAD option has been added to enable the compilation with the experimental
support of threads . Of course for now, there is nothing. And for a while,
HAProxy will be unstable. When we will be confident enough, this option will be
removed.

For this implementation and probably for a while, only the pthread library will
be supported.
2017-10-31 11:36:13 +01:00