Commit Graph

446 Commits

Author SHA1 Message Date
Tim Duesterhus
c8d19702f4 BUILD: Show the value of DEBUG= in haproxy -vv
Previously this was not visible after building.
2020-11-21 18:27:33 +01:00
Tim Duesterhus
81e948e051 BUILD: Make DEBUG part of .build_opts
This forces a recompilation if the value of DEBUG= changes.
2020-11-21 18:27:33 +01:00
Matthieu Guegan
496374e592 BUILD: makefile: enable crypt(3) for OpenBSD
Allow OpenBSD to support encrypted passwords in Userlists.

OpenBSD's crypt(3) function is provided directly by libc and does not
require -lcrypt.
Signed-off-by: Matthieu Guegan <matthieu.guegan@deindeal.ch>
2020-11-21 05:45:05 +01:00
Baptiste Assmann
e279ca6bbe MINOR: sample: Add converts to parses MQTT messages
This patch implements a couple of converters to validate and extract data from a
MQTT (Message Queuing Telemetry Transport) message. The validation consists of a
few checks as well as "packet size" validation. The extraction can get any field
from the variable header and the payload.

This is limited to CONNECT and CONNACK packet types only. All other messages are
considered as invalid. It is not a problem for now because only the first packet
on each side can be parsed (CONNECT for the client and CONNACK for the server).

MQTT 3.1.1 and 5.0 are supported.

Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
2020-11-05 19:27:03 +01:00
Baptiste Assmann
e138dda1e0 MINOR: sample: Add converters to parse FIX messages
This patch implements a couple of converters to validate and extract tag value
from a FIX (Financial Information eXchange) message. The validation consists in
a few checks such as mandatory fields and checksum computation. The extraction
can get any tag value based on a tag string or tag id.

This patch requires the istend() function. Thus it depends on "MINOR: ist: Add
istend() function to return a pointer to the end of the string".

Reviewed and Fixed by Christopher Faulet <cfaulet@haproxy.com>
2020-11-05 19:26:30 +01:00
Willy Tarreau
2404e37838 BUILD: makefile: usual reorder of objects for faster builds
Reordered the objets by reverse build times made the total build time
go down from 17.7s to 17.2s at -O2 using make -j8 on my PC, and from
~3.2 to ~2.7s on the build farm.
2020-11-05 16:46:24 +01:00
Willy Tarreau
5c643f37d0 BUILD: makefile: add entries to build common debugging tools
A few tools in contrib/ such as halog, flags, poll and tcploop are
occasionally useful at least to developers, and some of them such as
halog or flags can occasionally break due to some changes in the include
files. As reported in issue #907, their inability to inherit the global
build options also causes some warnings related to some specificities
of the main include files. Let's just add entries in the main makefile
to build them.
2020-10-22 05:17:08 +02:00
Brad Smith
ad5afbafea BUILD: makefile: Enable getaddrinfo() on OS/X
Enable getaddrinfo() on OS/X.
2020-10-10 10:09:29 +02:00
Brad Smith
0fdfe4179e BUILD: makefile: Update feature flags for NetBSD
This updates the feature flags for NetBSD.

NetBSD 8 adds support for accept4().

Enable getaddrinfo().
2020-10-09 09:53:56 +02:00
Willy Tarreau
4a539f343a BUG/MINOR: makefile: fix a tiny typo in the target list
Previous commit 382001b46 ("BUILD: Add a DragonFlyBSD target") introduced
a tiny typo in the target list ("iopenbs" vs "openbsd"). This will have to
be backported if that patch is backported.
2020-10-09 05:58:40 +02:00
Brad Smith
382001b46b BUILD: Add a DragonFlyBSD target
Add a target for DragonFlyBSD 4.3 and above.
2020-10-08 20:54:18 +02:00
Brad Smith
7c503bb459 BUILD: makefile: Enable closefrom() support on Solaris
Solaris 9 (released 2002) added support for closefrom().

I bumped the version in the comment to 10 as the default feature
flags already has event ports enabled which were introduced in
Solaris 10.
2020-10-02 08:32:33 +02:00
Willy Tarreau
ac32b4b98c BUILD: makefile: add an EXTRAVERSION variable to ease local naming
Sometimes it's desirable to append local version naming to packages,
and currently it can only be done using SUBVERS which is already set
by default to the git commit ID and patch count since last known tag,
making the addition a bit complicated.

Let's just add a new EXTRAVERSION field that is empty by default, and
systematically appended verbatim to the version string everywhere. This
way it becomes trivial to append some local strings, such as:

   make TARGET=foo EXTRAVERSION=+$(quilt applied|wc -l)
   -> 2.3-dev5-5018aa-15+1

or :

   make TARGET=foo EXTRAVERSION=-$(date +%F)
   -> 2.3-dev5-5018aa-15-20200110

Let's be careful not to add double quotes (used as the string delimiter)
nor spaces (which can confuse version parsers on the output). The extra
version is also used to name a tarball. It's always pre-initialized to an
empty string so that it's not accidently inherited from the environment.
It's not reported in "make version" to avoid fooling tools (it would be
pointless anyway).

As a side effect it also becomes possible to force VERSION and SUBVERS
to an empty string and use EXTRAVERSION alone to force a specific version
(could possibly be useful when bisecting from patch queues outside of Git
for example).
2020-10-01 04:13:36 +02:00
Brad Smith
5018aacae5 BUILD: makefile: Fix building with closefrom() support enabled
I noticed the USE_CLOSEFROM define was not being passed along like the rest
during the build.

Looking around I see this was broken with the following two commits and related
series..

BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
http://git.haproxy.org/?p=haproxy.git;a=commit;h=05fd82da76d1bbc8d65d63ab246bda7cbcf8481a

BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
http://git.haproxy.org/?p=haproxy.git;a=commit;h=824cd00d3bda8f7f6d4c30baf77ba6c19ab47811

Looks like this should be back ported to 2.0, 2.1 and 2.2.
2020-09-30 16:06:51 +02:00
Brad Smith
9d85eb02d5 BUILD: makefile: Update feature flags for FreeBSD
This updates the feature flags for FreeBSD.

FreeBSD 10 adds support for accept4().

Enable getaddrinfo().

From the FreeBSD port / package.
2020-09-29 11:11:04 +02:00
Brad Smith
90c9d78447 BUILD: makefile: Update feature flags for OpenBSD
Update the OpenBSD target features being enabled.

I updated the list of features after noticing
"BUILD: makefile: disable threads by default on OpenBSD".

The Makefile utilizing gcc(1) by default resulted in utilizing
our legacy and obsolete compiler (GCC 4.2.1) instead of the
proper system compiler (Clang), which does support TLS. With
"BUILD: makefile: change default value of CC from gcc to cc"
that is resolved.
2020-09-27 21:48:18 +02:00
Brad Smith
94636436df BUILD: makefile: change default value of CC from gcc to cc
Change the default value of CC from gcc to cc to be more appropriate
for modern day mix of compilers. On GCC based OS's cc -> gcc. On Clang
based OS's cc -> clang. FreeBSD / OpenBSD have switched to Clang and
this corrects building with the proper compiler on OS's using Clang
as the default compiler. This especially matters for the necessity for
TLS on OpenBSD. I would expect this affects OpenMandriva and other
Linux OS's using Clang as well.
2020-09-15 07:32:06 +02:00
Ilya Shipitsin
4a034f2212 BUILD: introduce possibility to define ABORT_NOW() conditionally
code analysis tools recognize abort() better, so let us introduce
such possibility
2020-09-12 13:11:27 +02:00
Willy Tarreau
0d06df6448 MINOR: sock: introduce sock_inet and sock_unix
These files will regroup everything specific to AF_INET, AF_INET6 and
AF_UNIX socket definitions and address management. Some code there might
be agnostic to the socket type and could later move to af_xxxx.c but for
now we only support regular sockets so no need to go too far.

The files are quite poor at this step, they only contain the address
comparison function for each address family.
2020-08-28 18:51:36 +02:00
Willy Tarreau
18b7df7a2b REORG: sock: start to move some generic socket code to sock.c
The new file sock.c will contain generic code for standard sockets
relying on file descriptors. We currently have way too much duplication
between proto_uxst, proto_tcp, proto_sockpair and proto_udp.

For now only get_src, get_dst and sock_create_server_socket were moved,
and are used where appropriate.
2020-08-28 18:51:36 +02:00
Willy Tarreau
1318034317 REORG: unix: move UNIX bind/server keywords from proto_uxst.c to cfgparse-unix.c
Let's finish the cleanup and get rid of all bind and server keywords
parsers from proto_uxst.c. They're now moved to cfgparse-unix.c. Now
proto_uxst.c is clean and only contains code related to binding and
connecting.
2020-08-28 18:51:36 +02:00
Willy Tarreau
de70ca5dfd REORG: tcp: move TCP bind/server keywords from proto_tcp.c to cfgparse-tcp.c
Let's continue the cleanup and get rid of all bind and server keywords
parsers from proto_tcp.c. They're now moved to cfgparse-tcp.c, just as
was done for ssl before 2.2 release. Nothing has changed beyond this.
Now proto_tcp.c is clean and only contains code related to binding and
connecting.
2020-08-28 18:51:36 +02:00
Willy Tarreau
8987e7a8c9 REORG: tcp: move TCP sample fetches from proto_tcp.c to tcp_sample.c
Let's continue the cleanup and get rid of all sample fetch functions
from proto_tcp.c. They're now moved to tcp_sample.c, just as was done
for ssl before 2.2 release. Nothing has changed beyond this.
2020-08-28 18:51:36 +02:00
Willy Tarreau
aeae66cf22 REORG: tcp: move TCP actions from proto_tcp.c to tcp_act.c
The file proto_tcp.c has become a real mess because it still contains
tons of definitions that have nothing to do with the TCP protocol setup.
This commit moves the ruleset actions "set-src-port", "set-dst-port",
"set-src", "set-dst", and "silent-drop" to a new file "tcp_act.c".
Nothing has changed beyond this.
2020-08-28 18:51:36 +02:00
Willy Tarreau
b36dcc66ed BUILD: makefile: don't disable -Wstringop-overflow anymore
This basically reverts commit c4e6460f6 ("MINOR: build: Disable
-Wstringop-overflow.") which is no more needed after previous one.
2020-08-11 10:31:18 +02:00
Ilya Shipitsin
f21023e1cf BUILD: Makefile: require SSL_LIB, SSL_INC to be explicitly set
The SSL_INC and SSL_LIB variables were not initialized in the Makefile,
so they could be accidently inherited from the environment. We require
that any makefile variable is explicitly set on the command line so they
must be initialized.

Note that the Travis scripts used to rely only on these variables to be
exported, so it was adjusted as well.
2020-08-05 11:37:32 +02:00
Emeric Brun
3835c0dcb5 MEDIUM: udp: adds minimal proto udp support for message listeners.
This patch introduce proto_udp.c targeting a further support of
log forwarding feature.

This code was originally produced by Frederic Lecaille working on
QUIC support and only minimal requirements for syslog support
have been merged.
2020-07-15 17:50:12 +02:00
Willy Tarreau
4c442b08b3 CLEANUP: makefile: update the outdated list of DEBUG_xxx options
A few options didn't exist anymore (FSM, HASH) and quite a few ones were
added since last update (MEM_STATS, DONT_SHARE_POOLS, NO_LOCKLESS_POOLS,
NO_LOCAL_POOLS, FAIL_ALLOC, STRICT_NOCRASH, HPACK.
2020-07-04 12:43:46 +02:00
Willy Tarreau
45e83e8c56 BUILD: makefile: disable threads by default on OpenBSD
As reported by Ilya in issue #725, building with threads on OpenBSD
is broken with gcc:

  include/haproxy/tinfo.h:30: error: thread-local storage not supported for this target

Better stay safe and disable it. Clang seems to support (or emulate)
thread-local, at least it builds. Those willing to experiment can
easily pass USE_THREAD=1.
2020-07-03 18:56:33 +02:00
Willy Tarreau
739879a23b REORG: buffer: rename buffer.c to dynbuf.c
The include part was renamed by commit 2741c8c but I somehow missed
the renaming of the C file, whose name didn't match the H file anymore.
2020-06-29 09:26:59 +02:00
Willy Tarreau
38e8a1c7b8 MINOR: debug: add a new DEBUG_FD build option
When DEBUG_FD is set at build time, we'll keep a counter of per-FD events
in the fdtab. This counter is reported in "show fd" even for closed FDs if
not zero. The purpose is to help spot situations where an apparently closed
FD continues to be reported in loops, or where some events are dismissed.
2020-06-23 10:04:54 +02:00
Tim Duesterhus
588b3148d9 BUILD: Re-enable -Wimplicit-fallthrough
Getting rid of this warning is cleaner solved using a 'fall through' comment,
because it clarifies intent to a human reader.

This patch adjust a few places that cause -Wimplicit-fallthrough to trigger:

- Fix typos in the comment.
- Remove redundant 'no break' that trips up gcc from comment.
- Move the comment out of the block when the 'case' is completely surrounded
  by braces.
- Add comments where I could determine that the fall through was intentional.

Changes tested on

    gcc (Debian 9.3.0-13) 9.3.0
    Copyright (C) 2019 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

using

    make -j4 all TARGET=linux-glibc USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1
2020-06-11 16:49:37 +02:00
Tim Duesterhus
4989de2811 BUILD: Remove nowarn for warnings that do not trigger
Tested with

    make -j4 all TARGET=linux-glibc USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1

against

    gcc (Debian 9.3.0-13) 9.3.0
    Copyright (C) 2019 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
2020-06-11 15:08:27 +02:00
Willy Tarreau
7c18b54106 REORG: dgram: rename proto_udp to dgram
The set of files proto_udp.{c,h} were misleadingly named, as they do not
provide anything related to the UDP protocol but to datagram handling
instead, since currently all UDP processing is hard-coded where it's used
(dns, logs). They are to UDP what connection.{c,h} are to proto_tcp. This
was causing confusion about how to insert UDP socket management code,
so let's rename them right now to dgram.{c,h} which more accurately
matches what's inside since every function and type is already prefixed
with "dgram_".
2020-06-11 10:18:59 +02:00
Willy Tarreau
920214e8c4 BUILD: reorder objects in the Makefile for faster builds
Splitting large files and changing includes has changed the per-file
build time. After a careful reordering based on build time, we're now
down to 5.8s at -O0 on the PC at -j8 and 2.4-2.6s on the farm at -j120.
Some room for at least one file name was left on each line to ease
future additions.
2020-06-11 10:18:59 +02:00
Willy Tarreau
bcc6733fab REORG: check: extract the external checks from check.{c,h}
The health check code is ugly enough, let's take the external checks
out of it to simplify the code and shrink the file a little bit.
2020-06-11 10:18:58 +02:00
Willy Tarreau
51cd5956ee REORG: check: move tcpchecks away from check.c
Checks.c remains one of the largest file of the project and it contains
too many things. The tcpchecks code represents half of this file, and
both parts are relatively isolated, so let's move it away into its own
file. We now have tcpcheck.c, tcpcheck{,-t}.h.

Doing so required to export quite a number of functions because check.c
has almost everything made static, which really doesn't help to split!
2020-06-11 10:18:58 +02:00
Willy Tarreau
4aa573da6f REORG: include: move checks.h to haproxy/check{,-t}.h
All includes that were not absolutely necessary were removed because
checks.h happens to very often be part of dependency loops. A warning
was added about this in check-t.h. The fields, enums and structs were
a bit tidied because it's particularly tedious to find anything there.
It would make sense to split this in two or more files (at least
extract tcp-checks).

The file was renamed to the singular because it was one of the rare
exceptions to have an "s" appended to its name compared to the struct
name.
2020-06-11 10:18:58 +02:00
Willy Tarreau
48fbcae07c REORG: tools: split common/standard.h into haproxy/tools{,-t}.h
And also rename standard.c to tools.c. The original split between
tools.h and standard.h dates from version 1.3-dev and was mostly an
accident. This patch moves the files back to what they were expected
to be, and takes care of not changing anything else. However this
time tools.h was split between functions and types, because it contains
a small number of commonly used macros and structures (e.g. name_desc)
which in turn cause the massive list of includes of tools.h to conflict
with the callers.

They remain the ugliest files of the whole project and definitely need
to be cleaned and split apart. A few types are defined there only for
functions provided there, and some parts are even OS-specific and should
move somewhere else, such as the symbol resolution code.
2020-06-11 10:18:57 +02:00
Willy Tarreau
d0ef439699 REORG: include: move common/memory.h to haproxy/pool.h
Now the file is ready to be stored into its final destination. A few
minor reorderings were performed to keep the file properly organized,
making the various sections more visible (cache & lockless).

In addition and to stay consistent, memory.c was renamed to pool.c.
2020-06-11 10:18:57 +02:00
Willy Tarreau
3f567e4949 REORG: include: split hathreads into haproxy/thread.h and haproxy/thread-t.h
This splits the hathreads.h file into types+macros and functions. Given
that most users of this file used to include it only to get the definition
of THREAD_LOCAL and MAXTHREADS, the bare minimum was placed into thread-t.h
(i.e. types and macros).

All the thread management was left to haproxy/thread.h. It's worth noting
the drop of the trailing "s" in the name, to remove the permanent confusion
that arises between this one and the system implementation (no "s") and the
makefile's option (no "s").

For consistency, src/hathreads.c was also renamed thread.c.

A number of files were updated to only include thread-t which is the one
they really needed.

Some future improvements are possible like replacing empty inlined
functions with macros for the thread-less case, as building at -O0 disables
inlining and causes these ones to be emitted. But this really is cosmetic.
2020-06-11 10:18:56 +02:00
Willy Tarreau
fc80e30217 REORG: ebtree: clean up remains of the ebtree/ directory
The only leftovers were the unused compiler.h file and the LICENSE file
which is already mentioned in each and every ebtree file header.

A few build paths were updated in the contrib/ directory not to mention
this directory anymore, and all its occurrences were dropped from the
main makefile. From now on no other include path but include/ will be
needed anymore to build any file.
2020-06-11 09:31:11 +02:00
Willy Tarreau
ff0e8a44a4 REORG: ebtree: move the C files from ebtree/ to src/
As part of the include files cleanup, we're going to kill the ebtree
directory. For this we need to host its C files in a different location
and src/ is the right one.
2020-06-11 09:31:11 +02:00
William Lallemand
6a66a5ec9b REORG: ssl: move utility functions to src/ssl_utils.c
These functions are mainly used to extract information from
certificates.
2020-05-15 14:11:54 +02:00
William Lallemand
15e169447d REORG: ssl: move sample fetches to src/ssl_sample.c
Move all SSL sample fetches to src/ssl_sample.c.
2020-05-15 14:11:54 +02:00
William Lallemand
dad3105157 REORG: ssl: move ssl configuration to cfgparse-ssl.c
Move all the configuration parsing of the ssl keywords in cfgparse-ssl.c
2020-05-15 14:11:54 +02:00
William Lallemand
03c331c80a REORG: ssl: move the ckch_store related functions to src/ssl_ckch.c
Move the cert_key_and_chain functions:

int ssl_sock_load_files_into_ckch(const char *path, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_pem_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch , char **err);
void ssl_sock_free_cert_key_and_chain_contents(struct cert_key_and_chain *ckch);

int ssl_sock_load_key_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch , char **err);
int ssl_sock_load_ocsp_response_from_file(const char *ocsp_path, char *buf, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_sctl_from_file(const char *sctl_path, char *buf, struct cert_key_and_chain *ckch, char **err);
int ssl_sock_load_issuer_file_into_ckch(const char *path, char *buf, struct cert_key_and_chain *ckch, char **err);

And the utility ckch_store functions:

void ckch_store_free(struct ckch_store *store)
struct ckch_store *ckch_store_new(const char *filename, int nmemb)
struct ckch_store *ckchs_dup(const struct ckch_store *src)
ckch_store *ckchs_lookup(char *path)
ckch_store *ckchs_load_cert_file(char *path, int multi, char **err)
2020-05-15 14:11:54 +02:00
William Lallemand
6e9556b635 REORG: ssl: move crtlist functions to src/ssl_crtlist.c
Move the crtlist functions to src/ssl_crtlist.c and their definitions to
proto/ssl_crtlist.h.

The following functions were moved:

/* crt-list entry functions */
void ssl_sock_free_ssl_conf(struct ssl_bind_conf *conf);
char **crtlist_dup_filters(char **args, int fcount);
void crtlist_free_filters(char **args);
void crtlist_entry_free(struct crtlist_entry *entry);
struct crtlist_entry *crtlist_entry_new();

/* crt-list functions */
void crtlist_free(struct crtlist *crtlist);
struct crtlist *crtlist_new(const char *filename, int unique);

/* file loading */
int crtlist_parse_line(char *line, char **crt_path, struct crtlist_entry *entry, const char *file, int linenum, char **err);
int crtlist_parse_file(char *file, struct bind_conf *bind_conf, struct proxy *curproxy, struct crtlist **crtlist, char **err);
int crtlist_load_cert_dir(char *path, struct bind_conf *bind_conf, struct crtlist **crtlist, char **err);
2020-05-15 14:11:54 +02:00
Willy Tarreau
39b2fda915 BUILD: Makefile: add linux-musl to TARGET
Other users are using musl, namely on Docker. It builds fine with
linux-glibc-legacy but not linux-glibc, which needs to first disable
USE_BACKTRACE. Better add a valid entry for it instead of hacking
around another libc.
2020-04-16 15:17:13 +02:00
Willy Tarreau
48e8603a9c BUILD: makefile: fix expression again to detect ARM platform
I messed up the fix in 67b095e ("BUILD: makefile: fix regex syntax in
ARM platform detection"), I tried it by hand in the shell without "-v"
but left it in the expression. It works on ARM because it only finds
lines starting with '#' but on other platforms it insists for -latomic.
2020-03-18 08:21:57 +01:00