Commit Graph

6593 Commits

Author SHA1 Message Date
Dragan Dosen
2f1cacb1aa BUG/MINOR: contrib/modsecurity: close the va_list ap before return
Make sure the va_list ap is closed before return in case the function
vsnprintf() returned an error.
2017-09-18 11:18:34 +02:00
Dragan Dosen
ccf6100e11 BUG/MINOR: contrib/mod_defender: close the va_list argp before return
Fix the case when the va_list argp could be left open if the function
vsnprintf() returned a negative value.
2017-09-18 11:18:09 +02:00
Christopher Faulet
f8bb0ce450 MINOR: ssl: Remove useless checks on bind_conf or bind_conf->is_ssl
bind_conf always exists at these steps and it is always for SSL listeners.
2017-09-15 18:42:23 +02:00
Christopher Faulet
3bbd65b23e BUG/MINOR: dns: Fix check on nameserver in snr_resolution_cb
snr_resolution_cb can be called with <nameserver> parameter set to NULL. So we
must check it before using it. This is done most of time, except when we deal
with invalid DNS response.
2017-09-15 18:42:23 +02:00
Christopher Faulet
ccbc3fd9f9 BUG/MINOR: spoe: Don't rely on SPOE ctx in debug message when its creation failed
If the SPOE context creation failed, we must not try to use it in the debug
message used to notice the error.

This patch must be backported in 1.7.
2017-09-15 18:42:23 +02:00
Christopher Faulet
3dc860d19d BUG/MINOR: compression: Check response headers before http-response rules eval
This is required if we want to use res.comp or res.comp_algo sample fetches in
http-response rules.

This patch must be backported in 1.7.
2017-09-15 18:42:23 +02:00
Christopher Faulet
03d85538b3 BUG/MEDIUM: compression: Fix check on txn in smp_fetch_res_comp_algo
The check was totally messed up. In the worse case, it led to a crash, when
res.comp_algo sample fetch was retrieved on uncompressed response (with the
compression enabled).

This patch must be backported in 1.7.
2017-09-15 18:42:23 +02:00
Willy Tarreau
0bf6fa5e40 MEDIUM: session: count the frontend's connections at a single place
There are several places where we see feconn++, feconn--, totalconn++ and
an increment on the frontend's number of connections and connection rate.
This is done exactly once per session in each direction, so better take
care of this counter in the session and simplify the callers. At least it
ensures a better symmetry. It also ensures consistency as till now the
lua/spoe/peers frontend didn't have these counters properly set, which can
be useful at least for troubleshooting.
2017-09-15 11:49:52 +02:00
Willy Tarreau
0c4ed35225 MEDIUM: session: factor out duplicated code for conn_complete_session
session_accept_fd() may either successfully complete a session creation,
or defer it to conn_complete_session() depending of whether a handshake
remains to be performed or not. The problem is that all the code after
the handshake was duplicated between the two functions.

This patch make session_accept_fd() synchronously call
conn_complete_session() to finish the session creation. It is only needed
to check if the session's task has to be released or not at the end, which
is fairly minimal. This way there is now a single place where the sessions
are created.
2017-09-15 11:49:52 +02:00
Willy Tarreau
eaa7e44ad7 MINOR: session: small cleanup of conn_complete_session()
Commit 8e3c6ce ("MEDIUM: connection: get rid of data->init() which was
not for data") simplified conn_complete_session() but introduced a
confusing check which cannot happen on CO_FL_HANDSHAKE. Make it clear
that this call is final and will either succeed and complete the
session or fail.
2017-09-15 11:49:52 +02:00
Willy Tarreau
05f5047d40 MINOR: listener: new function listener_release
Instead of duplicating some sensitive listener-specific code in the
session and in the stream code, let's call listener_release() when
releasing a connection attached to a listener.
2017-09-15 11:49:52 +02:00
Willy Tarreau
6f5e4b98df MEDIUM: session: take care of incrementing/decrementing jobs
Each user of a session increments/decrements the jobs variable at its
own place, resulting in a real mess and inconsistencies between them.
Let's have session_new() increment jobs and session_free() decrement
it.
2017-09-15 11:49:52 +02:00
Willy Tarreau
2cc5bae0b8 MINOR: listeners: make listeners count consistent with reality
Some places call delete_listener() then decrement the number of
listeners and jobs. At least one other place calls delete_listener()
without doing so, but since it's in deinit(), it's harmless and cannot
risk to cause zombie processes to survive. Given that the number of
listeners and jobs is incremented when creating the listeners, it's
much more logical to symmetrically decrement them when deleting such
listeners.
2017-09-15 11:49:52 +02:00
Willy Tarreau
0de59fd53a MINOR: listeners: new function create_listeners
This function is used to create a series of listeners for a specific
address and a port range. It automatically calls the matching protocol
handlers to add them to the relevant lists. This way cfgparse doesn't
need to manipulate listeners anymore. As an added bonus, the memory
allocation is checked.
2017-09-15 11:49:52 +02:00
Willy Tarreau
31794892af MINOR: unix: remove the now unused proto_uxst.h file
Since everything is self contained in proto_uxst.c there's no need to
export anything. The same should be done for proto_tcp.c but the file
contains other stuff that's not related to the TCP protocol itself
and which should first be moved somewhere else.
2017-09-15 11:49:52 +02:00
Willy Tarreau
9d5be5c823 MINOR: protocols: register the ->add function and stop calling them directly
cfgparse has no business directly calling each individual protocol's 'add'
function to create a listener. Now that they're all registered, better
perform a protocol lookup on the family and have a standard ->add method
for all of them.
2017-09-15 11:49:52 +02:00
Willy Tarreau
3228238c73 MINOR: protocols: always pass a "port" argument to the listener creation
It's a shame that cfgparse() has to make special cases of each protocol
just to cast the port to the target address family. Let's pass the port
in argument to the function. The unix listener simply ignores it.
2017-09-15 11:49:52 +02:00
Willy Tarreau
20814ff1fc MINOR: frontend: don't retrieve ALPN on the critical path
It's pointless to read it on each and every accept(), as we only need
it for reporting in debugging mode a few lines later. Let's move this
part to the relevant block.
2017-09-15 11:49:27 +02:00
Willy Tarreau
04b9286933 MINOR: peers: don't reference the incoming listener on outgoing connections
Since v1.7 it's pointless to reference a listener when greating a session
for an outgoing connection, it only complicates the code. SPOE and Lua were
cleaned up in 1.8-dev1 but the peers code was forgotten. This patch fixes
this by not assigning such a listener for outgoing connections. It also has
the extra benefit of not discounting the outgoing connections from the number
of allowed incoming connections (the code currently adds a safety marging of
3 extra connections to take care of this).
2017-09-15 11:01:04 +02:00
Willy Tarreau
b780166499 BUILD: Makefile: improve detection of support for compiler warnings
Some compiler versions don't emit an error when facing an unknown
no-warning unless another error is reported, resulting in all -Wno-*
options being enabled by default and being reported as wrong with
build errors. Let's create a new "cc-nowarn" function to disable
warnings only after checking that the positive one is supported.
2017-09-14 19:05:45 +02:00
Willy Tarreau
065843c876 BUILD: Makefile: shut certain gcc/clang stupid warnings
The recent gcc and clang are utterly broken and apparently written by
people who don't use them anymore, because they emit warnings that are
impossible to disable in the code, which is the opposite of what a
warning should do. It is however possible to disable these warnings on
the command line.

This patch adds when supported :
   -Wno-format-truncation: bogus warning which is triggered on each
    snprintf() call based on the input type instead of the variables
    ranges, resulting in the impossibility to use "%02d" and similar.

   -Wno-address-of-packed-member: emitted for each and every line in
    ebtree.h by recent clang. Probably that the warning's author has
    never understood the use cases of packed structs and should be
    taught the use cases of the language he writes the compiler for.

   -Wno-null-dereference: emitted by clang on *(int *)0 = 0. The code
    will be updated to use a volatile instead but this recent change
    of behaviour will certainly cause quite some bugs in decades of
    existing code.

Feel free to report new such stupid warnings and to propose patches
to complete this list.
2017-09-13 17:10:44 +02:00
Willy Tarreau
d3a7f40359 BUILD: Makefile: add a function to detect support by the compiler of certain options
The recent gcc and clang are utterly broken and apparently written by
people who don't use them anymore, because they emit warnings that are
impossible to disable in the code, which is the opposite of what a
warning should do. It is however possible to disable these warnings on
the command line, but not in a backwards-compatible way.

Thus here we create a new function which detect if the compiler supports
certain options, and which adds them if supported.
2017-09-13 16:57:52 +02:00
Andjelko Iharos
c3680ecdf8 MINOR: add severity information to cli feedback messages 2017-09-13 13:38:32 +02:00
Andjelko Iharos
c4df59e914 MINOR: cli: add socket commands and config to prepend informational messages with severity
Adds cli commands to change at runtime whether informational messages
are prepended with severity level or not, with support for numeric and
worded severity in line with syslog severity level.

Adds stats socket config keyword severity-output to set default behavior
per socket on startup.
2017-09-13 13:37:59 +02:00
Olivier Houchard
ed0d96cac4 MINOR: net_helper: Inline functions meant to be inlined. 2017-09-13 13:35:35 +02:00
Thierry FOURNIER
d697596c6c MINOR: tasks: Move Lua notification from Lua to tasks
These notification management function and structs are generic and
it will be better to move in common parts.

The notification management functions and structs have names
containing some "lua" references because it was written for
the Lua. This patch removes also these references.
2017-09-11 18:59:40 +02:00
Thierry FOURNIER
2da788e755 MEDIUM: xref/lua: Use xref for referencing cosocket relation between stream and lua
This relation will ensure that each was informed about death of another one.
2017-09-11 18:59:40 +02:00
Thierry FOURNIER
3c65b7a916 MINOR: xref: Add a new xref system
xref is used to create a relation between two elements.
Once an element is released, it breaks the relation. If the
relation is already broken, it frees the xref struct.
The pointer between two elements is a sort of refcount with
max value 1. The relation is only between two elements.
The pointer and the type of element a and b are conventional.

Note that xref is initialised from Lua files because Lua is
the only one user.
2017-09-11 18:59:40 +02:00
Thierry FOURNIER
b01d28f976 BUG/MINOR: Lua: The socket may be destroyed when we try to access.
When we try to access to other proxy context, we must check
its existence because haproxy can kill it between the creation
and the usage.

This patch should be backported in 1.6 and 1.7
2017-09-11 18:59:40 +02:00
Christopher Faulet
5d468ca97b BUG/MEDIUM: http: Close streams for connections closed before a redirect
A previous fix was made to prevent the connection to a server if a redirect was
performed during the request processing when we wait to keep the client
connection alive. This fix introduced a pernicious bug. If a client closes its
connection immediately after sending a request, it is possible to keep stream
alive infinitely. This happens when the connection closure is caught when the
request is received, before the request parsing.

To be more specific, this happens because the close event is not "forwarded",
first because of the call to "channel_dont_connect" in the function
"http_apply_redirect_rule", then because we want to keep the client connection
alive, we explicitly call "channel_dont_close" in the function
"http_request_forward_body".

So, to fix the bug, instead of blocking the server connection, we force its
shutdown. This will force the stream to re-evaluate all connexions states. So it
will detect the client has closed its connection.

This patch must be backported in 1.7.
2017-09-11 17:39:21 +02:00
Emmanuel Hocdet
ddcde195eb MINOR: ssl: rework smp_fetch_ssl_fc_cl_str without internal ssl use
smp_fetch_ssl_fc_cl_str as very limited usage (only work with openssl == 1.0.2
compiled with the option enable-ssl-trace). It use internal cipher.algorithm_ssl
attribut and SSL_CIPHER_standard_name (available with ssl-trace).
This patch implement this (debug) function in a standard way. It used common
SSL_CIPHER_get_name to display cipher name. It work with openssl >= 1.0.2
and boringssl.
2017-09-09 08:36:22 +02:00
Willy Tarreau
3d609a755e Revert "BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file"
This reverts commit 19e8aa58f7.

It causes some trouble reported by Manu :
   listen tls
     [...]
     server bla 127.0.0.1:8080

   [ALERT] 248/130258 (21960) : parsing [/etc/haproxy/test.cfg:53] : 'server bla' : no method found to resolve address '(null)'
   [ALERT] 248/130258 (21960) : Failed to initialize server(s) addr.

According to Nenad :
  "It's not a good way to fix the issue we were experiencing
   before. It will need a bigger rewrite, because the logic in
   srv_iterate_initaddr needs to be changed."
2017-09-06 14:22:45 +02:00
Nenad Merdanovic
19e8aa58f7 BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file
Historically the DNS was the only way of updating the server IP dynamically
and the init-addr processing and state file load required the server to have
an FQDN defined. Given that we can now update the IP through the socket as
well and also can have different init-addr values (like IP and 'none') - this
requirement needs to be removed.

This patch should be backported to 1.7.
2017-09-05 15:52:58 +02:00
Christopher Faulet
ab62f51959 MINOR: polling: Use fd_update_events to update events seen for a fd
Now, the same function is used by all pollers to update events seen for a
fd. This will ease the threads support integration.
2017-09-05 15:45:11 +02:00
Christopher Faulet
21e9267ac3 MINOR: fd: Add fd_update_events function
This function should be called by the poller to set FD_POLL_* flags on an FD and
update its state if needed. This function has been added to ease threads support
integration.
2017-09-05 15:43:09 +02:00
Willy Tarreau
9fab7bedfb BUG/MEDIUM: epoll: ensure we always consider HUP and ERR
Since commit 5be2f35 ("MAJOR: polling: centralize calls to I/O callbacks")
that came into 1.6-dev1, each poller deals with its own events and decides
to signal ability to receive or send on a file descriptor based on the
active events on the file descriptor.

The commit above was incorrectly done for the epoll code. Instead of
checking the active events on the fd, it checks for the new events. In
general these ones are the same for POLL_IN and POLL_OUT since they
are always cleared prior to being computed, but it is possible that
POLL_HUP and POLL_ERR were initially reported and are not reported
again (especially for HUP). This could happen for example if POLL_HUP
and POLL_IN were received together, the pending data exactly correspond
to a full buffer which is read at once, preventing the POLL_HUP from
being dealt with in the same call, and on the next call only POLL_OUT
is reported (eg: to emit some response or peers protocol ACKs). In this
case fd_may_recv() will not be enabled anymore and the close event will
be missed.

It seems quite hard to trigger this case, though it might explain some
of the rare missed close events that were detected in the past on the
peers.

This fix needs to be backported to 1.6 and 1.7.
2017-09-05 15:32:56 +02:00
Emeric Brun
52a91d3d48 MEDIUM: check: server states and weight propagation re-work
The server state and weight was reworked to handle
"pending" values updated by checks/CLI/LUA/agent.
These values are commited to be propagated to the
LB stack.

In further dev related to multi-thread, the commit
will be handled into a sync point.

Pending values are named using the prefix 'next_'
Current values used by the LB stack are named 'cur_'
2017-09-05 15:23:16 +02:00
Christopher Faulet
de2075fd21 MINOR: freq_ctr: Return the new value after an update
This will ease threads support integration.
2017-09-05 11:55:07 +02:00
Christopher Faulet
63fe65277a MINOR: fd: Move (de)allocation of fdtab and fdinfo in (de)init_pollers
This will be useful for the threads support integration.
2017-09-05 10:49:45 +02:00
Christopher Faulet
d82b180d6b MINOR: fd: Use inlined functions to check fd state in fd_*_send/recv functions
It these functions, the test is inverted and we rely on fd_recv/send_* function
to check the fd state. This will ease threads support integration.
2017-09-05 10:47:32 +02:00
Christopher Faulet
8db2fdfaba MINOR: fd: Add fd_active function
This inlined function is used to check if a fd is active for receive or send. It
will ease threads support integration.
2017-09-05 10:39:46 +02:00
Christopher Faulet
6988f678cd MINOR: http: Use a trash chunk to store decoded string of the HTTP auth header
This string is used in sample fetches so it is safe to use a preallocated trash
chunk instead of a buffer dynamically allocated during HAProxy startup.
2017-09-05 10:36:28 +02:00
Christopher Faulet
ca20d02ea8 MINOR: stick-tables: Make static_table_key a struct variable instead of a pointer
First, this variable does not need to be publicly exposed because it is only
used by stick_table functions. So we declare it as a global static in
stick_table.c file. Then, it is useless to use a pointer. Using a plain struct
variable avoids any dynamic allocation.
2017-09-05 10:35:07 +02:00
Christopher Faulet
ad405f1714 MINOR: buffers: Move swap_buffer into buffer.c and add deinit_buffer function
swap_buffer is a global variable only used by buffer_slow_realign. So it has
been moved from global.h to buffer.c and it is allocated by init_buffer
function. deinit_buffer function has been added to release it. It is also used
to destroy the buffers' pool.
2017-09-05 10:34:30 +02:00
Christopher Faulet
084aa9615b MINOR: logs: Realloc log buffers only after the config is parsed and checked
During the configuration parsing, log buffers are reallocated when
global.max_syslog_len is updated. This can be done serveral time. So, instead of
doing it serveral time, we do it only once after the configuration parsing.
2017-09-05 10:32:38 +02:00
Christopher Faulet
0132d06f68 MINOR: logs: Use dedicated function to init/deinit log buffers
Now, we use init_log_buffers and deinit_log_buffers to, respectively, initialize
and deinitialize log buffers used for syslog messages.

These functions have been introduced to be used by threads, to deal with
thread-local log buffers.
2017-09-05 10:29:31 +02:00
Christopher Faulet
3ef2639870 MEDIUM: chunks: Realloc trash buffers only after the config is parsed and checked
Trash buffers are reallocated when "tune.bufsize" parameter is changed. Here, we
just move the realloc after the configuration parsing.

Given that the config parser doesn't rely on the trash size, it should be
harmless.
2017-09-05 10:27:46 +02:00
Christopher Faulet
748919a4c7 MINOR: chunks: Use dedicated function to init/deinit trash buffers
Now, we use init_trash_buffers and deinit_trash_buffers to, respectively,
initialize and deinitialize trash buffers (trash, trash_buf1 and trash_buf2).

These functions have been introduced to be used by threads, to deal with
thread-local trash buffers.
2017-09-05 10:22:20 +02:00
Christopher Faulet
6c57dc9145 MINOR: applet: Check applets_active_queue before processing applets queue
This is useless for now, but it will allow a huge improvement when the
multithreading will be merged.
2017-09-05 10:21:29 +02:00
Christopher Faulet
8fe4891b11 MINOR: backends: Make get_server_* functions explicitly static
Not used outside.
2017-09-05 10:20:00 +02:00