Commit Graph

6829 Commits

Author SHA1 Message Date
Willy Tarreau
31794892af MINOR: unix: remove the now unused proto_uxst.h file
Since everything is self contained in proto_uxst.c there's no need to
export anything. The same should be done for proto_tcp.c but the file
contains other stuff that's not related to the TCP protocol itself
and which should first be moved somewhere else.
2017-09-15 11:49:52 +02:00
Willy Tarreau
9d5be5c823 MINOR: protocols: register the ->add function and stop calling them directly
cfgparse has no business directly calling each individual protocol's 'add'
function to create a listener. Now that they're all registered, better
perform a protocol lookup on the family and have a standard ->add method
for all of them.
2017-09-15 11:49:52 +02:00
Willy Tarreau
3228238c73 MINOR: protocols: always pass a "port" argument to the listener creation
It's a shame that cfgparse() has to make special cases of each protocol
just to cast the port to the target address family. Let's pass the port
in argument to the function. The unix listener simply ignores it.
2017-09-15 11:49:52 +02:00
Willy Tarreau
20814ff1fc MINOR: frontend: don't retrieve ALPN on the critical path
It's pointless to read it on each and every accept(), as we only need
it for reporting in debugging mode a few lines later. Let's move this
part to the relevant block.
2017-09-15 11:49:27 +02:00
Willy Tarreau
04b9286933 MINOR: peers: don't reference the incoming listener on outgoing connections
Since v1.7 it's pointless to reference a listener when greating a session
for an outgoing connection, it only complicates the code. SPOE and Lua were
cleaned up in 1.8-dev1 but the peers code was forgotten. This patch fixes
this by not assigning such a listener for outgoing connections. It also has
the extra benefit of not discounting the outgoing connections from the number
of allowed incoming connections (the code currently adds a safety marging of
3 extra connections to take care of this).
2017-09-15 11:01:04 +02:00
Willy Tarreau
b780166499 BUILD: Makefile: improve detection of support for compiler warnings
Some compiler versions don't emit an error when facing an unknown
no-warning unless another error is reported, resulting in all -Wno-*
options being enabled by default and being reported as wrong with
build errors. Let's create a new "cc-nowarn" function to disable
warnings only after checking that the positive one is supported.
2017-09-14 19:05:45 +02:00
Willy Tarreau
065843c876 BUILD: Makefile: shut certain gcc/clang stupid warnings
The recent gcc and clang are utterly broken and apparently written by
people who don't use them anymore, because they emit warnings that are
impossible to disable in the code, which is the opposite of what a
warning should do. It is however possible to disable these warnings on
the command line.

This patch adds when supported :
   -Wno-format-truncation: bogus warning which is triggered on each
    snprintf() call based on the input type instead of the variables
    ranges, resulting in the impossibility to use "%02d" and similar.

   -Wno-address-of-packed-member: emitted for each and every line in
    ebtree.h by recent clang. Probably that the warning's author has
    never understood the use cases of packed structs and should be
    taught the use cases of the language he writes the compiler for.

   -Wno-null-dereference: emitted by clang on *(int *)0 = 0. The code
    will be updated to use a volatile instead but this recent change
    of behaviour will certainly cause quite some bugs in decades of
    existing code.

Feel free to report new such stupid warnings and to propose patches
to complete this list.
2017-09-13 17:10:44 +02:00
Willy Tarreau
d3a7f40359 BUILD: Makefile: add a function to detect support by the compiler of certain options
The recent gcc and clang are utterly broken and apparently written by
people who don't use them anymore, because they emit warnings that are
impossible to disable in the code, which is the opposite of what a
warning should do. It is however possible to disable these warnings on
the command line, but not in a backwards-compatible way.

Thus here we create a new function which detect if the compiler supports
certain options, and which adds them if supported.
2017-09-13 16:57:52 +02:00
Andjelko Iharos
c3680ecdf8 MINOR: add severity information to cli feedback messages 2017-09-13 13:38:32 +02:00
Andjelko Iharos
c4df59e914 MINOR: cli: add socket commands and config to prepend informational messages with severity
Adds cli commands to change at runtime whether informational messages
are prepended with severity level or not, with support for numeric and
worded severity in line with syslog severity level.

Adds stats socket config keyword severity-output to set default behavior
per socket on startup.
2017-09-13 13:37:59 +02:00
Olivier Houchard
ed0d96cac4 MINOR: net_helper: Inline functions meant to be inlined. 2017-09-13 13:35:35 +02:00
Thierry FOURNIER
d697596c6c MINOR: tasks: Move Lua notification from Lua to tasks
These notification management function and structs are generic and
it will be better to move in common parts.

The notification management functions and structs have names
containing some "lua" references because it was written for
the Lua. This patch removes also these references.
2017-09-11 18:59:40 +02:00
Thierry FOURNIER
2da788e755 MEDIUM: xref/lua: Use xref for referencing cosocket relation between stream and lua
This relation will ensure that each was informed about death of another one.
2017-09-11 18:59:40 +02:00
Thierry FOURNIER
3c65b7a916 MINOR: xref: Add a new xref system
xref is used to create a relation between two elements.
Once an element is released, it breaks the relation. If the
relation is already broken, it frees the xref struct.
The pointer between two elements is a sort of refcount with
max value 1. The relation is only between two elements.
The pointer and the type of element a and b are conventional.

Note that xref is initialised from Lua files because Lua is
the only one user.
2017-09-11 18:59:40 +02:00
Thierry FOURNIER
b01d28f976 BUG/MINOR: Lua: The socket may be destroyed when we try to access.
When we try to access to other proxy context, we must check
its existence because haproxy can kill it between the creation
and the usage.

This patch should be backported in 1.6 and 1.7
2017-09-11 18:59:40 +02:00
Christopher Faulet
5d468ca97b BUG/MEDIUM: http: Close streams for connections closed before a redirect
A previous fix was made to prevent the connection to a server if a redirect was
performed during the request processing when we wait to keep the client
connection alive. This fix introduced a pernicious bug. If a client closes its
connection immediately after sending a request, it is possible to keep stream
alive infinitely. This happens when the connection closure is caught when the
request is received, before the request parsing.

To be more specific, this happens because the close event is not "forwarded",
first because of the call to "channel_dont_connect" in the function
"http_apply_redirect_rule", then because we want to keep the client connection
alive, we explicitly call "channel_dont_close" in the function
"http_request_forward_body".

So, to fix the bug, instead of blocking the server connection, we force its
shutdown. This will force the stream to re-evaluate all connexions states. So it
will detect the client has closed its connection.

This patch must be backported in 1.7.
2017-09-11 17:39:21 +02:00
Emmanuel Hocdet
ddcde195eb MINOR: ssl: rework smp_fetch_ssl_fc_cl_str without internal ssl use
smp_fetch_ssl_fc_cl_str as very limited usage (only work with openssl == 1.0.2
compiled with the option enable-ssl-trace). It use internal cipher.algorithm_ssl
attribut and SSL_CIPHER_standard_name (available with ssl-trace).
This patch implement this (debug) function in a standard way. It used common
SSL_CIPHER_get_name to display cipher name. It work with openssl >= 1.0.2
and boringssl.
2017-09-09 08:36:22 +02:00
Willy Tarreau
3d609a755e Revert "BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file"
This reverts commit 19e8aa58f7.

It causes some trouble reported by Manu :
   listen tls
     [...]
     server bla 127.0.0.1:8080

   [ALERT] 248/130258 (21960) : parsing [/etc/haproxy/test.cfg:53] : 'server bla' : no method found to resolve address '(null)'
   [ALERT] 248/130258 (21960) : Failed to initialize server(s) addr.

According to Nenad :
  "It's not a good way to fix the issue we were experiencing
   before. It will need a bigger rewrite, because the logic in
   srv_iterate_initaddr needs to be changed."
2017-09-06 14:22:45 +02:00
Nenad Merdanovic
19e8aa58f7 BUG/MINOR: server: Remove FQDN requirement for using init-addr and state file
Historically the DNS was the only way of updating the server IP dynamically
and the init-addr processing and state file load required the server to have
an FQDN defined. Given that we can now update the IP through the socket as
well and also can have different init-addr values (like IP and 'none') - this
requirement needs to be removed.

This patch should be backported to 1.7.
2017-09-05 15:52:58 +02:00
Christopher Faulet
ab62f51959 MINOR: polling: Use fd_update_events to update events seen for a fd
Now, the same function is used by all pollers to update events seen for a
fd. This will ease the threads support integration.
2017-09-05 15:45:11 +02:00
Christopher Faulet
21e9267ac3 MINOR: fd: Add fd_update_events function
This function should be called by the poller to set FD_POLL_* flags on an FD and
update its state if needed. This function has been added to ease threads support
integration.
2017-09-05 15:43:09 +02:00
Willy Tarreau
9fab7bedfb BUG/MEDIUM: epoll: ensure we always consider HUP and ERR
Since commit 5be2f35 ("MAJOR: polling: centralize calls to I/O callbacks")
that came into 1.6-dev1, each poller deals with its own events and decides
to signal ability to receive or send on a file descriptor based on the
active events on the file descriptor.

The commit above was incorrectly done for the epoll code. Instead of
checking the active events on the fd, it checks for the new events. In
general these ones are the same for POLL_IN and POLL_OUT since they
are always cleared prior to being computed, but it is possible that
POLL_HUP and POLL_ERR were initially reported and are not reported
again (especially for HUP). This could happen for example if POLL_HUP
and POLL_IN were received together, the pending data exactly correspond
to a full buffer which is read at once, preventing the POLL_HUP from
being dealt with in the same call, and on the next call only POLL_OUT
is reported (eg: to emit some response or peers protocol ACKs). In this
case fd_may_recv() will not be enabled anymore and the close event will
be missed.

It seems quite hard to trigger this case, though it might explain some
of the rare missed close events that were detected in the past on the
peers.

This fix needs to be backported to 1.6 and 1.7.
2017-09-05 15:32:56 +02:00
Emeric Brun
52a91d3d48 MEDIUM: check: server states and weight propagation re-work
The server state and weight was reworked to handle
"pending" values updated by checks/CLI/LUA/agent.
These values are commited to be propagated to the
LB stack.

In further dev related to multi-thread, the commit
will be handled into a sync point.

Pending values are named using the prefix 'next_'
Current values used by the LB stack are named 'cur_'
2017-09-05 15:23:16 +02:00
Christopher Faulet
de2075fd21 MINOR: freq_ctr: Return the new value after an update
This will ease threads support integration.
2017-09-05 11:55:07 +02:00
Christopher Faulet
63fe65277a MINOR: fd: Move (de)allocation of fdtab and fdinfo in (de)init_pollers
This will be useful for the threads support integration.
2017-09-05 10:49:45 +02:00
Christopher Faulet
d82b180d6b MINOR: fd: Use inlined functions to check fd state in fd_*_send/recv functions
It these functions, the test is inverted and we rely on fd_recv/send_* function
to check the fd state. This will ease threads support integration.
2017-09-05 10:47:32 +02:00
Christopher Faulet
8db2fdfaba MINOR: fd: Add fd_active function
This inlined function is used to check if a fd is active for receive or send. It
will ease threads support integration.
2017-09-05 10:39:46 +02:00
Christopher Faulet
6988f678cd MINOR: http: Use a trash chunk to store decoded string of the HTTP auth header
This string is used in sample fetches so it is safe to use a preallocated trash
chunk instead of a buffer dynamically allocated during HAProxy startup.
2017-09-05 10:36:28 +02:00
Christopher Faulet
ca20d02ea8 MINOR: stick-tables: Make static_table_key a struct variable instead of a pointer
First, this variable does not need to be publicly exposed because it is only
used by stick_table functions. So we declare it as a global static in
stick_table.c file. Then, it is useless to use a pointer. Using a plain struct
variable avoids any dynamic allocation.
2017-09-05 10:35:07 +02:00
Christopher Faulet
ad405f1714 MINOR: buffers: Move swap_buffer into buffer.c and add deinit_buffer function
swap_buffer is a global variable only used by buffer_slow_realign. So it has
been moved from global.h to buffer.c and it is allocated by init_buffer
function. deinit_buffer function has been added to release it. It is also used
to destroy the buffers' pool.
2017-09-05 10:34:30 +02:00
Christopher Faulet
084aa9615b MINOR: logs: Realloc log buffers only after the config is parsed and checked
During the configuration parsing, log buffers are reallocated when
global.max_syslog_len is updated. This can be done serveral time. So, instead of
doing it serveral time, we do it only once after the configuration parsing.
2017-09-05 10:32:38 +02:00
Christopher Faulet
0132d06f68 MINOR: logs: Use dedicated function to init/deinit log buffers
Now, we use init_log_buffers and deinit_log_buffers to, respectively, initialize
and deinitialize log buffers used for syslog messages.

These functions have been introduced to be used by threads, to deal with
thread-local log buffers.
2017-09-05 10:29:31 +02:00
Christopher Faulet
3ef2639870 MEDIUM: chunks: Realloc trash buffers only after the config is parsed and checked
Trash buffers are reallocated when "tune.bufsize" parameter is changed. Here, we
just move the realloc after the configuration parsing.

Given that the config parser doesn't rely on the trash size, it should be
harmless.
2017-09-05 10:27:46 +02:00
Christopher Faulet
748919a4c7 MINOR: chunks: Use dedicated function to init/deinit trash buffers
Now, we use init_trash_buffers and deinit_trash_buffers to, respectively,
initialize and deinitialize trash buffers (trash, trash_buf1 and trash_buf2).

These functions have been introduced to be used by threads, to deal with
thread-local trash buffers.
2017-09-05 10:22:20 +02:00
Christopher Faulet
6c57dc9145 MINOR: applet: Check applets_active_queue before processing applets queue
This is useless for now, but it will allow a huge improvement when the
multithreading will be merged.
2017-09-05 10:21:29 +02:00
Christopher Faulet
8fe4891b11 MINOR: backends: Make get_server_* functions explicitly static
Not used outside.
2017-09-05 10:20:00 +02:00
Christopher Faulet
576c5aa25c MINOR: fd: Set owner and iocb field before inserting a new fd in the fdtab
This will be needed for concurrent accesses.
2017-09-05 10:17:10 +02:00
Christopher Faulet
d531f88622 MINOR: fd: Don't forget to reset fdtab[fd].update when a fd is added/removed
It used to be guaranteed by the polling functions on a later call but
with concurrent accesses it cannot be granted anymore.
2017-09-05 10:16:42 +02:00
Christopher Faulet
f5b8adc5c0 MINOR: listeners: Change enable_listener and disable_listener into private functions
These functions are only used in listener.c.
2017-09-05 10:14:16 +02:00
Christopher Faulet
5580ba2e11 MINOR: listeners: Change listener_full and limit_listener into private functions
These functions are only used in listener_accept. So there is no need to export
them.
2017-09-05 10:13:55 +02:00
Christopher Faulet
ae459fd206 CLEANUP: memory: Remove unused function pool_destroy
This one was never used.
2017-09-05 10:13:20 +02:00
Daniel Schneller
b6c8b0db04 DOC: Add note about "* " prefix in CSV stats
The check_status field in the CSV stats output is conditionally prefixed
with "* " if a check is currently underway. This can trip tools that
parse the CSV output and compare against a well known list of values.

This commit just adds this bit to the documentation.
2017-09-05 10:02:41 +02:00
Christopher Faulet
35fe699ec7 BUG/MEDIUM: http: Fix a regression bug when a HTTP response is in TUNNEL mode
Unfortunatly, a regression bug was introduced in the commit 1486b0ab
("BUG/MEDIUM: http: Switch HTTP responses in TUNNEL mode when body length is
undefined"). HTTP responses with undefined body length are blocked until timeout
when the compression is enabled. This bug was fixed in commit 69744d92
("BUG/MEDIUM: http: Fix blocked HTTP/1.0 responses when compression is
enabled").

The bug is still the same. We do not forward response data because we are
waiting for the synchronization between the HTTP request and the response.

To fix the bug, conditions to infinitly forward channel data has been slightly
relaxed. Now, it is done if there is no more analyzer registered on the channel
or if _FLT_END analyzer is still there but without the flag CF_FLT_ANALYZE. This
last condition is only possible when a channel is waiting the end of the other
side. So, fundamentally, it means that no one is analyzing the channel
anymore. This is a transitional state during a sync phase.

This patch must be backported in 1.7.
2017-09-05 10:00:58 +02:00
Emmanuel Hocdet
4366476852 MINOR: ssl: remove duplicate ssl_methods in struct bind_conf
Patch "MINOR: ssl: support ssl-min-ver and ssl-max-ver with crt-list"
introduce ssl_methods in struct ssl_bind_conf. struct bind_conf have now
ssl_methods and ssl_conf.ssl_methods (unused). It's error-prone. This patch
remove the duplicate structure to avoid any confusion.
2017-09-05 09:42:30 +02:00
Daniel Schneller
87e4302707 DOC: Refer to Mozilla TLS info / config generator
As per a recent mailing list discussion, suggesting specific cipher
settings is not too helpful, because they depend on a lot of factors,
ranging from client capabilities, available TLS libraries, new
security research, and others.
To avoid the documentation from become stale -- and potentially
wrong/dangerous, this commit adds links to Mozilla's well-reknowned
TLS blog, as well as to their configuration generator.
2017-09-01 19:54:33 +02:00
Olivier Doucet
d8703e8cd7 DOC: add CLI info on privilege levels 2017-09-01 19:02:19 +02:00
Willy Tarreau
bbae3f0170 MEDIUM: connection: remove useless flag CO_FL_DATA_WR_SH
After careful inspection, this flag is set at exactly two places :
  - once in the health-check receive callback after receipt of a
    response
  - once in the stream interface's shutw() code where CF_SHUTW is
    always set on chn->flags

The flag was checked in the checks before deciding to send data, but
when it is set, the wake() callback immediately closes the connection
so the CO_FL_SOCK_WR_SH flag is also set.

The flag was also checked in si_conn_send(), but checking the channel's
flag instead is enough and even reveals that one check involving it
could never match.

So it's time to remove this flag and replace its check with a check of
CF_SHUTW in the stream interface. This way each layer is responsible
for its shutdown, this will ease insertion of the mux layer.
2017-08-30 10:05:49 +02:00
Willy Tarreau
cde5651c4d CLEANUP: connection: remove the unused conn_sock_shutw_pending()
This has never been used anywhere.
2017-08-30 08:18:53 +02:00
Willy Tarreau
54e917cfa1 MEDIUM: connection: remove useless flag CO_FL_DATA_RD_SH
This flag is both confusing and wrong. It is supposed to report the
fact that the data layer has received a shutdown, but in fact this is
reported by CO_FL_SOCK_RD_SH which is set by the transport layer after
this condition is detected. The only case where the flag above is set
is in the stream interface where CF_SHUTR is also set on the receiving
channel.

In addition, it was checked in the health checks code (while never set)
and was always test jointly with CO_FL_SOCK_RD_SH everywhere, except in
conn_data_read0_pending() which incorrectly doesn't match the second
time it's called and is fortunately protected by an extra check on
(ic->flags & CF_SHUTR).

This patch gets rid of the flag completely. Now conn_data_read0_pending()
accurately reports the fact that the transport layer has detected the end
of the stream, regardless of the fact that this state was already consumed,
and the stream interface watches ic->flags&CF_SHUTR to know if the channel
was already closed by the upper layer (which it already used to do).

The now unused conn_data_read0() function was removed.
2017-08-30 08:18:50 +02:00
Willy Tarreau
5790eb0a76 MINOR: stream: provide a new stream creation function for connections
The purpose will be to create new streams for a given connection so
that we can later abstract this from a mux.
2017-08-30 07:06:39 +02:00