Commit Graph

5289 Commits

Author SHA1 Message Date
Willy Tarreau
25f1310f33 MINOR: stream-int: implement a new stream_int_update() function
This function is designed to be called from within the stream handler to
update the channels' expiration timers and the stream interface's flags
based on the channels' flags. It needs to be called only once after the
channels' flags have settled down, and before they are cleared, though it
doesn't harm to call it as often as desired (it just slightly hurts
performance). It must not be called from outside of the stream handler,
as what it does will be used to compute the stream task's expiration.

The code was taken directly from stream_int_update_applet() and
stream_int_update_conn() which had exactly the same one except for
applet-specific or connection-specific status update.
2015-09-25 18:07:16 +02:00
Willy Tarreau
2f4e702031 MEDIUM: stream-int: split stream_int_update_conn() into si- and conn-specific parts
The purpose is to separate the connection-specific parts so that the
stream-int specific one can be factored out. There's no functional
change here, only code displacement.
2015-09-25 18:07:16 +02:00
Willy Tarreau
9994238adc BUG/MAJOR: applet: use a separate run queue to maintain list integrity
If an applet wakes up and causes the next one to sleep, the active list
is corrupted and cannot be scanned anymore, as the process then loops
over the next element.

In order to avoid this problem, we move the active applet list to a run
queue and reinit the active list. Only the first element of this queue
is checked, and if the element is not removed, it is removed and requeued
into the active list.

Since we're using a distinct list, if an applet wants to requeue another
applet into the active list, it properly gets added to the active list
and not to the run queue.

This stops the infinite loop issue that could be caused with Lua applets,
and in any future configuration where two applets could be attached
together.
2015-09-25 18:07:16 +02:00
Willy Tarreau
64bca9d36a MINOR: applet: rename applet_runq to applet_active_queue
This is not a real run queue and we're facing ugly bugs because
if this : if a an applet removes another applet from the queue,
typically the next one after itself, the list iterator loops
forever because the list's backup pointer is not valid anymore.
Before creating a run queue, let's rename this list.
2015-09-25 18:02:44 +02:00
Willy Tarreau
9bb49f6906 BUG/MEDIUM: acl: always accept match "found"
The pattern match "found" fails to parse on binary type samples. The
reason is that it presents itself as an integer type which bin cannot
be cast into. We must always accept this match since it validates
anything on input regardless of the type. Let's just relax the parser
to accept it.

This problem might also exist in 1.5.
(cherry picked from commit 91cc2368a73198bddc3e140d267cce4ee08cf20e)
2015-09-24 16:38:48 +02:00
Willy Tarreau
d7bdcb874b BUG/MEDIUM: payload: make req.payload and payload_lv aware of dynamic buffers
Due to a check between offset+len and buf->size, an empty buffer returns
"will never match". Check against tune.bufsize instead.
(cherry picked from commit 43e4039fd5d208fd9d32157d20de93d3ddf9bc0d)
2015-09-24 16:38:48 +02:00
Willy Tarreau
c4b56e4470 MINOR: stream-int: use si_release_endpoint() to close idle conns
We don't want to open-code the connection close code in
si_idle_conn_wake_cb() because we need to centralize some controls.
2015-09-24 11:57:34 +02:00
Thierry FOURNIER
8255a75e08 MINOR: lua: change actions registration
The current Lua action are not registered. The executed function is
selected according with a function name writed in the HAProxy configuration.

This patch add an action registration function. The configuration mode
described above disappear.

This change make some incompatibilities with existing configuration files for
HAProxy 1.6-dev.
2015-09-23 21:44:23 +02:00
Thierry FOURNIER
85c6c97830 MINOR: action: add reference to the original keywork matched for the called parser.
This is usefull because the keyword can contains some condifiguration
data set while the keyword registration.
2015-09-23 21:44:23 +02:00
Thierry FOURNIER
7ea160cb5f MINOR: action: add private configuration
This private configuration pointer is used for storing some configuration
data associated the keyword, So many keywords can use the same parse
function, and this one can use a discriminator.
2015-09-23 21:44:23 +02:00
Willy Tarreau
2c1068cb67 BUG/MEDIUM: stream: do not dereference strm_li(stream)
Some streams do not have a listener (eg: Lua's cosockets) so
let's check for this. For now this problem cannot happen but
it's definitely unsafe.
2015-09-23 13:42:08 +02:00
Willy Tarreau
b746329dc3 BUG/MEDIUM: proxy: do not dereference strm_li(stream)
Some streams do not have a listener (eg: Lua's cosockets) so
let's check for this. For now this problem cannot happen but
it's definitely unsafe.
2015-09-23 13:42:08 +02:00
Willy Tarreau
c29d0cda4b BUG/MEDIUM: http: do not dereference strm_li(stream)
Some streams do not have a listener (eg: Lua's cosockets) so
let's check for this. For now this problem cannot happen but
it's definitely unsafe.
2015-09-23 13:42:08 +02:00
Willy Tarreau
f1e0212d54 BUG/MAJOR: cli: do not dereference strm_li()->proto->name
Or it will crash when dumping streams instanciated by an applet,
like Lua's cosockets.
2015-09-23 13:42:08 +02:00
Willy Tarreau
0fd99094c3 CLEANUP: stream-int: remove obsolete function si_applet_call()
This one is not used anymore and is bogus due to the way applets
now work. Remove it instead of fixing it before someone finds it
fun to use it.
2015-09-23 13:42:08 +02:00
Willy Tarreau
ed17655666 DOC: add more entries to MAINTAINERS
Simon is willing to maintain mailers and external checks.
2015-09-23 07:41:56 +02:00
Emeric Brun
b058f1c548 BUG/MINOR: fct peer_prepare_ackmsg should not use trash.
function 'peer_prepare_ackmsg' is designed to use the argument 'msg'
instead of 'trash.str'.

There is currently no bug because the caller passes 'trash.str' in
the 'msg' argument.
2015-09-22 16:07:34 +02:00
Emeric Brun
a6a0998529 BUG/MEDIUM: peers: same table updates re-pushed after a re-connect
Some updates are pushed using an incremental update message after a
re-connection whereas the origin is forgotten by the peer.
These updates are never correctly acknowledged. So they are regularly
re-pushed after an idle timeout and a re-connect.

The fix consists to use an absolute update message in some cases.
2015-09-22 16:07:32 +02:00
Emeric Brun
c703a9d296 BUG/MEDIUM: peers: some table updates are randomly not pushed.
If an entry is still not present in the update tree, we could miss to schedule
for a push depending of an un-initialized value (upd.key remains un-initialized
for new sessions or isn't re-initalized for reused ones).

In the same way, if an entry is present in the tree, but its update's tick
is far in the past (> 2^31). We could consider it's still scheduled even if
it is not the case.

The fix consist to force the re-scheduling of an update if it was not present in
the updates tree or if the update is not in the scheduling window of every peers.
2015-09-22 16:07:27 +02:00
Baptiste Assmann
8c62c47cb2 BUG: dns: can't connect UDP socket on FreeBSD
PiBANL reported that HAProxy's DNS resolver can't "connect" its socker
on FreeBSD.
Remi Gacogne reported that we should use the function 'get_addr_len' to
get the addr structure size instead of sizeof.
2015-09-22 16:06:41 +02:00
Willy Tarreau
a9937f9519 DOC: add more entries to MAINTAINERS
Cyril is the most impacted by doc format changes, Thierry is the one who
knows maps and patterns infrastructure in great details.
2015-09-22 01:08:15 +02:00
Willy Tarreau
f7ead61388 BUG/MINOR: args: add name for ARGT_VAR
Commit 4834bc7 ("MEDIUM: vars: adds support of variables") introduced
ARGT_VAR but forgot to put it in the names array. No backport needed.
2015-09-21 20:57:12 +02:00
Willy Tarreau
a68f7629dd BUG/MEDIUM: stick-tables: fix double-decrement of tracked entries
Mailing list participant "mlist" reported negative conn_cur values in
stick tables as the result of "tcp-request connection track-sc". The
reason is that after the stick entry it copied from the session to the
stream, both the session and the stream grab a reference to the entry
and when the stream ends, it decrements one reference and one connection,
then the same is done for the session.

In fact this problem was already encountered slightly differently in the
past and addressed by Thierry using the patch below as it was believed by
then to be only a refcount issue since it was the observable symptom :

   827752e "BUG/MEDIUM: stick-tables: refcount error after copying SC..."

In reality the problem is that the stream must touch neither the refcount
nor the connection count for entries it inherits from the session. While
we have no way to tell whether a track entry was inherited from the session
(since they're simply memcpy'd), it is possible to prevent the stream from
touching an entry that already exists in the session because that's a
guarantee that it was inherited from it.

Note that it may be a temporary fix. Maybe in the future when a session
gives birth to multiple streams we'll face a situation where a session may
be updated to add more tracked entries after instanciating some streams.
The correct long-term fix is to mark some tracked entries as shared or
private (or RO/RW). That will allow the session to track more entries
even after the same trackers are being used by early streams.

No backport is needed, this is only caused by the session/stream split in 1.6.
2015-09-21 17:48:24 +02:00
Willy Tarreau
8f1b35b383 DOC: update coding-style to reference checkpatch.pl
Running the Linux kernel's checkpatch.pl is actually quite efficient
at spotting style issues and even sometimes bugs. The doc now suggests
how to use it to avoid the warnings that are specific to Linux's stricter
rules.

It properly reports errors like the following ones that were found on
real submissions so it should improve the situation for everyone :

ERROR: "foo * bar" should be "foo *bar"
+static char * tcpcheck_get_step_comment(struct check *, int);

ERROR: do not use assignment in if condition
+                       if ((comment = tcpcheck_get_step_comment(check, step)))

WARNING: trailing semicolon indicates no statements, indent implies otherwise
+                       if (elem->data && elem->free);
+                               elem->free(elem->data);

ERROR: do not initialise statics to 0 or NULL
+static struct lru64_head *ssl_ctx_lru_tree = NULL;

ERROR: space required after that ',' (ctx:VxV)
+           !X509_gmtime_adj(X509_get_notAfter(newcrt),(long)60*60*24*365))
                                                      ^
WARNING: space prohibited between function name and open parenthesis '('
+       else if (EVP_PKEY_type (capkey->type) == EVP_PKEY_RSA)

ERROR: trailing statements should be on next line
+       if (cacert) X509_free(cacert);

ERROR: space prohibited after that open parenthesis '('
+                                       !(         (srv_op_state == SRV_ST_STOPPED)
2015-09-21 16:45:45 +02:00
Willy Tarreau
37bb7be09c BUG/MAJOR: peers: fix a crash when stopping peers on unbound processes
Pradeep Jindal reported and troubleshooted a bug causing haproxy to die
during startup on all processes not making use of a peers section. It
only happens with nbproc > 1 when peers are declared. Technically it's
when the peers task is stopped on processes that don't use it that the
crash occurred (a task_free() called on a NULL task pointer).

This only affects peers v2 in the dev branch, no backport is needed.
2015-09-21 15:24:58 +02:00
Willy Tarreau
14a923cd2e DOC: add a MAINTAINERS file
This one indicates who to CC when sending changes affecting certain
parts of the product.
2015-09-21 15:24:34 +02:00
James Rosewell
63426cb6a6 MINOR: 51d: Improved string handling for LRU cache
Removed use of strlen with the data added to and retrived from the cache
by using chunk structures instead of string pointers.
2015-09-21 12:55:24 +02:00
James Rosewell
a28bbd53c4 MAJOR: 51d: Upgraded to support 51Degrees V3.2 and new features
Trie device detection doesn't benefit from caching compared to Pattern.
As such the LRU cache has been removed from the Trie method.

A new fetch  method has been added named 51d.all which uses all the
available HTTP headers for device device detection. The previous 51d
conv method has been changed to 51d.single where one HTTP header,
typically User-Agent, is used for detection. This method is marginally
faster but less accurate.

Three new properties are available with the Pattern method called
Method, Difference and Rank which provide insight into the validity of
the results returned.

A pool of worksets is used to avoid needing to create a new workset for
every request. The workset pool is thread safe ready to support a future
multi threaded version of HAProxy.
2015-09-21 12:44:59 +02:00
James Rosewell
3670eb1d74 BUILD: Changed 51Degrees option to support V3.2
Added support for city hash method, turned off multi threading support
and included maths library. Removed reference to compression library
which was never needed.
2015-09-21 12:14:11 +02:00
James Rosewell
a0c4c69b67 DOC: Added more explanation for 51Degrees V3.2
Changed examples to demonstrate the the new fetch and conv methods
available with the enhancements made in version 3.2 of 51Degrees.

Added reference to the accuracy indicators available with Pattern
detection method.
2015-09-21 12:13:50 +02:00
James Rosewell
10790964b0 MINOR: global: Added new fields for 51Degrees device detection
Added support for version 3.2 of 51Degrees C library.

Added fields to store HTTP header names important to device detection
other than User-Agent.

Included a pool of worksets for use with Pattern device detection.
2015-09-21 12:10:41 +02:00
James Rosewell
91a41cb32d MINOR: http: made CHECK_HTTP_MESSAGE_FIRST accessible to other functions
Added the definition of CHECK_HTTP_MESSAGE_FIRST and the declaration of
smp_prefetch_http to the header.

Changed smp_prefetch_http implementation to remove the static qualifier.
2015-09-21 12:05:26 +02:00
Willy Tarreau
11e334d972 DOC: add a CONTRIBUTING file
This file tries to explain in the most detailed way how to contribute
patches. A few parts of it were moved from the README. .gitignore was
updated.
2015-09-20 22:43:56 +02:00
Willy Tarreau
2f5cd60ed0 DOC: add the documentation about internal circular lists
This file was recovered from the first project where it was born 12 years
ago, but it's still convenient to understand how our circular lists work,
so let's add it.
2015-09-20 22:43:56 +02:00
Baptiste Assmann
71503d32a7 DOC: servers state seamless reload example
A short doc + example to help people start using seamless reload for
server state
2015-09-19 17:05:29 +02:00
Baptiste Assmann
9b6857e9b5 MINOR: cli: new stats socket command: show backend
new stats socket command which displays only the list of backends
available in the current process.
For now only the backend name is displayed.
2015-09-19 17:05:29 +02:00
Baptiste Assmann
6076d1c02d MINOR: server: startup slowstart task when using seamless reload of HAProxy
This patch uses the start up of the health check task to also start
the warmup task when required.

This is executed only once: when HAProxy has just started up and can
be started only if the load-server-state-from-file feature is enabled
and the server was in the warmup state before a reload occurs.
2015-09-19 17:05:28 +02:00
Baptiste Assmann
fecd2b53af MINOR: init: server state loaded from file
With this patch, HAProxy reads the content of server state file and
update state of servers accordingly.
2015-09-19 17:05:28 +02:00
Baptiste Assmann
01c6cc3670 DOC: load-server-state-from-file 2015-09-19 17:05:28 +02:00
Baptiste Assmann
e11cfcd2c9 MINOR: config: new backend directives: load-server-state-from-file and server-state-file-name
This directive gives HAProxy the ability to use the either the global
server-state-file directive or a local one using server-state-file-name to
load server states.
The state can be saved right before the reload by the init script, using
the "show servers state" command on the stats socket redirecting output into
a file.
2015-09-19 17:05:28 +02:00
Baptiste Assmann
ef1f0fc033 DOC: new global directive: server-state-file
Documentation related to a new global directive.
Purpose of this directive is to store a file path into the global
structure of HAProxy. The file pointed by the path may be used by
HAProxy to retrieve server state from the previous running process
after a reload occured.
2015-09-19 17:05:27 +02:00
Baptiste Assmann
e0882263e0 MINOR: config: new global section directive: server-state-file
This new global section directive is used to store the path to the file
where HAProxy will be able to retrieve server states across reloads.

The file pointed by this path is used to store a file which can contains
state of all servers from all backends.
2015-09-19 17:05:27 +02:00
Baptiste Assmann
5626f489ec DOC: global directive server-state-base
Documentation relative to the new global directive server-state-file.
2015-09-19 17:05:27 +02:00
Baptiste Assmann
6bc89366bb MINOR: config: new global directive server-state-base
This new global directive can be used to provide a base directory where
all the server state files could be loaded.
If a server state file name starts with a slash '/', then this directive
must not be applied.
2015-09-19 17:05:26 +02:00
Baptiste Assmann
6f9225ab55 DOC: stats socket command: show servers state
Documentation related to the new command available over the stats socket
"show servers state".

A description of the fields is also included.
2015-09-19 17:05:26 +02:00
Baptiste Assmann
2828946cb5 MINOR: cli: new stats socket command: show servers state
new command 'show servers state' which dumps all variable parameters
of a server during an HAProxy process life.
Purpose is to dump current server state at current run time in order to
read them right after the reload.

The format of the output is versionned and we support version 1 for now.
2015-09-19 16:52:46 +02:00
Baptiste Assmann
41472f4d98 MINOR: server: Macro definition for server-state
Introduces a few new macros used by server state save and application accros reloads:
- currently used state server file format version
- currently used state server file header fields
- MIN and MAX value for version number
- maximum number of fields that could be found in a server-state file
- an arbitrary state-file max line length
2015-09-19 11:34:41 +02:00
Baptiste Assmann
54a4730c65 BUG/MAJOR: can't enable a server through the stat socket
When a server is disabled in the configuration using the "disabled"
keyword, a single flag is positionned: SRV_ADMF_CMAINT (use to be
SRV_ADMF_FMAINT)..
That said, when providing the first version of this code, we also
changed the SRV_ADMF_MAINT mask to match any of the possible MAINT
cases: SRV_ADMF_FMAINT, SRV_ADMF_IMAINT, SRV_ADMF_CMAINT

Since SRV_ADMF_CMAINT is never (and is not supposed to be) altered at
run time, once a server has this flag set up, it can never ever be
enabled again using the stats socket.

In order to fix this, we should:
- consider SRV_ADMF_CMAINT as a simple flag to report the state in the
  old configuration file (will be used after a reload to deduce the
  state of the server in a new running process)
- enabling both SRV_ADMF_CMAINT and SRV_ADMF_FMAINT when the keyword
  "disabled" is in use in the configuration
- update the mask SRV_ADMF_MAINT as it was before, to only match
  SRV_ADMF_FMAINT and SRV_ADMF_IMAINT.

The following patch perform the changes above.
It allows fixing the regression without breaking the way the up coming
feature (seamless server state accross reloads) is going to work.

Note: this is 1.6-only, no backport needed.
2015-09-18 12:38:23 +02:00
Pieter Baauw
caa6a1bb46 MINOR: support cpu-map feature through the compile option USE_CPU_AFFINITY on FreeBSD 2015-09-17 22:11:09 +02:00
Pieter Baauw
44fc9dfa6d DOC: add references to rise/fall for the fastinter explanation 2015-09-17 22:11:04 +02:00