Commit Graph

4229 Commits

Author SHA1 Message Date
Cyril Bonté
9ede66b06d MEDIUM: checks: provide environment variables to the external checks
The external command accepted 4 arguments, some with the value "NOT_USED" when
not applicable. In order to make the exernal command more generic, this patch
also provides the values in environment variables. This allows to provide more
information.

Currently, the supported environment variables are :

PATH, as previously provided.

HAPROXY_PROXY_NAME, the backend name
HAPROXY_PROXY_ID, the backend id
HAPROXY_PROXY_ADDR, the first bind address if available (or empty)
HAPROXY_PROXY_PORT, the first bind port if available (or empty)

HAPROXY_SERVER_NAME, the server name
HAPROXY_SERVER_ID, the server id
HAPROXY_SERVER_ADDR, the server address
HAPROXY_SERVER_PORT, the server port if available (or empty)
HAPROXY_SERVER_MAXCONN, the server max connections
HAPROXY_SERVER_CURCONN, the current number of connections on the server
2014-12-02 21:44:33 +01:00
Cyril Bonté
777be861c5 MINOR: checks: allow external checks in backend sections
Previously, external checks required to find at least one listener in order to
pass the <proxy_address> and <proxy_port> arguments to the external script.
It prevented from declaring external checks in backend sections and haproxy
rejected the configuration.

The listener is now optional and values "NOT_USED" are passed if no listener is
found. For instance, this is the case with a backend section.

This is specific to the 1.6 branch.
2014-12-02 21:44:33 +01:00
Willy Tarreau
83f2592bcd BUG/MEDIUM: payload: ensure that a request channel is available
Denys Fedoryshchenko reported a segfault when using certain
sample fetch functions in the "tcp-request connection" rulesets
despite the warnings. This is because some tests for the existence
of the channel were missing.

The fetches which were fixed are :
  - req.ssl_hello_type
  - rep.ssl_hello_type
  - req.ssl_sni

This fix must be backported to 1.5.
2014-11-26 13:32:22 +01:00
Willy Tarreau
4deaf39243 BUG/MEDIUM: patterns: previous fix was incomplete
Dmitry Sivachenko <trtrmitya@gmail.com> reported that commit 315ec42
("BUG/MEDIUM: pattern: don't load more than once a pattern list.")
relies on an uninitialised variable in the stack. While it used to
work fine during the tests, if the uninitialized variable is non-null,
some patterns may be aggregated if loaded multiple times, resulting in
slower processing, which was the original issue it tried to address.

The fix needs to be backported to 1.5.
2014-11-26 13:17:03 +01:00
Willy Tarreau
3b24641745 BUG/MAJOR: sessions: unlink session from list on out of memory
Since embryonic sessions were introduced in 1.5-dev12 with commit
2542b53 ("MAJOR: session: introduce embryonic sessions"), a major
bug remained present. If haproxy cannot allocate memory during
session_complete() (for example, no more buffers), it will not
unlink the new session from the sessions list. This will cause
memory corruptions if the memory area from the session is reused
for anything else, and may also cause bogus output on "show sess"
on the CLI.

This fix must be backported to 1.5.
2014-11-25 22:09:05 +01:00
Emeric Brun
c9a0f6d023 MINOR: samples: add the word converter.
word(<index>,<delimiters>)
  Extracts the nth word considering given delimiters from an input string.
  Indexes start at 1 and delimiters are a string formatted list of chars.
2014-11-25 14:48:39 +01:00
Willy Tarreau
23a5c396ec DEBUG: pools: apply poisonning on every allocated pool
Till now, when memory poisonning was enabled, it used to be done only
after a calloc(). But sometimes it's not enough to detect unexpected
sharing, so let's ensure that we now poison every allocation once it's
in place. Note that enabling poisonning significantly hurts performance
(it can typically half the overall performance).
2014-11-25 13:48:43 +01:00
Emeric Brun
f399b0debf MINOR: samples: adds the field converter.
field(<index>,<delimiters>)
  Extracts the substring at the given index considering given delimiters from
  an input string. Indexes start at 1 and delimiters are a string formatted
  list of chars.
2014-11-24 17:44:02 +01:00
Emeric Brun
54c4ac8417 MINOR: samples: adds the bytes converter.
bytes(<offset>[,<length>])
  Extracts a some bytes from an input binary sample. The result is a
  binary sample starting at an offset (in bytes) of the original sample
  and optionnaly truncated at the given length.
2014-11-24 17:44:02 +01:00
Willy Tarreau
0f30d26dbf MINOR: sample: add a few basic internal fetches (nbproc, proc, stopping)
Sometimes, either for debugging or for logging we'd like to have a bit
of information about the running process. Here are 3 new fetches for this :

nbproc : integer
  Returns an integer value corresponding to the number of processes that were
  started (it equals the global "nbproc" setting). This is useful for logging
  and debugging purposes.

proc : integer
  Returns an integer value corresponding to the position of the process calling
  the function, between 1 and global.nbproc. This is useful for logging and
  debugging purposes.

stopping : boolean
  Returns TRUE if the process calling the function is currently stopping. This
  can be useful for logging, or for relaxing certain checks or helping close
  certain connections upon graceful shutdown.
2014-11-24 17:44:02 +01:00
Emeric Brun
4b9e80268e BUG/MINOR: samples: fix unnecessary memcopy converting binary to string. 2014-11-24 17:44:02 +01:00
Willy Tarreau
42fb809cf4 BUG/MINOR: peers: the buffer size is global.tune.bufsize, not trash.size
Currently this is harmless since trash.size is copied from
global.tune.bufsize, but this may soon change when buffers become
more dynamic.

At least for consistency it should be backported to 1.5.
2014-11-24 15:40:57 +01:00
Thierry FOURNIER
315ec4217f BUG/MEDIUM: pattern: don't load more than once a pattern list.
A memory optimization can use the same pattern expression for many
equal pattern list (same parse method, index method and index_smp
method).

The pattern expression is returned by "pattern_new_expr", but this
function dont indicate if the returned pattern is already in use.

So, the caller function reload the list of patterns in addition with
the existing patterns. This behavior is not a problem with tree indexed
pattern, but it grows the lists indexed patterns.

This fix add a "reuse" flag in return of the function "pattern_new_expr".
If the flag is set, I suppose that the patterns are already loaded.

This fix must be backported into 1.5.
2014-11-24 15:40:16 +01:00
Willy Tarreau
cbf8cf68f5 DOC: fix typo in the body parser documentation for msg.sov
"positive" ... "null or positive". The second one is "null or negative".
2014-11-21 21:11:30 +01:00
Willy Tarreau
5be2f35231 MAJOR: polling: centralize calls to I/O callbacks
In order for HTTP/2 not to eat too much memory, we'll have to support
on-the-fly buffer allocation, since most streams will have an empty
request buffer at some point. Supporting allocation on the fly means
being able to sleep inside I/O callbacks if a buffer is not available.

Till now, the I/O callbacks were called from two locations :
  - when processing the cached events
  - when processing the polled events from the poller

This change cleans up the design a bit further than what was started in
1.5. It now ensures that we never call any iocb from the poller itself
and that instead, events learned by the poller are put into the cache.
The benefit is important in terms of stability : we don't have to care
anymore about the risk that new events are added into the poller while
processing its events, and we're certain that updates are processed at
a single location.

To achieve this, we now modify all the fd_* functions so that instead of
creating updates, they add/remove the fd to/from the cache depending on
its state, and only create an update when the polling status reaches a
state where it will have to change. Since the pollers make use of these
functions to notify readiness (using fd_may_recv/fd_may_send), the cache
is always up to date with the poller.

Creating updates only when the polling status needs to change saves a
significant amount of work for the pollers : a benchmark showed that on
a typical TCP proxy test, the amount of updates per connection dropped
from 11 to 1 on average. This also means that the update list is smaller
and has more chances of not thrashing too many CPU cache lines. The first
observed benefit is a net 2% performance gain on the connection rate.

A second benefit is that when a connection is accepted, it's only when
we're processing the cache, and the recv event is automatically added
into the cache *after* the current one, resulting in this event to be
processed immediately during the same loop. Previously we used to have
a second run over the updates to detect if new events were added to
catch them before waking up tasks.

The next gain will be offered by the next steps on this subject consisting
in implementing an I/O queue containing all cached events ordered by priority
just like the run queue, and to be able to leave some events pending there
as long as needed. That will allow us *not* to perform some FD processing
if it's not the proper time for this (typically keep waiting for a buffer
to be allocated if none is available for an recv()). And by only processing
a small bunch of them, we'll allow priorities to take place even at the I/O
level.

As a result of this change, functions fd_alloc_or_release_cache_entry()
and fd_process_polled_events() have disappeared, and the code dedicated
to checking for new fd events after the callback during the poll() loop
was removed as well. Despite the patch looking large, it's mostly a
change of what function is falled upon fd_*() and almost nothing was
added.
2014-11-21 20:37:32 +01:00
Willy Tarreau
5506e3f8b6 BUG/MINOR: stats: correctly set the request/response analysers
When enabling stats, response analysers were set on the request
analyser list, which 1) has no effect, and 2) means we don't have
the response analysers properly set.

In practice these response analysers are set when the connection
to the server or applet is established so we don't need/must not
set them here.

Fortunately this bug had no impact since the flags are distinct,
but it definitely is confusing.

It should be backported to 1.5.
2014-11-21 17:53:08 +01:00
KOVACS Krisztian
b3e54fe387 MAJOR: namespace: add Linux network namespace support
This patch makes it possible to create binds and servers in separate
namespaces.  This can be used to proxy between multiple completely independent
virtual networks (with possibly overlapping IP addresses) and a
non-namespace-aware proxy implementation that supports the proxy protocol (v2).

The setup is something like this:

net1 on VLAN 1 (namespace 1) -\
net2 on VLAN 2 (namespace 2) -- haproxy ==== proxy (namespace 0)
net3 on VLAN 3 (namespace 3) -/

The proxy is configured to make server connections through haproxy and sending
the expected source/target addresses to haproxy using the proxy protocol.

The network namespace setup on the haproxy node is something like this:

= 8< =
$ cat setup.sh
ip netns add 1
ip link add link eth1 type vlan id 1
ip link set eth1.1 netns 1
ip netns exec 1 ip addr add 192.168.91.2/24 dev eth1.1
ip netns exec 1 ip link set eth1.$id up
...
= 8< =

= 8< =
$ cat haproxy.cfg
frontend clients
  bind 127.0.0.1:50022 namespace 1 transparent
  default_backend scb

backend server
  mode tcp
  server server1 192.168.122.4:2222 namespace 2 send-proxy-v2
= 8< =

A bind line creates the listener in the specified namespace, and connections
originating from that listener also have their network namespace set to
that of the listener.

A server line either forces the connection to be made in a specified
namespace or may use the namespace from the client-side connection if that
was set.

For more documentation please read the documentation included in the patch
itself.

Signed-off-by: KOVACS Tamas <ktamas@balabit.com>
Signed-off-by: Sarkozi Laszlo <laszlo.sarkozi@balabit.com>
Signed-off-by: KOVACS Krisztian <hidden@balabit.com>
2014-11-21 07:51:57 +01:00
KOVACS Krisztian
efd3aa9341 BUG/MEDIUM: connection: sanitize PPv2 header length before parsing address information
Previously, if hdr_v2->len was less than the length of the protocol
specific address information we could have read after the end of the
buffer and initialize the sockaddr structure with junk.

Signed-off-by: KOVACS Krisztian <hidden@balabit.com>

[WT: this is only tagged medium since proxy protocol is only used from
 trusted sources]

This must be backported to 1.5.
2014-11-21 07:45:17 +01:00
Willy Tarreau
9654e57fac BUG/MAJOR: frontend: initialize capture pointers earlier
Denys Fedoryshchenko reported and diagnosed a nasty bug caused by TCP
captures, introduced in late 1.5-dev by commit 18bf01e ("MEDIUM: tcp:
add a new tcp-request capture directive"). The problem is that we're
using the array of capture pointers initially designed for HTTP usage
only, and that this array was only reset when starting to process an
HTTP request. In a tcp-only frontend, the pointers are not reset, and
if the capture pool is shared, we can very well point to whatever other
memory location, resulting in random crashes when tcp-request content
captures are processed.

The fix simply consists in initializing these pointers when the pools
are prepared.

A workaround for existing versions consists in either disabling TCP
captures in tcp-only frontends, or in forcing the frontends to work in
HTTP mode.

Thanks to Denys for the amount of testing and detailed reports.

This fix must be backported to 1.5.
2014-11-18 18:53:43 +01:00
Willy Tarreau
743c128580 BUG/MINOR: config: don't inherit the default balance algorithm in frontends
Tom Limoncelli from Stack Exchange reported a minor bug : the frontend
inherits the LB parameters from the defaults sections. The impact is
that if a "balance" directive uses any L7 parameter in the defaults
sections and the frontend is in TCP mode, a warning is emitted about
their incompatibility. The warning is harmless but a valid, sane config
should never cause any warning to be reported.

This fix should be backported into 1.5 and possibly 1.4.
2014-11-18 15:04:29 +01:00
Christian Ruppert
de898712a0 MEDIUM: regex: Use pcre_study always when PCRE is used, regardless of JIT
pcre_study() has been around long before JIT has been added. It also seems to
affect the performance in some cases (positive).

Below I've attached some test restults. The test is based on
http://sljit.sourceforge.net/regex_perf.html (see bottom). It has been modified
to just test pcre_study vs. no pcre_study. Note: This test does not try to
match specific header it's instead run over a larger text with more and less
complex patterns to make the differences more clear.

% ./runtest
'mark.txt' loaded. (Length: 19665221 bytes)
-----------------
Regex: 'Twain'
[pcre-nostudy] time:    14 ms (2388 matches)
[pcre-study] time:    21 ms (2388 matches)
-----------------
Regex: '^Twain'
[pcre-nostudy] time:   109 ms (100 matches)
[pcre-study] time:   109 ms (100 matches)
-----------------
Regex: 'Twain$'
[pcre-nostudy] time:    14 ms (127 matches)
[pcre-study] time:    16 ms (127 matches)
-----------------
Regex: 'Huck[a-zA-Z]+|Finn[a-zA-Z]+'
[pcre-nostudy] time:   695 ms (83 matches)
[pcre-study] time:    26 ms (83 matches)
-----------------
Regex: 'a[^x]{20}b'
[pcre-nostudy] time:    90 ms (12495 matches)
[pcre-study] time:    91 ms (12495 matches)
-----------------
Regex: 'Tom|Sawyer|Huckleberry|Finn'
[pcre-nostudy] time:  1236 ms (3015 matches)
[pcre-study] time:    34 ms (3015 matches)
-----------------
Regex: '.{0,3}(Tom|Sawyer|Huckleberry|Finn)'
[pcre-nostudy] time:  5696 ms (3015 matches)
[pcre-study] time:  5655 ms (3015 matches)
-----------------
Regex: '[a-zA-Z]+ing'
[pcre-nostudy] time:  1290 ms (95863 matches)
[pcre-study] time:  1167 ms (95863 matches)
-----------------
Regex: '^[a-zA-Z]{0,4}ing[^a-zA-Z]'
[pcre-nostudy] time:   136 ms (4507 matches)
[pcre-study] time:   134 ms (4507 matches)
-----------------
Regex: '[a-zA-Z]+ing$'
[pcre-nostudy] time:  1334 ms (5360 matches)
[pcre-study] time:  1214 ms (5360 matches)
-----------------
Regex: '^[a-zA-Z ]{5,}$'
[pcre-nostudy] time:   198 ms (26236 matches)
[pcre-study] time:   197 ms (26236 matches)
-----------------
Regex: '^.{16,20}$'
[pcre-nostudy] time:   173 ms (4902 matches)
[pcre-study] time:   175 ms (4902 matches)
-----------------
Regex: '([a-f](.[d-m].){0,2}[h-n]){2}'
[pcre-nostudy] time:  1242 ms (68621 matches)
[pcre-study] time:   690 ms (68621 matches)
-----------------
Regex: '([A-Za-z]awyer|[A-Za-z]inn)[^a-zA-Z]'
[pcre-nostudy] time:  1215 ms (675 matches)
[pcre-study] time:   952 ms (675 matches)
-----------------
Regex: '"[^"]{0,30}[?!\.]"'
[pcre-nostudy] time:    27 ms (5972 matches)
[pcre-study] time:    28 ms (5972 matches)
-----------------
Regex: 'Tom.{10,25}river|river.{10,25}Tom'
[pcre-nostudy] time:   705 ms (2 matches)
[pcre-study] time:    68 ms (2 matches)

In some cases it's more or less the same but when it's faster than by a huge margin.
It always depends on the pattern, the string(s) to match against etc.

Signed-off-by: Christian Ruppert <c.ruppert@babiel.com>
2014-11-18 13:26:18 +01:00
Cyril Bonté
9ce1311ebc BUG/MEDIUM: checks: fix conflicts between agent checks and ssl healthchecks
Lasse Birnbaum Jensen reported an issue when agent checks are used at the same
time as standard healthchecks when SSL is enabled on the server side.

The symptom is that agent checks try to communicate in SSL while it should
manage raw data. This happens because the transport layer is shared between all
kind of checks.

To fix the issue, the transport layer is now stored in each check type,
allowing to use SSL healthchecks when required, while an agent check should
always use the raw_sock implementation.

The fix must be backported to 1.5.
2014-11-16 00:53:12 +01:00
Willy Tarreau
eb11889f1e MINOR: task: release the task pool when stopping
When we're stopping, we're not going to create new tasks anymore, so
let's release the task pool upon each task_free() in order to reduce
memory fragmentation.
2014-11-13 16:57:19 +01:00
Willy Tarreau
3a5e060bf6 MINOR: session: release a few other pools when stopping
We currently release all pools when a proxy is stopped, except the
connection, pendconn, and pipe pools. Doing so can improve further
reduce memory usage of old processes, eventhough the connection struct
is quite small, but there are a lot and they can participate to memory
fragmentation. The pipe pool is very small and limited, and not exported
so it's not done here.
2014-11-13 16:56:12 +01:00
Willy Tarreau
396a186def MEDIUM: ssl: add support for smaller SSL records
There's a very common openssl patch on the net meant to significantly
reduce openssl's memory usage. This patch has been provided for many
versions now, and it makes sense to add support for it given that it
is very simple. It only requires to add an extra SSL_MODE flag. Just
like for other flags, if the flag is unknown, it's unset. About 44kB
of memory may be saved per SSL session with the patch.
2014-11-13 14:08:53 +01:00
Willy Tarreau
fba03cdc5a BUG/MEDIUM: ssl: force a full GC in case of memory shortage
When memory becomes scarce and openssl refuses to allocate a new SSL
session, it is worth freeing the pools and trying again instead of
rejecting all incoming SSL connection. This can happen when some
memory usage limits have been assigned to the haproxy process using
-m or with ulimit -m/-v.

This is mostly an enhancement of previous fix and is worth backporting
to 1.5.
2014-11-13 13:48:58 +01:00
Emeric Brun
5547615cda BUG/MEDIUM: ssl: fix bad ssl context init can cause segfault in case of OOM.
Some SSL context's init functions errors were not handled and
can cause a segfault due to an incomplete SSL context
initialization.

This fix must be backported to 1.5.
2014-11-13 13:44:12 +01:00
Arcadiy Ivanov
3785311e64 BUILD: fix "make install" to support spaces in the install dirs
Makefile is unable to install into directories containing spaces.
2014-11-10 12:03:03 +01:00
Godbach
a6547c1338 BUG/MAJOR: buffer: check the space left is enough or not when input data in a buffer is wrapped
HAProxy will crash with the following configuration:

global
    ...
    tune.bufsize 1024
    tune.maxrewrite 0
frontend xxx
    ...
backend  yyy
   ...
   cookie cookie insert maxidle 300s

If client sends a request of which object size is more than tune.bufsize (1024
bytes), HAProxy will crash.

After doing some debugging, the crash was caused by http_header_add_tail2() ->
buffer_insert_line2() while inserting cookie at the end of response header.
Part codes of buffer_insert_line2() are as below:

int buffer_insert_line2(struct buffer *b, char *pos, const char *str, int len)
{
    int delta;

    delta = len + 2;

    if (bi_end(b) + delta >= b->data + b->size)
        return 0;  /* no space left */

    /* first, protect the end of the buffer */
    memmove(pos + delta, pos, bi_end(b) - pos);
    ...
}

Since tune.maxrewrite is 0, HAProxy can receive 1024 bytes once which is equals
to full buffer size.  Under such condition, the buffer is full and bi_end(b)
will be wrapped to the start of buffer which pointed to b->data. As a result,
though there is no space left in buffer, the check condition
	if (bi_end(b) + delta >= b->data + b->size)
will be true, then memmove() is called, and (pos + delta) will exceed the end
of buffer (b->data + b->size), HAProxy crashes

Just take buffer_replace2() as a reference, the other check when input data in
a buffer is wrapped should be also added into buffer_insert_line2().

This fix must be backported to 1.5.

Signed-off-by: Godbach <nylzhaowei@gmail.com>
2014-10-31 07:39:34 +01:00
Willy Tarreau
cd360cec7a BUG/BUILD: revert accidental change in the makefile from latest SSL fix
Commit 0bed994 ("BUG/MINOR: ssl: correctly initialize ssl ctx for
invalid certificates") accidently left a change in the Makefile
resulting in -ldl being appended to the LDFLAGS. As reported by
Dmitry Sivachenko, this will break build on systems without libdl
such as FreeBSD.

This fix must be backported to 1.5.
2014-10-31 07:39:04 +01:00
Emeric Brun
0bed9945ee BUG/MINOR: ssl: correctly initialize ssl ctx for invalid certificates
Bug reported by John Leach: no-sslv3 does not work using some certificates.

It appears that ssl ctx is not updated with configured options if the
CommonName of the certificate's subject is not found.

It applies only on the first cerificate of a configured bind line.

There is no security impact, because only invalid nameless certficates
are concerned.

This fix must be backported to 1.5
2014-10-30 20:02:33 +01:00
Emeric Brun
2c86cbf753 MINOR: ssl: add statement to force some ssl options in global.
Adds global statements 'ssl-default-server-options' and
'ssl-default-bind-options' to force on 'server' and 'bind' lines
some ssl options.

Currently available options are 'no-sslv3', 'no-tlsv10', 'no-tlsv11',
'no-tlsv12', 'force-sslv3', 'force-tlsv10', 'force-tlsv11',
'force-tlsv12', and 'no-tls-tickets'.

Example:
      global
        ssl-default-server-options no-sslv3
        ssl-default-bind-options no-sslv3
2014-10-30 17:06:29 +01:00
Willy Tarreau
5e0d0e046a BUG/MEDIUM: tcp: don't use SO_ORIGINAL_DST on non-AF_INET sockets
There's an issue when using SO_ORIGINAL_DST to retrieve the original
destination of a connection's address before being translated by
Netfilter's DNAT/REDIRECT or the old TPROXY. SO_ORIGINAL_DST is
able to retrieve an IPv4 address when the original destination was
IPv4 mapped into IPv6. At first glance it's not a big deal, but it
is for logging and for the proxy protocol, because we then have
two different address families for the source and destination. In
this case, the proxy protocol correctly detects the issue and emits
"UNKNOWN".

In order to fix this, we perform getsockname() first, and only if
the address family is AF_INET, then we perform the getsockopt() call.

This fix must be backported to 1.5, and probably even to 1.4 and 1.3.
2014-10-29 21:46:01 +01:00
Emeric Brun
43e7958def MINOR: add fetchs 'ssl_c_der' and 'ssl_f_der' to return DER formatted certs
ssl_c_der : binary
  Returns the DER formatted certificate presented by the client when the
  incoming connection was made over an SSL/TLS transport layer. When used for
  an ACL, the value(s) to match against can be passed in hexadecimal form.

ssl_f_der : binary
  Returns the DER formatted certificate presented by the frontend when the
  incoming connection was made over an SSL/TLS transport layer. When used for
  an ACL, the value(s) to match against can be passed in hexadecimal form.
2014-10-29 19:25:24 +01:00
Christian Ruppert
955f4613cb BUG/MEDIUM: regex: fix pcre_study error handling
pcre_study() may return NULL even though it succeeded. In this case error is
NULL otherwise error is not NULL. Also see man 3 pcre_study.

Previously a ACL pattern of e.g. ".*" would cause error because pcre_study did
not found anything to speed up matching and returned regex->extra = NULL and
error = NULL which in this case was a false-positive. That happend only when
PCRE_JIT was enabled for HAProxy but libpcre has been built without JIT.

Signed-off-by: Christian Ruppert <c.ruppert@babiel.com>

[wt: this needs to be backported to 1.5 as well]
2014-10-29 17:44:31 +01:00
Willy Tarreau
a616ba6f5f BUILD/MINOR: ssl: de-constify "ciphers" to avoid a warning on openssl-0.9.8
When building on openssl-0.9.8, since commit 23d5d37 ("MINOR: ssl: use
SSL_get_ciphers() instead of directly accessing the cipher list.") we get
the following warning :

	src/ssl_sock.c: In function 'ssl_sock_prepare_ctx':
	src/ssl_sock.c:1592: warning: passing argument 1 of 'SSL_CIPHER_description' discards qualifiers from pointer target type

This is because the openssl API has changed between 0.9.8 and 1.0.1 :

	0.9.8: char *SSL_CIPHER_description(SSL_CIPHER *,char *buf,int size);
	1.0.1: char *SSL_CIPHER_description(const SSL_CIPHER *,char *buf,int size);

So let's remove the "const" type qualifier to satisfy both versions.
Note that the fix above was backported to 1.5, so this one should as well.
2014-10-26 06:52:13 +01:00
Thierry FOURNIER
317e1c4f1e MINOR: sample: add "json" converter
This converter escapes string to use it as json/ascii escaped string.
It can read UTF-8 with differents behavior on errors and encode it in
json/ascii.

json([<input-code>])
  Escapes the input string and produces an ASCII ouput string ready to use as a
  JSON string. The converter tries to decode the input string according to the
  <input-code> parameter. It can be "ascii", "utf8", "utf8s", "utf8"" or
  "utf8ps". The "ascii" decoder never fails. The "utf8" decoder detects 3 types
  of errors:
   - bad UTF-8 sequence (lone continuation byte, bad number of continuation
     bytes, ...)
   - invalid range (the decoded value is within a UTF-8 prohibited range),
   - code overlong (the value is encoded with more bytes than necessary).

  The UTF-8 JSON encoding can produce a "too long value" error when the UTF-8
  character is greater than 0xffff because the JSON string escape specification
  only authorizes 4 hex digits for the value encoding. The UTF-8 decoder exists
  in 4 variants designated by a combination of two suffix letters : "p" for
  "permissive" and "s" for "silently ignore". The behaviors of the decoders
  are :
   - "ascii"  : never fails ;
   - "utf8"   : fails on any detected errors ;
   - "utf8s"  : never fails, but removes characters corresponding to errors ;
   - "utf8p"  : accepts and fixes the overlong errors, but fails on any other
                error ;
   - "utf8ps" : never fails, accepts and fixes the overlong errors, but removes
                characters corresponding to the other errors.

  This converter is particularly useful for building properly escaped JSON for
  logging to servers which consume JSON-formated traffic logs.

  Example:
     capture request header user-agent len 150
     capture request header Host len 15
     log-format {"ip":"%[src]","user-agent":"%[capture.req.hdr(1),json]"}

  Input request from client 127.0.0.1:
     GET / HTTP/1.0
     User-Agent: Very "Ugly" UA 1/2

  Output log:
     {"ip":"127.0.0.1","user-agent":"Very \"Ugly\" UA 1\/2"}
2014-10-26 06:41:12 +01:00
Willy Tarreau
fb20e4668d BUG/MEDIUM: tcp: fix outgoing polling based on proxy protocol
During a tcp connection setup in tcp_connect_server(), we check if
there are pending data to start polling for writes immediately. We
also use the same test to know if we can disable the quick ack and
merge the first data packet with the connection's ACK. This last
case is also valid for the proxy protocol.

The problem lies in the way it's done, as the "data" variable is
improperly completed with the presence of the proxy protocol, resulting
in the connection being polled for data writes if the proxy protocol is
enabled. It's not a big issue per se, except that the proxy protocol
uses the fact that we're polling for data to know if it can use MSG_MORE.

This causes no problem on HTTP/HTTPS, but with banner protocols, it
introduces a 200ms delay if the server waits for the PROXY header.

This has been caused by the connection management changes introduced in
1.5-dev12, specifically commit a1a7474 ("MEDIUM: proxy-proto: don't use
buffer flags in conn_si_send_proxy()"), so this fix must be backported
to 1.5.
2014-10-24 12:09:12 +02:00
Cyril Bonté
a83a50bd7d BUG/MINOR: log: fix request flags when keep-alive is enabled
Colin Ingarfield reported some unexplainable flags in the logs.
For example, a "LR" termination state was set on a request which was forwarded
to a server, where "LR" means that the request should have been handled
internally by haproxy.

This case happens when at least client side keep-alive is enabled. Next
requests in the connection will inherit the flags from the previous request.

2 fields are impacted : "termination_state" and "Tt" in the timing events,
where a "+" can be added, when a previous request was redispatched.

This is not critical for the service itself but can confuse troubleshooting.

The fix must be backported to 1.5 and 1.4.
2014-10-22 22:37:30 +02:00
Willy Tarreau
44cf545000 BUG/MAJOR: cli: explicitly call cli_release_handler() upon error
Dmitry Sivachenko reported an embarrassing problem where haproxy
would sometimes segfault upon reload. After careful analysis and
code inspection, what happens is related to the "show sess" command
on the CLI, and it is not limited to reload operations only.

When a "show sess" is running, once the output buffer is full, the
stats applet grabs a reference to the session being dumped in order
for the current pointer to be able to advance by itself should this
session disappear while the buffer is full. The applet also uses a
release handler that is called when the applet terminates to release
such references.

The problem is that upon error, the command line parser sets the
applet state to STAT_CLI_O_END indicating it wants to terminate the
processing. Unfortunately, the release handler which is called later
to clean everything up relies on the applet's state to know what
operations were in progress, and as such it does not release the
reference. A later "show sess" or the completion of the task being
watched lead to a LIST_DEL() on the task's list which point to a
location that does not match the applet's reference list anymore
and the process dies.

One solution to this would be to add a flag to the current applet's
state mentionning it must leave, without affecting the state indicating
the current operation. It's a bit invasive but could be the long term
solution. The short term solution simply consists in calling the
release handler just before changing the state to STAT_CLI_O_END.
That way everything that must be released is released in time.

Note that the probability to encounter this issue is very low.
It requires a lot of "show sess" or "show sess all" calls, and
that one of them dies before being completed. That can happen
if "show sess" is run in scripts which truncate the output (eg:
"echo show sess|socat|head"). This could be the worst case as it
almost ensures that haproxy fills a buffer, grabs a reference and
detects the error on the socket.

There's no config-based workaround to this issue, except refraining
from issuing "show sess" on large connection counts or "show sess all".
If that's not possible to block everyone, restricting permissions on
the stats socket ensures only authorized tools can connect.

This fix must be backported to 1.5 and to 1.4 (with some changes in
1.4 since the release function does not exist so the LIST_DEL sequence
must be open-coded).

Special thanks to Dmitry for the fairly complete report.
2014-10-22 19:25:30 +02:00
Willy Tarreau
7d59e90473 BUG/MEDIUM: http: don't dump debug headers on MSG_ERROR
When the HTTP parser is in state HTTP_MSG_ERROR, we don't know if it was
already initialized or not. If the error happens before HTTP_MSG_RQBEFORE,
random offsets might be present and we don't want to display such random
strings in debug mode.

While it's theorically possible to randomly crash the process when running
in debug mode here, this bug was not tagged MAJOR because it would not
make sense to run in debug mode in production.

This fix must be backported to 1.5 and 1.4.
2014-10-22 19:25:09 +02:00
Remi Gacogne
23d5d378d0 MINOR: ssl: use SSL_get_ciphers() instead of directly accessing the cipher list. 2014-10-20 18:01:06 +02:00
Willy Tarreau
fad4ffc893 BUG/MEDIUM: backend: fix URI hash when a query string is present
Commit 98634f0 ("MEDIUM: backend: Enhance hash-type directive with an
algorithm options") cleaned up the hashing code by using a centralized
function. A bug appeared in get_server_uh() which is the URI hashing
function. Prior to the patch, the function would stop hashing on the
question mark, or on the trailing slash of a maximum directory count.
Consecutive to the patch, this last character is included into the
hash computation. This means that :

    GET /0
    GET /0?

Are not hashed similarly. The following configuration reproduces it :

    mode http
    balance uri
    server s1 0.0.0.0:1234 redir /s1
    server s2 0.0.0.0:1234 redir /s2

Many thanks to Vedran Furac for reporting this issue. The fix must
be backported to 1.5.
2014-10-17 12:11:50 +02:00
Willy Tarreau
e1cfc1f2b4 BUG/MINOR: config: do not accept more track-sc than configured
MAX_SESS_STKCTR allows one to define the number of stick counters that can
be used in parallel in track-sc* rules. The naming of this macro creates
some confusion because the value there is sometimes used as a max instead
of a count, and the config parser accepts values from 0 to MAX_SESS_STKCTR
and the processing ignores anything tracked on the last one. This means
that by default, track-sc3 is allowed and ignored.

This fix must be backported to 1.5 where the problem there only affects
TCP rules.
2014-10-17 11:53:05 +02:00
Kristoffer Grönlund
91a9f1a2e1 MINOR: systemd: Check configuration before start
Adds a configuration check before starting the haproxy service.
2014-10-15 20:01:09 +02:00
Willy Tarreau
06cc905813 BUG/MEDIUM: config: avoid skipping disabled proxies
Paul Taylor and Bryan Talbot found that after commit 419ead8 ("MEDIUM:
config: compute the exact bind-process before listener's maxaccept"),
a backend marked "disabled" would cause the next backend to be skipped
and if it was the last one it would cause a segfault.

The reason is that the commit above changed the "while" loop for a "for"
loop but a "continue" statement still incrementing the current proxy was
left in the code for disabled proxies, causing the next one to be skipped
as well and the last one to try to dereference NULL when seeking ->next.

The quick workaround consists in not disabling backends, or adding an
empty dummy one after a disabled section.

This fix must be backported to 1.5.
2014-10-10 14:58:52 +02:00
Apollon Oikonomopoulos
74f016985a BUG/MEDIUM: systemd: set KillMode to 'mixed'
By default systemd will send SIGTERM to all processes in the service's
control group. In our case, this includes the wrapper, the master
process and all worker processes.

Since commit c54bdd2a the wrapper actually catches SIGTERM and survives
to see the master process getting killed by systemd and regard this as
an error, placing the unit in a failed state during "systemctl stop".

Since the wrapper now handles SIGTERM by itself, we switch the kill mode
to 'mixed', which means that systemd will deliver the initial SIGTERM to
the wrapper only, and if the actual haproxy processes don't exit after a
given amount of time (default: 90s), a SIGKILL is sent to all remaining
processes in the control group. See systemd.kill(5) for more
information.

This should also be backported to 1.5.
2014-10-09 11:44:42 +02:00
Willy Tarreau
c5af3a6d15 DOC: indicate that weight zero is reported as DRAIN
It's not the same state but reported as such.
2014-10-07 15:27:33 +02:00
Andrew Latham
fbc4e77321 DOC: Address issue where documentation is excluded due to a gitignore rule. 2014-10-07 15:05:19 +02:00
Cyril Bonté
51639696e0 BUG/MINOR: config: don't propagate process binding for dynamic use_backend
A segfault was reported with the introduction of the propagate_processes()
function. It was caused when a use_backend rule was declared with a dynamic
name, using a log-format string. The backend is not resolved during the
configuration, which lead to the segfault.

The patch prevents the process binding propagation for such dynamic rules, it
should also be backported to 1.5.
2014-10-02 21:17:11 +02:00