22 regression tests for state machines are managed by the new
file tests/test-fsm.cfg. Check it, they are all documented
inside. Most of the bugs introduced during the FSM extraction
have been found with these tests.
The new "wait_end" acl delays evaluation of the rule (and the next ones)
to the end of the analysis period. This is intented to be used with TCP
content analysis. A rule referencing such an ACL will not match until
the delay is over. An equivalent default ACL "WAIT_END" has been created.
This new keyword matches an dotted version mapped into an integer.
It permits to match an SSL message protocol version just as if it
was an integer, so that it is easy to map ranges, like this :
acl obsolete_ssl req_ssl_ver lt 3
acl correct_ssl req_ssl_ver 3.0-3.1
acl invalid_ssl req_ssl_ver gt 3.1
Both SSLv2 hello messages and SSLv3 messages are supported. The
test tries to be strict enough to avoid being easily fooled. In
particular, it waits for as many bytes as announced in the message
header if this header looks valid (bound to the buffer size).
The same decoder will be usable with minor changes to check the
response messages.
Some people need to inspect contents of TCP requests before
deciding to forward a connection or not. A future extension
of this demand might consist in selecting a server farm
depending on the protocol detected in the request.
For this reason, a new state CL_STINSPECT has been added on
the client side. It is immediately entered upon accept() if
the statement "tcp-request inspect-delay <xxx>" is found in
the frontend configuration. Haproxy will then wait up to
this amount of time trying to find a matching ACL, and will
either accept or reject the connection depending on the
"tcp-request content <action> {if|unless}" rules, where
<action> is either "accept" or "reject".
Note that it only waits that long if no definitive verdict
can be found earlier. That generally implies calling a fetch()
function which does not have enough information to decode
some contents, or a match() function which only finds the
beginning of what it's looking for.
It is only at the ACL level that partial data may be processed
as such, because we need to distinguish between MISS and FAIL
*before* applying the term negation.
Thus it is enough to add "| ACL_PARTIAL" to the last argument
when calling acl_exec_cond() to indicate that we expect
ACL_PAT_MISS to be returned if some data is missing (for
fetch() or match()). This is the only case we may return
this value. For this reason, the ACL check in process_cli()
has become a lot simpler.
A new ACL "req_len" of type "int" has been added. Right now
it is already possible to drop requests which talk too early
(eg: for SMTP) or which don't talk at all (eg: HTTP/SSL).
Also, the acl fetch() functions have been extended in order
to permit reporting of missing data in case of fetch failure,
using the ACL_TEST_F_MAY_CHANGE flag.
The default behaviour is unchanged, and if no rule matches,
the request is accepted.
As a side effect, all layer 7 fetching functions have been
cleaned up so that they now check for the validity of the
layer 7 pointer before dereferencing it.
The wait queues now rely on 4 trees for past, present and future
timers. The computations are cleaner and more reliable. The
wake_expired_tasks function has become simpler. Also, a bug
previously introduced in task_queue() by the first introduction
of eb_trees has been fixed (the eb->key was never updated).
A new "redirect" keyword adds the ability to send an HTTP 301/302/303
redirection to either an absolute location or to a prefix followed by
the original URI. The redirection is conditionned by ACL rules, so it
becomes very easy to move parts of a site to another site using this.
This work was almost entirely done at Exceliance by Emeric Brun.
A test-case has been added in the tests/ directory.
About: [BUG] Flush buffers also where there are exactly 0 bytes left
I'm also attaching a debug patch that helps to trigger this bug.
Without the fix:
# echo -ne "GET /haproxy?stats;csv;norefresh HTTP/1.0\r\n\r\n"|nc 127.0.0.1
801|wc -c
16384
With the fix:
# echo -ne "GET /haproxy?stats;csv;norefresh HTTP/1.0\r\n\r\n"|nc 127.0.0.1
801|wc -c
33089
Best regards,
Krzysztof Oledzki
If we want to support netmasks for IP address hashing,
we will need something better than a pure modulus, otherwise
people with even numbers of servers will get surprizes.
Bob Jenkins is known for his works on hashing, and his site
has a lot of very interesting researches and algorithms for
integer hashing. He also points to the work of Thomas Wang
who has similar findings.
The program here tests their algorithms in order to find one
well suited for IP address hashing.
The new "leastconn" LB algorithm selects the server which has the
least established or pending connections. The weights are considered,
so that a server with a weight of 20 will get twice as many connections
as the server with a weight of 10.
The algorithm respects the minconn/maxconn settings, as well as the
slowstart since it is a dynamic algorithm. It also correctly supports
backup servers (one and all).
It is generally suited for protocols with long sessions (such as remote
terminals and databases), as it will ensure that upon restart, a server
with no connection will take all new ones until its load is balanced
with others.
A test configuration has been added in order to ease regression testing.
In order to offer DoS protection, it may be required to lower the maximum
accepted time to receive a complete HTTP request without affecting the client
timeout. This helps protecting against established connections on which
nothing is sent. The client timeout cannot offer a good protection against
this abuse because it is an inactivity timeout, which means that if the
attacker sends one character every now and then, the timeout will not
trigger. With the HTTP request timeout, no matter what speed the client
types, the request will be aborted if it does not complete in time.
Add the "backlog" parameter to frontends, to give hints to
the system about the approximate listen backlog desired size.
In order to protect against SYN flood attacks, one solution is
to increase the system's SYN backlog size. Depending on the
system, sometimes it is just tunable via a system parameter,
sometimes it is not adjustable at all, and sometimes the system
relies on hints given by the application at the time of the
listen() syscall. By default, HAProxy passes the frontend's
maxconn value to the listen() syscall. On systems which can
make use of this value, it can sometimes be useful to be able
to specify a different value, hence this backlog parameter.
A new "timeout" keyword replaces old "{con|cli|srv}timeout", and
provides the ability to independantly set the following timeouts :
- client
- tarpit
- queue
- connect
- server
- appsession
Additionally, the "clitimeout", "contimeout" and "srvtimeout" values
are supported but deprecated. No warning is emitted yet when they are
used since the option is very new.
Other timeouts should follow soon now.
It is not always handy to manipulate large values exprimed
in milliseconds for timeouts. Also, some values are entered
in seconds (such as the stats refresh interval). This patch
adds support for time units. It knows about 'us', 'ms', 's',
'm', 'h', and 'd'. It automatically converts each value into
the caller's expected unit. Unit-less values are still passed
unchanged.
The unit must be passed as a suffix to the number. For instance:
clitimeout 15m
If any character is not understood, an error is returned.
In order to avoid issues in the future, we want to restrict
the set of allowed characters for identifiers. Starting from
now, only A-Z, a-z, 0-9, '-', '_', '.' and ':' will be allowed
for a proxy, a server or an ACL name.
A test file has been added to check the restriction.
Now we can compute the max place depending on the number of servers,
maximum weight and weight scale. The formula has been stored as a
comment so that it's easy to choose between smooth weight ramp up
and high number of servers. The default scale has been set to 16,
which permits 4000 servers with a granularity of 6% in the worst
case (weight=1).
The default_backend did not work in TCP mode since there was no
header state to assign the backend. This causes much trouble when
configs are created by copy-paste.
The solution was to fix the way the backend is assigned upon accept().
A wrong contimeout assignment was fixed too.
Some applications do not have a strict persistence requirement, yet
it is still desirable for performance considerations, due to local
caches on the servers. For some reasons, there are some applications
which cannot rely on cookies, and for which the last resort is to use
a parameter passed in the URL.
The new 'url_param' balance method is there to solve this issue. It
accepts a parameter name which is looked up from the URL and which
is then hashed to select a server. If the parameter is not found,
then the round robin algorithm is used in order to provide a normal
load balancing across the servers for the first requests. It would
have been possible to use a source IP hash instead, but since such
applications are generally buried behind multiple levels of
reverse-proxies, it would not provide a good balance.
The doc has been updated, and two regression testing configurations
have been added.
localtime() was called with pointers to tv_sec, which is time_t on
some platforms and long on others. A problem was encountered on
Sparc64 under OpenBSD where tv_sec is long (64 bits) and time_t is
32 bits. Since this architecture is big-endian, it exhibited the
bug because localtime() always worked with the high part of the
value which is always zero. This problem was identified and debugged
by Thierry Fournier.
The correct solution is to pass the date by value and not by pointer,
through an intermediate function. The use of localtime_r() instead of
localtime() also made it possible to get rid of the first call to
localtime() since it does not need to allocate memory anymore.
src/chtbl.c, src/hashpjw.c and src/list.c are distributed under
an obscure license. While Aleks and I believe that this license
is OK for haproxy, other people think it is not compatible with
the GPL.
Whether it is or not is not the problem. The fact that it rises
a doubt is sufficient for this problem to be addressed. Arnaud
Cornet rewrote the unclear parts with clean GPLv2 and LGPL code.
The hash algorithm has changed too and the code has been slightly
simplified in the process. A lot of care has been taken in order
to respect the original API as much as possible, including the
LGPL for the exportable parts.
The new code has not been thoroughly tested but it looks OK now.
added "wt_hash" which shows only 60 collisions in 575k values, which
sets it between hashword() and djbx33(). It's also between both in
terms of performance, but the most important part is that its variable
length rotation mechanism should make it really harder to predict and
attack than the other ones.
Compare the results of recv/send with the parameter passed and
detect whether the system has no free buffer space for send()
or has no data anymore for recv(). This dramatically reduces
the number of syscalls (by about 23%).
Aleksandar Lazic has collected many hashing algorithms and put them
in one file to ease benchmarking. Some algos look promising, some
of them have been checked further with uri_hash. Some results on
various systems/hardware are stored in hash_results.txt.
The uri_hash.c program makes it very easy to benchmark the
distribution of hash algos. Pass it one word per line, and
it will show the distribution per server for 1 to 10 servers.
* fixed a stupid bug introduced in 1.1.22 which caused second and subsequent
'default' sections to keep previous parameters, and not initialize logs
correctly.
* fixed a second stupid bug introduced in 1.1.22 which caused configurations
relying on 'dispatch' mode to segfault at the first connection.
* 'option httpchk' now supports method, HTTP version and a few headers.
* now, 'option httpchk', 'cookie' and 'capture' can be specified in
'defaults' section
* a fresh new english documentation
* large Makefile cleanup for increased portability
* new build script 'build.cfg' for Formilux-0.1.8
* new startup script 'init.haproxy.flx0' for Formilux-0.1.8
* 'listen' now supports optionnal address:port-range lists
* 'bind' introduced to add new listen addresses
* fixed a bug which caused a session to be kept established on a server till
it timed out if the client closed during the DATA phase.
* the port part of each server address can now be empty to make the proxy
connect to the server on the same port it was connected to, be an absolute
unsigned number to reflect a single port (as in older versions), or an
explicitly signed number (+N/-N) to indicate that this offset must be
applied to the port the proxy was connected to, when connecting to the
server.
* the 'port' server option allows the user to specify a different
health-check port than the service one. It is mandatory when only relative
ports have been specified and check is required. By default, the checks are
sent to the service port.
* new 'defaults' section which is rather similar to 'listen' except that all
values are only used as default values for future 'listen' sections, until
a new 'defaults' resets them. At the moment, server options, regexes,
cookie names and captures cannot be set in the 'defaults' section.
* Makefile now optimizes for Ultrasparc by default on Solaris/Sparc
* large documentation updates and fixes
* new 'tests' directory with some debug files
