It's useless to check for buffer timeouts every time we call
process_session() because we already control when we set the flag. So
let's check them at the precise moment where the flag is set.
We want to be able to keep information about errors and timeouts
as long as possible in the buffer. Let's not clear these flags
anymore and keep them static. This does not seem to cause any
trouble, though a finer review might be wise.
Sometimes it can be useful to limit the advertised TCP MSS on
incoming connections, for instance when requests come through
a VPN or when the system is running with jumbo frames enabled.
Passing the "mss <value>" arguments to a "bind" line will set
the value. This works under Linux >= 2.6.28, and maybe a few
earlier ones, though due to an old kernel bug most of earlier
versions will probably ignore it. It is also possible that some
other OSes will support this.
After considering various possibilities, we compiled haproxy under cygwin.
Attached is an updated full diff that also has the TARGET=cygwin documented.
The whole thing compiles and installs with this diff only.
In cygwin 1.7 (now in beta), there is apparently support for ipv6. Cygwin
1.5 (later versions, anyway) already includes some support in the form of a
define USE_IPV6. When defined, it declares the sockaddr_in6 struct and
possibly other things. The above definition AF_INET6=23 is taken from
their /usr/include/socket.h file (where it is #if 0'd out).
We are running into a socket limit. It appears that Cygwin (running on
Windows 2003 Server) will only allow us to set ulimit -n (maximum open
files) to 3200, which means we're a little short of 1600 connections.
The limit of 3200 is an internal Cygwin limit. Perhaps they can raise it in
the future. Using the nbproc option, I was able to bring up 10 servers. It
seems to me that they were able to handle over 2000 connections (even though
each had maxconn 1500 set, and the hard Cygwin fd limit).
This new option enables combining of request buffer data with
the initial ACK of an outgoing TCP connection. Doing so saves
one packet per connection which is quite noticeable on workloads
mostly consisting in small objects. The option is not enabled by
default.
Setting TCP_CORK on a socket before sending the last segment enables
automatic merging of this segment with the FIN from the shutdown()
call. Playing with TCP_CORK is not easy though as we have to track
the status of the TCP_NODELAY flag since both are mutually exclusive.
Doing so saves one more packet per session and offers about 5% more
performance.
There is no reason not to do it, so there is no associated option.
This option disables TCP quick ack upon accept. It is also
automatically enabled in HTTP mode, unless the option is
explicitly disabled with "no option tcp-smart-accept".
This saves one packet per connection which can bring reasonable
amounts of bandwidth for servers processing small requests.
A new keyword prefix "default" has been introduced in order to
reset some options to their default values. This can be needed
for instance when an option is forced disabled or enabled in a
defaults section and when later sections want to use automatic
settings regardless of what was specified there. Right now it
is only supported by options, just like the "no" prefix.
Sometimes we would want to implement implicit default options,
but for this we need to be able to disable them, which requires
to keep track of "no option" settings. With this change, an option
explicitly disabled in a defaults section will still be seen as
explicitly disabled. There should be no regression as nothing makes
use of this yet.
Some users are already hitting the 64k source port limit when
connecting to servers. The system usually maintains a list of
unused source ports, regardless of the source IP they're bound
to. So in order to go beyond the 64k concurrent connections, we
have to manage the source ip:port lists ourselves.
The solution consists in assigning a source port range to each
server and use a free port in that range when connecting to that
server, either for a proxied connection or for a health check.
The port must then be put back into the server's range when the
connection is closed.
This mechanism is used only when a port range is specified on
a server. It makes it possible to reach 64k connections per
server, possibly all from the same IP address. Right now it
should be more than enough even for huge deployments.
When a new process fails to grab some ports, it sends a signal to
the old process in order to release them. Then it tries to bind
again. If it still fails (eg: one of the ports is bound to a
completely different process), it must send the continue signal
to the old process so that this one re-binds to the ports. This
is correctly done, but the newly bound ports are not released
first, which sometimes causes the old process to remain running
with no port bound. The fix simply consists in unbinding all
ports before sending the signal to the old process.
It is recommended to have -D in init scripts, but -D also implies
quiet mode, which hides warning messages, and both options are now
completely unrelated. Remove the implication to get warnings with
-D.
Released version 1.3.18 with the following main changes :
- [MEDIUM] add support for "balance hdr(name)"
- [CLEANUP] give a little bit more information in error message
- [MINOR] add X-Original-To: header
- [BUG] x-original-to: fix missing initialization to default value
- [BUILD] spec file: fix broken pipe during rpmbuild and add man file
- [MINOR] improve reporting of misplaced acl/reqxxx rules
- [MEDIUM] http: add options to ignore invalid header names
- [MEDIUM] http: capture invalid requests/responses even if accepted
- [BUILD] add format(printf) to printf-like functions
- [MINOR] fix several printf formats and missing arguments
- [BUG] stats: total and lbtot are unsigned
- [MINOR] fix a few remaining printf-like formats on 64-bit platforms
- [CLEANUP] remove unused make option from haproxy.spec
- [BUILD] make it possible to pass alternative arch at build time
- [MINOR] switch all stat counters to 64-bit
- [MEDIUM] ensure we don't recursively call pool_gc2()
- [CRITICAL] uninitialized response field can sometimes cause crashes
- [BUG] fix wrong pointer arithmetics in HTTP message captures
- [MINOR] rhel init script : support the reload operation
- [MINOR] add basic signal handling functions
- [BUILD] add signal.o to all makefiles
- [MEDIUM] call signal_process_queue from run_poll_loop
- [MEDIUM] pollers: don't wait if a signal is pending
- [MEDIUM] convert all signals to asynchronous signals
- [BUG] O(1) pollers should check their FD before closing it
- [MINOR] don't close stdio fds twice
- [MINOR] add options dontlog-normal and log-separate-errors
- [DOC] minor fixes and rearrangements
- [BUG] fix parser crash on unconditional tcp content rules
- [DOC] rearrange the configuration manual and add a summary
- [MINOR] standard: provide a new 'my_strndup' function
- [MINOR] implement per-logger log level limitation
- [MINOR] compute the max of sessions/s on fe/be/srv
- [MINOR] stats: report max sessions/s and limit in CSV export
- [MINOR] stats: report max sessions/s and limit in HTML stats
- [MINOR] stats/html: use the arial font before helvetica
The stats HTML output were barely readable on some browsers such as
firefox on Linux, due to the selected helvetica font which is too
small. Specifying "arial" first fixes the issue without changing the
table size. Also, the default size of 0.8em choosen to get 10px out
of 12px is wrong because it gets 9px when rounded down.
Some users want to keep the max sessions/s seen on servers, frontends
and backends for capacity planning. It's easy to grab it while the
session count is updated, so let's keep it.
Some people are using haproxy in a shared environment where the
system logger by default sends alert and emerg messages to all
consoles, which happens when all servers go down on a backend for
instance. These people can not always change the system configuration
and would like to limit the outgoing messages level in order not to
disturb the local users.
The addition of an optional 4th field on the "log" line permits
exactly this. The minimal log level ensures that all outgoing logs
will have at least this level. So the logs are not filtered out,
just set to this level.
There is a patch made by me that allow for balancing on any http header
field.
[WT:
made minor changes:
- turned 'balance header name' into 'balance hdr(name)' to match more
closely the ACL syntax for easier future convergence
- renamed the proxy structure fields header_* => hh_*
- made it possible to use the domain name reduction to any header, not
only "host" since it makes sense to do it with other ones.
Otherwise patch looks good.
/WT]
Several people have asked for a summary in order to ease finding
of sections in the configuration manual. It was the opportunity to
tidy it up a bit and rearrange some sections.
Since 1.3.17, a config containing one of the following lines would
crash the parser :
tcp content reject
tcp content accept
This is because a check is performed on the condition which is not
specified. The obvious fix consists in checkinf for a condition
first.
Some big traffic sites have trouble dealing with logs and tend to
disable them. Here are two new options to help cope with massive
logs.
- dontlog-normal only disables logging for 100% successful
connections, other ones will still be logged
- log-separate-errors will cause non-100% successful connections
to be logged at level "err" instead of level "info" so that a
properly configured syslog daemon can send them to a different
file for longer conservation.
epoll, sepoll and kqueue pollers should check that their fd is not
closed before attempting to close it, otherwise we can end up with
multiple closes of fd #0 upon exit, which is harmless but dirty.
The small list of signals currently handled by haproxy were processed
as soon as they were received. This has caused trouble with calls to
pool_gc2() occuring in the middle of libc's memory management functions
seldom causing deadlocks preventing the old process from leaving.
Now these signals use the new async signal framework and are called
asynchronously, when there is no risk of recursion. This ensures more
reliable operation, especially for sensible processing such as memory
management.
If an asynchronous signal is received outside of the poller, we don't
want the poller to wait for a timeout to occur before processing it,
so we set its timeout to zero, just like we do with pending tasks in
the run queue.
These functions will be used to deliver asynchronous signals in order
to make the signal handling functions more robust. The goal is to keep
the same interface to signal handlers.
I have attached a patch which will add on every http request a new
header 'X-Original-To'. If you have HAProxy running in transparent mode
with a big number of SQUID servers behind it, it is very nice to have
the original destination ip as a common header to make decisions based
on it.
The whole thing is configurable with a new option 'originalto'. I have
updated the sourcecode as well as the documentation. The 'haproxy-en.txt'
and 'haproxy-fr.txt' files are untouched, due to lack of my french
language knowledge. ;)
Also the patch adds this header for IPv4 only. I haven't any IPv6 test
environment running here and don't know if getsockopt() with SO_ORIGINAL_DST
will work on IPv6. If someone knows it and wants to test it I can modify
the diff. Feel free to ask me questions or things which should be changed. :)
--Maik
The pointer arithmetics was wrong in http_capture_bad_message().
This has no impact right now because the error only msg->som was
affected and right now it's always 0. But this was a bug waiting
for keepalive support to strike.
The response message in the transaction structure was not properly
initialised at session initialisation. In theory it cannot cause any
trouble since the affected field os expected to always remain NULL.
However, in some circumstances, such as building on 64-bit platforms
with certain options, the struct session can be exactly 1024 bytes,
the same size of the requri field, so the pools are merged and the
uninitialised field may contain non-null data, causing crashes if
an invalid response is encountered and archived.
The fix simply consists in correctly initialising the missing fields.
This bug cannot affect architectures where the session pool is not
shared (32-bit architectures), but this is only by pure luck.
A race condition exists in the hot reconfiguration code. It is
theorically possible that the second signal is sent during a free()
in the first list, which can cause crashes or freezes (the later
have been observed). Just set up a counter to ensure we do not
recurse.
The byte counters have long been 64-bit to avoid overflows. But with
several sites nowadays, we see session counters wrap around every 10-days
or so. So it was the moment to switch counters to 64-bit, including
error and warning counters which can theorically rise as fast as session
counters even if in practice there is very low risk.
The performance impact should not be noticeable since those counters are
only updated once per session. The stats output have been carefully checked
for proper types on both 32- and 64-bit platforms.
When trying to build a 32-bit binary on a 64-bit platform, we generally
need to pass "-m32" to gcc, which is not convenient with current makefile.
Note that this option requires gcc >= 3.
In order to ease parameter passing, a new ARCH= makefile option has been
added. If it receives a target architecture, according "-m32"/"-m64" and
"-march=xxxx" will be passed to gcc. Only the generic makefile has been
changed to support this option right now as the need only appeared on Linux.
The spec file now makes use of this option so that rpmbuild can automatically
build with the proper architecture.
It's useful to be able to accept an invalid header name in a request
or response but still be able to monitor further such errors. Now,
when an invalid request/response is received and accepted due to
an "accept-invalid-http-{request|response}" option, the invalid
request will be captured for later analysis with "show errors" on
the stats socket.
