By default, when data is sent over a socket, both the write timeout and the
read timeout for that socket are refreshed, because we consider that there is
activity on that socket, and we have no other means of guessing if we should
receive data or not.
While this default behaviour is desirable for almost all applications, there
exists a situation where it is desirable to disable it, and only refresh the
read timeout if there are incoming data. This happens on sessions with large
timeouts and low amounts of exchanged data such as telnet session. If the
server suddenly disappears, the output data accumulates in the system's
socket buffers, both timeouts are correctly refreshed, and there is no way
to know the server does not receive them, so we don't timeout. However, when
the underlying protocol always echoes sent data, it would be enough by itself
to detect the issue using the read timeout. Note that this problem does not
happen with more verbose protocols because data won't accumulate long in the
socket buffers.
When this option is set on the frontend, it will disable read timeout updates
on data sent to the client. There probably is little use of this case. When
the option is set on the backend, it will disable read timeout updates on
data sent to the server. Doing so will typically break large HTTP posts from
slow lines, so use it with caution.
(cherry picked from commit f27b5ea8dc)
This patch implements "description" (proxy and global) and "node" (global)
options, removes "node-name" and adds "show-node" & "show-desc" options
for "stats". It also changes the way the header lines (with proxy name) and
the statistics are displayed, so stats no longer look so clumsy with very
long names.
Instead of "node-name" it is possible to use show-node/show-desc with
an optional parameter that overrides a default node/description.
backend cust-0045
# report specific values for this customer
stats show-node Europe
stats show-desc Master node for Europe, Asia, Africa
(cherry picked from commit 48cb2aed5a)
HTTP supports status codes 100 and 101 to report protocol indications,
which are followed by the requests's response. Till now, haproxy would
only see those responses without parsing subsequent ones. That means
that cookie additions were only performed on 1xx messages for instance,
which does not work since headers must be ignored with 1xx messages.
Also, logs were not terribly useful with the common 100 status code
in response to "Expect: 100-continue" during POST some requests.
This change adds support for such messages. Now haproxy sees them,
forwards them and skips them until it finds a correct response, which
it logs and processes. As an exception, header removal/rewriting still
work on 1xx responses in order to be able to strip out sensible
information that may have accidentely been left by another equipment
(possibly an older haproxy itself). But headers addition are disabled
however.
This change brings the ability to loop on response without data, which
is a starting point to support keepalive. The change is marked as major
as a few fixes had to be performed in the HTTP message parser.
Note: this change is sensible for version 1.3 but it appears correct
and has extensively been tested. Also it fixes a real misbehaviour.
(cherry picked from commit 816b979977)
The new "node-name" stats setting enables reporting of a node ID on
the stats page. It is possible to return the system's host name as
well as a specific name.
(cherry picked from commit 1d45b7cbae)
Released version 1.3.19 with the following main changes :
- [MINOR] startup: don't imply -q with -D
- [BUG] ensure that we correctly re-start old process in case of error
- [MEDIUM] add support for binding to source port ranges during connect
- [MEDIUM] support setting a server weight to zero
- [MINOR] make DEFAULT_MAXCONN user-configurable at build time
- [MEDIUM] config: split parser and checker in two functions
- [MEDIUM] config: support loading multiple configuration files
- [BUG] http: redirect rules were processed too early
- [CLEANUP] remove unused DEBUG_PARSE_NO_SPEEDUP define
- [BUG] default ACLs did not properly set the ->requires flag
- [BUILD] report commit date and not author's date as build date
- [BUG] stream_sock: always shutdown(SHUT_WR) before closing
- [BUG] stream_sock: don't stop reading when the poller reports an error
- [BUG] config: tcp-request content only accepts "if" or "unless"
- [BUG] task: fix possible timer drift after update
- [MINOR] stats: better displaying in MSIE
- [MINOR] config: improve error reporting in global section
- [MINOR] config: improve error reporting in listen sections
- [MINOR] config: the "capture" keyword is not allowed in backends
- [MINOR] config: improve error reporting when checking configuration
- [BUILD] fix a minor build warning on AIX
- [BUILD] use "git cmd" instead of "git-cmd"
- [CLEANUP] report 2009 not 2008 in the copyright banner.
- [MINOR] print usage on the stats sockets upon invalid commands
- [MINOR] acl: detect and report potential mistakes in ACLs
- [BUILD] fix incorrect printf arg count with tcp_splice
- [BUG] fix random pauses on last segment of a series
- [BUILD] add support for build under Cygwin
Sometimes it is useful to be able to set a server's weight to zero.
It allows the server to receive only persistent traffic but never
normal traffic.
(cherry picked from commit 6704d67d65)
Some users are already hitting the 64k source port limit when
connecting to servers. The system usually maintains a list of
unused source ports, regardless of the source IP they're bound
to. So in order to go beyond the 64k concurrent connections, we
have to manage the source ip:port lists ourselves.
The solution consists in assigning a source port range to each
server and use a free port in that range when connecting to that
server, either for a proxied connection or for a health check.
The port must then be put back into the server's range when the
connection is closed.
This mechanism is used only when a port range is specified on
a server. It makes it possible to reach 64k connections per
server, possibly all from the same IP address. Right now it
should be more than enough even for huge deployments.
(cherry picked from commit c6f4ce8fc4)
Released version 1.3.18 with the following main changes :
- [MEDIUM] add support for "balance hdr(name)"
- [CLEANUP] give a little bit more information in error message
- [MINOR] add X-Original-To: header
- [BUG] x-original-to: fix missing initialization to default value
- [BUILD] spec file: fix broken pipe during rpmbuild and add man file
- [MINOR] improve reporting of misplaced acl/reqxxx rules
- [MEDIUM] http: add options to ignore invalid header names
- [MEDIUM] http: capture invalid requests/responses even if accepted
- [BUILD] add format(printf) to printf-like functions
- [MINOR] fix several printf formats and missing arguments
- [BUG] stats: total and lbtot are unsigned
- [MINOR] fix a few remaining printf-like formats on 64-bit platforms
- [CLEANUP] remove unused make option from haproxy.spec
- [BUILD] make it possible to pass alternative arch at build time
- [MINOR] switch all stat counters to 64-bit
- [MEDIUM] ensure we don't recursively call pool_gc2()
- [CRITICAL] uninitialized response field can sometimes cause crashes
- [BUG] fix wrong pointer arithmetics in HTTP message captures
- [MINOR] rhel init script : support the reload operation
- [MINOR] add basic signal handling functions
- [BUILD] add signal.o to all makefiles
- [MEDIUM] call signal_process_queue from run_poll_loop
- [MEDIUM] pollers: don't wait if a signal is pending
- [MEDIUM] convert all signals to asynchronous signals
- [BUG] O(1) pollers should check their FD before closing it
- [MINOR] don't close stdio fds twice
- [MINOR] add options dontlog-normal and log-separate-errors
- [DOC] minor fixes and rearrangements
- [BUG] fix parser crash on unconditional tcp content rules
- [DOC] rearrange the configuration manual and add a summary
- [MINOR] standard: provide a new 'my_strndup' function
- [MINOR] implement per-logger log level limitation
- [MINOR] compute the max of sessions/s on fe/be/srv
- [MINOR] stats: report max sessions/s and limit in CSV export
- [MINOR] stats: report max sessions/s and limit in HTML stats
- [MINOR] stats/html: use the arial font before helvetica
Some people are using haproxy in a shared environment where the
system logger by default sends alert and emerg messages to all
consoles, which happens when all servers go down on a backend for
instance. These people can not always change the system configuration
and would like to limit the outgoing messages level in order not to
disturb the local users.
The addition of an optional 4th field on the "log" line permits
exactly this. The minimal log level ensures that all outgoing logs
will have at least this level. So the logs are not filtered out,
just set to this level.
There is a patch made by me that allow for balancing on any http header
field.
[WT:
made minor changes:
- turned 'balance header name' into 'balance hdr(name)' to match more
closely the ACL syntax for easier future convergence
- renamed the proxy structure fields header_* => hh_*
- made it possible to use the domain name reduction to any header, not
only "host" since it makes sense to do it with other ones.
Otherwise patch looks good.
/WT]
Several people have asked for a summary in order to ease finding
of sections in the configuration manual. It was the opportunity to
tidy it up a bit and rearrange some sections.
Some big traffic sites have trouble dealing with logs and tend to
disable them. Here are two new options to help cope with massive
logs.
- dontlog-normal only disables logging for 100% successful
connections, other ones will still be logged
- log-separate-errors will cause non-100% successful connections
to be logged at level "err" instead of level "info" so that a
properly configured syslog daemon can send them to a different
file for longer conservation.
I have attached a patch which will add on every http request a new
header 'X-Original-To'. If you have HAProxy running in transparent mode
with a big number of SQUID servers behind it, it is very nice to have
the original destination ip as a common header to make decisions based
on it.
The whole thing is configurable with a new option 'originalto'. I have
updated the sourcecode as well as the documentation. The 'haproxy-en.txt'
and 'haproxy-fr.txt' files are untouched, due to lack of my french
language knowledge. ;)
Also the patch adds this header for IPv4 only. I haven't any IPv6 test
environment running here and don't know if getsockopt() with SO_ORIGINAL_DST
will work on IPv6. If someone knows it and wants to test it I can modify
the diff. Feel free to ask me questions or things which should be changed. :)
--Maik
It's useful to be able to accept an invalid header name in a request
or response but still be able to monitor further such errors. Now,
when an invalid request/response is received and accepted due to
an "accept-invalid-http-{request|response}" option, the invalid
request will be captured for later analysis with "show errors" on
the stats socket.
Released version 1.3.17 with the following main changes :
- Update specfile to build for v2.6 kernel.
- [BUG] reset the stream_interface connect timeout upon connect or error
- [BUG] reject unix accepts when connection limit is reached
- [MINOR] show sess: report number of calls to each task
- [BUG] don't call epoll_ctl() on closed sockets
- [BUG] stream_sock: disable I/O on fds reporting an error
- [MINOR] sepoll: don't count two events on the same FD.
- [MINOR] show sess: report a lot more information about sessions
- [BUG] stream_sock: check for shut{r,w} before refreshing some timeouts
- [BUG] don't set an expiration date directly from now_ms
- [MINOR] implement ulltoh() to write HTML-formatted numbers
- [MINOR] stats/html: group digits by 3 to clarify numbers
- [BUILD] remove haproxy-small.spec
- [BUILD] makefile: remove unused references to linux24eold and EPOLL_CTL_WORKAROUND
Released version 1.3.16 with the following main changes :
- [BUILD] Fixed Makefile for linking pcre
- [CONTRIB] selinux policy for haproxy
- [MINOR] show errors: encode backslash as well as non-ascii characters
- [MINOR] cfgparse: some cleanups in the consistency checks
- [MINOR] cfgparse: set backends to "balance roundrobin" by default
- [MINOR] tcp-inspect: permit the use of no-delay inspection
- [MEDIUM] reverse internal proxy declaration order to match configuration
- [CLEANUP] config: catch and report some possibly wrong rule ordering
- [BUG] connect timeout is in the stream interface, not the buffer
- [BUG] session: errors were not reported in termination flags in TCP mode
- [MINOR] tcp_request: let the caller take care of errors and timeouts
- [CLEANUP] http: remove some commented out obsolete code in process_response
- [MINOR] update ebtree to version 4.1
- [MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1
- [BUG] sched: don't leave 3 lasts tasks unprocessed when niced tasks are present
- [BUG] scheduler: fix improper handling of duplicates __task_queue()
- [MINOR] sched: permit a task to stay up between calls
- [MINOR] task: keep a task count and clean up task creators
- [MINOR] stats: report number of tasks (active and running)
- [BUG] server check intervals must not be null
- [OPTIM] stream_sock: don't retry to read after a large read
- [OPTIM] buffer: new BF_READ_DONTWAIT flag reduces EAGAIN rates
- [MEDIUM] session: don't resync FSMs on non-interesting changes
- [BUG] check for global.maxconn before doing accept()
- [OPTIM] sepoll: do not re-check whole list upon accepts
Sometimes it may make sense to be able to immediately apply a verdict
without waiting at all. It was not possible because no inspect-delay
meant no inspection at all. This is now fixed.
When a backend has no LB algo specified and is not in dispatch, proxy
nor transparent mode, use "balance roundrobin" by default instead of
complaining. This will be particularly useful with stats and redirects.
Released version 1.3.16-rc1 with the following main changes :
- appsessions: cleanup DEBUG_HASH and initialize request_counter
- [MINOR] acl: add new keyword "connslots"
- [MINOR] cfgparse: fix off-by 2 in error message size
- [BUILD] fix build with gcc 4.3
- [BUILD] fix MANDIR default location to match documentation
- [TESTS] add a debug patch to help trigger the stats bug
- [BUG] Flush buffers also where there are exactly 0 bytes left
- [MINOR] Allow to specify a domain for a cookie
- [BUG/CLEANUP] cookiedomain -> cookie_domain rename + free(p->cookie_domain)
- [MEDIUM] Fix memory freeing at exit
- [MEDIUM] Fix memory freeing at exit, part 2
- [BUG] Fix listen & more of 2 couples <ip>:<port>
- [DOC] remove buggy comment for use_backend
- [CRITICAL] fix server state tracking: it was O(n!) instead of O(n)
- [MEDIUM] add support for URI hash depth and length limits
- [MINOR] permit renaming of x-forwarded-for header
- [BUILD] fix Makefile.bsd and Makefile.osx for stream_interface
- [BUILD] Haproxy won't compile if DEBUG_FULL is defined
- [MEDIUM] upgrade to ebtree v4.0
- [DOC] update the README file with new build options
- [MEDIUM] reduce risk of event starvation in ev_sepoll
- [MEDIUM] detect streaming buffers and tag them as such
- [MEDIUM] add support for conditional HTTP redirection
- [BUILD] make install should depend on haproxy not "all"
- [DEBUG] add a TRACE macro to facilitate runtime data extraction
- [BUG] event pollers must not wait if a task exists in the run queue
- [BUG] queue management: wake oldest request in queues
- [BUG] log: reported queue position was offed-by-one
- [BUG] fix the dequeuing logic to ensure that all requests get served
- [DOC] documentation for the "retries" parameter was missing.
- [MEDIUM] implement a monotonic internal clock
- [MEDIUM] further improve monotonic clock by check forward jumps
- [OPTIM] add branch prediction hints in list manipulations
- [MAJOR] replace ultree with ebtree in wait-queues
- [BUG] we could segfault during exit while freeing uri_auths
- [BUG] wqueue: perform proper timeout comparisons with wrapping values
- [MINOR] introduce now_ms, the current date in milliseconds
- [BUG] disable buffer read timeout when reading stats
- [MEDIUM] rework the wait queue mechanism
- [BUILD] change declaration of base64tab to fix build with Intel C++
- [OPTIM] shrink wake_expired_tasks() by using task_wakeup()
- [MAJOR] use an ebtree instead of a list for the run queue
- [MEDIUM] introduce task->nice and boot access to statistics
- [OPTIM] task_queue: assume most consecutive timers are equal
- [BUILD] silent a warning in unlikely() with gcc 4.x
- [MAJOR] convert all expiration timers from timeval to ticks
- [BUG] use_backend would not correctly consider "unless"
- [TESTS] added test-acl.cfg to test some ACL combinations
- [MEDIUM] add support for configuration keyword registration
- [MEDIUM] modularize the global "stats" keyword configuration parser
- [MINOR] cfgparse: add support for warnings in external functions
- [MEDIUM] modularize the "timeout" keyword configuration parser
- [MAJOR] implement tcp request content inspection
- [MINOR] acl: add a new parsing function: parse_dotted_ver
- [MINOR] acl: add req_ssl_ver in TCP, to match an SSL version
- [CLEANUP] remove unused include/types/client.h
- [CLEANUP] remove many #include <types/xxx> from C files
- [CLEANUP] remove dependency on obsolete INTBITS macro
- [DOC] document the new "tcp-request" keyword and associated ACLs
- [MINOR] acl: add REQ_CONTENT to the list of default acls
- [MEDIUM] acl: permit fetch() functions to set the result themselves
- [MEDIUM] acl: get rid of dummy values in always_true/always_false
- [MINOR] acl: add the "wait_end" acl verb
- [MEDIUM] acl: enforce ACL type checking
- [MEDIUM] acl: set types on all currently known ACL verbs
- [MEDIUM] acl: when possible, report the name and requirements of ACLs in warnings
- [CLEANUP] remove 65 useless NULL checks before free
- [MEDIUM] memory: update pool_free2() to support NULL pointers
- [MEDIUM] buffers: ensure buffer_shut* are properly called upon shutdowns
- [MEDIUM] process_srv: rely on buffer flags for client shutdown
- [MEDIUM] process_srv: don't rely at all on client state
- [MEDIUM] process_cli: don't rely at all on server state
- [BUG] fix segfault with url_param + check_post
- [BUG] server timeout was not considered in some circumstances
- [BUG] client timeout incorrectly rearmed while waiting for server
- [MAJOR] kill CL_STINSPECT and CL_STHEADERS (step 1)
- [MAJOR] get rid of SV_STANALYZE (step 2)
- [MEDIUM] simplify and centralize request timeout cancellation and request forwarding
- [MAJOR] completely separate HTTP and TCP states on the request path
- [BUG] fix recently introduced loop when client closes early
- [MAJOR] get rid of the SV_STHEADERS state
- [MAJOR] better separation of response processing and server state
- [MAJOR] clearly separate HTTP response processing from TCP server state
- [MEDIUM] remove unused references to {CL|SV}_STSHUT*
- [MINOR] term_trace: add better instrumentations to trace the code
- [BUG] ev_sepoll: closed file descriptors could persist in the spec list
- [BUG] process_response must not enable the read FD
- [BUG] buffers: remove BF_MAY_CONNECT and fix forwarding issue
- [BUG] process_response: do not touch srv_state
- [BUG] maintain_proxies must not disable backends
- [CLEANUP] get rid of BF_SHUT*_PENDING
- [MEDIUM] buffers: add BF_EMPTY and BF_FULL to remove dependency on req/rep->l
- [MAJOR] process_session: rely only on buffer flags
- [MEDIUM] use buffer->wex instead of buffer->cex for connect timeout
- [MEDIUM] centralize buffer timeout checks at the top of process_session
- [MINOR] ensure the termination flags are set by process_xxx
- [MEDIUM] session: move the analysis bit field to the buffer
- [OPTIM] process_cli/process_srv: reduce the number of tests
- [BUG] regparm is broken on gcc < 3
- [BUILD] fix warning in proto_tcp.c with gcc >= 4
- [MEDIUM] merge inspect_exp and txn->exp into request buffer
- [BUG] process_cli/process_srv: don't call shutdown when already done
- [BUG] process_request: HTTP body analysis must return zero if missing data
- [TESTS] test-fsm: 22 regression tests for state machines
- [BUG] Fix empty X-Forwarded-For header name when set in defaults section
- [BUG] fix harmless but wrong fd insertion sequence
- [MEDIUM] make it possible for analysers to follow the whole session
- [MAJOR] rework of the server FSM
- [OPTIM] remove useless fd_set(read) upon shutdown(write)
- [MEDIUM] massive cleanup of process_srv()
- [MEDIUM] second level of code cleanup for process_srv_data
- [MEDIUM] third cleanup and optimization of process_srv_data()
- [MEDIUM] process_srv_data: ensure that we always correctly re-arm timeouts
- [MEDIUM] stream_sock_process_data moved to stream_sock.c
- [MAJOR] make the client side use stream_sock_process_data()
- [MEDIUM] split stream_sock_process_data
- [OPTIM] stream_sock_read must check for null-reads more often
- [MINOR] only call flow analysers when their read side is connected.
- [MEDIUM] reintroduce BF_HIJACK with produce_content
- [MINOR] re-arrange buffer flags and rename some of them
- [MINOR] do not check for BF_SHUTR when computing write timeout
- [OPTIM] ev_sepoll: detect newly created FDs and check them once
- [OPTIM] reduce the number of calls to task_wakeup()
- [OPTIM] force inlining of large functions with gcc >= 3
- [MEDIUM] indicate a reason for a task wakeup
- [MINOR] change type of fdtab[]->owner to void*
- [MAJOR] make stream sockets aware of the stream interface
- [MEDIUM] stream interface: add the ->shutw method as well as in and out buffers
- [MEDIUM] buffers: add BF_READ_ATTACHED and BF_ANA_TIMEOUT
- [MEDIUM] process_session: make use of the new buffer flags
- [CLEANUP] process_session: move debug outputs out of the critical loop
- [MEDIUM] move QUEUE and TAR timers to stream interfaces
- [OPTIM] add compiler hints in tick_is_expired()
- [MINOR] add buffer_check_timeouts() to check what timeouts have fired.
- [MEDIUM] use buffer_check_timeouts instead of stream_sock_check_timeouts()
- [MINOR] add an expiration flag to the stream_sock_interface
- [MAJOR] migrate the connection logic to stream interface
- [MAJOR] add a connection error state to the stream_interface
- [MEDIUM] add the SN_CURR_SESS flag to the session to track open sessions
- [MEDIUM] continue layering cleanups.
- [MEDIUM] stream_interface: added a DISconnected state between CON/EST and CLO
- [MEDIUM] remove stream_sock_update_data()
- [MINOR] maintain a global session list in order to ease debugging
- [BUG] shutw must imply close during a connect
- [MEDIUM] process shutw during connection attempt
- [MEDIUM] make the stream interface control the SHUT{R,W} bits
- [MAJOR] complete layer4/7 separation
- [CLEANUP] move the session-related functions to session.c
- [MINOR] call session->do_log() for logging
- [MINOR] replace the ambiguous client_return function by stream_int_return
- [MINOR] replace client_retnclose() with stream_int_retnclose()
- [MINOR] replace srv_close_with_err() with http_server_error()
- [MEDIUM] make the http server error function a pointer in the session
- [CLEANUP] session.c: removed some migration left-overs in sess_establish()
- [MINOR] stream_sock_data_finish() should not expose fd
- [MEDIUM] extract TCP request processing from HTTP
- [MEDIUM] extract the HTTP tarpit code from process_request().
- [MEDIUM] move the HTTP request body analyser out of process_request().
- [MEDIUM] rename process_request to http_process_request
- [BUG] fix forgotten server session counter
- [MINOR] declare process_session in session.h, not proto_http.h
- [MEDIUM] first pass of lifting to proto_uxst.c:uxst_event_accept()
- [MINOR] add an analyser code for UNIX stats request
- [MINOR] pre-set analyser flags on the listener at registration time
- [BUG] do not forward close from cons to prod with analysers
- [MEDIUM] ensure that sock->shutw() also closes read for init states
- [MINOR] add an analyser state in struct session
- [MAJOR] make unix sockets work again with stats
- [MEDIUM] remove cli_fd, srv_fd, cli_state and srv_state from the session
- [MINOR] move the listener reference from fd to session
- [MEDIUM] reference the current hijack function in the buffer itself
- [MINOR] slightly rebalance stats_dump_{raw,http}
- [MINOR] add a new back-reference type : struct bref
- [MINOR] add back-references to sessions for later use by a dumper.
- [MEDIUM] add support for "show sess" in unix stats socket
- [BUG] do not release the connection slot during a retry
- [BUG] dynamic connection throttling could return a max of zero conns
- [BUG] do not try to pause backends during reload
- [BUG] ensure that listeners from disabled proxies are correctly unbound.
- [BUG] acl-related keywords are not allowed in defaults sections
- [BUG] cookie capture is declared in the frontend but checked on the backend
- [BUG] critical errors should be reported even in daemon mode
- [MINOR] redirect: add support for the "drop-query" option
- [MINOR] redirect: add support for "set-cookie" and "clear-cookie"
- [MINOR] redirect: in prefix mode a "/" means not to change the URI
- [BUG] do not dequeue requests on a dead server
- [BUG] do not dequeue the backend's pending connections on a dead server
- [MINOR] stats: indicate if a task is running in "show sess"
- [BUG] check timeout must not be changed if timeout.check is not set
- [BUG] "option transparent" is for backend, not frontend !
- [MINOR] transfer errors were not reported anymore in data phase
- [MEDIUM] add a send limit to a buffer
- [MEDIUM] don't report buffer timeout when there is I/O activity
- [MEDIUM] indicate when we don't care about read timeout
- [MINOR] add flags to indicate when a stream interface is waiting for space/data
- [MEDIUM] enable inter-stream_interface wakeup calls
- [MAJOR] implement autonomous inter-socket forwarding
- [MINOR] add the splice_len member to the buffer struct in preparation of splice support
- [MEDIUM] stream_sock: factor out the return path in case of no-writes
- [MEDIUM] i/o: rework ->to_forward and ->send_max
- [OPTIM] stream_sock: do not ask for polling on EAGAIN if we have read
- [OPTIM] buffer: replace rlim by max_len
- [OPTIM] stream_sock: factor out the buffer full handling out of the loop
- [CLEANUP] replace a few occurrences of (flags & X) && !(flags & Y)
- [CLEANUP] stream_sock: move the write-nothing condition out of the loop
- [MEDIUM] split stream_sock_write() into callback and core functions
- [MEDIUM] stream_sock_read: call ->chk_snd whenever there are data pending
- [MINOR] stream_sock: fix a few wrong empty calculations
- [MEDIUM] stream_sock: try to send pending data on chk_snd()
- [MINOR] global.maxpipes: add the ability to reserve file descriptors for pipes
- [MEDIUM] splice: add configuration options and set global.maxpipes
- [MINOR] introduce structures required to support Linux kernel splicing
- [MEDIUM] add definitions for Linux kernel splicing
- [MAJOR] complete support for linux 2.6 kernel splicing
- [BUG] reserve some pipes for backends with splice enabled
- [MEDIUM] splice: add hints to support older buggy kernels
- [MEDIUM] introduce pipe pools
- [MEDIUM] splice: make use of pipe pools
- [STATS] report pipe usage in the statistics
- [OPTIM] make global.maxpipes default to global.maxconn/4 when not specified
- [BUILD] fix snapshot date extraction with negative timezones
- [MEDIUM] move global tuning options to the global structure
- [MEDIUM] splice: add the global "nosplice" option
- [BUILD] add USE_LINUX_SPLICE to enable LINUX_SPLICE on linux 2.6
- [BUG] we must not exit if protocol binding only returns a warning
- [MINOR] add support for bind interface name
- [BUG] inform the user when root is expected but not set
- [MEDIUM] add support for source interface binding
- [MEDIUM] add support for source interface binding at the server level
- [MEDIUM] implement bind-process to limit service presence by process
- [DOC] document maxpipes, nosplice, option splice-{auto,request,response}
- [DOC] filled the logging section of the configuration manual
- [DOC] document HTTP status codes
- [DOC] document a few missing info about errorfile
- [BUG] fix random memory corruption using "show sess"
- [BUG] fix unix socket processing of interrupted output
- [DOC] add diagrams of queuing and future ACL design
- [BUILD] proto_http did not build on gcc-2.95
- [BUG] the "source" keyword must first clear optional settings
- [BUG] global.tune.maxaccept must be limited even in mono-process mode
- [MINOR] ensure that http_msg_analyzer updates pointer to invalid char
- [MEDIUM] store a complete dump of request and response errors in proxies
- [MEDIUM] implement error dump on unix socket with "show errors"
- [DOC] document "show errors"
- [MINOR] errors dump must use user-visible date, not internal date.
- [MINOR] time: add __usec_to_1024th to convert usecs to 1024th of second
- [MINOR] add curr_sec_ms and curr_sec_ms_scaled for current second.
- [MEDIUM] measure and report session rate on frontend, backends and servers
- [BUG] the "connslots" keyword was matched as "connlots"
- [MINOR] acl: add 2 new verbs: fe_sess_rate and be_sess_rate
- [MEDIUM] implement "rate-limit sessions" for the frontend
- [BUG] interface binding: length must include the trailing zero
- [BUG] typo in timeout error reporting : report *res and not *err
- [OPTIM] maintain_proxies: only wake up when the frontend will be ready
- [OPTIM] rate-limit: cleaner behaviour on low rates and reduce consumption
- [BUG] switch server-side stream interface to close in case of abort
- [CLEANUP] remove last references to term_trace
- [OPTIM] freq_ctr: do not rotate the counters when reading
- [BUG] disable any analysers for monitoring requests
- [BUG] rate-limit in defaults section was ignored
- [BUG] task: fix handling of duplicate keys
- [OPTIM] task: don't unlink a task from a wait queue when waking it up
- [OPTIM] displace tasks in the wait queue only if absolutely needed
- [MEDIUM] minor update to the task api: let the scheduler queue itself
- [BUG] event_accept() must always wake the task up, even in health mode
- [CLEANUP] task: distinguish between clock ticks and timers
- [OPTIM] task: reduce the number of calls to task_queue()
- [OPTIM] do not re-check req buffer when only response has changed
- [CLEANUP] don't enable kernel splicing when socket is closed
- [CLEANUP] buffer_flush() was misleading, rename it as buffer_erase
- [MINOR] buffers: implement buffer_flush()
- [MEDIUM] rearrange forwarding condition to enable splice during analysis
- [BUILD] build fixes for Solaris
- [BUILD] proto_http did not build on gcc-2.95 (again)
- [CONTRIB] halog: fast log parser for haproxy
- [CONTRIB] halog: faster fgets() and add support for percentile reporting
The new "rate-limit sessions" statement sets a limit on the number of
new connections per second on the frontend. As it is extremely accurate
(about 0.1%), it is efficient at limiting resource abuse or DoS.
These new ACLs match frontend session rate and backend session rate.
Examples are provided in the doc to explain how to use that in order
to limit abuse of service.
With this change, all frontends, backends, and servers maintain a session
counter and a timer to compute a session rate over the last second. This
value will be very useful because it varies instantly and can be used to
check thresholds. This value is also reported in the stats in a new "rate"
column.
On overloaded systems, it sometimes happens that hundreds or thousands
of incoming connections are queued in the system's backlog, and all get
dequeued at once. The problem is that when haproxy processes them and
does not apply any limit, this can take some time and the internal date
does not progress, resulting in wrong timer measures for all sessions.
The most common effect of this is that all of these sessions report a
large request time (around several hundreds of ms) which is in fact
caused by the time spent accepting other connections. This might happen
on shared systems when the machine swaps.
For this reason, we finally apply a reasonable limit even in mono-process
mode. Accepting 100 connections at once is fast enough for extreme cases
and will not cause that much of a trouble when the system is saturated.
Some parts from the previous doc about logging have been merged and
updated. Most of those parts have been reworked and completed. The
examples are now accurate and reflect recent versions.
The "bind-process" keyword lets the admin select which instances may
run on which process (in multi-process mode). It makes it easier to
more evenly distribute the load across multiple processes by avoiding
having too many listen to the same IP:ports.
Specifying "interface <name>" after the "source" statement allows
one to bind to a specific interface for proxy<->server traffic.
This makes it possible to use multiple links to reach multiple
servers, and to force traffic to pass via an interface different
from the one the system would have chosen based on the routing
table.
By appending "interface <name>" to a "bind" line, it is now possible
to specifically bind to a physical interface name. Note that this
currently only works on Linux and requires root privileges.
"option transparent" was set and checked on frontends only while it
is purely a backend thing as it replaces the "balance" mode. For this
reason, it did only work in "listen" sections. This change will then
not affect the rare users of this option.
If the prefix is set to "/", it means the user does not want to alter
the original URI, so we don't want to insert a new slash before the
original URI.
(cherry-picked from commit 02a35c74942c1bce762e996698add1270e6a5030)
It is now possible to set or clear a cookie during a redirection. This
is useful for logout pages, or for protecting against some DoSes. Check
the documentation for the options supported by the "redirect" keyword.
(cherry-picked from commit 4af993822e880d8c932f4ad6920db4c9242b0981)
If "drop-query" is present on a "redirect" line using the "prefix" mode,
then the returned Location header will be the request URI without the
query-string. This may be used on some login/logout pages, or when it
must be decided to redirect the user to a non-secure server.
(cherry-picked from commit f2d361ccd73aa16538ce767c766362dd8f0a88fd)
I'm in the process of setting up one haproxy instance now, and I find
the following acl option useful. I'm not too sure why this option has
not been available before, but I find this useful for my own usage, so
I'm submitting this patch in the hope that it will be useful as well.
The basic idea is to be able to measure the available connection slots
still available (connection, + queue) - anything beyond that can be
redirected to a different backend. 'connslots' = number of available
server connection slots, + number of available server queue slots. In
the case where we encounter srv maxconn = 0, or srv maxqueue = 0 (in
which case we dont need to care about connslots) the value you get is
-1. Note also that this code does not take care of dynamic connections
at this point in time.
The reason why I'm using this new acl (as opposed to 'nbsrv') is that
'nbsrv' only measures servers that are actually *down*. Whereas this
other acl is more fine-grained, and looks into the number of conn
slots available as well.
It is now possible to list all known sessions by issuing "show sess"
on the unix stats socket. The format is not much evolved but it is
very useful for debugging.
The doc has been updated to reflect the new keyword.
There were rare situations where it was not easy to detect that a failed
session attempt had occurred and needed some server cleanup. In particular,
client aborts sometimes lead to session leaks on the server side.
A new state "SI_ST_DIS" (disconnected) has been introduced for this. When
a session has been closed at a stream interface but the server cleanup has
not occurred, this state is entered instead of CLO. The cleanup is then
performed there and the state goes to CLO.
A new diagram has been added to show possible stream_interface state
transitions that can occur in a stream-sock. It makes debugging easier.
Because I needed it in my situation - here's a quick patch to
allow changing of the "x-forwarded-for" header by using a suboption to
"option forwardfor".
Suboption "header XYZ" will set the header from "x-forwarded-for" to "XYZ".
Default is still "x-forwarded-for" if the header value isn't defined.
Also the suboption 'except a.b.c.d/z' still works on the same line.
So it's now: option forwardfor [except a.b.c.d[/z]] [header XYZ]
The new "wait_end" acl delays evaluation of the rule (and the next ones)
to the end of the analysis period. This is intented to be used with TCP
content analysis. A rule referencing such an ACL will not match until
the delay is over. An equivalent default ACL "WAIT_END" has been created.
With content inspection, checking the presence of data in the
request buffer is very important. It's getting boring to always
add such an ACL, so let's add it by default.
A new "redirect" keyword adds the ability to send an HTTP 301/302/303
redirection to either an absolute location or to a prefix followed by
the original URI. The redirection is conditionned by ACL rules, so it
becomes very easy to move parts of a site to another site using this.
This work was almost entirely done at Exceliance by Emeric Brun.
A test-case has been added in the tests/ directory.
