Using get_ip_from_hdr2() we can look for occurrence #X or #-X and
extract the IP it contains. This is typically designed for use with
the X-Forwarded-For header.
Using "usesrc hdr_ip(name,occ)", it becomes possible to use the IP address
found in <name>, and possibly specify occurrence number <occ>, as the
source to connect to a server. This is possible both in a server and in
a backend's source statement. This is typically used to use the source
IP previously set by a upstream proxy.
Released version 1.4.3 with the following main changes :
- [CLEANUP] stats: remove printf format warning in stats_dump_full_sess_to_buffer()
- [MEDIUM] session: better fix for connection to servers with closed input
- [DOC] indicate in the doc how to bind to port ranges
- [BUG] backend: L7 hashing must not be performed on incomplete requests
- [TESTS] add a simple program to test connection resets
- [MINOR] cli: "show errors" should display "backend <NONE>" when backend was not used
- [MINOR] config: emit warnings when HTTP-only options are used in TCP mode
- [MINOR] config: allow "slowstart 0s"
- [BUILD] 'make tags' did not consider files ending in '.c'
- [MINOR] checks: add the ability to disable a server in the config
Released version 1.4.2 with the following main changes :
- [CLEANUP] product branch update
- [DOC] Some more documentation cleanups
- [BUG] clf logs segfault when capturing a non existant header
- [OPTIM] config: only allocate check buffer when checks are enabled
- [MEDIUM] checks: support multi-packet health check responses
- [CLEANUP] session: remove duplicate test
- [BUG] http: don't wait for response data to leave buffer is client has left
- [MINOR] proto_uxst: set accept_date upon accept() to the wall clock time
- [MINOR] stats: don't send empty lines in "show errors"
- [MINOR] stats: make the data dump function reusable for other purposes
- [MINOR] stats socket: add show sess <id> to dump details about a session
- [BUG] stats: connection reset counters must be plain ascii, not HTML
- [BUG] url_param hash may return a down server
- [MINOR] force null-termination of hostname
- [MEDIUM] connect to servers even when the input has already been closed
- [BUG] don't merge anonymous ACLs !
- [BUG] config: fix endless loop when parsing "on-error"
- [MINOR] http: don't mark a server as failed when it returns 501/505
- [OPTIM] checks: try to detect the end of response without polling again
- [BUG] checks: don't report an error when recv() returns an error after data
- [BUG] checks: don't abort when second poll returns an error
- [MINOR] checks: make shutdown() silently fail
- [BUG] http: fix truncated responses on chunk encoding when size divides buffer size
- [BUG] init: unconditionally catch SIGPIPE
- [BUG] checks: don't wait for a close to start parsing the response
Since the last documentation cleanups, I've found more typos that I kept
in a corner instead of sending you a mail just for one character :)
--
Cyril Bonté
When trying to spot some complex bugs, it's often needed to access
information on stuck sessions, which is quite difficult. This new
command helps one get detailed information about a session, with
flags, timers, states, etc... The buffer data are not dumped yet.
Released version 1.4.1 with the following main changes :
- [BUG] Clear-cookie path issue
- [DOC] fix typo on stickiness rules
- [BUILD] fix BSD and OSX makefiles for missing files
- [BUILD] includes order breaks OpenBSD build
- [BUILD] fix some build warnings on Solaris with is* macros
- [BUG] logs: don't report "last data" when we have just closed after an error
- [BUG] logs: don't report "proxy request" when server closes early
- [BUILD] fix platform-dependant build issues related to crypt()
- [STATS] count transfer aborts caused by client and by server
- [STATS] frontend requests were not accounted for failed requests
- [MINOR] report total number of processed connections when stopping a proxy
- [DOC] be more clear about the limitation to one single monitor-net entry
Often we need to understand why some transfers were aborted or what
constitutes server response errors. With those two counters, it is
now possible to detect an unexpected transfer abort during a data
phase (eg: too short HTTP response), and to know what part of the
server response errors may in fact be assigned to aborted transfers.
Released version 1.4.0 with the following main changes :
- [MINOR] stats: report maint state for tracking servers too
- [DOC] fix summary to add pattern extraction
- [DOC] Documentation cleanups
- [BUG] cfgparse memory leak and missing free calls in deinit()
- [BUG] pxid/puid/luid: don't shift IDs when some of them are forced
- [EXAMPLES] add auth.cfg
- [BUG] uri_auth: ST_SHLGNDS should be 0x00000008 not 0x0000008
- [BUG] uri_auth: do not attemp to convert uri_auth -> http-request more than once
- [BUILD] auth: don't use unnamed unions
- [BUG] config: report unresolvable host names as errors
- [BUILD] fix build breakage with DEBUG_FULL
- [DOC] fix a typo about timeout check and clarify the explanation.
- [MEDIUM] http: don't use trash to realign large buffers
- [STATS] report HTTP requests (total and rate) in frontends
- [STATS] separate frontend and backend HTTP stats
- [MEDIUM] http: revert to use a swap buffer for realignment
- [MINOR] stats: report the request rate in frontends as cell titles
- [MINOR] stats: mark areas with an underline when tooltips are available
- [DOC] reorder some entries to maintain the alphabetical order
- [DOC] cleanup of the keyword matrix
Hi Willy,
This patch make some cleanups on the documentation :
- strictly respect the 80 columns constraint
- use space indentation everywhere
- add some missing "(deprecated)"
- fix the "monitor fail" keyword syntax
- fix a keyword in a "See Also" line (interval instead of inter)
- the "no" prefix of "option http-use-proxy-header" is displayed as the others
I've also noticed that 3 keywords are missing from the keywords reference (I don't know what to do for that) :
- dispatch
- reqsetbe
- reqisetbe
--
Cyril Bonté
Released version 1.4-rc1 with the following main changes :
- [MEDIUM] add a maintenance mode to servers
- [MINOR] http-auth: last fix was wrong
- [CONTRIB] add base64rev-gen.c that was used to generate the base64rev table.
- [MINOR] Base64 decode
- [MINOR] generic auth support with groups and encrypted passwords
- [MINOR] add ACL_TEST_F_NULL_MATCH
- [MINOR] http-request: allow/deny/auth support for frontend/backend/listen
- [MINOR] acl: add http_auth and http_auth_group
- [MAJOR] use the new auth framework for http stats
- [DOC] add info about userlists, http-request and http_auth/http_auth_group acls
- [STATS] make it possible to change a CLI connection timeout
- [BUG] patterns: copy-paste typo in type conversion arguments
- [MINOR] pattern: make the converter more flexible by supporting void* and int args
- [MINOR] standard: str2mask: string to netmask converter
- [MINOR] pattern: add support for argument parsers for converters
- [MINOR] pattern: add the "ipmask()" converting function
- [MINOR] config: off-by-one in "stick-table" after list of converters
- [CLEANUP] acl, patterns: make use of my_strndup() instead of malloc+memcpy
- [BUG] restore accidentely removed line in last patch !
- [MINOR] checks: make the HTTP check code add the CRLF itself
- [MINOR] checks: add the server's status in the checks
- [BUILD] halog: make without arch-specific optimizations
- [BUG] halog: fix segfault in case of empty log in PCT mode (cherry picked from commit fe362fe476)
- [MINOR] http: disable keep-alive when process is going down
- [MINOR] acl: add build_acl_cond() to make it easier to add ACLs in config
- [CLEANUP] config: use build_acl_cond() instead of parse_acl_cond()
- [CLEANUP] config: use warnif_cond_requires_resp() to check for bad ACLs
- [MINOR] prepare req_*/rsp_* to receive a condition
- [CLEANUP] config: specify correct const char types to warnif_* functions
- [MEDIUM] config: factor out the parsing of 20 req*/rsp* keywords
- [MEDIUM] http: make the request filter loop check for optional conditions
- [MEDIUM] http: add support for conditional request filter execution
- [DOC] add some build info about the AIX platform (cherry picked from commit e41914c77e)
- [MEDIUM] http: add support for conditional request header addition
- [MEDIUM] http: add support for conditional response header rewriting
- [DOC] add some missing ACLs about response header matching
- [MEDIUM] http: add support for proxy authentication
- [MINOR] http-auth: make the 'unless' keyword work as expected
- [CLEANUP] config: use build_acl_cond() to simplify http-request ACL parsing
- [MEDIUM] add support for anonymous ACLs
- [MEDIUM] http: switch to tunnel mode after status 101 responses
- [MEDIUM] http: stricter processing of the CONNECT method
- [BUG] config: reset check request to avoid double free when switching to ssl/sql
- [MINOR] config: fix too large ssl-hello-check message.
- [BUG] fix error response in case of server error
A 101 response is accompanied with an Upgrade header indicating
a new protocol that is spoken on the connection after the exchange
completes. At least we should switch to tunnel mode after such a
response.
Anonymous ACLs allow the declaration of rules which rely directly on
ACL expressions without passing via the declaration of an ACL. Example :
With named ACLs :
acl site_dead nbsrv(dynamic) lt 2
acl site_dead nbsrv(static) lt 2
monitor fail if site_dead
With anonymous ACLs :
monitor fail if { nbsrv(dynamic) lt 2 } || { nbsrv(static) lt 2 }
This is a first attempt to add a maintenance mode on servers, using
the stat socket (in admin level).
It can be done with the following command :
- disable server <backend>/<server>
- enable server <backend>/<server>
In this mode, no more checks will be performed on the server and it
will be marked as a special DOWN state (MAINT).
If some servers were tracking it, they'll go DOWN until the server
leaves the maintenance mode. The stats page and the CSV export also
display this special state.
This can be used to disable the server in haproxy before doing some
operations on this server itself. This is a good complement to the
"http-check disable-on-404" keyword and works in TCP mode.
We're already able to know if a request is a proxy request or a
normal one, and we have an option "http-use-proxy-header" which states
that proxy headers must be checked. So let's switch to use the proxy
authentication headers and responses when this option is set and we're
facing a proxy request. That allows haproxy to enforce auth in front
of a proxy.
Just as for the req* rules, we can now condition rsp* rules with ACLs.
ACLs match on response, so volatile request information cannot be used.
A warning is emitted if a configuration contains such an anomaly.
All the req* rules except the reqadd rules can now be specified with
an if/unless condition. If a condition is specified and does not match,
the filter is ignored. This is particularly useful with reqidel, reqirep
and reqtarpit.
Now a server can check the contents of the header X-Haproxy-Server-State
to know how haproxy sees it. The same values as those reported in the stats
are provided :
- up/down status + check counts
- throttle
- weight vs backend weight
- active sessions vs backend sessions
- queue length
- haproxy node name
This converter can be applied on top of an IPv4-type pattern. It
applies a netmask which is suited for IP address storage and matching.
This can be used to make all hosts within a certain mask to share the
same table entries and as such use the same server.
The mask can be passed in dotted form (eg: 255.255.255.0) or in CIDR
form (eg: 24).
Hi Willy,
I've made a quick pass on the "defaults" column in the Proxy keywords matrix (chapter 4.1. in the documentation).
This patch resyncs the code and the documentation. I let you decide if some keywords that still work in the "defaults" section should be forbidden.
- default_backend : in the matrix, "defaults" was not supported but the keyword details say it is.
Tests also shows it works, then I've updated the matrix.
- capture cookie : in the keyword details, we can read `It is not possible to specify a capture in a "defaults" section.'.
Ok, even if the tests worked, I've added an alert in the configuration parser (as it is for capture request/response header).
- description : not supported in "defaults", I added an alert in the parser.
I've also noticed that this keyword doesn't appear in the documentation.
There's one "description" entry, but for the "global" section, which is for a different use (the patch doesn't update the documentation).
- grace : even if this is maybe useless, it works in "defaults". Documentation is updated.
- redirect : alert is added in the parser.
- rsprep : alert added in the parser.
--
Cyril Bonté
Despite what is explicitly stated in HTTP specifications,
browsers still use the undocumented Proxy-Connection header
instead of the Connection header when they connect through
a proxy. As such, proxies generally implement support for
this stupid header name, breaking the standards and making
it harder to support keep-alive between clients and proxies.
Thus, we add a new "option http-use-proxy-header" to tell
haproxy that if it sees requests which look like proxy
requests, it should use the Proxy-Connection header instead
of the Connection header.
Released version 1.4-dev7 with the following main changes :
- [BUG] appsession: possible memory leak in case of out of memory condition
- [MINOR] config: don't accept 'appsession' in defaults section
- [MINOR] Add function to parse a size in configuration
- [MEDIUM] Add stick table (persistence) management functions and types
- [MEDIUM] Add pattern fetch management types and functions
- [MEDIUM] Add src dst and dport pattern fetches.
- [MEDIUM] Add stick table configuration and init.
- [MEDIUM] Add stick and store rules analysers.
- [MINOR] add option "mysql-check" to use MySQL health checks
- [BUG] health checks: fix requeued message
- [OPTIM] remove SSP_O_VIA and SSP_O_STATUS
- [BUG] checks: fix newline termination
- [MINOR] acl: add fe_id/so_id to match frontend's and socket's id
- [BUG] appsession's sessid must be reset at end of transaction
- [BUILD] appsession did not build anymore under gcc-2.95
- [BUG] server redirection used an uninitialized string.
- [MEDIUM] http: fix handling of message pointers
- [MINOR] http: fix double slash prefix with server redirect
- [MINOR] http redirect: add the ability to append a '/' to the URL
- [BUG] stream_interface: fix retnclose and remove cond_close
- [MINOR] http redirect: don't explicitly state keep-alive on 1.1
- [MINOR] http: move appsession 'sessid' from session to http_txn
- [OPTIM] reorder http_txn to optimize cache lines placement
- [MINOR] http: differentiate waiting for new request and waiting for a complete requst
- [MINOR] http: add a separate "http-keep-alive" timeout
- [MINOR] config: remove undocumented and buggy 'timeout appsession'
- [DOC] fix various too large lines
- [DOC] remove several trailing spaces
- [DOC] add the doc about stickiness
- [BUILD] remove a warning in standard.h on AIX
- [BUG] checks: chars are unsigned on AIX, check was always true
- [CLEANUP] stream_sock: MSG_NOSIGNAL is only for send(), not recv()
- [BUG] check: we must not check for error before reading a response
- [BUG] buffers: remove remains of wrong obsolete length check
- [OPTIM] stream_sock: don't shutdown(write) when the socket is in error
- [BUG] http: don't count req errors on client resets or t/o during keep-alive
- [MEDIUM] http: don't switch to tunnel mode upon close
- [DOC] add documentation about connection header processing
- [MINOR] http: add http_remove_header2() to remove a header value.
- [MINOR] tools: add a "word_match()" function to match words and ignore spaces
- [MAJOR] http: rework request Connection header handling
- [MAJOR] http: rework response Connection header handling
- [MINOR] add the ability to force kernel socket buffer size.
- [BUG] http_server_error() must not purge a previous pending response
- [OPTIM] http: don't delay response if next request is incomplete
- [MINOR] add the "force-persist" statement to force persistence on down servers
- [MINOR] http: logs must report persistent connections to down servers
- [BUG] buffer_replace2 must never change the ->w entry
This is used to force access to down servers for some requests. This
is useful when validating that a change on a server correctly works
before enabling the server again.
Sometimes we need to be able to change the default kernel socket
buffer size (recv and send). Four new global settings have been
added for this :
- tune.rcvbuf.client
- tune.rcvbuf.server
- tune.sndbuf.client
- tune.sndbuf.server
Those can be used to reduce kernel memory footprint with large numbers
of concurrent connections, and to reduce risks of write timeouts with
very slow clients due to excessive kernel buffering.
This one is the next step of previous patch. It correctly computes
the response mode and the Connection flag transformations depending
on the request mode and version, and the response version and headers.
We're now also able to add "Connection: keep-alive", and to convert
server's close during a keep-alive connection to a server-close
connection.
The connection header is complex to handle, especially in the response
path, depending on request and response HTTP versions, desired mode,
etc... Let's document it. Note that only a subset of this document is
currently implemented.
Sometimes it can be desired to return a location which is the same
as the request with a slash appended when there was not one in the
request. A typical use of this is for sending a 301 so that people
don't reference links without the trailing slash. The name of the
new option is "append-slash" and it can be used on "redirect"
statements in prefix mode.
Released version 1.4-dev6 with the following main changes :
- [BUILD] warning in stream_interface.h
- [BUILD] warning ultoa_r returns char *
- [MINOR] hana: only report stats if it is enabled
- [MINOR] stats: add "a link" & "a href" for sockets
- [MINOR]: stats: add show-legends to report additional informations
- [MEDIUM] default-server support
- [BUG]: add 'observer', 'on-error', 'error-limit' to supported options list
- [MINOR] stats: add href to tracked server
- [BUG] stats: show UP/DOWN status also in tracking servers
- [DOC] Restore ability to search a keyword at the beginning of a line
- [BUG] stats: cookie should be reported under backend not under proxy
- [BUG] cfgparser/stats: fix error message
- [BUG] http: disable auto-closing during chunk analysis
- [BUG] http: fix hopefully last closing issue on data forwarding
- [DEBUG] add an http_silent_debug function to debug HTTP states
- [MAJOR] http: fix again the forward analysers
- [BUG] http_process_res_common() must not skip the forward analyser
- [BUG] http: some possible missed close remain in the forward chain
- [BUG] http: redirect needed to be updated after recent changes
- [BUG] http: don't set no-linger on response in case of forced close
- [MEDIUM] http: restore the original behaviour of option httpclose
- [TESTS] add a file to test various connection modes
- [BUG] http: check options before the connection header
- [MAJOR] session: fix the order by which the analysers are run
- [MEDIUM] session: also consider request analysers added during response
- [MEDIUM] http: make safer use of the DONT_READ and AUTO_CLOSE flags
- [BUG] http: memory leak with captures when using keep-alive
- [BUG] http: fix for capture memory leak was incorrect
- [MINOR] http redirect: use proper call to return last response
- [MEDIUM] http: wait for some flush of the response buffer before a new request
- [MEDIUM] session: limit the number of analyser loops
This patch implements default-server support allowing to change
default server options. It can be used in [defaults] or [backend]/[listen]
sections. Currently the following options are supported:
- error-limit
- fall
- inter
- fastinter
- downinter
- maxconn
- maxqueue
- minconn
- on-error
- port
- rise
- slowstart
- weight
Supported informations, available via "tr/td title":
- cap: capabilities (proxy)
- mode: one of tcp, http or health (proxy)
- id: SNMP ID (proxy, socket, server)
- IP (socket, server)
- cookie (backend, server)
Historically, "option httpclose" has always worked the same way. It
only mangles the "Connection" header in the request and the response
if needed, but does not affect the connection by itself, and ignores
any further data. It is dangerous to change this behaviour without
leaving any other alternative. If an active close is desired, it's
better to make use of "option forceclose" which does exactly what
it intends to do.
So as of now, "option httpclose" will only mangle the headers as
before, and will only affect the connection by itself when combined
with another connection-related option (eg: keepalive or server-close).
