Commit Graph

247 Commits

Author SHA1 Message Date
Willy Tarreau
88d349d25d [MEDIUM] http: add support for Proxy-Connection header
Despite what is explicitly stated in HTTP specifications,
browsers still use the undocumented Proxy-Connection header
instead of the Connection header when they connect through
a proxy. As such, proxies generally implement support for
this stupid header name, breaking the standards and making
it harder to support keep-alive between clients and proxies.

Thus, we add a new "option http-use-proxy-header" to tell
haproxy that if it sees requests which look like proxy
requests, it should use the Proxy-Connection header instead
of the Connection header.
2010-01-25 12:48:26 +01:00
Willy Tarreau
6939b5522d [RELEASE] Released version 1.4-dev7
Released version 1.4-dev7 with the following main changes :
    - [BUG] appsession: possible memory leak in case of out of memory condition
    - [MINOR] config: don't accept 'appsession' in defaults section
    - [MINOR] Add function to parse a size in configuration
    - [MEDIUM] Add stick table (persistence) management functions and types
    - [MEDIUM] Add pattern fetch management types and functions
    - [MEDIUM] Add src dst and dport pattern fetches.
    - [MEDIUM] Add stick table configuration and init.
    - [MEDIUM] Add stick and store rules analysers.
    - [MINOR] add option "mysql-check" to use MySQL health checks
    - [BUG] health checks: fix requeued message
    - [OPTIM] remove SSP_O_VIA and SSP_O_STATUS
    - [BUG] checks: fix newline termination
    - [MINOR] acl: add fe_id/so_id to match frontend's and socket's id
    - [BUG] appsession's sessid must be reset at end of transaction
    - [BUILD] appsession did not build anymore under gcc-2.95
    - [BUG] server redirection used an uninitialized string.
    - [MEDIUM] http: fix handling of message pointers
    - [MINOR] http: fix double slash prefix with server redirect
    - [MINOR] http redirect: add the ability to append a '/' to the URL
    - [BUG] stream_interface: fix retnclose and remove cond_close
    - [MINOR] http redirect: don't explicitly state keep-alive on 1.1
    - [MINOR] http: move appsession 'sessid' from session to http_txn
    - [OPTIM] reorder http_txn to optimize cache lines placement
    - [MINOR] http: differentiate waiting for new request and waiting for a complete requst
    - [MINOR] http: add a separate "http-keep-alive" timeout
    - [MINOR] config: remove undocumented and buggy 'timeout appsession'
    - [DOC] fix various too large lines
    - [DOC] remove several trailing spaces
    - [DOC] add the doc about stickiness
    - [BUILD] remove a warning in standard.h on AIX
    - [BUG] checks: chars are unsigned on AIX, check was always true
    - [CLEANUP] stream_sock: MSG_NOSIGNAL is only for send(), not recv()
    - [BUG] check: we must not check for error before reading a response
    - [BUG] buffers: remove remains of wrong obsolete length check
    - [OPTIM] stream_sock: don't shutdown(write) when the socket is in error
    - [BUG] http: don't count req errors on client resets or t/o during keep-alive
    - [MEDIUM] http: don't switch to tunnel mode upon close
    - [DOC] add documentation about connection header processing
    - [MINOR] http: add http_remove_header2() to remove a header value.
    - [MINOR] tools: add a "word_match()" function to match words and ignore spaces
    - [MAJOR] http: rework request Connection header handling
    - [MAJOR] http: rework response Connection header handling
    - [MINOR] add the ability to force kernel socket buffer size.
    - [BUG] http_server_error() must not purge a previous pending response
    - [OPTIM] http: don't delay response if next request is incomplete
    - [MINOR] add the "force-persist" statement to force persistence on down servers
    - [MINOR] http: logs must report persistent connections to down servers
    - [BUG] buffer_replace2 must never change the ->w entry
2010-01-25 01:54:37 +01:00
Willy Tarreau
4de9149f87 [MINOR] add the "force-persist" statement to force persistence on down servers
This is used to force access to down servers for some requests. This
is useful when validating that a change on a server correctly works
before enabling the server again.
2010-01-22 19:10:05 +01:00
Willy Tarreau
e803de2c6b [MINOR] add the ability to force kernel socket buffer size.
Sometimes we need to be able to change the default kernel socket
buffer size (recv and send). Four new global settings have been
added for this :
   - tune.rcvbuf.client
   - tune.rcvbuf.server
   - tune.sndbuf.client
   - tune.sndbuf.server

Those can be used to reduce kernel memory footprint with large numbers
of concurrent connections, and to reduce risks of write timeouts with
very slow clients due to excessive kernel buffering.
2010-01-22 11:49:41 +01:00
Willy Tarreau
6046652253 [MAJOR] http: rework response Connection header handling
This one is the next step of previous patch. It correctly computes
the response mode and the Connection flag transformations depending
on the request mode and version, and the response version and headers.

We're now also able to add "Connection: keep-alive", and to convert
server's close during a keep-alive connection to a server-close
connection.
2010-01-22 11:49:41 +01:00
Willy Tarreau
116f91e7a5 [DOC] add documentation about connection header processing
The connection header is complex to handle, especially in the response
path, depending on request and response HTTP versions, desired mode,
etc... Let's document it. Note that only a subset of this document is
currently implemented.
2010-01-17 11:43:59 +01:00
Krzysztof Piotr Oledzki
346f76ddbd [MINOR] acl: add fe_id/so_id to match frontend's and socket's id 2010-01-12 23:00:51 +01:00
Willy Tarreau
b937b7e3b3 [DOC] add the doc about stickiness 2010-01-12 16:01:24 +01:00
Willy Tarreau
d72758ded1 [DOC] remove several trailing spaces 2010-01-12 10:42:19 +01:00
Willy Tarreau
983e01ed86 [DOC] fix various too large lines 2010-01-12 10:38:50 +01:00
Hervé COMMOWICK
698ae00fc2 [MINOR] add option "mysql-check" to use MySQL health checks
This patch adds support for MySQL health checks. Those are
enabled using the new option "mysql-check".
2010-01-12 10:37:39 +01:00
Willy Tarreau
b16a5746b7 [MINOR] http: add a separate "http-keep-alive" timeout
This one is used to wait for next request after a response was sent
to the client.
2010-01-10 14:46:16 +01:00
Willy Tarreau
81e3b4f48d [MINOR] http redirect: add the ability to append a '/' to the URL
Sometimes it can be desired to return a location which is the same
as the request with a slash appended when there was not one in the
request. A typical use of this is for sending a 301 so that people
don't reference links without the trailing slash. The name of the
new option is "append-slash" and it can be used on "redirect"
statements in prefix mode.
2010-01-10 00:42:19 +01:00
Willy Tarreau
11f8f5458f [RELEASE] Released version 1.4-dev6
Released version 1.4-dev6 with the following main changes :
    - [BUILD] warning in stream_interface.h
    - [BUILD] warning ultoa_r returns char *
    - [MINOR] hana: only report stats if it is enabled
    - [MINOR] stats: add "a link" & "a href" for sockets
    - [MINOR]: stats: add show-legends to report additional informations
    - [MEDIUM] default-server support
    - [BUG]: add 'observer', 'on-error', 'error-limit' to supported options list
    - [MINOR] stats: add href to tracked server
    - [BUG] stats: show UP/DOWN status also in tracking servers
    - [DOC] Restore ability to search a keyword at the beginning of a line
    - [BUG] stats: cookie should be reported under backend not under proxy
    - [BUG] cfgparser/stats: fix error message
    - [BUG] http: disable auto-closing during chunk analysis
    - [BUG] http: fix hopefully last closing issue on data forwarding
    - [DEBUG] add an http_silent_debug function to debug HTTP states
    - [MAJOR] http: fix again the forward analysers
    - [BUG] http_process_res_common() must not skip the forward analyser
    - [BUG] http: some possible missed close remain in the forward chain
    - [BUG] http: redirect needed to be updated after recent changes
    - [BUG] http: don't set no-linger on response in case of forced close
    - [MEDIUM] http: restore the original behaviour of option httpclose
    - [TESTS] add a file to test various connection modes
    - [BUG] http: check options before the connection header
    - [MAJOR] session: fix the order by which the analysers are run
    - [MEDIUM] session: also consider request analysers added during response
    - [MEDIUM] http: make safer use of the DONT_READ and AUTO_CLOSE flags
    - [BUG] http: memory leak with captures when using keep-alive
    - [BUG] http: fix for capture memory leak was incorrect
    - [MINOR] http redirect: use proper call to return last response
    - [MEDIUM] http: wait for some flush of the response buffer before a new request
    - [MEDIUM] session: limit the number of analyser loops
2010-01-08 07:49:44 +01:00
Krzysztof Piotr Oledzki
c53601c878 [DOC] Restore ability to search a keyword at the beginning of a line 2010-01-07 00:34:27 +01:00
Krzysztof Piotr Oledzki
c6df066980 [MEDIUM] default-server support
This patch implements default-server support allowing to change
default server options. It can be used in [defaults] or [backend]/[listen]
sections. Currently the following options are supported:

 - error-limit
 - fall
 - inter
 - fastinter
 - downinter
 - maxconn
 - maxqueue
 - minconn
 - on-error
 - port
 - rise
 - slowstart
 - weight
2010-01-06 00:28:06 +01:00
Krzysztof Piotr Oledzki
15514c21a2 [MINOR]: stats: add show-legends to report additional informations
Supported informations, available via "tr/td title":
  - cap: capabilities (proxy)
  - mode: one of tcp, http or health (proxy)
  - id: SNMP ID (proxy, socket, server)
  - IP (socket, server)
  - cookie (backend, server)
2010-01-06 00:28:06 +01:00
Willy Tarreau
0dfdf19b64 [MEDIUM] http: restore the original behaviour of option httpclose
Historically, "option httpclose" has always worked the same way. It
only mangles the "Connection" header in the request and the response
if needed, but does not affect the connection by itself, and ignores
any further data. It is dangerous to change this behaviour without
leaving any other alternative. If an active close is desired, it's
better to make use of "option forceclose" which does exactly what
it intends to do.

So as of now, "option httpclose" will only mangle the headers as
before, and will only affect the connection by itself when combined
with another connection-related option (eg: keepalive or server-close).
2010-01-05 11:33:11 +01:00
Willy Tarreau
1f44589b71 [RELEASE] Released version 1.4-dev5
Released version 1.4-dev5 with the following main changes :
    - [MINOR] server tracking: don't care about the tracked server's mode
    - [MEDIUM] appsession: add "len", "prefix" and "mode" options
    - [MEDIUM] appsession: add the "request-learn" option
    - [BUG] Configuration parser bug when escaping characters
    - [MINOR] CSS & HTML fun
    - [MINOR] Collect & provide http response codes received from servers
    - [BUG] Fix silly typo: hspr_other -> hrsp_other
    - [MINOR] Add "a name" to stats page
    - [MINOR] add additional "a href"s to stats page
    - [MINOR] Collect & provide http response codes for frontends, fix backends
    - [DOC] some small spell fixes and unifications
    - [MEDIUM] Decrease server health based on http responses / events, version 3
    - [BUG] format '%d' expects type 'int', but argument 5 has type 'long int'
    - [BUG] config: fix erroneous check on cookie domain names, again
    - [BUG] Healthchecks: get a proper error code if connection cannot be completed immediately
    - [DOC] trivial fix for man page
    - [MINOR] config: report all supported options for the "bind" keyword
    - [MINOR] tcp: add support for the defer_accept bind option
    - [MINOR] unix socket: report the socket path in case of bind error
    - [CONTRIB] halog: support searching by response time
    - [DOC] add a reminder about obsolete documents
    - [DOC] point to 1.4 doc, not 1.3
    - [DOC] option tcp-smart-connect was missing from index
    - [MINOR] http: detect connection: close earlier
    - [CLEANUP] sepoll: clean up the fd_clr/fd_set functions
    - [OPTIM] move some rarely used fields out of fdtab
    - [MEDIUM] fd: merge fd_list into fdtab
    - [MAJOR] buffer: flag BF_DONT_READ to disable reads when not required
    - [MINOR] http: add new transaction flags for keep-alive and content-length
    - [MEDIUM] http request: parse connection, content-length and transfer-encoding
    - [MINOR] http request: update the TX_SRV_CONN_KA flag on rewrite
    - [MINOR] http request: simplify the test of no-data
    - [MEDIUM] http request: simplify POST length detection
    - [MEDIUM] http request: make use of pre-parsed transfer-encoding header
    - [MAJOR] http: create the analyser which waits for a response
    - [MINOR] http: pre-set the persistent flags in the transaction
    - [MEDIUM] http response: check body length and set transaction flags
    - [MINOR] http response: update the TX_CLI_CONN_KA flag on rewrite
    - [MINOR] http: remove the last call to stream_int_return
    - [IMPORT] import ebtree v5.0 into directory ebtree/
    - [MEDIUM] build: switch ebtree users to use new ebtree version
    - [CLEANUP] ebtree: remove old unused files
    - [BUG] definitely fix regparm issues between haproxy core and ebtree
    - [CLEANUP] ebtree: cast to char * to get rid of gcc warning
    - [BUILD] missing #ifndef in ebmbtree.h
    - [BUILD] missing #ifndef in ebsttree.h
    - [MINOR] tools: add hex2i() function to convert hex char to int
    - [MINOR] http: create new MSG_BODY sub-states
    - [BUG] stream_sock: BUF_INFINITE_FORWARD broke splice on 64-bit platforms
    - [DOC] option is "defer-accept", not "defer_accept"
    - [MINOR] http: keep pointer to beginning of data
    - [BUG] x-original-to: name was not set in default instance
    - [MINOR] http: detect tunnel mode and set it in the session
    - [BUG] config: fix error message when config file is not found
    - [BUG] config: fix wrong handling of too large argument count
    - [BUG] config: disable 'option httplog' on TCP proxies
    - [BUG] config: fix erroneous check on cookie domain names
    - [BUG] config: cookie domain was ignored in defaults sections
    - [MINOR] config: support passing multiple "domain" statements to cookies
    - [MINOR] ebtree: add functions to lookup non-null terminated strings
    - [MINOR] config: don't report error on all subsequent files on failure
    - [BUG] second fix for the printf format warning
    - [BUG] check_post: limit analysis to the buffer length
    - [MEDIUM] http: process request body in a specific analyser
    - [MEDIUM] backend: remove HTTP POST parsing from get_server_ph_post()
    - [MAJOR] http: completely process the "connection" header
    - [MINOR] http: only consider chunk encoding with HTTP/1.1
    - [MAJOR] buffers: automatically compute the maximum buffer length
    - [MINOR] http: move the http transaction init/cleanup code to proto_http
    - [MINOR] http: move 1xx handling earlier to eliminate a lot of ifs
    - [MINOR] http: introduce a new synchronisation state : HTTP_MSG_DONE
    - [MEDIUM] http: rework chunk-size parser
    - [MEDIUM] http: add a new transaction flags indicating if we know the transfer length
    - [MINOR] buffers: add buffer_ignore() to skip some bytes
    - [BUG] http: offsets are relative to the buffer, not to ->som
    - [MEDIUM] http: automatically re-aling request buffer
    - [BUG] http: body parsing must consider the start of message
    - [MINOR] new function stream_int_cond_close()
    - [MAJOR] http: implement body parser
    - [BUG] http: typos on several unlikely() around header insertion
    - [BUG] stream_sock: wrong max computation on recv
    - [MEDIUM] http: rework the buffer alignment logic
    - [BUG] buffers: wrong size calculation for displaced data
    - [MINOR] stream_sock: prepare for closing when all pending data are sent
    - [MEDIUM] http: add two more states for the closing period
    - [MEDIUM] http: properly handle "option forceclose"
    - [MINOR] stream_sock: add SI_FL_NOLINGER for faster close
    - [MEDIUM] http: make forceclose use SI_FL_NOLINGER
    - [MEDIUM] session: set SI_FL_NOLINGER when aborting on write timeouts
    - [MEDIUM] http: add some SI_FL_NOLINGER around server errors
    - [MINOR] config: option forceclose is valid in frontends too
    - [BUILD] halog: insufficient include path in makefile
    - [MEDIUM] http: make the analyser not rely on msg being initialized anymore
    - [MEDIUM] http: make the parsers able to wait for a buffer flush
    - [MAJOR] http: add support for option http-server-close
    - [BUG] http: ensure we abort data transfer on write error
    - [BUG] last fix was overzealous and disabled server-close
    - [BUG] http: fix erroneous trailers size computation
    - [MINOR] stream_sock: enable MSG_MORE when forwarding finite amount of data
    - [OPTIM] http: set MSG_MORE on response when a pipelined request is pending
    - [BUG] http: redirects were broken by chunk changes
    - [BUG] http: the request URI pointer is relative to the buffer
    - [OPTIM] http: don't immediately enable reading on request
    - [MINOR] http: move redirect messages to HTTP/1.1 with a content-length
    - [BUG] http: take care of errors, timeouts and aborts during the data phase
    - [MINOR] http: don't wait for sending requests to the server
    - [MINOR] http: make the conditional redirect support keep-alive
    - [BUG] http: fix cookie parser to support spaces and commas in values
    - [MINOR] config: some options were missing for "redirect"
    - [MINOR] redirect: add support for unconditional rules
    - [MINOR] config: centralize proxy struct initialization
    - [MEDIUM] config: remove the limitation of 10 reqadd/rspadd statements
    - [MEDIUM] config: remove the limitation of 10 config files
    - [CLEANUP] http: remove a remaining impossible condition
    - [OPTIM] http: optimize a bit the construct of the forward loops
2010-01-03 23:23:36 +01:00
Willy Tarreau
f285f54311 [MINOR] redirect: add support for unconditional rules
Sometimes it's useful to be able to specify an unconditional redirect
rule without adding "if TRUE".
2010-01-03 21:22:08 +01:00
Willy Tarreau
b608feb82a [MAJOR] http: add support for option http-server-close
This option enables HTTP keep-alive on the client side and close mode
on the server side. This offers the best latency on the slow client
side, and still saves as many resources as possible on the server side
by actively closing connections. Pipelining is supported on both requests
and responses, though there is currently no reason to get pipelined
responses.
2010-01-02 22:47:18 +01:00
Willy Tarreau
a31e5dff36 [MINOR] config: option forceclose is valid in frontends too
This option was disabled for frontends in the configuration because
it was useless in its initial implementation, though it was still
checked in the code. Let's officially enable it now.
2009-12-30 01:10:35 +01:00
Willy Tarreau
82eeaf2fae [MEDIUM] http: properly handle "option forceclose"
The "forceclose" option used to close the output channel to the
server once it started to respond. While this happened to work with
most servers, some of them considered this as a connection abort and
immediately stopped responding.

Now that we're aware of the end of a request and response, we're able
to trivially handle this option and properly close both sides when the
server's response is complete.

During this change it appeared that forwarding could be allowed when
the BF_SHUTW_NOW flag was set on a buffer, which obviously is not
acceptable and was causing some trouble. This has been fixed too and
is the reason for the MEDIUM status on this patch.
2009-12-29 14:26:42 +01:00
Krzysztof Piotr Oledzki
97f07b832f [MEDIUM] Decrease server health based on http responses / events, version 3
Implement decreasing health based on observing communication between
HAProxy and servers.

Changes in this version 2:
 - documentation
 - close race between a started check and health analysis event
 - don't force fastinter if it is not set
 - better names for options
 - layer4 support

Changes in this version 3:
 - add stats
 - port to the current 1.4 tree
2009-12-16 00:29:27 +01:00
Krzysztof Piotr Oledzki
f864533c05 [DOC] some small spell fixes and unifications
Traditionnally -> Traditionally
preceeded -> preceded
statictics -> statistics
(...)
2009-12-16 00:20:05 +01:00
Willy Tarreau
68a897b2be [MINOR] config: support passing multiple "domain" statements to cookies
In some environments it is not possible to rely on any wildcard for a
domain name (eg: .com, .net, .fr...) so it is required to send multiple
domain extensions. (Un)fortunately the syntax check on the domain name
prevented that from being done the dirty way. So let's just build a
domain list when multiple domains are passed on the same line.
(cherry picked from commit 950245ca2b)
2009-12-06 13:25:23 +01:00
Cyril Bonté
b21570ae0f [MEDIUM] appsession: add "len", "prefix" and "mode" options
To sum up :
- len : it's now the max number of characters for the value, preventing
  garbaged results.
- a new option "prefix" is added, this allows to use dynamic cookie
  names (e.g. ASPSESSIONIDXXX).

Previously in the thread, I wanted to use the value found with
"capture cookie" but when i started to update the documentation, I
found this solution quite weird. I've made a small rework to not
depend on "capture cookie".

- There's the posssiblity to define the URL parser mode (path parameters
  or query string).
2009-11-30 11:31:53 +01:00
Willy Tarreau
53319c91ef [DOC] option is "defer-accept", not "defer_accept" 2009-11-28 08:21:29 +01:00
Willy Tarreau
3e79479348 [CLEANUP] ebtree: remove old unused files 2009-10-26 21:15:10 +01:00
Cyril Bont
bf47aeb946 [MEDIUM] appsession: add the "request-learn" option
This patch has 2 goals :

1. I wanted to test the appsession feature with a small PHP code,
using PHPSESSID. The problem is that when PHP gets an unknown session
id, it creates a new one with this ID. So, when sending an unknown
session to PHP, persistance is broken : haproxy won't see any new
cookie in the response and will never attach this session to a
specific server.

This also happens when you restart haproxy : the internal hash becomes
empty and all sessions loose their persistance (load balancing the
requests on all backend servers, creating a new session on each one).
For a user, it's like the service is unusable.

The patch modifies the code to make haproxy also learn the persistance
from the client : if no session is sent from the server, then the
session id found in the client part (using the URI or the client cookie)
is used to associated the server that gave the response.

As it's probably not a feature usable in all cases, I added an option
to enable it (by default it's disabled). The syntax of appsession becomes :

  appsession <cookie> len <length> timeout <holdtime> [request-learn]

This helps haproxy repair the persistance (with the risk of losing its
session at the next request, as the user will probably not be load
balanced to the same server the first time).

2. This patch also tries to reduce the memory usage.
Here is a little example to explain the current behaviour :
- Take a Tomcat server where /session.jsp is valid.
- Send a request using a cookie with an unknown value AND a path
  parameter with another unknown value :

  curl -b "JSESSIONID=12345678901234567890123456789012" http://<haproxy>/session.jsp;jsessionid=00000000000000000000000000000001

(I know, it's unexpected to have a request like that on a live service)
Here, haproxy finds the URI session ID and stores it in its internal
hash (with no server associated). But it also finds the cookie session
ID and stores it again.

- As a result, session.jsp sends a new session ID also stored in the
  internal hash, with a server associated.

=> For 1 request, haproxy has stored 3 entries, with only 1 which will be usable

The patch modifies the behaviour to store only 1 entry (maximum).
2009-10-18 11:56:26 +02:00
Willy Tarreau
39bb9bee03 [DOC] option tcp-smart-connect was missing from index 2009-10-17 16:04:09 +02:00
Willy Tarreau
be6008f322 [DOC] point to 1.4 doc, not 1.3
The links were wrong.
2009-10-14 22:22:03 +02:00
Krzysztof Piotr Oledzki
5fb1882514 [MINOR] Collect & provide http response codes received from servers
Additional data is provided on both html & csv stats:
 - html: when passing a mouse over Sessions -> Total (servers, backends)
 - cvs: by 6 additional fields (hrsp_1xx, hrsp_2xx, hrsp_3xx, hrsp_4xx, hrsp_5xx, hspr_other)

Patch inspired by:
 http://www.formilux.org/archives/haproxy/0910/2528.html
 http://www.formilux.org/archives/haproxy/0910/2529.html
2009-10-14 21:49:53 +02:00
Michael Shuler
35928e84e2 [DOC] trivial fix for man page
I'm working on helping Arnaud update haproxy in Debian, and one of the
package build warnings I received was about "hyphen where a minus sign
was intended" in the man page - details:
http://lintian.debian.org/tags/hyphen-used-as-minus-sign.html

Patch included in my 1.3.20 Debian package is attached.
2009-10-14 21:12:39 +02:00
Willy Tarreau
a080eca533 [DOC] add a reminder about obsolete documents
haproxy-en.txt and haproxy-fr.txt are outdated but people still refer to
them quite often, generally causing a useless waste of time.
2009-10-14 20:39:14 +02:00
Willy Tarreau
cb6cd43725 [MINOR] tcp: add support for the defer_accept bind option
This can ensure that data is readily available on a socket when
we accept it, but a bug in the kernel ignores the timeout so the
socket can remain pending as long as the client does not talk.
Use with care.
2009-10-13 07:34:14 +02:00
Willy Tarreau
c82a9e59a7 [RELEASE] Released version 1.4-dev4
Released version 1.4-dev4 with the following main changes :
    - [DOC] add missing rate_lim and rate_max
    - [MAJOR] struct chunk rework
    - [MEDIUM] Health check reporting code rework + health logging, v3
    - [BUG] check if rise/fall has an argument and it is > 0
    - [MINOR] health checks logging unification
    - [MINOR] add "description", "node" and show-node"/"show-desc", remove "node-name", v2
    - [MINOR] Allow dots in show-node & add "white-space: nowrap" in th.pxname.
    - [DOC] Add information about http://haproxy.1wt.eu/contrib.html
    - [MINOR] Introduce include/types/counters.h
    - [CLEANUP] Move counters to dedicated structures
    - [MINOR] Add "clear counters" to clear statistics counters
    - [MEDIUM] Collect & provide separate statistics for sockets, v2
    - [BUG] Fix NULL pointer dereference in stats_check_uri_auth(), v2
    - [MINOR] acl: don't report valid acls as potential mistakes
    - [MINOR] Add cut_crlf(), ltrim(), rtrim() and alltrim()
    - [MINOR] Add chunk_htmlencode and chunk_asciiencode
    - [MINOR] Capture & display more data from health checks, v2
    - [BUG] task.c: don't assing last_timer to node-less entries
    - [BUG] http stats: large outputs sometimes got some parts chopped off
    - [MINOR] backend: export some functions to recount servers
    - [MINOR] backend: uninline some LB functions
    - [MINOR] include time.h from freq_ctr.h as is uses "now".
    - [CLEANUP] backend: move LB algos to individual files
    - [MINOR] lb_map: reorder code in order to ease integration of new hash functions
    - [CLEANUP] proxy: move last lb-specific bits to their respective files
    - [MINOR] backend: separate declarations of LB algos from their lookup method
    - [MINOR] backend: reorganize the LB algorithm selection
    - [MEDIUM] backend: introduce the "static-rr" LB algorithm
    - [MINOR] report list of supported pollers with -vv
    - [DOC] log-health-checks is an option, not a directive
    - [MEDIUM] new option "independant-streams" to stop updating read timeout on writes
    - [BUG] stats: don't call buffer_shutw(), but ->shutw() instead
    - [MINOR] stats: strip CR and LF from the input command line
    - [BUG] don't refresh timeouts late after detected activity
    - [MINOR] stats_dump_errors_to_buffer: use buffer_feed_chunk()
    - [MINOR] stats_dump_sess_to_buffer: use buffer_feed_chunk()
    - [MINOR] stats: make stats_dump_raw_to_buffer() use buffer_feed_chunk
    - [MEDIUM] stats: don't use s->ana_state anymore
    - [MINOR] remove now obsolete ana_state from the session struct
    - [MEDIUM] stats: make HTTP stats use an I/O handler
    - [MEDIUM] stream_int: adjust WAIT_ROOM handling
    - [BUG] config: look for ID conflicts in all sockets, not only last ones.
    - [MINOR] config: reference file and line with any listener/proxy/server declaration
    - [MINOR] config: report places of duplicate names or IDs
    - [MINOR] config: add pointer to file name in block/redirect/use_backend/monitor rules
    - [MINOR] tools: add a new get_next_id() function
    - [MEDIUM] config: automatically find unused IDs for proxies, servers and listeners
    - [OPTIM] counters: move some max numbers to the counters struct
    - [BUG] counters: fix segfault on missing counters for a listener
    - [MEDIUM] backend: implement consistent hashing variation
    - [MINOR] acl: add fe_conn, be_conn, queue, avg_queue
    - [MINOR] stats: use 'clear counters all' to clear all values
    - [MEDIUM] add access restrictions to the stats socket
    - [MINOR] buffers: add buffer_feed2() and make buffer_feed() measure string length
    - [MINOR] proxy: provide function to retrieve backend/server pointers
    - [MINOR] add the "initial weight" to the server struct.
    - [MEDIUM] stats: add the "get weight" command to report a server's weight
    - [MEDIUM] stats: add the "set weight" command
    - [BUILD] add a 'make tags' target
    - [MINOR] stats: add support for numeric IDs in set weight/get weight
    - [MINOR] stats: use a dedicated state to output static data
    - [OPTIM] stats: check free space before trying to print
2009-10-12 06:40:53 +02:00
Willy Tarreau
cfeaa476c6 [MINOR] stats: add support for numeric IDs in set weight/get weight
Krzysztof reported that using names only for get weight/set weight
was not enough because it's still possible to have multiple servers
with the same name (and my test config is one of those). He suggested
to be able to designate them by their unique numeric IDs by prefixing
the ID with a dash.

That way we can have :

     set weight #120/#2

as well as

     get weight static/srv1 10
2009-10-10 22:33:08 +02:00
Willy Tarreau
4483d43c66 [MEDIUM] stats: add the "set weight" command
It is now possible to change a server's weight from the stats socket.
Just use "set weight <back>/<serv> <weight>".
2009-10-10 20:18:44 +02:00
Willy Tarreau
38338fa0a0 [MEDIUM] stats: add the "get weight" command to report a server's weight
This command is unrestricted and simply reports a server's weight, as
well as the weight initially configured, which at the moment are the
sames.
2009-10-10 19:39:21 +02:00
Willy Tarreau
6162db2a81 [MEDIUM] add access restrictions to the stats socket
The stats socket can now run at 3 different levels :
  - user
  - operator (default one)
  - admin

These levels are used to restrict access to some information
and commands. Only the admin can clear all stats. A user cannot
clear anything nor access sensible data such as sessions or
errors.
2009-10-10 17:13:00 +02:00
Willy Tarreau
2f6bf2b82c [MINOR] stats: use 'clear counters all' to clear all values
The most common use of "clear counters" should be to only clear
max values without affecting cumulated values, for instance,
after an incident. So we change "clear counters" to only clear
max values, and add "clear counters all" to clear all counters.
2009-10-10 15:32:48 +02:00
Willy Tarreau
a36af91951 [MINOR] acl: add fe_conn, be_conn, queue, avg_queue
These ACLs are used to check the number of active connections on the
frontend, backend or in a backend's queue. The avg_queue returns the
average number of queued connections per server, and for this, divides
the total number of queued connections by the number of alive servers.

The dst_conn ACL has been slightly changed to more reflect its name and
original usage, which is to return the number of connections on the
destination address/port (the socket) and not the whole frontend.
2009-10-10 12:02:45 +02:00
Willy Tarreau
6b2e11be1e [MEDIUM] backend: implement consistent hashing variation
Consistent hashing provides some interesting advantages over common
hashing. It avoids full redistribution in case of a server failure,
or when expanding the farm. This has a cost however, the hashing is
far from being perfect, as we associate a server to a request by
searching the server with the closest key in a tree. Since servers
appear multiple times based on their weights, it is recommended to
use weights larger than approximately 10-20 in order to smoothen
the distribution a bit.

In some cases, playing with weights will be the only solution to
make a server appear more often and increase chances of being picked,
so stats are very important with consistent hashing.

In order to indicate the type of hashing, use :

   hash-type map-based      (default, old one)
   hash-type consistent     (new one)

Consistent hashing can make sense in a cache farm, in order not
to redistribute everyone when a cache changes state. It could also
probably be used for long sessions such as terminal sessions, though
that has not be attempted yet.

More details on this method of hashing here :
  http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/
2009-10-09 07:17:58 +02:00
Willy Tarreau
53fb4ae261 [MEDIUM] config: automatically find unused IDs for proxies, servers and listeners
Until now it was required that every custom ID was above 1000 in order to
avoid conflicts. Now we have the list of all assigned IDs and can automatically
pick the first unused one. This means that it is perfectly possible to interleave
automatic IDs with persistent IDs and the parser will automatically allocate
unused values starting with 1.
2009-10-04 23:04:08 +02:00
Krzysztof Piotr Oledzki
aeebf9ba65 [MEDIUM] Collect & provide separate statistics for sockets, v2
This patch allows to collect & provide separate statistics for each socket.
It can be very useful if you would like to distinguish between traffic
generate by local and remote users or between different types of remote
clients (peerings, domestic, foreign).

Currently no "Session rate" is supported, but adding it should be possible
if we found it useful.
2009-10-04 18:56:02 +02:00
Krzysztof Piotr Oledzki
719e726c97 [MINOR] Add "clear counters" to clear statistics counters
Now, when statistics counters are moved to separate
structures, adding "clear counters" is extremely easy.

[wt: updated the help message]
2009-10-04 18:40:08 +02:00
Willy Tarreau
f27b5ea8dc [MEDIUM] new option "independant-streams" to stop updating read timeout on writes
By default, when data is sent over a socket, both the write timeout and the
read timeout for that socket are refreshed, because we consider that there is
activity on that socket, and we have no other means of guessing if we should
receive data or not.

While this default behaviour is desirable for almost all applications, there
exists a situation where it is desirable to disable it, and only refresh the
read timeout if there are incoming data. This happens on sessions with large
timeouts and low amounts of exchanged data such as telnet session. If the
server suddenly disappears, the output data accumulates in the system's
socket buffers, both timeouts are correctly refreshed, and there is no way
to know the server does not receive them, so we don't timeout. However, when
the underlying protocol always echoes sent data, it would be enough by itself
to detect the issue using the read timeout. Note that this problem does not
happen with more verbose protocols because data won't accumulate long in the
socket buffers.

When this option is set on the frontend, it will disable read timeout updates
on data sent to the client. There probably is little use of this case. When
the option is set on the backend, it will disable read timeout updates on
data sent to the server. Doing so will typically break large HTTP posts from
slow lines, so use it with caution.
2009-10-03 22:01:18 +02:00
Willy Tarreau
211ad247ab [DOC] log-health-checks is an option, not a directive
Also fix alphabetical placement of the option in the list.
2009-10-03 21:47:47 +02:00
Willy Tarreau
9757a38feb [MEDIUM] backend: introduce the "static-rr" LB algorithm
The "static-rr" is just the old round-robin algorithm. It is still
in use when a hash algorithm is used and the data to hash is not
present, but it was impossible to configure it explicitly. This one
is cheaper in terms of CPU and supports unlimited numbers of servers,
so it makes sense to be able to use it.
2009-10-03 18:41:19 +02:00