Commit Graph

770 Commits

Author SHA1 Message Date
Willy Tarreau
bb2e669f9e BUG/MAJOR: http: correctly rewind the request body after start of forwarding
Daniel Dubovik reported an interesting bug showing that the request body
processing was still not 100% fixed. If a POST request contained short
enough data to be forwarded at once before trying to establish the
connection to the server, we had no way to correctly rewind the body.

The first visible case is that balancing on a header does not always work
on such POST requests since the header cannot be found. But there are even
nastier implications which are that http-send-name-header would apply to
the wrong location and possibly even affect part of the request's body
due to an incorrect rewinding.

There are two options to fix the problem :
  - first one is to force the HTTP_MSG_F_WAIT_CONN flag on all hash-based
    balancing algorithms and http-send-name-header, but there's always a
    risk that any new algorithm forgets to set it ;

  - the second option is to account for the amount of skipped data before
    the connection establishes so that we always know the position of the
    request's body relative to the buffer's origin.

The second option is much more reliable and fits very well in the spirit
of the past changes to fix forwarding. Indeed, at the moment we have
msg->sov which points to the start of the body before headers are forwarded
and which equals zero afterwards (so it still points to the start of the
body before forwarding data). A minor change consists in always making it
point to the start of the body even after data have been forwarded. It means
that it can get a negative value (so we need to change its type to signed)..

In order to avoid wrapping, we only do this as long as the other side of
the buffer is not connected yet.

Doing this definitely fixes the issues above for the requests. Since the
response cannot be rewound we don't need to perform any change there.

This bug was introduced/remained unfixed in 1.5-dev23 so the fix must be
backported to 1.5.
2014-07-10 19:29:45 +02:00
Willy Tarreau
0dbfdbaef1 MINOR: samples: add two converters for the date format
This patch adds two converters :

   ltime(<format>[,<offset>])
   utime(<format>[,<offset>])

Both use strftime() to emit the output string from an input date. ltime()
provides local time, while utime() provides the UTC time.
2014-07-10 16:43:44 +02:00
Willy Tarreau
d9f316ab83 MEDIUM: stick-table: add new converters to fetch table data
These new converters make it possible to look up any sample expression
in a table, and check whether an equivalent key exists or not, and if it
exists, to retrieve the associated data (eg: gpc0, request rate, etc...).

Till now it was only possible using tracking, but sometimes tracking is
not suited to only retrieving such counters, either because it's done too
early or because too many items need to be checked without necessarily
being tracked.

These converters all take a string on input, and then convert it again to
the table's type. This means that if an input sample is of type IPv4 and
the table is of type IP, it will first be converted to a string, then back
to an IP address. This is a limitation of the current design which does not
allow converters to declare that "any" type is supported on input. Since
strings are the only types which can be cast to any other one, this method
always works.

The following converters were added :

  in_table, table_bytes_in_rate, table_bytes_out_rate, table_conn_cnt,
  table_conn_cur, table_conn_rate, table_gpc0, table_gpc0_rate,
  table_http_err_cnt, table_http_err_rate, table_http_req_cnt,
  table_http_req_rate, table_kbytes_in, table_kbytes_out,
  table_server_id, table_sess_cnt, table_sess_rate, table_trackers.
2014-07-10 16:43:44 +02:00
Willy Tarreau
ffcb2e4b42 DOC: fix alphabetical sort of converters
For an unknown reason, these ones were not sorted.
2014-07-10 16:43:44 +02:00
Willy Tarreau
a01b974d5f DOC: minor fix on {sc,src}_kbytes_{in,out}
These ones report total amount of bytes, not byte rates.
This fix should be backported into 1.5 which has the same error.
2014-07-10 16:43:44 +02:00
James Westby
ebe62d645b DOC: expand the docs for the provided stats.
Indicate for each statistic which types may have a value for
that statistic.

Explain some of the provided statistics a little more deeply.
2014-07-08 20:35:05 +02:00
Willy Tarreau
70f72e0c90 DOC: explicitly mention the limits of abstract namespace sockets
Listening to an abstract namespace socket is quite convenient but
comes with some drawbacks that must be clearly understood when the
socket is being listened to by multiple processes. The trouble is
that the socket cannot be rebound if a new process attempts a soft
restart and fails, so only one of the initially bound processes
will still be bound to it, the other ones will fail to rebind. For
most situations it's not an issue but it needs to be indicated.
2014-07-08 01:13:35 +02:00
Willy Tarreau
2d0caa38e0 DOC: provide an example of how to use ssl_c_sha1
As suggested by Aydan Yumerefendi, a little bit of examples never hurts.
2014-07-02 19:02:10 +02:00
Willy Tarreau
18324f574f MEDIUM: log: support a user-configurable max log line length
With all the goodies supported by logformat, people find that the limit
of 1024 chars for log lines is too short. Some servers do not support
larger lines and can simply drop them, so changing the default value is
not always the best choice.

This patch takes a different approach. Log line length is specified per
log server on the "log" line, with a value between 80 and 65535. That
way it's possibly to satisfy all needs, even with some fat local servers
and small remote ones.
2014-06-27 18:13:53 +02:00
Simon Horman
98637e5bff MEDIUM: Add external check
Add an external check which makes use of an external process to
check the status of a server.
2014-06-20 07:10:07 +02:00
Willy Tarreau
15480d7250 [DEV] open new 1.6 development branch
This new branch is based on 1.5.0, which 1.6-dev0 is 100% equivalent to.
The README has been updated to mention that it is a development branch.

Released version 1.6-dev0 with the following main changes :
    - exact copy of 1.5.0
2014-06-19 21:11:06 +02:00
Willy Tarreau
9229f1248f [RELEASE] Released version 1.5.0
Released version 1.5.0 with the following main changes :
    - MEDIUM: ssl: ignored file names ending as '.issuer' or '.ocsp'.
    - MEDIUM: ssl: basic OCSP stapling support.
    - MINOR: ssl/cli: Fix unapropriate comment in code on 'set ssl ocsp-response'
    - MEDIUM: ssl: add 300s supported time skew on OCSP response update.
    - MINOR: checks: mysql-check: Add support for v4.1+ authentication
    - MEDIUM: ssl: Add the option to use standardized DH parameters >= 1024 bits
    - MEDIUM: ssl: fix detection of ephemeral diffie-hellman key exchange by using the cipher description.
    - MEDIUM: http: add actions "replace-header" and "replace-values" in http-req/resp
    - MEDIUM: Break out check establishment into connect_chk()
    - MEDIUM: Add port_to_str helper
    - BUG/MEDIUM: fix ignored values for half-closed timeouts (client-fin and server-fin) in defaults section.
    - BUG/MEDIUM: Fix unhandled connections problem with systemd daemon mode and SO_REUSEPORT.
    - MINOR: regex: fix a little configuration memory leak.
    - MINOR: regex: Create JIT compatible function that return match strings
    - MEDIUM: regex: replace all standard regex function by own functions
    - MEDIUM: regex: Remove null terminated strings.
    - MINOR: regex: Use native PCRE API.
    - MINOR: missing regex.h include
    - DOC: Add Exim as Proxy Protocol implementer.
    - BUILD: don't use type "uint" which is not portable
    - BUILD: stats: workaround stupid and bogus -Werror=format-security behaviour
    - BUG/MEDIUM: http: clear CF_READ_NOEXP when preparing a new transaction
    - CLEANUP: http: don't clear CF_READ_NOEXP twice
    - DOC: fix proxy protocol v2 decoder example
    - DOC: fix remaining occurrences of "pattern extraction"
    - MINOR: log: allow the HTTP status code to be logged even in TCP frontends
    - MINOR: logs: don't limit HTTP header captures to HTTP frontends
    - MINOR: sample: improve sample_fetch_string() to report partial contents
    - MINOR: capture: extend the captures to support non-header keys
    - MINOR: tcp: prepare support for the "capture" action
    - MEDIUM: tcp: add a new tcp-request capture directive
    - MEDIUM: session: allow shorter retry delay if timeout connect is small
    - MEDIUM: session: don't apply the retry delay when redispatching
    - MEDIUM: session: redispatch earlier when possible
    - MINOR: config: warn when tcp-check rules are used without option tcp-check
    - BUG/MINOR: connection: make proxy protocol v1 support the UNKNOWN protocol
    - DOC: proxy protocol example parser was still wrong
    - DOC: minor updates to the proxy protocol doc
    - CLEANUP: connection: merge proxy proto v2 header and address block
    - MEDIUM: connection: add support for proxy protocol v2 in accept-proxy
    - MINOR: tools: add new functions to quote-encode strings
    - DOC: clarify the CSV format
    - MEDIUM: stats: report the last check and last agent's output on the CSV status
    - MINOR: freq_ctr: introduce a new averaging method
    - MEDIUM: session: maintain per-backend and per-server time statistics
    - MEDIUM: stats: report per-backend and per-server time stats in HTML and CSV outputs
    - BUG/MINOR: http: fix typos in previous patch
    - DOC: remove the ultra-obsolete TODO file
    - DOC: update roadmap
    - DOC: minor updates to the README
    - DOC: mention the maxconn limitations with the select poller
    - DOC: commit a few old design thoughts files
2014-06-19 21:02:32 +02:00
Willy Tarreau
c14b7d94a0 DOC: commit a few old design thoughts files
These ones were design notes and ideas collected during the 1.5
development phase lying on my development machine. There might still
be some value in keeping them for future reference since they mention
certain corner cases.
2014-06-19 21:02:32 +02:00
Willy Tarreau
8274e105b7 DOC: mention the maxconn limitations with the select poller
Select()'s safe area is limited to 1024 FDs, and anything higher
than this will report "select: FAILED" on startup in debug mode,
so better document it.
2014-06-19 21:02:32 +02:00
Emeric Brun
4147b2ef10 MEDIUM: ssl: basic OCSP stapling support.
The support is all based on static responses. This doesn't add any
request / response logic to HAProxy, but allows a way to update
information through the socket interface.

Currently certificates specified using "crt" or "crt-list" on "bind" lines
are loaded as PEM files.
For each PEM file, haproxy checks for the presence of file at the same path
suffixed by ".ocsp". If such file is found, support for the TLS Certificate
Status Request extension (also known as "OCSP stapling") is automatically
enabled. The content of this file is optional. If not empty, it must contain
a valid OCSP Response in DER format. In order to be valid an OCSP Response
must comply with the following rules: it has to indicate a good status,
it has to be a single response for the certificate of the PEM file, and it
has to be valid at the moment of addition. If these rules are not respected
the OCSP Response is ignored and a warning is emitted. In order to  identify
which certificate an OCSP Response applies to, the issuer's certificate is
necessary. If the issuer's certificate is not found in the PEM file, it will
be loaded from a file at the same path as the PEM file suffixed by ".issuer"
if it exists otherwise it will fail with an error.

It is possible to update an OCSP Response from the unix socket using:

  set ssl ocsp-response <response>

This command is used to update an OCSP Response for a certificate (see "crt"
on "bind" lines). Same controls are performed as during the initial loading of
the response. The <response> must be passed as a base64 encoded string of the
DER encoded response from the OCSP server.

Example:
  openssl ocsp -issuer issuer.pem -cert server.pem \
               -host ocsp.issuer.com:80 -respout resp.der
  echo "set ssl ocsp-response $(base64 -w 10000 resp.der)" | \
               socat stdio /var/run/haproxy.stat

This feature is automatically enabled on openssl 0.9.8h and above.

This work was performed jointly by Dirkjan Bussink of GitHub and
Emeric Brun of HAProxy Technologies.
2014-06-18 18:28:56 +02:00
Sasha Pachev
218f064f55 MEDIUM: http: add actions "replace-header" and "replace-values" in http-req/resp
This patch adds two new actions to http-request and http-response rulesets :
  - replace-header : replace a whole header line, suited for headers
                     which might contain commas
  - replace-value  : replace a single header value, suited for headers
                     defined as lists.

The match consists in a regex, and the replacement string takes a log-format
and supports back-references.
2014-06-17 18:34:32 +02:00
Willy Tarreau
f5b1cc38b8 MEDIUM: stats: report per-backend and per-server time stats in HTML and CSV outputs
The time statistics computed by previous patches are now reported in the
HTML stats in the tips related to the total sessions for backend and servers,
and as separate columns for the CSV stats.
2014-06-17 17:15:56 +02:00
Willy Tarreau
a28df3e19a MEDIUM: stats: report the last check and last agent's output on the CSV status
Now that we can quote unsafe string, it becomes possible to dump the health
check responses on the CSV page as well. The two new fields are "last_chk"
and "last_agt".
2014-06-16 18:20:26 +02:00
Willy Tarreau
a3310dc66c DOC: clarify the CSV format
Indicate that the text cells in the CSV format may contain quotes to
escape ambiguous texts. We don't have this case right now since we limit
the output, but it may happen in the future.
2014-06-16 18:20:14 +02:00
Willy Tarreau
7799267f43 MEDIUM: connection: add support for proxy protocol v2 in accept-proxy
The "accept-proxy" statement of bind lines was still limited to version
1 of the protocol, while send-proxy-v2 is now available on the server
lines. This patch adds support for parsing v2 of the protocol on incoming
connections. The v2 header is automatically recognized so there is no
need for a new option.
2014-06-14 11:46:03 +02:00
Willy Tarreau
7a6f134121 DOC: minor updates to the proxy protocol doc
Update the release data, revision history and the link to the Forwarded
HTTP extension.
2014-06-14 11:46:02 +02:00
Willy Tarreau
01320c9a34 DOC: proxy protocol example parser was still wrong
Now that version and cmd are in the same byte, it is not possible
anymore to compare the version as a 13th byte.
2014-06-14 11:46:02 +02:00
Willy Tarreau
18bf01e900 MEDIUM: tcp: add a new tcp-request capture directive
This new directive captures the specified fetch expression, converts
it to text and puts it into the next capture slot. The capture slots
are shared with header captures so that it is possible to dump all
captures at once or selectively in logs and header processing.

The purpose is to permit logs to contain whatever payload is found in
a request, for example bytes at a fixed location or the SNI of forwarded
SSL traffic.
2014-06-13 16:45:53 +02:00
Willy Tarreau
d9ed3d2848 MINOR: logs: don't limit HTTP header captures to HTTP frontends
Similar to previous patches, HTTP header captures are performed when
a TCP frontend switches to an HTTP backend, but are not possible to
report. So let's relax the check to explicitly allow them to be present
in TCP frontends.
2014-06-13 16:32:48 +02:00
Willy Tarreau
4bf9963a78 MINOR: log: allow the HTTP status code to be logged even in TCP frontends
Log format is defined in the frontend, and some frontends may be chained to
an HTTP backend. Sometimes it's very convenient to be able to log the HTTP
status code of these HTTP backends. This status is definitely present in
the internal structures, it's just that we used to limit it to be used in
HTTP frontends. So let's simply relax the check to allow it to be used in
TCP frontends as well.
2014-06-13 16:32:48 +02:00
Willy Tarreau
be722a2d64 DOC: fix remaining occurrences of "pattern extraction" 2014-06-13 16:32:48 +02:00
Remi Gacogne
f46cd6e4ec MEDIUM: ssl: Add the option to use standardized DH parameters >= 1024 bits
When no static DH parameters are specified, this patch makes haproxy
use standardized (rfc 2409 / rfc 3526) DH parameters with prime lenghts
of 1024, 2048, 4096 or 8192 bits for DHE key exchange. The size of the
temporary/ephemeral DH key is computed as the minimum of the RSA/DSA server
key size and the value of a new option named tune.ssl.default-dh-param.
2014-06-12 16:12:23 +02:00
Willy Tarreau
0f6093a9b2 DOC: fix proxy protocol v2 decoder example
Richard Russo reported that the example code in the PP spec is wrong
now that we slightly changed the format to merge <ver> and <cmd>. Also
rename the field <ver_cmd> to avoid any ambiguity on the usage.
2014-06-11 21:21:26 +02:00
Nenad Merdanovic
6639a7cf0d MINOR: checks: mysql-check: Add support for v4.1+ authentication
MySQL will in stop supporting pre-4.1 authentication packets in the future
and is already giving us a hard time regarding non-silencable warnings
which are logged on each health check. Warnings look like the following:

"[Warning] Client failed to provide its character set. 'latin1' will be used
as client character set."

This patch adds basic support for post-4.1 authentication by sending the proper
authentication packet with the character set, along with the QUIT command.
2014-06-11 18:13:46 +02:00
Todd Lyons
d1dcea064c DOC: Add Exim as Proxy Protocol implementer. 2014-06-03 22:36:46 +02:00
Willy Tarreau
2e85840266 [RELEASE] Released version 1.5-dev26
Released version 1.5-dev26 with the following main changes :
    - BUG/MEDIUM: polling: fix possible CPU hogging of worker processes after receiving SIGUSR1.
    - BUG/MINOR: stats: fix a typo on a closing tag for a server tracking another one
    - OPTIM: stats: avoid the calculation of a useless link on tracking servers in maintenance
    - MINOR: fix a few memory usage errors
    - CONTRIB: halog: Filter input lines by date and time through timestamp
    - MINOR: ssl: SSL_CTX_set_options() and SSL_CTX_set_mode() take a long, not an int
    - BUG/MEDIUM: regex: fix risk of buffer overrun in exp_replace()
    - MINOR: acl: set "str" as default match for strings
    - DOC: Add some precisions about acl default matching method
    - MEDIUM: acl: strenghten the option parser to report invalid options
    - BUG/MEDIUM: config: a stats-less config crashes in 1.5-dev25
    - BUG/MINOR: checks: tcp-check must not stop on '\0' for binary checks
    - MINOR: stats: improve alignment of color codes to save one line of header
    - MINOR: checks: simplify and improve reporting of state changes when using log-health-checks
    - MINOR: server: remove the SRV_DRAIN flag which can always be deduced
    - MINOR: server: use functions to detect state changes and to update them
    - MINOR: server: create srv_was_usable() from srv_is_usable() and use a pointer
    - BUG/MINOR: stats: do not report "100%" in the thottle column when server is draining
    - BUG/MAJOR: config: don't free valid regex memory
    - BUG/MEDIUM: session: don't clear CF_READ_NOEXP if analysers are not called
    - BUG/MINOR: stats: tracking servers may incorrectly report an inherited DRAIN status
    - MEDIUM: proxy: make timeout parser a bit stricter
    - REORG/MEDIUM: server: split server state and flags in two different variables
    - REORG/MEDIUM: server: move the maintenance bits out of the server state
    - MAJOR: server: use states instead of flags to store the server state
    - REORG: checks: put the functions in the appropriate files !
    - MEDIUM: server: properly support and propagate the maintenance status
    - MEDIUM: server: allow multi-level server tracking
    - CLEANUP: checks: rename the server_status_printf function
    - MEDIUM: checks: simplify server up/down/nolb transitions
    - MAJOR: checks: move health checks changes to set_server_check_status()
    - MINOR: server: make the status reporting function support a reason
    - MINOR: checks: simplify health check reporting functions
    - MINOR: server: implement srv_set_stopped()
    - MINOR: server: implement srv_set_running()
    - MINOR: server: implement srv_set_stopping()
    - MEDIUM: checks: simplify failure notification using srv_set_stopped()
    - MEDIUM: checks: simplify success notification using srv_set_running()
    - MEDIUM: checks: simplify stopping mode notification using srv_set_stopping()
    - MEDIUM: stats: report a server's own state instead of the tracked one's
    - MINOR: server: make use of srv_is_usable() instead of checking eweight
    - MAJOR: checks: add support for a new "drain" administrative mode
    - MINOR: stats: use the admin flags for soft enable/disable/stop/start on the web page
    - MEDIUM: stats: introduce new actions to simplify admin status management
    - MINOR: cli: introduce a new "set server" command
    - MINOR: stats: report a distinct output for DOWN caused by agent
    - MINOR: checks: support specific check reporting for the agent
    - MINOR: checks: support a neutral check result
    - BUG/MINOR: cli: "agent" was missing from the "enable"/"disable" help message
    - MEDIUM: cli: add support for enabling/disabling health checks.
    - MEDIUM: stats: report down caused by agent prior to reporting up
    - MAJOR: agent: rework the response processing and support additional actions
    - MINOR: stats: improve the stats web page to support more actions
    - CONTRIB: halog: avoid calling time/localtime/mktime for each line
    - DOC: document the workarouds for Google Chrome's bogus pre-connect
    - MINOR: stats: report SSL key computations per second
    - MINOR: stats: add counters for SSL cache lookups and misses
2014-05-28 17:50:53 +02:00
Willy Tarreau
2705a61d8c DOC: document the workarouds for Google Chrome's bogus pre-connect
More and more people are complaining about the bugs experienced by
Chrome users due to the pre-connect feature and the fact that Chrome
does not monitor its connections and happily displays the error page
instead of re-opening a new connection. Since we can work around this
bug, let's document how to do it.
2014-05-23 17:38:34 +02:00
Willy Tarreau
81f5d94a0b MAJOR: agent: rework the response processing and support additional actions
We now retrieve a lot of information from a single line of response, which
can be made up of various words delimited by spaces/tabs/commas. We try to
arrange all this and report whatever unusual we detect. The agent now supports :
  - "up", "down", "stopped", "fail" for the operational states
  - "ready", "drain", "maint" for the administrative states
  - any "%" number for the weight
  - an optional reason after a "#" that can be reported on the stats page

The line parser and processor should move to its own function so that
we can reuse the exact same one for http-based agent checks later.
2014-05-23 15:42:49 +02:00
Willy Tarreau
9b5aecd5be MEDIUM: cli: add support for enabling/disabling health checks.
"enable health" and "disable health" are introduced to manipulate the
health check subsystem.
2014-05-23 15:42:49 +02:00
Willy Tarreau
2a4b70fffd MINOR: cli: introduce a new "set server" command
This command supports "agent", "health", "state" and "weight" to adjust
various server attributes as well as changing server health check statuses
on the fly or setting the drain mode.
2014-05-23 15:42:42 +02:00
Willy Tarreau
3209123fe7 MEDIUM: server: allow multi-level server tracking
Now that it is possible to know whether a server is in forced maintenance
or inherits its maintenance status from another one, it is possible to
allow server tracking at more than one level. We still provide a loop
detection however.

Note that for the stats it's a bit trickier since we have to report the
check state which corresponds to the state of the server at the end of
the chain.
2014-05-23 14:29:11 +02:00
Willy Tarreau
bef1b32c4e MINOR: checks: simplify and improve reporting of state changes when using log-health-checks
Function set_server_check_status() is very weird. It is called at the
end of a check to update the server's state before the new state is even
calculated, and possibly to log status changes, only if the proxy has
"option log-health-checks" set.

In order to do so, it employs an exhaustive list of the combinations
which can lead to a state change, while in practice almost all of
them may simply be deduced from the change of check status. Better,
some changes of check status are currently not detected while they
can be very valuable (eg: changes between L4/L6/TOUT/HTTP 500 for
example).

The doc was updated to reflect this.

Also, a minor change was made to consider s->uweight and not s->eweight
as meaning "DRAIN" since eweight can be null without the DRAIN mode (eg:
throttle, NOLB, ...).
2014-05-13 22:01:28 +02:00
Thierry FOURNIER
2a06e39a4c DOC: Add some precisions about acl default matching method 2014-05-12 16:06:41 +02:00
Willy Tarreau
a3393955da [RELEASE] Released version 1.5-dev25
Released version 1.5-dev25 with the following main changes :
    - MEDIUM: connection: Implement and extented PROXY Protocol V2
    - MINOR: ssl: clean unused ACLs declarations
    - MINOR: ssl: adds fetchs and ACLs for ssl back connection.
    - MINOR: ssl: merge client's and frontend's certificate functions.
    - MINOR: ssl: adds ssl_f_sha1 fetch to return frontend's certificate fingerprint
    - MINOR: ssl: adds sample converter base64 for binary type.
    - MINOR: ssl: convert to binary ssl_fc_unique_id and ssl_bc_unique_id.
    - BUG/MAJOR: ssl: Fallback to private session cache if current lock mode is not supported.
    - MAJOR: ssl: Change default locks on ssl session cache.
    - BUG/MINOR: chunk: Fix function chunk_strcmp and chunk_strcasecmp match a substring.
    - MINOR: ssl: add global statement tune.ssl.force-private-cache.
    - MINOR: ssl: remove fallback to SSL session private cache if lock init fails.
    - BUG/MEDIUM: patterns: last fix was still not enough
    - MINOR: http: export the smp_fetch_cookie function
    - MINOR: http: generic pointer to rule argument
    - BUG/MEDIUM: pattern: a typo breaks automatic acl/map numbering
    - BUG/MAJOR: patterns: -i and -n are ignored for inlined patterns
    - BUG/MINOR: proxy: unsafe initialization of HTTP transaction when switching from TCP frontend
    - BUG/MINOR: http: log 407 in case of proxy auth
    - MINOR: http: rely on the message body parser to send 100-continue
    - MEDIUM: http: move reqadd after execution of http_request redirect
    - MEDIUM: http: jump to dedicated labels after http-request processing
    - BUG/MINOR: http: block rules forgot to increment the denied_req counter
    - BUG/MINOR: http: block rules forgot to increment the session's request counter
    - MEDIUM: http: move Connection header processing earlier
    - MEDIUM: http: remove even more of the spaghetti in the request path
    - MINOR: http: silently support the "block" action for http-request
    - CLEANUP: proxy: rename "block_cond" to "block_rules"
    - MEDIUM: http: emulate "block" rules using "http-request" rules
    - MINOR: http: remove the now unused loop over "block" rules
    - MEDIUM: http: factorize the "auth" action of http-request and stats
    - MEDIUM: http: make http-request rules processing return a verdict instead of a rule
    - MINOR: config: add minimum support for emitting warnings only once
    - MEDIUM: config: inform the user about the deprecatedness of "block" rules
    - MEDIUM: config: inform the user that "reqsetbe" is deprecated
    - MEDIUM: config: inform the user only once that "redispatch" is deprecated
    - MEDIUM: config: warn that '{cli,con,srv}timeout' are deprecated
    - BUG/MINOR: auth: fix wrong return type in pat_match_auth()
    - BUILD: config: remove a warning with clang
    - BUG/MAJOR: http: connection setup may stall on balance url_param
    - BUG/MEDIUM: http/session: disable client-side expiration only after body
    - BUG/MEDIUM: http: correctly report request body timeouts
    - BUG/MEDIUM: http: disable server-side expiration until client has sent the body
    - MEDIUM: listener: make the accept function more robust against pauses
    - BUILD: syscalls: remove improper inline statement in front of syscalls
    - BUILD: ssl: SSL_CTX_set_msg_callback() needs openssl >= 0.9.7
    - BUG/MAJOR: session: recover the correct connection pointer in half-initialized sessions
    - DOC: add some explanation on the shared cache build options in the readme.
    - MEDIUM: proxy: only adjust the backend's bind-process when already set
    - MEDIUM: config: limit nbproc to the machine's word size
    - MEDIUM: config: check the bind-process settings according to nbproc
    - MEDIUM: listener: parse the new "process" bind keyword
    - MEDIUM: listener: inherit the process mask from the proxy
    - MAJOR: listener: only start listeners bound to the same processes
    - MINOR: config: only report a warning when stats sockets are bound to more than 1 process
    - CLEANUP: config: set the maxaccept value for peers listeners earlier
    - BUG/MINOR: backend: only match IPv4 addresses with RDP cookies
    - BUG/MINOR: checks: correctly configure the address family and protocol
    - MINOR: tools: split is_addr() and is_inet_addr()
    - MINOR: protocols: use is_inet_addr() when only INET addresses are desired
    - MEDIUM: unix: add preliminary support for connecting to servers over UNIX sockets
    - MEDIUM: checks: only complain about the missing port when the check uses TCP
    - MEDIUM: unix: implement support for Linux abstract namespace sockets
    - DOC: map_beg was missing from the table of map_* converters
    - DOC: ebtree: indicate that prefix insertion/lookup may be used with strings
    - MEDIUM: pattern: use ebtree's longest match to index/lookup string beginning
    - BUILD: remove the obsolete BSD and OSX makefiles
    - MEDIUM: unix: avoid a double connect probe when no data are sent
    - DOC: stop referencing the slow git repository in the README
    - BUILD: only build the systemd wrapper on Linux 2.6 and above
    - DOC: update roadmap with completed tasks
    - MEDIUM: session: implement half-closed timeouts (client-fin and server-fin)
2014-05-10 15:16:43 +02:00
Willy Tarreau
05cdd9655d MEDIUM: session: implement half-closed timeouts (client-fin and server-fin)
Long-lived sessions are often subject to half-closed sessions resulting in
a lot of sessions appearing in FIN_WAIT state in the system tables, and no
way for haproxy to get rid of them. This typically happens because clients
suddenly disconnect without sending any packet (eg: FIN or RST was lost in
the path), and while the server detects this using an applicative heart
beat, haproxy does not close the connection.

This patch adds two new timeouts : "timeout client-fin" and
"timeout server-fin". The former allows one to override the client-facing
timeout when a FIN has been received or sent. The latter does the same for
server-facing connections, which is less useful.
2014-05-10 15:14:05 +02:00
Willy Tarreau
787a4c025f DOC: map_beg was missing from the table of map_* converters 2014-05-10 07:55:30 +02:00
Willy Tarreau
ccfccefb80 MEDIUM: unix: implement support for Linux abstract namespace sockets
These sockets are the same as Unix sockets except that there's no need
for any filesystem access. The address may be whatever string both sides
agree upon. This can be really convenient for inter-process communications
as well as for chaining backends to frontends.

These addresses are forced by prepending their address with "abns@" for
"abstract namespace".
2014-05-10 01:53:58 +02:00
Willy Tarreau
ae30253c27 MAJOR: listener: only start listeners bound to the same processes
Now that we know what processes a "bind" statement is attached to, we
have the ability to avoid starting some of them when they're not on the
proper process. This feature is disabled when running in foreground
however, so that debug mode continues to work with everything bound to
the first and only process.

The main purpose of this change is to finally allow the global stats
sockets to be each bound to a different process.

It can also be used to force haproxy to use different sockets in different
processes for the same IP:port. The purpose is that under Linux 3.9 and
above (and possibly other OSes), when multiple processes are bound to the
same IP:port via different sockets, the system is capable of performing
a perfect round-robin between the socket queues instead of letting any
process pick all the connections from a queue. This results in a smoother
load balancing and may achieve a higher performance with a large enough
maxaccept setting.
2014-05-09 19:16:26 +02:00
Willy Tarreau
6ae1ba6f29 MEDIUM: listener: parse the new "process" bind keyword
This sets the bind_proc entry in the bind_conf config block. For now it's
still unused, but the doc was updated.
2014-05-09 19:16:26 +02:00
Willy Tarreau
102df613a9 MEDIUM: config: check the bind-process settings according to nbproc
When a bind-process setting is present in a frontend or backend, we
now verify that the specified process range at least shares one common
process with those defined globally by nbproc. Then if the value is
set, it is reduced to the one enforced by nbproc.

A warning is emitted if process count does not match, and the fix is
done the following way :
  - if a single process was specified in the range, it's remapped to
    process #1
  - if more than one process was specified, the binding is removed
    and all processes are usable.

Note that since backends may inherit their settings from frontends,
depending on the declaration order, they may or may not be reported
as warnings.
2014-05-09 19:16:26 +02:00
Willy Tarreau
a9db57ec5c MEDIUM: config: limit nbproc to the machine's word size
Some consistency checks cannot be performed between frontends, backends
and peers at the moment because there is no way to check for intersection
between processes bound to some processes when the number of processes is
higher than the number of bits in a word.

So first, let's limit the number of processes to the machine's word size.
This means nbproc will be limited to 32 on 32-bit machines and 64 on 64-bit
machines. This is far more than enough considering that configs rarely go
above 16 processes due to scalability and management issues, so 32 or 64
should be fine.

This way we'll ensure we can always build a mask of all the processes a
section is bound to.
2014-05-09 19:16:26 +02:00
Emeric Brun
8dc6039807 MINOR: ssl: add global statement tune.ssl.force-private-cache.
Boolean: used to force a private ssl session cache for each process in
case of nbproc > 1.
2014-05-09 19:16:13 +02:00
David S
afb768340c MEDIUM: connection: Implement and extented PROXY Protocol V2
This commit modifies the PROXY protocol V2 specification to support headers
longer than 255 bytes allowing for optional extensions.  It implements the
PROXY protocol V2 which is a binary representation of V1. This will make
parsing more efficient for clients who will know in advance exactly how
many bytes to read.  Also, it defines and implements some optional PROXY
protocol V2 extensions to send information about downstream SSL/TLS
connections.  Support for PROXY protocol V1 remains unchanged.
2014-05-09 08:25:38 +02:00
Emeric Brun
b73a9b039c MINOR: ssl: convert to binary ssl_fc_unique_id and ssl_bc_unique_id.
Previously ssl_fc_unique_id and ssl_bc_unique_id return a string encoded
in base64 of the RFC 5929 TLS unique identifier. This patch modify those fetches
to return directly the ID in the original binary format. The user can make the
choice to encode in base64 using the converter.

i.e. : ssl_fc_unique_id,base64
2014-04-30 22:31:11 +02:00
Emeric Brun
53d1a98270 MINOR: ssl: adds sample converter base64 for binary type.
The new converter encode binary type sample to base64 string.

i.e. : ssl_c_serial,base64
2014-04-30 22:31:11 +02:00