Also add some thoughts about the existing and new design.
Note: an earlier design used the names "head" and "tail" for both sides
of the buffer, but it appears awkward as these words may be understood
in two forms (feed by head, output by tail, or make the newcomers wait
at the tail of the queue). Also there were already a few functions in the
code making use of either terminology. So better avoid this terminology
and use "input" and "output" instead.
- Typo on "dispatch" keyword arguments.
- Reindent some blocks for better parsing by automated tools.
- "option mysql-check" and "option pgsql-check" arguments were not documented
as the others.
This patch is a group commit simplify the parsing of the documenation :
- remove remaining tabulations
- realign some lines
- break lines at 80 columns
- add missing chapters in the summary
- fix chapter numbering format
The Unique ID, is an ID generated with several informations. You can use
a log-format string to customize it, with the "unique-id-format" keyword,
and insert it in the request header, with the "unique-id-header" keyword.
%Fi: Frontend IP
%Fp: Frontend Port
%Si: Server IP
%Sp: Server Port
%Ts: Timestamp
%rt: HTTP request counter
%H: hostname
%pid: PID
+X: Hexadecimal represenation
The +X mode in logformat displays hexadecimal for the following flags
%Ci %Cp %Fi %Fp %Bi %Bp %Si %Sp %Ts %ct %pid
rename logformat_write_string() to lf_text()
Optimize size computation
The ACL matches rely on the extract_cookie_value() function as used for
for patterns. This permits ACLs to match cookie values based on the cookie
name instead of having to perform substring matching on the cookie header.
Sometimes it is desirable to forward a particular request to a specific
server without having to declare a dedicated backend for this server. This
can be achieved using the "use-server" rules. These rules are evaluated after
the "redirect" rules and before evaluating cookies, and they have precedence
on them. There may be as many "use-server" rules as desired. All of these
rules are evaluated in their declaration order, and the first one which
matches will assign the server.
Released version 1.5-dev8 with the following main changes :
- MINOR: patch for minor typo (ressources/resources)
- MEDIUM: http: add support for sending the server's name in the outgoing request
- DOC: mention that default checks are TCP connections
- BUG/MINOR: fix options forwardfor if-none when an alternative header name is specified
- CLEANUP: Make check_statuses, analyze_statuses and process_chk static
- CLEANUP: Fix HCHK spelling errors
- BUG/MINOR: fix typo in processing of http-send-name-header
- MEDIUM: log: Use linked lists for loggers
- BUILD: fix declaration inside a scope block
- REORG: log: split send_log function
- MINOR: config: Parse the string of the log-format config keyword
- MINOR: add ultoa, ulltoa, ltoa, lltoa implementations
- MINOR: Date and time fonctions that don't use snprintf
- MEDIUM: log: make http_sess_log use log_format
- DOC: log-format documentation
- MEDIUM: log: use log_format for mode tcplog
- MEDIUM: log-format: backend source address %Bi %Bp
- BUG/MINOR: log-format: fix %o flag
- BUG/MEDIUM: bad length in log_format and __send_log
- MINOR: logformat %st is signed
- BUILD/MINOR: fix the source URL in the spec file
- DOC: acl is http_first_req, not http_req_first
- BUG/MEDIUM: don't trim last spaces from headers consisting only of spaces
- MINOR: acl: add new matches for header/path/url length
- BUILD: halog: make halog build on solaris
- BUG/MINOR: don't use a wrong port when connecting to a server with mapped ports
- MINOR: remove the client/server side distinction in SI addresses
- MINOR: halog: add support for matching queued requests
- DOC: indicate that cookie "prefix" and "indirect" should not be mixed
- OPTIM/MINOR: move struct sockaddr_storage to the tail of structs
- OPTIM/MINOR: make it possible to change pipe size (tune.pipesize)
- BUILD/MINOR: silent a build warning in src/pipe.c (fcntl)
- OPTIM/MINOR: move the hdr_idx pools out of the proxy struct
- MEDIUM: tune.http.maxhdr makes it possible to configure the maximum number of HTTP headers
- BUG/MINOR: fix a segfault when parsing a config with undeclared peers
- CLEANUP: rename possibly confusing struct field "tracked"
- BUG/MEDIUM: checks: fix slowstart behaviour when server tracking is in use
- MINOR: config: tolerate server "cookie" setting in non-HTTP mode
- MEDIUM: buffers: add some new primitives and rework existing ones
- BUG: buffers: don't return a negative value on buffer_total_space_res()
- MINOR: buffers: make buffer_pointer() support negative pointers too
- CLEANUP: kill buffer_replace() and use an inline instead
- BUG: tcp: option nolinger does not work on backends
- CLEANUP: ebtree: remove a few annoying signedness warnings
- CLEANUP: ebtree: clarify licence and update to 6.0.6
- CLEANUP: ebtree: remove 4-year old harmless typo in duplicates insertion code
- CLEANUP: ebtree: remove another typo, a wrong initialization in insertion code
- BUG: ebtree: ebst_lookup() could return the wrong entry
- OPTIM: stream_sock: reduce the amount of in-flight spliced data
- OPTIM: stream_sock: save a failed recv syscall when splice returns EAGAIN
- MINOR: acl: add support for TLS server name matching using SNI
- BUG: http: re-enable TCP quick-ack upon incomplete HTTP requests
- BUG: proto_tcp: don't try to bind to a foreign address if sin_family is unknown
- MINOR: pattern: export the global temporary pattern
- CLEANUP: patterns: get rid of pattern_data_setstring()
- MEDIUM: acl: use temp_pattern to store fetched information in the "method" match
- MINOR: acl: include pattern.h to make pattern migration more transparent
- MEDIUM: pattern: change the pattern data integer from unsigned to signed
- MEDIUM: acl: use temp_pattern to store any integer-type information
- MEDIUM: acl: use temp_pattern to store any address-type information
- CLEANUP: acl: integer part of acl_test is not used anymore
- MEDIUM: acl: use temp_pattern to store any string-type information
- CLEANUP: acl: remove last data fields from the acl_test struct
- MEDIUM: http: replace get_ip_from_hdr2() with http_get_hdr()
- MEDIUM: patterns: the hdr() pattern is now of type string
- DOC: add minimal documentation on how ACLs work internally
- DOC: add a coding-style file
- OPTIM: halog: keep a fast path for the lines-count only
- CLEANUP: silence a warning when building on sparc
- BUG: http: tighten the list of allowed characters in a URI
- MEDIUM: http: block non-ASCII characters in URIs by default
- DOC: add some documentation from RFC3986 about URI format
- BUG/MINOR: cli: correctly remove the whole table on "clear table"
- BUG/MEDIUM: correctly disable servers tracking another disabled servers.
- BUG/MEDIUM: zero-weight servers must not dequeue requests from the backend
- MINOR: halog: add some help on the command line
- BUILD: fix build error on FreeBSD
- BUG: fix double free in peers config error path
- MEDIUM: improve config check return codes
- BUILD: make it possible to look for pcre in the default system paths
- MINOR: config: emit a warning when 'default_backend' masks servers
- MINOR: backend: rework the LC definition to support other connection-based algos
- MEDIUM: backend: add the 'first' balancing algorithm
- BUG: fix httplog trailing LF
- MEDIUM: increase chunk-size limit to 2GB-1
- BUG: queue: fix dequeueing sequence on HTTP keep-alive sessions
- BUG: http: disable TCP delayed ACKs when forwarding content-length data
- BUG: checks: fix server maintenance exit sequence
- BUG/MINOR: stream_sock: don't remove BF_EXPECT_MORE and BF_SEND_DONTWAIT on partial writes
- DOC: enumerate valid status codes for "observe layer7"
- MINOR: buffer: switch a number of buffer args to const
- CLEANUP: silence signedness warning in acl.c
- BUG: stream_sock: si->release was not called upon shutw()
- MINOR: log: use "%ts" to log term status only and "%tsc" to log with cookie
- BUG/CRITICAL: log: fix risk of crash in development snapshot
- BUG/MAJOR: possible crash when using capture headers on TCP frontends
- MINOR: config: disable header captures in TCP mode and complain
The main stats page says "ressources" (French spelling) rather than
"resources" (English spelling).
One little patch attached (against v1.4.20).
Many thanks,
Adrian
The difference could be seen when logging a request in HTTP mode with option
tcplog, as it would keep emitting 4 chars. Better use two distinct flags to
clear the confusion.
%Bi return the backend source IP
%Bp return the backend source port
Add a function pointer in logformat_type to do additional configuration
during the log-format variable parsing.
Merge http_sess_log() and tcp_sess_log() to sess_log() and move it to
log.c
A new field in logformat_type define if you can use a logformat
variable in TCP or HTTP mode.
doc: log-format in tcp mode
Note that due to the way log buffer allocation currently works, trying to
log an HTTP request without "option httplog" is still not possible. This
will change in the near future.
The principle behind this load balancing algorithm was first imagined
and modeled by Steen Larsen then iteratively refined through several
work sessions until it would totally address its original goal.
The purpose of this algorithm is to always use the smallest number of
servers so that extra servers can be powered off during non-intensive
hours. Additional tools may be used to do that work, possibly by
locally monitoring the servers' activity.
The first server with available connection slots receives the connection.
The servers are choosen from the lowest numeric identifier to the highest
(see server parameter "id"), which defaults to the server's position in
the farm. Once a server reaches its maxconn value, the next server is used.
It does not make sense to use this algorithm without setting maxconn. Note
that it can however make sense to use minconn so that servers are not used
at full load before starting new servers, and so that introduction of new
servers requires a progressively increasing load (the number of servers
would more or less follow the square root of the load until maxconn is
reached). This algorithm ignores the server weight, and is more beneficial
to long sessions such as RDP or IMAP than HTTP, though it can be useful
there too.
These ones are invalid and blocked unless "option accept-invalid-http-request"
is specified in the frontend. In any case, the faulty request is logged.
Note that some of the remaining invalid chars are still not checked against,
those are the invalid ones between 32 and 127 :
34 ('"'), 60 ('<'), 62 ('>'), 92 ('\'), 94 ('^'),
96 ('`'), 123 ('{'), 124 ('|'), 125 ('}')
Using a lookup table might be better at some point.
New option "http-send-name-header" specifies the name of a header which
will hold the server name in outgoing requests. This is the name of the
server the connection is really sent to, which means that upon redispatches,
the header's value is updated so that it always matches the server's name.
This pattern previously was limited to type IP. With the new header
extraction function, it becomes possible to extract strings, so that
the header can be returned as a string. This will not change anything
to existing configs, as string will automatically be converted to IP
when needed. However, new configs will be able to use IPv6 addresses
from headers in stick-tables, as well as stick on any non-IP header
(eg: host, user-agent, ...).
Server Name Indication (SNI) is a TLS extension which makes a client
present the name of the server it is connecting to in the client hello.
It allows a transparent proxy to take a decision based on the beginning
of an SSL/TLS stream without deciphering it.
The new ACL "req_ssl_sni" matches the name extracted from the TLS
handshake against a list of names which may be loaded from a file if
needed.
This patch settles the 2 loggers limitation.
Loggers are now stored in linked lists.
Using "global log", the global loggers list content is added at the end
of the current proxy list. Each "log" entries are added at the end of
the proxy list.
"no log" flush a logger list.
For a long time, the max number of headers was taken as a part of the buffer
size. Since the header size can be configured at runtime, it does not make
much sense anymore.
Nothing was making it necessary to have a static value, so let's turn this into
a tunable with a default value of 101 which equals what was previously used.
By default, pipes are the default size for the system. But sometimes when
using TCP splicing, it can improve performance to increase pipe sizes,
especially if it is suspected that pipes are not filled and that many
calls to splice() are performed. This has an impact on the kernel's
memory footprint, so this must not be changed if impacts are not understood.
When prefix and indirect are used together, a client which connects to
a server with a cookie will never get any cookie update from this server,
which will be removed by the "indirect" option.
This patch introduces hdr_len, path_len and url_len for matching these
respective parts lengths against integers. This can be used to detect
abuse or empty headers.
Released version 1.5-dev7 with the following main changes :
- [BUG] fix binary stick-tables
- [MINOR] http: *_dom matching header functions now also split on ":"
- [BUG] checks: fix support of Mysqld >= 5.5 for mysql-check
- [MINOR] acl: add srv_conn acl to count connections on a specific backend server
- [MINOR] check: add redis check support
- [DOC] small fixes to clearly distinguish between keyword and variables
- [MINOR] halog: add support for termination code matching (-tcn/-TCN)
- [DOC] Minor spelling fixes and grammatical enhancements
- [CLEANUP] dumpstats: make symbols static where possible
- [MINOR] Break out dumping table
- [MINOR] Break out processing of clear table
- [MINOR] Allow listing of stick table by key
- [MINOR] Break out all stick table socat command parsing
- [MINOR] More flexible clearing of stick table
- [MINOR] Allow showing and clearing by key of ipv6 stick tables
- [MINOR] Allow showing and clearing by key of integer stick tables
- [MINOR] Allow showing and clearing by key of string stick tables
- [CLEANUP] Remove assigned but unused variables
- [CLEANUP] peers.h: fix declarations
- [CLEANUP] session.c: Make functions static where possible
- [MINOR] Add active connection list to server
- [MINOR] Allow shutdown of sessions when a server becomes unavailable
- [MINOR] Add down termination condition
- [MINOR] Make appsess{,ion}_refresh static
- [MINOR] Add rdp_cookie pattern fetch function
- [CLEANUP] Remove unnecessary casts
- [MINOR] Add non-stick server option
- [MINOR] Consistently use error in tcp_parse_tcp_req()
- [MINOR] Consistently free expr on error in cfg_parse_listen()
- [MINOR] Free rdp_cookie_name on denint()
- [MINOR] Free tcp rules on denint()
- [MINOR] Free stick table pool on denint()
- [MINOR] Free stick rules on denint()
- [MEDIUM] Fix stick-table replication on soft-restart
- [MEDIUM] Correct ipmask() logic
- [MINOR] Correct type in table dump examples
- [MINOR] Fix build error in stream_int_register_handler()
- [MINOR] Use DPRINTF in assign_server()
- [BUG] checks: http-check expect could fail a check on multi-packet responses
- [DOC] fix minor typo in the "dispatch" doc
- [BUG] proto_tcp: fix address binding on remote source
- [MINOR] http: don't report the "haproxy" word on the monitoring response
- [REORG] http: move HTTP error codes back to proto_http.h
- [MINOR] http: make the "HTTP 200" status code configurable.
- [MINOR] http: partially revert the chunking optimization for now
- [MINOR] stream_sock: always clear BF_EXPECT_MORE upon complete transfer
- [CLEANUP] stream_sock: remove unneeded FL_TCP and factor out test
- [MEDIUM] http: add support for "http-no-delay"
- [OPTIM] http: optimize chunking again in non-interactive mode
- [OPTIM] stream_sock: avoid fast-forwarding of partial data
- [OPTIM] stream_sock: don't use splice on too small payloads
- [MINOR] config: make it possible to specify a cookie even without a server
- [BUG] stats: support url-encoded forms
- [MINOR] config: automatically compute a default fullconn value
- [CLEANUP] config: remove some left-over printf debugging code from previous patch
- [DOC] add missing entry or stick store-response
- [MEDIUM] http: add support for 'cookie' and 'set-cookie' patterns
- [BUG] halog: correctly handle truncated last line
- [MINOR] halog: make SKIP_CHAR stop on field delimiters
- [MINOR] halog: add support for HTTP log matching (-H)
- [MINOR] halog: gain back performance before SKIP_CHAR fix
- [OPTIM] halog: cache some common fields positions
- [OPTIM] halog: check once for correct line format and reuse the pointer
- [OPTIM] halog: remove many 'if' by using a function pointer for the filters
- [OPTIM] halog: remove support for tab delimiters in input data
- [BUG] session: risk of crash on out of memory (1.5-dev regression)
- [MINOR] session: try to emit a 500 response on memory allocation errors
- [OPTIM] stream_sock: reduce the default number of accepted connections at once
- [BUG] stream_sock: disable listener when system resources are exhausted
- [MEDIUM] proxy: add a PAUSED state to listeners and move socket tricks out of proxy.c
- [BUG] stream_sock: ensure orphan listeners don't accept too many connections
- [MINOR] listeners: add listen_full() to mark a listener full
- [MINOR] listeners: add support for queueing resource limited listeners
- [MEDIUM] listeners: put listeners in queue upon resource shortage
- [MEDIUM] listeners: queue proxy-bound listeners at the proxy's
- [MEDIUM] listeners: don't stop proxies when global maxconn is reached
- [MEDIUM] listeners: don't change listeners states anymore in maintain_proxies
- [CLEANUP] proxy: rename a few proxy states (PR_STIDLE and PR_STRUN)
- [MINOR] stats: report a "WAITING" state for sockets waiting for resource
- [MINOR] proxy: make session rate-limit more accurate
- [MINOR] sessions: only wake waiting listeners up if rate limit is OK
- [BUG] proxy: peers must only be stopped once, not upon every call to maintain_proxies
- [CLEANUP] proxy: merge maintain_proxies() operation inside a single loop
- [MINOR] task: new function task_schedule() to schedule a wake up
- [MAJOR] proxy: finally get rid of maintain_proxies()
- [BUG] proxy: stats frontend and peers were missing many initializers
- [MEDIUM] listeners: add a global listener management task
- [MINOR] proxy: make findproxy() return proxies from numeric IDs too
- [DOC] fix typos, "#" is a sharp, not a dash
- [MEDIUM] stats: add support for changing frontend's maxconn at runtime
- [MEDIUM] checks: group health checks methods by values and save option bits
- [MINOR] session-counters: add the ability to clear the counters
- [BUG] check: http-check expect + regex would crash in defaults section
- [MEDIUM] http: make x-forwarded-for addition conditional
- [REORG] build: move syscall redefinition to specific places
- [CLEANUP] update the year in the copyright banner
- [BUG] possible crash in 'show table' on stats socket
- [BUG] checks: use the correct destination port for sending checks
- [BUG] backend: risk of picking a wrong port when mapping is used with crossed families
- [MINOR] make use of set_host_port() and get_host_port() to get rid of family mismatches
- [DOC] fixed a few "sensible" -> "sensitive" errors
- [MINOR] make use of addr_to_str() and get_host_port() to replace many inet_ntop()
- [BUG] http: trailing white spaces must also be trimmed after headers
- [MINOR] stats: display "<NONE>" instead of the frontend name when unknown
- [MINOR] http: take a capture of too large requests and responses
- [MINOR] http: take a capture of truncated responses
- [MINOR] http: take a capture of bad content-lengths.
- [DOC] add a few old and uncommitted docs
- [CLEANUP] cfgparse: fix reported options for the "bind" keyword
- [MINOR] halog: add -hs/-HS to filter by HTTP status code range
- [MINOR] halog: support backslash-escaped quotes
- [CLEANUP] remove dirty left-over of a debugging message
- [MEDIUM] stats: disable complex socket reservation for stats socket
- [CLEANUP] remove a useless test in manage_global_listener_queue()
- [MEDIUM] stats: add the "set maxconn" setting to the command line interface
- [MEDIUM] add support for global.maxconnrate to limit the per-process conn rate.
- [MINOR] stats: report the current and max global connection rates
- [MEDIUM] stats: add the ability to adjust the global maxconnrate
- [BUG] peers: don't pre-allocate 65000 connections to each peer
- [MEDIUM] don't limit peers nor stats socket to maxconn nor maxconnrate
- [BUG] peers: the peer frontend must not emit any log
- [CLEANUP] proxy: make pause_proxy() perform the required controls and emit the logs
- [BUG] peers: don't keep a peers section which has a NULL frontend
- [BUG] peers: ensure the peers are resumed if they were paused
- [MEDIUM] stats: add the ability to enable/disable/shutdown a frontend at runtime
- [MEDIUM] session: make session_shutdown() an independant function
- [MEDIUM] stats: offer the possibility to kill a session from the CLI
- [CLEANUP] stats: centralize tests for backend/server inputs on the CLI
- [MEDIUM] stats: offer the possibility to kill sessions by server
- [MINOR] halog: do not consider byte 0x8A as end of line
- [MINOR] frontend: ensure debug message length is always initialized
- [OPTIM] halog: make fgets parse more bytes by blocks
- [OPTIM] halog: add assembly version of the field lookup code
- [MEDIUM] poll: add a measurement of idle vs work time
- [CLEANUP] startup: report only the basename in the usage message
- [MINOR] startup: add an option to change to a new directory
- [OPTIM] task: don't scan the run queue if we know it's empty
- [BUILD] stats: stdint is not present on solaris
- [DOC] update the README file to reflect new naming rules for patches
- [MINOR] stats: report the number of requests intercepted by the frontend
- [DOC] update ROADMAP file
We already had the ability to kill a connection, but it was only
for the checks. Now we can do this for any session, and for this we
add a specific flag "K" to the logs.
The stats socket now allows the admin to disable, enable or shutdown a frontend.
This can be used when a bug is discovered in a configuration and it's desirable
to fix it but the rules in place don't allow to change a running config. Thus it
becomes possible to kill the frontend to release the port and start a new one in
a separate process.
This can also be used to temporarily make haproxy return TCP resets to incoming
requests to pretend the service is not bound. For instance, this may be useful
to quickly flush a very deep SYN backlog.
The frontend check and lookup code was factored with the "set maxconn" usage.
This one enforces a per-process connection rate limit, regardless of what
may be set per frontend. It can be a way to limit the CPU usage of a process
being severely attacked.
The side effect is that the global process connection rate is now measured
for each incoming connection, so it will be possible to report it.
This option permits to change the global maxconn setting within the
limit that was set by the initial value, which is now reported as the
hard maxconn value. This allows to immediately accept more concurrent
connections or to stop accepting new ones until the value passes below
the indicated setting.
The main use of this option is on systems where many haproxy instances
are loaded and admins need to re-adjust resource sharing at run time
to regain a bit of fairness between processes.
If "option forwardfor" has the "if-none" argument, then the header is
only added when the request did not already have one. This option has
security implications, and should not be set blindly.
Manoj Kumar reported a case where haproxy would crash upon start-up. The
cause was an "http-check expect" statement declared in the defaults section,
which caused a NULL regex to be used during the check. This statement is not
allowed in defaults sections precisely because this requires saving a copy
of the regex in the default proxy. But the check was not made to prevent it
from being declared there, hence the issue.
Instead of adding code to detect its abnormal use, we decided to implement
it. It was not that much complex because the expect_str part was not used
with regexes, so it could hold the string form of the regex in order to
compile it again for every backend (there's no way to clone regexes).
This patch has been tested and works. So it's both a bugfix and a minor
feature enhancement.
It should be backported to 1.4 though it's not critical since the config
was not supposed to be supported.
This patch provides a new "option redis-check" statement to enable server health checks based on redis PING request (http://www.redis.io/commands/ping).
The new "set maxconn frontend XXX" statement on the stats socket allows
the admin to change a frontend's maxconn value. If some connections are
queued, they will immediately be accepted up to the new limit. If the
limit is lowered, new connections acceptation might be delayed. This can
be used to temporarily reduce or increase the impact of a specific frontend's
traffic on the whole process.
This is used to perform cookie-based stickiness with table replication
between multiple masters and across restarts. This partially overrides
some of the appsession capabilities.
Never add connections allocated to this sever to a stick-table.
This may be used in conjunction with backup to ensure that
stick-table persistence is disabled for backup servers.
This pattern fetch function extracts the value of the rdp cookie <name> as
a string and uses this value to match. This enables implementation of
persistence based on the mstshash cookie. This is typically done if there
is no msts cookie present.
This differs from "balance rdp-cookie" in that any balancing algorithm may
be used and thus the distribution of clients to backend servers is not
linked to a hash of the RDP cookie. It is envisaged that using a balancing
algorithm such as "balance roundrobin" or "balance leastconnect" will lead
to a more even distribution of clients to backend servers than the hash
used by "balance rdp-cookie".
Example :
listen tse-farm
bind 0.0.0.0:3389
# wait up to 5s for an RDP cookie in the request
tcp-request inspect-delay 5s
tcp-request content accept if RDP_COOKIE
# apply RDP cookie persistence
persist rdp-cookie
# Persist based on the mstshash cookie
# This is only useful makes sense if
# balance rdp-cookie is not used
stick-table type string size 204800
stick on rdp_cookie(mstshash)
server srv1 1.1.1.1:3389
server srv1 1.1.1.2:3389
If a connection is closed by because the backend became unavailable
then log 'D' as the termination condition.
Signed-off-by: Simon Horman <horms@verge.net.au>
This adds the "on-marked-down shutdown-sessions" statement on "server" lines,
which causes all sessions established on a server to be killed at once when
the server goes down. The task's priority is reniced to the highest value
(1024) so that servers holding many tasks don't cause a massive slowdown due
to the wakeup storm.
mysqld >= 5.5 want the client to announce 4.1+ authentication support, even if we have no password, so we do this.
I also check on a debian potato mysqld 3.22 and it works too so i assume we are good from 3.22 to 5.5.
[WT: this must be backported to 1.4]
The fullconn value is not easy to get right when doing dynamic regulation,
as it should depend on the maxconns of the frontends that can reach a
backend. Since the parameter is mandatory, many configs are found with
an inappropriate default value.
Instead of rejecting configs without a fullconn value, we now set it to
10% of the sum of the configured maxconns of all the frontends which are
susceptible to branch to the backend. That way if new frontends are added,
the backend's fullconn automatically adjusts itself.
There are some very rare server-to-server applications that abuse the HTTP
protocol and expect the payload phase to be highly interactive, with many
interleaved data chunks in both directions within a single request. This is
absolutely not supported by the HTTP specification and will not work across
most proxies or servers. When such applications attempt to do this through
haproxy, it works but they will experience high delays due to the network
optimizations which favor performance by instructing the system to wait for
enough data to be available in order to only send full packets. Typical
delays are around 200 ms per round trip. Note that this only happens with
abnormal uses. Normal uses such as CONNECT requests nor WebSockets are not
affected.
When "option http-no-delay" is present in either the frontend or the backend
used by a connection, all such optimizations will be disabled in order to
make the exchanges as fast as possible. Of course this offers no guarantee on
the functionality, as it may break at any other place. But if it works via
HAProxy, it will work as fast as possible. This option should never be used
by default, and should never be used at all unless such a buggy application
is discovered. The impact of using this option is an increase of bandwidth
usage and CPU usage, which may significantly lower performance in high
latency environments.
This change should be backported to 1.4 since the first report of such a
misuse was in 1.4. Next patch will also be needed.
This status code is used in response to requests matching "monitor-uri".
Some users need to adjust it to fit their needs (eg: make some strings
appear there). As it's already defined as a chunked string and used
exactly like other status codes, it makes sense to make it configurable
with the usual "errorfile", "errorloc", ...
Released version 1.5-dev6 with the following main changes :
- [BUG] stream_sock: use get_addr_len() instead of sizeof() on sockaddr_storage
- [BUG] TCP source tracking was broken with IPv6 changes
- [BUG] stick-tables did not work when converting IPv6 to IPv4
- [CRITICAL] fix risk of crash when dealing with space in response cookies
Released version 1.5-dev5 with the following main changes :
- [BUG] standard: is_addr return value for IPv4 was inverted
- [MINOR] update comment about IPv6 support for server
- [MEDIUM] use getaddrinfo to resolve names if gethostbyname fail
- [DOC] update IPv6 support for bind
- [DOC] document IPv6 support for server
- [DOC] fix a minor typo
- [MEDIUM] IPv6 support for syslog
- [DOC] document IPv6 support for syslog
- [MEDIUM] IPv6 support for stick-tables
- [DOC] document IPv6 support for stick-tables
- [DOC] update ROADMAP file
- [BUG] session: src_conn_cur was returning src_conn_cnt instead
- [MINOR] frontend: add a make_proxy_line function
- [MEDIUM] stream_sock: add support for sending the proxy protocol header line
- [MEDIUM] server: add support for the "send-proxy" option
- [DOC] update the spec on the proxy protocol
- [BUILD] proto_tcp: fix build issue with CTTPROXY
- [DOC] update ROADMAP file
- [MEDIUM] config: rework the IPv4/IPv6 address parser to support host-only addresses
- [MINOR] cfgparse: better report wrong listening addresses and make use of str2sa_range
- [BUILD] add the USE_GETADDRINFO build option
- [TESTS] provide a test case for various address formats
- [BUG] session: conn_retries was not always initialized
- [BUG] log: retrieve the target from the session, not the SI
- [BUG] http: fix possible incorrect forwarded wrapping chunk size (take 2)
- [MINOR] tools: add two macros MID_RANGE and MAX_RANGE
- [BUG] http: fix content-length handling on 32-bit platforms
- [OPTIM] buffers: uninline buffer_forward()
- [BUG] stream_sock: fix handling for server side PROXY protocol
- [MINOR] acl: add support for table_cnt and table_avl matches
- [DOC] update ROADMAP file
This option enables use of the PROXY protocol with the server, which
allows haproxy to transport original client's address across multiple
architecture layers.
Released version 1.5-dev4 with the following main changes :
- [MINOR] cfgparse: Check whether the path given for the stats socket actually fits into the sockaddr_un structure to avoid truncation.
- [MINOR] unix sockets : inherits the backlog size from the listener
- [CLEANUP] unix sockets : move create_uxst_socket() in uxst_bind_listener()
- [DOC] fix a minor typo
- [DOC] fix ignore-persist documentation
- [MINOR] add warnings on features not compatible with multi-process mode
- [BUG] http: fix http-pretend-keepalive and httpclose/tunnel mode
- [MINOR] stats: add support for several packets in stats admin
- [BUG] stats: admin commands must check the proxy state
- [BUG] stats: admin web interface must check the proxy state
- [MINOR] http: add pattern extraction method to stick on query string parameter
- [MEDIUM] add internal support for IPv6 server addresses
- [MINOR] acl: add be_id/srv_id to match backend's and server's id
- [MINOR] log: add support for passing the forwarded hostname
- [MINOR] log: ability to override the syslog tag
- [MINOR] checks: add PostgreSQL health check
- [DOC] update ROADMAP file
- [BUILD] pattern: use 'int' instead of 'int32_t'
- [OPTIM] linux: add support for bypassing libc to force using vsyscalls
- [BUG] debug: report the correct poller list in verbose mode
- [BUG] capture: do not capture a cookie if there is no memory left
- [BUG] appsession: fix possible double free in case of out of memory
- [CRITICAL] cookies: mixing cookies in indirect mode and appsession can crash the process
- [BUG] http: correctly update the header list when removing two consecutive headers
- [BUILD] add the CPU=native and ARCH=32/64 build options
- [BUILD] add -fno-strict-aliasing to fix warnings with gcc >= 4.4
- [CLEANUP] hash: move the avalanche hash code globally available
- [MEDIUM] hash: add support for an 'avalanche' hash-type
- [DOC] update roadmap file
- [BUG] http: do not re-enable the PROXY analyser on keep-alive
- [OPTIM] http: don't send each chunk in a separate packet
- [DOC] fix minor typos reported recently in the peers section
- [DOC] fix another typo in the doc
- [MINOR] stats: report HTTP message state and buffer flags in error dumps
- [BUG] http chunking: don't report a parsing error on connection errors
- [BUG] stream_interface: truncate buffers when sending error messages
- [MINOR] http: support wrapping messages in error captures
- [MINOR] http: capture incorrectly chunked message bodies
- [MINOR] stats: add global event ID and count
- [BUG] http: analyser optimizations broke pipelining
- [CLEANUP] frontend: only apply TCP-specific settings to TCP/TCP6 sockets
- [BUG] http: fix incorrect error reporting during data transfers
- [CRITICAL] session: correctly leave turn-around and queue states on abort
- [BUG] session: release slot before processing pending connections
- [MINOR] tcp: add support for dynamic MSS setting
- [BUG] stick-table: correctly terminate string keys during lookups
- [BUG] acl: fix handling of empty lines in pattern files
- [BUG] stick-table: use the private buffer when padding strings
- [BUG] ebtree: fix ebmb_lookup() with len smaller than the tree's keys
- [OPTIM] ebtree: ebmb_lookup: reduce stack usage by moving the return code out of the loop
- [OPTIM] ebtree: inline ebst_lookup_len and ebis_lookup_len
- [REVERT] undo the stick-table string key lookup fixes
- [MINOR] http: improve url_param pattern extraction to ignore empty values
- [BUILD] frontend: shut a warning with TCP_MAXSEG
- [BUG] http: update the header list's tail when removing the last header
- [DOC] fix minor typo in the proxy protocol doc
- [DOC] fix typos (http-request instead of http-check)
- [BUG] http: use correct ACL pointer when evaluating authentication
- [BUG] cfgparse: correctly count one socket per port in ranges
- [BUG] startup: set the rlimits before binding ports, not after.
- [BUG] acl: srv_id must return no match when the server is NULL
- [MINOR] acl: add ability to check for internal response-only parameters
- [MINOR] acl: srv_id is only valid in responses
- [MINOR] config: warn if response-only conditions are used in "redirect" rules
- [BUG] acl: fd leak when reading patterns from file
- [DOC] fix minor typo in "usesrc"
- [BUG] http: fix possible incorrect forwarded wrapping chunk size
- [BUG] http: fix computation of message body length after forwarding has started
- [BUG] http: balance url_param did not work with first parameters on POST
- [TESTS] update the url_param regression test to test check_post too
- [DOC] update ROADMAP
- [DOC] internal: reflect the fact that SI_ST_ASS is transient
- [BUG] config: don't crash on empty pattern files.
- [MINOR] stream_interface: make use of an applet descriptor for IO handlers
- [REORG] stream_interface: move the st0, st1 and private members to the applet
- [REORG] stream_interface: split the struct members in 3 parts
- [REORG] session: move client and server address to the stream interface
- [REORG] tcp: make tcpv4_connect_server() take the target address from the SI
- [MEDIUM] stream_interface: store the target pointer and type
- [CLEANUP] stream_interface: remove the applet.handler pointer
- [MEDIUM] log: take the logged server name from the stream interface
- [CLEANUP] session: remove data_source from struct session
- [CLEANUP] stats: make all dump functions only rely on the stream interface
- [REORG] session: move the data_ctx struct to the stream interface's applet
- [MINOR] proxy: add PR_O2_DISPATCH to detect dispatch mode
- [MINOR] cfgparse: only keep one of dispatch, transparent, http_proxy
- [MINOR] session: add a pointer to the new target into the session
- [MEDIUM] session: remove s->prev_srv which is not needed anymore
- [CLEANUP] stream_interface: use inline functions to manipulate targets
- [MAJOR] session: remove the ->srv pointer from struct session
- [MEDIUM] stats: split frontend and backend stats
- [MEDIUM] http: always evaluate http-request rules before stats http-request
- [REORG] http: move the http-request rules to proto_http
- [BUG] http: stats were not incremented on http-request deny
- [MINOR] checks: report it if checks fail due to socket creation error
Bryan Talbot reported that POST requests with a query string were not
correctly processed if the hash parameter was the first one, because
the delimiter that was looked for to trigger the parsing was '&' instead
of '?'.
Also, while checking the code, it became apparent that it was enough for
a query string to be present in the request for POST parameters to be
ignored, even if the url_param was in the body and not in the URL.
The code has then been fixed like this :
1) look for URL param. If found, return it.
2) if no URL param was found and method is POST, then look it up into
the body
The code now seems to pass all request combinations.
This patch must be backported to 1.4 since 1.4 is equally broken right now.
Some browsers send POST requests in several packets, which was not supported
by the "stats admin" function.
This patch allows to wait for more data when they are not fully received
(we are still limited to a certain size defined by the buffer size minus its
reserved space).
It also adds support for the "Expect: 100-Continue" header.
I have written a small patch to enable a correct PostgreSQL health check
It works similar to mysql-check with the very same parameters.
E.g.:
listen pgsql 127.0.0.1:5432
mode tcp
option pgsql-check user pgsql
server masterdb pgsql.server.com:5432 check inter 10000
One of the requirements we have is to run multiple instances of haproxy on a
single host; this is so that we can split the responsibilities (and change
permissions) between product teams. An issue we ran up against is how we
would distinguish between the logs generated by each instance. The solution
we came up with (please let me know if there is a better way) is to override
the application tag written to syslog. We can then configure syslog to write
these to different files.
I have attached a patch adding a global option 'log-tag' to override the
default syslog tag 'haproxy' (actually defaults to argv[0]).
By passing a negative value to the "mss" argument of "bind" lines, it
becomes possible to subtract this value to the MSS advertised by the
client, which results in segments smaller than advertised. The effect
is useful with some TCP stacks which ACK less often when segments are
not full, because they only ACK every other full segment as suggested
by RFC1122.
NOTE: currently this has no effect on Linux kernel 2.6, a kernel patch
is still required to change the MSS of established connections.
Haproxy does not include the hostname rather the IP of the machine in
the syslog headers it sends. Unfortunately this means that for each log
line rsyslog does a reverse dns on the client IP and in the case of
non-routable IPs one gets the public hostname not the internal one.
While this is valid according to RFC3164 as one might imagine this is
troublsome if you have some machines with public IPs, internal IPs, no
reverse DNS entries, etc and you want a standardized hostname based log
directory structure. The rfc says the preferred value is the hostname.
This patch adds a global "log-send-hostname" statement which accepts an
optional string to force the host name. If unset, the local host name
is used.
We've had several issues related to data transfers. First, if a
client aborted an upload before the server started to respond, it
would get a 502 followed by a 400. The same was true (in the other
way around) if the server suddenly aborted while the client was
uploading the data.
The flags reported in the logs were misleading. Request errors could
be reported while the transfer was stopped during the data phase. The
status codes could also be overwritten by a 400 eventhough the start
of the response was transferred to the client.
The stats were also wrong in case of data aborts. The server or the
client could sometimes be miscredited for being the author of the
abort depending on where the abort was detected. Some client aborts
could also be accounted as request errors and some server aborts as
response errors.
Now it seems like all such issues are fixed. Since we don't have a
specific state for data flowing from the client to the server
before the server responds, we're still counting the client aborted
transfers as "CH", and they become "CD" when the server starts to
respond. Ideally a "P" state would be desired.
This patch should be backported to 1.4.
Using haproxy in multi-process mode (nbproc > 1), some features can be
not fully compatible or not work at all. haproxy will now display a warning on
startup for :
- appsession
- sticking rules
- stats / stats admin
- stats socket
- peers (fatal error in that case)
During the documentation of the "ignore-persist" keyword, I documented an
invalid "option ignore-persist" and forgot to remove it. It's time to fix it.
When the number of servers is a multiple of the size of the input set,
map-based hash can be inefficient. This typically happens with 64
servers when doing URI hashing. The "avalanche" hash-type applies an
avalanche hash before performing a map lookup in order to smooth the
distribution. The result is slightly less smooth than the map for small
numbers of servers, but still better than the consistent hashing.
Released version 1.5-dev3 with the following main changes :
- [DOC] fix http-request documentation
- [MEDIUM] enable/disable servers from the stats web interface
- [MEDIUM] stats: add an admin level
- [DOC] stats: document the "stats admin" statement
- [MINOR] startup: print the proxy socket which caused an error
- [CLEANUP] Remove unneeded chars allocation
- [MINOR] config: detect options not supported due to compilation options
- [MINOR] Add pattern's fetchs payload and payload_lv
- [MINOR] frontend: improve accept-proxy header parsing
- [MINOR] frontend: add tcpv6 support on accept-proxy bind
- [MEDIUM] Enhance message errors management on binds
- [MINOR] Manage unix socket source field on logs
- [MINOR] Manage unix socket source field on session dump on sock stats
- [MINOR] Support of unix listener sockets for debug and log event messages on frontend.c
- [MINOR] Add some tests on sockets family for port remapping and mode transparent.
- [MINOR] Manage socket type unix for some logs
- [MINOR] Enhance controls of socket's family on acls and pattern fetch
- [MINOR] Support listener's sockets unix on http logs.
- [MEDIUM] Add supports of bind on unix sockets.
- [BUG] stick table purge failure if size less than 255
- [BUG] stick table entries expire on counters updates/read or show table, even if there is no "expire" parameter
- [MEDIUM] Implement tcp inspect response rules
- [DOC] tcp-response content and inspect
- [MINOR] new acls fetch req_ssl_hello_type and rep_ssl_hello_type
- [DOC] acls rep_ssl_hello and req_ssl_hello
- [MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated.
- [DOC] new type binary in stick-table
- [DOC] stick store-response and new patterns payload and payload_lv
- [MINOR] Manage all types (ip, integer, string, binary) on cli "show table" command
- [MEDIUM] Create updates tree on stick table to manage sync.
- [MAJOR] Add new files src/peer.c, include/proto/peers.h and include/types/peers.h for sync stick table management
- [MEDIUM] Manage peers section parsing and stick table registration on peers.
- [MEDIUM] Manage soft stop on peers proxy
- [DOC] add documentation for peers section
- [MINOR] checks: add support for LDAPv3 health checks
- [MINOR] add better support to "mysql-check"
- [BUG] Restore info about available active/backup servers
- [CONTRIB] Update haproxy.pl
- [CONTRIB] Update Cacti Tempates
- [CONTRIB] add templates for Cacti.
- [BUG] http: don't consider commas as a header delimitor within quotes
- [MINOR] support a global jobs counter
- [DOC] add a summary about cookie incompatibilities between specs and browsers
- [DOC] fix description of cookie "insert" and "indirect" modes
- [MEDIUM] http: fix space handling in the request cookie parser
- [MEDIUM] http: fix space handling in the response cookie parser
- [DOC] fix typo in the queue() definition (backend, not frontend)
- [BUG] deinit: unbind listeners before freeing them
- [BUG] stream_interface: only call si->release when both dirs are closed
- [MEDIUM] buffers: rework the functions to exchange between SI and buffers
- [DOC] fix typo in the avg_queue() and be_conn() definition (backend, not frontend)
- [MINOR] halog: add '-tc' to sort by termination codes
- [MINOR] halog: skip non-traffic logs for -st and -tc
- [BUG] stream_sock: cleanly disable the listener in case of resource shortage
- [BUILD] stream_sock: previous fix lacked the #include, causing a warning.
- [DOC] bind option is "defer-accept", not "defer_accept"
- [DOC] missing index entry for http-check send-state
- [DOC] tcp-request inspect-delay is for backends too
- [BUG] ebtree: string_equal_bits() could return garbage on identical strings
- [BUG] stream_sock: try to flush any extra pending request data after a POST
- [BUILD] proto_http: eliminate some build warnings with gcc-2.95
- [MEDIUM] make it possible to combine http-pretend-keepalived with httpclose
- [MEDIUM] tcp-request : don't wait for inspect-delay to expire when the buffer is full
- [MEDIUM] checks: add support for HTTP contents lookup
- [TESTS] add test-check-expect to test various http-check methods
- [MINOR] global: add "tune.chksize" to change the default check buffer size
- [MINOR] cookie: add options "maxidle" and "maxlife"
- [MEDIUM] cookie: support client cookies with some contents appended to their value
- [MINOR] http: make some room in the transaction flags to extend cookies
- [MINOR] cookie: add the expired (E) and old (O) flags for request cookies
- [MEDIUM] cookie: reassign set-cookie status flags to store more states
- [MINOR] add encode/decode function for 30-bit integers from/to base64
- [MEDIUM] cookie: check for maxidle and maxlife for incoming dated cookies
- [MEDIUM] cookie: set the date in the cookie if needed
- [DOC] document the cookie maxidle and maxlife parameters
- [BUG] checks: don't log backend down for all zero-weight servers
- [MEDIUM] checks: set server state to one state from failure when leaving maintenance
- [BUG] config: report correct keywords for "observe"
- [MINOR] checks: ensure that we can inherit binary checks from the defaults section
- [MINOR] acl: add the http_req_first match
- [DOC] fix typos about bind-process syntax
- [BUG] cookie: correctly unset default cookie parameters
- [MINOR] cookie: add support for the "preserve" option
- [BUG] ebtree: fix duplicate strings insertion
- [CONTRIB] halog: report per-url counts, errors and times
- [CONTRIB] halog: minor speed improvement in timer parser
- [MINOR] buffers: add a new request analyser flag for PROXY mode
- [MINOR] listener: add the "accept-proxy" option to the "bind" keyword
- [MINOR] standard: add read_uint() to parse a delimited unsigned integer
- [MINOR] standard: change arg type from const char* to char*
- [MINOR] frontend: add a new analyser to parse a proxied connection
- [MEDIUM] session: call the frontend_decode_proxy analyser on proxied connections
- [DOC] add the proxy protocol's specifications
- [DOC] document the 'accept-proxy' bind option
- [MINOR] cfgparse: report support of <path> for the 'bind' statements
- [DOC] add references to unix socket handling
- [MINOR] move MAXPATHLEN definition to compat.h
- [MEDIUM] unix sockets: cleanup the error reporting path
- [BUG] session: don't stop forwarding of data upon last packet
- [CLEANUP] accept: replace some inappropriate Alert() calls with send_log()
- [BUILD] peers: shut a printf format warning (key_size is a size_t)
- [BUG] accept: don't close twice upon error
- [OPTIM] session: don't recheck analysers when buffer flags have not changed
- [OPTIM] stream_sock: don't clear FDs that are already cleared
- [BUG] proto_tcp: potential bug on pattern fetch dst and dport
This option makes haproxy preserve any persistence cookie emitted by
the server, which allows the server to change it or to unset it, for
instance, after a logout request.
(cherry picked from commit 52e6d75374c7900c1fe691c5633b4ae029cae8d5)
This match returns true when the request calling it is the first one of
a connection.
(cherry picked from commit 922ca979c50653c415852531f36fe409190ad76b)
The MySQL check has been revamped to be able to send real MySQL data,
and to avoid Aborted connects on MySQL side.
It is however backward compatible with older version, but it is highly
recommended to use the new mode, by adding "user <username>" on the
"mysql-check" line.
The new check consists in sending two MySQL packet, one Client
Authentication packet, with "haproxy" username (by default), and one
QUIT packet, to correctly close MySQL session. We then parse the Mysql
Handshake Initialisation packet and/or Error packet. It is a basic but
useful test which does not produce error nor aborted connect on the
server.
(cherry picked from commit a1e4dcfe5718311b7653d7dabfad65c005d0439b)
- fix some erroneous keywords for http-request
- stats http-request was not documented
--
Cyril Bonté
(cherry picked from commit 17e07dad5074571c38cbeb92d54a20f83658449d)
HTTP content-based health checks will be involved in searching text in pages.
Some pages may not fit in the default buffer (16kB) and sometimes it might be
desired to have larger buffers in order to find patterns. Running checks on
smaller URIs is always preferred of course.
(cherry picked from commit 043f44aeb835f3d0b57626c4276581a73600b6b1)
This patch adds the "http-check expect [r]{string,status}" statements
which enable health checks based on whether the response status or body
to an HTTP request contains a string or matches a regex.
This probably is one of the oldest patches that remained unmerged. Over
the time, several people have contributed to it, among which FinalBSD
(first and second implementations), Nick Chalk (port to 1.4), Anze
Skerlavaj (tests and fixes), Cyril Bonté (general fixes), and of course
myself for the final fixes and doc during integration.
Some people already use an old version of this patch which has several
issues, among which the inability to search for a plain string that is
not at the beginning of the data, and the inability to look for response
contents that are provided in a second and subsequent recv() calls. But
since some configs are already deployed, it was quite important to ensure
a 100% compatible behaviour on the working cases.
Thus, that patch fixes the issues while maintaining config compatibility
with already deployed versions.
(cherry picked from commit b507c43a3ce9a8e8e4b770e52e4edc20cba4c37f)
This patch provides a new "option ldap-check" statement to enable
server health checks based on LDAPv3 bind requests.
(cherry picked from commit b76b44c6fed8a7ba6f0f565dd72a9cb77aaeca7c)
Some configs may involve httpclose in a frontend and http-pretend-keepalive
in a backend. httpclose used to take priority over keepalive, thus voiding
its effect. This change ensures that when both are combined, keepalive is
still announced to the server while close is announced to the client.
(cherry picked from commit 2be7ec90fa9caf66294f446423bbab2d00db9004)
The doc was wrong as the insert mode by default does not insert in
direct requests, and by default transmits the cookies to the server.
This was right in the old doc and it has not changed since the
beginning.
Released version 1.5-dev2 with the following main changes :
- [MINOR] startup: release unused structs after forking
- [MINOR] startup: don't wait for nothing when no old pid remains
- [CLEANUP] reference product branch 1.5
- [MEDIUM] signals: add support for registering functions and tasks
- [MEDIUM] signals: support redistribution of signal zero when stopping
- [BUG] http: don't set auto_close if more data are expected
Released version 1.5-dev1 with the following main changes :
- [BUG] stats: session rate limit gets garbaged in the stats
- [DOC] mention 'option http-server-close' effect in Tq section
- [DOC] summarize and highlight persistent connections behaviour
- [DOC] add configuration samples
- [BUG] http: dispatch and http_proxy modes were broken for a long time
- [BUG] http: the transaction must be initialized even in TCP mode
- [BUG] tcp: dropped connections must be counted as "denied" not "failed"
- [BUG] consistent hash: balance on all servers, not only 2 !
- [CONTRIB] halog: report per-server status codes, errors and response times
- [BUG] http: the transaction must be initialized even in TCP mode (part 2)
- [BUG] client: always ensure to zero rep->analysers
- [BUG] session: clear BF_READ_ATTACHED before next I/O
- [BUG] http: automatically close response if req is aborted
- [BUG] proxy: connection rate limiting was eating lots of CPU
- [BUG] http: report correct flags in case of client aborts during body
- [TESTS] refine non-regression tests and add 4 new tests
- [BUG] debug: wrong pointer was used to report a status line
- [BUG] debug: correctly report truncated messages
- [DOC] document the "dispatch" keyword
- [BUG] stick_table: fix possible memory leak in case of connection error
- [CLEANUP] acl: use 'L6' instead of 'L4' in ACL flags relying on contents
- [MINOR] accept: count the incoming connection earlier
- [CLEANUP] tcp: move some non tcp-specific layer6 processing out of proto_tcp
- [CLEANUP] client: move some ACLs away to their respective locations
- [CLEANUP] rename client -> frontend
- [MEDIUM] separate protocol-level accept() from the frontend's
- [MINOR] proxy: add a list to hold future layer 4 rules
- [MEDIUM] config: parse tcp layer4 rules (tcp-request accept/reject)
- [MEDIUM] tcp: check for pure layer4 rules immediately after accept()
- [OPTIM] frontend: tell the compiler that errors are unlikely to occur
- [MEDIUM] frontend: check for LI_O_TCP_RULES in the listener
- [MINOR] frontend: only check for monitor-net rules if LI_O_CHK_MONNET is set
- [CLEANUP] buffer->cto is not used anymore
- [MEDIUM] session: finish session establishment sequence in with I/O handlers
- [MEDIUM] session: initialize server-side timeouts after connect()
- [MEDIUM] backend: initialize the server stream_interface upon connect()
- [MAJOR] frontend: don't initialize the server-side stream_int anymore
- [MEDIUM] session: move the conn_retries attribute to the stream interface
- [MEDIUM] session: don't assign conn_retries upon accept() anymore
- [MINOR] frontend: rely on the frontend and not the backend for INDEPSTR
- [MAJOR] frontend: reorder the session initialization upon accept
- [MINOR] proxy: add an accept() callback for the application layer
- [MAJOR] frontend: split accept() into frontend_accept() and session_accept()
- [MEDIUM] stats: rely on the standard session_accept() function
- [MINOR] buffer: refine the flags that may wake an analyser up.
- [MINOR] stream_sock: don't dereference a non-existing frontend
- [MINOR] session: differenciate between accepted connections and received connections
- [MEDIUM] frontend: count the incoming connection earlier
- [MINOR] frontend: count denied TCP requests separately
- [CLEANUP] stick_table: add/clarify some comments
- [BUILD] memory: add a few missing parenthesis to the pool management macros
- [MINOR] stick_table: add support for variable-sized data
- [CLEANUP] stick_table: rename some stksess struct members to avoid confusion
- [CLEANUP] stick_table: move pattern to key functions to stick_table.c
- [MEDIUM] stick_table: add room for extra data types
- [MINOR] stick_table: add support for "conn_cum" data type.
- [MEDIUM] stick_table: don't overwrite data when storing an entry
- [MINOR] config: initialize stick tables after all the parsing
- [MINOR] stick_table: provide functions to return stksess data from a type
- [MEDIUM] stick_table: move the server ID to a generic data type
- [MINOR] stick_table: enable it for frontends too
- [MINOR] stick_table: export the stick_table_key
- [MINOR] tcp: add per-source connection rate limiting
- [MEDIUM] stick_table: separate storage and update of session entries
- [MEDIUM] stick-tables: add a reference counter to each entry
- [MINOR] session: add a pointer to the tracked counters for the source
- [CLEANUP] proto_tcp: make the config parser a little bit more flexible
- [BUG] config: report the correct proxy type in tcp-request errors
- [MINOR] config: provide a function to quote args in a more friendly way
- [BUG] stick_table: the fix for the memory leak caused a regression
- [MEDIUM] backend: support servers on 0.0.0.0
- [BUG] stick-table: correctly refresh expiration timers
- [MEDIUM] stream-interface: add a ->release callback
- [MINOR] proxy: add a "parent" member to the structure
- [MEDIUM] session: make it possible to call an I/O handler on both SI
- [MINOR] tools: add a fast div64_32 function
- [MINOR] freq_ctr: add new types and functions for periods different from 1s
- [MINOR] errors: provide new status codes for config parsing functions
- [BUG] http: denied requests must not be counted as denied resps in listeners
- [MINOR] tools: add a get_std_op() function to parse operators
- [MEDIUM] acl: make use of get_std_op() to parse intger ranges
- [MAJOR] stream_sock: better wakeup conditions on read()
- [BUG] session: analysers must be checked when SI state changes
- [MINOR] http: reset analysers to listener's, not frontend's
- [MEDIUM] session: support "tcp-request content" rules in backends
- [BUILD] always match official tags when doing git-tar
- [MAJOR] stream_interface: fix the wakeup conditions for embedded iohandlers
- [MEDIUM] buffer: make buffer_feed* support writing non-contiguous chunks
- [MINOR] tcp: src_count acl does not have a permanent result
- [MAJOR] session: add track-counters to track counters related to the session
- [MINOR] stick-table: provide a table lookup function
- [MINOR] stick-table: use suffix "_cnt" for cumulated counts
- [MEDIUM] session: move counter ACL fetches from proto_tcp
- [MEDIUM] session: add concurrent connections counter
- [MEDIUM] session: add data in and out volume counters
- [MINOR] session: add the trk_conn_cnt ACL keyword to track connection counts
- [MEDIUM] session-counters: automatically update tracked connection count
- [MINOR] session: add the trk_conn_cur ACL keyword to track concurrent connection
- [MINOR] session: add trk_kbytes_* ACL keywords to track data size
- [MEDIUM] session: add a counter on the cumulated number of sessions
- [MINOR] config: support a comma-separated list of store data types in stick-table
- [MEDIUM] stick-tables: add support for arguments to data_types
- [MEDIUM] stick-tables: add stored data argument type checking
- [MEDIUM] session counters: add conn_rate and sess_rate counters
- [MEDIUM] session counters: add bytes_in_rate and bytes_out_rate counters
- [MINOR] stktable: add a stktable_update_key() function
- [MINOR] session-counters: add a general purpose counter (gpc0)
- [MEDIUM] session-counters: add HTTP req/err tracking
- [MEDIUM] stats: add "show table [<name>]" to dump a stick-table
- [MEDIUM] stats: add "clear table <name> key <value>" to clear table entries
- [CLEANUP] stick-table: declare stktable_data_types as extern
- [MEDIUM] stick-table: make use of generic types for stored data
- [MINOR] stats: correctly report errors on "show table" and "clear table"
- [MEDIUM] stats: add the ability to dump table entries matching criteria
- [DOC] configuration: document all the new tracked counters
- [DOC] stats: document "show table" and "clear table"
- [MAJOR] session-counters: split FE and BE track counters
- [MEDIUM] tcp: accept the "track-counters" in "tcp-request content" rules
- [MEDIUM] session counters: automatically remove expired entries.
- [MEDIUM] config: replace 'tcp-request <action>' with "tcp-request connection"
- [MEDIUM] session-counters: make it possible to count connections from frontend
- [MINOR] session-counters: use "track-sc{1,2}" instead of "track-{fe,be}-counters"
- [MEDIUM] session-counters: correctly unbind the counters tracked by the backend
- [CLEANUP] stats: use stksess_kill() to remove table entries
- [DOC] update the references to session counters and to tcp-request connection
- [DOC] cleanup: split a few long lines
- [MEDIUM] http: forward client's close when abortonclose is set
- [BUG] queue: don't dequeue proxy-global requests on disabled servers
- [BUG] stats: global stats timeout may be specified before stats socket.
- [BUG] conf: add tcp-request content rules to the correct list
It began to be problematic to have "tcp-request" followed by an
immediate action, as sometimes it was a keyword indicating a hook
or setting ("content" or "inspect-delay") and sometimes it was an
action.
Now the prefix for connection-level tcp-requests is "tcp-request connection"
and the ones processing contents remain "tcp-request contents".
This has allowed a nice simplification of the config parser and to
clean up the doc a bit. Also now it's a bit more clear why tcp-request
connection are not allowed in backends.
The following statements and ACLs have been added to the doc :
- tcp-request track-counters <key> [table <table>] [{if | unless} <condition>]
- src_*
- trk_*
The "_cnt" suffix is already used by ACLs to count various data,
so it makes sense to use the same one in "conn_cnt" instead of
"conn_cum" to count cumulated connections.
This is not a problem because no version was emitted with those
keywords.
Thus we'll try to stick to the following rules :
xxxx_cnt : cumulated event count for criterion xxxx
xxxx_cur : current number of concurrent entries for criterion xxxx
xxxx_rate: event rate for criterion xxxx
Sometimes it's necessary to be able to perform some "layer 6" analysis
in the backend. TCP request rules were not available till now, although
documented in the diagram. Enable them in backend now.
Till now when a server was configured with address 0.0.0.0, the
connection was forwarded to this address which generally is intercepted
by the system as a local address, so this was completely useless.
One sometimes useful feature for outgoing transparent proxies is to
be able to forward the connection to the same address the client
requested. This patch fixes the meaning of 0.0.0.0 precisely to
ensure that the connection will be forwarded to the initial client's
destination address.
configuration.txt is thorough and accurate but lacked sample configurations
clarifying both the syntax and the relations between global, defaults,
frontend, backend and listen sections. Besides, almost all examples to be found
in haproxy-en.txt or online tutorials make use of the 'listen' syntax while
'frontend/backend' is really the one to know about.
(cherry picked from commit 01ac10ad189b11c563eeb835733fba58e6c5271d)
This change makes use of the stick-tables to keep track of any source
address activity. Two ACLs make it possible to check the count of an
entry or update it and act accordingly. The typical usage will be to
reject a TCP request upon match of an excess value.
This one can be parsed on the "stick-table" after with the "store"
keyword. It will hold the number of connections matching the entry,
for use with ACLs or anything else.
The stick_tables will now be able to store extra data for a same key.
A limited set of extra data types will be defined and for each of them
an offset in the sticky session will be assigned at startup time. All
of this information will be stored in the stick table.
The extra data types will have to be specified after the new "store"
keyword of the "stick-table" directive, which will reserve some space
for them.
It's very disturbing to see the "denied req" counter increase without
any other session counter moving. In fact, we can't count a rejected
TCP connection as "denied req" as we have not yet instanciated any
session at all. Let's use a new counter for that.
Released version 1.4.6 with the following main changes :
- [BUILD] ebtree: update to v6.0.1 to remove references to dprintf()
- [CLEANUP] acl: make use of eb_is_empty() instead of open coding the tree's emptiness test
- [MINOR] acl: add srv_is_up() to check that a specific server is up or not
- [DOC] add a few precisions about the use of RDP cookies
RDP cookies are not necessarily easy to implement because they require
some configuration on the servers. Add a few hints so that people know
what to check on their servers.
