This new sample-fetches captures the cipher list offer by the client
SSL connection during the client-hello phase. This is useful for
fingerprint the SSL connection.
This patch used boringssl's callback to analyse CLientHello before any
handshake to extract key signature capabilities.
Certificat with better signature (ECDSA before RSA) is choosed
transparenty, if client can support it. RSA and ECDSA certificates can
be declare in a row (without order). This makes it possible to set
different ssl and filter parameter with crt-list.
This patch change the names prefixing it by a "_". So "end" becomes "_end".
The backward compatibility with names without the prefix "_" is assured.
In other way, another the keyword "end" can be used like this: Map['end'].
Thanks Robin H. Johnson for the bug repport
This should be backported in version 1.6 and 1.7
crt-list is extend to support ssl configuration. You can now have
such line in crt-list <file>:
mycert.pem [npn h2,http/1.1]
Support include "npn", "alpn", "verify", "ca_file", "crl_file",
"ecdhe", "ciphers" configuration and ssl options.
"crt-base" is also supported to fetch certificates.
[wt: this one is in fact emulated using http-request deny. This
patch can thus be backported to 1.7, 1.6 and 1.5 so that users
of older versions do not add this keyword in their configs]
fc_rcvd_proxy : boolean
Returns true if the client initiated the connection with a PROXY protocol
header.
A flag is added on the struct connection if a PROXY header is successfully
parsed.
The older 'rsprep' directive allows modification of the status reason.
Extend 'http-response set-status' to take an optional string of the new
status reason.
http-response set-status 418 reason "I'm a coffeepot"
Matching updates in Lua code:
- AppletHTTP.set_status
- HTTP.res_set_status
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
It is important to defined analyzers (AN_REQ_* and AN_RES_*) in the same order
they are evaluated in process_stream. This order is really important because
during analyzers evaluation, we run them in the order of the lower bit to the
higher one. This way, when an analyzer adds/removes another one during its
evaluation, we know if it is located before or after it. So, when it adds an
analyzer which is located before it, we can switch to it immediately, even if it
has already been called once but removed since.
With the time, and introduction of new analyzers, this order was broken up. the
main problems come from the filter analyzers. We used values not related with
their evaluation order. Furthermore, we used same values for request and response
analyzers.
So, to fix the bug, filter analyzers have been splitted in 2 distinct lists to
have different analyzers for the request channel than those for the response
channel. And of course, we have moved them to the right place.
Some other analyzers have been reordered to respect the evaluation order:
* AN_REQ_HTTP_TARPIT has been moved just before AN_REQ_SRV_RULES
* AN_REQ_PRST_RDP_COOKIE has been moved just before AN_REQ_STICKING_RULES
* AN_RES_STORE_RULES has been moved just after AN_RES_WAIT_HTTP
Note today we have 29 analyzers, all stored into a 32 bits bitfield. So we can
still add 4 more analyzers before having a problem. A good way to fend off the
problem for a while could be to have a different bitfield for request and
response analyzers.
[wt: all of this must be backported to 1.7, and part of it must be backported
to 1.6 and 1.5]
The parameter "value" of the function TXN.set_var() was not documented.
This is a regression from the commit 85d79c94a9.
This patch must be backported in 1.7
These 2 patches add ability to fetch frontend/backend name in your
logic, so they can be used later to make routing decisions (fe_name) or
taking some actions based on backend which responded to request (be_name).
In our case we needed a fetcher to be able to extract information we
needed from frontend name.
Definitions and examples for 51d.single and 51d.all have been added to
configuration.txt so it now appears in online documentation in addition
to the README, The 51degrees-property-name-list entry has also been
updated to make it clear that multiple properties can be added.
In 1.5-dev20, commit 48bcfda ("MEDIUM: dumpstat: make the CLI parser
understand the backslash as an escape char") introduced support for
backslash on the CLI, but it strips all backslashes in all arguments
instead of only unescaping them, making it impossible to pass a
backslash in an argument.
This will allow us to use a backslash in a command over the socket, eg.
"add acl #0 ABC\\XYZ".
[wt: this should be backported to 1.7 and 1.6]
Released version 1.7.0 with the following main changes :
- SCRIPTS: make publish-release also copy the new SPOE doc
- BUILD: http: include types/sample.h in proto_http.h
- BUILD: debug/flags: remove test for SF_COMP_READY
- CONTRIB: debug/flags: add check for SF_ERR_CHK_PORT
- MINOR: lua: add function which return true if the channel is full.
- MINOR: lua: add ip addresses and network manipulation function
- CONTRIB: tcploop: scriptable TCP I/O for debugging purposes
- CONTRIB: tcploop: implement fork()
- CONTRIB: tcploop: implement logging when called with -v
- CONTRIB: tcploop: update the usage output
- CONTRIB: tcploop: support sending plain strings
- CONTRIB: tcploop: don't report failed send() or recv()
- CONTRIB: tcploop: add basic loops via a jump instruction
- BUG/MEDIUM: channel: bad unlikely macro
- CLEANUP: lua: move comment
- CLEANUP: lua: control executed twice
- BUG/MEDIUM: ssl: Store certificate filename in a variable
- BUG/MINOR: ssl: Print correct filename when error occurs reading OCSP
- CLEANUP: ssl: Remove goto after return dead code
- CLEANUP: ssl: Fix bind keywords name in comments
- DOC: ssl: Use correct wording for ca-sign-pass
- CLEANUP: lua: avoid directly calling getsockname/getpeername()
- BUG/MINOR: stick-table: handle out-of-memory condition gracefully
- MINOR: cli: add private pointer and release function
- MEDIUM: lua: Add cli handler for Lua
- BUG/MEDIUM: connection: check the control layer before stopping polling
- DEBUG: connection: mark the closed FDs with a value that is easier to detect
- BUG/MEDIUM: stick-table: fix regression caused by recent fix for out-of-memory
- BUG/MINOR: cli: properly decrement ref count on tables during failed dumps
- BUG/MEDIUM: lua: In some case, the return of sample-fetche is ignored
- MINOR: filters: Add check_timeouts callback to handle timers expiration on streams
- MINOR: spoe: Add 'timeout processing' option to limit time to process an event
- MINOR: spoe: Remove useless 'timeout ack' option
- MINOR: spoe: Add 'option continue-on-error' statement in spoe-agent section
- MINOR: spoe: Add "maxconnrate" and "maxerrrate" statements
- MINOR: spoe: Add "option set-on-error" statement
- MINOR: stats: correct documentation of process ID for typed output
- BUILD: contrib: fix ip6range build on Centos 7
- BUILD: fix build on Solaris 10/11
- BUG/MINOR: cli: fix pointer size when reporting data/transport layer name
- BUG/MINOR: cli: dequeue from the proxy when changing a maxconn
- BUG/MINOR: cli: wake up the CLI's task after a timeout update
- MINOR: connection: add a few functions to report the data and xprt layers' names
- MINOR: connection: add names for transport and data layers
- REORG: cli: split dumpstats.c in src/cli.c and src/stats.c
- REORG: cli: split dumpstats.h in stats.h and cli.h
- REORG: cli: move ssl CLI functions to ssl_sock.c
- REORG: cli: move map and acl code to map.c
- REORG: cli: move show stat resolvers to dns.c
- MINOR: cli: create new function cli_has_level() to validate permissions
- MINOR: server: create new function cli_find_server() to find a server
- MINOR: proxy: create new function cli_find_frontend() to find a frontend
- REORG: cli: move 'set server' to server.c
- REORG: cli: move 'show pools' to memory.c
- REORG: cli: move 'show servers' to proxy.c
- REORG: cli: move 'show sess' to stream.c
- REORG: cli: move 'show backend' to proxy.c
- REORG: cli: move get/set weight to server.c
- REORG: cli: move "show stat" to stats.c
- REORG: cli: move "show info" to stats.c
- REORG: cli: move dump_text(), dump_text_line(), and dump_binary() to standard.c
- REORG: cli: move table dump/clear/set to stick_table.c
- REORG: cli: move "show errors" out of cli.c
- REORG: cli: make "show env" also use the generic keyword registration
- REORG: cli: move "set timeout" to its own handler
- REORG: cli: move "clear counters" to stats.c
- REORG: cli: move "set maxconn global" to its own handler
- REORG: cli: move "set maxconn server" to server.c
- REORG: cli: move "set maxconn frontend" to proxy.c
- REORG: cli: move "shutdown sessions server" to stream.c
- REORG: cli: move "shutdown session" to stream.c
- REORG: cli: move "shutdown frontend" to proxy.c
- REORG: cli: move "{enable|disable} frontend" to proxy.c
- REORG: cli: move "{enable|disable} server" to server.c
- REORG: cli: move "{enable|disable} health" to server.c
- REORG: cli: move "{enable|disable} agent" to server.c
- REORG: cli: move the "set rate-limit" functions to their own parser
- CLEANUP: cli: rename STAT_CLI_* to CLI_ST_*
- CLEANUP: cli: simplify the request parser a little bit
- CLEANUP: cli: remove assignments to st0 and st2 in keyword parsers
- BUILD: server: remove a build warning introduced by latest series
- BUG/MINOR: log-format: uncatched memory allocation functions
- CLEANUP: log-format: useless file and line in json converter
- CLEANUP/MINOR: log-format: unexport functions parse_logformat_var_args() and parse_logformat_var()
- CLEANUP: log-format: fix return code of the function parse_logformat_var()
- CLEANUP: log-format: fix return code of function parse_logformat_var_args()
- CLEANUP: log-format: remove unused arguments
- MEDIUM: log-format: strict parsing and enable fail
- MEDIUM: log-format/conf: take into account the parse_logformat_string() return code
- BUILD: ssl: make the SSL layer build again with openssl 0.9.8
- BUILD: vars: remove a build warning on vars.c
- MINOR: lua: add utility function for check boolean argument
- MINOR: lua: Add tokenize function.
- BUG/MINOR: conf: calloc untested
- MINOR: http/conf: store the use_backend configuration file and line for logs
- MEDIUM: log-format: Use standard HAProxy log system to report errors
- CLEANUP: sample: report "converter" instead of "conv method" in error messages
- BUG: spoe: Fix parsing of SPOE actions in ACK frames
- MINOR: cli: make "show stat" support a proxy name
- MINOR: cli: make "show errors" support a proxy name
- MINOR: cli: make "show errors" capable of dumping only request or response
- BUG/MINOR: freq-ctr: make swrate_add() support larger values
- CLEANUP: counters: move from 3 types to 2 types
- CLEANUP: cfgparse: cascade the warnif_misplaced_* rules
- REORG: tcp-rules: move tcp rules processing to their own file
- REORG: stkctr: move all the stick counters processing to stick-tables.c
- DOC: update the roadmap file with the latest changes
When dealing with many proxies, it's hard to spot response errors because
all internet-facing frontends constantly receive attacks. This patch now
makes it possible to demand that only request or response errors are dumped
by appending "request" or "reponse" to the show errors command.
For tokenizing a string, standard Lua recommends to use regexes.
The followinf example splits words:
for i in string.gmatch(example, "%S+") do
print(i)
end
This is a little bit overkill for simply split words. This patch
adds a tokenize function which quick and do not use regexes.
It defines the variable to set when an error occurred during an event
processing. It will only be set when an error occurred in the scope of the
transaction. As for all other variables define by the SPOE, it will be
prefixed. So, if your variable name is "error" and your prefix is "my_spoe_pfx",
the variable will be "txn.my_spoe_pfx.error".
When set, the variable is the boolean "true". Note that if "option
continue-on-error" is set, the variable is not automatically removed between
events processing.
"maxconnrate" is the maximum number of connections per second. The SPOE will
stop to open new connections if the maximum is reached and will wait to acquire
an existing one.
"maxerrrate" is the maximum number of errors per second. The SPOE will stop its
processing if the maximum is reached.
These options replace hardcoded macros MAX_NEW_SPOE_APPLETS and
MAX_NEW_SPOE_APPLET_ERRS. We use it to limit SPOE activity, especially when
servers are down..
By default, for a specific stream, when an abnormal/unexpected error occurs, the
SPOE is disabled for all the transaction. So if you have several events
configured, such error on an event will disabled all followings. For TCP
streams, this will disable the SPOE for the whole session. For HTTP streams,
this will disable it for the transaction (request and response).
To bypass this behaviour, you can set 'continue-on-error' option in 'spoe-agent'
section. With this option, only the current event will be ignored.
It is a way to set the maximum time to wait for a stream to process an event,
i.e to acquire a stream to talk with an agent, to encode all messages, to send
the NOTIFY frame, to receive the corrsponding acknowledgement and to process all
actions. It is applied on the stream that handle the client and the server
sessions.
Released version 1.7-dev6 with the following main changes :
- DOC: fix the entry for hash-balance-factor config option
- DOC: Fix typo in description of `-st` parameter in man page
- CLEANUP: cfgparse: Very minor spelling correction
- MINOR: examples: Update haproxy.spec URLs to haproxy.org
- BUG/MEDIUM: peers: on shutdown, wake up the appctx, not the stream
- BUG/MEDIUM: peers: fix use after free in peer_session_create()
- MINOR: peers: make peer_session_forceshutdown() use the appctx and not the stream
- MINOR: peers: remove the pointer to the stream
- BUG/MEDIUM: systemd-wrapper: return correct exit codes
- DOC: stats: provide state details for show servers state
- MEDIUM: tools: make str2ip2() preserve existing ports
- CLEANUP: tools: make ipcpy() preserve the original port
- OPTIM: http: move all http character classs tables into a single one
- OPTIM: http: improve parsing performance of long header lines
- OPTIM: http: improve parsing performance of long URIs
- OPTIM: http: optimize lookup of comma and quote in header values
- BUG/MEDIUM: srv-state: properly restore the DRAIN state
- BUG/MINOR: srv-state: allow to have both CMAINT and FDRAIN flags
- MINOR: server: do not emit warnings/logs/alerts on server state changes at boot
- BUG/MEDIUM: servers: properly propagate the maintenance states during startup
- MEDIUM: wurfl: add Scientiamobile WURFL device detection module
- DOC: move the device detection modules documentation to their own files
- CLEANUP: wurfl: reduce exposure in the rest of the code
- MEDIUM: ssl: Add support for OpenSSL 1.1.0
- MINOR: stream: make option contstats usable again
- MEDIUM: tools: make str2sa_range() return the FQDN even when not resolving
- MINOR: init: move apply_server_state in haproxy.c before MODE_CHECK
- MAJOR: server: postpone address resolution
- MINOR: new srv_admin flag: SRV_ADMF_RMAINT
- MINOR: server: indicate in the logs when RMAINT is cleared
- MINOR: stats: indicate it when a server is down due to resolution
- MINOR: server: make srv_set_admin_state() capable of telling why this happens
- MINOR: dns: implement extra 'hold' timers.
- MAJOR: dns: runtime resolution can change server admin state
- MEDIUM: cli: leave the RMAINT state when setting an IP address on the CLI
- MEDIUM: server: add a new init-addr server line setting
- MEDIUM: server: make use of init-addr
- MINOR: server: implement init-addr none
- MEDIUM: server: make libc resolution failure non-fatal
- MINOR: server: add support for explicit numeric address in init-addr
- DOC: add some documentation for the "init-addr" server keyword
- MINOR: init: add -dr to ignore server address resolution failures
- MEDIUM: server: do not restrict anymore usage of IP address from the state file
- BUG: vars: Fix 'set-var' converter because of a typo
- CLEANUP: remove last references to 'ruleset' section
- MEDIUM: filters: Add attch/detach and stream_set_backend callbacks
- MINOR: filters: Update filters documentation accordingly to recent changes
- MINOR: filters: Call stream_set_backend callbacks before updating backend stats
- MINOR: filters: Remove backend filters attached to a stream only for HTTP streams
- MINOR: flt_trace: Add hexdump option to dump forwarded data
- MINOR: cfgparse: Add functions to backup and restore registered sections
- MINOR: cfgparse: Parse scope lines and save the last one parsed
- REORG: sample: move code to release a sample expression in sample.c
- MINOR: vars: Allow '.' in variable names
- MINOR: vars: Add vars_set_by_name_ifexist function
- MEDIUM: vars: Add a per-process scope for variables
- MINOR: vars: Add 'unset-var' action/converter
- MAJOR: spoe: Add an experimental Stream Processing Offload Engine
- MINOR: spoe: add random ip-reputation service as SPOA example
- MINOR: spoe/checks: Add support for SPOP health checks
- DOC: update ROADMAP file
A new "option spop-check" statement has been added to enable server health
checks based on SPOP HELLO handshake. SPOP is the protocol used by SPOE filters
to talk to servers.
SPOE makes possible the communication with external components to retrieve some
info using an in-house binary protocol, the Stream Processing Offload Protocol
(SPOP). In the long term, its aim is to allow any kind of offloading on the
streams. This first version, besides being experimental, won't do lot of
things. The most important today is to validate the protocol design and lay the
foundations of what will, one day, be a full offload engine for the stream
processing.
So, for now, the SPOE can offload the stream processing before "tcp-request
content", "tcp-response content", "http-request" and "http-response" rules. And
it only supports variables creation/suppression. But, in spite of these limited
features, we can easily imagine to implement a SSO solution, an ip reputation
service or an ip geolocation service.
Internally, the SPOE is implemented as a filter. So, to use it, you must use
following line in a proxy proxy section:
frontend my-front
...
filter spoe [engine <name>] config <file>
...
It uses its own configuration file to keep the HAProxy configuration clean. It
is also a easy way to disable it by commenting out the filter line.
See "doc/SPOE.txt" for all details about the SPOE configuration.
It does the opposite of 'set-var' action/converter. It is really useful for
per-process variables. But, it can be used for any scope.
The lua function 'unset_var' has also been added.
Now it is possible to use variables attached to a process. The scope name is
'proc'. These variables are released only when HAProxy is stopped.
'tune.vars.proc-max-size' directive has been added to confiure the maximum
amount of memory used by "proc" variables. And because memory accounting is
hierachical for variables, memory for "proc" vars includes memory for "sess"
vars.
Now that it is possible to decide whether we prefer to use libc or the
state file to resolve the server's IP address and it is possible to change
a server's IP address at run time on the CLI, let's not restrict the reuse
of the address from the state file anymore to the DNS only.
The impact is that by default the state file will be considered first
(which matches its purpose) and only then the libc. This way any address
change performed at run time over the CLI will be preserved regardless
of DNS usage or not.
It is very common when validating a configuration out of production not to
have access to the same resolvers and to fail on server address resolution,
making it difficult to test a configuration. This option simply appends the
"none" method to the list of address resolution methods for all servers,
ensuring that even if the libc fails to resolve an address, the startup
sequence is not interrupted.
This adds new "hold" timers : nx, refused, timeout, other. This timers
will be used to tell HAProxy to keep an erroneous response as valid for
the corresponding period. For now they're only configured, not enforced.
The server's state is now "MAINT (resolution)" just like we also have
"MAINT (via x/y)" when servers are tracked. The HTML stats page reports
"resolution" in the checks field similarly to what is done for the "via"
entry.
This flag has to be set when an IP address resolution fails (either
using libc at start up or using HAProxy's runtime resolver). This will
automatically trigger the administrative status "MAINT", through the
global mask SRV_ADMF_MAINT.
Quite a lot of people have been complaining about option contstats not
working correctly anymore since about 1.4. The reason was that one reason
for the significant performance boost between 1.3 and 1.4 was the ability
to forward data between a server and a client without waking up the stream
manager. And we couldn't afford to force sessions to constantly wake it
up given that most of the people interested in contstats are also those
interested in high performance transmission.
An idea was experimented with in the past, consisting in limiting the
amount of transmissible data before waking it up, but it was not usable
on slow connections (eg: FTP over modem lines, RDP, SSH) as stats would
be updated too rarely if at all, so that idea was dropped.
During a discussion today another idea came up : ensure that stats are
updated once in a while, since it's the only thing that matters. It
happens that we have the request channel's analyse_exp timeout that is
used to wake the stream up after a configured delay, and that by
definition this timeout is not used when there's no more analyser
(otherwise the stream would wake up and the stats would be updated).
Thus here the idea is to reuse this timeout when there's no analyser
and set it to now+5 seconds so that a stream wakes up at least once
every 5 seconds to update its stats. It should be short enough to
provide smooth traffic graphs and to allow to debug outputs of "show
sess" more easily without inflicting too much load even for very large
number of concurrent connections.
This patch is simple enough and safe enough to be backportable to 1.6
if there is some demand.
Both DeviceAtlas and 51Degrees used to put their building instructions
in the README, representing more than 1/3 of it. It's better to let the
README focus on generic stuff and building procedure and move the DD
docs to their own files.
WURFL is a high-performance and low-memory footprint mobile device
detection software component that can quickly and accurately detect
over 500 capabilities of visiting devices. It can differentiate between
portable mobile devices, desktop devices, SmartTVs and any other types
of devices on which a web browser can be installed.
In order to add WURFL device detection support, you would need to
download Scientiamobile InFuze C API and install it on your system.
Refer to www.scientiamobile.com to obtain a valid InFuze license.
Any useful information on how to configure HAProxy working with WURFL
may be found in:
doc/WURFL-device-detection.txt
doc/configuration.txt
examples/wurfl-example.cfg
Please find more information about WURFL device detection API detection
at https://docs.scientiamobile.com/documentation/infuze/infuze-c-api-user-guide
Released version 1.7-dev5 with the following main changes :
- MINOR: cfgparse: few memory leaks fixes.
- MEDIUM: log: Decompose %Tq in %Th %Ti %TR
- CLEANUP: logs: remove unused log format field definitions
- BUILD/MAJOR:updated 51d Trie implementation to incorperate latest update to 51Degrees.c
- BUG/MAJOR: stream: properly mark the server address as unset on connect retry
- CLEANUP: proto_http: Removing useless variable assignation
- CLEANUP: dumpstats: Removing useless variables allocation
- CLEANUP: dns: Removing usless variable & assignation
- BUG/MINOR: payload: fix SSLv2 version parser
- MINOR: cli: allow the semi-colon to be escaped on the CLI
- MINOR: cli: change a server health check port through the stats socket
- BUG/MINOR: Fix OSX compilation errors
- MAJOR: check: find out which port to use for health check at run time
- MINOR: server: introduction of 3 new server flags
- MINOR: new update_server_addr_port() function to change both server's ADDR and service PORT
- MINOR: cli: ability to change a server's port
- CLEANUP/MINOR dns: comment do not follow up code update
- MINOR: chunk: new strncat function
- MINOR: dns: wrong DNS_MAX_UDP_MESSAGE value
- MINOR: dns: new MAX values
- MINOR: dns: new macro to compute DNS header size
- MINOR: dns: new DNS structures to store received packets
- MEDIUM: dns: new DNS response parser
- MINOR: dns: query type change when last record is a CNAME
- MINOR: dns: proper domain name validation when receiving DNS response
- MINOR: dns: comments in types/dns.h about structures endianness
- BUG/MINOR: displayed PCRE version is running release
- MINOR: show Built with PCRE version
- MINOR: show Running on zlib version
- MEDIUM: make SO_REUSEPORT configurable
- MINOR: enable IP_BIND_ADDRESS_NO_PORT on backend connections
- BUG/MEDIUM: http/compression: Fix how chunked data are copied during the HTTP body parsing
- BUG/MINOR: stats: report the correct conn_time in backend's html output
- BUG/MEDIUM: dns: don't randomly crash on out-of-memory
- MINOR: Add fe_req_rate sample fetch
- MEDIUM: peers: Fix a peer stick-tables synchronization issue.
- MEDIUM: cli: register CLI keywords with cli_register_kw()
- BUILD: Make use of accept4() on OpenBSD.
- MINOR: tcp: make set-src/set-src-port and set-dst/set-dst-port commutative
- DOC: fix missed entry for "set-{src,dst}{,-port}"
- BUG/MINOR: vars: use sess and not s->sess in action_store()
- BUG/MINOR: vars: make smp_fetch_var() more robust against misuses
- BUG/MINOR: vars: smp_fetch_var() doesn't depend on HTTP but on the session
- MINOR: stats: output dcon
- CLEANUP: tcp rules: mention everywhere that tcp-conn rules are L4
- MINOR: counters: add new fields for denied_sess
- MEDIUM: tcp: add registration and processing of TCP L5 rules
- MINOR: stats: emit dses
- DOC: document tcp-request session
- MINOR: ssl: add debug traces
- BUILD/CLEANUP: ssl: Check BIO_reset() return code
- BUG/MINOR: ssl: Check malloc return code
- BUG/MINOR: ssl: prevent multiple entries for the same certificate
- BUG/MINOR: systemd: make the wrapper return a non-null status code on error
- BUG/MINOR: systemd: always restore signals before execve()
- BUG/MINOR: systemd: check return value of calloc()
- MINOR: systemd: report it when execve() fails
- BUG/MEDIUM: systemd: let the wrapper know that haproxy has completed or failed
- MINOR: proxy: add 'served' field to proxy, equal to total of all servers'
- MINOR: backend: add hash-balance-factor option for hash-type consistent
- MINOR: server: compute a "cumulative weight" to allow chash balancing to hit its target
- MEDIUM: server: Implement bounded-load hash algorithm
- SCRIPTS: make git-show-backports also dump a "git show" command
- MINOR: build: Allow linking to device-atlas library file
- MINOR: stats: Escape equals sign on socket dump
0 will mean no balancing occurs; otherwise it represents the ratio
between the highest-loaded server and the average load, times 100 (i.e.
a value of 150 means a 1.5x ratio), assuming equal weights.
Signed-off-by: Andrew Rodland <andrewr@vimeo.com>
These are denied conns. Strangely this wasn't emitted while it used to be
available for a while. It corresponds to the number of connections blocked
by "tcp-request connection reject".
When the tcp/http actions above were introduced in 1.7-dev4, we used to
proceed like this :
- set-src/set-dst would force the port to zero
- set-src-port/set-dst-port would not do anything if the address family is
neither AF_INET nor AF_INET6.
It was a stupid idea of mine to request this behaviour because it ensures
that these functions cannot be used in a wide number of situations. Because
of the first rule, it is necessary to save the source port one way or
another if only the address has to be changed (so you have to use an
variable). Due to the second rule, there's no way to set the source port
on a unix socket without first overwriting the address. And sometimes it's
really not convenient, especially when there's no way to guarantee that all
fields will properly be set.
In order to fix all this, this small change does the following :
- set-src/set-dst always preserve the original port even if the address
family changes. If the previous address family didn't have a port (eg:
AF_UNIX), then the port is set to zero ;
- set-src-port/set-dst-port always preserve the original address. If the
address doesn't have a port, then the family is forced to IPv4 and the
address to "0.0.0.0".
Thanks to this it now becomes possible to perform one action, the other or
both in any order.
The fe_req_rate is similar to fe_sess_rate, but fetches the number
of HTTP requests per second instead of connections/sessions per second.
Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
Enable IP_BIND_ADDRESS_NO_PORT on backend connections when the source
address is specified without port or port ranges. This is supported
since Linux 4.2/libc 2.23.
If the kernel supports it but the libc doesn't, we can define it at
build time:
make [...] DEFINE=-DIP_BIND_ADDRESS_NO_PORT=24
For more informations about this feature, see Linux commit 90c337da
With Linux officially introducing SO_REUSEPORT support in 3.9 and
its mainstream adoption we have seen more people running into strange
SO_REUSEPORT related issues (a process management issue turning into
hard to diagnose problems because the kernel load-balances between the
new and an obsolete haproxy instance).
Also some people simply want the guarantee that the bind fails when
the old process is still bound.
This change makes SO_REUSEPORT configurable, introducing the command
line argument "-dR" and the noreuseport configuration directive.
A backport to 1.6 should be considered.
Enrichment of the 'set server <b>/<s> addr' cli directive to allow changing
now a server's port.
The new syntax looks like:
set server <b>/<s> addr [port <port>]
Introduction of a new CLI command "set server <srv> check-port <port>' to
allow admins to change a server's health check port at run time.
This changes the equivalent of the configuration server parameter
called 'port'.
Today I was working on an auto-update script for some ACLs, and found
that I couldn't load ACL entries with a semi-colon in them no matter
how I tried to escape it.
As such, I wrote this patch (this one is for 1.7dev, but it applies to
1.5 the same with just line numbers changed), which seems to allow me
to execute a command such as "add acl /etc/foo.lst foo\;bar" over the
socket. It's worth noting that stats_sock_parse_request() already uses
the backslash to escape spaces in words so it makes sense to use it as
well to escape the semi-colon.
Tq is the time between the instant the connection is accepted and a
complete valid request is received. This time includes the handshake
(SSL / Proxy-Protocol), the idle when the browser does preconnect and
the request reception.
This patch decomposes %Tq in 3 measurements names %Th, %Ti, and %TR
which returns respectively the handshake time, the idle time and the
duration of valid request reception. It also adds %Ta which reports
the request's active time, which is the total time without %Th nor %Ti.
It replaces %Tt as the total time, reporting accurate measurements for
HTTP persistent connections.
%Th is avalaible for TCP and HTTP sessions, %Ti, %TR and %Ta are only
avalaible for HTTP connections.
In addition to this, we have new timestamps %tr, %trg and %trl, which
log the date of start of receipt of the request, respectively in the
default format, in GMT time and in local time (by analogy with %t, %T
and %Tl). All of them are obviously only available for HTTP. These values
are more relevant as they more accurately represent the request date
without being skewed by a browser's preconnect nor a keep-alive idle
time.
The HTTP log format and the CLF log format have been modified to
use %tr, %TR, and %Ta respectively instead of %t, %Tq and %Tt. This
way the default log formats now produce the expected output for users
who don't want to manually fiddle with the log-format directive.
Example with the following log-format :
log-format "%ci:%cp [%tr] %ft %b/%s h=%Th/i=%Ti/R=%TR/w=%Tw/c=%Tc/r=%Tr/a=%Ta/t=%Tt %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
The request was sent by hand using "openssl s_client -connect" :
Aug 23 14:43:20 haproxy[25446]: 127.0.0.1:45636 [23/Aug/2016:14:43:20.221] test~ test/test h=6/i=2375/R=261/w=0/c=1/r=0/a=262/t=2643 200 145 - - ---- 1/1/0/0/0 0/0 "GET / HTTP/1.1"
=> 6 ms of SSL handshake, 2375 waiting before sending the first char (in
fact the time to type the first line), 261 ms before the end of the request,
no time spent in queue, 1 ms spend connecting to the server, immediate
response, total active time for this request = 262ms. Total time from accept
to close : 2643 ms.
The timing now decomposes like this :
first request 2nd request
|<-------------------------------->|<-------------- ...
t tr t tr ...
---|----|----|----|----|----|----|----|----|--
: Th Ti TR Tw Tc Tr Td : Ti ...
:<---- Tq ---->: :
:<-------------- Tt -------------->:
:<--------- Ta --------->:
Released version 1.7-dev4 with the following main changes :
- MINOR: add list_append_word function
- MEDIUM: init: use list_append_word in haproxy.c
- MEDIUM: init: allow directory as argument of -f
- CLEANUP: config: detect double registration of a config section
- MINOR: log: add the %Td log-format specifier
- MEDIUM: filters: Move HTTP headers filtering in its own callback
- MINOR: filters: Simplify calls to analyzers using 2 new macros
- MEDIUM: filters: Add pre and post analyzer callbacks
- DOC: filters: Update the filters documentation accordingly to recent changes
- BUG/MEDIUM: init: don't use environment locale
- SCRIPTS: teach git-show-backports how to report upstream commits
- SCRIPTS: make git-show-backports capable of limiting its history
- BUG/MAJOR: fix listening IP address storage for frontends
- BUG/MINOR: fix listening IP address storage for frontends (cont)
- DOC: Fix typo so fetch is properly parsed by Cyril's converter
- BUG/MAJOR: http: fix breakage of "reqdeny" causing random crashes
- BUG/MEDIUM: stick-tables: fix breakage in table converters
- MINOR: stick-table: change all stick-table converters' inputs to SMP_T_ANY
- BUG/MEDIUM: dns: unbreak DNS resolver after header fix
- BUILD: fix build on Solaris 11
- BUG/MEDIUM: config: fix multiple declaration of section parsers
- BUG/MEDIUM: stats: show servers state may show an servers from another backend
- BUG/MEDIUM: fix risk of segfault with "show tls-keys"
- MEDIUM: dumpstats: 'show tls-keys' is now able to show secrets
- DOC: update doc about tls-tickets-keys dump
- MEDIUM: tcp: add 'set-src' to 'tcp-request connection'
- MINOR: set the CO_FL_ADDR_FROM_SET flags with 'set-src'
- MEDIUM: tcp/http: add 'set-src-port' action
- MEDIUM: tcp/http: new set-dst/set-dst-port actions
- BUG/MEDIUM: sticktables: segfault in some configuration error cases
- BUILD/MEDIUM: rebuild everything when an include file is changed
- BUILD/MEDIUM: force a full rebuild if some build options change
- BUG/MEDIUM: lua: converters doesn't work
- BUG/MINOR: http: add-header: header name copied twice
- BUG/MEDIUM: http: add-header: buffer overwritten
- BUG/MINOR: ssl: fix potential memory leak in ssl_sock_load_dh_params()
- MINOR: stream: export the function 'smp_create_src_stkctr'
- BUG/MEDIUM: dumpstats: undefined behavior in stats_tlskeys_list()
- MEDIUM: dumpstats: make stats_tlskeys_list() yield-aware during tls-keys dump
- BUG/MINOR: http: url32+src should use the big endian version of url32
- BUG/MINOR: http: url32+src should check cli_conn before using it
- DOC: http: add documentation for url32 and url32+src
- BUG/MINOR: fix http-response set-log-level parsing error
- MINOR: systemd: Use variable for config and pidfile paths
- MINOR: systemd: Perform sanity check on config before reload
- MEDIUM: ssl: support SNI filters with multicerts
- MINOR: ssl: crt-list parsing factor
- BUILD: ssl: fix typo causing a build failure in the multicert patch
- MINOR: listener: add the "accept-netscaler-cip" option to the "bind" keyword
- MINOR: tcp: add "tcp-request connection expect-netscaler-cip layer4"
- BUG/MINOR: init: always ensure that global.rlimit_nofile matches actual limits
- BUG/MINOR: init: ensure that FD limit is raised to the max allowed
- BUG/MEDIUM: external-checks: close all FDs right after the fork()
- BUG/MAJOR: external-checks: use asynchronous signal delivery
- BUG/MINOR: external-checks: do not unblock undesired signals
- CLEANUP: external-check: don't block/unblock SIGCHLD when manipulating the list
- BUG/MEDIUM: filters: Fix data filtering when data are modified
- BUG/MINOR: filters: Fix HTTP parsing when a filter loops on data forwarding
- BUG/MINOR: srv-state: fix incorrect output of state file
- BUG/MINOR: ssl: close ssl key file on error
- BUG/MINOR: http: fix misleading error message for response captures
- BUG/BUILD: don't automatically run "make" on "make install"
- DOC: add missing doc for http-request deny [deny_status <status>]
- CLEANUP: dumpstats: u64 field is an unsigned type.
- BUG/MEDIUM: http: unbreak uri/header/url_param hashing
- BUG/MINOR: Rework slightly commit 9962f8fc to clean code and avoid mistakes
- MINOR: new function my_realloc2 = realloc + free upon failure
- CLEANUP: fixed some usages of realloc leading to memory leak
- Revert "BUG/MINOR: ssl: fix potential memory leak in ssl_sock_load_dh_params()"
- CLEANUP: connection: using internal struct to hold source and dest port.
- DOC: spelling fixes
- BUG/MINOR: ssl: fix potential memory leak in ssl_sock_load_dh_params()
- BUG/MEDIUM: dns: fix alignment issues in the DNS response parser
- BUG/MINOR: Fix endiness issue in DNS header creation code
- BUG/MEDIUM: lua: the function txn_done() from sample fetches can crash
- BUG/MEDIUM: lua: the function txn_done() from action wrapper can crash
- MEDIUM: http: implement http-response track-sc* directive
- BUG/MINOR: peers: Fix peers data decoding issue
- BUG/MINOR: peers: don't count track-sc multiple times on errors
- MINOR: standard: add function "escape_string"
- BUG/MEDIUM: log: use function "escape_string" instead of "escape_chunk"
- MINOR: tcp: Return TCP statistics like RTT and RTT variance
- DOC: lua: remove old functions
- BUG/MEDIUM: lua: somme HTTP manipulation functions are called without valid requests
- DOC: fix json converter example and error message
- BUG/MEDIUM: stream-int: completely detach connection on connect error
- DOC: minor typo fixes to improve HTML parsing by haproxy-dconv
- BUILD: make proto_tcp.c compatible with musl library
- BUG/MAJOR: compression: initialize avail_in/next_in even during flush
- BUG/MEDIUM: samples: make smp_dup() always duplicate the sample
- MINOR: sample: implement smp_is_safe() and smp_make_safe()
- MINOR: sample: provide smp_is_rw() and smp_make_rw()
- BUG/MAJOR: server: the "sni" directive could randomly cause trouble
- BUG/MEDIUM: stick-tables: do not fail on string keys with no allocated size
- BUG/MEDIUM: stick-table: properly convert binary samples to keys
- MINOR: sample: use smp_make_rw() in upper/lower converters
- MINOR: tcp: add dst_is_local and src_is_local
- BUG/MINOR: peers: some updates are pushed twice after a resync.
- BUILD: protocol: fix some build errors on OpenBSD
- BUILD: log: iovec requires to include sys/uio.h on OpenBSD
- BUILD: tcp: do not include netinet/ip.h for IP_TTL
- BUILD: connection: fix build breakage on openbsd due to missing in_systm.h
- BUILD: checks: remove the last strcat and eliminate a warning on OpenBSD
- BUILD: tcp: define SOL_TCP when only IPPROTO_TCP exists
- BUILD: compression: remove a warning when no compression lib is used
- BUILD: poll: remove unused hap_fd_isset() which causes a warning with clang
- MINOR: tcp: add further tcp info fetchers
- BUG/MINOR: peers: empty chunks after a resync.
- BUG/MAJOR: stick-counters: possible crash when using sc_trackers with wrong table
- MINOR: standard.c: ipcmp() function to compare 2 IP addresses stored in 2 struct sockaddr_storage
- MINOR: standard.c: ipcpy() function to copy an IP address from a struct sockaddr_storage into an other one
- MAJOR: listen section: don't use first bind port anymore when no server ports are provided
Adding on to Thierry's work (http://git.haproxy.org/?p=haproxy.git;h=6310bef5)
I have added a few more fetchers for counters based on the tcp_info struct
maintained by the kernel :
fc_unacked, fc_sacked, fc_retrans, fc_fackets, fc_lost,
fc_reordering
Two fields were not added because they're version-dependant :
fc_rcv_rtt, fc_total_retrans
The fields name depend on the operating system. FreeBSD and NetBSD prefix
all the field names with "__" so we have to rely on a few #ifdef for
portability.
It is sometimes needed in application server environments to easily tell
if a source is local to the machine or a remote one, without necessarily
knowing all the local addresses (dhcp, vrrp, etc). Similarly in transparent
proxy configurations it is sometimes desired to tell the difference between
local and remote destination addresses.
This patch adds two new sample fetch functions for this :
dst_is_local : boolean
Returns true if the destination address of the incoming connection is local
to the system, or false if the address doesn't exist on the system, meaning
that it was intercepted in transparent mode. It can be useful to apply
certain rules by default to forwarded traffic and other rules to the traffic
targetting the real address of the machine. For example the stats page could
be delivered only on this address, or SSH access could be locally redirected.
Please note that the check involves a few system calls, so it's better to do
it only once per connection.
src_is_local : boolean
Returns true if the source address of the incoming connection is local to the
system, or false if the address doesn't exist on the system, meaning that it
comes from a remote machine. Note that UNIX addresses are considered local.
It can be useful to apply certain access restrictions based on where the
client comes from (eg: require auth or https for remote machines). Please
note that the check involves a few system calls, so it's better to do it only
once per connection.
The functions "req_replace_value()" and "res_replace_value()"
doesn't exists in the 1.6 version. There inherited from the 1.6dev.
This patch must be backported in 1.6 version
This patch adds 4 new sample fetches which returns the RTT of the
established connexion and the RTT variance. The established connection
can be between the client and HAProxy, and between HAProxy and the
server. This is very useful for statistics. A great use case is the
estimation of the TCP connection time of the client. Note that the
RTT of the server side is not so interesting because we already have
the connect() time.
This enables tracking of sticky counters from current response. The only
difference from "http-request track-sc" is the <key> sample expression
can only make use of samples in response (eg. res.*, status etc.) and
samples below Layer 6.
The function txn_done() ends a transaction. It does not make
sense to call this function from a lua sample-fetch wrapper,
because the role of a sample-fetch is not to terminate a
transaction.
This patch modify the role of the fucntion txn_done() if it
is called from a sample-fetch wrapper, now it just ends the
execution of the Lua code like the done() function.
Must be backported in 1.6
The feature was introduced in 1.6-dev2 by commit 108b1dd ("MEDIUM:
http: configurable http result codes for http-request deny") but the
doc was missing. Thanks to Cyril for noticing.
This must be backported into 1.6.
This configures the client-facing connection to receive a NetScaler
Client IP insertion protocol header before any byte is read from the
socket. This is equivalent to having the "accept-netscaler-cip" keyword
on the "bind" line, except that using the TCP rule allows the PROXY
protocol to be accepted only for certain IP address ranges using an ACL.
This is convenient when multiple layers of load balancers are passed
through by traffic coming from public hosts.
When NetScaler application switch is used as L3+ switch, informations
regarding the original IP and TCP headers are lost as a new TCP
connection is created between the NetScaler and the backend server.
NetScaler provides a feature to insert in the TCP data the original data
that can then be consumed by the backend server.
Specifications and documentations from NetScaler:
https://support.citrix.com/article/CTX205670https://www.citrix.com/blogs/2016/04/25/how-to-enable-client-ip-in-tcpip-option-of-netscaler/
When CIP is enabled on the NetScaler, then a TCP packet is inserted just after
the TCP handshake. This is composed as:
- CIP magic number : 4 bytes
Both sender and receiver have to agree on a magic number so that
they both handle the incoming data as a NetScaler Client IP insertion
packet.
- Header length : 4 bytes
Defines the length on the remaining data.
- IP header : >= 20 bytes if IPv4, 40 bytes if IPv6
Contains the header of the last IP packet sent by the client during TCP
handshake.
- TCP header : >= 20 bytes
Contains the header of the last TCP packet sent by the client during TCP
handshake.
SNI filters used to be ignored with multicerts (eg: those providing
ECDSA and RSA at the same time). This patch makes them work like
other certs.
Note: most of the changes in this patch are due to an extra level of
indent, read it with "git show -b".
The 'set-src' action was not available for tcp actions The action code
has been converted into a function in proto_tcp.c to be used for both
'http-request' and 'tcp-request connection' actions.
Both http and tcp keywords are registered in proto_tcp.c
This patch removes setlocale from the main function. It was introduced
by commit 379d9c7 ("MEDIUM: init: allow directory as argument of -f")
in 1.7-dev a few commits ago after a discussion on the mailing list.
Some regex may have different behaviours depending on the
locale. Some LUA scripts may change their behaviour too
(http://lua-users.org/wiki/LuaLocales).
Without this patch (haproxy is using setlocale) :
$ cat locale.cfg
defaults
mode http
frontend test
bind :9000
mode http
use_backend testbk if { hdr_reg(X-Test) ^\w+$ }
backend testbk
mode http
server s 127.0.0.1:80
$ LANG=fr_FR.UTF-8 ./haproxy -f locale.cfg
$ curl -i -H "X-Test: échec" localhost:9000
HTTP/1.1 200 OK
...
$ LANG=C ./haproxy -f locale.cfg
$ curl -i -H "X-Test: échec" localhost:9000
HTTP/1.0 503 Service Unavailable
...
As suggested by Pavlos, it's too bad that we didn't have a %Td log
format tag given that there are a few mentions of Td corresponding
to the data transmission time already in the doc, so this is now done.
Just like the other specifiers, we report -1 if the connection failed
before reaching the data transmission state.
If -f argument is a directory add all the files (and only files) it
containes to the config files list.
These files are added in lexical order (respecting LC_COLLATE).
Only files with ".cfg" extension are added.
Only non hidden files (not prefixed with ".") are added.
Symlink are followed.
The -f order is still respected:
$ tree -a rootdir
rootdir
|-- dir1
| |-- .6.cfg
| |-- 1.cfg
| |-- 2
| |-- 3.cfg
| |-- 4.cfg -> 1.cfg
| |-- 5 -> 1.cfg
| |-- 7.cfg -> .
| `-- dir4
| `-- 8.cfg
|-- dir2
| |-- 10.cfg
| `-- 9.cfg
|-- dir3
| `-- 11.cfg
|-- link -> dir3/
|-- root1
|-- root2
`-- root3
$ ./haproxy -C rootdir -f root2 -f dir2 -f root3 -f dir1 \
-f link -f root1
root2
dir2/10.cfg
dir2/9.cfg
root3
dir1/1.cfg
dir1/3.cfg
dir1/4.cfg
link/11.cfg
root1
This can be useful on systemd where you can't change the haproxy
commande line options on service reload.
Released version 1.7-dev3 with the following main changes :
- MINOR: sample: Moves ARGS underlying type from 32 to 64 bits.
- BUG/MINOR: log: Don't use strftime() which can clobber timezone if chrooted
- BUILD: namespaces: fix a potential build warning in namespaces.c
- MINOR: da: Using ARG12 macro for the sample fetch and the convertor.
- DOC: add encoding to json converter example
- BUG/MINOR: conf: "listener id" expects integer, but its not checked
- DOC: Clarify tunes.vars.xxx-max-size settings
- CLEANUP: chunk: adding NULL check to chunk_dup allocation.
- CLEANUP: connection: fix double negation on memcmp()
- BUG/MEDIUM: peers: fix incorrect age in frequency counters
- BUG/MEDIUM: Fix RFC5077 resumption when more than TLS_TICKETS_NO are present
- BUG/MAJOR: Fix crash in http_get_fhdr with exactly MAX_HDR_HISTORY headers
- BUG/MINOR: lua: can't load external libraries
- BUG/MINOR: prevent the dump of uninitialized vars
- CLEANUP: map: it seems that the map were planed to be chained
- MINOR: lua: move class registration facilities
- MINOR: lua: remove some useless checks
- CLEANUP: lua: Remove two same functions
- MINOR: lua: refactor the Lua object registration
- MINOR: lua: precise message when a critical error is catched
- MINOR: lua: post initialization
- MINOR: lua: Add internal function which strip spaces
- MINOR: lua: convert field to lua type
- DOC: "addr" parameter applies to both health and agent checks
- DOC: timeout client: pointers to timeout http-request
- DOC: typo on stick-store response
- DOC: stick-table: amend paragraph blaming the loss of table upon reload
- DOC: typo: ACL subdir match
- DOC: typo: maxconn paragraph is wrong due to a wrong buffer size
- DOC: regsub: parser limitation about the inability to use closing square brackets
- DOC: typo: req.uri is now replaced by capture.req.uri
- DOC: name set-gpt0 mismatch with the expected keyword
- MINOR: http: sample fetch which returns unique-id
- MINOR: dumpstats: extract stats fields enum and names
- MINOR: dumpstats: split stats_dump_info_to_buffer() in two parts
- MINOR: dumpstats: split stats_dump_fe_stats() in two parts
- MINOR: dumpstats: split stats_dump_li_stats() in two parts
- MINOR: dumpstats: split stats_dump_sv_stats() in two parts
- MINOR: dumpstats: split stats_dump_be_stats() in two parts
- MINOR: lua: dump general info
- MINOR: lua: add class proxy
- MINOR: lua: add class server
- MINOR: lua: add class listener
- BUG/MEDIUM: stick-tables: some sample-fetch doesn't work in the connection state.
- MEDIUM: proxy: use dynamic allocation for error dumps
- CLEANUP: remove unneeded casts
- CLEANUP: uniformize last argument of malloc/calloc
- DOC: fix "needed" typo
- BUG/MINOR: dumpstats: fix write to global chunk
- BUG/MINOR: dns: inapropriate way out after a resolution timeout
- BUG/MINOR: dns: trigger a DNS query type change on resolution timeout
- CLEANUP: proto_http: few corrections for gcc warnings.
- BUG/MINOR: DNS: resolution structure change
- BUG/MINOR : allow to log cookie for tarpit and denied request
- BUG/MEDIUM: ssl: rewind the BIO when reading certificates
- OPTIM/MINOR: session: abort if possible before connecting to the backend
- DOC: http: rename the unique-id sample and add the documentation
- BUG/MEDIUM: trace.c: rdtsc() is defined in two files
- BUG/MEDIUM: channel: fix miscalculation of available buffer space (2nd try)
- BUG/MINOR: server: risk of over reading the pref_net array.
- BUG/MINOR: cfgparse: couple of small memory leaks.
- BUG/MEDIUM: sample: initialize the pointer before parse_binary call.
- DOC: fix discrepancy in the example for http-request redirect
- MINOR: acl: Add predefined METH_DELETE, METH_PUT
- CLEANUP: .gitignore cleanup
- DOC: Clarify IPv4 address / mask notation rules
- CLEANUP: fix inconsistency between fd->iocb, proto->accept and accept()
- BUG/MEDIUM: fix maxaccept computation on per-process listeners
- BUG/MINOR: listener: stop unbound listeners on startup
- BUG/MINOR: fix maxaccept computation according to the frontend process range
- TESTS: add blocksig.c to run tests with all signals blocked
- MEDIUM: unblock signals on startup.
- MINOR: filters: Print the list of existing filters during HA startup
- MINOR: filters: Typo in an error message
- MINOR: filters: Filters must define the callbacks struct during config parsing
- DOC: filters: Add filters documentation
- BUG/MEDIUM: channel: don't allow to overwrite the reserve until connected
- BUG/MEDIUM: channel: incorrect polling condition may delay event delivery
- BUG/MEDIUM: channel: fix miscalculation of available buffer space (3rd try)
- BUG/MEDIUM: log: fix risk of segfault when logging HTTP fields in TCP mode
- MINOR: Add ability for agent-check to set server maxconn
- CLEANUP: Use server_parse_maxconn_change_request for maxconn CLI updates
- MINOR: filters: add opaque data
- BUG/MEDIUM: lua: protects the upper boundary of the argument list for converters/fetches.
- MINOR: lua: migrate the argument mask to 64 bits type.
- BUG/MINOR: dumpstats: Fix the "Total bytes saved" counter in backends stats
- BUG/MINOR: log: fix a typo that would cause %HP to log <BADREQ>
- BUG/MEDIUM: http: fix incorrect reporting of server errors
- MINOR: channel: add new function channel_congested()
- BUG/MEDIUM: http: fix risk of CPU spikes with pipelined requests from dead client
- BUG/MAJOR: channel: fix miscalculation of available buffer space (4th try)
- BUG/MEDIUM: stream: ensure the SI_FL_DONT_WAKE flag is properly cleared
- BUG/MEDIUM: channel: fix inconsistent handling of 4GB-1 transfers
- BUG/MEDIUM: stats: show servers state may show an empty or incomplete result
- BUG/MEDIUM: stats: show backend may show an empty or incomplete result
- MINOR: stats: fix typo in help messages
- MINOR: stats: show stat resolvers missing in the help message
- BUG/MINOR: dns: fix DNS header definition
- BUG/MEDIUM: dns: fix alignment issue when building DNS queries
- CLEANUP: don't ignore scripts in .gitignore
- BUILD: add a few release and backport scripts in scripts/
This is very useful in complex architecture systems where HAproxy
is balancing DB connections for example. We want to keep the maxconn
high in order to avoid issues with queueing on the LB level when
there is slowness on another part of the system. Example is a case of
an architecture where each thread opens multiple DB connections, which
if get stuck in queue cause a snowball effect (old connections aren't
closed, new ones cannot be established). These connections are mostly
idle and the DB server has no problem handling thousands of them.
Allowing us to dynamically set maxconn depending on the backend usage
(LA, CPU, memory, etc.) enables us to have high maxconn for situations
like above, but lowering it in case there are real issues where the
backend servers become overloaded (cache issues, DB gets hit hard).
The configuration documention has been updated. Doc about the filter line has
been added and a new chapter (§. 9) has been created to list and document
supported filters (for now, flt_trace and flt_http_comp).
The developer documentation about filters has also been added. The is a "pre"
version. Incoming changes in the filter API will require an update.
This documentation requires a deeper review and some TODO need to be complete.
Adds some examples regarding shorthand IPv4 address notation which might
be confused with RFC 4632 CIDR notation, leading to different than
expected results.
Commit c8f0e78 ("DOC: typo: req.uri is now replaced by capture.req.uri")
fixed a discrepancy in the doc but the scheme is still missing, resulting
in a redirect loop. Let's fix this as well. This should be backported to
1.5.
It worth mentionning "timeout http-request" in the "timeout client"
documentation paragraph, to ensure nobody misses this important setting.
backport: 1.5 and above
Adds a little more clarity to the description of the maximum sizes of
the different variable scopes and adds a note about what happens when
the space allocated for variables is too small.
Also fixes some typos and grammar/spelling issues re/ variables and
their naming conventions, copied throughout the document.
Regarding the minor update introduced in the
cd6c3c7cb4 commit, the DeviceAtlas
module is now able to use up to 12 device properties via the
new ARG12 macro.
Released version 1.7-dev2 with the following main changes :
- DOC: lua: fix lua API
- DOC: mailers: typo in 'hostname' description
- DOC: compression: missing mention of libslz for compression algorithm
- BUILD/MINOR: regex: missing header
- BUG/MINOR: stream: bad return code
- DOC: lua: fix somme errors and add implicit types
- MINOR: lua: add set/get priv for applets
- BUG/MINOR: http: fix several off-by-one errors in the url_param parser
- BUG/MINOR: http: Be sure to process all the data received from a server
- MINOR: filters/http: Use a wrapper function instead of stream_int_retnclose
- BUG/MINOR: chunk: make chunk_dup() always check and set dst->size
- DOC: ssl: fixed some formatting errors in crt tag
- MINOR: chunks: ensure that chunk_strcpy() adds a trailing zero
- MINOR: chunks: add chunk_strcat() and chunk_newstr()
- MINOR: chunk: make chunk_initstr() take a const string
- MEDIUM: tools: add csv_enc_append() to preserve the original chunk
- MINOR: tools: make csv_enc_append() always start at the first byte of the chunk
- MINOR: lru: new function to delete <nb> least recently used keys
- DOC: add Ben Shillito as the maintainer of 51d
- BUG/MINOR: 51d: Ensures a unique domain for each configuration
- BUG/MINOR: 51d: Aligns Pattern cache implementation with HAProxy best practices.
- BUG/MINOR: 51d: Releases workset back to pool.
- BUG/MINOR: 51d: Aligned const pointers to changes in 51Degrees.
- CLEANUP: 51d: Aligned if statements with HAProxy best practices and removed casts from malloc.
- MINOR: rename master process name in -Ds (systemd mode)
- DOC: fix a few spelling mistakes
- DOC: fix "workaround" spelling
- BUG/MINOR: examples: Fixing haproxy.spec to remove references to .cfg files
- MINOR: fix the return type for dns_response_get_query_id() function
- MINOR: server state: missing LF (\n) on error message printed when parsing server state file
- BUG/MEDIUM: dns: no DNS resolution happens if no ports provided to the nameserver
- BUG/MAJOR: servers state: server port is erased when dns resolution is enabled on a server
- BUG/MEDIUM: servers state: server port is used uninitialized
- BUG/MEDIUM: config: Adding validation to stick-table expire value.
- BUG/MEDIUM: sample: http_date() doesn't provide the right day of the week
- BUG/MEDIUM: channel: fix miscalculation of available buffer space.
- MEDIUM: pools: add a new flag to avoid rounding pool size up
- BUG/MEDIUM: buffers: do not round up buffer size during allocation
- BUG/MINOR: stream: don't force retries if the server is DOWN
- BUG/MINOR: counters: make the sc-inc-gpc0 and sc-set-gpt0 touch the table
- MINOR: unix: don't mention free ports on EAGAIN
- BUG/CLEANUP: CLI: report the proper field states in "show sess"
- MINOR: stats: send content-length with the redirect to allow keep-alive
- BUG: stream_interface: Reuse connection even if the output channel is empty
- DOC: remove old tunnel mode assumptions
- BUG/MAJOR: http-reuse: fix risk of orphaned connections
- BUG/MEDIUM: http-reuse: do not share private connections across backends
- BUG/MINOR: ssl: Be sure to use unique serial for regenerated certificates
- BUG/MINOR: stats: fix missing comma in stats on agent drain
- MAJOR: filters: Add filters support
- MINOR: filters: Do not reset stream analyzers if the client is gone
- REORG: filters: Prepare creation of the HTTP compression filter
- MAJOR: filters/http: Rewrite the HTTP compression as a filter
- MEDIUM: filters: Use macros to call filters callbacks to speed-up processing
- MEDIUM: filters: remove http_start_chunk, http_last_chunk and http_chunk_end
- MEDIUM: filters: Replace filter_http_headers callback by an analyzer
- MEDIUM: filters/http: Move body parsing of HTTP messages in dedicated functions
- MINOR: filters: Add stream_filters structure to hide filters info
- MAJOR: filters: Require explicit registration to filter HTTP body and TCP data
- MINOR: filters: Remove unused or useless stuff and do small optimizations
- MEDIUM: filters: Optimize the HTTP compression for chunk encoded response
- MINOR: filters/http: Slightly update the parsing of chunks
- MINOR: filters/http: Forward remaining data when a channel has no "data" filters
- MINOR: filters: Add an filter example
- MINOR: filters: Extract proxy stuff from the struct filter
- MINOR: map: Add regex matching replacement
- BUG/MINOR: lua: unsafe initialization
- DOC: lua: fix somme errors
- MINOR: lua: file dedicated to unsafe functions
- MINOR: lua: add "now" time function
- MINOR: standard: add RFC HTTP date parser
- MINOR: lua: Add date functions
- MINOR: lua: move common function
- MINOR: lua: merge function
- MINOR: lua: Add concat class
- MINOR: standard: add function "escape_chunk"
- MEDIUM: log: add a new log format flag "E"
- DOC: add server name at rate-limit sessions example
- BUG/MEDIUM: ssl: fix off-by-one in ALPN list allocation
- BUG/MEDIUM: ssl: fix off-by-one in NPN list allocation
- DOC: LUA: fix some typos and syntax errors
- MINOR: cli: add a new "show env" command
- MEDIUM: config: allow to manipulate environment variables in the global section
- MEDIUM: cfgparse: reject incorrect 'timeout retry' keyword spelling in resolvers
- MINOR: mailers: increase default timeout to 10 seconds
- MINOR: mailers: use <CRLF> for all line endings
- BUG/MAJOR: lua: segfault using Concat object
- DOC: lua: copyrights
- MINOR: common: mask conversion
- MEDIUM: dns: extract options
- MEDIUM: dns: add a "resolve-net" option which allow to prefer an ip in a network
- MINOR: mailers: make it possible to configure the connection timeout
- BUG/MAJOR: lua: applets can't sleep.
- BUG/MINOR: server: some prototypes are renamed
- BUG/MINOR: lua: Useless copy
- BUG/MEDIUM: stats: stats bind-process doesn't propagate the process mask correctly
- BUG/MINOR: server: fix the format of the warning on address change
- CLEANUP: server: add "const" to some message strings
- MINOR: server: generalize the "updater" source
- BUG/MEDIUM: chunks: always reject negative-length chunks
- BUG/MINOR: systemd: ensure we don't miss signals
- BUG/MINOR: systemd: report the correct signal in debug message output
- BUG/MINOR: systemd: propagate the correct signal to haproxy
- MINOR: systemd: ensure a reload doesn't mask a stop
- BUG/MEDIUM: cfgparse: wrong argument offset after parsing server "sni" keyword
- CLEANUP: stats: Avoid computation with uninitialized bits.
- CLEANUP: pattern: Ignore unknown samples in pat_match_ip().
- CLEANUP: map: Avoid memory leak in out-of-memory condition.
- BUG/MINOR: tcpcheck: fix incorrect list usage resulting in failure to load certain configs
- BUG/MAJOR: samples: check smp->strm before using it
- MINOR: sample: add a new helper to initialize the owner of a sample
- MINOR: sample: always set a new sample's owner before evaluating it
- BUG/MAJOR: vars: always retrieve the stream and session from the sample
- CLEANUP: payload: remove useless and confusing nullity checks for channel buffer
- BUG/MINOR: ssl: fix usage of the various sample fetch functions
- MINOR: stats: create fields types suitable for all CSV output data
- MINOR: stats: add all the "show info" fields in a table
- MEDIUM: stats: fill all the show info elements prior to displaying them
- MINOR: stats: add a function to emit fields into a chunk
- MINOR: stats: add stats_dump_info_fields() to dump one field per line
- MEDIUM: stats: make use of stats_dump_info_fields() for "show info"
- MINOR: stats: add a declaration of all stats fields
- MINOR: stats: don't hard-code the CSV fields list anymore
- MINOR: stats: create stats fields storage and CSV dump function
- MEDIUM: stats: convert stats_dump_fe_stats() to use stats_dump_fields_csv()
- MEDIUM: stats: make stats_dump_fe_stats() use stats fields for HTML dump
- MEDIUM: stats: convert stats_dump_li_stats() to use stats_dump_fields_csv()
- MEDIUM: stats: make stats_dump_li_stats() use stats fields for HTML dump
- MEDIUM: stats: convert stats_dump_be_stats() to use stats_dump_fields_csv()
- MEDIUM: stats: make stats_dump_be_stats() use stats fields for HTML dump
- MEDIUM: stats: convert stats_dump_sv_stats() to use stats_dump_fields_csv()
- MEDIUM: stats: make stats_dump_sv_stats() use the stats field for HTML
- MEDIUM: stats: move the server state coloring logic to the server dump function
- MINOR: stats: do not use srv->admin & STATS_ADMF_MAINT in HTML dumps
- MINOR: stats: do not check srv->state for SRV_ST_STOPPED in HTML dumps
- MINOR: stats: make CSV report server check status only when enabled
- MINOR: stats: only report backend's down time if it has servers
- MINOR: stats: prepend '*' in front of the check status when in progress
- MINOR: stats: make HTML stats dump rely on the table for the check status
- MINOR: stats: add agent_status, agent_code, agent_duration to output
- MINOR: stats: add check_desc and agent_desc to the output fields
- MINOR: stats: add check and agent's health values in the output
- MEDIUM: stats: make the HTML server state dump use the CSV states
- MEDIUM: stats: only report observe errors when observe is set
- MEDIUM: stats: expose the same flags for CLI and HTTP accesses
- MEDIUM: stats: report server's address in the CSV output
- MEDIUM: stats: report the cookie value in the server & backend CSV dumps
- MEDIUM: stats: compute the color code only in the HTML form
- MEDIUM: stats: report the listeners' address in the CSV output
- MEDIUM: stats: make it possible to report the WAITING state for listeners
- REORG: stats: dump the frontend's HTML stats via a generic function
- REORG: stats: dump the socket stats via the generic function
- REORG: stats: dump the server stats via the generic function
- REORG: stats: dump the backend stats via the generic function
- MEDIUM: stats: add a new "mode" column to report the proxy mode
- MINOR: stats: report the load balancing algorithm in CSV output
- MINOR: stats: add 3 fields to report the frontend-specific connection stats
- MINOR: stats: report number of intercepted requests for frontend and backends
- MINOR: stats: introduce stats_dump_one_line() to dump one stats line
- CLEANUP: stats: make stats_dump_fields_html() not rely on proxy anymore
- MINOR: stats: add ST_SHOWADMIN to pass the admin info in the regular flags
- MINOR: stats: make stats_dump_fields_html() not use &trash by default
- MINOR: stats: add functions to emit typed fields into a chunk
- MEDIUM: stats: support "show info typed" on the CLI
- MEDIUM: stats: implement a typed output format for stats
- DOC: document the "show info typed" and "show stat typed" output formats
- MINOR: cfgparse: warn when uid parameter is not a number
- MINOR: cfgparse: warn when gid parameter is not a number
- BUG/MINOR: standard: Avoid free of non-allocated pointer
- BUG/MINOR: pattern: Avoid memory leak on out-of-memory condition
- CLEANUP: http: fix a build warning introduced by a recent fix
- BUG/MINOR: log: GMT offset not updated when entering/leaving DST
This was reported in HTML dumps already but not CSV. It reports the
number of monitor and stats requests. Ideally use-service and redirs
should be accounted for as well.
Frontends have extra information compared to other entities, they can
report some statistics at the connection level while the other ones
are limited to the session level. This patch adds 3 more fields for
this :
- conn_rate
- conn_rate_max
- conn_tot
It's worth noting that listeners theorically have such statistics, except
that the distinction between connections and sessions is not clearly made
in the code, so that will have to be improved later.
The server's cookie value is now reported in the "cookie" column and
used as-is from the HTML dump. It was the last reference to the sv
pointer from this place.
The same was done for the backend's dump.
This new field "addr" presents the server's address:port if the client
is either enabled via "stats show legends" in case of HTTP dumps, or
has at least level operator on the CLI. The address formats might be :
- ipv4:port
- [ipv6]:port
- unix
- (error message)
This adds the following fields :
- check_rise [...S]: server's "rise" parameter used by checks
- check_fall [...S]: server's "fall" parameter used by checks
- check_health [...S]: server's health check value between 0 and rise+fall-1
- agent_rise [...S]: agent's "rise" parameter, normally 1
- agent_fall [...S]: agent's "fall" parameter, normally 1
- agent_health [...S]: agent's health parameter, between 0 and rise+fall-1
Added these two new fields to the CSV output :
- check_desc : short human-readable description of check_status
- agent_desc : short human-readable description of agent_status
Also factor two tests for enabled checks.
The agent check status is now reported :
- agent_status : status of last agent check
- agent_code : numeric code reported by agent if any (unused for now)
- agent_duration : time in ms taken to finish last check
This patch introduces a configurable connection timeout for mailers
with a new "timeout mail <time>" directive.
Acked-by: Simon Horman <horms@verge.net.au>
This options prioritize th choice of an ip address matching a network. This is
useful with clouds to prefer a local ip. In some cases, a cloud high
avalailibility service can be announced with many ip addresses on many
differents datacenters. The latency between datacenter is not negligible, so
this patch permitsto prefers a local datacenter. If none address matchs the
configured network, another address is selected.
With new init systems such as systemd, environment variables became a
real mess because they're only considered on startup but not on reload
since the init script's variables cannot be passed to the process that
is signaled to reload.
This commit introduces an alternative method consisting in making it
possible to modify the environment from the global section with directives
like "setenv", "unsetenv", "presetenv" and "resetenv".
Since haproxy supports loading multiple config files, it now becomes
possible to put the host-dependant variables in one file and to
distribute the rest of the configuration to all nodes, without having
to deal with the init system's deficiencies.
Environment changes take effect immediately when the directives are
processed, so it's possible to do perform the same operations as are
usually performed in regular service config files.
Using environment variables in configuration files can make troubleshooting
complicated because there's no easy way to verify that the variables are
correct. This patch introduces a new "show env" command which displays the
whole environment on the CLI, one variable per line.
The socket must at least have level operator to display the environment.
The +E mode escapes characters '"', '\' and ']' with '\' as prefix. It
mostly makes sense to use it in the RFC5424 structured-data log formats.
Example:
log-format-sd %{+Q,+E}o\ [exampleSDID@1234\ header=%[capture.req.hdr(0)]]
Now, http_parse_chunk_size and http_skip_chunk_crlf return the number of bytes
parsed on success. http_skip_chunk_crlf does not use msg->sol anymore.
On the other hand, http_forward_trailers is unchanged. It returns >0 if the end
of trailers is reached and 0 if not. In all cases (except if an error is
encountered), msg->sol contains the length of the last parsed part of the
trailer headers.
Internal doc and comments about msg->sol has been updated accordingly.
Michał Pasierb reported doc inconsistencies regarding the old default
HTTP tunnel mode.
This patch fixes a few of those inconsistencies and should be backported
to both 1.6 and 1.5.
The applet can't have access to the session private data. This patch
fix this problem. Now an applet can use private data stored by actions
and fecthes.
This patch fix the Lua API documentation, and adds some internal link
between values returned and associated class.
This patch can be backported in 1.6.
Released version 1.7-dev1 with the following main changes :
- DOC: specify that stats socket doc (section 9.2) is in management
- BUILD: install only relevant and existing documentation
- CLEANUP: don't ignore debian/ directory if present
- BUG/MINOR: dns: parsing error of some DNS response
- BUG/MEDIUM: namespaces: don't fail if no namespace is used
- BUG/MAJOR: ssl: free the generated SSL_CTX if the LRU cache is disabled
- MEDIUM: dns: Don't use the ANY query type
- BUILD: ssl: fix build error introduced in commit 7969a3 with OpenSSL < 1.0.0
- DOC: fix a typo for a "deviceatlas" keyword
- FIX: small typo in an example using the "Referer" header
- MINOR: cli: ability to set per-server maxconn
- DEBUG/MINOR: memory: add a build option to disable memory pools sharing
- DEBUG/MEDIUM: memory: optionally protect free data in pools
- DEBUG/MEDIUM: memory: add optional control pool memory operations
- MEDIUM: memory: add accounting for failed allocations
- BUG/MEDIUM: config: count memory limits on 64 bits, not 32
- BUG/MAJOR: dns: first DNS response packet not matching queried hostname may lead to a loop
- BUG/MINOR: dns: unable to parse CNAMEs response
- BUG/MINOR: examples/haproxy.init: missing brace in quiet_check()
- DOC: deviceatlas: more example use cases.
- MINOR: config: allow IPv6 bracketed literals
- BUG/BUILD: replace haproxy-systemd-wrapper with $(EXTRA) in install-bin.
- BUILD: add Haiku as supported target.
- BUG/MAJOR: http: don't requeue an idle connection that is already queued
- DOC: typo on capture.res.hdr and capture.req.hdr
- BUG/MINOR: dns: check for duplicate nameserver id in a resolvers section was missing
- CLEANUP: use direction names in place of numeric values
- BUG/MEDIUM: lua: sample fetches based on response doesn't work
- MINOR: check: add agent-send server parameter
- BUG/MINOR: http rule: http capture 'id' rule points to a non existing id
- BUG/MINOR: server: check return value of fgets() in apply_server_state()
- BUG/MINOR: acl: don't use record layer in req_ssl_ver
- BUILD: freebsd: double declaration
- BUG/MEDIUM: lua: clean output buffer
- BUILD: check for libressl to be able to build against it
- DOC: lua-api/index.rst small example fixes, spelling correction.
- DOC: lua: architecture and first steps
- DOC: relation between timeout http-request and option http-buffer-request
- BUILD: Make deviceatlas require PCRE
- BUG: http: do not abort keep-alive connections on server timeout
- BUG/MEDIUM: http: switch the request channel to no-delay once done.
- BUG/MINOR: lua: don't force-sslv3 LUA's SSL socket
- BUILD/MINOR: http: proto_http.h needs sample.h
- BUG/MEDIUM: http: don't enable auto-close on the response side
- BUG/MEDIUM: stream: fix half-closed timeout handling
- CLEANUP: compression: don't allocate DEFAULT_MAXZLIBMEM without USE_ZLIB
- BUG/MEDIUM: cli: changing compression rate-limiting must require admin level
- BUG/MEDIUM: sample: urlp can't match an empty value
- BUILD: dumpstats: silencing warning for printf format specifier / time_t
- CLEANUP: proxy: calloc call inverted arguments
- MINOR: da: silent logging by default and displaying DeviceAtlas support if built.
- BUG/MEDIUM: da: stop DeviceAtlas processing in the convertor if there is no input.
- DOC: Edited 51Degrees section of README/
- BUG/MEDIUM: checks: email-alert not working when declared in defaults
- BUG/MINOR: checks: email-alert causes a segfault when an unknown mailers section is configured
- BUG/MINOR: checks: typo in an email-alert error message
- BUG/MINOR: tcpcheck: conf parsing error when no port configured on server and last rule is a CONNECT with no port
- BUG/MINOR: tcpcheck: conf parsing error when no port configured on server and first rule(s) is (are) COMMENT
- BUG/MEDIUM: http: fix http-reuse when frontend and backend differ
- DOC: prefer using http-request/response over reqXXX/rspXXX directives
- CLEANUP: haproxy: using _GNU_SOURCE instead of __USE_GNU macro.
- MINOR: ssl: Added cert_key_and_chain struct
- MEDIUM: ssl: Added support for creating SSL_CTX with multiple certs
- MINOR: ssl: Added multi cert support for crt-list config keyword
- MEDIUM: ssl: Added multi cert support for loading crt directories
- MEDIUM: ssl: Added support for Multi-Cert OCSP Stapling
- BUILD: ssl: set SSL_SOCK_NUM_KEYTYPES with openssl < 1.0.2
- MINOR: config: make tune.recv_enough configurable
- BUG/MEDIUM: config: properly adjust maxconn with nbproc when memmax is forced
- DOC: ssl: Adding docs for Multi-Cert bundling
- BUG/MEDIUM: peers: table entries learned from a remote are pushed to others after a random delay.
- BUG/MEDIUM: peers: old stick table updates could be repushed.
- MINOR: lua: service/applet can have access to the HTTP headers when a POST is received
- REORG/MINOR: lua: convert boolean "int" to bitfield
- BUG/MEDIUM: lua: Lua applets must not fetch samples using http_txn
- BUG/MINOR: lua: Lua applets must not use http_txn
- BUG/MEDIUM: lua: Forbid HTTP applets from being called from tcp rulesets
- BUG/MAJOR: lua: Do not force the HTTP analysers in use-services
- CLEANUP: lua: bad error messages
- CONTRIB: initiate a debugging suite to make debugging easier
When a POST is processed by a Lua service, the HTTP header are
potentially gone. So, we cannot retrieve their content using
the standard "hdr" sample fetchs (which will soon become invalid
anyway) from an applet.
This patch add an entry "headers" to the object applet_http. This
entry is an array containing all the headers. It permits to use the
HTTP headers during the processing of the service.
Many thanks to Jan Bruder for reporting this issue with enough
details to reproduce it.
This patch will have to be backported to 1.6 since it will be the
only way to access headers from Lua applets.
When memmax is forced using "-m", the per-process memory limit is enforced
using setrlimit(), but this value is not used to compute the automatic
maxconn limit. In addition, the per-process memory limit didn't consider
the fact that the shared SSL cache only needs to be accounted once.
The doc was also fixed to clearly state that "-m" is global and not per
process. It makes sense because people who use -m want to protect the
system's resources regardless of whatever appears in the configuration.
This setting used to be assigned to a variable tunable from a constant
and for an unknown reason never made its way into the config parser.
tune.recv_enough <number>
Haproxy uses some hints to detect that a short read indicates the end of the
socket buffers. One of them is that a read returns more than <recv_enough>
bytes, which defaults to 10136 (7 segments of 1448 each). This default value
may be changed by this setting to better deal with workloads involving lots
of short messages such as telnet or SSH sessions.
add referrence for "http-request" or "http-response" in reqXXX/rspXXX
directives.
add a paragraph in "http-request" and "http-response" stating that
reqXXX/rspXXX directives are discouraged
The documentation missed the explanation and relation between the
timeout http-request and option http-buffer-request.
Combined together, it helps protecting against slow POST types of
attacks.
[wt: backport to 1.6]
It is possible to create a http capture rule which points to a capture slot
id which does not exist.
Current patch prevent this when parsing configuration and prevent running
configuration which contains such rules.
This configuration is now invalid:
frontend f
bind :8080
http-request capture req.hdr(User-Agent) id 0
default_backend b
this one as well:
frontend f
bind :8080
declare capture request len 32 # implicit id is 0 here
http-request capture req.hdr(User-Agent) id 1
default_backend b
It applies of course to both http-request and http-response rules.
Causes HAProxy to emit a static string to the agent on every check,
so that you can independently control multiple services running
behind a single agent port.
This commit adds support for setting a per-server maxconn from the stats
socket. The only really notable part of this commit is that we need to
check if maxconn == minconn before changing things, as this indicates
that we are NOT using dynamic maxconn. When we are not using dynamic
maxconn, we should update maxconn/minconn in lockstep.
It was reported that an example was manipulating a "Referrer" header instead
of the known "Referer" one. Even if it's an example wich doesn't break things,
the typo can be fixed.
The fix should be backported in 1.4/1.5/1.6 branches.
Commit 44aed90ce1 moved the stats socket
documentation from config to management but the remaining references to
section 9.2 were not updated; improve it to be less confusing.
Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
Released version 1.6.0 with the following main changes :
- BUG/MINOR: Handle interactive mode in cli handler
- DOC: global section missing parameters
- DOC: backend section missing parameters
- DOC: stats paramaters available in frontend
- MINOR: lru: do not allocate useless memory in lru64_lookup
- BUG/MINOR: http: Add OPTIONS in supported http methods (found by find_http_meth)
- BUG/MINOR: ssl: fix management of the cache where forged certificates are stored
- MINOR: ssl: Release Servers SSL context when HAProxy is shut down
- MINOR: ssl: Read the file used to generate certificates in any order
- MINOR: ssl: Add support for EC for the CA used to sign generated certificates
- MINOR: ssl: Add callbacks to set DH/ECDH params for generated certificates
- BUG/MEDIUM: logs: fix time zone offset format in RFC5424
- BUILD: Fix the build on OSX (htonll/ntohll)
- BUILD: enable build on Linux/s390x
- BUG/MEDIUM: lua: direction test failed
- MINOR: lua: fix a spelling error in some error messages
- CLEANUP: cli: ensure we can never double-free error messages
- BUG/MEDIUM: lua: force server-close mode on Lua services
- MEDIUM: init: support more command line arguments after pid list
- MEDIUM: init: support a list of files on the command line
- MINOR: debug: enable memory poisonning to use byte 0
- BUILD: ssl: fix build error introduced by recent commit
- BUG/MINOR: config: make the stats socket pass the correct proxy to the parsers
- MEDIUM: server: implement TCP_USER_TIMEOUT on the server
- DOC: mention the "namespace" options for bind and server lines
- DOC: add the "management" documentation
- DOC: move the stats socket documentation from config to management
- MINOR: examples: update haproxy.spec to mention new docs
- DOC: mention management.txt in README
- DOC: remove haproxy-{en,fr}.txt
- BUILD: properly report when USE_ZLIB and USE_SLZ are used together
- MINOR: init: report use of libslz instead of "no compression"
- CLEANUP: examples: remove some obsolete and confusing files
- CLEANUP: examples: remove obsolete configuration file samples
- CLEANUP: examples: fix the example file content-sw-sample.cfg
- CLEANUP: examples: update sample file option-http_proxy.cfg
- CLEANUP: examples: update sample file ssl.cfg
- CLEANUP: tests: move a test file from examples/ to tests/
- CLEANUP: examples: shut up warnings in transparent proxy example
- CLEANUP: tests: removed completely obsolete test files
- DOC: update ROADMAP to remove what was done in 1.6
- BUG/MEDIUM: pattern: fixup use_after_free in the pat_ref_delete_by_id
This doc explains how to start/stop haproxy, what signals are used
and a few debugging tricks. It's far from being complete but should
already help a number of users.
The stats part will be taken from the config doc.
This is equivalent to commit 2af207a ("MEDIUM: tcp: implement tcp-ut
bind option to set TCP_USER_TIMEOUT") except that this time it works
on the server side. The purpose is to detect dead server connections
even when checks are rare, disabled, or after a soft reload (since
checks are disabled there as well), and to ensure client connections
will get killed faster.
Released version 1.6-dev7 with the following main changes :
- MINOR: cli: Dump all resolvers stats if no resolver section is given
- BUG: config: external-check command validation is checking for incorrect arguments.
- DOC: documentation format cleanups
- DOC: lua: few typos.
- BUG/MEDIUM: str2ip: make getaddrinfo() consider local address selection policy
- BUG/MEDIUM: logs: segfault writing to log from Lua
- DOC: fix lua use-service example
- MINOR: payload: add support for tls session ticket ext
- MINOR: lua: remove the run flag
- MEDIUM: lua: change the timeout execution
- MINOR: lua: rename the tune.lua.applet-timeout
- DOC: lua: update Lua doc
- DOC: lua: update doc according with the last Lua changes
- MINOR: http/tcp: fill the avalaible actions
- DOC: reorder misplaced res.ssl_hello_type in the doc
- BUG/MINOR: tcp: make silent-drop always force a TCP reset
- CLEANUP: tcp: silent-drop: only drain the connection when quick-ack is disabled
- BUILD: tcp: use IPPROTO_IP when SOL_IP is not available
- BUILD: server: fix build warnings introduced by load-server-state
- BUG/MEDIUM: server: fix misuse of format string in load-server-state's warnings
This commit adds support for dumping all resolver stats. Specifically
if a command 'show stats resolvers' is issued withOUT a resolver section
id, we dump all known resolver sections. If none are configured, a
message is displayed indicating that.
req.ssl_st_ext : integer
Returns 0 if the client didn't send a SessionTicket TLS Extension (RFC5077)
Returns 1 if the client sent SessionTicket TLS Extension
Returns 2 if the client also sent non-zero length TLS SessionTicket
Chapter "Mailers" was missing from the summary, tabulations were used and
several lines exceeded 80 columns : some lines still exceeds the 80 columns
limit but are left as is, as it mostly concerns syntax and configuration
examples.
Released version 1.6-dev6 with the following main changes :
- BUG/MAJOR: can't enable a server through the stat socket
- MINOR: server: Macro definition for server-state
- MINOR: cli: new stats socket command: show servers state
- DOC: stats socket command: show servers state
- MINOR: config: new global directive server-state-base
- DOC: global directive server-state-base
- MINOR: config: new global section directive: server-state-file
- DOC: new global directive: server-state-file
- MINOR: config: new backend directives: load-server-state-from-file and server-state-file-name
- DOC: load-server-state-from-file
- MINOR: init: server state loaded from file
- MINOR: server: startup slowstart task when using seamless reload of HAProxy
- MINOR: cli: new stats socket command: show backend
- DOC: servers state seamless reload example
- BUG: dns: can't connect UDP socket on FreeBSD
- MINOR: cfgparse: New function cfg_unregister_sections()
- MINOR: chunk: New function free_trash_buffers()
- BUG/MEDIUM: main: Freeing a bunch of static pointers
- MINOR: proto_http: Externalisation of previously internal functions
- MINOR: global: Few new struct fields for da module
- MAJOR: da: Update of the DeviceAtlas API module
- DOC: DeviceAtlas new keywords
- DOC: README: DeviceAtlas sample configuration updates
- MEDIUM: log: replace sendto() with sendmsg() in __send_log()
- MEDIUM: log: use a separate buffer for the header and for the message
- MEDIUM: logs: remove the hostname, tag and pid part from the logheader
- MEDIUM: logs: add support for RFC5424 header format per logger
- MEDIUM: logs: add a new RFC5424 log-format for the structured-data
- DOC: mention support for the RFC5424 syslog message format
- MEDIUM: logs: have global.log_send_hostname not contain the trailing space
- MEDIUM: logs: pass the trailing "\n" as an iovec
- BUG/MEDIUM: peers: some table updates are randomly not pushed.
- BUG/MEDIUM: peers: same table updates re-pushed after a re-connect
- BUG/MINOR: fct peer_prepare_ackmsg should not use trash.
- MINOR: http: made CHECK_HTTP_MESSAGE_FIRST accessible to other functions
- MINOR: global: Added new fields for 51Degrees device detection
- DOC: Added more explanation for 51Degrees V3.2
- BUILD: Changed 51Degrees option to support V3.2
- MAJOR: 51d: Upgraded to support 51Degrees V3.2 and new features
- MINOR: 51d: Improved string handling for LRU cache
- DOC: add references to rise/fall for the fastinter explanation
- MINOR: support cpu-map feature through the compile option USE_CPU_AFFINITY on FreeBSD
- BUG/MAJOR: lua: potential unexpected aborts()
- BUG/MINOR: lua: breaks the log message if his size exceed one buffer
- MINOR: action: add private configuration
- MINOR: action: add reference to the original keywork matched for the called parser.
- MINOR: lua: change actions registration
- MEDIUM: proto_http: smp_prefetch_http initialize txn
- MINOR: channel: rename function chn_sess to chn_strm
- CLEANUP: lua: align defines
- MINOR: http: export http_get_path() function
- MINOR: http: export the get_reason() function
- MINOR: http: export function http_msg_analyzer()
- MINOR: http: split initialization
- MINOR: lua: reset pointer after use
- MINOR: lua: identify userdata objects
- MEDIUM: lua: use the function lua_rawset in place of lua_settable
- BUG/MAJOR: lua: segfault after the channel data is modified by some Lua action.
- CLEANUP: lua: use calloc in place of malloc
- BUG/MEDIUM: lua: longjmp function must be unregistered
- BUG/MEDIUM: lua: forces a garbage collection
- BUG/MEDIUM: lua: wakeup task on bad conditions
- MINOR: standard: avoid DNS resolution from the function str2sa_range()
- MINOR: lua: extend socket address to support non-IP families
- MINOR: lua/applet: the cosocket applet should use appctx_wakeup in place of task_wakeup
- BUG/MEDIUM: lua: socket destroy before reading pending data
- MEDIUM: lua: change the GC policy
- OPTIM/MEDIUM: lua: executes the garbage collector only when using cosocket
- BUG/MEDIUM: lua: don't reset undesired flags in hlua_ctx_resume
- MINOR: applet: add init function
- MINOR: applet: add an execution timeout
- MINOR: stream/applet: add use-service action
- MINOR: lua: add AppletTCP class and service
- MINOR: lua: add AppletHTTP class and service
- DOC: lua: some documentation update
- DOC: add the documentation about internal circular lists
- DOC: add a CONTRIBUTING file
- DOC: add a MAINTAINERS file
- BUG/MAJOR: peers: fix a crash when stopping peers on unbound processes
- DOC: update coding-style to reference checkpatch.pl
- BUG/MEDIUM: stick-tables: fix double-decrement of tracked entries
- BUG/MINOR: args: add name for ARGT_VAR
- DOC: add more entries to MAINTAINERS
- DOC: add more entries to MAINTAINERS
- CLEANUP: stream-int: remove obsolete function si_applet_call()
- BUG/MAJOR: cli: do not dereference strm_li()->proto->name
- BUG/MEDIUM: http: do not dereference strm_li(stream)
- BUG/MEDIUM: proxy: do not dereference strm_li(stream)
- BUG/MEDIUM: stream: do not dereference strm_li(stream)
- MINOR: stream-int: use si_release_endpoint() to close idle conns
- BUG/MEDIUM: payload: make req.payload and payload_lv aware of dynamic buffers
- BUG/MEDIUM: acl: always accept match "found"
- MINOR: applet: rename applet_runq to applet_active_queue
- BUG/MAJOR: applet: use a separate run queue to maintain list integrity
- MEDIUM: stream-int: split stream_int_update_conn() into si- and conn-specific parts
- MINOR: stream-int: implement a new stream_int_update() function
- MEDIUM: stream-int: factor out the stream update functions
- MEDIUM: stream-int: call stream_int_update() from si_update()
- MINOR: stream-int: export stream_int_update_*
- MINOR: stream-int: move the applet_pause call out of the stream updates
- MEDIUM: stream-int: clean up the conditions to enable reading in si_conn_wake_cb
- MINOR: stream-int: implement the stream_int_notify() function
- MEDIUM: stream-int: use the same stream notification function for applets and conns
- MEDIUM: stream-int: completely remove stream_int_update_embedded()
- MINOR: stream-int: rename si_applet_done() to si_applet_wake_cb()
- BUG/MEDIUM: applet: fix reporting of broken write situation
- BUG/MINOR: stats: do not call cli_release_handler 3 times
- BUG/MEDIUM: cli: properly handle closed output
- MINOR: cli: do not call the release handler on internal error.
- BUG/MEDIUM: stream-int: avoid double-call to applet->release
- DEBUG: add p_malloc() to return a poisonned memory area
- CLEANUP: lua: remove unneeded memset(0) after calloc()
- MINOR: lua: use the proper applet wakeup mechanism
- BUG/MEDIUM: lua: better fix for the protocol check
- BUG/MEDIUM: lua: properly set the target on the connection
- MEDIUM: actions: pass a new "flags" argument to custom actions
- MEDIUM: actions: add new flag ACT_FLAG_FINAL to notify about last call
- MEDIUM: http: pass ACT_FLAG_FINAL to custom actions
- MEDIUM: lua: only allow actions to yield if not in a final call
- DOC: clarify how to make use of abstract sockets in socat
- CLEANUP: config: make the errorloc/errorfile messages less confusing
- MEDIUM: action: add a new flag ACT_FLAG_FIRST
- BUG/MINOR: config: check that tune.bufsize is always positive
- MEDIUM: config: set tune.maxrewrite to 1024 by default
- DOC: add David Carlier as maintainer of da.c
- DOC: fix some broken unexpected unicode chars in the Lua doc.
- BUG/MEDIUM: proxy: ignore stopped peers
- BUG/MEDIUM: proxy: do not wake stopped proxies' tasks during soft_stop()
- MEDIUM: init: completely deallocate unused peers
- BUG/MEDIUM: tcp: fix inverted condition to call custom actions
- DOC: remove outdated actions lists on tcp-request/response
- MEDIUM: tcp: add new tcp action "silent-drop"
- DOC: add URLs to optional libraries in the README
This stops the evaluation of the rules and makes the client-facing
connection suddenly disappear using a system-dependant way that tries
to prevent the client from being notified. The effect it then that the
client still sees an established connection while there's none on
HAProxy. The purpose is to achieve a comparable effect to "tarpit"
except that it doesn't use any local resource at all on the machine
running HAProxy. It can resist much higher loads than "tarpit", and
slow down stronger attackers. It is important to undestand the impact
of using this mechanism. All stateful equipments placed between the
client and HAProxy (firewalls, proxies, load balancers) will also keep
the established connection for a long time and may suffer from this
action. On modern Linux systems running with enough privileges, the
TCP_REPAIR socket option is used to block the emission of a TCP
reset. On other systems, the socket's TTL is reduced to 1 so that the
TCP reset doesn't pass the first router, though it's still delivered to
local networks.
