This patch remove the limit of 32 groups. It also permit to use standard
"pat_parse_str()" function in place of "pat_parse_strcat()". The
"pat_parse_strcat()" is no longer used and its removed. Before this
patch, the groups are stored in a bitfield, now they are stored in a
list of strings. The matching is slower, but the number of groups is
low and generally the list of allowed groups is short.
The fetch function "smp_fetch_http_auth_grp()" used with the name
"http_auth_group" return valid username. It can be used as string for
displaying the username or with the acl "http_auth_group" for checking
the group of the user.
Maybe the names of the ACL and fetch methods are no longer suitable, but
I keep the current names for conserving the compatibility with existing
configurations.
The function "userlist_postinit()" is created from verification code
stored in the big function "check_config_validity()". The code is
adapted to the new authentication storage system and it is moved in the
"src/auth.c" file. This function is used to check the validity of the
users declared in groups and to check the validity of groups declared
on the "user" entries.
This resolve function is executed before the check of all proxy because
many acl needs solved users and groups.
This new filter converts BIN type to its hexadecimal
representation in STR type. It is used to keep the
compatibility with the original bin2str cast.
It will be useful when bin2str changes to copy the
string as-is without encoding anymore.
Sometimes it can be useful to generate a random value, at least
for debugging purposes, but also to take routing decisions or to
pass such a value to a backend server.
The ability to globally override the default client and server cipher
suites has been requested multiple times since the introduction of SSL.
This commit adds two new keywords to the global section for this :
- ssl-default-bind-ciphers
- ssl-default-server-ciphers
It is still possible to preset them at build time by setting the macros
LISTEN_DEFAULT_CIPHERS and CONNECT_DEFAULT_CIPHERS.
The new tune.idletimer value allows one to set a different value for
idle stream detection. The default value remains set to one second.
It is possible to disable it using zero, and to change the default
value at build time using DEFAULT_IDLE_TIMER.
It happens that latest change broke some monitoring tools which expect the
field to be found at the same position as indicated in the doc. Let's move
it to the last column instead.
Add 2 sample fetchs allowing to extract the method and the uri of an
HTTP request.
FIXME: the sample fetches parser can't add the LW_REQ requirement, at
the moment this flag is used automatically when you use sample fetches.
Note: also fixed the alphabetical order of other capture.req.* keywords
in the doc.
Released version 1.5-dev22 with the following main changes :
- MEDIUM: tcp-check new feature: connect
- MEDIUM: ssl: Set verify 'required' as global default for servers side.
- MINOR: ssl: handshake optim for long certificate chains.
- BUG/MINOR: pattern: pattern comparison executed twice
- BUG/MEDIUM: map: segmentation fault with the stats's socket command "set map ..."
- BUG/MEDIUM: pattern: Segfault in binary parser
- MINOR: pattern: move functions for grouping pat_match_* and pat_parse_* and add documentation.
- MINOR: standard: The parse_binary() returns the length consumed and his documentation is updated
- BUG/MINOR: payload: the patterns of the acl "req.ssl_ver" are no parsed with the good function.
- BUG/MEDIUM: pattern: "pat_parse_dotted_ver()" set bad expect_type.
- BUG/MINOR: sample: The c_str2int converter does not fail if the entry is not an integer
- BUG/MEDIUM: http/auth: Sometimes the authentication credentials can be mix between two requests
- MINOR: doc: Bad cli function name.
- MINOR: http: smp_fetch_capture_header_* fetch captured headers
- BUILD: last release inadvertently prepended a "+" in front of the date
- BUG/MEDIUM: stream-int: fix the keep-alive idle connection handler
- BUG/MEDIUM: backend: do not re-initialize the connection's context upon reuse
- BUG: Revert "OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes"
- BUG/MINOR: checks: successful check completion must not re-enable MAINT servers
- MINOR: http: try to stick to same server after status 401/407
- BUG/MINOR: http: always disable compression on HTTP/1.0
- OPTIM: poll: restore polling after a poll/stop/want sequence
- OPTIM: http: don't stop polling for read on the client side after a request
- BUG/MEDIUM: checks: unchecked servers could not be enabled anymore
- BUG/MEDIUM: stats: the web interface must check the tracked servers before enabling
- BUG/MINOR: channel: CHN_INFINITE_FORWARD must be unsigned
- BUG/MINOR: stream-int: do not clear the owner upon unregister
- MEDIUM: stats: add support for HTTP keep-alive on the stats page
- BUG/MEDIUM: stats: fix HTTP/1.0 breakage introduced in previous patch
- Revert "MEDIUM: stats: add support for HTTP keep-alive on the stats page"
- MAJOR: channel: add a new flag CF_WAKE_WRITE to notify the task of writes
- OPTIM: session: set the READ_DONTWAIT flag when connecting
- BUG/MINOR: http: don't clear the SI_FL_DONT_WAKE flag between requests
- MINOR: session: factor out the connect time measurement
- MEDIUM: session: prepare to support earlier transitions to the established state
- MEDIUM: stream-int: make si_connect() return an established state when possible
- MINOR: checks: use an inline function for health_adjust()
- OPTIM: session: put unlikely() around the freewheeling code
- MEDIUM: config: report a warning when multiple servers have the same name
- BUG: Revert "OPTIM: poll: restore polling after a poll/stop/want sequence"
- BUILD/MINOR: listener: remove a glibc warning on accept4()
- BUG/MAJOR: connection: fix mismatch between rcv_buf's API and usage
- BUILD: listener: fix recent accept4() again
- BUG/MAJOR: ssl: fix breakage caused by recent fix abf08d9
- BUG/MEDIUM: polling: ensure we update FD status when there's no more activity
- MEDIUM: listener: fix polling management in the accept loop
- MINOR: protocol: improve the proto->drain() API
- MINOR: connection: add a new conn_drain() function
- MEDIUM: tcp: report in tcp_drain() that lingering is already disabled on close
- MEDIUM: connection: update callers of ctrl->drain() to use conn_drain()
- MINOR: connection: add more error codes to report connection errors
- MEDIUM: tcp: report connection error at the connection level
- MEDIUM: checks: make use of chk_report_conn_err() for connection errors
- BUG/MEDIUM: unique_id: HTTP request counter is not stable
- DOC: fix misleading information about SIGQUIT
- BUG/MAJOR: fix freezes during compression
- BUG/MEDIUM: stream-interface: don't wake the task up before end of transfer
- BUILD: fix VERDATE exclusion regex
- CLEANUP: polling: rename "spec_e" to "state"
- DOC: add a diagram showing polling state transitions
- REORG: polling: rename "spec_e" to "state" and "spec_p" to "cache"
- REORG: polling: rename "fd_spec" to "fd_cache"
- REORG: polling: rename the cache allocation functions
- REORG: polling: rename "fd_process_spec_events()" to "fd_process_cached_events()"
- MAJOR: polling: rework the whole polling system
- MAJOR: connection: remove the CO_FL_WAIT_{RD,WR} flags
- MEDIUM: connection: remove conn_{data,sock}_poll_{recv,send}
- MEDIUM: connection: add check for readiness in I/O handlers
- MEDIUM: stream-interface: the polling flags must always be updated in chk_snd_conn
- MINOR: stream-interface: no need to call fd_stop_both() on error
- MEDIUM: connection: no need to recheck FD state
- CLEANUP: connection: use conn_ctrl_ready() instead of checking the flag
- CLEANUP: connection: use conn_xprt_ready() instead of checking the flag
- CLEANUP: connection: fix comments in connection.h to reflect new behaviour.
- OPTIM: raw-sock: don't speculate after a short read if polling is enabled
- MEDIUM: polling: centralize polled events processing
- MINOR: polling: create function fd_compute_new_polled_status()
- MINOR: cli: add more information to the "show info" output
- MEDIUM: listener: add support for limiting the session rate in addition to the connection rate
- MEDIUM: listener: apply a limit on the session rate submitted to SSL
- REORG: stats: move the stats socket states to dumpstats.c
- MINOR: cli: add the new "show pools" command
- BUG/MEDIUM: counters: flush content counters after each request
- BUG/MEDIUM: counters: fix stick-table entry leak when using track-sc2 in connection
- MINOR: tools: add very basic support for composite pointers
- MEDIUM: counters: stop relying on session flags at all
- BUG/MINOR: cli: fix missing break in command line parser
- BUG/MINOR: config: correctly report when log-format headers require HTTP mode
- MAJOR: http: update connection mode configuration
- MEDIUM: http: make keep-alive + httpclose be passive mode
- MAJOR: http: switch to keep-alive mode by default
- BUG/MEDIUM: http: fix regression caused by recent switch to keep-alive by default
- BUG/MEDIUM: listener: improve detection of non-working accept4()
- BUILD: listener: add fcntl.h and unistd.h
- BUG/MINOR: raw_sock: correctly set the MSG_MORE flag
A new tcp-check rule type: connect.
It allows HAProxy to test applications which stand on multiple ports or
multiple applications load-balanced through the same backend.
Since we support HTTP keep-alive, there is no more reason for staying
in tunnel mode by default. It is confusing for new users and creates
more issues than it solves. Option "http-tunnel" is available to force
to use it if really desired.
Switching to KA by default has implied to change the value of some
option flags and some transaction flags so that value zero (default)
matches keep-alive. That explains why more code has been changed than
expected. Tests have been run on the 25 combinations of frontend and
backend options, plus a few with option http-pretend-keepalive, and
no anomaly was found.
The relation between frontend and backends remains the same. Options
have been updated to take precedence over http-keep-alive which is now
implicit.
All references in the doc to haproxy not supporting keep-alive have
been fixed, and the doc for config options has been updated.
At the very beginning of haproxy, there was "option httpclose" to make
haproxy add a "Connection: close" header in both directions to invite
both sides to agree on closing the connection. It did not work with some
rare products, so "option forceclose" was added to do the same and actively
close the connection. Then client-side keep-alive was supported, so option
http-server-close was introduced. Now we have keep-alive with a fourth
option, not to mention the implicit tunnel mode.
The connection configuration has become a total mess because all the
options above may be combined together, despite almost everyone thinking
they cancel each other, as judging from the common problem reports on the
mailing list. Unfortunately, re-reading the doc shows that it's not clear
at all that options may be combined, and the opposite seems more obvious
since they're compared. The most common issue is options being set in the
defaults section that are not negated in other sections, but are just
combined when the user expects them to be overloaded. The migration to
keep-alive by default will only make things worse.
So let's start to address the first problem. A transaction can only work in
5 modes today :
- tunnel : haproxy doesn't bother with what follows the first req/resp
- passive close : option http-close
- forced close : option forceclose
- server close : option http-server-close with keep-alive on the client side
- keep-alive : option http-keep-alive, end to end
All 16 combination for each section fall into one of these cases. Same for
the 256 combinations resulting from frontend+backend different modes.
With this patch, we're doing something slightly different, which will not
change anything for users with valid configs, and will only change the
behaviour for users with unsafe configs. The principle is that these options
may not combined anymore, and that the latest one always overrides all the
other ones, including those inherited from the defaults section. The "no
option xxx" statement is still supported to cancel one option and fall back
to the default one. It is mainly needed to ignore defaults sections (eg:
force the tunnel mode). The frontend+backend combinations have not changed.
So for examplen the following configuration used to put the connection
into forceclose :
defaults http
mode http
option httpclose
frontend foo.
option http-server-close
=> http-server-close+httpclose = forceclose before this patch! Now
the frontend's config replaces the defaults config and results in
the more expected http-server-close.
All 25 combinations of the 5 modes in (frontend,backend) have been
successfully tested.
In order to prepare for upcoming changes, a new "option http-tunnel" was
added. It currently only voids all other options, and has the lowest
precedence when mixed with another option in another frontend/backend.
If no CA file specified on a server line, the config parser will show an error.
Adds an cmdline option '-dV' to re-set verify 'none' as global default on
servers side (previous behavior).
Also adds 'ssl-server-verify' global statement to set global default to
'none' or 'required'.
WARNING: this changes the default verify mode from "none" to "required" on
the server side, and it *will* break insecure setups.
One year ago, commit 5d5b5d8 ("MEDIUM: proto_tcp: add support for tracking
L7 information") brought support for tracking L7 information in tcp-request
content rules. Two years earlier, commit 0a4838c ("[MEDIUM] session-counters:
correctly unbind the counters tracked by the backend") used to flush the
backend counters after processing a request.
While that earliest patch was correct at the time, it became wrong after
the second patch was merged. The code does what it says, but the concept
is flawed. "TCP request content" rules are evaluated for each HTTP request
over a single connection. So if such a rule in the frontend decides to
track any L7 information or to track L4 information when an L7 condition
matches, then it is applied to all requests over the same connection even
if they don't match. This means that a rule such as :
tcp-request content track-sc0 src if { path /index.html }
will count one request for index.html, and another one for each of the
objects present on this page that are fetched over the same connection
which sent the initial matching request.
Worse, it is possible to make the code do stupid things by using multiple
counters:
tcp-request content track-sc0 src if { path /foo }
tcp-request content track-sc1 src if { path /bar }
Just sending two requests first, one with /foo, one with /bar, shows
twice the number of requests for all subsequent requests. Just because
both of them persist after the end of the request.
So the decision to flush backend-tracked counters was not the correct
one. In practice, what is important is to flush countent-based rules
since they are the ones evaluated for each request.
Doing so requires new flags in the session however, to keep track of
which stick-counter was tracked by what ruleset. A later change might
make this easier to maintain over time.
This bug is 1.5-specific, no backport to stable is needed.
show pools
Dump the status of internal memory pools. This is useful to track memory
usage when suspecting a memory leak for example. It does exactly the same
as the SIGQUIT when running in foreground except that it does not flush
the pools.
Just like the previous commit, we sometimes want to limit the rate of
incoming SSL connections. While it can be done for a frontend, it was
not possible for a whole process, which makes sense when multiple
processes are running on a system to server multiple customers.
The new global "maxsslrate" setting is usable to fix a limit on the
session rate going to the SSL frontends. The limits applies before
the SSL handshake and not after, so that it saves the SSL stack from
expensive key computations that would finally be aborted before being
accounted for.
The same setting may be changed at run time on the CLI using
"set rate-limit ssl-session global".
It's sometimes useful to be able to limit the connection rate on a machine
running many haproxy instances (eg: per customer) but it removes the ability
for that machine to defend itself against a DoS. Thus, better also provide a
limit on the session rate, which does not include the connections rejected by
"tcp-request connection" rules. This permits to have much higher limits on
the connection rate without having to raise the session rate limit to insane
values.
The limit can be changed on the CLI using "set rate-limit sessions global",
or in the global section using "maxsessrate".
Patrick Hemmer reported that using unique_id_format and logs did not
report the same unique ID counter since commit 9f09521 ("BUG/MEDIUM:
unique_id: HTTP request counter must be unique!"). This is because
the increment was done while producing the log message, so it was
performed twice.
A better solution consists in fetching a new value once per request
and saving it in the request or session context for all of this
request's life.
It happens that sessions already have a unique ID field which is used
for debugging and reporting errors, and which differs from the one
sent in logs and unique_id header.
So let's change this to reuse this field to have coherent IDs everywhere.
As of now, a session gets a new unique ID once it is instanciated. This
means that TCP sessions will also benefit from a unique ID that can be
logged. And this ID is renewed for each extra HTTP request received on
an existing session. Thus, all TCP sessions and HTTP requests will have
distinct IDs that will be stable along all their life, and coherent
between all places where they're used (logs, unique_id header,
"show sess", "show errors").
This feature is 1.5-specific, no backport to 1.4 is needed.
In HTTP keep-alive mode, if we receive a 401, we still have a chance
of being able to send the visitor again to the same server over the
same connection. This is required by some broken protocols such as
NTLM, and anyway whenever there is an opportunity for sending the
challenge to the proper place, it's better to do it (at least it
helps with debugging).
Released version 1.5-dev21 with the following main changes :
- MINOR: stats: don't use a monospace font to report numbers
- MINOR: session: remove debugging code
- BUG/MAJOR: patterns: fix double free caused by loading strings from files
- MEDIUM: http: make option http_proxy automatically rewrite the URL
- BUG/MEDIUM: http: cook_cnt() forgets to set its output type
- BUG/MINOR: stats: correctly report throttle rate of low weight servers
- BUG/MEDIUM: checks: servers must not start in slowstart mode
- BUG/MINOR: acl: parser must also stop at comma on ACL-only keywords
- MEDIUM: stream-int: implement a very simplistic idle connection manager
- DOC: update the ROADMAP file
Released version 1.5-dev20 with the following main changes :
- DOC: add missing options to the manpage
- DOC: add manpage references to all system calls
- DOC: update manpage reference to haproxy-en.txt
- DOC: remove -s and -l options from the manpage
- DOC: missing information for the "description" keyword
- DOC: missing http-send-name-header keyword in keyword table
- MINOR: tools: function my_memmem() to lookup binary contents
- MEDIUM: checks: add send/expect tcp based check
- MEDIUM: backend: Enhance hash-type directive with an algorithm options
- MEDIUM: backend: Implement avalanche as a modifier of the hashing functions.
- DOC: Documentation for hashing function, with test results.
- BUG/MEDIUM: ssl: potential memory leak using verifyhost
- BUILD: ssl: compilation issue with openssl v0.9.6.
- BUG/MINOR: ssl: potential memory leaks using ssl_c_key_alg or ssl_c_sig_alg.
- MINOR: ssl: optimization of verifyhost on wildcard certificates.
- BUG/MINOR: ssl: verifyhost does not match empty strings on wildcard.
- MINOR: ssl: Add statement 'verifyhost' to "server" statements
- CLEANUP: session: remove event_accept() which was not used anymore
- BUG/MINOR: deinit: free fdinfo while doing cleanup
- DOC: minor typo fix in documentation
- BUG/MEDIUM: server: set the macro for server's max weight SRV_UWGHT_MAX to SRV_UWGHT_RANGE
- BUG/MINOR: use the same check condition for server as other algorithms
- DOC: fix typo in comments
- BUG/MINOR: deinit: free server map which is allocated in init_server_map()
- CLEANUP: stream_interface: cleanup loop information in si_conn_send_loop()
- MINOR: buffer: align the last output line of buffer_dump()
- MINOR: buffer: align the last output line if there are less than 8 characters left
- DOC: stick-table: modify the description
- OPTIM: stream_interface: return directly if the connection flag CO_FL_ERROR has been set
- CLEANUP: code style: use tabs to indent codes
- DOC: checkcache: block responses with cacheable cookies
- BUG/MINOR: check_config_validity: check the returned value of stktable_init()
- MEDIUM: haproxy-systemd-wrapper: Use haproxy in same directory
- MEDIUM: systemd-wrapper: Kill child processes when interrupted
- LOW: systemd-wrapper: Write debug information to stdout
- BUG/MINOR: http: fix "set-tos" not working in certain configurations
- MEDIUM: http: add IPv6 support for "set-tos"
- DOC: ssl: update build instructions to use new SSL_* variables
- BUILD/MINOR: systemd: fix compiler warning about unused result
- url32+src - like base32+src but whole url including parameters
- BUG/MINOR: fix forcing fastinter in "on-error"
- CLEANUP: Make parameters of srv_downtime and srv_getinter const
- CLEANUP: Remove unused 'last_slowstart_change' field from struct peer
- MEDIUM: Split up struct server's check element
- MEDIUM: Move result element to struct check
- MEDIUM: Paramatise functions over the check of a server
- MEDIUM: cfgparse: Factor out check initialisation
- MEDIUM: Add state to struct check
- MEDIUM: Move health element to struct check
- MEDIUM: Add helper for task creation for checks
- MEDIUM: Add helper function for failed checks
- MEDIUM: Log agent fail, stopped or down as info
- MEDIUM: Remove option lb-agent-chk
- MEDIUM: checks: Add supplementary agent checks
- MEDIUM: Do not mark a server as down if the agent is unavailable
- MEDIUM: Set rise and fall of agent checks to 1
- MEDIUM: Add enable and disable agent unix socket commands
- MEDIUM: Add DRAIN state and report it on the stats page
- BUILD/MINOR: missing header file
- CLEANUP: regex: Create regex_comp function that compiles regex using compilation options
- CLEANUP: The function "regex_exec" needs the string length but in many case they expect null terminated char.
- MINOR: http: some exported functions were not in the header file
- MINOR: http: change url_decode to return the size of the decoded string.
- BUILD/MINOR: missing header file
- BUG/MEDIUM: sample: The function v4tov6 cannot support input and output overlap
- BUG/MINOR: arg: fix error reporting for add-header/set-header sample fetch arguments
- MINOR: sample: export the generic sample conversion parser
- MINOR: sample: export sample_casts
- MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()' into the ACL keyword
- MINOR: stick-table: use smp_expr_output_type() to retrieve the output type of a "struct sample_expr"
- MINOR: sample: provide the original sample_conv descriptor struct to the argument checker function.
- MINOR: tools: Add a function to convert buffer to an ipv6 address
- MINOR: acl: export acl arrays
- MINOR: acl: Extract the pattern parsing and indexation from the "acl_read_patterns_from_file()" function
- MINOR: acl: Extract the pattern matching function
- MINOR: sample: Define new struct sample_storage
- MEDIUM: acl: associate "struct sample_storage" to each "struct acl_pattern"
- REORG: acl/pattern: extract pattern matching from the acl file and create pattern.c
- MEDIUM: pattern: create pattern expression
- MEDIUM: pattern: rename "acl" prefix to "pat"
- MEDIUM: sample: let the cast functions set their output type
- MINOR: sample: add a private field to the struct sample_conv
- MINOR: map: Define map types
- MEDIUM: sample: add the "map" converter
- MEDIUM: http: The redirect strings follows the log format rules.
- BUG/MINOR: acl: acl parser does not recognize empty converter list
- BUG/MINOR: map: The map list was declared in the map.h file
- MINOR: map: Cleanup the initialisation of map descriptors.
- MEDIUM: map: merge identical maps
- BUG/MEDIUM: pattern: Pattern node has type of "struct pat_idx_elt" in place of "struct eb_node"
- BUG/MEDIUM: map: Bad map file parser
- CLEANUP/MINOR: standard: use the system define INET6_ADDRSTRLEN in place of MAX_IP6_LEN
- BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end
- MINOR: map: export map_get_reference() function
- MINOR: pattern: Each pattern sets the expected input type
- MEDIUM: acl: Last patch change the output type
- MEDIUM: pattern: Extract the index process from the pat_parse_*() functions
- MINOR: standard: The function parse_binary() can use preallocated buffer
- MINOR: regex: Change the struct containing regex
- MINOR: regex: Copy the original regex expression into string.
- MINOR: pattern: add support for compiling patterns for lookups
- MINOR: pattern: make the pattern matching function return a pointer to the matched element
- MINOR: map: export parse output sample functions
- MINOR: pattern: add function to lookup a specific entry in pattern list
- MINOR: pattern/map: Each pattern must free the associated sample
- MEDIUM: dumpstat: make the CLI parser understand the backslash as an escape char
- MEDIUM: map: dynamic manipulation of maps
- BUG/MEDIUM: unique_id: junk in log on empty unique_id
- BUG/MINOR: log: junk at the end of syslog packet
- MINOR: Makefile: provide cscope rule
- DOC: compression: chunk are not compressed anymore
- MEDIUM: session: disable lingering on the server when the client aborts
- BUG/MEDIUM: prevent gcc from moving empty keywords lists into BSS
- DOC: remove the comment saying that SSL certs are not checked on the server side
- BUG: counters: third counter was not stored if others unset
- BUG/MAJOR: http: don't emit the send-name-header when no server is available
- BUG/MEDIUM: http: "option checkcache" fails with the no-cache header
- BUG/MAJOR: http: sample prefetch code was not properly migrated
- BUG/MEDIUM: splicing: fix abnormal CPU usage with splicing
- BUG/MINOR: stream_interface: don't call chk_snd() on polled events
- OPTIM: splicing: use splice() for the last block when relevant
- MEDIUM: sample: handle comma-delimited converter list
- MINOR: sample: fix sample_process handling of unstable data
- CLEANUP: acl: move the 3 remaining sample fetches to samples.c
- MINOR: sample: add a new "date" fetch to return the current date
- MINOR: samples: add the http_date([<offset>]) sample converter.
- DOC: minor improvements to the part on the stats socket.
- MEDIUM: sample: systematically pass the keyword pointer to the keyword
- MINOR: payload: split smp_fetch_rdp_cookie()
- MINOR: counters: factor out smp_fetch_sc*_tracked
- MINOR: counters: provide a generic function to retrieve a stkctr for sc* and src.
- MEDIUM: counters: factor out smp_fetch_sc*_get_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_gpc0_rate
- MEDIUM: counters: factor out smp_fetch_sc*_inc_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_clr_gpc0
- MEDIUM: counters: factor out smp_fetch_sc*_conn_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_conn_rate
- MEDIUM: counters: factor out smp_fetch_sc*_conn_cur
- MEDIUM: counters: factor out smp_fetch_sc*_sess_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_sess_rate
- MEDIUM: counters: factor out smp_fetch_sc*_http_req_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_http_req_rate
- MEDIUM: counters: factor out smp_fetch_sc*_http_err_cnt
- MEDIUM: counters: factor out smp_fetch_sc*_http_err_rate
- MEDIUM: counters: factor out smp_fetch_sc*_kbytes_in
- MEDIUM: counters: factor out smp_fetch_sc*_bytes_in_rate
- MEDIUM: counters: factor out smp_fetch_sc*_kbytes_out
- MEDIUM: counters: factor out smp_fetch_sc*_bytes_out_rate
- MEDIUM: counters: factor out smp_fetch_sc*_trackers
- MINOR: session: make the number of stick counter entries more configurable
- MEDIUM: counters: support passing the counter number as a fetch argument
- MEDIUM: counters: support looking up a key in an alternate table
- MEDIUM: cli: adjust the method for feeding frequency counters in tables
- MINOR: cli: make it possible to enter multiple values at once with "set table"
- MINOR: payload: allow the payload sample fetches to retrieve arbitrary lengths
- BUG/MINOR: cli: "clear table" must not kill entries that don't match condition
- MINOR: ssl: use MAXPATHLEN instead of PATH_MAX
- MINOR: config: warn when a server with no specific port uses rdp-cookie
- BUG/MEDIUM: unique_id: HTTP request counter must be unique!
- DOC: add a mention about the limited chunk size
- BUG/MEDIUM: fix broken send_proxy on FreeBSD
- MEDIUM: stick-tables: flush old entries upon soft-stop
- MINOR: tcp: add new "close" action for tcp-response
- MINOR: payload: provide the "res.len" fetch method
- BUILD: add SSL_INC/SSL_LIB variables to force the path to openssl
- MINOR: http: compute response time before processing headers
- BUG/MINOR: acl: fix improper string size assignment in proxy argument
- BUG/MEDIUM: http: accept full buffers on smp_prefetch_http
- BUG/MINOR: acl: implicit arguments of ACL keywords were not properly resolved
- BUG/MEDIUM: session: risk of crash on out of memory conditions
- BUG/MINOR: peers: set the accept date in outgoing connections
- BUG/MEDIUM: tcp: do not skip tracking rules on second pass
- BUG/MEDIUM: acl: do not evaluate next terms after a miss
- MINOR: acl: add a warning when an ACL keyword is used without any value
- MINOR: tcp: don't use tick_add_ifset() when timeout is known to be set
- BUG/MINOR: acl: remove patterns from the tree before freeing them
- MEDIUM: backend: add support for the wt6 hash
- OPTIM/MEDIUM: epoll: fuse active events into polled ones during polling changes
- OPTIM/MINOR: mark the source address as already known on accept()
- BUG/MINOR: stats: don't count tarpitted connections twice
- CLEANUP: http: homogenize processing of denied req counter
- CLEANUP: http: merge error handling for req* and http-request *
- BUG/MEDIUM: http: fix possible parser crash when parsing erroneous "http-request redirect" rules
- BUG/MINOR: http: fix build warning introduced with url32/url32_src
- BUG/MEDIUM: checks: fix slow start regression after fix attempt
- BUG/MAJOR: server: weight calculation fails for map-based algorithms
- MINOR: stats: report correct throttling percentage for servers in slowstart
- OPTIM: connection: fold the error handling with handshake handling
- MINOR: peers: accept to learn strings of different lengths
- BUG/MAJOR: fix haproxy crash when using server tracking instead of checks
- BUG/MAJOR: check: fix haproxy crash during soft-stop/soft-start
- BUG/MINOR: stats: do not report "via" on tracking servers in maintenance
- BUG/MINOR: connection: fix typo in error message report
- BUG/MINOR: backend: fix target address retrieval in transparent mode
- BUG/MINOR: config: report the correct track-sc number in tcp-rules
- BUG/MINOR: log: fix log-format parsing errors
- DOC: add some information about how to apply converters to samples
- MINOR: acl/pattern: use types different from int to clarify who does what.
- MINOR: pattern: import acl_find_match_name() into pattern.h
- MEDIUM: stick-tables: support automatic conversion from ipv4<->ipv6
- MEDIUM: log-format: relax parsing of '%' followed by unsupported characters
- BUG/MINOR: http: usual deinit stuff in last commit
- BUILD: log: silent a warning about isblank() with latest patches
- BUG/MEDIUM: checks: fix health check regression causing them to depend on declaration order
- BUG/MEDIUM: checks: fix a long-standing issue with reporting connection errors
- BUG/MINOR: checks: don't consider errno and use conn->err_code
- BUG/MEDIUM: checks: also update the DRAIN state from the web interface
- MINOR: stats: remove some confusion between the DRAIN state and NOLB
- BUG/MINOR: tcp: check that no error is pending during a connect probe
- BUG/MINOR: connection: check EINTR when sending a PROXY header
- MEDIUM: connection: set the socket shutdown flags on socket errors
- BUG/MEDIUM: acl: fix regression introduced by latest converters support
- MINOR: connection: clear errno prior to checking for errors
- BUG/MINOR: checks: do not trust errno in write event before any syscall
- MEDIUM: checks: centralize error reporting
- OPTIM: checks: don't poll on recv when using plain TCP connects
- OPTIM: checks: avoid setting SO_LINGER twice
- MINOR: tools: add a generic binary hex string parser
- BUG/MEDIUM: checks: tcp-check: do not poll when there's nothing to send
- BUG/MEDIUM: check: tcp-check might miss some outgoing data when socket buffers are full
- BUG/MEDIUM: args: fix double free on error path in argument expression parser
- BUG/MINOR: acl: fix sample expression error reporting
- BUG/MINOR: checks: tcp-check actions are enums, not flags
- MEDIUM: checks: make tcp-check perform multiple send() at once
- BUG/MEDIUM: stick: completely remove the unused flag from the store entries
- OPTIM: ebtree: pack the struct eb_node to avoid holes on 64-bit
- BUG/MEDIUM: stick-tables: complete the latest fix about store-responses
- CLEANUP: stream_interface: remove unused field err_loc
- MEDIUM: stats: don't use conn->xprt_st anymore
- MINOR: session: add a simple function to retrieve a session from a task
- MEDIUM: stats: don't use conn->xprt_ctx anymore
- MEDIUM: peers: don't rely on conn->xprt_ctx anymore
- MINOR: http: prevent smp_fetch_url_{ip,port} from using si->conn
- MINOR: connection: make it easier to emit proxy protocol for unknown addresses
- MEDIUM: stats: prepare the HTTP stats I/O handler to support more states
- MAJOR: stats: move the HTTP stats handling to its applet
- MEDIUM: stats: move request argument processing to the final step
- MEDIUM: session: detect applets from the session by using s->target
- MAJOR: session: check for a connection to an applet in sess_prepare_conn_req()
- MAJOR: session: pass applet return traffic through the response analysers
- MEDIUM: stream-int: split the shutr/shutw functions between applet and conn
- MINOR: stream-int: make the shutr/shutw functions void
- MINOR: obj: provide a safe and an unsafe access to pointed objects
- MINOR: connection: add a field to store an object type
- MINOR: connection: always initialize conn->objt_type to OBJ_TYPE_CONN
- MEDIUM: stream interface: move the peers' ptr into the applet context
- MINOR: stream-interface: move the applet context to its own struct
- MINOR: obj: introduce a new type appctx
- MINOR: stream-int: rename ->applet to ->appctx
- MINOR: stream-int: split si_prepare_embedded into si_prepare_none and si_prepare_applet
- MINOR: stream-int: add a new pointer to the end point
- MEDIUM: stream-interface: set the pointer to the applet into the applet context
- MAJOR: stream interface: remove the ->release function pointer
- MEDIUM: stream-int: make ->end point to the connection or the appctx
- CLEANUP: stream-int: remove obsolete si_ctrl function
- MAJOR: stream-int: stop using si->conn and use si->end instead
- MEDIUM: stream-int: do not allocate a connection in parallel to applets
- MEDIUM: session: attach incoming connection to target on embryonic sessions
- MINOR: connection: add conn_init() to (re)initialize a connection
- MINOR: checks: call conn_init() to properly initialize the connection.
- MINOR: peers: make use of conn_init() to initialize the connection
- MINOR: session: use conn_init() to initialize the connections
- MINOR: http: use conn_init() to reinitialize the server connection
- MEDIUM: connection: replace conn_prepare with conn_assign
- MINOR: get rid of si_takeover_conn()
- MINOR: connection: add conn_new() / conn_free()
- MAJOR: connection: add two new flags to indicate readiness of control/transport
- MINOR: stream-interface: introduce si_reset() and si_set_state()
- MINOR: connection: reintroduce conn_prepare to set the protocol and transport
- MINOR: connection: replace conn_assign with conn_attach
- MEDIUM: stream-interface: introduce si_attach_conn to replace si_prepare_conn
- MAJOR: stream interface: dynamically allocate the outgoing connection
- MEDIUM: connection: move the send_proxy offset to the connection
- MINOR: connection: check for send_proxy during the connect(), not the SI
- MEDIUM: connection: merge the send_proxy and local_send_proxy calls
- MEDIUM: stream-int: replace occurrences of si->appctx with si_appctx()
- MEDIUM: stream-int: return the allocated appctx in stream_int_register_handler()
- MAJOR: stream-interface: dynamically allocate the applet context
- MEDIUM: session: automatically register the applet designated by the target
- MEDIUM: stats: delay appctx initialization
- CLEANUP: peers: use less confusing state/status code names
- MEDIUM: peers: delay appctx initialization
- MINOR: stats: provide some appctx information in "show sess all"
- DIET/MINOR: obj: pack the obj_type enum to 8 bits
- DIET/MINOR: connection: rearrange a few fields to save 8 bytes in the struct
- DIET/MINOR: listener: rearrange a few fields in struct listener to save 16 bytes
- DIET/MINOR: proxy: rearrange a few fields in struct proxy to save 16 bytes
- DIET/MINOR: session: reduce the struct session size by 8 bytes
- DIET/MINOR: stream-int: rearrange a few fields in struct stream_interface to save 8 bytes
- DIET/MINOR: http: reduce the size of struct http_txn by 8 bytes
- MINOR: http: switch the http state to an enum
- MINOR: http: use an enum for the auth method in http_auth_data
- DIET/MINOR: task: reduce struct task size by 8 bytes
- MINOR: stream_interface: add reporting of ressouce allocation errors
- MINOR: session: report lack of resources using the new stream-interface's error code
- BUILD: simplify the date and version retrieval in the makefile
- BUILD: prepare the makefile to skip format lines in SUBVERS and VERDATE
- BUILD: use format tags in VERDATE and SUBVERS files
- BUG/MEDIUM: channel: bo_getline() must wait for \n until buffer is full
- CLEANUP: check: server port is unsigned
- BUG/MEDIUM: checks: agent doesn't get the response if server does not closes
- MINOR: tools: buf2ip6 must not modify output on failure
- MINOR: pattern: do not assign SMP_TYPES by default to patterns
- MINOR: sample: make sample_parse_expr() use memprintf() to report parse errors
- MINOR: arg: improve wording on error reporting
- BUG/MEDIUM: sample: simplify and fix the argument parsing
- MEDIUM: acl: fix the argument parser to let the lower layer report detailed errors
- MEDIUM: acl: fix the initialization order of the ACL expression
- CLEANUP: acl: remove useless blind copy-paste from sample converters
- TESTS: add regression tests for ACL and sample expression parsers
- BUILD: time: adapt the type of TV_ETERNITY to the local system
- MINOR: chunks: allocate the trash chunks before parsing the config
- BUILD: definitely silence some stupid GCC warnings
- MINOR: chunks: always initialize the output chunk in get_trash_chunk()
- MINOR: checks: improve handling of the servers tracking chain
- REORG: checks: retrieve the check-specific defines from server.h to checks.h
- MINOR: checks: use an enum instead of flags to report a check result
- MINOR: checks: rename the state flags
- MINOR: checks: replace state DISABLED with CONFIGURED and ENABLED
- MINOR: checks: use check->state instead of srv->state & SRV_CHECKED
- MINOR: checks: fix agent check interval computation
- MINOR: checks: add a PAUSED state for the checks
- MINOR: checks: create the agent tasks even when no check is configured
- MINOR: checks: add a flag to indicate what check is an agent
- MEDIUM: checks: enable agent checks even if health checks are disabled
- BUG/MEDIUM: checks: ensure we can enable a server after boot
- BUG/MEDIUM: checks: tracking servers must not inherit the MAINT flag
- BUG/MAJOR: session: repair tcp-request connection rules
- BUILD: fix SUBVERS extraction in the Makefile
- BUILD: pattern: silence a warning about uninitialized value
- BUILD: log: fix build warning on Solaris
- BUILD: dumpstats: fix build error on Solaris
- DOC: move option pgsql-check to the correct place
- DOC: move option tcp-check to the proper place
- MINOR: connection: add simple functions to report connection readiness
- MEDIUM: connection: centralize handling of nolinger in fd management
- OPTIM: http: set CF_READ_DONTWAIT on response message
- OPTIM: http: do not re-enable reading on client side while closing the server side
- MINOR: config: add option http-keep-alive
- MEDIUM: connection: inform si_alloc_conn() whether existing conn is OK or not
- MAJOR: stream-int: handle the connection reuse in si_connect()
- MAJOR: http: add the keep-alive transition on the server side
- MAJOR: backend: enable connection reuse
- MINOR: http: add option prefer-last-server
- MEDIUM: http: do not report connection errors for second and further requests
When the load balancing algorithm in use is not deterministic, and a previous
request was sent to a server to which haproxy still holds a connection, it is
sometimes desirable that subsequent requests on a same session go to the same
server as much as possible. Note that this is different from persistence, as
we only indicate a preference which haproxy tries to apply without any form
of warranty. The real use is for keep-alive connections sent to servers. When
this option is used, haproxy will try to reuse the same connection that is
attached to the server instead of rebalancing to another server, causing a
close of the connection. This can make sense for static file servers. It does
not make much sense to use this in combination with hashing algorithms.
This new option enables HTTP keep-alive processing on the connections.
It can be overwritten by http-server-close, httpclose and forceclose.
Right now full-chain keep-alive is not yet implemented, but we need
the option to work on it. The doc will come later.
This patch adds map manipulation commands to the socket interface.
add map <map> <key> <value>
Add the value <value> in the map <map>, at the entry corresponding to
the key <key>. This command does not verify if the entry already
exists.
clear map <map>
Remove entries from the map <map>
del map <map> <key>
Delete all the map entries corresponding to the <key> value in the map
<map>.
set map <map> <key> <value>
Modify the value corresponding to each key <key> in a map <map>. The
new value is <value>.
show map [<map>]
Dump info about map converters. Without argument, the list of all
available maps are returned. If a <map> is specified, is content is
dumped.
This patch remove the compatibility check from the input type and the
match method. Now, it checks if a casts from the input type to output
type exists and the pattern_exec_match() function apply casts before
each pattern matching.
The commit 37e340c (BUG/MEDIUM: stick: completely remove the unused flag
from the store entries) was incomplete. We also need to ensure that only
the first store-response for a table is applied and that it may coexist
with a possible store-request that was already done on this table.
This patch with the previous one should be backported to 1.4.
This is a generic health check which can be used to match a
banner or send a request and analyse a server response.
It works in a send/expect ways and many exchange can be done between
HAProxy and a server to decide the server status, making HAProxy able to
speak the server's protocol.
It can send arbitrary regular or binary strings and match content as a
regular or binary string or a regex.
Signed-off-by: Baptiste Assmann <bedis9@gmail.com>
The stickiness table can be declared in such sections as frontend, listen
and backend, but the original manual only mentioned backend. Modify the
description simply as below:
"current backend" -> "current section"
Signed-off-by: Godbach <nylzhaowei@gmail.com>
We handle "http-request redirect" with a log-format string now, but we
leave "redirect" unaffected.
Note that the control of the special "/" case is move from the runtime
execution to the configuration parsing. If the format rule list is
empty, the build_logline() function does nothing.
At the moment when a '%' character is followed by any unhandled character,
it is considered as a variable name, and if it cannot be resolved, a warning
is emitted and the configuration goes on.
When we start using log-format for redirect rules, it may happen that some
people accidently use '%' instead of '%%' without understanding the cause
of the issue. Thus we do two things here :
- if a single '%' is followed by a blank or a digit, we fix it and emit a
warning explaining how this should be done ; this ensures that existing
configs continue to work ;
- if a single '%' is followed by an unknown variable name, we report it
and explain how to emit a verbatim '%' in case this is what the user
desired.
Add a new converter with the following prototype :
map(<map_file>[,<default_value>])
map_<match_type>(<map_file>[,<default_value>])
map_<match_type>_<output_type>(<map_file>[,<default_value>])
It searches the for input value from <map_file> using the <match_type>
matching method, and return the associated value converted to the type
<output_type>. If the input value cannot be found in the <map_file>,
the converter returns the <default_value>. If the <default_value> is
not set, the converter fails and acts as if no input value could be
fetched. If the <match_type> is not set, it defaults to "str".
Likewise, if the <output_type> is not set, it defaults to "str". For
convenience, the "map" keyword is an alias for "map_str" and maps a
string to another string. The following array contains contains the
list of all the map* converters.
+----+----------+---------+-------------+------------+
| `-_ out | | | |
| input `-_ | str | int | ip |
| / match `-_ | | | |
+---------------+---------+-------------+------------+
| str / str | map_str | map_str_int | map_str_ip |
| str / sub | map_sub | map_sub_int | map_sub_ip |
| str / dir | map_dir | map_dir_int | map_dir_ip |
| str / dom | map_dom | map_dom_int | map_dom_ip |
| str / end | map_end | map_end_int | map_end_ip |
| str / reg | map_reg | map_reg_int | map_reg_ip |
| int / int | map_int | map_int_int | map_int_ip |
| ip / ip | map_ip | map_ip_int | map_ip_ip |
+---------------+---------+-------------+------------+
The names are intentionally chosen to reflect the same match methods
as ACLs use.
We've had the feature for log-format, unique-id-format and add-header for
a while now. It has just been implemented for ACLs but some doc was still
lacking.
The syntax of this new commands are:
enable agent <backend>/<server>
disable agent <backend>/<server>
These commands allow temporarily stopping and subsequently
re-starting an auxiliary agent check. The effect of this is as follows:
New checks are only initialised when the agent is in the enabled. Thus,
disable agent will prevent any new agent checks from begin initiated until
the agent re-enabled using enable agent.
When an agent is disabled the processing of an auxiliary agent check that
was initiated while the agent was set as enabled is as follows: All
results that would alter the weight, specifically "drain" or a weight
returned by the agent, are ignored. The processing of agent check is
otherwise unchanged.
The motivation for this feature is to allow the weight changing effects
of the agent checks to be paused to allow the weight of a server to be
configured using set weight without being overridden by the agent.
Signed-off-by: Simon Horman <horms@verge.net.au>
In the case where agent-port is used and the agent
check is a secondary check to not mark a server as down
if the agent becomes unavailable.
In this configuration the agent should only cause a server to be marked
as down if the agent returns "fail", "stopped" or "down".
Signed-off-by: Simon Horman <horms@verge.net.au>
Allow an auxiliary agent check to be run independently of the
regular a regular health check. This is enabled by the agent-check
server setting.
The agent-port, which specifies the TCP port to use for the agent's
connections, is required.
The agent-inter, which specifies the interval between agent checks and
timeout of agent checks, is optional. If not set the value for regular
checks is used.
e.g.
server web1_1 127.0.0.1:80 check agent-port 10000
If either the health or agent check determines that a server is down
then it is marked as being down, otherwise it is marked as being up.
An agent health check performed by opening a TCP socket and reading an
ASCII string. The string should have one of the following forms:
* An ASCII representation of an positive integer percentage.
e.g. "75%"
Values in this format will set the weight proportional to the initial
weight of a server as configured when haproxy starts.
* The string "drain".
This will cause the weight of a server to be set to 0, and thus it
will not accept any new connections other than those that are
accepted via persistence.
* The string "down", optionally followed by a description string.
Mark the server as down and log the description string as the reason.
* The string "stopped", optionally followed by a description string.
This currently has the same behaviour as "down".
* The string "fail", optionally followed by a description string.
This currently has the same behaviour as "down".
Signed-off-by: Simon Horman <horms@verge.net.au>
Remove option lb-agent-chk and thus the facility to configure
a stand-alone agent health check. This feature was added by
"MEDIUM: checks: Add agent health check". It will be replaced
by subsequent patches with a features to allow an agent check
to be run as either a secondary check, along with any of the existing
checks, or as part of an http check with the status returned
in an HTTP header.
This patch does not entirely revert "MEDIUM: checks: Add agent health
check". The infrastructure it provides to parse the results of an
agent health check remains and will be re-used by the planned features
that are mentioned above.
Signed-off-by: Simon Horman <horms@verge.net.au>
Summary:
Added a document for hashing under internal docs explaining
hashing in haproxy along with the results of tests under the test
folder.
These documents together explain the motivation for adding
options for hashing algorithms with the option of enabling or
disabling of avalanche.
This function was designed for haproxy while testing other functions
in the past. Initially it was not planned to be used given the not
very interesting numbers it showed on real URL data : it is not as
smooth as the other ones. But later tests showed that the other ones
are extremely sensible to the server count and the type of input data,
especially DJB2 which must not be used on numeric input. So in fact
this function is still a generally average performer and it can make
sense to merge it in the end, as it can provide an alternative to
sdbm+avalanche or djb2+avalanche for consistent hashing or when hashing
on numeric data such as a source IP address or a visitor identifier in
a URL parameter.
Summary:
Avalanche is supported not as a native hashing choice, but a modifier
on the hashing function. Note that this means that possible configs
written after 1.5-dev4 using "hash-type avalanche" will get an informative
error instead. But as discussed on the mailing list it seems nobody ever
used it anyway, so let's fix it before the final 1.5 release.
The default values were selected for backward compatibility with previous
releases, as discussed on the mailing list, which means that the consistent
hashing will still apply the avalanche hash by default when no explicit
algorithm is specified.
Examples
(default) hash-type map-based
Map based hashing using sdbm without avalanche
(default) hash-type consistent
Consistent hashing using sdbm with avalanche
Additional Examples:
(a) hash-type map-based sdbm
Same as default for map-based above
(b) hash-type map-based sdbm avalanche
Map based hashing using sdbm with avalanche
(c) hash-type map-based djb2
Map based hashing using djb2 without avalanche
(d) hash-type map-based djb2 avalanche
Map based hashing using djb2 with avalanche
(e) hash-type consistent sdbm avalanche
Same as default for consistent above
(f) hash-type consistent sdbm
Consistent hashing using sdbm without avalanche
(g) hash-type consistent djb2
Consistent hashing using djb2 without avalanche
(h) hash-type consistent djb2 avalanche
Consistent hashing using djb2 with avalanche
Summary:
In testing at tumblr, we found that using djb2 hashing instead of the
default sdbm hashing resulted is better workload distribution to our backends.
This commit implements a change, that allows the user to specify the hash
function they want to use. It does not limit itself to consistent hashing
scenarios.
The supported hash functions are sdbm (default), and djb2.
For a discussion of the feature and analysis, see mailing list thread
"Consistent hashing alternative to sdbm" :
http://marc.info/?l=haproxy&m=138213693909219
Note: This change does NOT make changes to new features, for instance,
applying an avalance hashing always being performed before applying
consistent hashing.
