Commit Graph

1524 Commits

Author SHA1 Message Date
John Roesler
fb2fce1680 DOC: Fix typos and grammer in configuration.txt 2019-07-11 10:25:53 +02:00
Alain Belkadi
ac52095098 DOC: Fix typo in intro.txt 2019-07-06 11:41:37 +02:00
Frédéric Lécaille
1b9423d214 MINOR: server: Add "no-tfo" option.
Simple patch to add "no-tfo" option to "default-server" and "server" lines
to disable any usage of TCP fast open.

Must be backported to 2.0.
2019-07-04 14:45:52 +02:00
Tim Duesterhus
d437630237 MINOR: sample: Add sha2([<bits>]) converter
This adds a converter for the SHA-2 family, supporting SHA-224, SHA-256
SHA-384 and SHA-512.

The converter relies on the OpenSSL implementation, thus only being available
when HAProxy is compiled with USE_OPENSSL.

See GitHub issue #123. The hypothetical `ssl_?_sha256` fetch can then be
simulated using `ssl_?_der,sha2(256)`:

  http-response set-header Server-Cert-FP %[ssl_f_der,sha2(256),hex]
2019-06-17 13:36:42 +02:00
Tim Duesterhus
24915a55da MEDIUM: Remove 'option independant-streams'
It is deprecated with HAProxy 1.5. Time to remove it.
2019-06-17 13:35:54 +02:00
Tim Duesterhus
86e6b6ebf8 MEDIUM: Make '(cli|con|srv)timeout' directive fatal
They were deprecated with HAProxy 1.5. Time to remove them.
2019-06-17 13:35:54 +02:00
Tim Duesterhus
dac168bc15 MEDIUM: Make 'redispatch' directive fatal
It was deprecated with HAProxy 1.5. Time to remove it.
2019-06-17 13:35:54 +02:00
Tim Duesterhus
7b7c47f05c MEDIUM: Make 'block' directive fatal
It was deprecated with HAProxy 1.5. Time to remove it.
2019-06-17 13:35:54 +02:00
Willy Tarreau
9dc6b97429 [RELEASE] Released version 2.1-dev0
Released version 2.1-dev0 with the following main changes :
    - exact copy of 2.0.0
2019-06-16 21:49:47 +02:00
Willy Tarreau
ba23630ad0 [RELEASE] Released version 2.0.0
Released version 2.0.0 with the following main changes :
    - MINOR: fd: Don't use atomic operations when it's not needed.
    - DOC: mworker-prog: documentation for the program section
    - MINOR: http: add a new "http-request replace-uri" action
    - BUG/MINOR: 51d/htx: The _51d_fetch method, and the methods it calls are now HTX aware.
    - MINOR: 51d: Added dummy libraries for the 51Degrees module for testing.
    - MINOR: mworker: change formatting in uptime field of "show proc"
    - MINOR: mworker: add the HAProxy version in "show proc"
    - MINOR: doc: Remove -Ds option in man page
    - MINOR: doc: add master-worker in the man page
    - MINOR: doc: mention HAPROXY_LOCALPEER in the man
    - BUILD: Silence gcc warning about unused return value
    - CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real lib
    - BUILD: travis-ci: add 51Degree device detection, update openssl to 1.1.1c
    - MINOR: doc: update the manpage and usage message about -S
    - BUILD/MINOR: 51d: Updated build registration output to indicate thatif the library is a dummy one or not.
    - BUG/MEDIUM: h1: Don't wait for handshake if we had an error.
    - BUG/MEDIUM: h1: Wait for the connection if the handshake didn't complete.
    - BUG/MINOR: task: prevent schedulable tasks from starving under high I/O activity
    - BUG/MINOR: fl_trace/htx: Be sure to always forward trailers and EOM
    - BUG/MINOR: channel/htx: Call channel_htx_full() from channel_full()
    - BUG/MINOR: http: Use the global value to limit the number of parsed headers
    - BUG/MINOR: htx: Detect when tail_addr meet end_addr to maximize free rooms
    - BUG/MEDIUM: htx: Don't change position of the first block during HTX analysis
    - CLEANUP: channel: Remove channel_htx_fwd_payload() and channel_htx_fwd_all()
    - BUG/MEDIUM: proto_htx: Introduce the state ENDING during forwarding
    - MINOR: htx: Add 3 flags on the start-line to deal with the request schemes
    - MINOR: h2: Set flags about the request's scheme on the start-line
    - MINOR: mux-h1: Set flags about the request's scheme on the start-line
    - MINOR: mux-h2: Forward clients scheme to servers checking start-line flags
    - MEDIUM: server: server-state only rely on server name
    - CLEANUP: connection: rename the wait_event.task field to .tasklet
    - CLEANUP: tasks: rename task_remove_from_tasklet_list() to tasklet_remove_*
    - BUG/MEDIUM: connections: Don't call shutdown() if we want to disable linger.
    - DOC: add some environment variables in section 2.3
    - BUILD: makefile: clarify the "help" output and list options
    - BUG/MINOR: mux-h1: Wake busy mux for I/O when message is fully sent
    - BUG: tasks: fix bug introduced by latest scheduler cleanup
    - BUG/MEDIUM: mux-h2: fix early close with option abortonclose
    - BUG/MEDIUM: connections: Don't use ALPN to pick mux when in mode TCP.
    - BUG/MEDIUM: connections: Don't try to send early data if we have no mux.
    - BUG/MEDIUM: mux-h2: properly account for the appended data in HTX
    - BUILD: makefile: further clarify the "help" output and list targets
    - BUILD: makefile: rename "linux2628" to "linux-glibc" and remove older targets
    - BUILD: travis-ci: switch to linux-glibc instead of linux2628
    - DOC: update few references to the linux* targets and change them to linux-glibc
    - BUILD: makefile: detect and reject recently removed linux targets
    - BUILD: makefile: enable linux namespaces by default on linux
    - BUILD: makefile: enable TFO on linux platforms
    - BUILD: makefile: enable getaddrinfo on the linux-glibc target
    - DOC: small updates to the CONTRIBUTING file
    - BUG/MEDIUM: ssl: Make sure we initiate the handshake after using early data.
    - CLEANUP: removed obsolete examples an move a few to better places
    - DOC: Fix typos in CONTRIBUTING
    - DOC: update the outdated ROADMAP file
    - DOC: create a BRANCHES file to explain the life cycle
    - DOC: mention in INSTALL haproxy 2.0 is a long-term supported stable version
    - BUILD: travis-ci: TFO and GETADDRINFO are now enabled by default
    - BUILD: makefile: make the obsolete target detection compatible with make-3.80
    - BUILD: tools: work around an internal compiler bug in gcc-3.4
    - BUILD: pattern: work around an internal compiler bug in gcc-3.4
    - BUILD: makefile: enable USE_RT on Solaris
    - BUILD: makefile: do not use echo -n
    - DOC: mention a few common build errors in the INSTALL file
2019-06-16 20:00:26 +02:00
Willy Tarreau
a8ee4b199f CLEANUP: removed obsolete examples an move a few to better places
The following example files awere removed as irrelevant by this
time :
  auth.cfg check.conf ssl.cfg haproxy.spec

The following scripts were removed as having been unused for more
than a decade :
  debug2ansi debug2html debugfind check init.haproxy stats_haproxy.sh

seemless_reload.txt was moved to doc/ where it's more suitable.

haproxy.vim was moved to contrib/syntax-highlight/

scripts/create-release was updated not to try to update haproxy.spec
anymore.
2019-06-15 21:25:06 +02:00
Willy Tarreau
d254aa8139 DOC: update few references to the linux* targets and change them to linux-glibc
The INSTALL guide, the Lua doc and the Prometheus exporter's README all
used to reference "linux2628", "linux26" or even "linux". These were all
updated to consistently reflect "linux-glibc" instead. The default options
were updated there as well so that it should build cleanly on most distros.
2019-06-15 18:03:48 +02:00
William Lallemand
4d03e431b3 DOC: add some environment variables in section 2.3
Add the missing environment variable in the 2.3 section.
2019-06-14 15:38:05 +02:00
William Lallemand
63329e36ab MINOR: doc: update the manpage and usage message about -S
Add -S in the manpage, and update the usage message.

Should be backported to 1.9.
2019-06-13 17:09:27 +02:00
Willy Tarreau
5e4c5003c5 CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real lib
This way the directory structure remains the same as with the real lib and
one can apply the same build options regardless of where the lib is stored,
removing any possible confusion.
2019-06-13 15:56:10 +02:00
William Lallemand
c5473e507e MINOR: doc: mention HAPROXY_LOCALPEER in the man
vMention the HAPROXY_LOCALPEER environment variable in the -L argument
of the manpage.

Should be backported in 1.9.
2019-06-13 15:39:48 +02:00
William Lallemand
95635ddac8 MINOR: doc: add master-worker in the man page
Add some information about the master-worker in the man page.

Should be backported in every version since 1.8.
2019-06-13 15:32:39 +02:00
Kazuo Yagi
971c3943be MINOR: doc: Remove -Ds option in man page
Remove -Ds option in man page.

Should be backported in every version since 1.8.
2019-06-13 11:22:08 +02:00
William Lallemand
1dc6963086 MINOR: mworker: add the HAProxy version in "show proc"
Displays the HAProxy version so you can compare the version of old
processes and new ones.
2019-06-12 19:19:57 +02:00
Ben51Degrees
31c3d51a18 MINOR: 51d: Added dummy libraries for the 51Degrees module for testing.
These are intended for use by HAProxy developers to ensure any changes
did not affect the 51Degrees implementation. The 51Degrees module can be
enabled and used by using the source in contrib/51d. This will run
without breaking, but will not return any meaningful information.

This is ideal for testing HAProxy core code, and other modules alongside
51Degrees, but should never be used as an actual module as it does
nothing.
2019-06-12 18:06:59 +02:00
Willy Tarreau
3381022d88 MINOR: http: add a new "http-request replace-uri" action
This action is particularly convenient to replace some deprecated usees
of "reqrep". It takes a match and a format string including back-
references. The reqrep warning was updated to suggest it as well.
2019-06-12 18:06:59 +02:00
William Lallemand
c9515529a0 DOC: mworker-prog: documentation for the program section
This patch documents the program feature.
2019-06-12 18:06:59 +02:00
Willy Tarreau
ca3551f005 [RELEASE] Released version 2.0-dev7
Released version 2.0-dev7 with the following main changes :
    - BUG/MEDIUM: mux-h2: make sure the connection timeout is always set
    - MINOR: tools: add new bitmap manipulation functions
    - MINOR: logs: use the new bitmap functions instead of fd_sets for encoding maps
    - MINOR: chunks: Make sure trash_size is only set once.
    - Revert "MINOR: chunks: Make sure trash_size is only set once."
    - MINOR: threads: serialize threads initialization
    - MINOR peers: data structure simplifications for server names dictionary cache.
    - DOC: peers: Update for dictionary cache entries for peers protocol.
    - MINOR: dict: Store the length of the dictionary entries.
    - MINOR: peers: A bit of optimization when encoding cached server names.
    - MINOR: peers: Optimization for dictionary cache lookup.
    - MEDIUM: tools: improve time format error detection
    - BUG/MEDIUM: H1: When upgrading, make sure we don't free the buffer too early.
    - BUG/MEDIUM: stream_interface: Make sure we call si_cs_process() if CS_FL_EOI.
    - MINOR: threads: avoid clearing harmless twice in thread_release()
    - MEDIUM: threads: add thread_sync_release() to synchronize steps
    - BUG/MEDIUM: init/threads: prevent initialized threads from starting before others
    - OPTIM/MINOR: init/threads: only call protocol_enable_all() on first thread
    - BUG/MINOR: dict: race condition fix when inserting dictionary entries.
    - MEDIUM: init/threads: don't use spinlocks during the init phase
    - BUG/MINOR: cache/htx: Fix the counting of data already sent by the cache applet
    - BUG/MEDIUM: compression/htx: Fix the adding of the last data block
    - MINOR: flt_trace: Don't scrash the original offset during the random forwarding
    - MAJOR: htx: Rework how free rooms are tracked in an HTX message
    - MINOR: htx: Add the function htx_move_blk_before()
    - Revert "BUG/MEDIUM: H1: When upgrading, make sure we don't free the buffer too early."
    - BUG/MINOR: http-rules: mention "deny_status" for "deny" in the error message
    - MINOR: http: turn default error files to HTTP/1.1
    - BUG/MEDIUM: h1: Don't try to subscribe if we had a connection error.
    - BUG/MEDIUM: h1: Don't consider we're connected if the handshake isn't done.
    - MINOR: contrib/spoa_server: Upgrade SPOP to 2.0
    - BUG/MEDIUM: contrib/spoa_server: Set FIN flag on agent frames
    - MINOR: contrib/spoa_server: Add random IP score
    - DOC/MINOR: contrib/spoa_server: Fix typo in README
2019-06-11 19:28:00 +02:00
Frdric Lcaille
fdfa9e39ed DOC: peers: Update for dictionary cache entries for peers protocol.
Add information about how the peers protocol send/receive entries of
LRU caches for literal dictionaries (e.g. server names in replacement
for server IDs).
2019-06-07 15:47:54 +02:00
Willy Tarreau
b57f109966 [RELEASE] Released version 2.0-dev6
Released version 2.0-dev6 with the following main changes :
    - BUG/MEDIUM: connection: fix multiple handshake polling issues
    - MINOR: connection: also stop receiving after a SOCKS4 response
    - MINOR: mux-h1: don't try to recv() before the connection is ready
    - BUG/MEDIUM: mux-h1: only check input data for the current stream, not next one
    - MEDIUM: mux-h1: don't use CS_FL_REOS anymore
    - CLEANUP: connection: remove the now unused CS_FL_REOS flag
    - CONTRIB: debug: add 4 missing connection/conn_stream flags
    - MEDIUM: stream: make a full process_stream() loop when completing I/O on exit
    - MINOR: server: increase the default pool-purge-delay to 5 seconds
    - BUILD: tools: do not use the weak attribute for trace() on obsolete linkers
    - BUG/MEDIUM: vars: make sure the scope is always valid when accessing vars
    - BUG/MEDIUM: vars: make the tcp/http unset-var() action support conditions
    - BUILD: task: fix a build warning when threads are disabled
    - CLEANUP: peers: Remove tabs characters.
    - CLEANUP: peers: Replace hard-coded values by macros.
    - BUG/MINOR: peers: Wrong stick-table update message building.
    - MINOR: dict: Add dictionary new data structure.
    - MINOR: peers: Add a LRU cache implementation for dictionaries.
    - MINOR: stick-table: Add "server_name" new data type.
    - MINOR: cfgparse: Space allocation for "server_name" stick-table data type.
    - MINOR: proxy: Add a "server by name" tree to proxy.
    - MINOR: server: Add a dictionary for server names.
    - MINOR: stream: Stickiness server lookup by name.
    - MINOR: peers: Make peers protocol support new "server_name" data type.
    - MINOR: stick-table: Make the CLI stick-table handler support dictionary entry data type.
    - REGTEST: Add a basic server by name stickiness reg test.
    - MINOR: peers: Add dictionary cache information to "show peers" CLI command.
    - MINOR: peers: Replace hard-coded for peer protocol 64-bits value encoding by macros.
    - MINOR: peers: Replace hard-coded values for peer protocol messaging by macros.
    - CLEANUP: ssl: remove unneeded defined(OPENSSL_IS_BORINGSSL)
    - BUILD: travis-ci improvements
    - MINOR: SSL: add client/server random sample fetches
    - BUG/MINOR: channel/htx: Don't alter channel during forward for empty HTX message
    - BUG/MINOR: contrib/prometheus-exporter: Add HTX data block in one time
    - BUG/MINOR: mux-h1: errflag must be set on H1S and not H1M during output processing
    - MEDIUM: mux-h1: refactor output processing
    - MINOR: mux-h1: Add the flag HAVE_O_CONN on h1s
    - MINOR: mux-h1: Add h1_eval_htx_hdrs_size() to estimate size of the HTX headers
    - MINOR: mux-h1: Don't count the EOM in the estimated size of headers
    - MEDIUM: cache/htx: Always store info about HTX blocks in the cache
    - MEDIUM: htx: Add the parsing of trailers of chunked messages
    - MINOR: htx: Don't use end-of-data blocks anymore
    - BUG/MINOR: mux-h1: Don't send more data than expected
    - BUG/MINOR: flt_trace/htx: Only apply the random forwarding on the message body.
    - BUG/MINOR: peers: Wrong "server_name" decoding.
    - BUG/MEDIUM: servers: Don't attempt to destroy idle connections if disabled.
    - MEDIUM: checks: Make sure we unsubscribe before calling cs_destroy().
    - MEDIUM: connections: Wake the upper layer even if sending/receiving is disabled.
    - MEDIUM: ssl: Handle subscribe by itself.
    - MINOR: ssl: Make ssl_sock_handshake() static.
    - MINOR: connections: Add a new xprt method, remove_xprt.
    - MINOR: connections: Add a new xprt method, add_xprt().
    - MEDIUM: connections: Introduce a handshake pseudo-XPRT.
    - MEDIUM: connections: Remove CONN_FL_SOCK*
    - BUG/MEDIUM: ssl: Don't forget to initialize ctx->send_recv and ctx->recv_wait.
    - BUG/MINOR: peers: Wrong server name parsing.
    - MINOR: server: really increase the pool-purge-delay default to 5 seconds
    - BUG/MINOR: stream: don't emit a send-name-header in conn error or disconnect states
    - MINOR: stream-int: use bit fields to match multiple stream-int states at once
    - MEDIUM: stream-int: remove dangerous interval checks for stream-int states
    - MEDIUM: stream-int: introduce a new state SI_ST_RDY
    - MAJOR: stream-int: switch from SI_ST_CON to SI_ST_RDY on I/O
    - MEDIUM: stream-int: make idle-conns switch to ST_RDY
    - MEDIUM: stream: re-arrange the connection setup status reporting
    - MINOR: stream-int: split si_update() into si_update_rx() and si_update_tx()
    - MINOR: stream-int: make si_sync_send() from the send code of si_update_both()
    - MEDIUM: stream: rearrange the events to remove the loop
    - MEDIUM: stream: only loop on flags relevant to the analysers
    - MEDIUM: stream: don't abusively loop back on changes on CF_SHUT*_NOW
    - BUILD: stream-int: avoid a build warning in dev mode in si_state_bit()
    - BUILD: peers: fix a build warning about an incorrect intiialization
    - BUG/MINOR: time: make sure only one thread sets global_now at boot
    - BUG/MEDIUM: tcp: Make sure we keep the polling consistent in tcp_probe_connect.
2019-06-07 06:12:59 +02:00
Willy Tarreau
4f283fa604 MEDIUM: stream-int: introduce a new state SI_ST_RDY
The main reason for all the trouble we're facing with stream interface
error or timeout reports during the connection phase is that we currently
can't make the difference between a connection attempt and a validated
connection attempt. It is problematic because we tend to switch early
to SI_ST_EST but can't always do what we want in this state since it's
supposed to be set when we don't need to visit sess_establish() again.

This patch introduces a new state betwen SI_ST_CON and SI_ST_EST, which
is SI_ST_RDY. It indicates that we've verified that the connection is
ready. It's a transient state, like SI_ST_DIS, that cannot persist when
leaving process_stream(). For now it is not set, only verified in various
tests where SI_ST_CON was used or SI_ST_EST depending on the cases.

The stream-int state diagram was minimally updated to reflect the new
state, though it is largely obsolete and would need to be seriously
updated.
2019-06-06 16:36:19 +02:00
Patrick Hemmer
65674662b4 MINOR: SSL: add client/server random sample fetches
This adds 4 sample fetches:
- ssl_fc_client_random
- ssl_fc_server_random
- ssl_bc_client_random
- ssl_bc_server_random

These fetches retrieve the client or server random value sent during the
handshake.

Their use is to be able to decrypt traffic sent using ephemeral ciphers. Tools
like wireshark expect a TLS log file with lines in a few known formats
(https://code.wireshark.org/review/gitweb?p=wireshark.git;a=blob;f=epan/dissectors/packet-tls-utils.c;h=28a51fb1fb029eae5cea52d37ff5b67d9b11950f;hb=HEAD#l5209).
Previously the only format supported using data retrievable from HAProxy state
was the one utilizing the Session-ID. However an SSL/TLS session ID is
optional, and thus cannot be relied upon for this purpose.

This change introduces the ability to extract the client random instead which
can be used for one of the other formats. The change also adds the ability to
extract the server random, just in case it might have some other use, as the
code change to support this was trivial.
2019-06-05 10:07:44 +02:00
Willy Tarreau
fb55365f9e MINOR: server: increase the default pool-purge-delay to 5 seconds
The default used to be a very aggressive delay of 1 second before starting
to purge idle connections, but tests show that with bursty traffic it's a
bit short. Let's increase this to 5 seconds.
2019-06-04 14:06:31 +02:00
Willy Tarreau
abc874ea45 [RELEASE] Released version 2.0-dev5
Released version 2.0-dev5 with the following main changes :
    - BUILD: watchdog: use si_value.sival_int, not si_int for the timer's value
    - BUILD: signals: FreeBSD has SI_LWP instead of SI_TKILL
    - BUILD: watchdog: condition it to USE_RT
    - MINOR: raw_sock: report global traffic statistics
    - MINOR: stats: report the global output bit rate in human readable form
    - BUG/MINOR: proto-htx: Try to keep connections alive on redirect
    - BUG/MEDIUM: spoe: Don't use the SPOE applet after releasing it
    - BUG/MINOR: lua: Set right direction and flags on new HTTP objects
    - BUG/MINOR: mux-h2: Count EOM in bytes sent when a HEADERS frame is formatted
    - BUG/MINOR: mux-h1: Report EOI instead EOS on parsing error or H2 upgrade
    - BUG/MEDIUM: proto-htx: Not forward too much data when 1xx reponses are handled
    - BUG/MINOR: htx: Remove a forgotten while loop in htx_defrag()
    - DOC: fix typos
    - BUG/MINOR: ssl_sock: Fix memory leak when disabling compression
    - OPTIM: freq-ctr: don't take the date lock for most updates
    - MEDIUM: mux-h2: avoid doing expensive buffer realigns when not absolutely needed
    - CLEANUP: debug: remove the TRACE() macro
    - MINOR: buffer: introduce b_make() to make a buffer from its parameters
    - MINOR: buffer: add a new buffer ring API to manipulate rings of buffers
    - MEDIUM: mux-h2: replace all occurrences of mbuf with a buffer ring
    - MEDIUM: mux-h2: make the conditions to send based on mbuf, not just its tail
    - MINOR: mux-h2: introduce h2_release_mbuf() to release all buffers in the mbuf ring
    - MEDIUM: mux-h2: make the send() function iterate over all mux buffers
    - CLEANUP: mux-h2: consistently use a local variable for the mbuf
    - MINOR: mux-h2: report the mbuf's head and tail in "show fd"
    - MAJOR: mux-h2: switch to next mux buffer on buffer full condition.
    - BUILD: connections: shut up gcc about impossible out-of-bounds warning
    - BUILD: ssl: fix latest LibreSSL reg-test error
    - MINOR: cli/activity: remove "fd_del" and "fd_skip" from show activity
    - MINOR: cli/activity: add 3 general purpose counters in development mode
    - BUG/MAJOR: lb/threads: make sure the avoided server is not full on second pass
    - BUG/MEDIUM: queue: fix the tree walk in pendconn_redistribute.
    - BUG/MEDIUM: threads: fix double-word CAS on non-optimized 32-bit platforms
    - MEDIUM: config: now alert when two servers have the same name
    - MINOR: htx: Remove the macro IS_HTX_SMP() and always use IS_HTX_STRM() instead
    - MINOR: htx: Move the macro IS_HTX_STRM() in proto/stream.h
    - MINOR: htx: Store the head position instead of the wrap one
    - MINOR: htx: Store start-line block's position instead of address of its payload
    - MINOR: htx: Add functions to get the first block of an HTX message
    - MINOR: mux-h2/htx: Get the start-line from the head when HEADERS frame is built
    - MINOR: htx: Replace the function http_find_stline() by http_get_stline()
    - CLEANUP: htx: Remove unused function htx_get_stline()
    - MINOR: http/htx: Use sl_pos directly to replace the start-line
    - MEDIUM: http/htx: Perform analysis relatively to the first block
    - MINOR: channel/htx: Call channel_htx_recv_max() from channel_recv_max()
    - MINOR: htx: Add function htx_get_max_blksz()
    - BUG/MINOR: htx: Change htx_xfer_blk() to also count metadata
    - MEDIUM: mux-h1: Use the count value received from the SI in h1_rcv_buf()
    - MINOR: mux-h2: Use the count value received from the SI in h2_rcv_buf()
    - MINOR: stream-int: Don't use the flag CO_RFL_KEEP_RSV anymore in si_cs_recv()
    - MINOR: connection: Remove the unused flag CO_RFL_KEEP_RSV
    - MINOR: mux-h2/htx: Support zero-copy when possible in h2_rcv_buf()
    - MINOR: htx: Add a field to set the memory used by headers in the HTX start-line
    - MINOR: h2/htx: Set hdrs_bytes on the SL when an HTX message is produced
    - MINOR: mux-h1: Set hdrs_bytes on the SL when an HTX message is produced
    - MINOR: htx: Be sure to xfer all headers in one time in htx_xfer_blks()
    - MEDIUM: htx: 1xx messages are now part of the final reponses
    - MINOR: channel/htx: Add function to forward headers of an HTX message
    - MINOR: filters/htx: Use channel_htx_fwd_headers() after headers filtering
    - MINOR: proto-htx: Use channel_htx_fwd_headers() to forward 1xx responses
    - MEDIUM: htx: Store the first block position instead of the start-line one
    - MINOR: stats/htx: don't use the first block position but the head one
    - MINOR: channel/htx: Add functions to forward a part or all HTX payload
    - MINOR: proto-htx: Use channel_htx_fwd_all() when unfiltered body are forwarded
    - MEDIUM: filters/htx: Filter body relatively to the first block
    - MINOR: htx: Optimize htx_drain() when all data are drained
    - MINOR: htx: don't rely on htx_find_blk() anymore in the function htx_truncate()
    - MINOR: htx: remove the unused function htx_find_blk()
    - MINOR: htx: Remove support of pseudo headers because it is unused
    - BUG/MEDIUM: http: fix "http-request reject" when not final
    - MINOR: ssl: Make sure the underlying xprt's init method doesn't fail.
    - MINOR: ssl: Don't forget to call the close method of the underlying xprt.
    - MINOR: htx: rename htx_append_blk_value() to htx_add_data_atonce()
    - MINOR: htx: make htx_add_data() return the transmitted byte count
    - MEDIUM: htx: make htx_add_data() never defragment the buffer
    - MINOR: activity: write totals on the "show activity" output
    - MINOR: activity: report totals and average separately
    - MEDIUM: poller: separate the wait time from the wake events
    - MINOR: activity: report the number of failed pool/buffer allocations
    - MEDIUM: buffers: relax the buffer lock a little bit
    - MINOR: task: turn the WQ lock to an RW_LOCK
    - MEDIUM: task: don't grab the WR lock just to check the WQ
    - BUG/MEDIUM: mux-h1: Don't skip the TCP splicing when there is no more data to read
    - MEDIUM: sessions: Introduce session flags.
    - BUG/MEDIUM: h2: Don't forget to set h2s->cs to NULL after having free'd cs.
    - BUG/MEDIUM: mux-h2: fix the conditions to end the h2_send() loop
    - BUG/MEDIUM: mux-h2: don't refrain from offering oneself a used buffer
    - BUG/MEDIUM: connection: Use the session to get the origin address if needed.
    - MEDIUM: tasks: Get rid of active_tasks_mask.
    - MEDIUM: connection: Upstream SOCKS4 proxy support
    - BUILD: contrib/prometheus: fix build breakage caused by move of idle_pct
    - BUG/MINOR: deinit/threads: make hard-stop-after perform a clean exit
2019-06-02 12:06:08 +02:00
Alexander Liu
2a54bb74cd MEDIUM: connection: Upstream SOCKS4 proxy support
Have "socks4" and "check-via-socks4" server keyword added.
Implement handshake with SOCKS4 proxy server for tcp stream connection.
See issue #82.

I have the "SOCKS: A protocol for TCP proxy across firewalls" doc found
at "https://www.openssh.com/txt/socks4.protocol". Please reference to it.

[wt: for now connecting to the SOCKS4 proxy over unix sockets is not
 supported, and mixing IPv4/IPv6 is discouraged; indeed, the control
 layer is unique for a connection and will be used both for connecting
 and for target address manipulation. As such it may for example report
 incorrect destination addresses in logs if the proxy is reached over
 IPv6]
2019-05-31 17:24:06 +02:00
Michael Prokop
4438c6061d DOC: fix typos
s/accidently/accidentally/
s/any ot these messages/any of theses messages/
s/catched/caught/
s/completly/completely/
s/convertor/converter/
s/desribing/describing/
s/developper/developer/
s/eventhough/even though/
s/exectution/execution/
s/functionnality/functionality/
s/If it receive a/If it receives a/
s/In can even/It can even/
s/informations/information/
s/it will be remove /it will be removed /
s/langage/language/
s/mentionned/mentioned/
s/negociated/negotiated/
s/Optionnaly/Optionally/
s/ouputs/outputs/
s/outweights/outweighs/
s/ressources/resources/
2019-05-25 07:34:24 +02:00
Willy Tarreau
567406949b [RELEASE] Released version 2.0-dev4
Released version 2.0-dev4 with the following main changes :
    - BUILD: enable freebsd builds on cirrus-ci
    - BUG/MINOR: http_fetch: Rely on the smp direction for "cookie()" and "hdr()"
    - MEDIUM: Make 'option forceclose' actually warn
    - MEDIUM: Make 'resolution_pool_size' directive fatal
    - DOC: management: place "show activity" at the right place
    - MINOR: cli/activity: show the dumping thread ID starting at 1
    - MINOR: task: export global_task_mask
    - MINOR: cli/debug: add a thread dump function
    - BUG/MEDIUM: streams: Don't use CF_EOI to decide if the request is complete.
    - BUG/MEDIUM: streams: Try to L7 retry before aborting the connection.
    - BUG/MINOR: debug: make ha_task_dump() always check the task before dumping it
    - BUG/MINOR: debug: make ha_task_dump() actually dump the requested task
    - MINOR: debug: make ha_thread_dump() and ha_task_dump() take a buffer
    - BUG/MINOR: debug: don't check the call date on tasklets
    - MINOR: thread: implement ha_thread_relax()
    - MINOR: task: put barriers after each write to curr_task
    - MINOR: task: always reset curr_task when freeing a task or tasklet
    - MINOR: stream: detach the stream from its own task on stream_free()
    - MEDIUM: debug/threads: implement an advanced thread dump system
    - REGTEST: extend the check duration on tls_health_checks and mark it slow
    - DOC: fix "successful" typo
    - MINOR: init: setenv HAPROXY_CFGFILES
    - MINOR: threads/init: synchronize the threads startup
    - MEDIUM: init/mworker: make the pipe register function a regular initcall
    - CLEANUP: memory: make the fault injection code use the OTHER_LOCK label
    - CLEANUP: threads: remove the now unused START_LOCK label
    - MINOR: init/threads: make the global threads an array of structs
    - MINOR: threads: add each thread's clockid into the global thread_info
    - CLEANUP: stream: remove an obsolete debugging test
    - MINOR: tools: add dump_hex()
    - MINOR: debug: implement ha_panic()
    - MINOR: debug/cli: add some debugging commands for developers
    - MINOR: tools: provide a may_access() function and make dump_hex() use it
    - MINOR: debug: make ha_panic() report threads starting at 1
    - REORG: compat: move some integer limit definitions from standard.h to compat.h
    - REORG: threads: move the struct thread_info from global.h to hathreads.h
    - MINOR: compat: make sure to always define clockid_t
    - MINOR: threads: always place the clockid in the struct thread_info
    - MINOR: threads: add a thread-local thread_info pointer "ti"
    - MINOR: time: move the cpu, mono, and idle time to thread_info
    - MINOR: time: add a function to retrieve another thread's cputime
    - MINOR: debug: report each thread's cpu usage in "show thread"
    - BUILD: threads: only assign the clock_id when supported
    - BUILD: makefile: use USE_OBSOLETE_LINKER for solaris
    - BUILD: makefile: remove -fomit-frame-pointer optimisation (solaris)
    - MAJOR: polling: add event ports support (Solaris)
    - BUG/MEDIUM: streams: Don't switch from SI_ST_CON to SI_ST_DIS on read0.
    - CLEANUP: time: refine the test on _POSIX_TIMERS
    - MINOR: compat: define a new empty type empty_t for non-implemented fields
    - CLEANUP: time: switch clockid_t to empty_t when not available
    - BUG/MINOR: mworker: Fix memory leak of mworker_proc members
    - CLEANUP: objtype: make obj_type() and obj_type_name() take consts
    - MINOR: debug: switch to SIGURG for thread dumps
    - CLEANUP: threads: really move thread_info to hathreads.c
    - MINOR: threads: make threads_{harmless|want_rdv}_mask constant 0 without threads
    - CLEANUP: debug: always report harmless/want_rdv even without threads
    - MINOR: threads: implement ha_tkill() and ha_tkillall()
    - CLEANUP: debug: make use of ha_tkill() and remove ifdefs
    - MINOR: stream: introduce a stream_dump() function and use it in stream_dump_and_crash()
    - MINOR: debug: dump streams when an applet, iocb or stream is known
    - MINOR: threads: add a "stuck" flag to the thread_info struct
    - MINOR: threads: add a timer_t per thread in thread_info
    - MAJOR: watchdog: implement a thread lockup detection mechanism
    - MINOR: stream: remove the cpu time detection from process_stream()
    - MINOR: connection: report the mux names in "haproxy -vv"
    - CLEANUP: mux-h1: use "H1" and not "h1" as the mux's name
    - BUG/MEDIUM: WURFL: segfault in wurfl-get() with missing info.
    - MINOR: WURFL: call header_retireve_callback() in dummy library
    - MINOR: WURFL: fixed Engine load failed error when wurfl-information-list contains wurfl_root_id
    - MINOR: WURFL: shows log messages during module initialization
    - MINOR: WURFL: removes heading wurfl-information-separator from wurfl-get-all() and wurfl-get() results
    - MINOR: WURFL: wurfl_get() and wurfl_get_all() now return an empty string if device detection fails
    - MEDIUM: WURFL: HTX awareness.
    - MINOR: WURFL: module version bump to 2.0
    - MINOR: WURFL: do not emit warnings when not configured
    - CONTRIB: wurfl: address 3 build issues in the wurfl dummy library
    - BUG/MEDIUM: init/threads: provide per-thread alloc/free function callbacks
    - BUILD: travis: add sanitizers to travis-ci builds
    - BUILD: time: remove the test on _POSIX_C_SOURCE
    - CLEANUP: build: rename some build macros to use the USE_* ones
    - CLEANUP: raw_sock: remove support for very old linux splice bug workaround
    - BUG/MEDIUM: dns: make the port numbers unsigned
    - MEDIUM: config: deprecate the antique req* and rsp* commands
2019-05-22 20:48:33 +02:00
Willy Tarreau
96d5195862 MEDIUM: config: deprecate the antique req* and rsp* commands
These commands don't follow the same flow as the rest of the commands,
each of them iterates over all header lines before switching to the
next directive. In addition they make no distinction between start
line and headers and can lead to unparsable rewrites which are very
difficult to deal with internally.

Most of them are still occasionally found in configurations, mainly
because of the usual "we've always done this way". By marking them
deprecated and emitting a warning and recommendation on first use of
each of them, we will raise users' awareness of users regarding the
cleaner, faster and more reliable alternatives.

Some use cases of "reqrep" still appear from time to time for URL
rewriting that is not so convenient with other rules. But at least
users facing this requirement will explain their use case so that we
can best serve them. Some discussion started on this subject in a
thread linked to from github issue #100.

The goal is to remove them in 2.1 since they require to reparse the
result before indexing it and we don't want this hack to live long.
The following directives were marked deprecated :

  -reqadd
  -reqallow
  -reqdel
  -reqdeny
  -reqiallow
  -reqidel
  -reqideny
  -reqipass
  -reqirep
  -reqitarpit
  -reqpass
  -reqrep
  -reqtarpit
  -rspadd
  -rspdel
  -rspdeny
  -rspidel
  -rspideny
  -rspirep
  -rsprep
2019-05-22 20:43:45 +02:00
Willy Tarreau
e6a02fa65a MINOR: threads: add a "stuck" flag to the thread_info struct
This flag is constantly cleared by the scheduler and will be set by the
watchdog timer to detect stuck threads. It is also set by the "show
threads" command so that it is easy to spot if the situation has evolved
between two subsequent calls : if the first "show threads" shows no stuck
thread and the second one shows such a stuck thread, it indicates that
this thread didn't manage to make any forward progress since the previous
call, which is extremely suspicious.
2019-05-22 11:50:48 +02:00
Emmanuel Hocdet
0ba4f483d2 MAJOR: polling: add event ports support (Solaris)
Event ports are kqueue/epoll polling class for Solaris. Code is based
on https://github.com/joyent/haproxy-1.8/tree/joyent/dev-v1.8.8.
Event ports are available only on SunOS systems derived from
Solaris 10 and later (including illumos systems).
2019-05-21 15:16:45 +02:00
Willy Tarreau
6bdf3e9b11 MINOR: debug/cli: add some debugging commands for developers
When haproxy is built with DEBUG_DEV, the following commands are added
to the CLI :

  debug dev close <fd>        : close this file descriptor
  debug dev delay [ms]        : sleep this long
  debug dev exec  [cmd] ...   : show this command's output
  debug dev exit  [code]      : immediately exit the process
  debug dev hex   <addr> [len]: dump a memory area
  debug dev log   [msg] ...   : send this msg to global logs
  debug dev loop  [ms]        : loop this long
  debug dev panic             : immediately trigger a panic
  debug dev tkill [thr] [sig] : send signal to thread

These are essentially aimed at helping developers trigger certain
conditions and are expected to be complemented over time.
2019-05-20 16:59:30 +02:00
Willy Tarreau
c7091d89ae MEDIUM: debug/threads: implement an advanced thread dump system
The current "show threads" command was too limited as it was not possible
to dump other threads' detailed states (e.g. their tasks). This patch
goes further by using thread signals so that each thread can dump its
own state in turn into a shared buffer provided by the caller. Threads
are synchronized using a mechanism very similar to the rendez-vous point
and using this method, each thread can safely dump any of its contents
and the caller can finally report the aggregated ones from the buffer.

It is important to keep in mind that the list of signal-safe functions
is limited, so we take care of only using chunk_printf() to write to a
pre-allocated buffer.

This mechanism is enabled by USE_THREAD_DUMP and is enabled by default
on Linux 2.6.28+. On other platforms it falls back to the previous
solution using the loop and the less precise dump.
2019-05-17 17:16:20 +02:00
Willy Tarreau
4e2b646d60 MINOR: cli/debug: add a thread dump function
The new function ha_thread_dump() will dump debugging info about all known
threads. The current thread will contain a bit more info. The long-term goal
is to make it possible to use it in signal handlers to improve the accuracy
of some dumps.

The function dumps its output into the trash so as it was trivial to add,
a new "show threads" command appeared on the CLI.
2019-05-16 18:06:45 +02:00
Willy Tarreau
4c35693eb0 DOC: management: place "show activity" at the right place
This time it seems to be me who had trouble with alphabetical ordering.
This should be backported to 1.9.
2019-05-16 18:02:03 +02:00
Tim Duesterhus
3506dae342 MEDIUM: Make 'resolution_pool_size' directive fatal
This directive never appeared in a stable release and instead was
introduced and deprecated within 1.8-dev. While it technically could
be outright removed we detect it and error out for good measure.
2019-05-16 18:02:03 +02:00
Tim Duesterhus
10c6c16cde MEDIUM: Make 'option forceclose' actually warn
It is deprecated since 315b39c391 (1.9-dev),
but only was deprecated in the docs.

Make it warn when being used and remove it from the docs.
2019-05-16 18:02:03 +02:00
Willy Tarreau
a257a9b015 [RELEASE] Released version 2.0-dev3
Released version 2.0-dev3 with the following main changes :
    - BUG/MINOR: peers: Really close the sessions with no heartbeat.
    - CLEANUP: peers: remove useless annoying tabulations.
    - CLEANUP: peers: replace timeout constants by macros.
    - REGTEST: Enable again reg tests with HEAD HTTP method usage.
    - DOC: The option httplog is no longer valid in a backend.
    - DOC: peers: Peers protocol documentation update.
    - REGTEST: remove unexpected "nbthread" statement from Lua test cases
    - BUILD: Makefile: remove 11-years old workarounds for deprecated options
    - BUILD: remove 10-years old error message for obsolete option USE_TCPSPLICE
    - BUILD: Makefile: remove outdated support for dlmalloc
    - BUILD: Makefile: consider a variable's origin and not its value for the options list
    - BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
    - BUILD: Makefile: shorten default settings declaration
    - BUILD: Makefile: clean up the target declarations
    - BUILD: report the whole feature set with their status in haproxy -vv
    - BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
    - REGTEST: script: make the script use the new features list
    - REGTEST: script: remove platform-specific assigments of OPTIONS
    - BUG/MINOR: peers: Missing initializations after peer session shutdown.
    - BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes
    - BUILD/MINOR: listener: Silent a few signedness warnings.
    - BUG/MINOR: mux-h1: Only skip invalid C-L headers on output
    - BUG/MEDIUM: mworker: don't free the wrong child when not found
    - BUG/MEDIUM: checks: Don't bother subscribing if we have a connection error.
    - BUG/MAJOR: checks: segfault during tcpcheck_main
    - BUILD: makefile: work around an old bug in GNU make-3.80
    - BUILD: makefile: work around another bug in make 3.80
    - BUILD: http: properly mark some struct as extern
    - BUILD: chunk: properly declare pool_head_trash as extern
    - BUILD: cache: avoid a build warning with some compilers/linkers
    - MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
    - MINOR: tools: add an unsetenv() implementation
    - BUILD: re-implement an initcall variant without using executable sections
    - BUILD: use inttypes.h instead of stdint.h
    - BUILD: connection: fix naming of ip_v field
    - BUILD: makefile: fix build of IPv6 header on aix51
    - BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
    - BUILD: define unsetenv on AIX 5.1
    - BUILD: Makefile: disable shared cache on AIX 5.1
    - MINOR: ssl: Add aes_gcm_dec converter
    - REORG: mworker: move serializing functions to mworker.c
    - REORG: mworker: move signals functions to mworker.c
    - REORG: mworker: move IPC functions to mworker.c
    - REORG: mworker: move signal handlers and related functions
    - REORG: mworker: move mworker_cleanlisteners to mworker.c
    - MINOR: mworker: calloc mworker_proc structures
    - MINOR: mworker: don't use children variable anymore
    - MINOR: cli: export cli_parse_default() definition in cli.h
    - REORG: mworker/cli: move CLI functions to mworker.c
    - MEDIUM: mworker-prog: implement program for master-worker
    - MINOR: mworker/cli: show programs in 'show proc'
    - BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
    - MINOR: cli: start addresses by a prefix in 'show cli sockets'
    - MINOR: cli: export HAPROXY_CLI environment variable
    - BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing error
    - BUG/MINOR: proto_htx: Reset to_forward value when a message is set to DONE
    - REGTEST: http-capture/h00000: Relax a regex matching the log message
    - REGTEST: http-messaging/h00000: Fix the test when the HTX is enabled
    - REGTEST: http-rules/h00003: Use a different client for requests expecting a 301
    - REGTEST: log/b00000: Be sure the client always hits its timeout
    - REGTEST: lua/b00003: Relax the regex matching the log message
    - REGTEST: lua/b00003: Specify the HAProxy pid when the command ss is executed
    - BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on release.
    - BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we subscribed.
    - BUG/MEDIUM: htx: fix random premature abort of data transfers
    - BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
    - BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
    - BUG/MEDIUM: stream: Don't clear the stream_interface flags in si_update_both.
    - MINOR: initcall: Don't forget to define the __start/stop_init_##stg symbols.
    - MINOR: threads: Implement thread_cpus_enabled() for FreeBSD.
    - BUG/MEDIUM: pattern: assign pattern IDs after checking the config validity
    - MINOR: skip get_gmtime where tm is unused
    - MINOR: ssl: Activate aes_gcm_dec converter for BoringSSL
    - BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected state.
    - BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not connected.
    - BUG/MEDIUM: task/threads: address a fairness issue between local and global tasks
    - BUG/MINOR: tasks: make sure the first task to be queued keeps its nice value
    - BUG/MINOR: listener: renice the accept ring processing task
    - MINOR: cli/listener: report the number of accepts on "show activity"
    - MINOR: cli/activity: report the accept queue sizes in "show activity"
    - BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to the stream
    - BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented messages
    - BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is encoded
    - BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads wrap
    - BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a data block
    - MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
    - MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
    - BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
    - MINOR: mux-h1: Simplify handling of 1xx responses
    - MINOR: stats/htx: Don't add "Connection: close" header anymore in stats responses
    - MEDIUM: h1: Add an option to sanitize connection headers during parsing
    - MEDIUM: mux-h1: Simplify the connection mode management by sanitizing headers
    - MINOR: mux-h1: Don't release the conn_stream anymore when h1s is destroyed
    - BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown read/write
    - MINOR: mux-h2: Add a mux_ops dedicated to the HTX mode
    - MINOR: muxes: Add a flag to specify a multiplexer uses the HTX
    - MINOR: stream: Set a flag when the stream uses the HTX
    - MINOR: http: update the macro IS_HTX_STRM() to check the stream flag SF_HTX
    - MINOR: http_fetch/htx: Use stream flags instead of px mode in smp_prefetch_htx
    - MINOR: filters/htx: Use stream flags instead of px mode to instanciate a filter
    - MINOR: muxes: Rely on conn_is_back() during init to handle front/back conn
    - MEDIUM: muxes: Add an optional input buffer during mux initialization
    - MINOR: muxes: Pass the context of the mux to destroy() instead of the connection
    - MEDIUM: muxes: Be prepared to don't own connection during the release
    - MEDIUM: connection: Add conn_upgrade_mux_fe() to handle mux upgrades
    - MEDIUM: htx: Allow the option http-use-htx to be used on TCP proxies too
    - MAJOR: proxy/htx: Handle mux upgrades from TCP to HTTP in HTX mode
    - MAJOR: muxes/htx: Handle inplicit upgrades from h1 to h2
    - MAJOR: htx: Enable the HTX mode by default for all proxies
    - REGTEST: Use HTX by default and add '--no-htx' option to disable it
    - BUG/MEDIUM: muxes: Don't dereference mux context if null in release functions
    - CLEANUP: task: do not export rq_next anymore
    - MEDIUM: tasks: improve fairness between the local and global queues
    - MEDIUM: tasks: only base the nice offset on the run queue depth
    - MINOR: tasks: restore the lower latency scheduling when niced tasks are present
    - BUG/MEDIUM: map: Fix memory leak in the map converter
    - BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation
    - BUILD: task/thread: fix single-threaded build of task.c
    - BUILD: cli/threads: fix build in single-threaded mode
    - BUG/MEDIUM: muxes: Make sure we unsubcribed when destroying mux ctx.
    - BUG/MEDIUM: h2: Make sure we're not already in the send_list in h2_subscribe().
    - BUG/MEDIUM: h2: Revamp the way send subscriptions works.
    - MINOR: connections: Remove the SUB_CALL_UNSUBSCRIBE flag.
    - BUG/MEDIUM: Threads: Only use the gcc >= 4.7 builtins when using gcc >= 4.7.
    - BUILD: address a few cases of "static <type> inline foo()"
    - BUILD: do not specify "const" on functions returning structs or scalars
    - BUILD: htx: fix a used uninitialized warning on is_cookie2
    - MINOR: peers: Add a new command to the CLI for peers.
    - DOC: update for "show peers" CLI command.
    - BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
    - MEDIUM: mworker: store the leaving state of a process
    - MEDIUM: mworker-prog: implements 'option start-on-reload'
    - CLEANUP: mworker: remove the type field in mworker_proc
    - MEDIUM: mworker/cli: export the HAPROXY_MASTER_CLI variable
    - MINOR: cli: don't add a semicolon at the end of HAPROXY_CLI
    - MINOR: mworker: export HAPROXY_MWORKER=1 when running in mworker mode
    - MINOR: init: add a "set-dumpable" global directive to enable core dumps
    - BUG/MINOR: listener/mq: correctly scan all bound threads under low load
    - BUG/MINOR: mworker: mworker_kill should apply on every children
    - BUG/MINOR: mworker: don't exit with an ambiguous value
    - BUG/MINOR: mworker: ensure that we still quits with SIGINT
    - REGTESTS: exclude tests that require ssl, pcre if no such feature is enabled
    - BUG/MINOR: mux-h1: Process input even if the input buffer is empty
    - BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side has done
    - BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf is empty
    - BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
    - MINOR: mux-h1: Handle read0 during TCP splicing
    - BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
    - BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
    - BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
    - BUG/MINOR: http_fetch/htx: Use HTX versions if the proxy enables the HTX mode
    - BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to the rq.
    - BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the rq_lock.
    - MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
    - MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
    - MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet list.
    - BUG/MEDIUM: applets: Don't use task_in_rq().
    - BUG/MAJOR: task: make sure never to delete a queued task
    - MINOR: task/thread: factor out a wake-up condition
    - CLEANUP: task: remain consistent when using the task's handler
    - MEDIUM: tasks: Merge task_delete() and task_free() into task_destroy().
    - MEDIUM: tasks: Don't account a destroyed task as a runned task.
    - BUG/MINOR: contrib/prometheus-exporter: Fix a typo in the run-queue metric type
    - MINOR: contrib/prometheus-exporter: Remove usless rate metrics
    - MINOR: contrib/prometheus-exporter: Rename some metrics to be more usable
    - MINOR: contrib/prometheus-exporter: Follow best practices about metrics type
    - BUG/MINOR: mworker: disable busy polling in the master process
    - MEDIUM: tasks: Use __ha_barrier_store after modifying global_tasks_mask.
    - MEDIUM: ssl: Give ssl_sock its own context.
    - MEDIUM: connections: Move some fields from struct connection to ssl_sock_ctx.
    - MEDIUM: ssl: provide its own subscribe/unsubscribe function.
    - MEDIUM: connections: Provide a xprt_ctx for each xprt method.
    - MEDIUM: ssl: provide our own BIO.
    - BUILD/medium: ssl: Fix build with OpenSSL < 1.1.0
    - MINOR: peers: adds counters on show peers about tasks calls.
    - MEDIUM: enable travis-ci builds
    - MINOR: fd: Add a counter of used fds.
    - MEDIUM: connections: Add a way to control the number of idling connections.
    - BUG/MEDIUM: maps: only try to parse the default value when it's present
    - BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
    - REGTEST: Missing REQUIRE_VERSION declarations.
    - MINOR: proto_tcp: tcp-request content: enable set-dst and set-dst-var
    - BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
    - BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP rules
    - BUG/MEDIUM: stream: Don't request a server connection if a shutw was scheduled
    - BUG/MINOR: 51d: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
    - BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
    - MINOR: gcc: Fix a silly gcc warning in connect_server()
    - MINOR: ssl/cli: async fd io-handlers printable on show fd
    - Revert "CLEANUP: wurfl: remove dead, broken and unmaintained code"
    - BUILD: add USE_WURFL to the list of known build options
    - MINOR: wurfl: indicate in haproxy -vv the wurfl version in use
    - BUILD: wurfl: build fix for 1.9/2.0 code base
    - CLEANUP: wurfl: removed deprecated methods
    - DOC: wurfl: added point of contact in MAINTAINERS file
    - MINOR: wurfl: enabled multithreading mode
    - MINOR: contrib: dummy wurfl library
    - MINOR: dns: dns_requester structures are now in a memory pool
    - MINOR: dns: move callback affection in dns_link_resolution()
    - MINOR: obj_type: new object type for struct stream
    - MINOR: action: new '(http-request|tcp-request content) do-resolve' action
    - MINOR: log: Extract some code to send syslog messages.
    - REGTEST: replace LEVEL option by a more human readable one.
    - REGTEST: rename the reg test files.
    - REGTEST: adapt some reg tests after renaming.
    - REGTEST: make the "run-regtests" script search for tests in reg-tests by default
    - BUG/MAJOR: stream: Missing DNS context initializations.
    - BUG/MEDIUM: stream: Fix the way early aborts on the client side are handled
    - BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet handler
    - BUG/MEDIUM: ssl: Return -1 on recv/send if we got EAGAIN.
    - BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB
    - BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies only
    - BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during startup
    - CLEANUP: task: report calls as unsigned in show sess
    - MINOR: tasks/activity: report the context switch and task wakeup rates
    - MINOR: stream: measure and report a stream's call rate in "show sess"
    - MINOR: applet: measure and report an appctx's call rate in "show sess"
    - BUILD: extend Travis CI config to support more platforms
    - REGTEST: exclude osx and generic targets for 40be_2srv_odd_health_checks
    - REGTEST: relax the IPv6 address format checks in converters_ipmask_concat_strcmp_field_word
    - REGTEST: exclude OSX and generic targets from abns_socket.vtc
    - BUILD: travis: remove the "allow_failures" entry
    - BUG/MINOR: activity: always initialize the profiling variable
    - MINOR: activity: make the profiling status per thread and not global
    - MINOR: activity: enable automatic profiling turn on/off
    - CLEANUP: standard: use proper const to addr_to_str() and port_to_str()
    - BUG/MINOR: proto_http: properly reset the stream's call rate on keep-alive
    - MINOR: connection: make the debugging helper functions safer
    - MINOR: stream/debug: make a stream dump and crash function
    - MEDIUM: appctx/debug: force a crash if an appctx spins over itself forever
    - MEDIUM: stream/debug: force a crash if a stream spins over itself forever
    - MEDIUM: streams: measure processing time and abort when detecting bugs
    - BUILD/MEDIUM: contrib: Dummy DeviceAtlas API.
    - MEDIUM: da: HTX mode support.
    - BUG/MEDIUM: mux-h2: properly deal with too large headers frames
    - BUG/MINOR: http: Call stream_inc_be_http_req_ctr() only one time per request
    - BUG/MEDIUM: spoe: arg len encoded in previous frag frame but len changed
    - MINOR: spoe: Use the sample context to pass frag_ctx info during encoding
    - DOC: contrib/modsecurity: Typos and fix the reject example
    - BUG/MEDIUM: contrib/modsecurity: If host header is NULL, don't try to strdup it
    - MINOR: log: Add "sample" new keyword to "log" lines.
    - MINOR: log: Enable the log sampling and load-balancing feature.
    - DOC: log: Document the sampling and load-balancing logging feature.
    - REGTEST: Add a new reg test for log load-balancing feature.
    - BUG/MAJOR: map/acl: real fix segfault during show map/acl on CLI
    - REGTEST: Make this reg test be Linux specific.
    - CLEANUP: task: move the task_per_thread definition to task.h
    - MINOR: activity: report context switch counts instead of rates
    - MINOR: threads: Implement HA_ATOMIC_LOAD().
    - BUG/MEDIUM: port_range: Make the ring buffer lock-free.
    - BUG/MEDIUM: listener: Fix how unlimited number of consecutive accepts is handled
    - MINOR: config: Test validity of tune.maxaccept during the config parsing
    - CLEANUP: config: Don't alter listener->maxaccept when nbproc is set to 1
    - BUG/MEDIUM: servers: fix typo "src" instead of "srv"
    - BUG/MEDIUM: ssl: Don't pretend we can retry a recv/send if we got a shutr/w.
    - BUG/MINOR: haproxy: fix rule->file memory leak
    - BUG/MINOR: log: properly free memory on logformat parse error and deinit()
    - BUG/MINOR: checks: free memory allocated for tasklets
    - BUG/MEDIUM: pattern: fix memory leak in regex pattern functions
    - BUG/MEDIUM: channels: Don't forget to reset output in channel_erase().
    - BUG/MEDIUM: connections: Make sure we remove CO_FL_SESS_IDLE on disown.
    - MINOR: threads: flatten the per-thread cpu-map
    - MINOR: init/threads: remove the useless tids[] array
    - MINOR: init/threads: make the threads array global
    - BUG/MEDIUM: ssl: Use the early_data API the right way.
    - BUG/MEDIUM: streams: Don't add CF_WRITE_ERROR if early data were rejected.
    - MEDIUM: streams: Add the ability to retry a request on L7 failure.
    - MEDIUM: streams: Add a way to replay failed 0rtt requests.
    - MEDIUM: streams: Add a new keyword for retry-on, "junk-response"
    - BUG/MINOR: stream: also increment the retry stats counter on L7 retries
    - BUG/MEDIUM: checks: make sure the warmup task takes the server lock
    - BUG/MINOR: logs/threads: properly split the log area upon startup
    - BUILD: extend travis-ci matrix
    - CLEANUP: Remove appsession documentation
    - DOC: Fix typo in keyword matrix
    - BUILD: remove "build_libressl" duplicate declaration
    - BUILD: travis-ci: get back to osx without openssl support
    - BUILD: enable several LibreSSL hacks, including
    - BUILD: temporarily mark LibreSSL builds as allowed to fail
    - BUILD: travis: TMPDIR replacement.
    - BUG/MEDIUM: ssl: Don't attempt to use early data with libressl.
    - MINOR: doc: Document allow-0rtt on the server line.
    - MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
    - MEDIUM: proto: Change the prototype of the connect() method.
    - MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
    - MINOR: config: Extract the code of "stick-table" line parsing.
    - BUILD/MINOR: stick-table: Compilation fix.
    - MEDIUM: stick-table: Stop handling stick-tables as proxies.
    - MINOR: stick-tables: Add peers process binding computing.
    - MINOR: stick-table: Add prefixes to stick-table names.
    - MINOR: peers: Do not emit global stick-table names.
    - DOC: Update for "table" lines in "peers" section.
    - REGTEST: Add reg tests for "table" lines in "peers" sections.
    - MEDIUM: regex: modify regex_comp() to atomically allocate/free the my_regex struct
    - REGTEST: make the tls_health_checks test much faster
    - REGTEST: make the "table in peers" test require v2.0
    - BUG/MINOR: mux-h2: rely on trailers output not input to turn them to empty data
    - BUG/MEDIUM: h2/htx: always fail on too large trailers
    - MEDIUM: mux-h2: discard contents that are to be sent after a shutdown
    - BUG/MEDIUM: mux-h2/htx: never wait for EOM when processing trailers
    - BUG/MEDIUM: h2/htx: never leave a trailers block alone with no EOM block
    - REGTEST: Flag some slow reg tests.
    - REGTEST: Reg tests file renaming.
    - REGTEST: Wrong renaming for one reg test.
    - REGTEST: Wrong assumption in IP:port logging test.
    - BUG/MINOR: mworker/ssl: close OpenSSL FDs on reload
    - MINOR: systemd: Use the variables from /etc/default/haproxy
    - MINOR: systemd: Make use of master socket in systemd unit
    - MINOR: systemd: support /etc/sysconfig/ for redhat based distrib
    - BUG/MEDIUM: stick-table: fix regression caused by a change in proxy struct
    - BUG/MEDIUM: tasks: fix possible segfault on task_destroy()
    - CLEANUP: task: remove unneeded tests before task_destroy()
    - MINOR: mworker: support a configurable maximum number of reloads
    - BUG/MINOR: mux-h2: fix the condition to close a cs-less h2s on the backend
    - BUG/MEDIUM: spoe: Be sure the sample is found before setting its context
    - BUG/MINOR: mux-h1: Fix the parsing of trailers
    - BUG/MINOR: htx: Never transfer more than expected in htx_xfer_blks()
    - MINOR: htx: Split on DATA blocks only when blocks are moved to an HTX message
    - MINOR: htx: Don't try to append a trailer block with the previous one
    - MINOR: htx: Remove support for unused OOB HTX blocks
    - BUILD: travis-ci bugfixes and improvements
    - BUG/MEDIUM: servers: Don't use the same srv flag for cookie-set and TFO.
    - BUG/MEDIUM: h2: Make sure we set send_list to NULL in h2_detach().
    - BUILD: ssl: fix again a libressl build failure after the openssl FD leak fix
    - CLEANUP: ssl-sock: use HA_OPENSSL_VERSION_NUMBER instead of OPENSSL_VERSION_NUMBER
    - BUILD: ssl: make libressl use its own version numbers
    - CLEANUP: ssl: remove 57 occurrences of useless tests on LIBRESSL_VERSION_NUMBER
    - MINOR: ssl: enable aes_gcm_dec on LibreSSL
    - BUILD: ssl: fix libressl build again after aes-gcm-enc
    - REORG: ssl: move openssl-compat from proto to common
    - REORG: ssl: move some OpenSSL defines from ssl_sock to openssl-compat
    - CLEANUP: ssl: never include openssl/*.h outside of openssl-compat.h anymore
    - CLEANUP: ssl: make inclusion of openssl headers safe
    - BUILD: add BoringSSL to travis-ci build matrix
    - BUILD: threads: Add __ha_cas_dw fallback for single threaded builds
    - BUG/MINOR: stream: Attach the read side on the response as soon as possible
    - BUG/MEDIUM: http: Use pointer to the begining of input to parse message headers
    - BUG/MEDIUM: h2: Don't check send_wait to know if we're in the send_list.
    - BUG/MEDIUM: streams: Make sur SI_FL_L7_RETRY is set before attempting a retry.
    - MEDIUM: streams: Add a new http action, disable-l7-retry.
    - MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
    - BUG/MINOR: vars: Fix memory leak in vars_check_arg
    - BUILD: travis-ci: make TMPDIR global variable in travis-ci
    - CLEANUP: ssl: move the SSL_OP_* and SSL_MODE_* definitions to openssl-compat
    - CLEANUP: ssl: remove ifdef around SSL_CTX_get_extra_chain_certs()
    - CLEANUP: ssl: move all BIO_* definitions to openssl-compat
    - BUILD: threads: fix again the __ha_cas_dw() definition
    - BUG/MAJOR: mux-h2: do not add a stream twice to the send list
    - Revert "BUG/MINOR: vars: Fix memory leak in vars_check_arg"
    - BUG/MINOR: peers: Fix memory leak in cfg_parse_peers
    - BUG/MINOR: htx: make sure to always initialize the HTTP method when parsing a buffer
    - REGTEST: fix tls_health_checks random failures on MacOS in Travis-CI
    - MINOR: spoe: Set the argument chunk size to 0 when SPOE variables are checked
    - BUG/MINOR: vars: Fix memory leak in vars_check_arg
    - BUG/MAJOR: ssl: segfault upon an heartbeat request
    - MINOR: spoa-server: Clone the v1.7 spoa-example project
    - MINOR: spoa-server: move some definition from spoa_server.c to spoa_server.h
    - MINOR: spoa-server: Externalise debug functions
    - MINOR: spoe-server: rename "worker" functions
    - MINOR: spoa-server: Replace the thread init system by processes
    - MINOR: spoa-server: With debug mode, start only one process
    - MINOR: spoa-server: Allow registering external processes
    - MINOR: spoa-server: Allow registering message processors
    - MINOR: spoa-server: Load files
    - MINOR: spoa-server: Prepare responses
    - MINOR: spoa-server: Execute registered callbacks
    - MINOR: spoa-server: Add Lua processing
    - MINOR: spoa-server: Add python
    - MINOR/DOC: spoe-server: Add documentation
    - BUG/MEDIUM: connections: Don't forget to set xprt_ctx to NULL on close.
    - MINOR: lists: add LIST_ADDED() to check if an element belongs to a list
    - CLEANUP: mux-h2: use LIST_ADDED() instead of LIST_ISEMPTY() where relevant
    - MINOR: mux-h2: add two H2S flags to report the need for shutr/shutw
    - CLEANUP: mux-h2: simply use h2s->flags instead of ret in h2_deferred_shut()
    - CLEANUP: connection: remove the handle field from the wait_event struct
    - BUG/MINOR: log: Wrong log format initialization.
    - BUG/MINOR: mux-h2: make the do_shut{r,w} functions more robust against retries
    - BUG/MINOR: mworker: use after free when the PID not assigned
    - MINOR: mux-h2: remove useless test on stream ID vs last in wake function
    - MINOR: mux-h2: make h2_wake_some_streams() not depend on the CS flags
    - MINOR: mux-h2: make h2s_wake_one_stream() the only function to deal with CS
    - MINOR: mux-h2: make h2s_wake_one_stream() not depend on temporary CS flags
    - BUG/MINOR: mux-h2: make sure to honor KILL_CONN in do_shut{r,w}
    - CLEANUP: mux-h2: don't test for impossible CS_FL_REOS conditions
    - MINOR: mux-h2: add macros to check multiple stream states at once
    - MINOR: mux-h2: stop relying on CS_FL_REOS
    - BUG/MEDIUM: mux-h2: Set EOI on the conn_stream during h2_rcv_buf()
    - BUILD: debug: make gcc not complain on the ABORT_NOW() macro
    - MINOR: debug: add a new BUG_ON macro
    - MINOR: h2: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - MINOR: h1: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - MINOR: connections: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - BUILD: ist: turn the lower/upper case tables to literal on obsolete linkers
2019-05-15 16:51:48 +02:00
Olivier Houchard
ddf0e03585 MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
Add a new retry-on keyword, "all-retryable-errors", that activates retry
for all errors that are considered retryable.
This currently activates retry for "conn-failure", "empty-response",
"junk-respones", "response-timeout", "0rtt-rejected", "500", "502", "503" and
"504".
2019-05-10 18:05:35 +02:00
Olivier Houchard
602bf7d2ea MEDIUM: streams: Add a new http action, disable-l7-retry.
Add a new action for http-request, disable-l7-retry, that can be used to
disable any attempt at retry requests (see retry-on) if it fails for any
reason other than a connection failure.
This is useful for example to make sure POST requests aren't retried.
2019-05-10 17:49:09 +02:00
William Lallemand
27edc4b915 MINOR: mworker: support a configurable maximum number of reloads
This patch implements a new global parameter for the master-worker mode.
When setting the mworker-max-reloads value, a worker receive a SIGTERM
if its number of reloads is greater than this value.
2019-05-07 19:09:01 +02:00
Frdric Lcaille
4f5b77c57c DOC: Update for "table" lines in "peers" section. 2019-05-07 06:54:07 +02:00
Willy Tarreau
034c88cf03 MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
This implements support for the new API which relies on a call to
setsockopt().
On systems that support it (currently, only Linux >= 4.11), this enables
using TCP fast open when connecting to server.
Please note that you should use the retry-on "conn-failure", "empty-response"
and "response-timeout" keywords, or the request won't be able to be retried
on failure.

Co-authored-by: Olivier Houchard <ohouchard@haproxy.com>
2019-05-06 22:29:39 +02:00
Olivier Houchard
22c9b440dc MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
Document that haproxy will try to use early data when connecting to the
server if we have retry-on 0rtt-rejected, as well.
2019-05-06 19:01:19 +02:00
Olivier Houchard
8cb2d2e941 MINOR: doc: Document allow-0rtt on the server line.
Briefly document allow-0rtt on the server line, and only the part that apply
to 1.8 and 1.9.

This should be backported to 1.8 and 1.9.
2019-05-06 19:01:14 +02:00
Tim Duesterhus
44864ac7b0 DOC: Fix typo in keyword matrix
It should read 'deprecated' instead of 'deprectated'.
2019-05-06 07:15:08 +02:00