A regression was introduced in the commit 89dc49935 ("BUG/MAJOR: http_fetch: Get
the channel depending on the keyword used") on the samples "cookie()" and
"hdr()". Unlike other samples manipulating the HTTP headers, these ones depend
on the sample direction. To fix the bug, these samples use now their own
functions. Depending on the sample direction, they call smp_fetch_cookie() and
smp_fetch_hdr() with the appropriate keyword.
Thanks to Yves Lafon to report this issue.
This patch must be backported wherever the commit 89dc49935 was backported. For
now, 1.9 and 1.8.
Released version 2.0-dev3 with the following main changes :
- BUG/MINOR: peers: Really close the sessions with no heartbeat.
- CLEANUP: peers: remove useless annoying tabulations.
- CLEANUP: peers: replace timeout constants by macros.
- REGTEST: Enable again reg tests with HEAD HTTP method usage.
- DOC: The option httplog is no longer valid in a backend.
- DOC: peers: Peers protocol documentation update.
- REGTEST: remove unexpected "nbthread" statement from Lua test cases
- BUILD: Makefile: remove 11-years old workarounds for deprecated options
- BUILD: remove 10-years old error message for obsolete option USE_TCPSPLICE
- BUILD: Makefile: remove outdated support for dlmalloc
- BUILD: Makefile: consider a variable's origin and not its value for the options list
- BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
- BUILD: Makefile: shorten default settings declaration
- BUILD: Makefile: clean up the target declarations
- BUILD: report the whole feature set with their status in haproxy -vv
- BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
- REGTEST: script: make the script use the new features list
- REGTEST: script: remove platform-specific assigments of OPTIONS
- BUG/MINOR: peers: Missing initializations after peer session shutdown.
- BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes
- BUILD/MINOR: listener: Silent a few signedness warnings.
- BUG/MINOR: mux-h1: Only skip invalid C-L headers on output
- BUG/MEDIUM: mworker: don't free the wrong child when not found
- BUG/MEDIUM: checks: Don't bother subscribing if we have a connection error.
- BUG/MAJOR: checks: segfault during tcpcheck_main
- BUILD: makefile: work around an old bug in GNU make-3.80
- BUILD: makefile: work around another bug in make 3.80
- BUILD: http: properly mark some struct as extern
- BUILD: chunk: properly declare pool_head_trash as extern
- BUILD: cache: avoid a build warning with some compilers/linkers
- MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
- MINOR: tools: add an unsetenv() implementation
- BUILD: re-implement an initcall variant without using executable sections
- BUILD: use inttypes.h instead of stdint.h
- BUILD: connection: fix naming of ip_v field
- BUILD: makefile: fix build of IPv6 header on aix51
- BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
- BUILD: define unsetenv on AIX 5.1
- BUILD: Makefile: disable shared cache on AIX 5.1
- MINOR: ssl: Add aes_gcm_dec converter
- REORG: mworker: move serializing functions to mworker.c
- REORG: mworker: move signals functions to mworker.c
- REORG: mworker: move IPC functions to mworker.c
- REORG: mworker: move signal handlers and related functions
- REORG: mworker: move mworker_cleanlisteners to mworker.c
- MINOR: mworker: calloc mworker_proc structures
- MINOR: mworker: don't use children variable anymore
- MINOR: cli: export cli_parse_default() definition in cli.h
- REORG: mworker/cli: move CLI functions to mworker.c
- MEDIUM: mworker-prog: implement program for master-worker
- MINOR: mworker/cli: show programs in 'show proc'
- BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
- MINOR: cli: start addresses by a prefix in 'show cli sockets'
- MINOR: cli: export HAPROXY_CLI environment variable
- BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing error
- BUG/MINOR: proto_htx: Reset to_forward value when a message is set to DONE
- REGTEST: http-capture/h00000: Relax a regex matching the log message
- REGTEST: http-messaging/h00000: Fix the test when the HTX is enabled
- REGTEST: http-rules/h00003: Use a different client for requests expecting a 301
- REGTEST: log/b00000: Be sure the client always hits its timeout
- REGTEST: lua/b00003: Relax the regex matching the log message
- REGTEST: lua/b00003: Specify the HAProxy pid when the command ss is executed
- BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on release.
- BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we subscribed.
- BUG/MEDIUM: htx: fix random premature abort of data transfers
- BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
- BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
- BUG/MEDIUM: stream: Don't clear the stream_interface flags in si_update_both.
- MINOR: initcall: Don't forget to define the __start/stop_init_##stg symbols.
- MINOR: threads: Implement thread_cpus_enabled() for FreeBSD.
- BUG/MEDIUM: pattern: assign pattern IDs after checking the config validity
- MINOR: skip get_gmtime where tm is unused
- MINOR: ssl: Activate aes_gcm_dec converter for BoringSSL
- BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected state.
- BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not connected.
- BUG/MEDIUM: task/threads: address a fairness issue between local and global tasks
- BUG/MINOR: tasks: make sure the first task to be queued keeps its nice value
- BUG/MINOR: listener: renice the accept ring processing task
- MINOR: cli/listener: report the number of accepts on "show activity"
- MINOR: cli/activity: report the accept queue sizes in "show activity"
- BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to the stream
- BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented messages
- BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is encoded
- BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads wrap
- BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a data block
- MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
- MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
- BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
- MINOR: mux-h1: Simplify handling of 1xx responses
- MINOR: stats/htx: Don't add "Connection: close" header anymore in stats responses
- MEDIUM: h1: Add an option to sanitize connection headers during parsing
- MEDIUM: mux-h1: Simplify the connection mode management by sanitizing headers
- MINOR: mux-h1: Don't release the conn_stream anymore when h1s is destroyed
- BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown read/write
- MINOR: mux-h2: Add a mux_ops dedicated to the HTX mode
- MINOR: muxes: Add a flag to specify a multiplexer uses the HTX
- MINOR: stream: Set a flag when the stream uses the HTX
- MINOR: http: update the macro IS_HTX_STRM() to check the stream flag SF_HTX
- MINOR: http_fetch/htx: Use stream flags instead of px mode in smp_prefetch_htx
- MINOR: filters/htx: Use stream flags instead of px mode to instanciate a filter
- MINOR: muxes: Rely on conn_is_back() during init to handle front/back conn
- MEDIUM: muxes: Add an optional input buffer during mux initialization
- MINOR: muxes: Pass the context of the mux to destroy() instead of the connection
- MEDIUM: muxes: Be prepared to don't own connection during the release
- MEDIUM: connection: Add conn_upgrade_mux_fe() to handle mux upgrades
- MEDIUM: htx: Allow the option http-use-htx to be used on TCP proxies too
- MAJOR: proxy/htx: Handle mux upgrades from TCP to HTTP in HTX mode
- MAJOR: muxes/htx: Handle inplicit upgrades from h1 to h2
- MAJOR: htx: Enable the HTX mode by default for all proxies
- REGTEST: Use HTX by default and add '--no-htx' option to disable it
- BUG/MEDIUM: muxes: Don't dereference mux context if null in release functions
- CLEANUP: task: do not export rq_next anymore
- MEDIUM: tasks: improve fairness between the local and global queues
- MEDIUM: tasks: only base the nice offset on the run queue depth
- MINOR: tasks: restore the lower latency scheduling when niced tasks are present
- BUG/MEDIUM: map: Fix memory leak in the map converter
- BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation
- BUILD: task/thread: fix single-threaded build of task.c
- BUILD: cli/threads: fix build in single-threaded mode
- BUG/MEDIUM: muxes: Make sure we unsubcribed when destroying mux ctx.
- BUG/MEDIUM: h2: Make sure we're not already in the send_list in h2_subscribe().
- BUG/MEDIUM: h2: Revamp the way send subscriptions works.
- MINOR: connections: Remove the SUB_CALL_UNSUBSCRIBE flag.
- BUG/MEDIUM: Threads: Only use the gcc >= 4.7 builtins when using gcc >= 4.7.
- BUILD: address a few cases of "static <type> inline foo()"
- BUILD: do not specify "const" on functions returning structs or scalars
- BUILD: htx: fix a used uninitialized warning on is_cookie2
- MINOR: peers: Add a new command to the CLI for peers.
- DOC: update for "show peers" CLI command.
- BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
- MEDIUM: mworker: store the leaving state of a process
- MEDIUM: mworker-prog: implements 'option start-on-reload'
- CLEANUP: mworker: remove the type field in mworker_proc
- MEDIUM: mworker/cli: export the HAPROXY_MASTER_CLI variable
- MINOR: cli: don't add a semicolon at the end of HAPROXY_CLI
- MINOR: mworker: export HAPROXY_MWORKER=1 when running in mworker mode
- MINOR: init: add a "set-dumpable" global directive to enable core dumps
- BUG/MINOR: listener/mq: correctly scan all bound threads under low load
- BUG/MINOR: mworker: mworker_kill should apply on every children
- BUG/MINOR: mworker: don't exit with an ambiguous value
- BUG/MINOR: mworker: ensure that we still quits with SIGINT
- REGTESTS: exclude tests that require ssl, pcre if no such feature is enabled
- BUG/MINOR: mux-h1: Process input even if the input buffer is empty
- BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side has done
- BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf is empty
- BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
- MINOR: mux-h1: Handle read0 during TCP splicing
- BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
- BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
- BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
- BUG/MINOR: http_fetch/htx: Use HTX versions if the proxy enables the HTX mode
- BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to the rq.
- BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the rq_lock.
- MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
- MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
- MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet list.
- BUG/MEDIUM: applets: Don't use task_in_rq().
- BUG/MAJOR: task: make sure never to delete a queued task
- MINOR: task/thread: factor out a wake-up condition
- CLEANUP: task: remain consistent when using the task's handler
- MEDIUM: tasks: Merge task_delete() and task_free() into task_destroy().
- MEDIUM: tasks: Don't account a destroyed task as a runned task.
- BUG/MINOR: contrib/prometheus-exporter: Fix a typo in the run-queue metric type
- MINOR: contrib/prometheus-exporter: Remove usless rate metrics
- MINOR: contrib/prometheus-exporter: Rename some metrics to be more usable
- MINOR: contrib/prometheus-exporter: Follow best practices about metrics type
- BUG/MINOR: mworker: disable busy polling in the master process
- MEDIUM: tasks: Use __ha_barrier_store after modifying global_tasks_mask.
- MEDIUM: ssl: Give ssl_sock its own context.
- MEDIUM: connections: Move some fields from struct connection to ssl_sock_ctx.
- MEDIUM: ssl: provide its own subscribe/unsubscribe function.
- MEDIUM: connections: Provide a xprt_ctx for each xprt method.
- MEDIUM: ssl: provide our own BIO.
- BUILD/medium: ssl: Fix build with OpenSSL < 1.1.0
- MINOR: peers: adds counters on show peers about tasks calls.
- MEDIUM: enable travis-ci builds
- MINOR: fd: Add a counter of used fds.
- MEDIUM: connections: Add a way to control the number of idling connections.
- BUG/MEDIUM: maps: only try to parse the default value when it's present
- BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
- REGTEST: Missing REQUIRE_VERSION declarations.
- MINOR: proto_tcp: tcp-request content: enable set-dst and set-dst-var
- BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
- BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP rules
- BUG/MEDIUM: stream: Don't request a server connection if a shutw was scheduled
- BUG/MINOR: 51d: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
- BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
- MINOR: gcc: Fix a silly gcc warning in connect_server()
- MINOR: ssl/cli: async fd io-handlers printable on show fd
- Revert "CLEANUP: wurfl: remove dead, broken and unmaintained code"
- BUILD: add USE_WURFL to the list of known build options
- MINOR: wurfl: indicate in haproxy -vv the wurfl version in use
- BUILD: wurfl: build fix for 1.9/2.0 code base
- CLEANUP: wurfl: removed deprecated methods
- DOC: wurfl: added point of contact in MAINTAINERS file
- MINOR: wurfl: enabled multithreading mode
- MINOR: contrib: dummy wurfl library
- MINOR: dns: dns_requester structures are now in a memory pool
- MINOR: dns: move callback affection in dns_link_resolution()
- MINOR: obj_type: new object type for struct stream
- MINOR: action: new '(http-request|tcp-request content) do-resolve' action
- MINOR: log: Extract some code to send syslog messages.
- REGTEST: replace LEVEL option by a more human readable one.
- REGTEST: rename the reg test files.
- REGTEST: adapt some reg tests after renaming.
- REGTEST: make the "run-regtests" script search for tests in reg-tests by default
- BUG/MAJOR: stream: Missing DNS context initializations.
- BUG/MEDIUM: stream: Fix the way early aborts on the client side are handled
- BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet handler
- BUG/MEDIUM: ssl: Return -1 on recv/send if we got EAGAIN.
- BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB
- BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies only
- BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during startup
- CLEANUP: task: report calls as unsigned in show sess
- MINOR: tasks/activity: report the context switch and task wakeup rates
- MINOR: stream: measure and report a stream's call rate in "show sess"
- MINOR: applet: measure and report an appctx's call rate in "show sess"
- BUILD: extend Travis CI config to support more platforms
- REGTEST: exclude osx and generic targets for 40be_2srv_odd_health_checks
- REGTEST: relax the IPv6 address format checks in converters_ipmask_concat_strcmp_field_word
- REGTEST: exclude OSX and generic targets from abns_socket.vtc
- BUILD: travis: remove the "allow_failures" entry
- BUG/MINOR: activity: always initialize the profiling variable
- MINOR: activity: make the profiling status per thread and not global
- MINOR: activity: enable automatic profiling turn on/off
- CLEANUP: standard: use proper const to addr_to_str() and port_to_str()
- BUG/MINOR: proto_http: properly reset the stream's call rate on keep-alive
- MINOR: connection: make the debugging helper functions safer
- MINOR: stream/debug: make a stream dump and crash function
- MEDIUM: appctx/debug: force a crash if an appctx spins over itself forever
- MEDIUM: stream/debug: force a crash if a stream spins over itself forever
- MEDIUM: streams: measure processing time and abort when detecting bugs
- BUILD/MEDIUM: contrib: Dummy DeviceAtlas API.
- MEDIUM: da: HTX mode support.
- BUG/MEDIUM: mux-h2: properly deal with too large headers frames
- BUG/MINOR: http: Call stream_inc_be_http_req_ctr() only one time per request
- BUG/MEDIUM: spoe: arg len encoded in previous frag frame but len changed
- MINOR: spoe: Use the sample context to pass frag_ctx info during encoding
- DOC: contrib/modsecurity: Typos and fix the reject example
- BUG/MEDIUM: contrib/modsecurity: If host header is NULL, don't try to strdup it
- MINOR: log: Add "sample" new keyword to "log" lines.
- MINOR: log: Enable the log sampling and load-balancing feature.
- DOC: log: Document the sampling and load-balancing logging feature.
- REGTEST: Add a new reg test for log load-balancing feature.
- BUG/MAJOR: map/acl: real fix segfault during show map/acl on CLI
- REGTEST: Make this reg test be Linux specific.
- CLEANUP: task: move the task_per_thread definition to task.h
- MINOR: activity: report context switch counts instead of rates
- MINOR: threads: Implement HA_ATOMIC_LOAD().
- BUG/MEDIUM: port_range: Make the ring buffer lock-free.
- BUG/MEDIUM: listener: Fix how unlimited number of consecutive accepts is handled
- MINOR: config: Test validity of tune.maxaccept during the config parsing
- CLEANUP: config: Don't alter listener->maxaccept when nbproc is set to 1
- BUG/MEDIUM: servers: fix typo "src" instead of "srv"
- BUG/MEDIUM: ssl: Don't pretend we can retry a recv/send if we got a shutr/w.
- BUG/MINOR: haproxy: fix rule->file memory leak
- BUG/MINOR: log: properly free memory on logformat parse error and deinit()
- BUG/MINOR: checks: free memory allocated for tasklets
- BUG/MEDIUM: pattern: fix memory leak in regex pattern functions
- BUG/MEDIUM: channels: Don't forget to reset output in channel_erase().
- BUG/MEDIUM: connections: Make sure we remove CO_FL_SESS_IDLE on disown.
- MINOR: threads: flatten the per-thread cpu-map
- MINOR: init/threads: remove the useless tids[] array
- MINOR: init/threads: make the threads array global
- BUG/MEDIUM: ssl: Use the early_data API the right way.
- BUG/MEDIUM: streams: Don't add CF_WRITE_ERROR if early data were rejected.
- MEDIUM: streams: Add the ability to retry a request on L7 failure.
- MEDIUM: streams: Add a way to replay failed 0rtt requests.
- MEDIUM: streams: Add a new keyword for retry-on, "junk-response"
- BUG/MINOR: stream: also increment the retry stats counter on L7 retries
- BUG/MEDIUM: checks: make sure the warmup task takes the server lock
- BUG/MINOR: logs/threads: properly split the log area upon startup
- BUILD: extend travis-ci matrix
- CLEANUP: Remove appsession documentation
- DOC: Fix typo in keyword matrix
- BUILD: remove "build_libressl" duplicate declaration
- BUILD: travis-ci: get back to osx without openssl support
- BUILD: enable several LibreSSL hacks, including
- BUILD: temporarily mark LibreSSL builds as allowed to fail
- BUILD: travis: TMPDIR replacement.
- BUG/MEDIUM: ssl: Don't attempt to use early data with libressl.
- MINOR: doc: Document allow-0rtt on the server line.
- MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
- MEDIUM: proto: Change the prototype of the connect() method.
- MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
- MINOR: config: Extract the code of "stick-table" line parsing.
- BUILD/MINOR: stick-table: Compilation fix.
- MEDIUM: stick-table: Stop handling stick-tables as proxies.
- MINOR: stick-tables: Add peers process binding computing.
- MINOR: stick-table: Add prefixes to stick-table names.
- MINOR: peers: Do not emit global stick-table names.
- DOC: Update for "table" lines in "peers" section.
- REGTEST: Add reg tests for "table" lines in "peers" sections.
- MEDIUM: regex: modify regex_comp() to atomically allocate/free the my_regex struct
- REGTEST: make the tls_health_checks test much faster
- REGTEST: make the "table in peers" test require v2.0
- BUG/MINOR: mux-h2: rely on trailers output not input to turn them to empty data
- BUG/MEDIUM: h2/htx: always fail on too large trailers
- MEDIUM: mux-h2: discard contents that are to be sent after a shutdown
- BUG/MEDIUM: mux-h2/htx: never wait for EOM when processing trailers
- BUG/MEDIUM: h2/htx: never leave a trailers block alone with no EOM block
- REGTEST: Flag some slow reg tests.
- REGTEST: Reg tests file renaming.
- REGTEST: Wrong renaming for one reg test.
- REGTEST: Wrong assumption in IP:port logging test.
- BUG/MINOR: mworker/ssl: close OpenSSL FDs on reload
- MINOR: systemd: Use the variables from /etc/default/haproxy
- MINOR: systemd: Make use of master socket in systemd unit
- MINOR: systemd: support /etc/sysconfig/ for redhat based distrib
- BUG/MEDIUM: stick-table: fix regression caused by a change in proxy struct
- BUG/MEDIUM: tasks: fix possible segfault on task_destroy()
- CLEANUP: task: remove unneeded tests before task_destroy()
- MINOR: mworker: support a configurable maximum number of reloads
- BUG/MINOR: mux-h2: fix the condition to close a cs-less h2s on the backend
- BUG/MEDIUM: spoe: Be sure the sample is found before setting its context
- BUG/MINOR: mux-h1: Fix the parsing of trailers
- BUG/MINOR: htx: Never transfer more than expected in htx_xfer_blks()
- MINOR: htx: Split on DATA blocks only when blocks are moved to an HTX message
- MINOR: htx: Don't try to append a trailer block with the previous one
- MINOR: htx: Remove support for unused OOB HTX blocks
- BUILD: travis-ci bugfixes and improvements
- BUG/MEDIUM: servers: Don't use the same srv flag for cookie-set and TFO.
- BUG/MEDIUM: h2: Make sure we set send_list to NULL in h2_detach().
- BUILD: ssl: fix again a libressl build failure after the openssl FD leak fix
- CLEANUP: ssl-sock: use HA_OPENSSL_VERSION_NUMBER instead of OPENSSL_VERSION_NUMBER
- BUILD: ssl: make libressl use its own version numbers
- CLEANUP: ssl: remove 57 occurrences of useless tests on LIBRESSL_VERSION_NUMBER
- MINOR: ssl: enable aes_gcm_dec on LibreSSL
- BUILD: ssl: fix libressl build again after aes-gcm-enc
- REORG: ssl: move openssl-compat from proto to common
- REORG: ssl: move some OpenSSL defines from ssl_sock to openssl-compat
- CLEANUP: ssl: never include openssl/*.h outside of openssl-compat.h anymore
- CLEANUP: ssl: make inclusion of openssl headers safe
- BUILD: add BoringSSL to travis-ci build matrix
- BUILD: threads: Add __ha_cas_dw fallback for single threaded builds
- BUG/MINOR: stream: Attach the read side on the response as soon as possible
- BUG/MEDIUM: http: Use pointer to the begining of input to parse message headers
- BUG/MEDIUM: h2: Don't check send_wait to know if we're in the send_list.
- BUG/MEDIUM: streams: Make sur SI_FL_L7_RETRY is set before attempting a retry.
- MEDIUM: streams: Add a new http action, disable-l7-retry.
- MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
- BUG/MINOR: vars: Fix memory leak in vars_check_arg
- BUILD: travis-ci: make TMPDIR global variable in travis-ci
- CLEANUP: ssl: move the SSL_OP_* and SSL_MODE_* definitions to openssl-compat
- CLEANUP: ssl: remove ifdef around SSL_CTX_get_extra_chain_certs()
- CLEANUP: ssl: move all BIO_* definitions to openssl-compat
- BUILD: threads: fix again the __ha_cas_dw() definition
- BUG/MAJOR: mux-h2: do not add a stream twice to the send list
- Revert "BUG/MINOR: vars: Fix memory leak in vars_check_arg"
- BUG/MINOR: peers: Fix memory leak in cfg_parse_peers
- BUG/MINOR: htx: make sure to always initialize the HTTP method when parsing a buffer
- REGTEST: fix tls_health_checks random failures on MacOS in Travis-CI
- MINOR: spoe: Set the argument chunk size to 0 when SPOE variables are checked
- BUG/MINOR: vars: Fix memory leak in vars_check_arg
- BUG/MAJOR: ssl: segfault upon an heartbeat request
- MINOR: spoa-server: Clone the v1.7 spoa-example project
- MINOR: spoa-server: move some definition from spoa_server.c to spoa_server.h
- MINOR: spoa-server: Externalise debug functions
- MINOR: spoe-server: rename "worker" functions
- MINOR: spoa-server: Replace the thread init system by processes
- MINOR: spoa-server: With debug mode, start only one process
- MINOR: spoa-server: Allow registering external processes
- MINOR: spoa-server: Allow registering message processors
- MINOR: spoa-server: Load files
- MINOR: spoa-server: Prepare responses
- MINOR: spoa-server: Execute registered callbacks
- MINOR: spoa-server: Add Lua processing
- MINOR: spoa-server: Add python
- MINOR/DOC: spoe-server: Add documentation
- BUG/MEDIUM: connections: Don't forget to set xprt_ctx to NULL on close.
- MINOR: lists: add LIST_ADDED() to check if an element belongs to a list
- CLEANUP: mux-h2: use LIST_ADDED() instead of LIST_ISEMPTY() where relevant
- MINOR: mux-h2: add two H2S flags to report the need for shutr/shutw
- CLEANUP: mux-h2: simply use h2s->flags instead of ret in h2_deferred_shut()
- CLEANUP: connection: remove the handle field from the wait_event struct
- BUG/MINOR: log: Wrong log format initialization.
- BUG/MINOR: mux-h2: make the do_shut{r,w} functions more robust against retries
- BUG/MINOR: mworker: use after free when the PID not assigned
- MINOR: mux-h2: remove useless test on stream ID vs last in wake function
- MINOR: mux-h2: make h2_wake_some_streams() not depend on the CS flags
- MINOR: mux-h2: make h2s_wake_one_stream() the only function to deal with CS
- MINOR: mux-h2: make h2s_wake_one_stream() not depend on temporary CS flags
- BUG/MINOR: mux-h2: make sure to honor KILL_CONN in do_shut{r,w}
- CLEANUP: mux-h2: don't test for impossible CS_FL_REOS conditions
- MINOR: mux-h2: add macros to check multiple stream states at once
- MINOR: mux-h2: stop relying on CS_FL_REOS
- BUG/MEDIUM: mux-h2: Set EOI on the conn_stream during h2_rcv_buf()
- BUILD: debug: make gcc not complain on the ABORT_NOW() macro
- MINOR: debug: add a new BUG_ON macro
- MINOR: h2: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
- MINOR: h1: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
- MINOR: connections: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
- BUILD: ist: turn the lower/upper case tables to literal on obsolete linkers
Gil Bahat reported build issues on Cygwin starting with 1.9 due to a
difference in the way the linker handles the weak symbols there,
causing multiple declarations of ist_lc[] and ist_uc[]. It's likely
that this issue could also happen on any older or non-ELF linker.
This patch addresses this by using literals instead on such platforms,
leaving it to the compiler to merge the constants when it can. On other
platforms the resulting executable is slightly larger due to strings
that could not be merged but this is a minor detail compared to not
being able to build at all.
If this change alone is confirmed to fix these issues, it's safe to
backport to 1.9.
It is not legal to subscribe if we're already subscribed, or to unsubscribe
if we did not subscribe, so instead of trying to handle those cases, just
assert that it's ok using the new BUG_ON() macro.
It is not legal to subscribe if we're already subscribed, or to unsubscribe
if we did not subscribe, so instead of trying to handle those cases, just
assert that it's ok using the new BUG_ON() macro.
It is not legal to subscribe if we're already subscribed, or to unsubscribe
if we did not subscribe, so instead of trying to handle those cases, just
assert that it's ok using the new BUG_ON() macro.
We do have some code paths testing for impossible errors that tend to
be quite confusing, first for maintenance (what to do on such errors,
and how far to guess the bug), second for developers as it tends to
hide the main purpose and expectations of these call places. Also
most of the time impossible errors are ignored by the callers so the
tests are not even usable during debugging.
Let's instead implement a BUG_ON macro which takes a condition, which
if true, will cause a message to be emitted and optionally to crash the
process. Additionally, these calls inserted at various places server as
hints and documentation for developers to know that such conditions
must absolutely not happen.
This is only enabled when DEBUG_STRICT or DEBUG_STRICT_NOCRASH are set.
As its name implies, DEBUG_STRICT_NOCRASH only performs the test but
does not crash, which can be useful to track some checkpoints.
At the moment nothing uses this code.
On recent gcc versions with the null-deref checks, ABORT_NOW() rightfully
emits such a warning. But here it's on purpose. Simply changing the memory
address to 1 makes gcc happy.
Just like CS_FL_REOS previously, the CS_FL_EOI flag is abused as a proxy
for H2_SF_ES_RCVD. The problem is that this flag is consumed by the
application layer and is set immediately when an end of stream was met,
which is too early since the application must retrieve the rxbuf's
contents first. The effect is that some transfers are truncated (mostly
the first one of a connection in most tests).
The problem of mixing CS flags and H2S flags in the H2 mux is not new
(and is currently being addressed) but this specific one was emphasized
in commit 63768a63d ("MEDIUM: mux-h2: Don't mix the end of the message
with the end of stream") which was backported to 1.9. Note that other
flags, particularly CS_FL_REOS still need to be asynchronously reported,
though their impact seems more limited for now.
This patch makes sure that all internal uses of CS_FL_EOI are replaced
with a test on H2_SF_ES_RCVD (as there is a 1-to-1 equivalence) and that
CS_FL_EOI is only reported once the rxbuf is empty.
This should ideally be backported to 1.9 unless it causes too much
trouble due to the recent changes in this area, as 1.9 *seems* not
to be directly affected by this bug.
This flag was introduced early in 1.9 development (a3f7efe00) to report
the fact that the rxbuf that was present on the conn_stream was followed
by a shutr. Since then the rxbuf moved from the conn_stream to the h2s
(638b799b0) but the flag remained on the conn_stream. It is problematic
because some state transitions inside the mux depend on it, thus depend
on the CS, and as such have to test for its existence before proceeding.
This patch replaces the test on CS_FL_REOS with a test on the only
states that set this flag (H2_SS_CLOSED, H2_SS_HREM, H2_SS_ERROR).
The few places where the flag was set were removed (the flag is not
used by the data layer).
This flag is currently set when an incoming close was received, which
results in the stream being in either H2_SS_HREM, H2_SS_CLOSED, or
H2_SS_ERROR states, so let's remove the test for the OPEN and HLOC
cases.
If the stream closes and quits while there's no room in the mux buffer
to send an RST frame, next time it is attempted it will not lead to
the connection being closed because the conn_stream will have been
released and the KILL_CONN flag with it as well.
This patch reserves a new H2_SF_KILL_CONN flag that is copied from
the CS when calling shut{r,w} so that the stream remains autonomous
on this even when the conn_stream leaves.
This should ideally be backported to 1.9 though it depends on several
previous patches that may or may not be suitable for backporting. The
severity is very low so there's no need to insist in case of trouble.
In h2s_wake_one_stream() we used to rely on the temporary flags used to
adjust the CS to determine the new h2s state. This really is not convenient
and creates far too many dependencies. This commit just moves the same
condition to the places where the temporary flags were set so that we
don't have to rely on these anymore. Whether these are relevant or not
was not the subject of the operation, what matters was to make sure the
conditions to adjust the stream's state and the CS's flags remain the
same. Later it could be studied if these conditions are correct or not.
h2s_wake_one_stream() has access to all the required elements to update
the connstream's flags and figure the necessary state transitions, so
let's move the conditions there from h2_wake_some_streams().
It's problematic to have to pass some CS flags to this function because
that forces some h2s state transistions to update them just in time
while some of them are supposed to only be updated during I/O operations.
As a first step this patch transfers the decision to pass CS_FL_ERR_PENDING
from the caller to the leaf function h2s_wake_one_stream(). It is easy
since this is the only flag passed there and it depends on the position of
the stream relative to the last_sid if it was set.
h2_wake_some_streams() first looks up streams whose IDs are greater than
or equal to last+1, then checks if the id is lower than or equal to last,
which by definition will never match. Let's remove this confusing leftover
from ancient code.
Commit 4528611 ("MEDIUM: mworker: store the leaving state of a process")
introduced a bug in the mworker_env_to_proc_list() function.
This is very unlikely to occur since the PID should always be assigned.
It can probably happen if the environment variable is corrupted.
No backport needed.
These functions may fail to emit an RST or an empty DATA frame because
the mux is full or busy. Then they subscribe the h2s and try again.
However when doing so, they will already have marked the error state on
the stream and will not pass anymore through the sequence resulting in
the failed frame to be attempted to be sent again nor to the close to
be done, instead they will return a success.
It is important to only leave when the stream is already closed, but
to go through the whole sequence otherwise.
This patch should ideally be backported to 1.9 though it's possible that
the lack of the WANT_SHUT* flags makes this difficult or dangerous. The
severity is low enough to avoid this in case of trouble.
This patch fixes an issue introduced by 0bad840b commit
"MINOR: log: Extract some code to send syslog messages" which leaded
to wrong log format variable initializations at least for "short" and "raw" format.
This commit skipped the cases where even if passed to __do_send_log(), the
syslog tag and syslog pid string must not be used to format the log message
with "short" and "raw". This is done iniatilizing "tag_max" and "pid_max"
variables (the lengths of the tag and pid strings) to 0, then updating to them to
the length of the tag and pid strings passed as variables to __do_send_log()
depending on the log format and in every cases using this length for the iovec
variable used to send() the log.
This bug is specific to 2.0.
It was only set and not consumed after the previous change. The reason
is that the task's context always contains the relevant information,
so there is no need for a second pointer.
This one used to rely on the combined return statuses of the shutr/w
functions but now that we have the H2_SF_WANT_SHUT{R,W} flags we don't
need this anymore if we properly remove these flags after their operations
succeed. This is what this patch does.
Currently when a shutr/shutw fails due to lack of buffer space, we abuse
the wait_event's handle pointer to place up to two bits there in addition
to the original pointer. This pointer is not used for anything but this
and overall the intent becomes clearer with h2s flags than with these
two alien bits in the pointer, so let's use clean flags now.
Lots of places were using LIST_ISEMPTY() to detect if a stream belongs
to one of the send lists or to detect if a connection was already
waiting for a buffer or attached to an idle list. Since these ones are
not list heads but list elements, let's use LIST_ADDED() instead.
Some code parts use LIST_ISEMPTY() a lot on list elements to detect
if they were reset consecutive to their removal from a list, but this
test is always confusing as this was initially designed for list heads.
Instead let's have a new macro, LIST_ADDED(), which returns true when
the element is in a list (i.e. it's not "empty").
In conn_xprt_close(), after calling xprt->close(), don't forget to set
conn->xprt_ctx to NULL, or we may attempt to reuse the now-free'd
conn->xprt_ctx if the connection failed and we're retrying it.
I will replace thread by processes. Note that, I keep the pthread_key
system for identifiying process in the same way that threads. Note
also that I keep commented out the original thread code because I hope
to reactivate it.
7b5fd1e ("MEDIUM: connections: Move some fields from struct connection
to ssl_sock_ctx.") introduced a bug in the heartbleed mitigation code.
Indeed the code used conn->ctx instead of conn->xprt_ctx for the ssl
context, resulting in a null dereference.
vars_check_arg previously leaked the string containing the variable
name:
Consider this config:
frontend fe1
mode http
bind :8080
http-request set-header X %[var(txn.host)]
Starting HAProxy and immediately stopping it by sending a SIGINT makes
Valgrind report this leak:
==7795== 9 bytes in 1 blocks are definitely lost in loss record 15 of 71
==7795== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==7795== by 0x4AA2AD: my_strndup (standard.c:2227)
==7795== by 0x51FCC5: make_arg_list (arg.c:146)
==7795== by 0x4CF095: sample_parse_expr (sample.c:897)
==7795== by 0x4BA7D7: add_sample_to_logformat_list (log.c:495)
==7795== by 0x4BBB62: parse_logformat_string (log.c:688)
==7795== by 0x4E70A9: parse_http_req_cond (http_rules.c:239)
==7795== by 0x41CD7B: cfg_parse_listen (cfgparse-listen.c:1466)
==7795== by 0x480383: readcfgfile (cfgparse.c:2089)
==7795== by 0x47A081: init (haproxy.c:1581)
==7795== by 0x4049F2: main (haproxy.c:2591)
This leak can be detected even in HAProxy 1.6, this patch thus should
be backported to all supported branches
[Cf: This fix was reverted because the chunk's area was inconditionnaly
released, making haproxy to crash when spoe was enabled. Now the chunk is
released by calling chunk_destroy(). This function takes care of the
chunk's size to release it or not. It is the responsibility of callers to
set or not the chunk's size.]
When SPOE variables are registered during HAProxy startup, the argument used to
call the function vars_check_arg() uses the trash area. To be sure it is never
released by the callee function, the size of the internal chunk (arg.data.str)
is set to 0. It is important to do so because, to fix a memory leak, this buffer
must be released by the function vars_check_arg().
This patch must be backported to 1.9.
Since commit 2eb1c79df ("REGTEST: make the tls_health_checks test much
faster") the build tests randomly fail on MacOS on Travis-CI. Each time
this test is reponsible for the failure, showing huge response times
possibly indicating that the VMs running the tests are sometimes
overloaded. Since this delay directly impacts the whole regtest execution
time everywhere, it's important not to inflate it too much. It was bumped
to 100ms instead of 40, that doesn't add significantly to the perceived
execution time and should be enough for Travis since test reports have
shown around 60-70 ms.
smp_prefetch_htx() is used when trying to access the contents of an HTTP
buffer from the TCP rulesets. The method was not properly set in this
case, which will cause the sample fetch methods relying on the method
to randomly fail in this case.
Thanks to Tim Düsterhus for reporting this issue (#97).
This fix must be backported to 1.9.
cfg_parse_peers previously leaked the contents of the `kws` string,
as it was unconditionally filled using bind_dump_kws, but only used
(and freed) within the error case.
Move the dumping into the error case to:
1. Ensure that the registered keywords are actually printed as least once.
2. The contents of kws are not leaked.
This move allows to narrow the scope of `kws`, so this is done as well.
This bug was found using valgrind:
==28217== 590 bytes in 1 blocks are definitely lost in loss record 51 of 71
==28217== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28217== by 0x4AD4C7: indent_msg (standard.c:3676)
==28217== by 0x47E962: cfg_parse_peers (cfgparse.c:700)
==28217== by 0x480273: readcfgfile (cfgparse.c:2147)
==28217== by 0x479D51: init (haproxy.c:1585)
==28217== by 0x404A02: main (haproxy.c:2585)
with this super simple configuration:
peers peers
bind :8081
server A
This bug exists since the introduction of cfg_parse_peers in commit
355b2033ec (which was introduced for HAProxy
2.0, but marked as backportable). It should be backported to all branches
containing that commit.
This reverts commit 6ea00195c4.
As found by Christopher, this fix is not correct due to the way args
are built at various places. For example some config or runtime parsers
will place a substring pointer there, and calling free() on it will
immediately crash the program. A quick audit of the code shows that
there are not that many users, but the way it's done requires to
properly set the string as a regular chunk (size=0 if free not desired,
then call chunk_destroy() at release time), and given that the size is
currently set to len+1 in all parsers, a deeper audit needs to be done
to figure the impacts of not setting it anymore.
Thus for now better leave this harmless leak which impacts only the
config parsing time.
This fix must be backported to all branches containing the fix above.
In this long thread, Maciej Zdeb reported that the H2 mux was still
going through endless loops from time to time :
https://www.mail-archive.com/haproxy@formilux.org/msg33709.html
What happens is the following :
- in h2s_frt_make_resp_data() we can set H2_SF_BLK_SFCTL and remove the
stream from the send_list
- then in h2_shutr() and h2_shutw(), we check if the list is empty before
subscribing the element, which is true after the case above
- then in h2c_update_all_ws() we still have H2_SF_BLK_SFCTL with the item
in the send_list, thus LIST_ADDQ() adds it a second time.
This patch adds a check of list emptiness before performing the LIST_ADDQ()
when the flow control window opens. Maciej reported that it reliably fixed
the problem for him.
As later discussed with Olivier, this fixes the consequence of the issue
rather than its cause. The root cause is that a stream should never be in
the send_list with a blocking flag set and the various places that can lead
to this situation must be revisited. Thus another fix is expected soon for
this issue, which will require some observation. In the mean time this one
is easy enough to validate and to backport.
Many thanks to Maciej for testing several versions of the patch, each
time providing detailed traces which allowed to nail the problem down.
This patch must be backported to 1.9.
This low-level asm implementation of a double CAS was implemented only
for certain architectures (x86_64, armv7, armv8). When threads are not
used, they were not defined, but since they were called directly from
a few locations, they were causing build issues on certain platforms
with threads disabled. This was addressed in commit f4436e1 ("BUILD:
threads: Add __ha_cas_dw fallback for single threaded builds") by
making it fall back to HA_ATOMIC_CAS() when threads are not defined,
but this actually made the situation worse by breaking other cases.
This patch fixes this by creating a high-level macro HA_ATOMIC_DWCAS()
which is similar to HA_ATOMIC_CAS() except that it's intended to work
on a double word, and which rely on the asm implementations when threads
are in use, and uses its own open-coded implementation when threads are
not used. The 3 call places relying on __ha_cas_dw() were updated to
use HA_ATOMIC_DWCAS() instead.
This change was tested on i586, x86_64, armv7, armv8 with and without
threads with gcc 4.7, armv8 with gcc 5.4 with and without threads, as
well as i586 with gcc-3.4 without threads. It will need to be backported
to 1.9 along with the fix above to fix build on armv7 with threads
disabled.
