Commit Graph

106 Commits

Author SHA1 Message Date
Willy Tarreau
bc84657410 DOC: internals: document the IST API
This one was missing. It should be easier to use now. It is obvious that
some functions are missing, and it looks like ist2str() and istpad() are
exactly the same.
2021-11-08 16:50:48 +01:00
Willy Tarreau
08d3220de5 [RELEASE] Released version 2.5-dev13
Released version 2.5-dev13 with the following main changes :
    - SCRIPTS: git-show-backports: re-enable file-based filtering
    - MINOR: jwt: Make invalid static JWT algorithms an error in `jwt_verify` converter
    - MINOR: mux-h2: add trace on extended connect usage
    - BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
    - MINOR: stream/mux: implement websocket stream flag
    - MINOR: connection: implement function to update ALPN
    - MINOR: connection: add alternative mux_ops param for conn_install_mux_be
    - MEDIUM: server/backend: implement websocket protocol selection
    - MINOR: server: add ws keyword
    - BUG/MINOR: resolvers: fix sent messages were counted twice
    - BUG/MINOR: resolvers: throw log message if trash not large enough for query
    - MINOR: resolvers/dns: split dns and resolver counters in dns_counter struct
    - MEDIUM: resolvers: rename dns extra counters to resolvers extra counters
    - BUG/MINOR: jwt: Fix jwt_parse_alg incorrectly returning JWS_ALG_NONE
    - DOC: add QUIC instruction in INSTALL
    - CLEANUP: halog: Remove dead stores
    - DEV: coccinelle: Add ha_free.cocci
    - CLEANUP: Apply ha_free.cocci
    - DEV: coccinelle: Add rule to use `istnext()` where possible
    - CLEANUP: Apply ist.cocci
    - REGTESTS: Use `feature cmd` for 2.5+ tests (2)
    - DOC: internals: move some API definitions to an "api" subdirectory
    - MINOR: quic: Allocate listener RX buffers
    - CLEANUP: quic: Remove useless code
    - MINOR: quic: Enhance the listener RX buffering part
    - MINOR: quic: Remove a useless lock for CRYPTO frames
    - MINOR: quic: Use QUIC_LOCK QUIC specific lock label.
    - MINOR: backend: Get client dst address to set the server's one only if needful
    - MINOR: compression: Warn for 'compression offload' in defaults sections
    - MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
    - DOC: configuration: move the default log formats to their own section
    - MINOR: ssl: make the ssl_fc_sni() sample-fetch function always available
    - MEDIUM: log: add the client's SNI to the default HTTPS log format
    - DOC: config: add an example of reasonably complete error-log-format
    - DOC: config: move error-log-format before custom log format
2021-11-06 09:25:57 +01:00
Willy Tarreau
a62f184d3d DOC: internals: move some API definitions to an "api" subdirectory
It's not always easy to figure that there are some docs on internal API
stuff, let's move them to their own directory. There's a diagram for
lists that could be placed there but instead would deserve a greppable
description for quick lookups, so it was not moved there.
2021-11-05 11:53:22 +01:00
Ilya Shipitsin
01881087fc CLEANUP: assorted typo fixes in the code and comments
This is 25th iteration of typo fixes
2021-08-16 12:37:59 +02:00
Willy Tarreau
f69fea64e0 MAJOR: fd: get rid of the DWCAS when setting the running_mask
Right now we're using a DWCAS to atomically set the running_mask while
being constrained by the thread_mask. This DWCAS is annoying because we
may seriously need it later when adding support for thread groups, for
checking that the running_mask applies to the correct group.

It turns out that the DWCAS is not strictly necessary because we never
need it to set the thread_mask based on the running_mask, only the other
way around. And in fact, the running_mask is always cleared alone, and
the thread_mask is changed alone as well. The running_mask is only
relevant to indicate a takeover when the thread_mask matches it. Any
bit set in running and not present in thread_mask indicates a transition
in progress.

As such, it is possible to re-arrange this by using a regular CAS around a
consistency check between running_mask and thread_mask in fd_update_events
and by making a CAS on running_mask then an atomic store on the thread_mask
in fd_takeover(). The only other case is fd_delete() but that one already
sets the running_mask before clearing the thread_mask, which is compatible
with the consistency check above.

This change has happily survived 10 billion takeovers on a 16-thread
machine at 800k requests/s.

The fd-migration doc was updated to reflect this change.
2021-08-04 16:03:36 +02:00
Willy Tarreau
88babd9944 DOC: internals: document the FD takeover process
This explains the traps to avoid and the sequence that leads to
consistent use of an FD known by multiple threads at once. This
was co-authored with Olivier.
2021-07-30 17:41:55 +02:00
Remi Tricot-Le Breton
653a16769a DOC: internals: update the SSL architecture schema
Add the new relation between the CA file tree entries and the ckch
instances introduced during CA file hot update implementation (see
GitHub #1057).
2021-05-17 10:50:24 +02:00
Willy Tarreau
46b93afdb3 [RELEASE] Released version 2.4-dev19
Released version 2.4-dev19 with the following main changes :
    - BUG/MINOR: hlua: Don't rely on top of the stack when using Lua buffers
    - BUG/MEDIUM: cli: prevent memory leak on write errors
    - BUG/MINOR: ssl/cli: fix a lock leak when no memory available
    - MINOR: debug: add a new "debug dev sym" command in expert mode
    - MINOR: pools/debug: slightly relax DEBUG_DONT_SHARE_POOLS
    - CI: Github Actions: switch to LibreSSL-3.3.3
    - MINOR: srv: close all idle connections on shutdown
    - MINOR: connection: move session_list member in a union
    - MEDIUM: mux_h1: release idling frontend conns on soft-stop
    - MEDIUM: connection: close front idling connection on soft-stop
    - MINOR: tools: add functions to retrieve the address of a symbol
    - CLEANUP: activity: mark the profiling and task_profiling_mask __read_mostly
    - MINOR: activity: add a "memory" entry to "profiling"
    - MINOR: activity: declare the storage for memory usage statistics
    - MEDIUM: activity: collect memory allocator statistics with USE_MEMORY_PROFILING
    - MINOR: activity: clean up the show profiling io_handler a little bit
    - MINOR: activity: make "show profiling" support a few arguments
    - MINOR: activity: make "show profiling" also dump the memoery usage
    - MINOR: activity: add the profiling.memory global setting
    - BUILD: makefile: add new option USE_MEMORY_PROFILING
    - MINOR: channel: Rely on HTX version if appropriate in channel_may_recv()
    - BUG/MINOR: stream-int: Don't block reads in si_update_rx() if chn may receive
    - MINOR: conn-stream: Force mux to wait for read events if abortonclose is set
    - MEDIUM: mux-h1: Don't block reads when waiting for the other side
    - BUG/MEDIUM: mux-h1: Properly report client close if abortonclose option is set
    - REGTESTS: Add script to test abortonclose option
    - MINOR: mux-h1: clean up conditions to enabled and disabled splicing
    - MINOR: mux-h1: Subscribe for sends if output buffer is not empty in h1_snd_pipe
    - MINOR: mux-h1: Always subscribe for reads when splicing is disabled
    - MEDIUM: mux-h1: Wake H1 stream when both sides a synchronized
    - CLEANUP: mux-h1: rename WAIT_INPUT/WAIT_OUTPUT flags
    - MINOR: mux-h1: Manage processing blocking flags on the H1 stream
    - BUG/MINOR: stream: Decrement server current session counter on L7 retry
    - BUG/MINOR: config: fix uninitialized initial state in ".if" block evaluator
    - BUG/MINOR: config: add a missing "ELIF_TAKE" test for ".elif" condition evaluator
    - BUG/MINOR: config: .if/.elif should also accept negative integers
    - MINOR: config: centralize the ".if"/".elif" condition parser and evaluator
    - MINOR: config: keep up-to-date current file/line/section in the global struct
    - MINOR: config: support some pseudo-variables for file/line/section
    - BUILD: activity: do not include malloc.h
    - MINOR: arg: improve the error message on missing closing parenthesis
    - MINOR: global: export the build features string list
    - MINOR: global: add version comparison functions
    - MINOR: config: improve .if condition error reporting
    - MINOR: config: make cfg_eval_condition() support predicates with arguments
    - MINOR: config: add predicate "defined()" to conditional expression blocks
    - MINOR: config: add predicates "streq()" and "strneq()" to conditional expressions
    - MINOR: config: add predicate "feature" to detect certain built-in features
    - MINOR: config: add predicates "version_atleast" and "version_before" to cond blocks
    - BUG/MINOR: activity: use the new pointer to calculate the new size in realloc()
    - BUG/MINOR: stream: properly clear the previous error mask on L7 retries
    - MEDIUM: log: slightly refine the output format of alerts/warnings/etc
    - MINOR: config: add a new message directive: .diag
    - CLEANUP: cli/tree-wide: properly re-align the CLI commands' help messages
    - BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
    - BUG/MINOR: checks: Handle synchronous connect when a tcpcheck is started
    - BUG/MINOR: checks: Reschedule check on observe mode only if fastinter is set
    - MINOR: global: define tainted flag
    - MINOR: cfgparse: add a new field flags in cfg_keyword
    - MINOR: cfgparse: implement experimental config keywords
    - MINOR: action: replace match_pfx by a keyword flags field
    - MINOR: action: implement experimental actions
    - MINOR: cli: set tainted when using CLI expert/experimental mode
    - MINOR: stats: report tainted on show info
    - MINOR: http_act: mark normalize-uri as experimental
    - BUILD: fix usage of ha_alert without format string
    - MINOR: proxy: define PR_CAP_LB
    - BUG/MINOR: server: do not report diag for peer servers with null weight
    - DOC: ssl: Extra files loading now works for backends too
    - ADDONS: make addons/ discoverable by git via .gitignore
    - DOC: ssl: Add information about crl-file option
    - MINOR: sample: improve error reporting on missing arg to strcmp() converter
    - DOC: management: mention that some fields may be emitted as floats
    - MINOR: tools: implement trimming of floating point numbers
    - MINOR: tools: add a float-to-ascii conversion function
    - MINOR: freq_ctr: add new functions to report float measurements
    - MINOR: stats: avoid excessive padding of float values with trailing zeroes
    - MINOR: stats: add the HTML conversion for float types
    - MINOR: stats: pass the appctx flags to stats_fill_info()
    - MINOR: stats: support an optional "float" option to "show info"
    - MINOR: stats: use tv_remain() to precisely compute the uptime
    - MINOR: stats: report uptime and start time as floats with subsecond resolution
    - MINOR: stats: make "show info" able to report rates as floats when asked
    - MINOR: config: mark tune.fd.edge-triggered as experimental
    - REORG: vars: move the "proc" scope variables out of the global struct
    - REORG: threads: move all_thread_mask() to thread.h
    - BUILD: wdt: include signal-t.h
    - BUILD: auth: include missing list.h
    - REORG: mworker: move proc_self from global to mworker
    - BUILD: ssl: ssl_utils requires chunk.h
    - BUILD: config: cfgparse-ssl.c needs tools.h
    - BUILD: wurfl: wurfl.c needs tools.h
    - BUILD: spoe: flt_spoe.c needs tools.h
    - BUILD: promex: service-prometheus.c needs tools.h
    - BUILD: resolvers: include tools.h
    - BUILD: config: include tools.h in cfgparse-listen.c
    - BUILD: htx: include tools.h in http_htx.c
    - BUILD: proxy: include tools.h in proxy.c
    - BUILD: session: include tools.h in session.c
    - BUILD: cache: include tools.h in cache.c
    - BUILD: sink: include tools.h in sink.c
    - BUILD: connection: include tools.h in connection.c
    - BUILD: server-state: include tools.h from server_state.c
    - BUILD: dns: include tools.h in dns.c
    - BUILD: payload: include tools.h in payload.c
    - BUILD: vars: include tools.h in vars.c
    - BUILD: compression: include tools.h in compression.c
    - BUILD: mworker: include tools.h from mworker.c
    - BUILD: queue: include tools.h from queue.c
    - BUILD: udp: include tools.h from proto_udp.c
    - BUILD: stick-table: include freq_ctr.h from stick_table.h
    - BUILD: server: include tools.h from server.c
    - BUILD: server: include missing proxy.h in server.c
    - BUILD: sink: include proxy.h in sink.c
    - BUILD: mworker: include proxy.h in mworker.c
    - BUILD: filters: include proxy.h in filters.c
    - BUILD: fcgi-app: include proxy.h in fcgi-app.c
    - BUILD: connection: move list_mux_proto() to connection.c
    - REORG: stick-table: uninline stktable_alloc_data_type()
    - REORG: stick-table: move composite address functions to stick_table.h
    - REORG: config: uninline warnifnotcap() and failifnotcap()
    - BUILD: task: remove unused includes from task.c
    - MINOR: task: stop including stream.h from task.c
    - BUILD: connection: stop including listener-t.h
    - BUILD: hlua: include proxy.h from hlua.c
    - BUILD: mux-h1: include proxy.h from mux-h1.c
    - BUILD: mux-fcgi: include proxy.h from mux-fcgi.c
    - BUILD: listener: include proxy.h from listener.c
    - BUILD: http-rules: include proxy.h from http_rules.c
    - BUILD: thread: include log.h from thread.c
    - BUILD: comp: include proxy.h from flt_http_comp.c
    - BUILD: fd: include log.h from fd.c
    - BUILD: config: do not include proxy.h nor errors.h anymore in cfgparse.h
    - BUILD: makefile: reorder object files by build time
    - DOC: Fix a few grammar/spelling issues and casing of HAProxy
    - REGTESTS: run-regtests: match both "HAProxy" and "HA-Proxy" in the version
    - MINOR: version: report "HAProxy" not "HA-Proxy" in the version output
    - DOC: remove last occurrences of "HA-Proxy" syntax
    - DOC: peers: fix the protocol tag name in the doc
    - ADMIN: netsnmp: report "HAProxy" and not "Haproxy" in output descriptions
    - MEDIUM: mailers: use "HAProxy" nor "HAproxy" in the subject of messages
    - DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
    - MINOR: tools/rnd: compute the result outside of the CAS loop
    - BUILD: http_fetch: address a few aliasing warnings with older compilers
    - BUILD: ssl: define HAVE_CRYPTO_memcmp() based on the library version
    - BUILD: errors: include stdarg in errors.h
    - REGTESTS: disable inter-thread idle connection sharing on sensitive tests
    - MINOR: cli: make "help" support a command in argument
    - MINOR: cli: sort the output of the "help" keywords
    - CLEANUP: cli/mworker: properly align the help messages
    - BUILD: memprof: make the old caller pointer a const in get_prof_bin()
    - BUILD: compat: include malloc_np.h for USE_MEMORY_PROFILING on FreeBSD
    - CI: Github Actions: enable USE_QUIC=1 for BoringSSL builds
    - BUG/MEDIUM: quic: fix null deref on error path in qc_conn_init()
    - BUILD: cli: appease a null-deref warning in cli_gen_usage_msg()
2021-05-10 07:50:26 +02:00
Willy Tarreau
714f34580e DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
Some of the Lua doc and a few places still used "Haproxy" or "HAproxy".
There was even one "HA proxy". A few of them were in an example of VTest
output, indicating that VTest ought to be fixed as well. No big deal but
better address all the remaining ones so that these inconsistencies stop
spreading around.
2021-05-09 06:50:46 +02:00
Willy Tarreau
58000fe16d DOC: remove last occurrences of "HA-Proxy" syntax
There were only a few more used as output examples and comments in a few
docs, it was the right moment to get rid of them. The file intro.txt
which explains how to parse the version also got a hint about the possible
presence of a hyphen in the name in older versions.
2021-05-09 06:29:40 +02:00
Willy Tarreau
bfd19d68f0 [RELEASE] Released version 2.4-dev17
Released version 2.4-dev17 with the following main changes :
    - MINOIR: mux-pt/trace: Register a new trace source with its events
    - BUG/MINOR: mux-pt: Fix a possible UAF because of traces in mux_pt_io_cb
    - CI: travis: Drastically clean up .travis.yml
    - CLEANUP: pattern: make all pattern tables read-only
    - MINOR: trace: replace the trace() inline function with an equivalent macro
    - MINOR: initcall: uniformize the section names between MacOS and other unixes
    - CLEANUP: initcall: rename HA_SECTION to HA_INIT_SECTION
    - MINOR: compiler: add macros to declare section names
    - CLEANUP: initcall: rely on HA_SECTION_* instead of defining its own
    - MINOR: global: declare a read_mostly section
    - MINOR: fd: move a few read-mostly variables to their own section
    - MINOR: epoll: move epoll_fd to read_mostly
    - MINOR: kqueue: move kqueue_fd to read_mostly
    - MINOR: pool: move pool declarations to read_mostly
    - MINOR: threads: mark all_threads_mask as read_mostly
    - MINOR: server: move idle_conn_task to read_mostly
    - MINOR: protocol: move __protocol_by_family to read_mostly
    - MINOR: pattern: make the pat_lru_seed read_mostly
    - MINOR: trace: make trace sources read_mostly
    - MINOR: freq_ctr: add a generic function to report the total value
    - MEDIUM: freq_ctr: make read_freq_ctr_period() use freq_ctr_total()
    - MEDIUM: freq_ctr: reimplement freq_ctr_remain_period() from freq_ctr_total()
    - MINOR: freq_ctr: add the missing next_event_delay_period()
    - MINOR: freq_ctr: unify freq_ctr and freq_ctr_period into freq_ctr
    - MEDIUM: freq_ctr: replace the per-second counters with the generic ones
    - MINOR: freq_ctr: add cpu_relax in the rotation loop of update_freq_ctr_period()
    - MINOR: freq_ctr: simplify and improve the update function
    - CLEANUP: time: remove the now unused ms_left_scaled
    - MINOR: time: move the time initialization out of tv_update_date()
    - MINOR: time: remove useless variable copies in tv_update_date()
    - MINOR: time: change the global timeval and the the global tick at once
    - MEDIUM: time: make the clock offset global and no per-thread
    - MINOR: atomic: reimplement the relaxed version of x86 BTS/BTR
    - MINOR: trace: Add the checks as a possible trace source
    - MINOIR: checks/trace: Register a new trace source with its events
    - MINOR: hlua: Add function to release a lua function
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a task
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a converter
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a fetch
    - BUG/MINOR: hlua: Fix memory leaks on error path when parsing a lua action
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering an action
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a service
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a cli keyword
    - BUG/MINOR: cfgparse/proxy: Fix some leaks during proxy section parsing
    - BUG/MINOR: listener: Handle allocation error when allocating a new bind_conf
    - BUG/MINOR: cfgparse/proxy: Hande allocation errors during proxy section parsing
    - MINOR: cfgparse/proxy: Group alloc error handling during proxy section parsing
    - DOC: internals: update the SSL architecture schema
    - BUG/MEDIUM: sample: Fix adjusting size in field converter
    - MINOR: sample: add ub64dec and ub64enc converters
    - CLEANUP: sample: align samples list in sample.c
    - MINOR: ist: Add `istclear(struct ist*)`
    - CI: cirrus: install "pcre" package
    - MINOR: opentracing: correct calculation of the number of arguments in the args[]
    - MINOR: opentracing: transfer of context names without prefix
    - MINOR: sample: converter: Add mjson library.
    - MINOR: sample: converter: Add json_query converter
    - CI: travis-ci: enable weekly graviton2 builds
    - DOC: ssl: Certificate hot update only works on fronted certificates
    - DOC: ssl: Certificate hot update works on server certificates
    - BUG/MEDIUM: threads: Ignore current thread to end its harmless period
    - MINOR: threads: Only consider running threads to end a thread harmeless period
    - BUG/MINOR: checks: Set missing id to the dummy checks frontend
    - MINOR: logs: Add support of checks as session origin to format lf strings
    - BUG/MINOR: connection: Fix fc_http_major and bc_http_major for TCP connections
    - MINOR: connection: Make bc_http_major compatible with tcp-checks
    - BUG/MINOR: ssl-samples: Fix ssl_bc_* samples when called from a health-check
    - BUG/MINOR: http-fetch: Make method smp safe if headers were already forwarded
    - MINOR: tcp_samples: Add samples to get src/dst info of the backend connection
    - MINOR: tcp_samples: Be able to call bc_src/bc_dst from the health-checks
    - BUG/MINOR: http_htx: Remove BUG_ON() from http_get_stline() function
    - BUG/MINOR: logs: Report the true number of retries if there was no connection
    - BUILD: makefile: Redirect stderr to /dev/null when probing options
    - MINOR: uri_normalizer: Add uri_normalizer module
    - MINOR: uri_normalizer: Add `enum uri_normalizer_err`
    - MINOR: uri_normalizer: Add `http-request normalize-uri`
    - MINOR: uri_normalizer: Add a `merge-slashes` normalizer to http-request normalize-uri
    - MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri
    - MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer
    - MINOR: uri_normalizer: Add a `sort-query` normalizer
    - MINOR: uri_normalizer: Add a `percent-upper` normalizer
    - MEDIUM: http_act: Rename uri-normalizers
    - DOC: Add introduction to http-request normalize-uri
    - DOC: Note that URI normalization is experimental
    - BUG/MINOR: pools: maintain consistent ->allocated count on alloc failures
    - BUG/MINOR: pools/buffers: make sure to always reserve the required buffers
    - MINOR: pools: drop the unused static history of artificially failed allocs
    - CLEANUP: pools: remove unused arguments to pool_evict_from_cache()
    - MEDIUM: pools: move the cache into the pool header
    - MINOR: pool: remove the size field from pool_cache_head
    - MINOR: pools: rename CONFIG_HAP_LOCAL_POOLS to CONFIG_HAP_POOLS
    - MINOR: pools: enable the fault injector in all allocation modes
    - MINOR: pools: make the basic pool_refill_alloc()/pool_free() update needed_avg
    - MEDIUM: pools: unify pool_refill_alloc() across all models
    - CLEANUP: pools: re-merge pool_refill_alloc() and __pool_refill_alloc()
    - MINOR: pools: call pool_alloc_nocache() out of the pool's lock
    - CLEANUP: pools: move the lock to the only __pool_get_first() that needs it
    - CLEANUP: pools: rename __pool_get_first() to pool_get_from_shared_cache()
    - CLEANUP: pools: rename pool_*_{from,to}_cache() to *_local_cache()
    - CLEANUP: pools: rename __pool_free() to pool_put_to_shared_cache()
    - MINOR: tools: add statistical_prng_range() to get a random number over a range
    - MINOR: pools: use cheaper randoms for fault injections
    - MINOR: pools: move the fault injector to __pool_alloc()
    - MINOR: pools: split the OS-based allocator in two
    - MINOR: pools: always use atomic ops to maintain counters
    - MINOR: pools: move pool_free_area() out of the lock in the locked version
    - MINOR: pools: factor the release code into pool_put_to_os()
    - MEDIUM: pools: make CONFIG_HAP_POOLS control both local and shared pools
    - MINOR: pools: create unified pool_{get_from,put_to}_cache()
    - MINOR: pools: evict excess objects using pool_evict_from_local_cache()
    - MEDIUM: pools: make pool_put_to_cache() always call pool_put_to_local_cache()
    - CLEANUP: pools: make the local cache allocator fall back to the shared cache
    - CLEANUP: pools: merge pool_{get_from,put_to}_local_caches with generic ones
    - CLEANUP: pools: uninline pool_put_to_cache()
    - CLEANUP: pools: declare dummy pool functions to remove some ifdefs
    - BUILD: pools: fix build with DEBUG_FAIL_ALLOC
    - BUG/MINOR: server: make srv_alloc_lb() allocate lb_nodes for consistent hash
    - CONTRIB: mod_defender: import the minimal number of includes
    - CONTRIB: mod_defender: make the code build with the embedded includes
    - CONTRIB: modsecurity: import the minimal number of includes
    - CONTRIB: modsecurity: make the code build with the embedded includes
    - CLEANUP: sample: Improve local variables in sample_conv_json_query
    - CLEANUP: sample: Explicitly handle all possible enum values from mjson
    - CLEANUP: sample: Use explicit return for successful `json_query`s
    - CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
    - CONTRIB: move spoa_example out of the tree
    - BUG/MINOR: server: free srv.lb_nodes in free_server
    - BUG/MINOR: logs: free logsrv.conf.file on exit
    - BUG/MEDIUM: server: ensure thread-safety of server runtime creation
    - MINOR: server: add log on dynamic server creation
    - MINOR: server: implement delete server cli command
    - CONTRIB: move spoa_server out of the tree
    - CONTRIB: move modsecurity out of the tree
    - BUG/MINOR: server: fix potential null gcc error in delete server
    - BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers
    - BUG/MEDIUM: mux-h2: Fix dfl calculation when merging CONTINUATION frames
    - BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort
    - CLEANUP: uri_normalizer: Remove trailing whitespace
    - MINOR: uri_normalizer: Add a `strip-dot` normalizer
    - CONTRIB: move mod_defender out of the tree
    - CLEANUP: contrib: remove the last references to the now dead contrib/ directory
    - BUG/MEDIUM: config: fix cpu-map notation with both process and threads
    - MINOR: config: add a diag for invalid cpu-map statement
    - BUG/MINOR: mworker/init: don't reset nb_oldpids in non-mworker cases
    - BUG/MINOR: mworker: don't use oldpids[] anymore for reload
    - BUILD: makefile: fix the "make clean" target on strict bourne shells
    - IMPORT: slz: import slz into the tree
    - BUILD: compression: switch SLZ from out-of-tree to in-tree
    - CI: github: do not build libslz any more
    - CLEANUP: compression: remove calls to SLZ init functions
    - BUG/MEDIUM: mux-h2: Properly handle shutdowns when received with data
    - MINOR: cpuset: define a platform-independent cpuset type
    - MINOR: cfgparse: use hap_cpuset for parse_cpu_set
    - MEDIUM: config: use platform independent type hap_cpuset for cpu-map
    - MINOR: thread: implement the detection of forced cpu affinity
    - MINOR: cfgparse: support the comma separator on parse_cpu_set
    - MEDIUM: cfgparse: detect numa and set affinity if needed
    - MINOR: global: add option to disable numa detection
    - BUG/MINOR: haproxy: fix compilation on macOS
    - BUG/MINOR: cpuset: fix compilation on platform without cpu affinity
    - MINOR: time: avoid unneeded updates to now_offset
    - MINOR: time: avoid overwriting the same values of global_now
    - CLEANUP: time: use __tv_to_ms() in tv_update_date() instead of open-coding
    - MINOR: time: avoid u64 needlessly expensive computations for the 32-bit now_ms
    - BUG/MINOR: peers: remove useless table check if initial resync is finished
    - BUG/MEDIUM: peers: re-work connection to new process during reload.
    - BUG/MEDIUM: peers: re-work refcnt on table to protect against flush
    - BUG/MEDIUM: config: fix missing initialization in numa_detect_topology()
2021-04-23 19:11:10 +02:00
Willy Tarreau
2b71810cb3 CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
The current "ADD" vs "ADDQ" is confusing because when thinking in terms
of appending at the end of a list, "ADD" naturally comes to mind, but
here it does the opposite, it inserts. Several times already it's been
incorrectly used where ADDQ was expected, the latest of which was a
fortunate accident explained in 6fa922562 ("CLEANUP: stream: explain
why we queue the stream at the head of the server list").

Let's use more explicit (but slightly longer) names now:

   LIST_ADD        ->       LIST_INSERT
   LIST_ADDQ       ->       LIST_APPEND
   LIST_ADDED      ->       LIST_INLIST
   LIST_DEL        ->       LIST_DELETE

The same is true for MT_LISTs, including their "TRY" variant.
LIST_DEL_INIT keeps its short name to encourage to use it instead of the
lazier LIST_DELETE which is often less safe.

The change is large (~674 non-comment entries) but is mechanical enough
to remain safe. No permutation was performed, so any out-of-tree code
can easily map older names to new ones.

The list doc was updated.
2021-04-21 09:20:17 +02:00
Remi Tricot-Le Breton
59846b6773 DOC: internals: update the SSL architecture schema
This commit adds the new fields added to the ckch_inst structure in
order to manage the backend certificate hot update (GitHub #427) and the
bug of the default certificate update (GitHub #1143).
2021-04-13 11:34:44 +02:00
Willy Tarreau
7be7ffac15 CLEANUP: dynbuf: remove the unused b_alloc_fast() function
It is never used anymore since 1.7 where it was used by b_alloc_margin()
then replaced by direct calls to the pools function, and it maintains a
dependency on the exposed pools functions. It's time to get rid of it,
as it's not even certain it still works.
2021-03-22 16:28:05 +01:00
Willy Tarreau
f44ca97fcb CLEANUP: dynbuf: remove b_alloc_margin()
It's not used anymore, let's completely remove it before anyone uses it
again by accident.
2021-03-22 16:28:02 +01:00
Willy Tarreau
766b6cf206 MINOR: dynbuf: make b_alloc() always check if the buffer is allocated
Right now there is a discrepancy beteween b_alloc() and b_allow_margin():
the former forcefully overwrites the target pointer while the latter tests
it and returns it as-is if already allocated.

As a matter of fact, all callers of b_alloc() either preliminary test the
buffer, or assume it's already null.

Let's remove this pain and make the function test the buffer's allocation
before doing it again, and match call places' expectations.
2021-03-22 16:14:45 +01:00
Ilya Shipitsin
d7a988c14a CLEANUP: assorted typo fixes in the code and comments
This is 19th iteration of typo fixes
2021-03-05 21:22:47 +01:00
Willy Tarreau
8ab65c201a [RELEASE] Released version 2.4-dev10
Released version 2.4-dev10 with the following main changes :
    - BUILD: SSL: introduce fine guard for RAND_keep_random_devices_open
    - MINOR: Configure the `cpp` userdiff driver for *.[ch] in .gitattributes
    - BUG/MINOR: ssl/cli: potential null pointer dereference in "set ssl cert"
    - BUG/MINOR: sample: secure convs that accept base64 string and var name as args
    - BUG/MEDIUM: vars: make functions vars_get_by_{name,desc} thread-safe
    - CLEANUP: vars: make smp_fetch_var() to reuse vars_get_by_desc()
    - DOC: muxes: add a diagram of the exchanges between muxes and outer world
    - BUG/MEDIUM: proxy: use thread-safe stream killing on hard-stop
    - BUG/MEDIUM: cli/shutdown sessions: make it thread-safe
    - BUG/MINOR: proxy: wake up all threads when sending the hard-stop signal
    - MINOR: stream: add an "epoch" to figure which streams appeared when
    - MINOR: cli/streams: make "show sess" dump all streams till the new epoch
    - MINOR: streams: use one list per stream instead of a global one
    - MEDIUM: streams: do not use the streams lock anymore
    - BUILD: dns: avoid a build warning when threads are disabled (dss unused)
    - MEDIUM: task: remove the tasks_run_queue counter and have one per thread
    - MINOR: tasks: do not maintain the rqueue_size counter anymore
    - CLEANUP: tasks: use a less confusing name for task_list_size
    - CLEANUP: task: move the tree root detection from __task_wakeup() to task_wakeup()
    - MINOR: task: limit the remote thread wakeup to the global runqueue only
    - MINOR: task: move the allocated tasks counter to the per-thread struct
    - CLEANUP: task: split the large tasklet_wakeup_on() function in two
    - BUG/MINOR: fd: properly wait for !running_mask in fd_set_running_excl()
    - BUG/MINOR: resolvers: Fix condition to release received ARs if not assigned
    - BUG/MINOR: resolvers: Only renew TTL for SRV records with an additional record
    - BUG/MINOR: resolvers: new callback to properly handle SRV record errors
    - BUG/MEDIUM: resolvers: Reset server address and port for obselete SRV records
    - BUG/MEDIUM: resolvers: Reset address for unresolved servers
    - DOC: Update the module list in MAINTAINERS file
    - MINOR: htx: Add function to reserve the max possible size for an HTX DATA block
    - DOC: Update the HTX API documentation
    - DOC: Update the filters guide
    - BUG/MEDIUM: contrib/prometheus-exporter: fix segfault in listener name dump
    - MINOR: task: split the counts of local and global tasks picked
    - MINOR: task: do not use __task_unlink_rq() from process_runnable_tasks()
    - MINOR: task: don't decrement then increment the local run queue
    - CLEANUP: task: re-merge __task_unlink_rq() with task_unlink_rq()
    - MINOR: task: make grq_total atomic to move it outside of the grq_lock
    - MINOR: tasks: also compute the tasklet latency when DEBUG_TASK is set
    - MINOR: task: make tasklet wakeup latency measurements more accurate
    - MINOR: server: Be more strict on the server-state line parsing
    - MINOR: server: Only fill one array when parsing a server-state line
    - MEDIUM: server: Refactor apply_server_state() to make it more readable
    - CLEANUP: server: Rename state_line node to node instead of name_name
    - CLEANUP: server: Rename state_line structure into server_state_line
    - CLEANUP: server: Use a local eb-tree to store lines of the global server-state file
    - MINOR: server: Be more strict when reading the version of a server-state file
    - MEDIUM: server: Store parsed params of a server-state line in the tree
    - MINOR: server: Remove cached line from global server-state tree when found
    - MINOR: server: Move loading state of servers in a dedicated function
    - MEDIUM: server: Use a tree to store local server-state lines
    - MINOR: server: Parse and store server-state lines in a dedicated function
    - MEDIUM: server: Don't load server-state file if a line is corrupted
    - REORG: server: Export and rename some functions updating server info
    - REORG: server-state: Move functions to deal with server-state in its own file
    - MINOR: server-state: Don't load server-state file for serverless proxies
    - CLEANUP: muxes: Remove useless if condition in show_fd function
    - BUG/MINOR: stats: fix compare of no-maint url suffix
    - MINOR: task: limit the number of subsequent heavy tasks with flag TASK_HEAVY
    - MINOR: ssl: mark the SSL handshake tasklet as heavy
    - CLEANUP: server: rename srv_cleanup_{idle,toremove}_connections()
    - BUG/MINOR: ssl: potential null pointer dereference in ckchs_dup()
    - MINOR: task: add one extra tasklet class: TL_HEAVY
    - MINOR: task: place the heavy elements in TL_HEAVY
    - MINOR: task: only limit TL_HEAVY tasks but not others
    - BUG/MINOR: http-ana: Only consider dst address to process originalto option
    - MINOR: tools: Add net_addr structure describing a network addess
    - MINOR: tools: Add function to compare an address to a network address
    - MEDIUM: http-ana: Add IPv6 support for forwardfor and orignialto options
    - CLEANUP: hlua: Use net_addr structure internally to parse and compare addresses
    - REGTESTS: Add script to test except param for fowardedfor/originalto options
    - DOC: scheduler: add a diagram showing the different queues and their usages
    - CLEANUP: tree-wide: replace free(x);x=NULL with ha_free(&x)
    - CLEANUP: config: replace a few free() with ha_free()
    - CLEANUP: vars: always zero the pointers after a free()
    - CLEANUP: ssl: remove a useless "if" before freeing an error message
    - CLEANUP: ssl: make ssl_sock_free_srv_ctx() zero the pointers after free
    - CLEANUP: ssl: use realloc() instead of free()+malloc()
2021-02-26 22:49:10 +01:00
Willy Tarreau
ee17b97eea DOC: scheduler: add a diagram showing the different queues and their usages
The scheduler has become complex over time and the latest updates were a
good opportunity to document it. This diagram shows the time-based wait
queue(s), the priority-based run queue(s), and the class-based tasklet
queues, trying to emphasize what is local-only and what is shared between
threads. The diagram is provided in .fig, .svg, .png, and .pdf.
2021-02-26 17:49:37 +01:00
Christopher Faulet
74d7b6e992 DOC: Update the filters guide
The filters guide was totally outdated. Callbacks to filter payload were
changed, especially the HTTP one because of the HTX. All the HTTP legacy
part is removed. This new guide now reflects the reality.

This patch may be backported as far as 2.2.
2021-02-24 22:10:01 +01:00
Christopher Faulet
9a2cec4953 DOC: Update the HTX API documentation
Missing functions have been added. And because the EOM block was removed,
some parts have been adapted to better explain how the end of the message
may be detected.
2021-02-24 22:10:01 +01:00
Willy Tarreau
61d095ed37 DOC: muxes: add a diagram of the exchanges between muxes and outer world
Since the muxes API is far from being obvious, let's show a stream being
forwarded between two sides through muxes with their buffers and the
transport layers. The diagram is provided in .fig, .svg, .png, and .pdf.
2021-02-24 09:13:00 +01:00
Christopher Faulet
d1ac2b90cd MAJOR: htx: Remove the EOM block type and use HTX_FL_EOM instead
The EOM block may be removed. The HTX_FL_EOM flags is enough. Most of time,
to know if the end of the message is reached, we just need to have an empty
HTX message with HTX_FL_EOM flag set. It may also be detected when the last
block of a message with HTX_FL_EOM flag is manipulated.

Removing EOM blocks simplifies the HTX message filling. Indeed, there is no
more edge problems when the message ends but there is no more space to write
the EOM block. However, some part are more tricky. Especially the
compression filter or the FCGI mux. The compression filter must finish the
compression on the last DATA block. Before it was performed on the EOM
block, an extra DATA block with the checksum was added. Now, we must detect
the last DATA block to be sure to finish the compression. The FCGI mux on
its part must be sure to reserve the space for the empty STDIN record on the
last DATA block while this record was inserted on the EOM block.

The H2 multiplexer is probably the part that benefits the most from this
change. Indeed, it is now fairly easier to known when to set the ES flag.

The HTX documentaion has been updated accordingly.
2021-01-28 16:37:14 +01:00
Thayne McCombs
cdbcca9995 DOC: fix some spelling issues over multiple files
This is from the output of codespell and may be backported.
2021-01-08 14:53:47 +01:00
Willy Tarreau
9d58c9b251 [RELEASE] Released version 2.3-dev7
Released version 2.3-dev7 with the following main changes :
    - CI: travis-ci: replace not defined SSL_LIB, SSL_INC for BotringSSL builds
    - BUG/MINOR: init: only keep rlim_fd_cur if max is unlimited
    - BUG/MINOR: mux-h2: do not stop outgoing connections on stopping
    - MINOR: fd: report an error message when failing initial allocations
    - MINOR: proto-tcp: make use of connect(AF_UNSPEC) for the pause
    - MINOR: sock: add sock_accept_conn() to test a listening socket
    - MINOR: protocol: make proto_tcp & proto_uxst report listening sockets
    - MINOR: sockpair: implement the .rx_listening function
    - CLEANUP: tcp: make use of sock_accept_conn() where relevant
    - CLEANUP: unix: make use of sock_accept_conn() where relevant
    - BUG/MINOR: listener: detect and handle shared sockets stopped in other processes
    - CONTRIB: tcploop: implement a disconnect operation 'D'
    - CLEANUP: protocol: intitialize all of the sockaddr when disconnecting
    - BUG/MEDIUM: deinit: check fdtab before fdtab[fd].owner
    - BUG/MINOR: connection: fix loop iter on connection takeover
    - BUG/MEDIUM: connection: fix srv idle count on conn takeover
    - MINOR: connection: improve list api usage
    - MINOR: mux/connection: add a new mux flag for HOL risk
    - MINOR: connection: don't check priv flag on free
    - MEDIUM: backend: add new conn to session if mux marked as HOL blocking
    - MEDIUM: backend: add reused conn to sess if mux marked as HOL blocking
    - MEDIUM: h2: remove conn from session on detach
    - MEDIUM: fcgi: remove conn from session on detach
    - DOC: Describe reuse safe for HOL handling
    - MEDIUM: proxy: remove obsolete "mode health"
    - MEDIUM: proxy: remove obsolete "monitor-net"
    - CLEANUP: protocol: remove the ->drain() function
    - CLEANUP: fd: finally get rid of fd_done_recv()
    - MINOR: connection: make sockaddr_alloc() take the address to be copied
    - MEDIUM: listener: allocate the connection before queuing a new connection
    - MINOR: session: simplify error path in session_accept_fd()
    - MINOR: connection: add new error codes for accept_conn()
    - MINOR: sock: rename sock_accept_conn() to sock_accepting_conn()
    - MINOR: protocol: add a new function accept_conn()
    - MINOR: sock: implement sock_accept_conn() to accept a connection
    - MINOR: sockpair: implement sockpair_accept_conn() to accept a connection
    - MEDIUM: listener: use protocol->accept_conn() to accept a connection
    - MEDIUM: listener: remove the second pass of fd manipulation at the end
    - MINOR: protocol: add a default I/O callback and put it into the receiver
    - MINOR: log: set the UDP receiver's I/O handler in the receiver
    - MINOR: protocol: register the receiver's I/O handler and not the protocol's
    - CLEANUP: protocol: remove the now unused <handler> field of proto_fam->bind()
    - DOC: improve the documentation for "option nolinger"
    - BUG/MEDIUM: proxy: properly stop backends
    - BUG/MEDIUM: task: bound the number of tasks picked from the wait queue at once
    - MINOR: threads: augment rwlock debugging stats to report seek lock stats
    - MINOR: threads: add the transitions to/from the seek state
    - MEDIUM: task: use an upgradable seek lock when scanning the wait queue
    - BUILD: listener: avoir a build warning when threads are disabled
    - BUG/MINOR: peers: Possible unexpected peer seesion reset after collisions.
    - MINOR: ssl: add volatile flags to ssl samples
    - MEDIUM: backend: reuse connection if using a static sni
    - BUG/MEDIUM: spoe: Unset variable instead of set it if no data provided
    - BUG/MEDIUM: mux-h1: Get the session from the H1S when capturing bad messages
    - BUG/MEDIUM: lb: Always lock the server when calling server_{take,drop}_conn
    - DOC: fix typo in MAX_SESS_STKCTR
2020-10-17 10:31:50 +02:00
Willy Tarreau
0138f51f93 CLEANUP: fd: finally get rid of fd_done_recv()
fd_done_recv() used to be useful with the FD cache because it used to
allow to keep a file descriptor active in the poller without being
marked as ready in the cache, saving it from ringing immediately,
without incurring any system call. It was a way to make it yield
to wait for new events leaving a bit of time for others. The only
user left was the connection accepter (listen_accept()). We used
to suspect that with the FD cache removal it had become totally
useless since changing its readiness or not wouldn't change its
status regarding the poller itself, which would be the only one
deciding to report it again.

Careful tests showed that it indeed has exactly zero effect nowadays,
the syscall numbers are exactly the same with and without, including
when enabling edge-triggered polling.

Given that there's no more API available to manipulate it and that it
was directly called as an optimization from listener_accept(), it's
about time to remove it.
2020-10-15 21:47:56 +02:00
Willy Tarreau
b7ffe1975a [RELEASE] Released version 2.3-dev6
Released version 2.3-dev6 with the following main changes :
    - REGTESTS: use "command" instead of "which" for better POSIX compatibility
    - BUILD: makefile: Update feature flags for OpenBSD
    - DOC: agent-check: fix typo in "fail" word expected reply
    - DOC: crt: advise to move away from cert bundle
    - BUG/MINOR: ssl/crt-list: exit on warning out of crtlist_parse_line()
    - REGTEST: fix host part in balance-uri-path-only.vtc
    - REGTEST: make ssl_client_samples and ssl_server_samples requiret to 2.3
    - REGTEST: the iif converter test requires 2.3
    - REGTEST: make agent-check.vtc require 1.8
    - REGTEST: make abns_socket.vtc require 1.8
    - REGTEST: make map_regm_with_backref require 1.7
    - BUILD: makefile: Update feature flags for FreeBSD
    - OPTIM: backend/random: never queue on the server, always on the backend
    - OPTIM: backend: skip LB when we know the backend is full
    - BUILD: makefile: Fix building with closefrom() support enabled
    - BUILD: makefile: add an EXTRAVERSION variable to ease local naming
    - MINOR: tools: support for word expansion of environment in parse_line
    - BUILD: tools: fix minor build issue on isspace()
    - BUILD: makefile: Enable closefrom() support on Solaris
    - CLEANUP: ssl: Use structured format for error line report during crt-list parsing
    - MINOR: ssl: Add error if a crt-list might be truncated
    - MINOR: ssl: remove uneeded check in crtlist_parse_file
    - BUG/MINOR: Fix several leaks of 'log_tag' in init().
    - DOC: tcp-rules: Refresh details about L7 matching for tcp-request content rules
    - MEDIUM: tcp-rules: Warn if a track-sc* content rule doesn't depend on content
    - BUG/MINOR: tcpcheck: Set socks4 and send-proxy flags before the connect call
    - DOC: ssl: new "cert bundle" behavior
    - BUG/MEDIUM: queue: make pendconn_cond_unlink() really thread-safe
    - CLEANUP: ssl: "bundle" is not an OpenSSL wording
    - MINOR: counters: fix a typo in comment
    - BUG/MINOR: stats: fix validity of the json schema
    - REORG: stats: export some functions
    - MINOR: stats: add stats size as a parameter for csv/json dump
    - MINOR: stats: hide px/sv/li fields in applet struct
    - REORG: stats: extract proxy json dump
    - REORG: stats: extract proxies dump loop in a function
    - MINOR: hlua: Display debug messages on stderr only in debug mode
    - MINOR: stats: define the concept of domain for statistics
    - MINOR: stats: define additional flag px cap on domain
    - MEDIUM: stats: add delimiter for static proxy stats on csv
    - MEDIUM: stats: define an API to register stat modules
    - MEDIUM: stats: add abstract type to store counters
    - MEDIUM: stats: integrate static proxies stats in new stats
    - MINOR: stats: support clear counters for dynamic stats
    - MINOR: stats: display extra proxy stats on the html page
    - MINOR: stats: add config "stats show modules"
    - MINOR: dns/stats: integrate dns counters in stats
    - MINOR: stats: remove for loop declaration
    - DOC: ssl: fix typo about ocsp files
    - BUG/MINOR: peers: Inconsistency when dumping peer status codes.
    - DOC: update INSTALL with supported OpenBSD / FreeBSD versions
    - BUG/MINOR: proto_tcp: Report warning messages when listeners are bound
    - CLEANUP: cache: Fix leak of cconf->c.name during config check
    - CLEANUP: ssl: Release cached SSL sessions on deinit
    - BUG/MINOR: mux-h1: Be sure to only set CO_RFL_READ_ONCE for the first read
    - BUG/MINOR: mux-h1: Always set the session on frontend h1 stream
    - MINOR: mux-h1: Don't wakeup the H1C when output buffer become available
    - CLEANUP: sock-unix: Remove an unreachable goto clause
    - BUG/MINOR: proxy: inc req counter on new syslog messages.
    - BUG/MEDIUM: log: old processes with log foward section don't die on soft stop.
    - MINOR: stats: inc req counter on listeners.
    - MINOR: channel: new getword and getchar functions on channel.
    - MEDIUM: log: syslog TCP support on log forward section.
    - BUG/MINOR: proxy/log: frontend/backend and log forward names must differ
    - DOC: re-work log forward bind statement documentation.
    - DOC: fix a confusing typo on a regsub example
    - BUILD: Add a DragonFlyBSD target
    - BUG/MINOR: makefile: fix a tiny typo in the target list
    - BUILD: makefile: Update feature flags for NetBSD
    - CI: travis-ci: help Coverity to detect BUG_ON() as a real stop
    - DOC: Add missing stats fields in the management doc
    - BUG/MEDIUM: mux-fcgi: Don't handle pending read0 too early on streams
    - BUG/MEDIUM: mux-h2: Don't handle pending read0 too early on streams
    - DOC: Fix typos in configuration.txt
    - BUG/MINOR: http: Fix content-length of the default 500 error
    - BUG/MINOR: http-htx: Expect no body for 204/304 internal HTTP responses
    - REGTESTS: mark abns_socket as broken
    - MEDIUM: fd: always wake up one thread when enabling a foreing FD
    - MEDIUM: listeners: don't bounce listeners management between queues
    - MEDIUM: init: stop disabled proxies after initializing fdtab
    - MEDIUM: listeners: make unbind_listener() converge if needed
    - MEDIUM: deinit: close all receivers/listeners before scanning proxies
    - MEDIUM: listeners: remove the now unused ZOMBIE state
    - MINOR: listeners: do not uselessly try to close zombie listeners in soft_stop()
    - CLEANUP: proxy: remove the first_to_listen hack in zombify_proxy()
    - MINOR: listeners: introduce listener_set_state()
    - MINOR: proxy: maintain per-state counters of listeners
    - MEDIUM: proxy: remove the unused PR_STFULL state
    - MEDIUM: proxy: remove the PR_STERROR state
    - MEDIUM: proxy: remove state PR_STPAUSED
    - MINOR: startup: don't rely on PR_STNEW to check for listeners
    - CLEANUP: peers: don't use the PR_ST* states to mark enabled/disabled
    - MEDIUM: proxy: replace proxy->state with proxy->disabled
    - MEDIUM: proxy: remove start_proxies()
    - MEDIUM: proxy: merge zombify_proxy() with stop_proxy()
    - MINOR: listeners: check the current listener state in pause_listener()
    - MINOR: listeners: check the current listener earlier state in resume_listener()
    - MEDIUM: listener/proxy: make the listeners notify about proxy pause/resume
    - MINOR: protocol: introduce protocol_{pause,resume}_all()
    - MAJOR: signals: use protocol_pause_all() and protocol_resume_all()
    - CLEANUP: proxy: remove the now unused pause_proxies() and resume_proxies()
    - MEDIUM: proto_tcp: make the pause() more robust in multi-process
    - BUG/MEDIUM: listeners: correctly report pause() errors
    - MINOR: listeners: move fd_stop_recv() to the receiver's socket code
    - CLEANUP: protocol: remove the ->disable_all method
    - CLEANUP: listeners: remove unused disable_listener and disable_all_listeners
    - MINOR: listeners: export enable_listener()
    - MINOR: protocol: directly call enable_listener() from protocol_enable_all()
    - CLEANUP: protocol: remove the ->enable_all method
    - CLEANUP: listeners: remove the now unused enable_all_listeners()
    - MINOR: protocol: rename the ->listeners field to ->receivers
    - MINOR: protocol: replace ->pause(listener) with ->rx_suspend(receiver)
    - MINOR: protocol: implement an ->rx_resume() method
    - MINOR: listener: use the protocol's ->rx_resume() method when available
    - MINOR: sock: provide a set of generic enable/disable functions
    - MINOR: protocol: add a new pair of rx_enable/rx_disable methods
    - MINOR: protocol: add a new pair of enable/disable methods for listeners
    - MEDIUM: listeners: now use the listener's ->enable/disable
    - MINOR: listeners: split delete_listener() in two versions
    - MINOR: listeners: count unstoppable jobs on creation, not deletion
    - MINOR: listeners: add a new stop_listener() function
    - MEDIUM: proxy: make stop_proxy() now use stop_listener()
    - MEDIUM: proxy: add mode PR_MODE_PEERS to flag peers frontends
    - MEDIUM: proxy: centralize proxy status update and reporting
    - MINOR: protocol: add protocol_stop_now() to instant-stop listeners
    - MEDIUM: proxy: make soft_stop() stop most listeners using protocol_stop_now()
    - MEDIUM: udp: implement udp_suspend() and udp_resume()
    - MINOR: listener: add a few BUG_ON() statements to detect inconsistencies
    - MEDIUM: listeners: always close master vs worker listeners
    - BROKEN/MEDIUM: listeners: rework the unbind logic to make it idempotent
    - MEDIUM: listener: let do_unbind_listener() decide whether to close or not
    - CLEANUP: listeners: remove the do_close argument to unbind_listener()
    - MINOR: listeners: move the LI_O_MWORKER flag to the receiver
    - MEDIUM: receivers: add an rx_unbind() method in the protocols
    - MINOR: listeners: split do_unbind_listener() in two
    - MEDIUM: listeners: implement protocol level ->suspend/resume() calls
    - MEDIUM: config: mark "grace" as deprecated
    - MEDIUM: config: remove the deprecated and dangerous global "debug" directive
    - BUG/MINOR: proxy: respect the proper format string in sig_pause/sig_listen
    - MINOR: peers: heartbeat, collisions and handshake information for "show peers" command.
    - BUILD: makefile: Enable getaddrinfo() on OS/X
2020-10-10 10:45:13 +02:00
Willy Tarreau
c6dac6c7f5 MEDIUM: listeners: remove the now unused ZOMBIE state
The zombie state is not used anymore by the listeners, because in the
last two cases where it was tested it couldn't match as it was covered
by the test on the process mask. Instead now the FD is either in the
LISTEN state or the INIT state. This also avoids forcing the listener
to be single-dimensional because actually belonging to another process
isn't totally exclusive with the other states, which explains some of
the difficulties requiring to check the proc_mask and the fd sometimes.

So let's get rid of it now not to be tempted to reuse it.

The doc on the listeners state was updated.
2020-10-09 11:27:29 +02:00
Ilya Shipitsin
11057a3590 DOC: assorted typo fixes in the documentation
this is 10th iteration of typo fixes
2020-06-26 11:27:10 +02:00
William Lallemand
d7f8bd9b40 DOC: internals: update the SSL architecture schema
This commit updates the SSL files architecture schema and adds the
crtlist structures in it.
2020-04-23 16:30:12 +02:00
Willy Tarreau
d008930265 [RELEASE] Released version 2.2-dev6
Released version 2.2-dev6 with the following main changes :
    - BUG/MINOR: ssl: memory leak when find_chain is NULL
    - CLEANUP: ssl: rename ssl_get_issuer_chain to ssl_get0_issuer_chain
    - MINOR: ssl: rework add cert chain to CTX to be libssl independent
    - BUG/MINOR: peers: init bind_proc to 1 if it wasn't initialized
    - BUG/MINOR: peers: avoid an infinite loop with peers_fe is NULL
    - BUG/MINOR: peers: Use after free of "peers" section.
    - CI: github actions: add weekly h2spec test
    - BUG/MEDIUM: mux_h1: Process a new request if we already received it.
    - MINOR: build: Fix build in mux_h1
    - CLEANUP: remove obsolete comments
    - BUG/MEDIUM: dns: improper parsing of aditional records
    - MINOR: ssl: skip self issued CA in cert chain for ssl_ctx
    - MINOR: listener: add so_name sample fetch
    - MEDIUM: stream: support use-server rules with dynamic names
    - MINOR: servers: Add a counter for the number of currently used connections.
    - MEDIUM: connections: Revamp the way idle connections are killed
    - MINOR: cli: add a general purpose pointer in the CLI struct
    - MINOR: ssl: add a list of bind_conf in struct crtlist
    - REORG: ssl: move SETCERT enum to ssl_sock.h
    - BUG/MINOR: ssl: ckch_inst wrongly inserted in crtlist_entry
    - REORG: ssl: move some functions above crtlist_load_cert_dir()
    - MINOR: ssl: use crtlist_free() upon error in directory loading
    - MINOR: ssl: add a list of crtlist_entry in ckch_store
    - MINOR: ssl: store a ptr to crtlist in crtlist_entry
    - MINOR: ssl/cli: update pointer to store in 'commit ssl cert'
    - MEDIUM: ssl/cli: 'add ssl crt-list' command
    - REGTEST: ssl/cli: test the 'add ssl crt-list' command
    - BUG/MINOR: ssl: entry->ckch_inst not initialized
    - REGTEST: ssl/cli: change test type to devel
    - REGTEST: make the PROXY TLV validation depend on version 2.2
    - CLEANUP: assorted typo fixes in the code and comments
    - BUG/MINOR: stats: Fix color of draining servers on stats page
    - DOC: internals: Fix spelling errors in filters.txt
    - MINOR: connections: Don't mark conn flags 0x00000001 and 0x00000002 as unused.
    - REGTEST: make the unique-id test depend on version 2.0
    - BUG/MEDIUM: dns: Consider the fact that dns answers are case-insensitive
    - MINOR: ssl: split the line parsing of the crt-list
    - MINOR: ssl/cli: support filters and options in add ssl crt-list
    - MINOR: ssl: add a comment above the ssl_bind_conf keywords
    - REGTEST: ssl/cli: tests options and filters w/ add ssl crt-list
    - REGTEST: ssl: pollute the crt-list file
    - BUG/CRITICAL: hpack: never index a header into the headroom after wrapping
    - BUG/MINOR: protocol_buffer: Wrong maximum shifting.
    - CLEANUP: src/fd.c: mask setsockopt with DISGUISE
    - BUG/MINOR: ssl/cli: initialize fcount int crtlist_entry
    - REGTEST: ssl/cli: add other cases of 'add ssl crt-list'
    - CLEANUP: assorted typo fixes in the code and comments
    - DOC: management: add the new crt-list CLI commands
    - BUG/MINOR: ssl/cli: fix spaces in 'show ssl crt-list'
    - MINOR: ssl/cli: 'del ssl crt-list' delete an entry
    - MINOR: ssl/cli: replace dump/show ssl crt-list by '-n' option
    - CI: use better SSL library definition
    - CI: travis-ci: enable DEBUG_STRICT=1 for CI builds
    - CI: travis-ci: upgrade openssl to 1.1.1f
    - MINOR: ssl: improve the errors when a crt can't be open
    - CI: cirrus-ci: rename openssl package after it is renamed in FreeBSD
    - CI: adopt openssl download script to download all versions
    - BUG/MINOR: ssl/cli: lock the ckch structures during crt-list delete
    - MINOR: ssl/cli: improve error for bundle in add/del ssl crt-list
    - MINOR: ssl/cli: 'del ssl cert' deletes a certificate
    - BUG/MINOR: ssl: trailing slashes in directory names wrongly cached
    - BUG/MINOR: ssl/cli: memory leak in 'set ssl cert'
    - CLEANUP: ssl: use the refcount for the SSL_CTX'
    - CLEANUP: ssl/cli: use the list of filters in the crtlist_entry
    - BUG/MINOR: ssl: memleak of the struct cert_key_and_chain
    - CLEANUP: ssl: remove a commentary in struct ckch_inst
    - MINOR: ssl: initialize all list in ckch_inst_new()
    - MINOR: ssl: free instances and SNIs with ckch_inst_free()
    - MINOR: ssl: replace ckchs_free() by ckch_store_free()
    - BUG/MEDIUM: ssl/cli: trying to access to free'd memory
    - MINOR: ssl: ckch_store_new() alloc and init a ckch_store
    - MINOR: ssl: crtlist_new() alloc and initialize a struct crtlist
    - REORG: ssl: move some free/new functions
    - MINOR: ssl: crtlist_entry_{new, free}
    - BUG/MINOR: ssl: ssl_conf always set to NULL on crt-list parsing
    - MINOR: ssl: don't alloc ssl_conf if no option found
    - BUG/MINOR: connection: always send address-less LOCAL PROXY connections
    - BUG/MINOR: peers: Incomplete peers sections should be validated.
    - MINOR: init: report in "haproxy -c" whether there were warnings or not
    - MINOR: init: add -dW and "zero-warning" to reject configs with warnings
    - MINOR: init: report the compiler version in haproxy -vv
    - CLEANUP: assorted typo fixes in the code and comments
    - MINOR: init: report the haproxy version and executable path once on errors
    - DOC: Make how "option redispatch" works more explicit
    - BUILD: Makefile: add linux-musl to TARGET
    - CLEANUP: assorted typo fixes in the code and comments
    - CLEANUP: http: Fixed small typo in parse_http_return
    - DOC: hashing: update link to hashing functions
2020-04-17 14:19:38 +02:00
Adam Mills
bbf697e752 DOC: hashing: update link to hashing functions
Bret Mulvey, the author of the article cited in this pulication
has migrated his work to papa.bretmulvey.com. I was able to
view an archival version of Bret M.'s original post
(http://home.comcast.net/~bretm/hash/3.html) and have validated
that this is the same paper that is originally cited.
2020-04-17 13:59:46 +02:00
Miroslav Zagorac
d80f5c0d0c DOC: internals: Fix spelling errors in filters.txt 2020-03-31 17:24:07 +02:00
Ilya Shipitsin
1fae8db7b7 DOC: assorted typo fixes in the documentation
This is the fourth round of cleanups in various docs
2020-03-18 11:34:33 +01:00
Willy Tarreau
5a753bd7b7 [RELEASE] Released version 2.2-dev4
Released version 2.2-dev4 with the following main changes :
    - MEDIUM: buffer: remove the buffer_wq lock
    - MINOR: ssl: move find certificate chain code to its own function
    - MINOR: ssl: resolve issuers chain later
    - MINOR: ssl: resolve ocsp_issuer later
    - MINOR: ssl/cli: "show ssl cert" command should print the "Chain Filename:"
    - BUG/MINOR: h2: reject again empty :path pseudo-headers
    - MINOR: wdt: always clear sigev_value to make valgrind happy
    - MINOR: epoll: always initialize all of epoll_event to please valgrind
    - BUG/MINOR: sample: Make sure to return stable IDs in the unique-id fetch
    - BUG/MEDIUM: ssl: chain must be initialized with sk_X509_new_null()
    - BUILD: cirrus-ci: suppress OS version check when installing packages
    - BUG/MINOR: http_ana: make sure redirect flags don't have overlapping bits
    - CLEANUP: fd: remove the FD_EV_STATUS aggregate
    - CLEANUP: fd: remove some unneeded definitions of FD_EV_* flags
    - MINOR: fd: merge the read and write error bits into RW error
    - BUG/MINOR: dns: ignore trailing dot
    - MINOR: contrib/prometheus-exporter: Add the last heathcheck duration metric
    - BUG/MINOR: http-htx: Do case-insensive comparisons on Host header name
    - MINOR: mux-h1: Remove useless case-insensitive comparisons
    - MINOR: rawsock: always mark the FD not ready when we're certain it happens
    - MEDIUM: connection: make the subscribe() call able to wakeup if ready
    - MEDIUM: connection: don't stop receiving events in the FD handler
    - MEDIUM: mux-h1: do not blindly wake up the tasklet at end of request anymore
    - BUG/MINOR: arg: don't reject missing optional args
    - MINOR: tools: make sure to correctly check the returned 'ms' in date2std_log
    - MINOR: debug: report the task handler's pointer relative to main
    - BUG/MEDIUM: debug: make the debug_handler check for the thread in threads_to_dump
    - MINOR: haproxy: export main to ease access from debugger
    - MINOR: haproxy: export run_poll_loop
    - MINOR: task: export run_tasks_from_list
    - BUILD: tools: remove obsolete and conflicting trace() from standard.c
    - MINOR: tools: add new function dump_addr_and_bytes()
    - MINOR: tools: add resolve_sym_name() to resolve function pointers
    - MINOR: debug: use resolve_sym_name() to dump task handlers
    - MINOR: cli: make "show fd" rely on resolve_sym_name()
    - MEDIUM: debug: add support for dumping backtraces of stuck threads
    - MINOR: debug: call backtrace() once upon startup
    - MINOR: ssl: add "ca-verify-file" directive
    - BUG/MINOR: wdt: do not return an error when the watchdog couldn't be enabled
    - BUILD: Makefile: include librt before libpthread
    - MEDIUM: wdt: fall back to CLOCK_REALTIME if CLOCK_THREAD_CPUTIME is not available
    - MINOR: wdt: do not depend on USE_THREAD
    - MINOR: debug: report the number of entries in the backtrace
    - MINOR: debug: improve backtrace() on aarch64 and possibly other systems
    - MINOR: debug: use our own backtrace function on clang+x86_64
    - MINOR: debug: dump the whole trace if we can't spot the starting point
    - BUILD: tools: unbreak resolve_sym_name() on non-GNU platforms
    - BUILD: tools: rely on __ELF__ not USE_DL to enable use of dladdr()
    - CLEANUP: contrib/spoa_example: Fix several typos
    - BUILD: makefile: do not modify the build options during make reg-tests
    - BUG/MEDIUM: connection: stop polling for sending when the event is ready
    - MEDIUM: stream-int: make sure to try to immediately validate the connection
    - MINOR: tcp/uxst/sockpair: only ask for I/O when really waiting for a connect()
    - MEDIUM: connection: only call ->wake() for connect() without I/O
    - OPTIM: connection: disable receiving on disabled events when the run queue is too high
    - OPTIM: mux-h1: subscribe rather than waking up at a few other places
    - REGTEST: Add unique-id reg-test
    - MINOR: stream: Add stream_generate_unique_id function
    - MINOR: stream: Use stream_generate_unique_id
    - BUG/MINOR: connection/debug: do not enforce !event_type on subscribe() anymore
    - MINOR: ssl/cli: support crt-list filters
    - MINOR: ssl: reach a ckch_store from a sni_ctx
    - DOC: fix incorrect indentation of http_auth_*
    - BUG/MINOR: ssl-sock: do not return an uninitialized pointer in ckch_inst_sni_ctx_to_sni_filters
    - MINOR: debug: add CLI command "debug dev write" to write an arbitrary size
    - MINOR: ist: Add `IST_NULL` macro
    - MINOR: ist: Add `int isttest(const struct ist)`
    - MINOR: ist: Add `struct ist istalloc(size_t)` and `void istfree(struct ist*)`
    - CLEANUP: Use `isttest()` and `istfree()`
    - MINOR: ist: Add `struct ist istdup(const struct ist)`
    - MINOR: proxy: Make `header_unique_id` a `struct ist`
    - MEDIUM: stream: Make the `unique_id` member of `struct stream` a `struct ist`
    - OPTIM: startup: fast unique_id allocation for acl.
    - DOC: configuration.txt: fix various typos
    - DOC: assorted typo fixes in the documentation and Makefile
    - BUG/MINOR: init: make the automatic maxconn consider the max of soft/hard limits
    - BUG/MAJOR: proxy_protocol: Properly validate TLV lengths
    - CLEANUP: proxy_protocol: Use `size_t` when parsing TLVs
    - MINOR: buf: Add function to insert a string at an absolute offset in a buffer
    - MINOR: htx: Add a function to return a block at a specific offset
    - MINOR: htx: Use htx_find_offset() to truncate an HTX message
    - MINOR: flt_trace: Use htx_find_offset() to get the available payload length
    - BUG/MINOR: filters: Use filter offset to decude the amount of forwarded data
    - BUG/MINOR: filters: Forward everything if no data filters are called
    - BUG/MEDIUM: cache/filters: Fix loop on HTX blocks caching the response payload
    - BUG/MEDIUM: compression/filters: Fix loop on HTX blocks compressing the payload
    - BUG/MINOR: http-ana: Reset request analysers on a response side error
    - BUG/MINOR: lua: Abort when txn:done() is called from a Lua action
    - BUG/MINOR: lua: Ignore the reserve to know if a channel is full or not
    - MINOR: lua: Add function to know if a channel is a response one
    - MINOR: lua: Stop using the lua txn in hlua_http_get_headers()
    - MINOR: lua: Stop using the lua txn in hlua_http_rep_hdr()
    - MINOR: lua: Stop using lua txn in hlua_http_del_hdr() and hlua_http_add_hdr()
    - MINOR: lua: Remove the flag HLUA_TXN_HTTP_RDY
    - MINOR: lua: Rename hlua_action_wake_time() to hlua_set_wake_time()
    - BUG/MINOR: lua: Init the lua wake_time value before calling a lua function
    - BUG/MINOR: http-rules: Return ACT_RET_ABRT to abort a transaction
    - BUG/MINOR: http-rules: Preserve FLT_END analyzers on reject action
    - BUG/MINOR: http-rules: Fix a typo in the reject action function
    - MINOR: cache/filters: Initialize the cache filter when stream is created
    - MINOR: compression/filters: Initialize the comp filter when stream is created
    - BUG/MINOR: rules: Preserve FLT_END analyzers on silent-drop action
    - BUG/MINOR: rules: Return ACT_RET_ABRT when a silent-drop action is executed
    - BUG/MINOR: rules: Increment be_counters if backend is assigned for a silent-drop
    - BUG/MINOR: http-rules: Abort transaction when a redirect is applied on response
    - BUILD: buffer: types/{ring.h,checks.h} should include buf.h, not buffer.h
    - BUILD: ssl: include mini-clist.h
    - BUILD: global: must not include common/standard.h but only types/freq_ctr.h
    - BUILD: freq_ctr: proto/freq_ctr needs to include common/standard.h
    - BUILD: listener: types/listener.h must not include standard.h
    - BUG/MEDIUM: random: initialize the random pool a bit better
    - BUG/MEDIUM: random: implement per-thread and per-process random sequences
    - Revert "BUG/MEDIUM: random: implement per-thread and per-process random sequences"
    - BUILD: cirrus-ci: get rid of unstable freebsd images
    - MINOR: tools: add 64-bit rotate operators
    - BUG/MEDIUM: random: implement a thread-safe and process-safe PRNG
    - MINOR: backend: use a single call to ha_random32() for the random LB algo
    - BUG/MINOR: checks/threads: use ha_random() and not rand()
    - MINOR: sample: make all bits random on the rand() sample fetch
    - MINOR: tools: add a generic function to generate UUIDs
    - DOC: fix typo about no-tls-tickets
    - DOC: improve description of no-tls-tickets
    - DOC: assorted typo fixes in the documentation
    - CLEANUP: remove unused code in 'my_ffsl/my_flsl' functions
2020-03-09 14:57:20 +01:00
Ilya Shipitsin
2075ca8a93 DOC: assorted typo fixes in the documentation
This is the third round of cleanups in various docs
2020-03-09 14:45:58 +01:00
William Lallemand
90de53dc79 DOC: schematic of the SSL certificates architecture
This patch provides a schematic of the new architecture based on the
struct cert_key_and_chain which appeared with haproxy 2.1.

Could be backported in 2.1
2020-02-10 11:46:59 +01:00
Willy Tarreau
71f95fa20e [RELEASE] Released version 2.2-dev1
Released version 2.2-dev1 with the following main changes :
    - DOC: this is development again
    - MINOR: version: this is development again, update the status
    - SCRIPTS: update create-release to fix the changelog on new branches
    - CLEANUP: ssl: Clean up error handling
    - BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only
    - BUG/MINOR: h1: Don't test the host header during response parsing
    - BUILD/MINOR: trace: fix use of long type in a few printf format strings
    - DOC: Clarify behavior of server maxconn in HTTP mode
    - MINOR: ssl: deduplicate ca-file
    - MINOR: ssl: compute ca-list from deduplicate ca-file
    - MINOR: ssl: deduplicate crl-file
    - CLEANUP: dns: resolution can never be null
    - BUG/MINOR: http-htx: Don't make http_find_header() fail if the value is empty
    - DOC: ssl/cli: set/commit/abort ssl cert
    - BUG/MINOR: ssl: fix SSL_CTX_set1_chain compatibility for openssl < 1.0.2
    - BUG/MINOR: fcgi-app: Make the directive pass-header case insensitive
    - BUG/MINOR: stats: Fix HTML output for the frontends heading
    - BUG/MINOR: ssl: fix X509 compatibility for openssl < 1.1.0
    - DOC: clarify matching strings on binary fetches
    - DOC: Fix ordered list in summary
    - DOC: move the "group" keyword at the right place
    - MEDIUM: init: prevent process and thread creation at runtime
    - BUG/MINOR: ssl/cli: 'ssl cert' cmd only usable w/ admin rights
    - BUG/MEDIUM: stream-int: don't subscribed for recv when we're trying to flush data
    - BUG/MINOR: stream-int: avoid calling rcv_buf() when splicing is still possible
    - BUG/MINOR: ssl/cli: don't overwrite the filters variable
    - BUG/MEDIUM: listener/thread: fix a race when pausing a listener
    - BUG/MINOR: ssl: certificate choice can be unexpected with openssl >= 1.1.1
    - BUG/MEDIUM: mux-h1: Never reuse H1 connection if a shutw is pending
    - BUG/MINOR: mux-h1: Don't rely on CO_FL_SOCK_RD_SH to set H1C_F_CS_SHUTDOWN
    - BUG/MINOR: mux-h1: Fix conditions to know whether or not we may receive data
    - BUG/MEDIUM: tasks: Make sure we switch wait queues in task_set_affinity().
    - BUG/MEDIUM: checks: Make sure we set the task affinity just before connecting.
    - MINOR: debug: replace popen() with pipe+fork() in "debug dev exec"
    - MEDIUM: init: set NO_NEW_PRIVS by default when supported
    - BUG/MINOR: mux-h1: Be sure to set CS_FL_WANT_ROOM when EOM can't be added
    - BUG/MEDIUM: mux-fcgi: Handle cases where the HTX EOM block cannot be inserted
    - BUG/MINOR: proxy: make soft_stop() also close FDs in LI_PAUSED state
    - BUG/MINOR: listener/threads: always use atomic ops to clear the FD events
    - BUG/MINOR: listener: also clear the error flag on a paused listener
    - BUG/MEDIUM: listener/threads: fix a remaining race in the listener's accept()
    - MINOR: listener: make the wait paths cleaner and more reliable
    - MINOR: listener: split dequeue_all_listener() in two
    - REORG: listener: move the global listener queue code to listener.c
    - DOC: document the listener state transitions
    - BUG/MEDIUM: kqueue: Make sure we report read events even when no data.
    - BUG/MAJOR: dns: add minimalist error processing on the Rx path
    - BUG/MEDIUM: proto_udp/threads: recv() and send() must not be exclusive.
    - DOC: listeners: add a few missing transitions
    - BUG/MINOR: tasks: only requeue a task if it was already in the queue
    - MINOR: tasks: split wake_expired_tasks() in two parts to avoid useless wakeups
    - DOC: proxies: HAProxy only supports 3 connection modes
    - DOC: remove references to the outdated architecture.txt
    - BUG/MINOR: log: fix minor resource leaks on logformat error path
    - BUG/MINOR: mworker: properly pass SIGTTOU/SIGTTIN to workers
    - BUG/MINOR: listener: do not immediately resume on transient error
    - BUG/MINOR: server: make "agent-addr" work on default-server line
    - BUG/MINOR: listener: fix off-by-one in state name check
    - BUILD/MINOR: unix sockets: silence an absurd gcc warning about strncpy()
    - MEDIUM: h1-htx: Add HTX EOM block when the message is in H1_MSG_DONE state
    - MINOR: http-htx: Add some htx sample fetches for debugging purpose
    - REGTEST: Add an HTX reg-test to check an edge case
    - DOC: clarify the fact that replace-uri works on a full URI
    - BUG/MINOR: sample: fix the closing bracket and LF in the debug converter
    - BUG/MINOR: sample: always check converters' arguments
    - MINOR: sample: Validate the number of bits for the sha2 converter
    - BUG/MEDIUM: ssl: Don't set the max early data we can receive too early.
    - MINOR: ssl/cli: 'show ssl cert' give information on the certificates
    - BUG/MINOR: ssl/cli: fix build for openssl < 1.0.2
    - MINOR: debug: support logging to various sinks
    - MINOR: http: add a new "replace-path" action
    - REGTEST: ssl: test the "set ssl cert" CLI command
    - REGTEST: run-regtests: implement #REQUIRE_BINARIES
    - MINOR: task: only check TASK_WOKEN_ANY to decide to requeue a task
    - BUG/MAJOR: task: add a new TASK_SHARED_WQ flag to fix foreing requeuing
    - BUG/MEDIUM: ssl: Revamp the way early data are handled.
    - MINOR: fd/threads: make _GET_NEXT()/_GET_PREV() use the volatile attribute
    - BUG/MEDIUM: fd/threads: fix a concurrency issue between add and rm on the same fd
    - REGTEST: make the "set ssl cert" require version 2.1
    - BUG/MINOR: ssl: openssl-compat: Fix getm_ defines
    - BUG/MEDIUM: state-file: do not allocate a full buffer for each server entry
    - BUG/MINOR: state-file: do not store duplicates in the global tree
    - BUG/MINOR: state-file: do not leak memory on parse errors
    - BUG/MAJOR: mux-h1: Don't pretend the input channel's buffer is full if empty
    - BUG/MEDIUM: stream: Be sure to never assign a TCP backend to an HTX stream
    - BUILD: ssl: improve SSL_CTX_set_ecdh_auto compatibility
    - BUILD: travis-ci: link with ssl libraries using rpath instead of LD_LIBRARY_PATH/DYLD_LIBRARY_PATH
    - BUILD: travis-ci: reenable address sanitizer for clang builds
    - BUG/MINOR: checks: refine which errno values are really errors.
    - BUG/MINOR: connection: only wake send/recv callbacks if the FD is active
    - CLEANUP: connection: conn->xprt is never NULL
    - MINOR: pollers: add a new flag to indicate pollers reporting ERR & HUP
    - MEDIUM: tcp: make tcp_connect_probe() consider ERR/HUP
    - REORG: connection: move tcp_connect_probe() to conn_fd_check()
    - MINOR: connection: check for connection validation earlier
    - MINOR: connection: remove the double test on xprt_done_cb()
    - CLEANUP: connection: merge CO_FL_NOTIFY_DATA and CO_FL_NOTIFY_DONE
    - MINOR: poller: do not call the IO handler if the FD is not active
    - OPTIM: epoll: always poll for recv if neither active nor ready
    - OPTIM: polling: do not create update entries for FD removal
    - BUG/MEDIUM: checks: Only attempt to do handshakes if the connection is ready.
    - BUG/MEDIUM: connections: Hold the lock when wanting to kill a connection.
    - BUILD: CI: modernize cirrus-ci
    - MINOR: config: disable busy polling on old processes
    - MINOR: ssl: Remove unused variable "need_out".
    - BUG/MINOR: h1: Report the right error position when a header value is invalid
    - BUG/MINOR: proxy: Fix input data copy when an error is captured
    - BUG/MEDIUM: http-ana: Truncate the response when a redirect rule is applied
    - BUG/MINOR: channel: inject output data at the end of output
    - BUG/MEDIUM: session: do not report a failure when rejecting a session
    - MEDIUM: dns: implement synchronous send
    - MINOR: raw_sock: make sure to disable polling once everything is sent
    - MINOR: http: Add 410 to http-request deny
    - MINOR: http: Add 404 to http-request deny
    - CLEANUP: mux-h2: remove unused goto "out_free_h2s"
    - BUILD: cirrus-ci: choose proper openssl package name
    - BUG/MAJOR: listener: do not schedule a task-less proxy
    - CLEANUP: server: remove unused err section in server_finalize_init
    - REGTEST: set_ssl_cert.vtc: replace "echo" with "printf"
    - BUG/MINOR: stream-int: Don't trigger L7 retry if max retries is already reached
    - BUG/MEDIUM: tasks: Use the MT macros in tasklet_free().
    - BUG/MINOR: mux-h2: use a safe list_for_each_entry in h2_send()
    - BUG/MEDIUM: mux-h2: fix missing test on sending_list in previous patch
    - CLEANUP: ssl: remove opendir call in ssl_sock_load_cert
    - MEDIUM: lua: don't call the GC as often when dealing with outgoing connections
    - BUG/MEDIUM: mux-h2: don't stop sending when crossing a buffer boundary
    - BUG/MINOR: cli/mworker: can't start haproxy with 2 programs
    - REGTEST: mcli/mcli_start_progs: start 2 programs
    - BUG/MEDIUM: mworker: remain in mworker mode during reload
    - DOC: clarify crt-base usage
    - CLEANUP: compression: remove unused deinit_comp_ctx section
    - BUG/MEDIUM: mux_h1: Don't call h1_send if we subscribed().
    - BUG/MEDIUM: raw_sock: Make sur the fd and conn are sync.
    - CLEANUP: proxy: simplify proxy_parse_rate_limit proxy checks
    - BUG/MAJOR: hashes: fix the signedness of the hash inputs
    - REGTEST: add sample_fetches/hashes.vtc to validate hashes
    - BUG/MEDIUM: cli: _getsocks must send the peers sockets
    - CLEANUP: cli: deduplicate the code in _getsocks
    - BUG/MINOR: stream: don't mistake match rules for store-request rules
    - BUG/MEDIUM: connection: add a mux flag to indicate splice usability
    - BUG/MINOR: pattern: handle errors from fgets when trying to load patterns
    - MINOR: connection: move the CO_FL_WAIT_ROOM cleanup to the reader only
    - MINOR: stream-int: remove dependency on CO_FL_WAIT_ROOM for rcv_buf()
    - MEDIUM: connection: get rid of CO_FL_CURR_* flags
    - BUILD: pattern: include errno.h
    - MEDIUM: mux-h2: do not try to stop sending streams on blocked mux
    - MEDIUM: mux-fcgi: do not try to stop sending streams on blocked mux
    - MEDIUM: mux-h2: do not make an h2s subscribe to itself on deferred shut
    - MEDIUM: mux-fcgi: do not make an fstrm subscribe to itself on deferred shut
    - REORG: stream/backend: move backend-specific stuff to backend.c
    - MEDIUM: backend: move the connection finalization step to back_handle_st_con()
    - MEDIUM: connection: merge the send_wait and recv_wait entries
    - MEDIUM: xprt: merge recv_wait and send_wait in xprt_handshake
    - MEDIUM: ssl: merge recv_wait and send_wait in ssl_sock
    - MEDIUM: mux-h1: merge recv_wait and send_wait
    - MEDIUM: mux-h2: merge recv_wait and send_wait event notifications
    - MEDIUM: mux-fcgi: merge recv_wait and send_wait event notifications
    - MINOR: connection: make the last arg of subscribe() a struct wait_event*
    - MINOR: ssl: Add support for returning the dn samples from ssl_(c|f)_(i|s)_dn in LDAP v3 (RFC2253) format.
    - DOC: Fix copy and paste mistake in http-response replace-value doc
    - BUG/MINOR: cache: Fix leak of cache name in error path
    - BUG/MINOR: dns: Make dns_query_id_seed unsigned
    - BUG/MINOR: 51d: Fix bug when HTX is enabled
    - MINOR: http-htx: Move htx sample fetches in the scope "internal"
    - MINOR: http-htx: Rename 'internal.htx_blk.val' to 'internal.htx_blk.data'
    - MINOR: http-htx: Make 'internal.htx_blk_data' return a binary string
    - DOC: Add a section to document the internal sample fetches
    - MINOR: mux-h1: Inherit send flags from the upper layer
    - MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics
    - BUG/MINOR: http-ana/filters: Wait end of the http_end callback for all filters
    - BUG/MINOR: http-rules: Remove buggy deinit functions for HTTP rules
    - BUG/MINOR: stick-table: Use MAX_SESS_STKCTR as the max track ID during parsing
    - MEDIUM: http-rules: Register an action keyword for all http rules
    - MINOR: tcp-rules: Always set from which ruleset a rule comes from
    - MINOR: actions: Use ACT_RET_CONT code to ignore an error from a custom action
    - MINOR: tcp-rules: Kill connections when custom actions return ACT_RET_ERR
    - MINOR: http-rules: Return an error when custom actions return ACT_RET_ERR
    - MINOR: counters: Add a counter to report internal processing errors
    - MEDIUM: http-ana: Properly handle internal processing errors
    - MINOR: http-rules: Add a rule result to report internal error
    - MINOR: http-rules: Handle internal errors during HTTP rules evaluation
    - MINOR: http-rules: Add more return codes to let custom actions act as normal ones
    - MINOR: tcp-rules: Handle denied/aborted/invalid connections from TCP rules
    - MINOR: http-rules: Handle denied/aborted/invalid connections from HTTP rules
    - MINOR: stats: Report internal errors in the proxies/listeners/servers stats
    - MINOR: contrib/prometheus-exporter: Export internal errors per proxy/server
    - MINOR: counters: Remove failed_secu counter and use denied_resp instead
    - MINOR: counters: Review conditions to increment counters from analysers
    - MINOR: http-ana: Add a txn flag to support soft/strict message rewrites
    - MINOR: http-rules: Handle all message rewrites the same way
    - MINOR: http-rules: Add a rule to enable or disable the strict rewriting mode
    - MEDIUM: http-rules: Enable the strict rewriting mode by default
    - REGTEST: Fix format of set-uri HTTP request rule in h1or2_to_h1c.vtc
    - MINOR: actions: Add a function pointer to release args used by actions
    - MINOR: actions: Regroup some info about HTTP rules in the same struct
    - MINOR: http-rules/tcp-rules: Call the defined action function first if defined
    - MINOR: actions: Rename the act_flag enum into act_opt
    - MINOR: actions: Add flags to configure the action behaviour
    - MINOR: actions: Use an integer to set the action type
    - MINOR: http-rules: Use a specific action type for some custom HTTP actions
    - MINOR: http-rules: Make replace-header and replace-value custom actions
    - MINOR: http-rules: Make set-header and add-header custom actions
    - MINOR: http-rules: Make set/del-map and add/del-acl custom actions
    - MINOR: http-rules: Group all processing of early-hint rule in its case clause
    - MEDIUM: http-rules: Make early-hint custom actions
    - MINOR: http-rule/tcp-rules: Make track-sc* custom actions
    - MINOR: tcp-rules: Make tcp-request capture a custom action
    - MINOR: http-rules: Add release functions for existing HTTP actions
    - BUG/MINOR: http-rules: Fix memory releases on error path during action parsing
    - MINOR: tcp-rules: Add release functions for existing TCP actions
    - BUG/MINOR: tcp-rules: Fix memory releases on error path during action parsing
    - MINOR: http-htx: Add functions to read a raw error file and convert it in HTX
    - MINOR: http-htx: Add functions to create HTX redirect message
    - MINOR: config: Use dedicated function to parse proxy's errorfiles
    - MINOR: config: Use dedicated function to parse proxy's errorloc
    - MEDIUM: http-htx/proxy: Use a global and centralized storage for HTTP error messages
    - MINOR: proxy: Register keywords to parse errorfile and errorloc directives
    - MINOR: http-htx: Add a new section to create groups of custom HTTP errors
    - MEDIUM: proxy: Add a directive to reference an http-errors section in a proxy
    - MINOR: http-rules: Update txn flags and status when a deny rule is executed
    - MINOR: http-rules: Support an optional status on deny rules for http reponses
    - MINOR: http-rules: Use same function to parse request and response deny actions
    - MINOR: http-ana: Add an error message in the txn and send it when defined
    - MEDIUM: http-rules: Support an optional error message in http deny rules
    - REGTEST: Add a strict rewriting mode reg test
    - REGEST: Add reg tests about error files
    - MINOR: ssl: accept 'verify' bind option with 'set ssl cert'
    - BUG/MINOR: ssl: ssl_sock_load_ocsp_response_from_file memory leak
    - BUG/MINOR: ssl: ssl_sock_load_issuer_file_into_ckch memory leak
    - BUG/MINOR: ssl: ssl_sock_load_sctl_from_file memory leak
    - BUG/MINOR: http_htx: Fix some leaks on error path when error files are loaded
    - CLEANUP: http-ana: Remove useless test on txn when the error message is retrieved
    - BUILD: CI: introduce ARM64 builds
    - BUILD: ssl: more elegant anti-replay feature presence check
    - MINOR: proxy/http-ana: Add support of extra attributes for the cookie directive
    - MEDIUM: dns: use Additional records from SRV responses
    - CLEANUP: Consistently `unsigned int` for bitfields
    - CLEANUP: pattern: remove the pat_time definition
    - BUG/MINOR: http_act: don't check capture id in backend
    - BUG/MINOR: ssl: fix build on development versions of openssl-1.1.x
2020-01-22 10:34:58 +01:00
Willy Tarreau
4ac36d691a DOC: listeners: add a few missing transitions
Some disable() transitions were missing, and the distinction between
multi-threaded and single-threaded transitions was not mentioned.
2019-12-11 07:44:34 +01:00
Willy Tarreau
977afab3f8 DOC: document the listener state transitions
This was done by reading all the code affecting a listener's state,
hopefully it will save some time in the future.
2019-12-10 16:06:53 +01:00
Willy Tarreau
64a18534e3 DOC: internal: document the init calls
INITCALLs are used a lot in the code now and were not documented, resulting
in each user having to grep for functions in other files. This doc is not
perfect but aims at improving the situation. It documents what's been
available since 1.9 and may be backported there if it helps though it's
unlikely to be needed as it's mostly aimed at developers.
2019-11-20 16:52:56 +01:00
Willy Tarreau
cb8f03fc46 [RELEASE] Released version 2.1-dev2
Released version 2.1-dev2 with the following main changes :
    - DOC: management: document reuse and connect counters in the CSV format
    - DOC: management: document cache_hits and cache_lookups in the CSV format
    - BUG/MINOR: dns: remove irrelevant dependency on a client connection
    - MINOR: applet: make appctx use their own pool
    - BUG/MEDIUM: checks: Don't attempt to receive data if we already subscribed.
    - BUG/MEDIUM: http/htx: unbreak option http_proxy
    - BUG/MINOR: backend: do not try to install a mux when the connection failed
    - MINOR: mux-h2: Don't adjust anymore the amount of data sent in h2_snd_buf()
    - BUG/MINOR: http_fetch: Fix http_auth/http_auth_group when called from TCP rules
    - BUG/MINOR: http_htx: Initialize HTX error messages for TCP proxies
    - BUG/MINOR: cache/htx: Make maxage calculation HTX aware
    - BUG/MINOR: hlua: Make the function txn:done() HTX aware
    - MINOR: proto_htx: Directly call htx_check_response_for_cacheability()
    - MINOR: proto_htx: Rely on the HTX function to apply a redirect rules
    - MINOR: proto_htx: Add the function htx_return_srv_error()
    - MINOR: backend/htx: Don't rewind output data to set the sni on a srv connection
    - MINOR: proto_htx: Don't stop forwarding when there is a post-connect processing
    - DOC: htx: Update comments in HTX files
    - CLEANUP: htx: Remove the unsued function htx_add_blk_type_size()
    - MINOR: htx: Deduce the number of used blocks from tail and head values
    - MINOR: htx: Use an array of char to store HTX blocks
    - MINOR: htx: Slightly update htx_dump() to report better messages
    - DOC: htx: Add internal documentation about the HTX
    - MAJOR: http: Deprecate and ignore the option "http-use-htx"
    - MEDIUM: mux-h2: Remove support of the legacy HTTP mode
    - CLEANUP: h2: Remove functions converting h2 requests to raw HTTP/1.1 ones
    - MINOR: connection: Remove the multiplexer protocol PROTO_MODE_HTX
    - MINOR: stream: Rely on HTX analyzers instead of legacy HTTP ones
    - MEDIUM: http_fetch: Remove code relying on HTTP legacy mode
    - MINOR: config: Remove tests on the option 'http-use-htx'
    - MINOR: stream: Remove tests on the option 'http-use-htx' in stream_new()
    - MINOR: proxy: Remove tests on the option 'http-use-htx' during H1 upgrade
    - MINOR: hlua: Remove tests on the option 'http-use-htx' to reject TCP applets
    - MINOR: cache: Remove tests on the option 'http-use-htx'
    - MINOR: contrib/prometheus-exporter: Remove tests on the option 'http-use-htx'
    - CLEANUP: proxy: Remove the flag PR_O2_USE_HTX
    - MINOR: proxy: Don't adjust connection mode of HTTP proxies anymore
    - MEDIUM: backend: Remove code relying on the HTTP legacy mode
    - MEDIUM: hlua: Remove code relying on the legacy HTTP mode
    - MINOR: http_act: Remove code relying on the legacy HTTP mode
    - MEDIUM: cache: Remove code relying on the legacy HTTP mode
    - MEDIUM: compression: Remove code relying on the legacy HTTP mode
    - MINOR: flt_trace: Remove code relying on the legacy HTTP mode
    - MINOR: stats: Remove code relying on the legacy HTTP mode
    - MAJOR: filters: Remove code relying on the legacy HTTP mode
    - MINOR: stream: Remove code relying on the legacy HTTP mode
    - MAJOR: http: Remove the HTTP legacy code
    - MINOR: hlua: Remove useless test on TX_CON_WANT_* flags
    - MINOR: proto_http: Remove unused http txn flags
    - MINOR: proto_http: Remove the unused flag HTTP_MSGF_WAIT_CONN
    - CLEANUP: proto_http: Group remaining flags of the HTTP transaction
    - CLEANUP: channel: Remove the unused flag CF_WAKE_CONNECT
    - CLEANUP: proto_http: Remove unecessary includes and comments
    - CLEANUP: proto_http: Move remaining code from proto_http.c to proto_htx.c
    - REORG: proto_htx: Move HTX analyzers & co to http_ana.{c,h} files
    - BUG/MINOR: debug: Remove flags CO_FL_SOCK_WR_ENA/CO_FL_SOCK_RD_ENA
    - MINOR: proxy: Remove support of the option 'http-tunnel'
    - DOC: config: Update as a result of the legacy HTTP removal
    - MEDIUM: config: Remove parsing of req* and rsp* directives
    - MINOR: proxy: Remove the unused list of block rules
    - MINOR: proxy/http_ana: Remove unused req_exp/rsp_exp and req_add/rsp_add lists
    - DOC: config: Remove unsupported req* and rsp* keywords
    - MINOR: global: Preset tune.max_http_hdr to its default value
    - MINOR: http: Don't store raw HTTP errors in chunks anymore
    - BUG/MINOR: session: Emit an HTTP error if accept fails only for H1 connection
    - BUG/MINOR: session: Send a default HTTP error if accept fails for a H1 socket
    - CLEANUP: mux-h2: Remove unused flags H2_SF_CHNK_*
    - BUG/MINOR: checks: do not exit tcp-checks from the middle of the loop
    - MINOR: config: Warn only if the option http-use-htx is used with "no" prefix
    - BUG/MEDIUM: mux-h1: Trim excess server data at the end of a transaction
    - MINOR: connection: add conn_get_src() and conn_get_dst()
    - MINOR: frontend: switch to conn_get_{src,dst}() for logging and debugging
    - MINOR: backend: switch to conn_get_{src,dst}() for port and address mapping
    - MINOR: ssl: switch to conn_get_dst() to retrieve the destination address
    - MINOR: tcp: replace various calls to conn_get_{from,to}_addr with conn_get_{src,dst}
    - MINOR: stream-int: use conn_get_{src,dst} in conn_si_send_proxy()
    - MINOR: stream/cli: use conn_get_{src,dst} in "show sess" and "show peers" output
    - MINOR: log: use conn_get_{dst,src}() to retrieve the cli/frt/bck/srv/ addresses
    - MINOR: http/htx: use conn_get_dst() to retrieve the destination address
    - MINOR: lua: use conn_get_{src,dst} to retrieve connection addresses
    - MINOR: http: check the source address via conn_get_src() in sample fetch functions
    - CLEANUP: connection: remove the now unused conn_get_{from,to}_addr()
    - MINOR: connection: add new src and dst fields
    - MINOR: connection: use conn->{src,dst} instead of &conn->addr.{from,to}
    - MINOR: ssl-sock: use conn->dst instead of &conn->addr.to
    - MINOR: lua: switch to conn->dst for a connection's target address
    - MINOR: peers: use conn->dst for the peer's target address
    - MINOR: htx: switch from conn->addr.{from,to} to conn->{src,dst}
    - MINOR: stream: switch from conn->addr.{from,to} to conn->{src,dst}
    - MINOR: proxy: switch to conn->src in error snapshots
    - MINOR: session: use conn->src instead of conn->addr.from
    - MINOR: tcp: replace conn->addr.{from,to} with conn->{src,dst}
    - MINOR: unix: use conn->dst for the target address in ->connect()
    - MINOR: sockpair: use conn->dst for the target address in ->connect()
    - MINOR: log: use conn->{src,dst} instead of conn->addr.{from,to}
    - MINOR: checks: replace conn->addr.to with conn->dst
    - MINOR: frontend: switch from conn->addr.{from,to} to conn->{src,dst}
    - MINOR: http: convert conn->addr.from to conn->src in sample fetches
    - MEDIUM: backend: turn all conn->addr.{from,to} to conn->{src,dst}
    - MINOR: connection: create a new pool for struct sockaddr_storage
    - MEDIUM: connection: make sure all address producers allocate their address
    - MAJOR: connection: remove the addr field
    - MINOR: connection: don't use clear_addr() anymore, just release the address
    - MINOR: stream: add a new target_addr entry in the stream structure
    - MAJOR: stream: store the target address into s->target_addr
    - MINOR: peers: now remove the remote connection setup code
    - MEDIUM: lua: do not allocate the remote connection anymore
    - MEDIUM: backend: always release any existing prior connection in connect_server()
    - MEDIUM: backend: remove impossible cases from connect_server()
    - BUG/MINOR: mux-h1: Close server connection if input data remains in h1_detach()
    - BUG/MEDIUM: tcp-checks: do not dereference inexisting conn_stream
    - BUG/MINOR: http_ana: Be sure to have an allocated buffer to generate an error
    - BUG/MINOR: http_htx: Support empty errorfiles
    - BUG/CRITICAL: http_ana: Fix parsing of malformed cookies which start by a delimiter
    - BUG/MEDIUM: protocols: add a global lock for the init/deinit stuff
    - BUG/MINOR: proxy: always lock stop_proxy()
    - MEDIUM: mux-h1: Add the support of headers adjustment for bogus HTTP/1 apps
    - BUILD: threads: add the definition of PROTO_LOCK
    - BUG/MEDIUM: lb-chash: Fix the realloc() when the number of nodes is increased
    - BUG/MEDIUM: streams: Don't switch the SI to SI_ST_DIS if we have data to send.
    - BUG/MINOR: log: make sure writev() is not interrupted on a file output
    - DOC: improve the wording in CONTRIBUTING about how to document a bug fix
    - MEDIUM: h1: Don't try to subscribe if we managed to read data.
    - MEDIUM: h1: Don't wake the H1 tasklet if we got the whole request.
    - REGTESTS: checks: exclude freebsd target for tcp-check_multiple_ports.vtc
    - BUG/MINOR: hlua/htx: Reset channels analyzers when txn:done() is called
    - BUG/MEDIUM: hlua: Check the calling direction in lua functions of the HTTP class
    - MINOR: hlua: Don't set request analyzers on response channel for lua actions
    - MINOR: hlua: Add a flag on the lua txn to know in which context it can be used
    - BUG/MINOR: hlua: Only execute functions of HTTP class if the txn is HTTP ready
    - BUG/MINOR: htx: Fix free space addresses calculation during a block expansion
    - MINOR: ssl: merge ssl_sock_load_cert_file() and ssl_sock_load_cert_chain_file()
    - MEDIUM: ssl: use cert_key_and_chain struct in ssl_sock_load_cert_file()
    - MEDIUM: ssl: split the loading of the certificates
    - MEDIUM: ssl: lookup and store in a ckch_node tree
    - MEDIUM: ssl: load DH param in struct cert_key_and_chain
    - BUG/MAJOR: queue/threads: avoid an AB/BA locking issue in process_srv_queue()
    - MINOR: ssl: use STACK_OF for chain certs
    - MINOR: ssl: add extra chain compatibility
    - MINOR: ssl: check private key consistency in loading
    - MINOR: ssl: do not look at DHparam with OPENSSL_NO_DH
    - CLEANUP: ssl: ssl_sock_load_crt_file_into_ckch
    - MINOR: ssl: clean ret variable in ssl_sock_load_ckchn
    - MAJOR: fd: Get rid of the fd cache.
    - MEDIUM: pollers: Remember the state for read and write for each threads.
    - MEDIUM: mux-h2: don't try to read more than needed
    - BUG/BUILD: ssl: fix build with openssl < 1.0.2
    - BUG/MEDIUM: ssl: does not try to free a DH in a ckch
    - BUG/MINOR: debug: fix a small race in the thread dumping code
    - MINOR: wdt: also consider that waiting in the thread dumper is normal
    - REGTESTS: checks: make 4be_1srv_health_checks more reliable
    - BUILD: ssl: BoringSSL add EVP_PKEY_base_id
    - BUG/MEDIUM: ssl: don't free the ckch in multi-cert bundle
    - BUG/MINOR: ssl: fix ressource leaks on error
    - BUG/MEDIUM: lb-chash: Ensure the tree integrity when server weight is increased
    - BUG/MAJOR: http/sample: use a static buffer for raw -> htx conversion
    - BUG/MINOR: stream-int: make sure to always release empty buffers after sending
    - BUG/MEDIUM: ssl: open the right path for multi-cert bundle
    - BUG/MINOR: stream-int: also update analysers timeouts on activity
    - BUG/MEDIUM: mux-h2: unbreak receipt of large DATA frames
    - BUG/MEDIUM: mux-h2: split the stream's and connection's window sizes
    - BUG/MEDIUM: proxy: Make sure to destroy the stream on upgrade from TCP to H2
    - DOC: Add 'Question.md' issue template, discouraging asking questions
    - BUG/MEDIUM: fd: Always reset the polled_mask bits in fd_dodelete().
    - BUG/MEDIUM: pollers: Clear the poll_send bits as well.
    - BUILD: travis-ci: enable daily Coverity scan
    - BUG/MINOR: mux-h2: don't refrain from sending an RST_STREAM after another one
    - BUG/MINOR: mux-h2: use CANCEL, not STREAM_CLOSED in h2c_frt_handle_data()
    - BUG/MINOR: mux-h2: do not send REFUSED_STREAM on aborted uploads
    - BUG/MEDIUM: mux-h2: do not recheck a frame type after a state transition
    - BUG/MINOR: mux-h2: always send stream window update before connection's
    - BUG/MINOR: mux-h2: always reset rcvd_s when switching to a new frame
    - BUG/MEDIUM: checks: make sure to close nicely when we're the last to speak
    - BUG/MEDIUM: stick-table: Wrong stick-table backends parsing.
    - CLEANUP: mux-h2: move the demuxed frame check code in its own function
    - MINOR: cache: add method to cache hash
    - MINOR: cache: allow caching of OPTIONS request
    - BUG/MINOR: ssl: fix 0-RTT for BoringSSL
    - MINOR: ssl: ssl_fc_has_early should work for BoringSSL
    - BUG/MINOR: pools: don't mark the thread harmless if already isolated
    - BUG/MINOR: buffers/threads: always clear a buffer's head before releasing it
    - CLEANUP: buffer: replace b_drop() with b_free()
    - CLEANUP: task: move the cpu_time field to the task-only part
    - MINOR: cli: add two new states to print messages on the CLI
    - MINOR: cli: add cli_msg(), cli_err(), cli_dynmsg(), cli_dynerr()
    - CLEANUP: cli: replace all occurrences of manual handling of return messages
    - BUG/MEDIUM: proxy: Don't forget the SF_HTX flag when upgrading TCP=>H1+HTX.
    - BUG/MEDIUM: proxy: Don't use cs_destroy() when freeing the conn_stream.
    - BUG/MINOR: lua: fix setting netfilter mark
    - BUG/MINOR: Fix prometheus '# TYPE' and '# HELP' headers
    - BUG/MEDIUM: lua: Fix test on the direction to set the channel exp timeout
    - BUG/MINOR: stats: Wait the body before processing POST requests
    - MINOR: fd: make sure to mark the thread as not stuck in fd_update_events()
    - BUG/MEDIUM: mux_pt: Don't call unsubscribe if we did not subscribe.
    - BUILD: travis-ci: trigger non-mainstream configurations only on daily builds.
    - MINOR: debug: indicate the applet name when the task is task_run_applet()
    - MINOR: tools: add append_prefixed_str()
    - MINOR: lua: export applet and task handlers
    - MEDIUM: debug: make the thread dump code show Lua backtraces
    - BUG/MEDIUM: h1: Always try to receive more in h1_rcv_buf().
    - MINOR: list: add LIST_SPLICE() to merge one list into another
    - MINOR: tools: add a DEFNULL() macro to use NULL for empty args
    - REORG: trace: rename trace.c to calltrace.c and mention it's not thread-safe
    - MINOR: sink: create definitions a minimal code for event sinks
    - MINOR: sink: add a support for file descriptors
    - MINOR: trace: start to create a new trace subsystem
    - MINOR: trace: add allocation of buffer-sized trace buffers
    - MINOR: trace/cli: register the "trace" CLI keyword to list the sources
    - MINOR: trace/cli: parse the "level" argument to configure the trace verbosity
    - MINOR: trace/cli: add "show trace" to report trace state and statistics
    - MINOR: trace: implement a very basic trace() function
    - MINOR: trace: add the file name and line number in the prefix
    - MINOR: trace: make trace() now also take a level in argument
    - MINOR: trace: implement a call to a decode function
    - MINOR: trace: add per-level macros to produce traces
    - MINOR: trace: add a definition of typed arguments to trace()
    - MINOR: trace: make sure to always stop the locking when stopping or pausing
    - MINOR: trace: add the possibility to lock on some arguments
    - MINOR: trace: parse the "lock" argument to trace
    - MINOR: trace: retrieve useful pointers and enforce lock-on
    - DOC: management: document the "trace" and "show trace" commands
    - BUILD: trace: make the lockon_ptr const to silence a warning without threads
    - BUG/MEDIUM: mux-h1: do not truncate trailing 0CRLF on buffer boundary
    - BUG/MEDIUM: mux-h1: do not report errors on transfers ending on buffer full
    - DOC: fixed typo in management.txt
    - BUG/MINOR: mworker: disable SIGPROF on re-exec
    - BUG/MEDIUM: listener/threads: fix an AB/BA locking issue in delete_listener()
    - BUG/MEDIUM: url32 does not take the path part into account in the returned hash.
    - MINOR: backend: Add srv_queue converter
    - MINOR: sink: set the fd-type sinks to non-blocking
    - MINOR: tools: add a function varint_bytes() to report the size of a varint
    - MINOR: buffer: add functions to read/write varints from/to buffers
    - MINOR: fd: add fd_write_frag_line() to send a fragmented line to an fd
    - MINOR: sink: now call the generic fd write function
    - MINOR: ring: add a new mechanism for retrieving/storing ring data in buffers
    - MINOR: ring: add a ring_write() function
    - MINOR: ring: add a generic CLI io_handler to dump a ring buffer
    - MINOR: sink: add support for ring buffers
    - MINOR: sink: implement "show events" to show supported sinks and dump the rings
    - MINOR: sink: now report the number of dropped events on output
    - MINOR: trace: support a default callback for the source
    - MINOR: trace: extend the source location to 13 chars
    - MINOR: trace: show thread number and source name in the trace
    - MINOR: trace: change the TRACE() calling convention to put the args and cb last
    - MINOR: connection: add the fc_pp_authority fetch -- authority TLV, from PROXYv2
    - MINOR: tools: add a generic struct "name_desc" for name-description pairs
    - MINOR: trace: replace struct trace_lockon_args with struct name_desc
    - MINOR: trace: change the "payload" level to "data" and move it
    - MINOR: trace: prepend the function name for developer level traces
    - MINOR: trace: also report the trace level in the output
    - MINOR: trace: change the detail_level to per-source verbosity
    - MINOR: mux-h2/trace: register a new trace source with its events
    - MINOR: mux-h2/trace: add the default decoding callback
    - MEDIUM: mux-h2/trace: add lots of traces all over the code
    - MINOR: mux-h2: add functions to convert an h2c/h2s state to a string
    - MINOR: mux-h2/trace: add a new verbosity level "clean"
    - MINOR: mux-h2/trace: only decode the start-line at verbosity other than "minimal"
    - MINOR: mux-h2/trace: always report the h2c/h2s state and flags
    - MINOR: mux-h2/trace: report h2s->id before h2c->dsi for the stream ID
    - CLEANUP: mux-h2/trace: reformat the "received" messages for better alignment
    - CLEANUP: mux-h2/trace: lower-case event names
    - MINOR: trace: extend default event names to 12 chars
    - BUG/MINOR: ring: fix the way watchers are counted
    - MINOR: cli: extend the CLI context with a list and two offsets
    - MINOR: mux-h2/trace: report the connection pointer and state before FRAME_H
    - MEDIUM: ring: implement a wait mode for watchers
    - BUG/MEDIUM: mux-h2/trace: do not dereference h2c->conn after failed idle
    - BUG/MEDIUM: mux-h2/trace: fix missing braces added with traces
    - BUG/MINOR: ring: b_peek_varint() returns a uint64_t, not a size_t
    - CLEANUP: fd: remove leftovers of the fdcache
    - MINOR: fd: add a new "initialized" bit in the fdtab struct
    - MINOR: fd/log/sink: make the non-blocking initialization depend on the initialized bit
    - MEDIUM: log: use the new generic fd_write_frag_line() function
    - MINOR: log: add a target type instead of hacking the address family
    - MEDIUM: log: add support for logging to a ring buffer
    - MINOR: send-proxy-v2: sends authority TLV according to TLV received
    - MINOR: build: add linux-glibc-legacy build TARGET
    - BUG/MEDIUM: peers: local peer socket not bound.
    - BUILD: connection: silence gcc warning with extra parentheses
    - BUG/MINOR: http-ana: Reset response flags when 1xx messages are handled
    - BUG/MINOR: h1: Properly reset h1m when parsing is restarted
    - BUG/MINOR: mux-h1: Fix size evaluation of HTX messages after headers parsing
    - BUG/MINOR: mux-h1: Don't stop anymore input processing when the max is reached
    - BUG/MINOR: mux-h1: Be sure to update the count before adding EOM after trailers
    - BUG/MEDIUM: cache: Properly copy headers splitted on several shctx blocks
    - BUG/MEDIUM: cache: Don't cache objects if the size of headers is too big
    - BUG/MINOR: mux-h1: Fix a possible null pointer dereference in h1_subscribe()
    - MEDIUM: fd: remove the FD_EV_POLLED status bit
    - MEDIUM: fd: simplify the fd_*_{recv,send} functions using BTS/BTR
    - MINOR: fd: make updt_fd_polling() a normal function
    - CONTRIB: debug: add new program "poll" to test poll() events
    - BUG/MINOR: checks: stop polling for write when we have nothing left to send
    - BUG/MINOR: checks: start sending the request right after connect()
    - BUG/MINOR: checks: make __event_chk_srv_r() report success before closing
    - BUG/MINOR: checks: do not uselessly poll for reads before the connection is up
    - BUG/MINOR: mux-h1: Fix a UAF in cfg_h1_headers_case_adjust_postparser()
    - BUILD: CI: add basic CentOS 6 cirrus build
    - MINOR: contrib/prometheus-exporter: Report DRAIN/MAINT/NOLB status for servers
    - BUG/MINOR: lb/leastconn: ignore the server weights for empty servers
    - BUG/MAJOR: ssl: ssl_sock was not fully initialized.
    - MEDIUM: fd: mark the FD as ready when it's inserted
    - MINOR: fd: add two new calls fd_cond_{recv,send}()
    - MEDIUM: connection: enable reading only once the connection is confirmed
    - MINOR: fd: add two flags ERR and SHUT to describe FD states
    - MEDIUM: fd: do not use the FD_POLL_* flags in the pollers anymore
    - BUG/MEDIUM: connection: don't keep more idle connections than ever needed
    - MINOR: stats: report the number of idle connections for each server
    - BUILD: CI: skip reg-tests/connection/proxy_protocol_random_fail.vtc on CentOS 6
    - BUILD/MINOR: auth: enabling for osx
    - BUG/MINOR: listener: Fix a possible null pointer dereference
    - BUG/MINOR: ssl: always check for ssl connection before getting its XPRT context
    - MINOR: stats: Add JSON export from the stats page
    - BUG/MINOR: filters: Properly set the HTTP status code on analysis error
    - MINOR: sample: Add UUID-fetch
    - CLEANUP: mux-h2: Remove unused flag H2_SF_DATA_CHNK
    - BUG/MINOR: acl: Fix memory leaks when an ACL expression is parsed
    - BUG/MINOR: backend: Fix a possible null pointer dereference
    - BUG/MINOR: Missing stat_field_names (since f21d17bb)
    - BUG/MEDIUM: stick-table: Properly handle "show table" with a data type argument
    - BUILD: CI: temporarily disable ASAN
    - MINOR: htx: Add a flag on HTX message to report processing errors
    - MINOR: mux-h1: Report a processing error during output processing
    - MINOR: http-ana: Handle HTX errors first during message analysis
    - MINOR: http-ana: Remove err_state field from http_msg
    - MINOR: config: Support per-proxy and per-server deinit functions callbacks
    - MINOR: config: Support per-proxy and per-server post-check functions callbacks
    - MINOR: http_fetch: Add sample fetches to get auth method/user/pass
    - MINOR: istbuf: Add the function b_isteqi()
    - MINOR: log: Provide a function to emit a log for an application
    - MINOR: http: Add function to parse value of the header Status
    - MEDIUM: mux-h1/h1-htx: move HTX convertion of H1 messages in dedicated file
    - MINOR: h1-htx: Use the same function to copy message payload in all cases
    - MINOR: muxes/htx: Ignore pseudo header during message formatting
    - MINOR: fcgi: Add code related to FCGI protocol
    - MEDIUM: fcgi-app: Add FCGI application and filter
    - MEDIUM: mux-fcgi: Add the FCGI multiplexer
    - MINOR: doc: Add documentation about the FastCGI support
    - BUG/MINOR: build: Fix compilation of mux_fcgi.c when compiled without SSL
    - BUILD: CI: install golang-1.13 when building BoringSSL
    - BUG/MINOR: mux-h2: Be sure to have a connection to unsubcribe
    - BUG/MINOR: mux-fcgi: Be sure to have a connection to unsubcribe
    - CLEANUP: fcgi-app: Remove useless test on fcgi_conf pointer
    - BUG/MINOR: mux-fcgi: Don't compare the filter name in its parsing callback
    - BUG/MAJOR: mux-h2: Handle HEADERS frames received after a RST_STREAM frame
    - BUG/MEDIUM: check/threads: make external checks run exclusively on thread 1
    - MEDIUM: list: Separate "locked" list from regular list.
    - MINOR: mt_lists: Add new macroes.
    - MEDIUM: servers: Use LIST_DEL_INIT() instead of LIST_DEL().
    - MINOR: mt_lists: Do nothing in MT_LIST_ADD/MT_LIST_ADDQ if already in list.
    - MINOR: mt_lists: Give MT_LIST_ADD, MT_LIST_ADDQ and MT_LIST_DEL a return value.
    - MEDIUM: tasklets: Make the tasklet list a struct mt_list.
    - TESTS: Add a stress-test for mt_lists.
    - BUILD: travis-ci: add PCRE2, SLZ build
    - BUG/MINOR: build: fix event ports (Solaris)
    - BUG/MEDIUM: namespace: fix fd leak in master-worker mode
    - OPTIM: listeners: use tasklets for the multi-queue rings
    - BUILD: makefile: work around yet another GCC fantasy (-Wstring-plus-int)
    - BUG/MINOR: stream-int: Process connection/CS errors first in si_cs_send()
    - BUG/MEDIUM: stream-int: Process connection/CS errors during synchronous sends
    - BUG/MEDIUM: checks: make sure the connection is ready before trying to recv
    - CLEANUP: task: remove impossible test
    - CLEANUP: task: cache the task_per_thread pointer
    - MINOR: task: split the tasklet vs task code in process_runnable_tasks()
    - MINOR: task: introduce a thread-local "sched" variable for local scheduler stuff
    - CLEANUP: mux-fcgi: Remove the unused function fcgi_strm_id()
    - BUG/MINOR: mux-fcgi: Use a literal string as format in app_log()
    - BUG/MEDIUM: tasklets: Make sure we're waking the target thread if it sleeps.
    - MINOR: h2/trace: indicate 'F' or 'B' to locate the side of an h2c in traces
    - MINOR: h2/trace: report the frame type when known
    - BUG/MINOR: mux-h2: do not wake up blocked streams before the mux is ready
    - BUG/MEDIUM: namespace: close open namespaces during soft shutdown
    - MINOR: time: add timeofday_as_iso_us() to return instant time as ISO
    - MINOR: sink: finally implement support for SINK_FMT_{TIMED,ISO}
    - MINOR: sink: change ring buffer "buf0"'s format to "timed"
    - BUG/MEDIUM: mux-h2: don't reject valid frames on closed streams
    - BUG/MINOR: mux-fcgi: silence a gcc warning about null dereference
    - BUG/MINOR: mux-h2: Fix missing braces because of traces in h2_detach()
    - BUG/MINOR: mux-h2: Use the dummy error when decoding headers for a closed stream
    - BUG/MAJOR: mux_h2: Don't consume more payload than received for skipped frames
    - BUG/MINOR: mux-h1: Do h2 upgrade only on the first request
    - BUG/MEDIUM: spoe: Use a different engine-id per process
    - MINOR: spoe: Improve generation of the engine-id
    - MINOR: spoe: Support the async mode with several threads
    - MINOR: http: Add server name header from HTTP multiplexers
    - CLEANUP: http-ana: Remove the unused function http_send_name_header()
    - MINOR: stats: Add the support of float fields in stats
    - BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds
    - DOC: Fix documentation about the cli command to get resolver stats
    - BUG/MEDIUM: fcgi: fix missing list tail in sample fetch registration
    - BUG/MINOR: stats: Add a missing break in a switch statement
    - BUG/MINOR: lua: Properly initialize the buffer's fields for string samples in hlua_lua2(smp|arg)
    - CLEANUP: lua: Get rid of obsolete (size_t *) cast in hlua_lua2(smp|arg)
    - BUG/MEDIUM: lua: Store stick tables into the sample's `t` field
    - CLEANUP: proxy: Remove `proxy_tbl_by_name`
    - BUILD: ssl: fix a warning when built with openssl < 1.0.2
    - DOC: replace utf-8 quotes by ascii ones
    - BUG/MEDIUM: fd: HUP is an error only when write is active
    - BUG/MINOR: action: do-resolve does not yield on requests with body
    - Revert "MINOR: cache: allow caching of OPTIONS request"
2019-10-01 18:13:09 +02:00
Willy Tarreau
5bee3e2f47 MEDIUM: fd: remove the FD_EV_POLLED status bit
Since commit 7ac0e35f2 in 1.9-dev1 ("MAJOR: fd: compute the new fd polling
state out of the fd lock") we've started to update the FD POLLED bit a
bit more aggressively. Lately with the removal of the FD cache, this bit
is always equal to the ACTIVE bit. There's no point continuing to watch
it and update it anymore, all it does is create confusion and complicate
the code. One interesting side effect is that it now becomes visible that
all fd_*_{send,recv}() operations systematically call updt_fd_polling(),
except fd_cant_recv()/fd_cant_send() which never saw it change.
2019-09-05 09:31:18 +02:00
Willy Tarreau
e0d0b4089d CLEANUP: buffer: replace b_drop() with b_free()
Since last commit there's no point anymore in having two variants of the
same function, let's switch to b_free() only. __b_drop() was renamed to
__b_free() for obvious consistency reasons.
2019-08-08 08:07:45 +02:00
Willy Tarreau
3b091f80aa BUG/MINOR: buffers/threads: always clear a buffer's head before releasing it
A small race exists in buffers with "show sess all". This one wants to show
some information grabbed from the buffer (especially in HTX mode). But the
thread owning this buffer might just be releasing its area, right after a
free() or munmap() call, resulting in a head that is not seen as empty yet
though the area was released. It may then be dereferenced by "show sess all"
causing a crash. Note that in practice it only happens in debug mode with
UAF enabled, but it's tricky enough to fix it right now.

This should be backported to stable versions which support threads and a
store barrier. It's worth noting that by performing the clearing first,
b_free() and b_drop() now become two exact equivalent.
2019-08-08 08:07:45 +02:00
Christopher Faulet
bda62e756a DOC: htx: Add internal documentation about the HTX 2019-07-19 09:18:27 +02:00
Willy Tarreau
b57f109966 [RELEASE] Released version 2.0-dev6
Released version 2.0-dev6 with the following main changes :
    - BUG/MEDIUM: connection: fix multiple handshake polling issues
    - MINOR: connection: also stop receiving after a SOCKS4 response
    - MINOR: mux-h1: don't try to recv() before the connection is ready
    - BUG/MEDIUM: mux-h1: only check input data for the current stream, not next one
    - MEDIUM: mux-h1: don't use CS_FL_REOS anymore
    - CLEANUP: connection: remove the now unused CS_FL_REOS flag
    - CONTRIB: debug: add 4 missing connection/conn_stream flags
    - MEDIUM: stream: make a full process_stream() loop when completing I/O on exit
    - MINOR: server: increase the default pool-purge-delay to 5 seconds
    - BUILD: tools: do not use the weak attribute for trace() on obsolete linkers
    - BUG/MEDIUM: vars: make sure the scope is always valid when accessing vars
    - BUG/MEDIUM: vars: make the tcp/http unset-var() action support conditions
    - BUILD: task: fix a build warning when threads are disabled
    - CLEANUP: peers: Remove tabs characters.
    - CLEANUP: peers: Replace hard-coded values by macros.
    - BUG/MINOR: peers: Wrong stick-table update message building.
    - MINOR: dict: Add dictionary new data structure.
    - MINOR: peers: Add a LRU cache implementation for dictionaries.
    - MINOR: stick-table: Add "server_name" new data type.
    - MINOR: cfgparse: Space allocation for "server_name" stick-table data type.
    - MINOR: proxy: Add a "server by name" tree to proxy.
    - MINOR: server: Add a dictionary for server names.
    - MINOR: stream: Stickiness server lookup by name.
    - MINOR: peers: Make peers protocol support new "server_name" data type.
    - MINOR: stick-table: Make the CLI stick-table handler support dictionary entry data type.
    - REGTEST: Add a basic server by name stickiness reg test.
    - MINOR: peers: Add dictionary cache information to "show peers" CLI command.
    - MINOR: peers: Replace hard-coded for peer protocol 64-bits value encoding by macros.
    - MINOR: peers: Replace hard-coded values for peer protocol messaging by macros.
    - CLEANUP: ssl: remove unneeded defined(OPENSSL_IS_BORINGSSL)
    - BUILD: travis-ci improvements
    - MINOR: SSL: add client/server random sample fetches
    - BUG/MINOR: channel/htx: Don't alter channel during forward for empty HTX message
    - BUG/MINOR: contrib/prometheus-exporter: Add HTX data block in one time
    - BUG/MINOR: mux-h1: errflag must be set on H1S and not H1M during output processing
    - MEDIUM: mux-h1: refactor output processing
    - MINOR: mux-h1: Add the flag HAVE_O_CONN on h1s
    - MINOR: mux-h1: Add h1_eval_htx_hdrs_size() to estimate size of the HTX headers
    - MINOR: mux-h1: Don't count the EOM in the estimated size of headers
    - MEDIUM: cache/htx: Always store info about HTX blocks in the cache
    - MEDIUM: htx: Add the parsing of trailers of chunked messages
    - MINOR: htx: Don't use end-of-data blocks anymore
    - BUG/MINOR: mux-h1: Don't send more data than expected
    - BUG/MINOR: flt_trace/htx: Only apply the random forwarding on the message body.
    - BUG/MINOR: peers: Wrong "server_name" decoding.
    - BUG/MEDIUM: servers: Don't attempt to destroy idle connections if disabled.
    - MEDIUM: checks: Make sure we unsubscribe before calling cs_destroy().
    - MEDIUM: connections: Wake the upper layer even if sending/receiving is disabled.
    - MEDIUM: ssl: Handle subscribe by itself.
    - MINOR: ssl: Make ssl_sock_handshake() static.
    - MINOR: connections: Add a new xprt method, remove_xprt.
    - MINOR: connections: Add a new xprt method, add_xprt().
    - MEDIUM: connections: Introduce a handshake pseudo-XPRT.
    - MEDIUM: connections: Remove CONN_FL_SOCK*
    - BUG/MEDIUM: ssl: Don't forget to initialize ctx->send_recv and ctx->recv_wait.
    - BUG/MINOR: peers: Wrong server name parsing.
    - MINOR: server: really increase the pool-purge-delay default to 5 seconds
    - BUG/MINOR: stream: don't emit a send-name-header in conn error or disconnect states
    - MINOR: stream-int: use bit fields to match multiple stream-int states at once
    - MEDIUM: stream-int: remove dangerous interval checks for stream-int states
    - MEDIUM: stream-int: introduce a new state SI_ST_RDY
    - MAJOR: stream-int: switch from SI_ST_CON to SI_ST_RDY on I/O
    - MEDIUM: stream-int: make idle-conns switch to ST_RDY
    - MEDIUM: stream: re-arrange the connection setup status reporting
    - MINOR: stream-int: split si_update() into si_update_rx() and si_update_tx()
    - MINOR: stream-int: make si_sync_send() from the send code of si_update_both()
    - MEDIUM: stream: rearrange the events to remove the loop
    - MEDIUM: stream: only loop on flags relevant to the analysers
    - MEDIUM: stream: don't abusively loop back on changes on CF_SHUT*_NOW
    - BUILD: stream-int: avoid a build warning in dev mode in si_state_bit()
    - BUILD: peers: fix a build warning about an incorrect intiialization
    - BUG/MINOR: time: make sure only one thread sets global_now at boot
    - BUG/MEDIUM: tcp: Make sure we keep the polling consistent in tcp_probe_connect.
2019-06-07 06:12:59 +02:00
Willy Tarreau
4f283fa604 MEDIUM: stream-int: introduce a new state SI_ST_RDY
The main reason for all the trouble we're facing with stream interface
error or timeout reports during the connection phase is that we currently
can't make the difference between a connection attempt and a validated
connection attempt. It is problematic because we tend to switch early
to SI_ST_EST but can't always do what we want in this state since it's
supposed to be set when we don't need to visit sess_establish() again.

This patch introduces a new state betwen SI_ST_CON and SI_ST_EST, which
is SI_ST_RDY. It indicates that we've verified that the connection is
ready. It's a transient state, like SI_ST_DIS, that cannot persist when
leaving process_stream(). For now it is not set, only verified in various
tests where SI_ST_CON was used or SI_ST_EST depending on the cases.

The stream-int state diagram was minimally updated to reflect the new
state, though it is largely obsolete and would need to be seriously
updated.
2019-06-06 16:36:19 +02:00
Willy Tarreau
a257a9b015 [RELEASE] Released version 2.0-dev3
Released version 2.0-dev3 with the following main changes :
    - BUG/MINOR: peers: Really close the sessions with no heartbeat.
    - CLEANUP: peers: remove useless annoying tabulations.
    - CLEANUP: peers: replace timeout constants by macros.
    - REGTEST: Enable again reg tests with HEAD HTTP method usage.
    - DOC: The option httplog is no longer valid in a backend.
    - DOC: peers: Peers protocol documentation update.
    - REGTEST: remove unexpected "nbthread" statement from Lua test cases
    - BUILD: Makefile: remove 11-years old workarounds for deprecated options
    - BUILD: remove 10-years old error message for obsolete option USE_TCPSPLICE
    - BUILD: Makefile: remove outdated support for dlmalloc
    - BUILD: Makefile: consider a variable's origin and not its value for the options list
    - BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
    - BUILD: Makefile: shorten default settings declaration
    - BUILD: Makefile: clean up the target declarations
    - BUILD: report the whole feature set with their status in haproxy -vv
    - BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
    - REGTEST: script: make the script use the new features list
    - REGTEST: script: remove platform-specific assigments of OPTIONS
    - BUG/MINOR: peers: Missing initializations after peer session shutdown.
    - BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes
    - BUILD/MINOR: listener: Silent a few signedness warnings.
    - BUG/MINOR: mux-h1: Only skip invalid C-L headers on output
    - BUG/MEDIUM: mworker: don't free the wrong child when not found
    - BUG/MEDIUM: checks: Don't bother subscribing if we have a connection error.
    - BUG/MAJOR: checks: segfault during tcpcheck_main
    - BUILD: makefile: work around an old bug in GNU make-3.80
    - BUILD: makefile: work around another bug in make 3.80
    - BUILD: http: properly mark some struct as extern
    - BUILD: chunk: properly declare pool_head_trash as extern
    - BUILD: cache: avoid a build warning with some compilers/linkers
    - MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
    - MINOR: tools: add an unsetenv() implementation
    - BUILD: re-implement an initcall variant without using executable sections
    - BUILD: use inttypes.h instead of stdint.h
    - BUILD: connection: fix naming of ip_v field
    - BUILD: makefile: fix build of IPv6 header on aix51
    - BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
    - BUILD: define unsetenv on AIX 5.1
    - BUILD: Makefile: disable shared cache on AIX 5.1
    - MINOR: ssl: Add aes_gcm_dec converter
    - REORG: mworker: move serializing functions to mworker.c
    - REORG: mworker: move signals functions to mworker.c
    - REORG: mworker: move IPC functions to mworker.c
    - REORG: mworker: move signal handlers and related functions
    - REORG: mworker: move mworker_cleanlisteners to mworker.c
    - MINOR: mworker: calloc mworker_proc structures
    - MINOR: mworker: don't use children variable anymore
    - MINOR: cli: export cli_parse_default() definition in cli.h
    - REORG: mworker/cli: move CLI functions to mworker.c
    - MEDIUM: mworker-prog: implement program for master-worker
    - MINOR: mworker/cli: show programs in 'show proc'
    - BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
    - MINOR: cli: start addresses by a prefix in 'show cli sockets'
    - MINOR: cli: export HAPROXY_CLI environment variable
    - BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing error
    - BUG/MINOR: proto_htx: Reset to_forward value when a message is set to DONE
    - REGTEST: http-capture/h00000: Relax a regex matching the log message
    - REGTEST: http-messaging/h00000: Fix the test when the HTX is enabled
    - REGTEST: http-rules/h00003: Use a different client for requests expecting a 301
    - REGTEST: log/b00000: Be sure the client always hits its timeout
    - REGTEST: lua/b00003: Relax the regex matching the log message
    - REGTEST: lua/b00003: Specify the HAProxy pid when the command ss is executed
    - BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on release.
    - BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we subscribed.
    - BUG/MEDIUM: htx: fix random premature abort of data transfers
    - BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
    - BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
    - BUG/MEDIUM: stream: Don't clear the stream_interface flags in si_update_both.
    - MINOR: initcall: Don't forget to define the __start/stop_init_##stg symbols.
    - MINOR: threads: Implement thread_cpus_enabled() for FreeBSD.
    - BUG/MEDIUM: pattern: assign pattern IDs after checking the config validity
    - MINOR: skip get_gmtime where tm is unused
    - MINOR: ssl: Activate aes_gcm_dec converter for BoringSSL
    - BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected state.
    - BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not connected.
    - BUG/MEDIUM: task/threads: address a fairness issue between local and global tasks
    - BUG/MINOR: tasks: make sure the first task to be queued keeps its nice value
    - BUG/MINOR: listener: renice the accept ring processing task
    - MINOR: cli/listener: report the number of accepts on "show activity"
    - MINOR: cli/activity: report the accept queue sizes in "show activity"
    - BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to the stream
    - BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented messages
    - BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is encoded
    - BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads wrap
    - BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a data block
    - MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
    - MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
    - BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
    - MINOR: mux-h1: Simplify handling of 1xx responses
    - MINOR: stats/htx: Don't add "Connection: close" header anymore in stats responses
    - MEDIUM: h1: Add an option to sanitize connection headers during parsing
    - MEDIUM: mux-h1: Simplify the connection mode management by sanitizing headers
    - MINOR: mux-h1: Don't release the conn_stream anymore when h1s is destroyed
    - BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown read/write
    - MINOR: mux-h2: Add a mux_ops dedicated to the HTX mode
    - MINOR: muxes: Add a flag to specify a multiplexer uses the HTX
    - MINOR: stream: Set a flag when the stream uses the HTX
    - MINOR: http: update the macro IS_HTX_STRM() to check the stream flag SF_HTX
    - MINOR: http_fetch/htx: Use stream flags instead of px mode in smp_prefetch_htx
    - MINOR: filters/htx: Use stream flags instead of px mode to instanciate a filter
    - MINOR: muxes: Rely on conn_is_back() during init to handle front/back conn
    - MEDIUM: muxes: Add an optional input buffer during mux initialization
    - MINOR: muxes: Pass the context of the mux to destroy() instead of the connection
    - MEDIUM: muxes: Be prepared to don't own connection during the release
    - MEDIUM: connection: Add conn_upgrade_mux_fe() to handle mux upgrades
    - MEDIUM: htx: Allow the option http-use-htx to be used on TCP proxies too
    - MAJOR: proxy/htx: Handle mux upgrades from TCP to HTTP in HTX mode
    - MAJOR: muxes/htx: Handle inplicit upgrades from h1 to h2
    - MAJOR: htx: Enable the HTX mode by default for all proxies
    - REGTEST: Use HTX by default and add '--no-htx' option to disable it
    - BUG/MEDIUM: muxes: Don't dereference mux context if null in release functions
    - CLEANUP: task: do not export rq_next anymore
    - MEDIUM: tasks: improve fairness between the local and global queues
    - MEDIUM: tasks: only base the nice offset on the run queue depth
    - MINOR: tasks: restore the lower latency scheduling when niced tasks are present
    - BUG/MEDIUM: map: Fix memory leak in the map converter
    - BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation
    - BUILD: task/thread: fix single-threaded build of task.c
    - BUILD: cli/threads: fix build in single-threaded mode
    - BUG/MEDIUM: muxes: Make sure we unsubcribed when destroying mux ctx.
    - BUG/MEDIUM: h2: Make sure we're not already in the send_list in h2_subscribe().
    - BUG/MEDIUM: h2: Revamp the way send subscriptions works.
    - MINOR: connections: Remove the SUB_CALL_UNSUBSCRIBE flag.
    - BUG/MEDIUM: Threads: Only use the gcc >= 4.7 builtins when using gcc >= 4.7.
    - BUILD: address a few cases of "static <type> inline foo()"
    - BUILD: do not specify "const" on functions returning structs or scalars
    - BUILD: htx: fix a used uninitialized warning on is_cookie2
    - MINOR: peers: Add a new command to the CLI for peers.
    - DOC: update for "show peers" CLI command.
    - BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
    - MEDIUM: mworker: store the leaving state of a process
    - MEDIUM: mworker-prog: implements 'option start-on-reload'
    - CLEANUP: mworker: remove the type field in mworker_proc
    - MEDIUM: mworker/cli: export the HAPROXY_MASTER_CLI variable
    - MINOR: cli: don't add a semicolon at the end of HAPROXY_CLI
    - MINOR: mworker: export HAPROXY_MWORKER=1 when running in mworker mode
    - MINOR: init: add a "set-dumpable" global directive to enable core dumps
    - BUG/MINOR: listener/mq: correctly scan all bound threads under low load
    - BUG/MINOR: mworker: mworker_kill should apply on every children
    - BUG/MINOR: mworker: don't exit with an ambiguous value
    - BUG/MINOR: mworker: ensure that we still quits with SIGINT
    - REGTESTS: exclude tests that require ssl, pcre if no such feature is enabled
    - BUG/MINOR: mux-h1: Process input even if the input buffer is empty
    - BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side has done
    - BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf is empty
    - BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
    - MINOR: mux-h1: Handle read0 during TCP splicing
    - BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
    - BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
    - BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
    - BUG/MINOR: http_fetch/htx: Use HTX versions if the proxy enables the HTX mode
    - BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to the rq.
    - BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the rq_lock.
    - MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
    - MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
    - MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet list.
    - BUG/MEDIUM: applets: Don't use task_in_rq().
    - BUG/MAJOR: task: make sure never to delete a queued task
    - MINOR: task/thread: factor out a wake-up condition
    - CLEANUP: task: remain consistent when using the task's handler
    - MEDIUM: tasks: Merge task_delete() and task_free() into task_destroy().
    - MEDIUM: tasks: Don't account a destroyed task as a runned task.
    - BUG/MINOR: contrib/prometheus-exporter: Fix a typo in the run-queue metric type
    - MINOR: contrib/prometheus-exporter: Remove usless rate metrics
    - MINOR: contrib/prometheus-exporter: Rename some metrics to be more usable
    - MINOR: contrib/prometheus-exporter: Follow best practices about metrics type
    - BUG/MINOR: mworker: disable busy polling in the master process
    - MEDIUM: tasks: Use __ha_barrier_store after modifying global_tasks_mask.
    - MEDIUM: ssl: Give ssl_sock its own context.
    - MEDIUM: connections: Move some fields from struct connection to ssl_sock_ctx.
    - MEDIUM: ssl: provide its own subscribe/unsubscribe function.
    - MEDIUM: connections: Provide a xprt_ctx for each xprt method.
    - MEDIUM: ssl: provide our own BIO.
    - BUILD/medium: ssl: Fix build with OpenSSL < 1.1.0
    - MINOR: peers: adds counters on show peers about tasks calls.
    - MEDIUM: enable travis-ci builds
    - MINOR: fd: Add a counter of used fds.
    - MEDIUM: connections: Add a way to control the number of idling connections.
    - BUG/MEDIUM: maps: only try to parse the default value when it's present
    - BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
    - REGTEST: Missing REQUIRE_VERSION declarations.
    - MINOR: proto_tcp: tcp-request content: enable set-dst and set-dst-var
    - BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
    - BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP rules
    - BUG/MEDIUM: stream: Don't request a server connection if a shutw was scheduled
    - BUG/MINOR: 51d: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
    - BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
    - MINOR: gcc: Fix a silly gcc warning in connect_server()
    - MINOR: ssl/cli: async fd io-handlers printable on show fd
    - Revert "CLEANUP: wurfl: remove dead, broken and unmaintained code"
    - BUILD: add USE_WURFL to the list of known build options
    - MINOR: wurfl: indicate in haproxy -vv the wurfl version in use
    - BUILD: wurfl: build fix for 1.9/2.0 code base
    - CLEANUP: wurfl: removed deprecated methods
    - DOC: wurfl: added point of contact in MAINTAINERS file
    - MINOR: wurfl: enabled multithreading mode
    - MINOR: contrib: dummy wurfl library
    - MINOR: dns: dns_requester structures are now in a memory pool
    - MINOR: dns: move callback affection in dns_link_resolution()
    - MINOR: obj_type: new object type for struct stream
    - MINOR: action: new '(http-request|tcp-request content) do-resolve' action
    - MINOR: log: Extract some code to send syslog messages.
    - REGTEST: replace LEVEL option by a more human readable one.
    - REGTEST: rename the reg test files.
    - REGTEST: adapt some reg tests after renaming.
    - REGTEST: make the "run-regtests" script search for tests in reg-tests by default
    - BUG/MAJOR: stream: Missing DNS context initializations.
    - BUG/MEDIUM: stream: Fix the way early aborts on the client side are handled
    - BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet handler
    - BUG/MEDIUM: ssl: Return -1 on recv/send if we got EAGAIN.
    - BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB
    - BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies only
    - BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during startup
    - CLEANUP: task: report calls as unsigned in show sess
    - MINOR: tasks/activity: report the context switch and task wakeup rates
    - MINOR: stream: measure and report a stream's call rate in "show sess"
    - MINOR: applet: measure and report an appctx's call rate in "show sess"
    - BUILD: extend Travis CI config to support more platforms
    - REGTEST: exclude osx and generic targets for 40be_2srv_odd_health_checks
    - REGTEST: relax the IPv6 address format checks in converters_ipmask_concat_strcmp_field_word
    - REGTEST: exclude OSX and generic targets from abns_socket.vtc
    - BUILD: travis: remove the "allow_failures" entry
    - BUG/MINOR: activity: always initialize the profiling variable
    - MINOR: activity: make the profiling status per thread and not global
    - MINOR: activity: enable automatic profiling turn on/off
    - CLEANUP: standard: use proper const to addr_to_str() and port_to_str()
    - BUG/MINOR: proto_http: properly reset the stream's call rate on keep-alive
    - MINOR: connection: make the debugging helper functions safer
    - MINOR: stream/debug: make a stream dump and crash function
    - MEDIUM: appctx/debug: force a crash if an appctx spins over itself forever
    - MEDIUM: stream/debug: force a crash if a stream spins over itself forever
    - MEDIUM: streams: measure processing time and abort when detecting bugs
    - BUILD/MEDIUM: contrib: Dummy DeviceAtlas API.
    - MEDIUM: da: HTX mode support.
    - BUG/MEDIUM: mux-h2: properly deal with too large headers frames
    - BUG/MINOR: http: Call stream_inc_be_http_req_ctr() only one time per request
    - BUG/MEDIUM: spoe: arg len encoded in previous frag frame but len changed
    - MINOR: spoe: Use the sample context to pass frag_ctx info during encoding
    - DOC: contrib/modsecurity: Typos and fix the reject example
    - BUG/MEDIUM: contrib/modsecurity: If host header is NULL, don't try to strdup it
    - MINOR: log: Add "sample" new keyword to "log" lines.
    - MINOR: log: Enable the log sampling and load-balancing feature.
    - DOC: log: Document the sampling and load-balancing logging feature.
    - REGTEST: Add a new reg test for log load-balancing feature.
    - BUG/MAJOR: map/acl: real fix segfault during show map/acl on CLI
    - REGTEST: Make this reg test be Linux specific.
    - CLEANUP: task: move the task_per_thread definition to task.h
    - MINOR: activity: report context switch counts instead of rates
    - MINOR: threads: Implement HA_ATOMIC_LOAD().
    - BUG/MEDIUM: port_range: Make the ring buffer lock-free.
    - BUG/MEDIUM: listener: Fix how unlimited number of consecutive accepts is handled
    - MINOR: config: Test validity of tune.maxaccept during the config parsing
    - CLEANUP: config: Don't alter listener->maxaccept when nbproc is set to 1
    - BUG/MEDIUM: servers: fix typo "src" instead of "srv"
    - BUG/MEDIUM: ssl: Don't pretend we can retry a recv/send if we got a shutr/w.
    - BUG/MINOR: haproxy: fix rule->file memory leak
    - BUG/MINOR: log: properly free memory on logformat parse error and deinit()
    - BUG/MINOR: checks: free memory allocated for tasklets
    - BUG/MEDIUM: pattern: fix memory leak in regex pattern functions
    - BUG/MEDIUM: channels: Don't forget to reset output in channel_erase().
    - BUG/MEDIUM: connections: Make sure we remove CO_FL_SESS_IDLE on disown.
    - MINOR: threads: flatten the per-thread cpu-map
    - MINOR: init/threads: remove the useless tids[] array
    - MINOR: init/threads: make the threads array global
    - BUG/MEDIUM: ssl: Use the early_data API the right way.
    - BUG/MEDIUM: streams: Don't add CF_WRITE_ERROR if early data were rejected.
    - MEDIUM: streams: Add the ability to retry a request on L7 failure.
    - MEDIUM: streams: Add a way to replay failed 0rtt requests.
    - MEDIUM: streams: Add a new keyword for retry-on, "junk-response"
    - BUG/MINOR: stream: also increment the retry stats counter on L7 retries
    - BUG/MEDIUM: checks: make sure the warmup task takes the server lock
    - BUG/MINOR: logs/threads: properly split the log area upon startup
    - BUILD: extend travis-ci matrix
    - CLEANUP: Remove appsession documentation
    - DOC: Fix typo in keyword matrix
    - BUILD: remove "build_libressl" duplicate declaration
    - BUILD: travis-ci: get back to osx without openssl support
    - BUILD: enable several LibreSSL hacks, including
    - BUILD: temporarily mark LibreSSL builds as allowed to fail
    - BUILD: travis: TMPDIR replacement.
    - BUG/MEDIUM: ssl: Don't attempt to use early data with libressl.
    - MINOR: doc: Document allow-0rtt on the server line.
    - MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
    - MEDIUM: proto: Change the prototype of the connect() method.
    - MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
    - MINOR: config: Extract the code of "stick-table" line parsing.
    - BUILD/MINOR: stick-table: Compilation fix.
    - MEDIUM: stick-table: Stop handling stick-tables as proxies.
    - MINOR: stick-tables: Add peers process binding computing.
    - MINOR: stick-table: Add prefixes to stick-table names.
    - MINOR: peers: Do not emit global stick-table names.
    - DOC: Update for "table" lines in "peers" section.
    - REGTEST: Add reg tests for "table" lines in "peers" sections.
    - MEDIUM: regex: modify regex_comp() to atomically allocate/free the my_regex struct
    - REGTEST: make the tls_health_checks test much faster
    - REGTEST: make the "table in peers" test require v2.0
    - BUG/MINOR: mux-h2: rely on trailers output not input to turn them to empty data
    - BUG/MEDIUM: h2/htx: always fail on too large trailers
    - MEDIUM: mux-h2: discard contents that are to be sent after a shutdown
    - BUG/MEDIUM: mux-h2/htx: never wait for EOM when processing trailers
    - BUG/MEDIUM: h2/htx: never leave a trailers block alone with no EOM block
    - REGTEST: Flag some slow reg tests.
    - REGTEST: Reg tests file renaming.
    - REGTEST: Wrong renaming for one reg test.
    - REGTEST: Wrong assumption in IP:port logging test.
    - BUG/MINOR: mworker/ssl: close OpenSSL FDs on reload
    - MINOR: systemd: Use the variables from /etc/default/haproxy
    - MINOR: systemd: Make use of master socket in systemd unit
    - MINOR: systemd: support /etc/sysconfig/ for redhat based distrib
    - BUG/MEDIUM: stick-table: fix regression caused by a change in proxy struct
    - BUG/MEDIUM: tasks: fix possible segfault on task_destroy()
    - CLEANUP: task: remove unneeded tests before task_destroy()
    - MINOR: mworker: support a configurable maximum number of reloads
    - BUG/MINOR: mux-h2: fix the condition to close a cs-less h2s on the backend
    - BUG/MEDIUM: spoe: Be sure the sample is found before setting its context
    - BUG/MINOR: mux-h1: Fix the parsing of trailers
    - BUG/MINOR: htx: Never transfer more than expected in htx_xfer_blks()
    - MINOR: htx: Split on DATA blocks only when blocks are moved to an HTX message
    - MINOR: htx: Don't try to append a trailer block with the previous one
    - MINOR: htx: Remove support for unused OOB HTX blocks
    - BUILD: travis-ci bugfixes and improvements
    - BUG/MEDIUM: servers: Don't use the same srv flag for cookie-set and TFO.
    - BUG/MEDIUM: h2: Make sure we set send_list to NULL in h2_detach().
    - BUILD: ssl: fix again a libressl build failure after the openssl FD leak fix
    - CLEANUP: ssl-sock: use HA_OPENSSL_VERSION_NUMBER instead of OPENSSL_VERSION_NUMBER
    - BUILD: ssl: make libressl use its own version numbers
    - CLEANUP: ssl: remove 57 occurrences of useless tests on LIBRESSL_VERSION_NUMBER
    - MINOR: ssl: enable aes_gcm_dec on LibreSSL
    - BUILD: ssl: fix libressl build again after aes-gcm-enc
    - REORG: ssl: move openssl-compat from proto to common
    - REORG: ssl: move some OpenSSL defines from ssl_sock to openssl-compat
    - CLEANUP: ssl: never include openssl/*.h outside of openssl-compat.h anymore
    - CLEANUP: ssl: make inclusion of openssl headers safe
    - BUILD: add BoringSSL to travis-ci build matrix
    - BUILD: threads: Add __ha_cas_dw fallback for single threaded builds
    - BUG/MINOR: stream: Attach the read side on the response as soon as possible
    - BUG/MEDIUM: http: Use pointer to the begining of input to parse message headers
    - BUG/MEDIUM: h2: Don't check send_wait to know if we're in the send_list.
    - BUG/MEDIUM: streams: Make sur SI_FL_L7_RETRY is set before attempting a retry.
    - MEDIUM: streams: Add a new http action, disable-l7-retry.
    - MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
    - BUG/MINOR: vars: Fix memory leak in vars_check_arg
    - BUILD: travis-ci: make TMPDIR global variable in travis-ci
    - CLEANUP: ssl: move the SSL_OP_* and SSL_MODE_* definitions to openssl-compat
    - CLEANUP: ssl: remove ifdef around SSL_CTX_get_extra_chain_certs()
    - CLEANUP: ssl: move all BIO_* definitions to openssl-compat
    - BUILD: threads: fix again the __ha_cas_dw() definition
    - BUG/MAJOR: mux-h2: do not add a stream twice to the send list
    - Revert "BUG/MINOR: vars: Fix memory leak in vars_check_arg"
    - BUG/MINOR: peers: Fix memory leak in cfg_parse_peers
    - BUG/MINOR: htx: make sure to always initialize the HTTP method when parsing a buffer
    - REGTEST: fix tls_health_checks random failures on MacOS in Travis-CI
    - MINOR: spoe: Set the argument chunk size to 0 when SPOE variables are checked
    - BUG/MINOR: vars: Fix memory leak in vars_check_arg
    - BUG/MAJOR: ssl: segfault upon an heartbeat request
    - MINOR: spoa-server: Clone the v1.7 spoa-example project
    - MINOR: spoa-server: move some definition from spoa_server.c to spoa_server.h
    - MINOR: spoa-server: Externalise debug functions
    - MINOR: spoe-server: rename "worker" functions
    - MINOR: spoa-server: Replace the thread init system by processes
    - MINOR: spoa-server: With debug mode, start only one process
    - MINOR: spoa-server: Allow registering external processes
    - MINOR: spoa-server: Allow registering message processors
    - MINOR: spoa-server: Load files
    - MINOR: spoa-server: Prepare responses
    - MINOR: spoa-server: Execute registered callbacks
    - MINOR: spoa-server: Add Lua processing
    - MINOR: spoa-server: Add python
    - MINOR/DOC: spoe-server: Add documentation
    - BUG/MEDIUM: connections: Don't forget to set xprt_ctx to NULL on close.
    - MINOR: lists: add LIST_ADDED() to check if an element belongs to a list
    - CLEANUP: mux-h2: use LIST_ADDED() instead of LIST_ISEMPTY() where relevant
    - MINOR: mux-h2: add two H2S flags to report the need for shutr/shutw
    - CLEANUP: mux-h2: simply use h2s->flags instead of ret in h2_deferred_shut()
    - CLEANUP: connection: remove the handle field from the wait_event struct
    - BUG/MINOR: log: Wrong log format initialization.
    - BUG/MINOR: mux-h2: make the do_shut{r,w} functions more robust against retries
    - BUG/MINOR: mworker: use after free when the PID not assigned
    - MINOR: mux-h2: remove useless test on stream ID vs last in wake function
    - MINOR: mux-h2: make h2_wake_some_streams() not depend on the CS flags
    - MINOR: mux-h2: make h2s_wake_one_stream() the only function to deal with CS
    - MINOR: mux-h2: make h2s_wake_one_stream() not depend on temporary CS flags
    - BUG/MINOR: mux-h2: make sure to honor KILL_CONN in do_shut{r,w}
    - CLEANUP: mux-h2: don't test for impossible CS_FL_REOS conditions
    - MINOR: mux-h2: add macros to check multiple stream states at once
    - MINOR: mux-h2: stop relying on CS_FL_REOS
    - BUG/MEDIUM: mux-h2: Set EOI on the conn_stream during h2_rcv_buf()
    - BUILD: debug: make gcc not complain on the ABORT_NOW() macro
    - MINOR: debug: add a new BUG_ON macro
    - MINOR: h2: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - MINOR: h1: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - MINOR: connections: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - BUILD: ist: turn the lower/upper case tables to literal on obsolete linkers
2019-05-15 16:51:48 +02:00
Olivier Houchard
a254a37ad7 MEDIUM: streams: Add the ability to retry a request on L7 failure.
When running in HTX mode, if we sent the request, but failed to get the
answer, either because the server just closed its socket, we hit a server
timeout, or we get a 404, 408, 425, 500, 501, 502, 503 or 504 error,
attempt to retry the request, exactly as if we just failed to connect to
the server.

To do so, add a new backend keyword, "retry-on".

It accepts a list of keywords, which can be "none" (never retry),
"conn-failure" (we failed to connect, or to do the SSL handshake),
"empty-response" (the server closed the connection without answering),
"response-timeout" (we timed out while waiting for the server response),
or "404", "408", "425", "500", "501", "502", "503" and "504".

The default is "conn-failure".
2019-05-04 10:19:56 +02:00