Commit Graph

1278 Commits

Author SHA1 Message Date
Tim Duesterhus
6edab865f6 BUG/MEDIUM: lua: Fix IPv6 with separate port support for Socket.connect
The `socket.tcp.connect` method of Lua requires at least two parameters:
The host and the port. The `Socket.connect` method of haproxy requires
only one when a host with a combined port is provided. This stems from
the fact that `str2sa_range` is used internally in `hlua_socket_connect`.
This very fact unfortunately causes a diversion in the behaviour of
Lua's socket class and haproxy's for IPv6 addresses:

  sock:connect("::1", "80")

works fine with Lua, but fails with:

  connect: cannot parse destination address '::1'

in haproxy, because `str2sa_range` parses the trailing `:1` as the port.

This patch forcefully adds a `:` to the end of the address iff a port
number greater than `0` is given as the second parameter.

Technically this breaks backwards compatibility, because the docs state:

> The syntax "127.0.0.1:1234" is valid. in this case, the
> parameter *port* is ignored.

But: The connect() call can only succeed if the second parameter is left
out (which causes no breakage) or if the second parameter is an integer
or a numeric string.

It seems unlikely that someone would provide an address with a port number
and would also provide a second parameter containing a number other than
zero. Thus I feel this breakage is warranted to fix the mismatch between
haproxy's socket class and Lua's one.

This commit should be backported to haproxy 1.8 only, because of the
possible breakage of existing Lua scripts.
2018-01-09 15:22:55 +01:00
Davor Ocelic
e9ed281e9f DOC/MINOR: configuration: typo, formatting fixes
- Add simple typo and formatting fixes
- Eliminate a couple > 80 column lines

Changes do not affect technical content and can be backported.
2017-12-27 19:03:32 +01:00
Willy Tarreau
c55ddce65c MINOR: http: update the list of cacheable status codes as per RFC7231
Since RFC2616, the following codes were added to the list of codes
cacheable by default : 204, 404, 405, 414, 501. For now this it only
checked by the checkcache option to detect cacheable cookies.
2017-12-22 14:43:26 +01:00
Willy Tarreau
24ea0bcb1d MINOR: http: adjust the list of supposedly cacheable methods
We used to have a rule inherited from RFC2616 saying that the POST
method was the only uncacheable one, but things have changed since
and RFC7231+7234 made it clear that in fact only GET/HEAD/OPTIONS/TRACE
are cacheable. Currently this rule is only used to detect cacheable
cookies.
2017-12-22 14:43:26 +01:00
Etienne Carriere
aec8989e53 MINOR: spoe: add force-set-var option in spoe-agent configuration
For security reasons, the spoe filter was only able to change values of
existing variables. In specific cases (ex : with LUA code), the name of
variables are unknown at the configuration parsing phase.
The force-set-var option can be enabled to register all variables.
2017-12-20 08:55:18 +01:00
Bertrand Jacquin
72fa1ec24e MEDIUM: netscaler: add support for standard NetScaler CIP protocol
It looks like two version of the protocol exist as reported by
Andreas Mahnke. This patch add support for both legacy and standard CIP
protocol according to NetScaler specifications.
2017-12-20 07:04:07 +01:00
Davor Ocelic
4094ce1a23 DOC/MINOR: intro: typo, wording, formatting fixes
- Fix a couple typos
- Introduce a couple simple rewordings
- Eliminate > 80 column lines

Changes do not affect technical content and can be backported.
2017-12-20 07:01:36 +01:00
Willy Tarreau
60a2ee7945 MINOR: sample: rename the "len" converter to "length"
This converter was recently introduced by commit ed0d24e ("MINOR:
sample: add len converter").

As found by Cyril, it causes an issue in "http-request capture"
statements. The non-obvious problem is that an old syntax for sample
expressions and converters used to support a series of words, each
representing a converter. This used to be how the "stick" directives
were created initially. By having a converter called "len", a
statement such as "http-request capture foo len 10" considers "len"
as a converter and not as the capture length.

This obsolete syntax needs to be changed in 1.9 but it's too late
for other versions. It's worth noting that the same problem can
happen if converters are registered on the fly using Lua. Other
language keywords that currently have to be avoided in converters
include "id", "table", "if", "unless".
2017-12-15 07:13:48 +01:00
Etienne Carriere
ed0d24ebed MINOR: sample: add len converter
Add len converter that returns the length of a string
2017-12-14 14:36:10 +01:00
Willy Tarreau
721d8e0286 MINOR: config: report when "monitor fail" rules are misplaced
"monitor-uri" may rely on "monitor fail" rules, which are processed
very early, immediately after the HTTP request is parsed and before
any http rulesets. It's not reported by the config parser when this
ruleset is misplaces, causing some configurations not to work like
users would expect. Let's just add the warning for a misplaced rule.
2017-12-01 18:25:08 +01:00
Cyril Bonté
7b888f1235 DOC: cache: update sections and fix some typos
Cache sections were not defined as the others, preventing them to be
correctly parsed by the HTML converter. Also, the "Cache" subsections
where not added to the summary.

This patch should be backported to the 1.8 branch.
2017-11-26 23:19:35 +01:00
Willy Tarreau
b306650c2a [RELEASE] Released version 1.9-dev0
Released version 1.9-dev0 with the following main changes :
    - BUG/MEDIUM: stream: don't automatically forward connect nor close
    - BUG/MAJOR: stream: ensure analysers are always called upon close
    - BUG/MINOR: stream-int: don't try to read again when CF_READ_DONTWAIT is set
    - MEDIUM: mworker: Add systemd `Type=notify` support
    - BUG/MEDIUM: cache: free callback to remove from tree
    - CLEANUP: cache: remove unused struct
    - MEDIUM: cache: enable the HTTP analysers
    - CLEANUP: cache: remove wrong comment
    - MINOR: threads/atomic: rename local variables in macros to avoid conflicts
    - MINOR: threads/plock: rename local variables in macros to avoid conflicts
    - MINOR: threads/atomic: implement pl_mb() in asm on x86
    - MINOR: threads/atomic: implement pl_bts() on non-x86
    - MINOR: threads/build: atomic: replace the few inlines with macros
    - BUILD: threads/plock: fix a build issue on Clang without optimization
    - BUILD: ebtree: don't redefine types u32/s32 in scope-aware trees
    - BUILD: compiler: add a new type modifier __maybe_unused
    - BUILD: h2: mark some inlined functions "unused"
    - BUILD: server: check->desc always exists
    - BUG/MEDIUM: h2: properly report connection errors in headers and data handlers
    - MEDIUM: h2: add a function to emit an HTTP/1 request from a headers list
    - MEDIUM: h2: change hpack_decode_headers() to only provide a list of headers
    - BUG/MEDIUM: h2: always reassemble the Cookie request header field
    - BUG/MINOR: systemd: ignore daemon mode
    - CONTRIB: spoa_example: allow to compile outside HAProxy.
    - CONTRIB: spoa_example: remove bref, wordlist, cond_wordlist
    - CONTRIB: spoa_example: remove last dependencies on type "sample"
    - CONTRIB: spoa_example: remove SPOE enums that are useless for clients
    - CLEANUP: cache: reorder includes
    - MEDIUM: shctx: use unsigned int for len and block_count
    - MEDIUM: cache: "show cache" on the cli
    - BUG/MEDIUM: cache: use key=0 as a condition for freeing
    - BUG/MEDIUM: cache: refcount forbids to free the objects
    - BUG/MEDIUM: cache fix cli_kws structure
    - BUG/MEDIUM: deinit: correctly deinitialize the proxy and global listener tasks
    - BUG/MINOR: ssl: Always start the handshake if we can't send early data.
    - MINOR: ssl: Don't disable early data handling if we could not write.
    - MINOR: pools: prepare functions to override malloc/free in pools
    - MINOR: pools: implement DEBUG_UAF to detect use after free
    - BUG/MEDIUM: threads/time: fix time drift correction
    - BUG/MEDIUM: threads/time: maintain a common time reference between all threads
    - MINOR: sample: Add "thread" sample fetch
    - BUG/MINOR: Use crt_base instead of ca_base when crt is parsed on a server line
    - BUG/MINOR: stream: fix tv_request calculation for applets
    - BUG/MAJOR: h2: always remove a stream from the send list before freeing it
    - BUG/MAJOR: threads/task: dequeue expired tasks under the WQ lock
    - MINOR: ssl: Handle reading early data after writing better.
    - MINOR: mux: Make sure every string is woken up after the handshake.
    - MEDIUM: cache: store sha1 for hashing the cache key
    - MINOR: http: implement the "http-request reject" rule
    - MINOR: h2: send RST_STREAM before GOAWAY on reject
    - MEDIUM: h2: don't gracefully close the connection anymore on Connection: close
    - MINOR: h2: make use of client-fin timeout after GOAWAY
    - MEDIUM: config: ensure that tune.bufsize is at least 16384 when using HTTP/2
    - MINOR: ssl: Handle early data with BoringSSL
    - BUG/MEDIUM: stream: always release the stream-interface on abort
    - BUG/MEDIUM: cache: free ressources in chn_end_analyze
    - MINOR: cache: move the refcount decrease in the applet release
    - BUG/MINOR: listener: Allow multiple "process" options on "bind" lines
    - MINOR: config: Support a range to specify processes in "cpu-map" parameter
    - MINOR: config: Slightly change how parse_process_number works
    - MINOR: config: Export parse_process_number and use it wherever it's applicable
    - MINOR: standard: Add my_ffsl function to get the position of the bit set to one
    - MINOR: config: Add auto-increment feature for cpu-map
    - MINOR: config: Support partial ranges in cpu-map directive
    - MINOR:: config: Remove thread-map directive
    - MINOR: config: Add the threads support in cpu-map directive
    - MINOR: config: Add threads support for "process" option on "bind" lines
    - MEDIUM: listener: Bind listeners on a thread subset if specified
    - CLEANUP: debug: Use DPRINTF instead of fprintf into #ifdef DEBUG_FULL/#endif
    - CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning
    - MINOR/CLEANUP: proxy: rename "proxy" to "proxies_list"
    - CLEANUP: pools: rename all pool functions and pointers to remove this "2"
    - DOC: update the roadmap file with the latest changes merged in 1.8
    - DOC: fix mangled version in peers protocol documentation
    - DOC: add initial peers protovol v2.0 documentation.
    - DOC: mention William as maintainer of the cache and master-worker
    - DOC: add Christopher and Emeric as maintainers of the threads
    - MINOR: cache: replace a fprint() by an abort()
    - MEDIUM: cache: max-age configuration keyword
    - DOC: explain HTTP2 timeout behavior
    - DOC: cache: configuration and management
    - MAJOR: mworker: exits the master on failure
    - BUG/MINOR: threads: don't drop "extern" on the lock in include files
    - MINOR: task: keep a pointer to the currently running task
    - MINOR: task: align the rq and wq locks
    - MINOR: fd: cache-align fdtab and fdcache locks
    - MINOR: buffers: cache-align buffer_wq_lock
    - CLEANUP: server: reorder some fields in struct server to save 40 bytes
    - CLEANUP: proxy: slightly reorder the struct proxy to reduce holes
    - CLEANUP: checks: remove 16 bytes of holes in struct check
    - CLEANUP: cache: more efficiently pack the struct cache
    - CLEANUP: fd: place the lock at the beginning of struct fdtab
    - CLEANUP: pools: align pools on a cache line
    - DOC: config: add a few bits about how to configure HTTP/2
    - BUG/MAJOR: threads/queue: avoid recursive locking in pendconn_get_next_strm()
    - BUILD: Makefile: reorder object files by size
2017-11-26 19:50:17 +01:00
Willy Tarreau
0b78792bbe [RELEASE] Released version 1.8.0
Released version 1.8.0 with the following main changes :
    - BUG/MEDIUM: stream: don't automatically forward connect nor close
    - BUG/MAJOR: stream: ensure analysers are always called upon close
    - BUG/MINOR: stream-int: don't try to read again when CF_READ_DONTWAIT is set
    - MEDIUM: mworker: Add systemd `Type=notify` support
    - BUG/MEDIUM: cache: free callback to remove from tree
    - CLEANUP: cache: remove unused struct
    - MEDIUM: cache: enable the HTTP analysers
    - CLEANUP: cache: remove wrong comment
    - MINOR: threads/atomic: rename local variables in macros to avoid conflicts
    - MINOR: threads/plock: rename local variables in macros to avoid conflicts
    - MINOR: threads/atomic: implement pl_mb() in asm on x86
    - MINOR: threads/atomic: implement pl_bts() on non-x86
    - MINOR: threads/build: atomic: replace the few inlines with macros
    - BUILD: threads/plock: fix a build issue on Clang without optimization
    - BUILD: ebtree: don't redefine types u32/s32 in scope-aware trees
    - BUILD: compiler: add a new type modifier __maybe_unused
    - BUILD: h2: mark some inlined functions "unused"
    - BUILD: server: check->desc always exists
    - BUG/MEDIUM: h2: properly report connection errors in headers and data handlers
    - MEDIUM: h2: add a function to emit an HTTP/1 request from a headers list
    - MEDIUM: h2: change hpack_decode_headers() to only provide a list of headers
    - BUG/MEDIUM: h2: always reassemble the Cookie request header field
    - BUG/MINOR: systemd: ignore daemon mode
    - CONTRIB: spoa_example: allow to compile outside HAProxy.
    - CONTRIB: spoa_example: remove bref, wordlist, cond_wordlist
    - CONTRIB: spoa_example: remove last dependencies on type "sample"
    - CONTRIB: spoa_example: remove SPOE enums that are useless for clients
    - CLEANUP: cache: reorder includes
    - MEDIUM: shctx: use unsigned int for len and block_count
    - MEDIUM: cache: "show cache" on the cli
    - BUG/MEDIUM: cache: use key=0 as a condition for freeing
    - BUG/MEDIUM: cache: refcount forbids to free the objects
    - BUG/MEDIUM: cache fix cli_kws structure
    - BUG/MEDIUM: deinit: correctly deinitialize the proxy and global listener tasks
    - BUG/MINOR: ssl: Always start the handshake if we can't send early data.
    - MINOR: ssl: Don't disable early data handling if we could not write.
    - MINOR: pools: prepare functions to override malloc/free in pools
    - MINOR: pools: implement DEBUG_UAF to detect use after free
    - BUG/MEDIUM: threads/time: fix time drift correction
    - BUG/MEDIUM: threads/time: maintain a common time reference between all threads
    - MINOR: sample: Add "thread" sample fetch
    - BUG/MINOR: Use crt_base instead of ca_base when crt is parsed on a server line
    - BUG/MINOR: stream: fix tv_request calculation for applets
    - BUG/MAJOR: h2: always remove a stream from the send list before freeing it
    - BUG/MAJOR: threads/task: dequeue expired tasks under the WQ lock
    - MINOR: ssl: Handle reading early data after writing better.
    - MINOR: mux: Make sure every string is woken up after the handshake.
    - MEDIUM: cache: store sha1 for hashing the cache key
    - MINOR: http: implement the "http-request reject" rule
    - MINOR: h2: send RST_STREAM before GOAWAY on reject
    - MEDIUM: h2: don't gracefully close the connection anymore on Connection: close
    - MINOR: h2: make use of client-fin timeout after GOAWAY
    - MEDIUM: config: ensure that tune.bufsize is at least 16384 when using HTTP/2
    - MINOR: ssl: Handle early data with BoringSSL
    - BUG/MEDIUM: stream: always release the stream-interface on abort
    - BUG/MEDIUM: cache: free ressources in chn_end_analyze
    - MINOR: cache: move the refcount decrease in the applet release
    - BUG/MINOR: listener: Allow multiple "process" options on "bind" lines
    - MINOR: config: Support a range to specify processes in "cpu-map" parameter
    - MINOR: config: Slightly change how parse_process_number works
    - MINOR: config: Export parse_process_number and use it wherever it's applicable
    - MINOR: standard: Add my_ffsl function to get the position of the bit set to one
    - MINOR: config: Add auto-increment feature for cpu-map
    - MINOR: config: Support partial ranges in cpu-map directive
    - MINOR:: config: Remove thread-map directive
    - MINOR: config: Add the threads support in cpu-map directive
    - MINOR: config: Add threads support for "process" option on "bind" lines
    - MEDIUM: listener: Bind listeners on a thread subset if specified
    - CLEANUP: debug: Use DPRINTF instead of fprintf into #ifdef DEBUG_FULL/#endif
    - CLEANUP: log: Rename Alert/Warning in ha_alert/ha_warning
    - MINOR/CLEANUP: proxy: rename "proxy" to "proxies_list"
    - CLEANUP: pools: rename all pool functions and pointers to remove this "2"
    - DOC: update the roadmap file with the latest changes merged in 1.8
    - DOC: fix mangled version in peers protocol documentation
    - DOC: add initial peers protovol v2.0 documentation.
    - DOC: mention William as maintainer of the cache and master-worker
    - DOC: add Christopher and Emeric as maintainers of the threads
    - MINOR: cache: replace a fprint() by an abort()
    - MEDIUM: cache: max-age configuration keyword
    - DOC: explain HTTP2 timeout behavior
    - DOC: cache: configuration and management
    - MAJOR: mworker: exits the master on failure
    - BUG/MINOR: threads: don't drop "extern" on the lock in include files
    - MINOR: task: keep a pointer to the currently running task
    - MINOR: task: align the rq and wq locks
    - MINOR: fd: cache-align fdtab and fdcache locks
    - MINOR: buffers: cache-align buffer_wq_lock
    - CLEANUP: server: reorder some fields in struct server to save 40 bytes
    - CLEANUP: proxy: slightly reorder the struct proxy to reduce holes
    - CLEANUP: checks: remove 16 bytes of holes in struct check
    - CLEANUP: cache: more efficiently pack the struct cache
    - CLEANUP: fd: place the lock at the beginning of struct fdtab
    - CLEANUP: pools: align pools on a cache line
    - DOC: config: add a few bits about how to configure HTTP/2
    - BUG/MAJOR: threads/queue: avoid recursive locking in pendconn_get_next_strm()
    - BUILD: Makefile: reorder object files by size
2017-11-26 19:25:23 +01:00
Willy Tarreau
95c4e14d0c DOC: config: add a few bits about how to configure HTTP/2
There's hardly anything to say in this file beyond the ALPN
configuration and some precisions about how the HTTP transaction model
applies to HTTP/2.
2017-11-26 12:18:55 +01:00
William Lallemand
4cfede87a3 MAJOR: mworker: exits the master on failure
This patch changes the behavior of the master during the exit of a
worker.

When a worker exits with an error code, for example in the case of a
segfault, all workers are now killed and the master leaves.

If you don't want this behavior you can use the option
"master-worker no-exit-on-failure".
2017-11-24 22:48:27 +01:00
William Lallemand
86d0df0b66 DOC: cache: configuration and management 2017-11-24 22:48:27 +01:00
Lukas Tribus
75df9d7a7a DOC: explain HTTP2 timeout behavior
Clarifies that in HTTP2 we don't consider "timeout http-keep-alive", but
"timeout client" instead.
2017-11-24 19:33:27 +01:00
Emeric Brun
9c05c48b1b DOC: add initial peers protovol v2.0 documentation.
[wt: the new version is 2.1 but it's useful to document the different
 versions since they're found in field. There's some overlap with the
 new one and they complement on certain areas. Most likely they'll
 ultimately be merged.]
2017-11-24 18:28:05 +01:00
Willy Tarreau
99795b1952 DOC: fix mangled version in peers protocol documentation
Tim Düsterhus noticed that the create-release script had mangled the
version in the peers protocol doc, forcing it to 1.8 due to its syntax
matching the format of an haproxy version. Let's just slightly readjust
the header not to match this by removing the word "version" and placing
it on the same line as the title.
2017-11-24 18:10:24 +01:00
Willy Tarreau
bafbe01028 CLEANUP: pools: rename all pool functions and pointers to remove this "2"
During the migration to the second version of the pools, the new
functions and pool pointers were all called "pool_something2()" and
"pool2_something". Now there's no more pool v1 code and it's a real
pain to still have to deal with this. Let's clean this up now by
removing the "2" everywhere, and by renaming the pool heads
"pool_head_something".
2017-11-24 17:49:53 +01:00
Christopher Faulet
c644fa9bf5 MINOR: config: Add threads support for "process" option on "bind" lines
It is now possible on a "bind" line (or a "stats socket" line) to specify the
thread set allowed to process listener's connections. For instance:

    # HTTPS connections will be processed by all threads but the first and HTTP
    # connection will be processed on the first thread.
    bind *:80 process 1/1
    bind *:443 ssl crt mycert.pem process 1/2-
2017-11-24 15:38:50 +01:00
Christopher Faulet
cb6a94510d MINOR: config: Add the threads support in cpu-map directive
Now, it is possible to bind CPU at the thread level instead of the process level
by defining a thread set in "cpu-map" directives. Thus, its format is now:

  cpu-map [auto:]<process-set>[/<thread-set>] <cpu-set>...

where <process-set> and <thread-set> must follow the format:

  all | odd | even | number[-[number]]

Having a process range and a thread range in same time with the "auto:" prefix
is not supported. Only one range is supported, the other one must be a fixed
number. But it is allowed when there is no "auto:" prefix.

Because it is possible to define a mapping for a process and another for a
thread on this process, threads will be bound on the intersection of their
mapping and the one of the process on which they are attached. If the
intersection is null, no specific binding will be set for the threads.
2017-11-24 15:38:50 +01:00
Christopher Faulet
11da456e77 MINOR:: config: Remove thread-map directive
It was a temporary directive used for development purpose. Now, CPU mapping for
at the thread level should be done using the cpu-map directive. This feature
will be added in a next commit.
2017-11-24 15:38:50 +01:00
Christopher Faulet
ff4121f741 MINOR: config: Support partial ranges in cpu-map directive
Now, processa and CPU ranges can be partially defined. The higher bound can be
omitted. In such case, it is replaced by the corresponding maximum value, 32 or
64 depending on the machine's word size.

By extension, It is also true for the "bind-process" directive and "process"
parameter on a "bind" or a "stats socket" line.
2017-11-24 15:38:50 +01:00
Christopher Faulet
26028f6209 MINOR: config: Add auto-increment feature for cpu-map
The prefix "auto:" can be added before the process set to let HAProxy
automatically bind a process to a CPU by incrementing process and CPU sets. To
be valid, both sets must have the same size. No matter the declaration order of
the CPU sets, it will be bound from the lower to the higher bound.

  Examples:
      # all these lines bind the process 1 to the cpu 0, the process 2 to cpu 1
      #  and so on.
      cpu-map auto:1-4   0-3
      cpu-map auto:1-4   0-1 2-3
      cpu-map auto:1-4   3 2 1 0

      # bind each process to exaclty one CPU using all/odd/even keyword
      cpu-map auto:all   0-63
      cpu-map auto:even  0-31
      cpu-map auto:odd   32-63

      # invalid cpu-map because process and CPU sets have different sizes.
      cpu-map auto:1-4   0    # invalid
      cpu-map auto:1     0-3  # invalid
2017-11-24 15:38:49 +01:00
Christopher Faulet
1dcb9cb81c MINOR: config: Support a range to specify processes in "cpu-map" parameter
Now, you can define processes concerned by a cpu-map line using a range. For
instance, the following line binds the first 32 processes on CPUs 0 to 3:

  cpu-map 1-32 0-3
2017-11-24 15:38:49 +01:00
Willy Tarreau
45a66ccc55 MEDIUM: config: ensure that tune.bufsize is at least 16384 when using HTTP/2
HTTP/2 mandates the support of 16384 bytes frames by default, so we need
a large enough buffer to process them. Till now if tune.bufsize was too
small, H2 connections were simply rejected during their establishment,
making it quite hard to troubleshoot the issue.

Now we detect when HTTP/2 is enabled on an HTTP frontend and emit an
error if tune.bufsize is not large enough, with the appropriate
recommendation.
2017-11-24 11:28:00 +01:00
Willy Tarreau
599391a7c2 MINOR: h2: make use of client-fin timeout after GOAWAY
At the moment, the "client" timeout is used on an HTTP/2 connection once
it's idle with no active stream. With this patch, this timeout is replaced
by client-fin once a GOAWAY frame is sent. This closely matches what is
done on HTTP/1 since the principle is the same, as it indicates a willing
ness to quickly close a connection on which we don't expect to see anything
anymore.
2017-11-24 10:16:00 +01:00
Willy Tarreau
53275e8b02 MINOR: http: implement the "http-request reject" rule
This one acts similarly to its tcp-request counterpart. It immediately
closes the request without emitting any response. It can be suitable in
certain DoS conditions, as well as to close an HTTP/2 connection.
2017-11-24 07:52:01 +01:00
Christopher Faulet
34adb2af96 MINOR: sample: Add "thread" sample fetch
It returns id of the thread calling the function.
2017-11-23 16:33:13 +01:00
Lukas Tribus
f46bf95d2b BUG/MINOR: systemd: ignore daemon mode
Since we switched to notify mode in the systemd unit file in commit
d6942c8, haproxy won't start if the daemon keyword is present in the
configuration.

This change makes sure that haproxy remains in foreground when using
systemd mode and adds a note in the documentation.
2017-11-21 21:21:35 +01:00
Willy Tarreau
cfe14669f7 [RELEASE] Released version 1.8-rc4
Released version 1.8-rc4 with the following main changes :
    - BUG/MEDIUM: cache: does not cache if no Content-Length
    - BUILD: thread/pipe: fix build without threads
    - BUG/MINOR: spoe: check buffer size before acquiring or releasing it
    - MINOR: debug/flags: Add missing flags
    - MINOR: threads: Use __decl_hathreads to declare locks
    - BUG/MINOR: buffers: Fix b_alloc_margin to be "fonctionnaly" thread-safe
    - BUG/MAJOR: ebtree/scope: fix insertion and removal of duplicates in scope-aware trees
    - BUG/MAJOR: ebtree/scope: fix lookup of next node in scope-aware trees
    - MINOR: ebtree/scope: add a function to find next node from a parent
    - MINOR: ebtree/scope: simplify the lookup functions by using eb32sc_next_with_parent()
    - BUG/MEDIUM: mworker: Fix re-exec when haproxy is started from PATH
    - BUG/MEDIUM: cache: use msg->sov to forward header
    - MINOR: cache: forward data with headers
    - MINOR: cache: disable cache if shctx_row_data_append fail
    - BUG/MINOR: threads: tid_bit must be a unsigned long
    - CLEANUP: tasks: Remove useless double test on rq_next
    - BUG/MEDIUM: standard: itao_str/idx and quote_str/idx must be thread-local
    - MINOR: tools: add a function to dump a scope-aware tree to a file
    - MINOR: tools: improve the DOT dump of the ebtree
    - MINOR: tools: emphasize the node being worked on in the tree dump
    - BUG/MAJOR: ebtree/scope: properly tag upper nodes during insertion
    - DOC: peers: Add a first version of peers protocol v2.1.
    - CONTRIB: Wireshark dissector for HAProxy Peer Protocol.
    - MINOR: mworker: display an accurate error when the reexec fail
    - BUG/MEDIUM: mworker: wait again for signals when execvp fail
    - BUG/MEDIUM: mworker: does not deinit anymore
    - BUG/MEDIUM: mworker: does not close inherited FD
    - MINOR: tests: add a python wrapper to test inherited fd
    - BUG/MINOR: Allocate the log buffers before the proxies startup
    - MINOR: tasks: Use a bitfield to track tasks activity per-thread
    - MAJOR: polling: Use active_tasks_mask instead of tasks_run_queue
    - MINOR: applets: Use a bitfield to track applets activity per-thread
    - MAJOR: polling: Use active_appels_mask instead of applets_active_queue
    - MEDIUM: applets: Don't process more than 200 active applets at once
    - MINOR: stream: Add thread-mask of tasks/FDs/applets in "show sess all" command
    - MINOR: SSL: Store the ASN1 representation of client sessions.
    - MINOR: ssl: Make sure we don't shutw the connection before the handshake.
    - BUG/MEDIUM: deviceatlas: ignore not valuable HTTP request data
2017-11-19 09:55:29 +01:00
Frédéric Lécaille
4b6645d8a7 DOC: peers: Add a first version of peers protocol v2.1.
This documentation has to be completed.
2017-11-15 19:48:22 +01:00
Willy Tarreau
34650d5a7b [RELEASE] Released version 1.8-rc3
Released version 1.8-rc3 with the following main changes :
    - BUILD: use MAXPATHLEN instead of NAME_MAX.
    - BUG/MAJOR: threads/checks: add 4 missing spin_unlock() in various functions
    - BUG/MAJOR: threads/server: missing unlock in CLI fqdn parser
    - BUG/MINOR: cli: do not perform an invalid action on "set server check-port"
    - BUG/MAJOR: threads/checks: wrong use of SPIN_LOCK instead of SPIN_UNLOCK
    - CLEANUP: checks: remove return statements in locked functions
    - BUG/MINOR: cli: add severity in "set server addr" parser
    - CLEANUP: server: get rid of return statements in the CLI parser
    - BUG/MAJOR: cli/streams: missing unlock on exit "show sess"
    - BUG/MAJOR: threads/dns: add missing unlock on allocation failure path
    - BUG/MAJOR: threads/lb: fix missing unlock on consistent hash LB
    - BUG/MAJOR: threads/lb: fix missing unlock on map-based hash LB
    - BUG/MEDIUM: threads/stick-tables: close a race condition on stktable_trash_expired()
    - BUG/MAJOR: h2: set the connection's task to NULL when no client timeout is set
    - BUG/MAJOR: thread/listeners: enable_listener must not call unbind_listener()
    - BUG/MEDIUM: threads: don't try to free build option message on exit
    - MINOR: applets: no need to check for runqueue's emptiness in appctx_res_wakeup()
    - MINOR: add master-worker in the warning about nbproc
    - MINOR: mworker: allow pidfile in mworker + foreground
    - MINOR: mworker: write parent pid in the pidfile
    - MINOR: mworker: do not store child pid anymore in the pidfile
    - MINOR: ebtree: implement the scope-aware functions for eb32
    - MEDIUM: ebtree: specify the scope of every node inserted via eb32sc
    - MINOR: ebtree: update the eb32sc parent node's scope on delete
    - MEDIUM: ebtree: only consider the branches matching the scope in lookups
    - MINOR: ebtree: implement eb32sc_lookup_ge_or_first()
    - MAJOR: task: make use of the scope-aware ebtree functions
    - MINOR: task: simplify wake_expired_tasks() to avoid unlocking in the loop
    - MEDIUM: task: change the construction of the loop in process_runnable_tasks()
    - MINOR: threads: use faster locks for the spin locks
    - MINOR: tasks: only visit filled task slots after processing them
    - MEDIUM: tasks: implement a lockless scheduler for single-thread usage
    - BUG/MINOR: dns: Don't try to get the server lock if it's already held.
    - BUG/MINOR: dns: Don't lock the server lock in snr_check_ip_callback().
    - DOC: Add note about encrypted password CPU usage
    - BUG/MINOR: h2: set the "HEADERS_SENT" flag on stream, not connection
    - BUG/MEDIUM: h2: properly send an RST_STREAM on mux stream error
    - BUG/MEDIUM: h2: properly send the GOAWAY frame in the mux
    - BUG/MEDIUM: h2: don't try (and fail) to send non-existing data in the mux
    - MEDIUM: h2: remove the H2_SS_RESET intermediate state
    - BUG/MEDIUM: h2: fix some wrong error codes on connections
    - BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix
    - BUILD: enable USE_THREAD for Solaris build.
    - BUG/MEDIUM: h2: don't close the connection is there are data left
    - MINOR: h2: don't re-enable the connection's task when we're closing
    - BUG/MEDIUM: h2: properly set H2_SF_ES_SENT when sending the final frame
    - BUG/MINOR: h2: correctly check for H2_SF_ES_SENT before closing
    - MINOR: h2: add new stream flag H2_SF_OUTGOING_DATA
    - BUG/MINOR: h2: don't send GOAWAY on failed response
    - BUG/MEDIUM: splice/threads: pipe reuse list was not protected.
    - BUG/MINOR: comp: fix compilation warning compiling without compression.
    - BUG/MINOR: stream-int: don't set MSG_MORE on closed request path
    - BUG/MAJOR: threads/tasks: fix the scheduler again
    - BUG/MINOR; ssl: Don't assume we have a ssl_bind_conf because a SNI is matched.
    - MINOR: ssl: Handle session resumption with TLS 1.3
    - MINOR: ssl: Spell 0x10101000L correctly.
    - MINOR: ssl: Handle sending early data to server.
    - BUILD: ssl: fix build of backend without ssl
    - BUILD: shctx: do not depend on openssl anymore
    - BUG/MINOR: h1: the HTTP/1 make status code parser check for digits
    - BUG/MEDIUM: h2: reject non-3-digit status codes
    - BUG/MEDIUM: stream-int: Don't loss write's notifs when a stream is woken up
    - BUG/MINOR: pattern: Rely on the sample type to copy it in pattern_exec_match
    - BUG/MEDIUM: h2: split the function to send RST_STREAM
    - BUG/MEDIUM: h1: ensure the chunk size parser can deal with full buffers
    - MINOR: tools: don't use unlikely() in hex2i()
    - BUG/MEDIUM: h2: support orphaned streams
    - BUG/MEDIUM: threads/cli: fix "show sess" locking on release
    - CLEANUP: mux: remove the unused "release()" function
    - MINOR: cli: make "show fd" report the fd's thread mask
    - BUG/MEDIUM: stream: don't ignore res.analyse_exp anymore
    - CLEANUP: global: introduce variable pid_bit to avoid shifts with relative_pid
    - MEDIUM: http: always reject the "PRI" method
2017-11-11 09:06:48 +01:00
Daniel Schneller
d06f31c443 DOC: Add note about encrypted password CPU usage
From first-hand experience I realized that using encrypted passwords in
userlists can quickly become overwhelming for busy sites. In my case
just about 100 rq/s were enough to drive (user) CPU usage from 2-3% up
to >90%. While it is perfectly explicable why this is the case, having
it mentioned in the relevant documentation section might spare someone
some confusion in the future.
2017-11-06 18:37:35 +01:00
Willy Tarreau
a8d8d6e8f6 [RELEASE] Released version 1.8-rc2
Released version 1.8-rc2 with the following main changes :
    - BUG/MINOR: send-proxy-v2: fix dest_len in make_tlv call
    - BUG/MINOR: send-proxy-v2: string size must include ('\0')
    - MINOR: mux: Only define pipe functions on linux.
    - MINOR: cache: Remove useless test for nonzero.
    - MINOR: cache: Don't confuse act_return and act_parse_ret.
    - BUG/MEDIUM: h2: don't try to parse incomplete H1 responses
    - BUG/MEDIUM: checks/mux: always enable send-polling after connecting
    - BUG/MAJOR: fix deadlock on healthchecks.
    - BUG/MINOR: thread: fix a typo in the debug code
    - BUILD: shctx: allow to be built without openssl
    - BUG/MEDIUM: cache: don't try to resolve wrong filters
    - BUG/MAJOR: buffers: fix get_buffer_nc() for data at end of buffer
    - BUG/MINOR: freq: fix infinite loop on freq_ctr_period.
    - BUG/MINOR: stdarg.h inclusion
    - BUG/MINOR: dns: fix missing lock protection on server.
    - BUG/MINOR: lua: fix missing lock protection on server.
    - BUILD: enable USE_THREAD for OpenBSD build.
    - BUG/MAJOR: mux_pt: don't dereference a connstream after ->wake()
    - MINOR: thread: report multi-thread support in haproxy -vv
2017-11-03 23:52:47 +01:00
Willy Tarreau
901f75c4a6 [RELEASE] Released version 1.8-rc1
Released version 1.8-rc1 with the following main changes :
    - BUG/MEDIUM: server: Allocate tmptrash before using it.
    - CONTRIB: trace: add the possibility to place trace calls in the code
    - CONTRIB: trace: try to display the function's return value on exit
    - CONTRIB: trace: report the base name only for file names
    - BUILD: ssl: support OPENSSL_NO_ASYNC #define
    - MINOR: ssl: build with recent BoringSSL library
    - BUG/MINOR: ssl: OCSP_single_get0_status can return -1
    - BUG/MINOR: cli: restore "set ssl tls-key" command
    - CLEANUP: cli: remove undocumented "set ssl tls-keys" command
    - IMPORT: sha1: import SHA1 functions
    - MINOR: sample: add the sha1 converter
    - MINOR: sample: add the hex2i converter
    - MINOR: stream-int: stop checking for useless connection flags in chk_snd_conn
    - MINOR: ssl: don't abort after sending 16kB
    - MINOR: connection: move the cleanup of flag CO_FL_WAIT_ROOM
    - MINOR: connection: add flag CO_FL_WILL_UPDATE to indicate when updates are granted
    - MEDIUM: connection: make use of CO_FL_WILL_UPDATE in conn_sock_shutw()
    - MINOR: raw_sock: make use of CO_FL_WILL_UPDATE
    - MINOR: ssl_sock: make use of CO_FL_WILL_UPDATE
    - BUG/MINOR: checks: Don't forget to release the connection on error case.
    - MINOR: buffer: add the buffer input manipulation functions
    - BUG/MEDIUM: prevent buffers being overwritten during build_logline() execution
    - MEDIUM: cfgparse: post section callback
    - MEDIUM: cfgparse: post parsing registration
    - MINOR: lua: add uuid to the Class Proxy
    - MINOR: hlua: Add regex class
    - MINOR: http: Mark the 425 code as "Too Early".
    - MEDIUM: ssl: convert CBS (BoringSSL api) usage to neutral code
    - MINOR: ssl: support Openssl 1.1.1 early callback for switchctx
    - MINOR: ssl: generated certificate is missing in switchctx early callback
    - MEDIUM: ssl: Handle early data with OpenSSL 1.1.1
    - BUILD: Makefile: disable -Wunused-label
    - MINOR: ssl/proto_http: Add keywords to take care of early data.
    - BUG/MINOR: lua: const attribute of a string is overridden
    - MINOR: ssl: Don't abuse ssl_options.
    - MINOR: update proxy-protocol-v2 #define
    - MINOR: merge ssl_sock_get calls for log and ppv2
    - MINOR: add ALPN information to send-proxy-v2
    - MEDIUM: h1: ensure that 1xx, 204 and 304 don't have a payload body
    - CLEANUP: shctx: get ride of the shsess_packet{_hdr} structures
    - MEDIUM: lists: list_for_each_entry{_safe}_from functions
    - REORG: shctx: move lock functions and struct
    - MEDIUM: shctx: allow the use of multiple shctx
    - REORG: shctx: move ssl functions to ssl_sock.c
    - MEDIUM: shctx: separate ssl and shctx
    - MINOR: shctx: rename lock functions
    - MINOR: h1: store the status code in the H1 message
    - BUG/MINOR: spoe: Don't compare engine name and SPOE scope when both are NULL
    - BUG/MINOR: spoa: Update pointer on the end of the frame when a reply is encoded
    - MINOR: action: Add trk_idx inline function
    - MINOR: action: Use trk_idx instead of tcp/http_trk_idx
    - MINOR: action: Add a function pointer in act_rule struct to check its validity
    - MINOR: action: Add function to check rules using an action ACT_ACTION_TRK_*
    - MINOR: action: Add a functions to check http capture rules
    - MINOR: action: Factorize checks on rules calling check_ptr if defined
    - MINOR: acl: Pass the ACLs as an explicit parameter of build_acl_cond
    - MEDIUM: spoe: Add support of ACLS to enable or disable sending of SPOE messages
    - MINOR: spoe: Check uniqness of SPOE engine names during config parsing
    - MEDIUM: spoe: Parse new "spoe-group" section in SPOE config file
    - MEDIUM: spoe/rules: Add "send-spoe-group" action for tcp/http rules
    - MINOR: spoe: Move message encoding in its own function
    - MINOR: spoe: Add a type to qualify the message list during encoding
    - MINOR: spoe: Add a generic function to encode a list of SPOE message
    - MEDIUM: spoe/rules: Process "send-spoe-group" action
    - BUG/MINOR: dns: Fix CLI keyword declaration
    - MAJOR: dns: Refactor the DNS code
    - BUG/MINOR: mailers: Fix a memory leak when email alerts are released
    - MEDIUM: mailers: Init alerts during conf parsing and refactor their processing
    - MINOR: mailers: Use pools to allocate email alerts and its tcpcheck_rules
    - MINOR: standard: Add memvprintf function
    - MINOR: log: Save alerts and warnings emitted during HAProxy startup
    - MINOR: cli: Add "show startup-logs" command
    - MINOR: startup: Extend the scope the MODE_STARTING flag
    - MINOR: threads: Prepare makefile to link with pthread
    - MINOR: threads: Add THREAD_LOCAL macro
    - MINOR: threads: Add atomic-ops and plock includes in import dir
    - MEDIUM: threads: Add hathreads header file
    - MINOR: threads: Add mechanism to register per-thread init/deinit functions
    - MINOR: threads: Add nbthread parameter
    - MEDIUM: threads: Adds a set of functions to handle sync-point
    - MAJOR: threads: Start threads to experiment multithreading
    - MINOR: threads: Define the sync-point inside run_poll_loop
    - MEDIUM: threads/buffers: Define and register per-thread init/deinit functions
    - MEDIUM: threads/chunks: Transform trash chunks in thread-local variables
    - MEDIUM: threads/time: Many global variables from time.h are now thread-local
    - MEDIUM: threads/logs: Make logs thread-safe
    - MEDIUM: threads/pool: Make pool thread-safe by locking all access to a pool
    - MAJOR: threads/fd: Make fd stuffs thread-safe
    - MINOR: threads/fd: Add a mask of threads allowed to process on each fd in fdtab array
    - MEDIUM: threads/fd: Initialize the process mask during the call to fd_insert
    - MINOR: threads/fd: Process cached events of FDs depending on the process mask
    - MINOR: threads/polling: pollers now handle FDs depending on the process mask
    - WIP: SQUASH WITH SYNC POINT
    - MAJOR: threads/task: handle multithread on task scheduler
    - MEDIUM: threads/signal: Add a lock to make signals thread-safe
    - MEDIUM: threads/listeners: Make listeners thread-safe
    - MEDIUM: threads/proxy: Add a lock per proxy and atomically update proxy vars
    - MEDIUM: threads/server: Make connection list (priv/idle/safe) thread-safe
    - MEDIUM: threads/server: Add a lock per server and atomically update server vars
    - MINOR: threads/server: Add a lock to deal with insert in updates_servers list
    - MEDIUM: threads/lb: Make LB algorithms (lb_*.c) thread-safe
    - MEDIUM: threads/stick-tables: handle multithreads on stick tables
    - MINOR: threads/sample: Change temp_smp into a thread local variable
    - MEDIUM: threads/http: Make http_capture_bad_message thread-safe
    - MINOR: threads/regex: Change Regex trash buffer into a thread local variable
    - MAJOR: threads/applet: Handle multithreading for applets
    - MAJOR: threads/peers: Make peers thread safe
    - MAJOR: threads/buffer: Make buffer wait queue thread safe
    - MEDIUM: threads/stream: Make streams list thread safe
    - MAJOR: threads/ssl: Make SSL part thread-safe
    - MEDIUM: threads/queue: Make queues thread-safe
    - MAJOR: threads/map: Make acls/maps thread safe
    - MEDIUM: threads/freq_ctr: Make the frequency counters thread-safe
    - MEDIUM: thread/vars: Make vars thread-safe
    - MEDIUM: threads/filters: Add init/deinit callback per thread
    - MINOR: threads/filters: Update trace filter to add _per_thread callbacks
    - MEDIUM: threads/compression: Make HTTP compression thread-safe
    - MEDIUM: threads/lua: Makes the jmpbuf and some other buffers local to the current thread.
    - MEDIUM: threads/lua: Add locks around the Lua execution parts.
    - MEDIUM: threads/lua: Ensure that the launched tasks runs on the same threads than me
    - MEDIUM: threads/lua: Cannot acces to the socket if we try to access from another thread.
    - MEDIUM: threads/xref: Convert xref function to a thread safe model
    - MEDIUM: threads/tasks: Add lock around notifications
    - MEDIUM: thread/spoe: Make the SPOE thread-safe
    - MEDIUM: thread/dns: Make DNS thread-safe
    - MINOR: threads: Add thread-map config parameter in the global section
    - MINOR: threads/checks: Add a lock to protect the pid list used by external checks
    - MINOR: threads/checks: Set the task process_mask when a check is executed
    - MINOR: threads/mailers: Add a lock to protect queues of email alerts
    - MEDIUM: threads/server: Use the server lock to protect health check and cli concurrency
    - MINOR: threads: Don't start when device a detection module is used
    - BUG/MEDIUM: threads: Run the poll loop on the main thread too
    - BUG/MINOR: threads: Add missing THREAD_LOCAL on static here and there
    - MAJOR: threads: Offically enable the threads support in HAProxy
    - BUG/MAJOR: threads/freq_ctr: fix lock on freq counters.
    - BUG/MAJOR: threads/time: Store the time deviation in an 64-bits integer
    - BUILD: stick-tables: silence an uninitialized variable warning
    - BUG/MINOR: dns: Fix SRV records with the new thread code.
    - MINOR: ssl: Remove the global allow-0rtt option.
    - CLEANUP: threads: replace the last few 1UL<<tid with tid_bit
    - CLEANUP: threads: rename process_mask to thread_mask
    - MINOR: h1: add a function to measure the trailers length
    - MINOR: threads: add a portable barrier for threads and non-threads
    - BUG/MAJOR: threads/freq_ctr: use a memory barrier to detect changes
    - BUG/MEDIUM: threads: Initialize the sync-point
    - MEDIUM: connection: start to introduce a mux layer between xprt and data
    - MINOR: connection: implement alpn registration of muxes
    - MINOR: mux: register the pass-through mux for any ALPN string
    - MEDIUM: session: use the ALPN token and proxy mode to select the mux
    - MINOR: connection: report the major HTTP version from the MUX for logging (fc_http_major)
    - MINOR: connection: introduce conn_stream
    - MINOR: mux: add more methods to mux_ops
    - MINOR: connection: introduce the conn_stream manipulation functions
    - MINOR: mux_pt: implement remaining mux_ops methods
    - MAJOR: connection : Split struct connection into struct connection and struct conn_stream.
    - MINOR: connection: make conn_stream users also check for per-stream error flag
    - MINOR: conn_stream: new shutr/w status flags
    - MINOR: conn_stream: modify cs_shut{r,w} API to pass the desired mode
    - MEDIUM: connection: make conn_sock_shutw() aware of lingering
    - MINOR: connection: add cs_close() to close a conn_stream
    - MEDIUM: mux_pt: make cs_shutr() / cs_shutw() properly close the connection
    - MEDIUM: connection: replace conn_full_close() with cs_close()
    - MEDIUM: connection: make mux->detach() release the connection
    - MEDIUM: stream: do not forcefully close the client connection anymore
    - MEDIUM: checks: exclusively use cs_destroy() to release a connection
    - MEDIUM: connection: add a destroy callback
    - MINOR: session: release the listener with the session, not the stream
    - MEDIUM: session: make use of the connection's destroy callback
    - CONTRIB: hpack: implement a reverse huffman table generator for hpack
    - MINOR: hpack: implement the HPACK Huffman table decoder
    - MINOR: hpack: implement the header tables management
    - MINOR: hpack: implement the decoder
    - MEDIUM: hpack: implement basic hpack encoding
    - MINOR: h2: centralize all HTTP/2 protocol elements and constants
    - MINOR: h2: create a very minimalistic h2 mux
    - MINOR: h2: expose tune.h2.header-table-size to configure the table size
    - MINOR: h2: expose tune.h2.initial-window-size to configure the window size
    - MINOR: h2: expose tune.h2.max-concurrent-streams to limit the number of streams
    - MINOR: h2: create the h2c struct and allocate its pool
    - MINOR: h2: create the h2s struct and the associated pool
    - MINOR: h2: handle two extra stream states for errors
    - MINOR: h2: add a frame header descriptor for incoming frames
    - MEDIUM: h2: allocate and release the h2c context on connection init/end
    - MEDIUM: h2: implement basic recv/send/wake functions
    - MEDIUM: h2: dynamically allocate the demux buffer on Rx
    - MEDIUM: h2: implement the mux buffer allocator
    - MINOR: h2: add the connection and stream flags listing the causes for blocking
    - MINOR: h2: add function h2s_id() to report a stream's ID
    - MINOR: h2: small function to know when the mux is busy
    - MINOR: h2: new function h2c_error to mark an error on the connection
    - MINOR: h2: new function h2s_error() to mark an error on a stream
    - MINOR: h2: add h2_set_frame_size() to update the size in a binary frame
    - MINOR: h2: new function h2_peek_frame_hdr() to retrieve a new frame header
    - MINOR: h2: add a few functions to retrieve contents from a wrapping buffer
    - MINOR: h2: add stream lookup function based on the stream ID
    - MINOR: h2: create dummy idle and closed streams
    - MINOR: h2: add the function to create a new stream
    - MINOR: h2: update the {MUX,DEM}_{M,D}ALLOC flags on buffer availability
    - MEDIUM: h2: start to consider the H2_CF_{MUX,DEM}_* flags for polling
    - MINOR: h2: also terminate the connection on shutr
    - MEDIUM: h2: properly consider all conditions for end of connection
    - MEDIUM: h2: wake the connection up for send on pending streams
    - MEDIUM: h2: start to implement the frames processing loop
    - MINOR: h2: add a function to send a GOAWAY error frame
    - MINOR: h2: match the H2 connection preface on init
    - MEDIUM: h2: enable connection polling for send when a cs wants to emit
    - MEDIUM: h2: enable reading again on the connection if it was blocked on stream buffer full
    - MEDIUM: h2: process streams pending for sending
    - MINOR: h2: send a real SETTINGS frame based on the configuration
    - MEDIUM: h2: detect the presence of the first settings frame
    - MINOR: h2: create a stream parser for the demuxer
    - MINOR: h2: implement PING frames
    - MEDIUM: h2: decode SETTINGS frames and extract relevant settings
    - MINOR: h2: lookup the stream during demuxing
    - MEDIUM: h2: honor WINDOW_UPDATE frames
    - MINOR: h2: implement h2_send_rst_stream() to send RST_STREAM frames
    - MINOR: h2: handle CONTINUATION frames
    - MEDIUM: h2: partial implementation of h2_detach()
    - MEDIUM: h2: unblock a connection when its current stream detaches
    - MEDIUM: h2: basic processing of HEADERS frame
    - MEDIUM: h2: don't use trash to decode headers!
    - MEDIUM: h2: implement the response HEADERS frame to encode the H1 response
    - MEDIUM: h2: send the H1 response body as DATA frames
    - MEDIUM: h2: skip the response trailers if any
    - MEDIUM: h2: properly continue to parse header block when facing a 1xx response
    - MEDIUM: h2: send WINDOW_UPDATE frames for connection
    - MEDIUM: h2: handle request body in DATA frames
    - MINOR: h2: handle RST_STREAM frames
    - MEDIUM: h2: send DATA+ES or RST_STREAM on shutw/shutr
    - MINOR: h2: use a common function to signal some and all streams.
    - MEDIUM: h2: handle GOAWAY frames
    - MINOR: h2: centralize the check for the idle streams
    - MINOR: h2: centralize the check for the half-closed(remote) streams
    - MEDIUM: h2: silently ignore frames higher than last_id after GOAWAY
    - MINOR: h2: properly reject PUSH_PROMISE frames coming from the client
    - MEDIUM: h2: perform a graceful shutdown on "Connection: close"
    - MEDIUM: h2: send a GOAWAY frame when dealing with an empty response
    - MEDIUM: h2: apply a timeout to h2 connections
    - BUG/MEDIUM: h2: fix incorrect timeout handling on the connection
    - MEDIUM: shctx: forbid shctx to read more than expected
    - MEDIUM: cache: configuration parsing and initialization
    - MEDIUM: cache: store objects in cache
    - MEDIUM: cache: deliver objects from cache
2017-10-31 23:18:29 +01:00
Willy Tarreau
5242ef8095 MINOR: h2: expose tune.h2.max-concurrent-streams to limit the number of streams
This will be advertised in the settings frame.
2017-10-31 18:03:24 +01:00
Willy Tarreau
e6baec0e23 MINOR: h2: expose tune.h2.initial-window-size to configure the window size
This will be advertised in the settings frame.
2017-10-31 18:03:24 +01:00
Willy Tarreau
fe20e5b8c7 MINOR: h2: expose tune.h2.header-table-size to configure the table size
It's the HPACK header table size which is to be advertised in the settings
frames. It defaults to 4096.
2017-10-31 18:03:24 +01:00
Willy Tarreau
60ca10a372 MINOR: connection: report the major HTTP version from the MUX for logging (fc_http_major)
A new sample fetch function reports either 1 or 2 for the on-wire encoding,
to indicate if the request was received using the HTTP/1.x format or HTTP/2
format. Note that it reports the on-wire encoding, not the version presented
in the request header.

This will possibly have to evolve if it becomes necessary to report the
encoding on the server side as well.
2017-10-31 18:03:23 +01:00
Olivier Houchard
79a481ddde MINOR: ssl: Remove the global allow-0rtt option. 2017-10-31 15:48:42 +01:00
Christopher Faulet
6251902e67 MINOR: threads: Add thread-map config parameter in the global section
By default, no affinity is set for threads. To bind threads on CPU, you must
define a "thread-map" in the global section. The format is the same than the
"cpu-map" parameter, with a small difference. The process number must be
defined, with the same format than cpu-map ("all", "even", "odd" or a number
between 1 and 31/63).

A thread will be bound on the intersection of its mapping and the one of the
process on which it is attached. If the intersection is null, no specific bind
will be set for the thread.
2017-10-31 13:58:33 +01:00
Christopher Faulet
71a6a8efaa MEDIUM: threads/filters: Add init/deinit callback per thread
Now, it is possible to define init_per_thread and deinit_per_thread callbacks to
deal with ressources allocation for each thread.

This is the filter responsibility to deal with concurrency. This is also the
filter responsibility to know if HAProxy is started with some threads. A good
way to do so is to check "global.nbthread" value. If it is greater than 1, then
_per_thread callbacks will be called.
2017-10-31 13:58:32 +01:00
Christopher Faulet
be0faa2e47 MINOR: threads: Add nbthread parameter
It is only parsed and initialized for now. It will be used later. This parameter
is only available when support for threads was built in.
2017-10-31 13:58:29 +01:00
Christopher Faulet
67957bd59e MAJOR: dns: Refactor the DNS code
This is a huge patch with many changes, all about the DNS. Initially, the idea
was to update the DNS part to ease the threads support integration. But quickly,
I started to refactor some parts. And after several iterations, it was
impossible for me to commit the different parts atomically. So, instead of
adding tens of patches, often reworking the same parts, it was easier to merge
all my changes in a uniq patch. Here are all changes made on the DNS.

First, the DNS initialization has been refactored. The DNS configuration parsing
remains untouched, in cfgparse.c. But all checks have been moved in a post-check
callback. In the function dns_finalize_config, for each resolvers, the
nameservers configuration is tested and the task used to manage DNS resolutions
is created. The links between the backend's servers and the resolvers are also
created at this step. Here no connection are kept alive. So there is no needs
anymore to reopen them after HAProxy fork. Connections used to send DNS queries
will be opened on demand.

Then, the way DNS requesters are linked to a DNS resolution has been
reworked. The resolution used by a requester is now referenced into the
dns_requester structure and the resolution pointers in server and dns_srvrq
structures have been removed. wait and curr list of requesters, for a DNS
resolution, have been replaced by a uniq list. And Finally, the way a requester
is removed from a DNS resolution has been simplified. Now everything is done in
dns_unlink_resolution.

srv_set_fqdn function has been simplified. Now, there is only 1 way to set the
server's FQDN, independently it is done by the CLI or when a SRV record is
resolved.

The static DNS resolutions pool has been replaced by a dynamoc pool. The part
has been modified by Baptiste Assmann.

The way the DNS resolutions are triggered by the task or by a health-check has
been totally refactored. Now, all timeouts are respected. Especially
hold.valid. The default frequency to wake up a resolvers is now configurable
using "timeout resolve" parameter.

Now, as documented, as long as invalid repsonses are received, we really wait
all name servers responses before retrying.

As far as possible, resources allocated during DNS configuration parsing are
releases when HAProxy is shutdown.

Beside all these changes, the code has been cleaned to ease code review and the
doc has been updated.
2017-10-31 11:36:12 +01:00
Christopher Faulet
344c4ab6a9 MEDIUM: spoe/rules: Process "send-spoe-group" action
The messages processing is done using existing functions. So here, the main task
is to find the SPOE engine to use. To do so, we loop on all filter instances
attached to the stream. For each, we check if it is a SPOE filter and, if yes,
if its name is the one used to declare the "send-spoe-group" action.

We also take care to return an error if the action processing is interrupted by
HAProxy (because of a timeout or an error at the HAProxy level). This is done by
checking if the flag ACT_FLAG_FINAL is set.

The function spoe_send_group is the action_ptr callback ot
2017-10-31 11:36:12 +01:00
Christopher Faulet
76c09ef8de MEDIUM: spoe/rules: Add "send-spoe-group" action for tcp/http rules
This action is used to trigger sending of a group of SPOE messages. To do so,
the SPOE engine used to send messages must be defined, as well as the SPOE group
to send. Of course, the SPOE engine must refer to an existing SPOE filter. If
not engine name is provided on the SPOE filter line, the SPOE agent name must be
used. For example:

   http-request send-spoe-group my-engine some-group

This action is available for "tcp-request content", "tcp-response content",
"http-request" and "http-response" rulesets. It cannot be used for tcp
connection/session rulesets because actions for these rulesets cannot yield.

For now, the action keyword is parsed and checked. But it does nothing. Its
processing will be added in another patch.
2017-10-31 11:36:12 +01:00
Christopher Faulet
11610f3b5a MEDIUM: spoe: Parse new "spoe-group" section in SPOE config file
For now, this section is only parsed. It should have the following format:

    spoe-group <grp-name>
      messages <msg-name> ...

And then SPOE groups must be referenced in spoe-agent section:

    spoe-agnt <name>
        ...
	groups <grp-name> ...

The purpose of these groups is to trigger messages sending from TCP or HTTP
rules, directly from HAProxy configuration, and not on specific event. This part
will be added in another patch.

It is important to note that a message belongs at most to a group.
2017-10-31 11:36:12 +01:00
Christopher Faulet
7ee8667c99 MINOR: spoe: Check uniqness of SPOE engine names during config parsing
The engine name is now kept in "spoe_config" struture. Because a SPOE filter can
be declared without engine name, we use the SPOE agent name by default. Then,
its uniqness is checked against all others SPOE engines configured for the same
proxy.

  * TODO: Add documentation
2017-10-31 11:36:12 +01:00