1
0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2025-02-28 08:30:46 +00:00
Commit Graph

1338 Commits

Author SHA1 Message Date
Willy Tarreau
7cc040cc74 DOC: add more design feedback on the new layering model
Introduce the distinction between structured messages and raw data,
and how to make them coexist in a buffer. This is still a design draft.
2018-07-23 17:29:37 +02:00
Willy Tarreau
9382cdd8e1 DOC: add some design notes about the new layering model
This explains how streams and connection should interact.
2018-07-20 19:21:43 +02:00
Willy Tarreau
f148888d19 MINOR: buffers: add b_xfer() to transfer data between buffers
Instead of open-coding buffer-to-buffer transfers using blocks, let's
have a dedicated function for this. It also adjusts the buffer counts.
2018-07-20 19:21:43 +02:00
Willy Tarreau
23d465c48c DOC: buffers: remove obsolete docs about buffers
A number of outdated docs dating 2012 about buffers implementation
and management were totally irrelevant to the current code (and even
to most 1.8 code as well). These docs have all been removed so that
only the up to date documentation remains.
2018-07-19 16:23:43 +02:00
Willy Tarreau
9d752e96d4 DOC: buffers: document the new buffers API
Most of the new functions have been documented in buffer-api.txt, with
a diagram of what a buffer looks like and some hints to convert older
code.
2018-07-19 16:23:43 +02:00
Willy Tarreau
83061a820e MAJOR: chunks: replace struct chunk with struct buffer
Now all the code used to manipulate chunks uses a struct buffer instead.
The functions are still called "chunk*", and some of them will progressively
move to the generic buffer handling code as they are cleaned up.
2018-07-19 16:23:43 +02:00
Willy Tarreau
bcbd39370f MINOR: channel/buffer: replace b_{adv,rew} with c_{adv,rew}
These ones manipulate the output data count which will be specific to
the channel soon, so prepare the call points to use the channel only.
The b_* functions are now unused and were removed.
2018-07-19 16:23:40 +02:00
Baptiste Assmann
8e2d9430c0 MINOR: dns: new DNS options to allow/prevent IP address duplication
By default, HAProxy's DNS resolution at runtime ensure that there is no
IP address duplication in a backend (for servers being resolved by the
same hostname).
There are a few cases where people want, on purpose, to disable this
feature.

This patch introduces a couple of new server side options for this purpose:
"resolve-opts allow-dup-ip" or "resolve-opts prevent-dup-ip".
2018-07-12 17:56:44 +02:00
Fr�d�ric L�caille
fd9655c54d DOC: regression testing: Add a short starting guide.
This documentation describes how to write varnish test case (VTC)
files to reg test haproxy.
2018-06-12 14:40:21 +02:00
Kevin Zhu
730323e56e DOC: SPOE.txt: fix a typo 2018-06-01 05:38:00 +02:00
Ben Draut
44e609bfa5 MINOR: dns: Implement parse-resolv-conf directive
This introduces a new directive for the `resolvers` section:
`parse-resolv-conf`. When present, it will attempt to add any
nameservers in `/etc/resolv.conf` to the list of nameservers
for the current `resolvers` section.

[Mailing list thread][1].

[1]: https://www.mail-archive.com/haproxy@formilux.org/msg29600.html
2018-05-30 05:17:16 +02:00
Willy Tarreau
ea96a823c6 DOC: management: add the new wrew stats column
This is the number of failed rewrite warnings, per front/listener/back/server.
2018-05-28 15:16:23 +02:00
Olivier Houchard
1599b80360 MINOR: tasks: Make the number of tasks to run at once configurable.
Instead of hardcoding 200, make the number of tasks to be run configurable
using tune.runqueue-depth. 200 is still the default.
2018-05-26 20:03:24 +02:00
Lukas Tribus
926594f606 MINOR: ssl: set SSL_OP_PRIORITIZE_CHACHA
Sets OpenSSL 1.1.1's SSL_OP_PRIORITIZE_CHACHA unconditionally, as per [1]:

When SSL_OP_CIPHER_SERVER_PREFERENCE is set, temporarily reprioritize
ChaCha20-Poly1305 ciphers to the top of the server cipher list if a
ChaCha20-Poly1305 cipher is at the top of the client cipher list. This
helps those clients (e.g. mobile) use ChaCha20-Poly1305 if that cipher
is anywhere in the server cipher list; but still allows other clients to
use AES and other ciphers. Requires SSL_OP_CIPHER_SERVER_PREFERENCE.

[1] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_clear_options.html
2018-05-23 16:55:15 +02:00
William Lallemand
8a16fe0d05 BUG/MEDIUM: cache: don't cache when an Authorization header is present
RFC 7234 says:

A cache MUST NOT store a response to any request, unless:
[...] the Authorization header field (see Section 4.2 of [RFC7235]) does
      not appear in the request, if the cache is shared, unless the
      response explicitly allows it (see Section 3.2), [...]

In this patch we completely disable the cache upon the receipt of an
Authorization header in the request. In this case it's not possible to
either use the cache or store into the cache anymore.

Thanks to Adam Eijdenberg of Digital Transformation Agency for raising
this issue.

This patch must be backported to 1.8.
2018-05-23 10:36:44 +02:00
Willy Tarreau
fa9f9ccd6f DOC: add some description of the pending rework of the buffer structure
The "struct buffer" needs to be reworked, this new doc lists the changes
and steps to do this.
2018-05-18 16:18:17 +02:00
Christopher Faulet
6e0d5e7f67 DOC: spoe: fix a typo
s/STATUC/STATUS/
2018-05-18 15:05:17 +02:00
Thierry FOURNIER
c4dcaff3f0 BUG/MEDIUM: spoe: Flags are not encoded in network order
The flags are direct copy of the "unsigned int" in the network stream,
so the stream contains a 32 bits field encoded with the host endian.
 - This is not reliable for stream betwen different architecture host
 - For x86, the bits doesn't correspond to the documentation.

This patch add some precision in the documentation and put the bitfield
in the stream usig network butes order.

Warning: this patch can break compatibility with existing agents.

This patch should be backported in all version supporing SPOE

Original network capture:

   12:28:16.181343 IP 127.0.0.1.46782 > 127.0.0.1.12345: Flags [P.], seq 134:168, ack 59, win 342, options [nop,nop,TS val 2855241281 ecr 2855241281], length 34
           0x0000:  4500 0056 6b94 4000 4006 d10b 7f00 0001  E..Vk.@.@.......
           0x0010:  7f00 0001 b6be 3039 a3d1 ee54 7d61 d6f7  ......09...T}a..
           0x0020:  8018 0156 fe4a 0000 0101 080a aa2f 8641  ...V.J......./.A
           0x0030:  aa2f 8641 0000 001e 0301 0000 0000 010f  ./.A............
                                          ^^^^^^^^^^
           0x0040:  6368 6563 6b2d 636c 6965 6e74 2d69 7001  check-client-ip.
           0x0050:  0006 7f00 0001                           ......

Fixed network capture:

   12:24:26.948165 IP 127.0.0.1.46706 > 127.0.0.1.12345: Flags [P.], seq 4066280627:4066280661, ack 3148908096, win 342, options [nop,nop,TS val 2855183972 ecr 2855177690], length 34
           0x0000:  4500 0056 0538 4000 4006 3768 7f00 0001  E..V.8@.@.7h....
           0x0010:  7f00 0001 b672 3039 f25e 84b3 bbb0 8640  .....r09.^.....@
           0x0020:  8018 0156 fe4a 0000 0101 080a aa2e a664  ...V.J.........d
           0x0030:  aa2e 8dda 0000 001e 0300 0000 0114 010f  ................
                                          ^^^^^^^^^^
           0x0040:  6368 6563 6b2d 636c 6965 6e74 2d69 7001  check-client-ip.
           0x0050:  0006 7f00 0001                           ......
2018-05-18 13:50:53 +02:00
Patrick Hemmer
32d539fa88 MINOR: lua: add get_maxconn and set_maxconn to LUA Server class. 2018-05-03 18:53:42 +02:00
Patrick Hemmer
a62ae7ed9a MINOR: lua: Add server name & puid to LUA Server class. 2018-05-03 18:44:44 +02:00
Patrick Hemmer
c6a1d711a4 DOC/MINOR: clean up LUA documentation re: servers & array/table.
* A few typos
* Fix definitions of values which are tables, not arrays.
* Consistent US English naming for "server" instead of "serveur".

[tfo: should be backported to 1.6 and higher]
2018-05-03 18:42:24 +02:00
Willy Tarreau
760e81d356 MINOR: backend: implement random-based load balancing
For large farms where servers are regularly added or removed, picking
a random server from the pool can ensure faster load transitions than
when using round-robin and less traffic surges on the newly added
servers than when using leastconn.

This commit introduces "balance random". It internally uses a random as
the key to the consistent hashing mechanism, thus all features available
in consistent hashing such as weights and bounded load via hash-balance-
factor are usable. It is extremely convenient because one common concern
when using random is what happens when a server is hammered a bit too
much. Here that can trivially be avoided, like in the configuration below :

    backend bk0
        balance random
        hash-balance-factor 110
        server-template s 1-100 127.0.0.1:8000 check inter 1s

Note that while "balance random" internally relies on a hash algorithm,
it holds the same properties as round-robin and as such is compatible with
reusing an existing server connection with "option prefer-last-server".
2018-05-03 07:20:40 +02:00
Patrick Hemmer
e027547f8d MINOR: ssl: add fetch 'ssl_fc_session_key' and 'ssl_bc_session_key'
These fetches return the SSL master key of the front/back connection.
This is useful to decrypt traffic encrypted with ephemeral ciphers.
2018-04-30 14:56:19 +02:00
Tim Duesterhus
ca097c16a8 MINOR: sample: Add strcmp sample converter
This converter supplements the existing string matching by allowing
strings to be converted to a variable.

Example usage:

  http-request set-var(txn.host) hdr(host)
  # Check whether the client is attempting domain fronting.
  acl ssl_sni_http_host_match ssl_fc_sni,strcmp(txn.host) eq 0
2018-04-28 07:03:39 +02:00
Aur�lien Nephtali
1e0867cfbc MINOR: ssl: Add payload support to "set ssl ocsp-response"
It is now possible to use a payload with the "set ssl ocsp-response"
command.  These syntaxes will work the same way:

 # echo "set ssl ocsp-response $(base64 -w 10000 ocsp.der)" | \
     socat /tmp/sock1 -

 # echo -e "set ssl ocsp-response <<\n$(base64 ocsp.der)\n" | \
     socat /tmp/sock1 -

Signed-off-by: Aur�lien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-26 14:20:09 +02:00
Aur�lien Nephtali
25650ce513 MINOR: map: Add payload support to "add map"
It is now possible to use a payload with the "add map" command.
These syntaxes will work the same way:

 # echo "add map #-1 key value" | socat /tmp/sock1 -

 # echo -e "add map #-1 <<\n$(cat data)\n" | socat /tmp/sock1 -

with

 # cat data
 key1 value1 with spaces
 key2 value2
 key3 value3 also with spaces

Signed-off-by: Aur�lien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-26 14:20:01 +02:00
Aur�lien Nephtali
abbf607105 MEDIUM: cli: Add payload support
In order to use arbitrary data in the CLI (multiple lines or group of words
that must be considered as a whole, for example), it is now possible to add a
payload to the commands. To do so, the first line needs to end with a special
pattern: <<\n. Everything that follows will be left untouched by the CLI parser
and will be passed to the commands parsers.

Per-command support will need to be added to take advantage of this
feature.

Signed-off-by: Aur�lien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-26 14:19:33 +02:00
Willy Tarreau
daac1e4c79 DOC: lua: update the links to the config and Lua API
The links were still stuck to version 1.6. Let's update them.

The patch needs to be carefully backported to 1.8 and 1.7 after
editing the respective version (replace 1.9dev with 1.8 or 1.7).
2018-04-19 15:12:26 +02:00
William Lallemand
daf4cd209a MINOR: export localpeer as an environment variable
Export localpeer as the environment variable $HAPROXY_LOCALPEER,
allowing to use this variable in the configuration file.

It's useful to use this variable in the case of synchronized
configuration between peers.
2018-04-17 17:17:58 +02:00
Marcin Deranek
9631a28275 MEDIUM: sample: Extend functionality for field/word converters
Extend functionality of field/word converters, so it's possible
to extract field(s)/word(s) counting from the beginning/end and/or
extract multiple fields/words (including separators) eg.

str(f1_f2_f3__f5),field(2,_,2)  # f2_f3
str(f1_f2_f3__f5),field(2,_,0)  # f2_f3__f5
str(f1_f2_f3__f5),field(-2,_,3) # f2_f3_
str(f1_f2_f3__f5),field(-3,_,0) # f1_f2_f3

str(w1_w2_w3___w4),word(3,_,2)  # w3___w4
str(w1_w2_w3___w4),word(2,_,0)  # w2_w3___w4
str(w1_w2_w3___w4),word(-2,_,3) # w1_w2_w3
str(w1_w2_w3___w4),word(-3,_,0) # w1_w2

Change is backward compatible.
2018-04-17 11:27:48 +02:00
Marcin Deranek
9a66dfbd6c MINOR: proxy: Add fe_defbe fetcher
Patch adds ability to fetch frontend's default backend name in your
logic, so it can be used later to derive other backend names to make routing
decisions.
2018-04-16 15:51:57 +02:00
Christopher Faulet
caf2feca62 MINOR: spoe: Add counters to log info about SPOE agents
In addition to metrics about time spent in the SPOE, following counters have
been added:

  * applets : number of SPOE applets.
  * idles : number of idle applets.
  * nb_sending : number of streams waiting to send data.
  * nb_waiting : number of streams waiting for a ack.
  * nb_processed : number of events/groups processed by the SPOE (from the
                   stream point of view).
  * nb_errors : number of errors during the processing (from the stream point of
                view).

Log messages has been updated to report these counters. Following pattern has
been added at the end of the log message:

    ... <idles>/<applets> <nb_sending>/<nb_waiting> <nb_error>/<nb_processed>
2018-04-05 15:13:54 +02:00
Christopher Faulet
3b8e34902b MINOR: spoe: use agent's logger to log SPOE messages
Instead of using the logger of the stream, we now use dedicated logger of the
SPOE. This means a logger should be defined.
2018-04-05 15:13:54 +02:00
Christopher Faulet
0e0f085a73 MINOR: spoe: Add support for option dontlog-normal in the SPOE agent section
It does the same than for proxies.
2018-04-05 15:13:54 +02:00
Christopher Faulet
7250b8fb5c MINOR: spoe: Add loggers dedicated to the SPOE agent
Now it is possible to configure a logger in a spoe-agent section using a "log"
line, as for a proxy. "no log", "log global" and "log <address> ..." syntaxes
are supported.
2018-04-05 15:13:54 +02:00
Christopher Faulet
36bda1cd4a MINOR: spoe: Add options to store processing times in variables
"set-process-time" and "set-total-time" options have been added to store
processing times in the transaction scope, at each event and group processing,
the current one and the total one. So it is possible to get them.

TODO: documentation
2018-04-05 15:13:54 +02:00
Christopher Faulet
b2dd1e034c MINOR: spoe: Add metrics in to know time spent in the SPOE
Following metrics are added for each event or group of messages processed in the
SPOE:

  * processing time: the delay to process the event or the group. From the
                     stream point of view, it is the latency added by the SPOE
                     processing.
  * request time : It is the encoding time. It includes ACLs processing, if
                   any. For fragmented frames, it is the sum of all fragments.
  * queue time : the delay before the request gets out the sending queue. For
                 fragmented frames, it is the sum of all fragments.
  * waiting time: the delay before the reponse is received. No fragmentation
                  supported here.
  * response time: the delay to process the response. No fragmentation supported
                   here.
  * total time: (unused for now). It is the sum of all events or groups
                processed by the SPOE for a specific threads.

Log messages has been updated. Before, only errors was logged (status_code !=
0). Now every processing is logged, following this format:

  SPOE: [AGENT] <TYPE:NAME> sid=STREAM-ID st=STATUC-CODE reqT/qT/wT/resT/pT

where:

  AGENT              is the agent name
  TYPE               is EVENT of GROUP
  NAME               is the event or the group name
  STREAM-ID          is an integer, the unique id of the stream
  STATUS_CODE        is the processing's status code
  reqT/qT/wT/resT/pT are delays descrive above

For all these delays, -1 means the processing was interrupted before the end. So
-1 for the queue time means the request was never dequeued. For fragmented
frames it is harder to know when the interruption happened.

For now, messages are logged using the same logger than the backend of the
stream which initiated the request.
2018-04-05 15:13:53 +02:00
Mark Lakes
56cc12509c MINOR: lua: allow socket api settimeout to accept integers, float, and doubles
Instead of hlua_socket_settimeout() accepting only integers, allow user
to specify float and double as well. Convert to milliseconds much like
cli_parse_set_timeout but also sanity check the value.

http://w3.impa.br/~diego/software/luasocket/tcp.html#settimeout

T. Fournier edit:

The main goal is to keep compatibility with the LuaSocket API. This
API only accept seconds, so using a float to specify milliseconds is
an acceptable way.

Update doc.
2018-03-27 14:17:02 +02:00
Emmanuel Hocdet
50791a7df3 MINOR: samples: add crc32c converter
This patch adds the support of CRC32c (rfc4960).
2018-03-21 16:17:00 +01:00
Emmanuel Hocdet
4399c75f6c MINOR: proxy-v2-options: add crc32c
This patch add option crc32c (PP2_TYPE_CRC32C) to proxy protocol v2.
It compute the checksum of proxy protocol v2 header as describe in
"doc/proxy-protocol.txt".
2018-03-21 05:04:01 +01:00
Cyril Bont�
3e9548777e DOC: log: more than 2 log servers are allowed
Since commit 0f99e3497, loggers are not limited to 2 instances anymore.
2018-03-21 04:56:33 +01:00
Baptiste Assmann
1fa7d2acce BUG/MINOR: dns: don't downgrade DNS accepted payload size automatically
Automatic downgrade of DNS accepted payload size may have undesired side
effect, which could make a backend with all servers DOWN.

After talking with Lukas on the ML, I realized this "feature" introduces
more issues that it fixes problem.
The "best" way to handle properly big responses will be to implement DNS
over TCP.

To be backported to 1.8.
2018-03-19 11:41:52 +01:00
Cyril Bont�
4288c5a9d8 BUG/MINOR: force-persist and ignore-persist only apply to backends
>From the very first day of force-persist and ignore-persist features,
they only applied to backends, except that the documentation stated it
could also be applied to frontends.

In order to make it clear, the documentation is updated and the parser
will raise a warning if the keywords are used in a frontend section.

This patch should be backported up to the 1.5 branch.
2018-03-12 22:52:24 +01:00
Emmanuel Hocdet
253c3b7516 MINOR: connection: add proxy-v2-options authority
This patch add option PP2_TYPE_AUTHORITY to proxy protocol v2 when a TLS
connection was negotiated. In this case, authority corresponds to the sni.
2018-03-01 11:38:32 +01:00
Emmanuel Hocdet
fa8d0f1875 MINOR: connection: add proxy-v2-options ssl-cipher,cert-sig,cert-key
This patch implement proxy protocol v2 options related to crypto information:
ssl-cipher (PP2_SUBTYPE_SSL_CIPHER), cert-sig (PP2_SUBTYPE_SSL_SIG_ALG) and
cert-key (PP2_SUBTYPE_SSL_KEY_ALG).
2018-03-01 11:38:28 +01:00
Emeric Brun
74f7ffa229 MINOR: ssl/sample: adds ssl_bc_is_resumed fetch keyword.
Returns true when the back connection was made over an SSL/TLS transport
layer and the newly created SSL session was resumed using a cached
session or a TLS ticket.
2018-02-19 16:50:20 +01:00
Willy Tarreau
280f42b99e MINOR: sample: add a new "concat" converter
It's always a pain not to be able to combine variables. This commit
introduces the "concat" converter, which appends a delimiter, a variable's
contents and another delimiter to an existing string. The result is a string.
This makes it easier to build composite variables made of other variables.
2018-02-19 15:34:12 +01:00
Tim Duesterhus
9ad9f3517e DOC: cfgparse: Warn on option (tcp|http)log in backend
The option does not seem to have any effect since at least haproxy
1.3. Also the `log-format` directive already warns when being used
in a backend.
2018-02-19 13:57:32 +01:00
Thierry FOURNIER
c5d11c6b33 DOC: lua: new prototype for function "register_action()"
This patch should be backported to version 1.8.
2018-02-19 08:23:35 +01:00
Pavlos Parissis
f65f257871 DOC: Mention -Ws in the list of available options 2018-02-08 09:35:56 +01:00