Commit Graph

1179 Commits

Author SHA1 Message Date
Thierry FOURNIER
12ba0c29d6 MINOR: samples: rename a struct from sample_storage to sample_data
This a first step of sample reorganization.
2015-08-20 17:13:46 +02:00
Thierry FOURNIER
fd7edd3968 MINOR: Move http method enum from proto_http to sample
This is useful to prevent cross includes. The header file sample.h
needs to include proto_http, stick_tables.h will need to include
sample.h and proto_http includes stick_tables.h.

I choose to move the known http method define because this enum is
mainly used in sample.h. This enum is used for the sample type method.
2015-08-11 14:14:10 +02:00
Thierry FOURNIER
422a3af4ce MINOR: proto_tcp: add session in the action prototype
Some actions require the "struct session" while the "struct stream" is not
avalaible. This patch adds a pointer to the session.
2015-08-11 14:08:29 +02:00
Thierry FOURNIER
a6b6343cff CLEANUP: http/tcp actions: remove the scope member
The scope member is not used. This patch removes this entry.
2015-08-11 13:44:53 +02:00
Willy Tarreau
3580b03fc0 CLEANUP: proxy: remove last references to appsession
The code entirely builds without using these elements, let's get rid
of them now.
2015-08-10 19:42:30 +02:00
Andrew Hayworth
e63ac871f8 MINOR: log: Add log-format variable %HQ, to log HTTP query strings
Since sample fetches are not always available in the response phase,
this patch implements %HQ such that:

  GET /foo?bar=baz HTTP/1.0

...would be logged as:

  ?bar=baz
2015-08-09 10:16:49 +02:00
Baptiste Assmann
9f5ada32e4 MINOR: server: add new SRV_ADMF_CMAINT flag
The purpose of SRV_ADMF_CMAINT flag is to keep in mind the server was
forced to maintenance status because of the configuration file.
2015-08-08 18:18:17 +02:00
Willy Tarreau
7017cb040c MINOR: server: add a list of safe, already reused idle connections
These ones are considered safe as they have already been reused.
They will be useful in "aggressive" and "always" http-reuse modes
in order to place the first request of a connection with the least
risk.
2015-08-06 16:29:01 +02:00
Willy Tarreau
d8fecee291 MINOR: config: add new setting "http-reuse"
For now it only supports "never", meaning that we never want to reuse a
shared connection, and "always", meaning that we can use any connection
that was not marked private. When "never" is set, this also implies that
no idle connection may become a shared one.
2015-08-06 11:14:20 +02:00
Willy Tarreau
387ebf84dd MINOR: connection: add a new flag CO_FL_PRIVATE
This flag is set on an outgoing connection when this connection gets
some properties that must not be shared with other connections, such
as dynamic transparent source binding, SNI or a proxy protocol header,
or an authentication challenge from the server. This will be needed
later to implement connection reuse.
2015-08-06 11:14:17 +02:00
Willy Tarreau
173a1c6b43 MINOR: server: add a list of already used idle connections
There's a difference with the other idle conns in that these new
ones have already been used and may be reused by other streams.
2015-08-06 11:13:47 +02:00
Willy Tarreau
d75d40e9a8 MINOR: connection: add a new list member in the connection struct
This list member will be used to attach a connection to a list of
idle, reusable or queued connections. It's unused for now. Given
that it's not expected to be used more than a few times per session,
the member was put after the target, in the area starting at the
second cache line of the structure.
2015-08-06 11:00:34 +02:00
Willy Tarreau
600802aef0 MINOR: server: add a list of private idle connections
For now it's not populated but we have the list entry. It will carry
all idle connections that sessions don't want to share. They may be
used later to reclaim connections upon socket shortage for example.
2015-08-06 10:59:08 +02:00
Thierry FOURNIER
bf65cd4d77 MAJOR: arg: converts uint and sint in sint
This patch removes the 32 bits unsigned integer and the 32 bit signed
integer. It replaces these types by a unique type 64 bit signed.
2015-07-22 00:48:23 +02:00
Thierry FOURNIER
07ee64ef4d MAJOR: sample: converts uint and sint in 64 bits signed integer
This patch removes the 32 bits unsigned integer and the 32 bit signed
integer. It replaces these types by a unique type 64 bit signed.

This makes easy the usage of integer and clarify signed and unsigned use.
With the previous version, signed and unsigned are used ones in place of
others, and sometimes the converter loose the sign. For example, divisions
are processed with "unsigned", if one entry is negative, the result is
wrong.

Note that the integer pattern matching and dotted version pattern matching
are already working with signed 64 bits integer values.

There is one user-visible change : the "uint()" and "sint()" sample fetch
functions which used to return a constant integer have been replaced with
a new more natural, unified "int()" function. These functions were only
introduced in the latest 1.6-dev2 so there's no impact on regular
deployments.
2015-07-22 00:48:23 +02:00
Baptiste Assmann
7cc419ae1d MINOR: server: new server flag: SRV_F_FORCED_ID
This flag aims at reporting whether the server unique id (srv->puid) has
been forced by the administrator in HAProxy's configuration.
If not set, it means HAProxy has generated automatically the server's
unique id.
2015-07-21 23:24:16 +02:00
Baptiste Assmann
8a027ccb38 MINOR: proxy: bit field for proxy_find_best_match diff status
function proxy_find_best_match can update the caller by updating an int
provided in argument.
For now, proxy_find_best_match hardcode bit values 0x01, 0x02 and 0x04,
which is not understandable when reading a code exploiting them.

This patch defines 3 macros with a more explicit wording, so further
reading of a code exploiting the magic bit values will be understandable
more easily.
2015-07-21 23:24:16 +02:00
Thierry FOURNIER
25f4e3ef33 CLEANUP: vars: remove unused struct
The commit "MEDIUM: vars: move the session variables to the session, not the stream" (ebcd4844e82a4198ea5d98fe491a46267da1d1ec")
moves the variables from the stream to the session. It forgot to remove
the stream definition of the "vars_sess".
2015-07-10 16:30:08 +02:00
Willy Tarreau
732eac41f4 MEDIUM: ssl: add sni support on the server lines
The new "sni" server directive takes a sample fetch expression and
uses its return value as a hostname sent as the TLS SNI extension.
A typical use case consists in forwarding the front connection's SNI
value to the server in a bridged HTTPS forwarder :

   sni ssl_fc_sni
2015-07-10 11:43:15 +02:00
Willy Tarreau
28d976d5ee MINOR: args: add new context for servers
We'll have to support fetch expressions and args on server lines for
"usesrc", "usedst", "sni", etc...
2015-07-09 11:39:33 +02:00
Adis Nezirovic
2fbcafc9ce MEDIUM: http: Add new 'set-src' option to http-request
This option enables overriding source IP address in a HTTP request. It is
useful when we want to set custom source IP (e.g. front proxy rewrites address,
but provides the correct one in headers) or we wan't to mask source IP address
for privacy or compliance.

It acts on any expression which produces correct IP address.
2015-07-06 16:17:28 +02:00
Dragan Dosen
105c8e6368 MEDIUM: 51d: add LRU-based cache on User-Agent string detection
This cache is used by 51d converter. The input User-Agent string, the
converter args and a random seed are used as a hashing key. The cached
entries contains a pointer to the resulting string for specific
User-Agent string detection.

The cache size can be tuned using 51degrees-cache-size parameter.
2015-06-30 10:43:03 +02:00
Dragan Dosen
93b38d9191 MEDIUM: 51Degrees code refactoring and cleanup
Moved 51Degrees code from src/haproxy.c, src/sample.c and src/cfgparse.c
into a separate files src/51d.c and include/import/51d.h.

Added two new functions init_51degrees() and deinit_51degrees(), updated
Makefile and other code reorganizations related to 51Degrees.
2015-06-30 10:43:03 +02:00
Willy Tarreau
ebcd4844e8 MEDIUM: vars: move the session variables to the session, not the stream
It's important that the session-wide variables are in the session and not
in the stream.
2015-06-19 11:59:02 +02:00
Emeric Brun
aaf5860fd6 MINOR: peers: avoid re-scheduling of pending stick-table's updates still not pushed. 2015-06-16 16:11:12 +02:00
Thierry FOURNIER
4834bc773c MEDIUM: vars: adds support of variables
This patch adds support of variables during the processing of each stream. The
variables scope can be set as 'session', 'transaction', 'request' or 'response'.
The variable type is the type returned by the assignment expression. The type
can change while the processing.

The allocated memory can be controlled for each scope and each request, and for
the global process.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
a9ff994461 MINOR: build: fix build dependency
fix include dependency. The header file sample.h don't need to known
the content of the struct arg, so I remove the include, and replace
it by a simple pointer declaration.

This prevent an include dependecy issue with the next patch.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
0e11863a6f MINOR: tcp/http/conf: extends the keyword registration options
This patch permits to register a new keyword with the keyword "tcp-request content"
'tcp-request connection", tcp-response content", http-request" and "http-response"
which is identified only by matching the start of the keyword.

for example, we register the keyword "set-var" with the option "match_pfx"
and the configuration keyword "set-var(var_name)" matchs this entry.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
fbdb77582d MINOR: tcp: increase the opaque data array
This patch increase the opaque data array for the tcp_rules.
It is used by the "store" action (next commited) which deal
with variables.
2015-06-13 23:01:37 +02:00
Thierry FOURNIER
9c627e84b2 MEDIUM: sample: Add type any
This type is used to accept any type of sample as input, and prevent
any automatic "cast". It runs like the type "ADDR" which accept the
type "IPV4" and "IPV6".
2015-06-13 22:59:14 +02:00
Baptiste Assmann
3863f97349 MINOR: dns: add DNS statistics
add a new command on the stats socket to print a DNS resolvers section
(including per server) statistics: "show stats resolvers <id>"
2015-06-13 22:07:35 +02:00
Baptiste Assmann
a68ca96375 MAJOR: server: add DNS-based server name resolution
Relies on the DNS protocol freshly implemented in HAProxy.
It performs a server IP addr resolution based on a server hostname.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
325137d603 MEDIUM: dns: implement a DNS resolver
Implementation of a DNS client in HAProxy to perform name resolution to
IP addresses.

It relies on the freshly created UDP client to perform the DNS
resolution. For now, all UDP socket calls are performed in the
DNS layer, but this might change later when the protocols are
extended to be more suited to datagram mode.

A new section called 'resolvers' is introduced thanks to this patch. It
is used to describe DNS servers IP address and also many parameters.
2015-06-13 22:07:35 +02:00
Baptiste Assmann
5d4e4f7a57 MEDIUM: protocol: add minimalist UDP protocol client
Basic introduction of a UDP layer in HAProxy. It can be used as a
client only and manages UDP exchanges with servers.

It can't be used to load-balance UDP protocols, but only used by
internal features such as DNS resolution.
2015-06-13 22:07:35 +02:00
Christopher Faulet
31af49d62b MEDIUM: ssl: Add options to forge SSL certificates
With this patch, it is possible to configure HAProxy to forge the SSL
certificate sent to a client using the SNI servername. We do it in the SNI
callback.

To enable this feature, you must pass following BIND options:

 * ca-sign-file <FILE> : This is the PEM file containing the CA certitifacte and
   the CA private key to create and sign server's certificates.

 * (optionally) ca-sign-pass <PASS>: This is the CA private key passphrase, if
   any.

 * generate-certificates: Enable the dynamic generation of certificates for a
   listener.

Because generating certificates is expensive, there is a LRU cache to store
them. Its size can be customized by setting the global parameter
'tune.ssl.ssl-ctx-cache-size'.
2015-06-12 18:06:59 +02:00
Thomas Holmes
4d441a759c MEDIUM: sample: add trie support to 51Degrees
Trie or pattern algorithm is used depending on what 51Degrees source
files are provided to MAKE.
2015-06-02 19:30:53 +02:00
Thomas Holmes
eef3f9fbd3 MINOR: global: add several 51Degrees members to global
These are :
  fiftyoneDegreesDataSet _51d_data_set
  char *_51d_data_file_path
  char _51d_property_seperator
  struct list _51d_property_names
2015-06-02 14:00:22 +02:00
Willy Tarreau
ba3dd2b7e0 CLEANUP: global: remove one ifdef USE_DEVICEATLAS
The include file already has the ifdef, let's remove it from the global
file.
2015-06-02 13:42:12 +02:00
David Carlier
88d13cbf4d MEDIUM: global: add the DeviceAtlas required elements to struct global
This diff is the raw C struct definition of all DeviceAtlas module
data needed added to the main global struct haproxy configuration.
The three first members are needed for both init and deinit phases
as some dynamic memory allocations are done.
The useragentid serves to hold during the whole lifecycle of the
module the User-Agent HTTP Header identifier from the DeviceAtlas
data during the init process.
2015-06-02 13:24:49 +02:00
Thierry FOURNIER
561a0f989d MINOR: tcp: add custom actions that can continue tcp-(request|response) processing
Actually, the tcp-request and tcp-response custom ation are always final
actions. This patch create a new type of action that can permit to
continue the evaluation of tcp-request and tcp-response processing.
2015-05-29 17:49:48 +02:00
Emeric Brun
b3971ab062 MAJOR: peers: peers protocol version 2.0
This patch does'nt add any new feature: the functional behavior
is the same than version 1.0.

Technical differences:

In this version all updates on different stick tables are
multiplexed on the same tcp session. There is only one established
tcp session per peer whereas in first version there was one established
tcp session per peer and per stick table.

Messages format was reviewed to be more evolutive and to support
further types of data exchange such as SSL sessions or other sticktable's
data types (currently only the sticktable's server id is supported).
2015-05-29 15:50:33 +02:00
Willy Tarreau
51d861a44f MEDIUM: http: implement http-response redirect rules
Sometimes it's problematic not to have "http-response redirect" rules,
for example to perform a browser-based redirect based on certain server
conditions (eg: match of a header).

This patch adds "http-response redirect location <fmt>" which gives
enough flexibility for most imaginable operations. The connection to
the server is closed when this is performed so that we don't risk to
forward any pending data from the server.

Any pending response data are trimmed so that we don't risk to
forward anything pending to the client. It's harmless to also do that
for requests so we don't need to consider the direction.
2015-05-28 17:45:43 +02:00
Thierry FOURNIER
cdb67cad01 MINOR: http: add array of generic pointers in http_res_rules
This patch adds an array of generic pointers in the http_res_rules
struct. It is used later by the "response capture" functions.
2015-05-28 13:50:29 +02:00
Willy Tarreau
0d1fdf7df5 MINOR: proxy: add a flag to memorize that the proxy's ID was forced
This will be used to know if proxy's ID should be considered when names
mismatch upon check status reload.
2015-05-27 16:51:28 +02:00
Willy Tarreau
98d0485a90 MAJOR: config: remove the deprecated reqsetbe / reqisetbe actions
These ones were already obsoleted in 1.4, marked for removal in 1.5,
and not documented anymore. They used to emit warnings, and do still
require quite some code to stay in place. Let's remove them now.
2015-05-26 12:18:29 +02:00
Joseph Lynch
726ab7145c MEDIUM: backend: Allow redispatch on retry intervals
For backend load balancing it sometimes makes sense to redispatch rather
than retrying against the same server. For example, when machines or routers
fail you may not want to waste time retrying against a dead server and
would instead prefer to immediately redispatch against other servers.

This patch allows backend sections to specify that they want to
redispatch on a particular interval. If the interval N is positive the
redispatch occurs on every Nth retry, and if the interval N is negative then
the redispatch occurs on the Nth retry prior to the last retry (-1 is the
default and maintains backwards compatibility). In low latency environments
tuning this setting can save a few hundred milliseconds when backends fail.
2015-05-22 07:07:40 +02:00
Nenad Merdanovic
200b0facde MEDIUM: Add support for updating TLS ticket keys via socket
Until now, HAproxy needed to be restarted to change the TLS ticket
keys. With this patch, the TLS keys can be updated on a per-file
basis using the admin socket. Two new socket commands have been
introduced: "show tls-keys" and "set ssl tls-keys".

Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-05-16 11:28:04 +02:00
Nenad Merdanovic
146defaff4 MINOR: Add TLS ticket keys reference and use it in the listener struct
Within the listener struct we need to use a reference to the TLS
ticket keys which binds the actual keys with the filename. This will
make it possible to update the keys through the socket

Signed-off-by: Nenad Merdanovic <nmerdan@anine.io>
2015-05-16 11:28:04 +02:00
Baptiste Assmann
22b09d2393 MINOR: include comment in tcpcheck error log
tcpcheck error messages include the step id where the error occurs.
In some cases, this is not enough. Now, HAProxy also use the comment
field of the latest tcpcheck rule which has been run.
This commit allows HAProxy to parse a new directive in the tcpcheck
ruleset: 'comment'.
It is used to setup comments on the current tcpcheck rules.
2015-05-12 11:04:39 +02:00
Baptiste Assmann
aa12b47b06 MINOR: tcpcheck_rule structure update
A new field is added into the tcpcheck_rule structure.
This field will host a string used as a comment to describe the rule.
Then this comment can be used in logs to report a more user friendly
message on the step which failed during the tcpcheck ruleset.
2015-05-12 10:44:49 +02:00