1
0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2025-01-20 12:40:46 +00:00
Commit Graph

295 Commits

Author SHA1 Message Date
Cyril Bont�
474be415af [MEDIUM] stats: add an admin level
The stats web interface must be read-only by default to prevent security
holes. As it is now allowed to enable/disable servers, a new keyword
"stats admin" is introduced to activate this admin level, conditioned by ACLs.
(cherry picked from commit 5334bab92ca7debe36df69983c19c21b6dc63f78)
2010-10-30 19:04:34 +02:00
Cyril Bont�
70be45dbdf [MEDIUM] enable/disable servers from the stats web interface
Based on a patch provided by Judd Montgomery, it is now possible to
enable/disable servers from the stats web interface. This allows to select
several servers in a backend and apply the action to them at the same time.

Currently, there are 2 known limitations :
- The POST data are limited to one packet
  (don't alter too many servers at a time).
- Expect: 100-continue is not supported.
(cherry picked from commit 7693948766cb5647ac03b48e782cfee2b1f14491)
2010-10-30 19:04:34 +02:00
Willy Tarreau
74b08c9ab7 [MEDIUM] buffers: rework the functions to exchange between SI and buffers
There was no consistency between all the functions used to exchange data
between a buffer and a stream interface. Also, the functions used to send
data to a buffer did not consider the possibility that the buffer was
shutdown for read.

Now the functions are called buffer_{put,get}_{char,block,chunk,string}.

The old buffer_feed* functions have been left available for existing code
but marked deprecated.
2010-09-08 17:04:31 +02:00
Willy Tarreau
d0807c3c60 [MEDIUM] signals: support redistribution of signal zero when stopping
Signal zero is never delivered by the system. However having a signal to
which functions and tasks can subscribe to be notified of a stopping event
is useful. So this patch does two things :
  1) allow signal zero to be delivered from any function of signal handler
  2) make soft_stop() deliver this signal so that tasks can be notified of
     a stopping condition.
2010-08-27 18:26:11 +02:00
Willy Tarreau
24f4efa670 [MEDIUM] signals: add support for registering functions and tasks
The two new functions below make it possible to register any number
of functions or tasks to a system signal. They will be called in the
registration order when the signal is received.

    struct sig_handler *signal_register_fct(int sig, void (*fct)(struct sig_handler *), int arg);
    struct sig_handler *signal_register_task(int sig, struct task *task, int reason);
2010-08-27 18:00:40 +02:00
Willy Tarreau
0a4838cd31 [MEDIUM] session-counters: correctly unbind the counters tracked by the backend
In case of HTTP keepalive processing, we want to release the counters tracked
by the backend. Till now only the second set of counters was released, while
it could have been assigned by the frontend, or the backend could also have
assigned the first set. Now we reuse to unused bits of the session flags to
mark which stick counters were assigned by the backend and to release them as
appropriate.
2010-08-10 18:04:16 +02:00
Willy Tarreau
56123282ef [MINOR] session-counters: use "track-sc{1,2}" instead of "track-{fe,be}-counters"
The assumption that there was a 1:1 relation between tracked counters and
the frontend/backend role was wrong. It is perfectly possible to track the
track-fe-counters from the backend and the track-be-counters from the
frontend. Thus, in order to reduce confusion, let's remove this useless
{fe,be} reference and simply use {1,2} instead. The keywords have also been
renamed in order to limit confusion. The ACL rule action now becomes
"track-sc{1,2}". The ACLs are now "sc{1,2}_*" instead of "trk{fe,be}_*".

That means that we can reasonably document "sc1" and "sc2" (sticky counters
1 and 2) as sort of patterns that are available during the whole session's
life and use them just like any other pattern.
2010-08-10 18:04:15 +02:00
Willy Tarreau
f6efda1189 [MEDIUM] session counters: automatically remove expired entries.
When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-10 18:04:15 +02:00
Willy Tarreau
f059a0f63a [MAJOR] session-counters: split FE and BE track counters
Having a single tracking pointer for both frontend and backend counters
does not work. Instead let's have one for each. The keyword has changed
to "track-be-counters" and "track-fe-counters", and the ACL "trk_*"
changed to "trkfe_*" and "trkbe_*".
2010-08-10 18:04:15 +02:00
Willy Tarreau
3b9c6e053e [MEDIUM] stick-table: make use of generic types for stored data
It's a bit cumbersome to have to know all possible storable types
from the stats interface. Instead, let's have generic types for
all data, which will facilitate their manipulation.
2010-08-10 18:04:14 +02:00
Willy Tarreau
69f58c8058 [MEDIUM] stats: add "show table [<name>]" to dump a stick-table
It is now possible to dump a table's contents with keys, expire,
use count, and various data using the command above on the stats
socket.

"show table" only shows main table stats, while "show table <name>"
dumps table contents, only if the socket level is admin.
2010-08-10 18:04:14 +02:00
Willy Tarreau
da7ff64aa9 [MEDIUM] session-counters: add HTTP req/err tracking
This patch adds support for the following session counters :
  - http_req_cnt : HTTP request count
  - http_req_rate: HTTP request rate
  - http_err_cnt : HTTP request error count
  - http_err_rate: HTTP request error rate

The equivalent ACLs have been added to check the tracked counters
for the current session or the counters of the current source.
2010-08-10 18:04:14 +02:00
Willy Tarreau
1f7e925d6a [MINOR] stktable: add a stktable_update_key() function
This function looks up a key, updates its expiration date, or creates
it if it was not found. acl_fetch_src_updt_conn_cnt() was updated to
make use of it.
2010-08-10 18:04:14 +02:00
Willy Tarreau
91c43d7fe4 [MEDIUM] session counters: add conn_rate and sess_rate counters
These counters maintain incoming connection rates and session rates
in a stick-table, over a period which is defined in the configuration
(2 ms to 24 days). They can be used to detect service abuse and
enforce a certain accept rate per source address for instance, and
block if the rate is passed over.

Example :
	# block if more than 50 requests per 5 seconds from a source.
	stick-table type ip size 200k expire 1m store conn_rate(5s),sess_rate(5s)
	tcp-request track-counters src
	tcp-request reject if { trk_conn_rate gt 50 }

	# cause a 3 seconds pause to requests from sources in excess of 20 requests/5s
	tcp-request inspect-delay 3s
	tcp-request content accept if { trk_sess_rate gt 20 } WAIT_END
2010-08-10 18:04:13 +02:00
Willy Tarreau
ac78288eaf [MEDIUM] stick-tables: add stored data argument type checking
We're now able to return errors based on the validity of an argument
passed to a stick-table store data type. We also support ARG_T_DELAY
to pass delays to stored data types (eg: for rate counters).
2010-08-10 18:04:13 +02:00
Willy Tarreau
888617dc3b [MEDIUM] stick-tables: add support for arguments to data_types
Some data types will require arguments (eg: period for a rate counter).
This patch adds support for such arguments between parenthesis in the
"store" directive of the stick-table statement. Right now only integers
are supported.
2010-08-10 18:04:13 +02:00
Willy Tarreau
e348793696 [MEDIUM] session-counters: automatically update tracked connection count
When a session tracks a counter, automatically increase the cumulated
connection count. This makes src_updt_conn_cnt() almost useless. In
fact it might still be used to update different tables.
2010-08-10 18:04:12 +02:00
Willy Tarreau
38285c18f4 [MEDIUM] session: add concurrent connections counter
The new "conn_cur" session counter has been added. It is automatically
updated upon "track XXX" directives, and the entry is touched at the
moment we increment the value so that we don't consider further counter
updates as real updates, otherwise we would end up updating upon completion,
which may not be desired. Probably that some other event counters (eg: HTTP
requests) will have to be updated upon each event though.

This counter can be matched against current session's source address using
the "src_conn_cur" ACL.
2010-08-10 18:04:12 +02:00
Willy Tarreau
4a0347add0 [MINOR] stick-table: provide a table lookup function
We'll often need to lookup a table by its name. This will change
in the future once we can resolve these names on startup.
2010-08-10 18:04:12 +02:00
Willy Tarreau
9ba2dcc86c [MAJOR] session: add track-counters to track counters related to the session
This patch adds the ability to set a pointer in the session to an
entry in a stick table which holds various counters related to a
specific pattern.

Right now the syntax matches the target syntax and only the "src"
pattern can be specified, to track counters related to the session's
IPv4 source address. There is a special function to extract it and
convert it to a key. But the goal is to be able to later support as
many patterns as for the stick rules, and get rid of the specific
function.

The "track-counters" directive may only be set in a "tcp-request"
statement right now. Only the first one applies. Probably that later
we'll support multi-criteria tracking for a single session and that
we'll have to name tracking pointers.

No counter is updated right now, only the refcount is. Some subsequent
patches will have to bring that feature.
2010-08-10 18:04:12 +02:00
Willy Tarreau
591fedc2c3 [MEDIUM] buffer: make buffer_feed* support writing non-contiguous chunks
The buffer_feed* functions that are used to send data to buffers did only
support sending contiguous chunks while they're relying on memcpy(). This
patch improves on this by making them able to write in two chunks if needed.
Thus, the buffer_almost_full() function has been improved to really consider
the remaining space and not just what can be written at once.
2010-08-10 17:48:57 +02:00
Willy Tarreau
2970b0bedf [MINOR] freq_ctr: add new types and functions for periods different from 1s
Some freq counters will have to work on periods different from 1 second.
The original freq counters rely on the period to be exactly one second.
The new ones (freq_ctr_period) let the user define the period in ticks,
and all computations are operated over that period. When reading a value,
it indicates the amount of events over that period too.
2010-08-10 14:01:09 +02:00
Willy Tarreau
cb18364ca7 [MEDIUM] stick_table: separate storage and update of session entries
When an entry already exists, we just need to update its expiration
timer. Let's have a dedicated function for that instead of spreading
open code everywhere.

This change also ensures that an update of an existing sticky session
really leads to an update of its expiration timer, which was apparently
not the case till now. This point needs to be checked in 1.4.
2010-06-14 15:10:26 +02:00
Willy Tarreau
41883e2041 [MINOR] stick_table: export the stick_table_key
This one is huge and will be needed by other portions of code for various
data lookups. Let's not have them allocate it in the stack.
2010-06-14 15:10:25 +02:00
Willy Tarreau
68129b90eb [MINOR] stick_table: provide functions to return stksess data from a type
This function does the indirection job in the table to find the pointer
to the real data matching the requested type.
2010-06-14 15:10:25 +02:00
Willy Tarreau
f16d2b8c1b [MEDIUM] stick_table: don't overwrite data when storing an entry
Till now sticky sessions only held server IDs. Now there are other
data types so it is not acceptable anymore to overwrite the server ID
when writing something. The server ID must then only be written from
the caller when appropriate. Doing this has also led to separate
lookup and storage.
2010-06-14 15:10:24 +02:00
Willy Tarreau
08d5f98294 [MEDIUM] stick_table: add room for extra data types
The stick_tables will now be able to store extra data for a same key.
A limited set of extra data types will be defined and for each of them
an offset in the sticky session will be assigned at startup time. All
of this information will be stored in the stick table.

The extra data types will have to be specified after the new "store"
keyword of the "stick-table" directive, which will reserve some space
for them.
2010-06-14 15:10:24 +02:00
Willy Tarreau
f0b38bfc33 [CLEANUP] stick_table: move pattern to key functions to stick_table.c
pattern.c depended on stick_table while in fact it should be the opposite.
So we move from pattern.c everything related to stick_tables and invert the
dependency. That way the code becomes more logical and intuitive.
2010-06-14 15:10:24 +02:00
Willy Tarreau
393379c3e0 [MINOR] stick_table: add support for variable-sized data
Right now we're only able to store a server ID in a sticky session.
The goal is to be able to store anything whose size is known at startup
time. For this, we store the extra data before the stksess pointer,
using a negative offset. It will then be easy to cumulate multiple
data provided they each have their own offset.
2010-06-14 15:10:23 +02:00
Willy Tarreau
b36b4244a2 [MINOR] session: differenciate between accepted connections and received connections
Now we're able to reject connections very early, so we need to use a
different counter for the connections that are received and the ones
that are accepted and converted into sessions, so that the rate limits
can still apply to the accepted ones. The session rate must still be
used to compute the rate limit, so that we can reject undesired traffic
without affecting the rate.
2010-06-14 10:53:19 +02:00
Willy Tarreau
decd14d298 [MEDIUM] stats: rely on the standard session_accept() function
The stats' accept() function is now ridiculously small. It could
even be reduced by moving some parts to the common accept code.
2010-06-14 10:53:18 +02:00
Willy Tarreau
81f9aa3bf2 [MAJOR] frontend: split accept() into frontend_accept() and session_accept()
A new function session_accept() is now called from the lower layer to
instanciate a new session. Once the session is instanciated, the upper
layer's frontent_accept() is called. This one can be service-dependant.

That way, we have a 3-phase accept() sequence :
  1) protocol-specific, session-less accept(), which is pointed to by
     the listener. It defaults to the generic stream_sock_accept().
  2) session_accept() which relies on a frontend but not necessarily
     for use in a proxy (eg: stats or any future service).
  3) frontend_accept() which performs the accept for the service
     offerred by the frontend. It defaults to frontend_accept() which
     is really what is used by a proxy.

The TCP/HTTP proxies have been moved to this mode so that we can now rely on
frontend_accept() for any type of session initialization relying on a frontend.

The next step will be to convert the stats to use the same system for the stats.
2010-06-14 10:53:17 +02:00
Willy Tarreau
a8f55d5473 [MEDIUM] backend: initialize the server stream_interface upon connect()
It's not normal to initialize the server-side stream interface from the
accept() function, because it may change later. Thus, we introduce a new
stream_sock_prepare_interface() function which is called just before the
connect() and which sets all of the stream_interface's callbacks to the
default ones used for real sockets. The ->connect function is also set
at the same instant so that we can easily add new server-side protocols
soon.
2010-06-14 10:53:15 +02:00
Willy Tarreau
a5c0ab200b [MEDIUM] frontend: check for LI_O_TCP_RULES in the listener
The new LI_O_TCP_RULES listener option indicates that some TCP rules
must be checked upon accept on this listener. It is now checked by
the frontend and the L4 rules are evaluated only in this case. The
flag is only set when at least one tcp-req rule is present in the
frontend.

The L4 rules check function has now been moved to proto_tcp.c where
it ought to be.
2010-06-14 10:53:13 +02:00
Willy Tarreau
eb472685cb [MEDIUM] separate protocol-level accept() from the frontend's
For a long time we had two large accept() functions, one for TCP
sockets instanciating proxies, and another one for UNIX sockets
instanciating the stats interface.

A lot of code was duplicated and both did not work exactly the same way.

Now we have a stream_sock layer accept() called for either TCP or UNIX
sockets, and this function calls the frontend-specific accept() function
which does the rest of the frontend-specific initialisation.

Some code is still duplicated (session & task allocation, stream interface
initialization), and might benefit from having an intermediate session-level
accept() callback to perform such initializations. Still there are some
minor differences that need to be addressed first. For instance, the monitor
nets should only be checked for proxies and not for other connection templates.

Last, we renamed l->private as l->frontend. The "private" pointer in
the listener is only used to store a frontend, so let's rename it to
eliminate this ambiguity. When we later support detached listeners
(eg: FTP), we'll add another field to avoid the confusion.
2010-06-14 10:53:11 +02:00
Willy Tarreau
03fa5df64a [CLEANUP] rename client -> frontend
The 'client.c' file now only contained frontend-specific functions,
so it has naturally be renamed 'frontend.c'. Same for client.h. This
has also been an opportunity to remove some cross references from
files that should not have depended on it.

In the end, this file should contain a protocol-agnostic accept()
code, which would initialize a session, task, etc... based on an
accept() from a lower layer. Right now there are still references
to TCP.
2010-06-14 10:53:10 +02:00
Willy Tarreau
44b90cc4d8 [CLEANUP] tcp: move some non tcp-specific layer6 processing out of proto_tcp
Some functions which act on generic buffer contents without being
tcp-specific were historically in proto_tcp.c. This concerns ACLs
and RDP cookies. Those have been moved away to more appropriate
locations. Ideally we should create some new files for each layer6
protocol parser. Let's do that later.
2010-06-14 10:53:09 +02:00
Willy Tarreau
bce7088275 [MEDIUM] add ability to connect to a server from an IP found in a header
Using get_ip_from_hdr2() we can look for occurrence #X or #-X and
extract the IP it contains. This is typically designed for use with
the X-Forwarded-For header.

Using "usesrc hdr_ip(name,occ)", it becomes possible to use the IP address
found in <name>, and possibly specify occurrence number <occ>, as the
source to connect to a server. This is possible both in a server and in
a backend's source statement. This is typically used to use the source
IP previously set by a upstream proxy.
2010-03-30 10:39:43 +02:00
Willy Tarreau
b1d67749db [MEDIUM] backend: move the transparent proxy address selection to backend
The transparent proxy address selection was set in the TCP connect function
which is not the most appropriate place since this function has limited
access to the amount of parameters which could produce a source address.

Instead, now we determine the source address in backend.c:connect_server(),
right after calling assign_server_address() and we assign this address in
the session and pass it to the TCP connect function. This cannot be performed
in assign_server_address() itself because in some cases (transparent mode,
dispatch mode or http_proxy mode), we assign the address somewhere else.

This change will open the ability to bind to addresses extracted from many
other criteria (eg: from a header).
2010-03-30 09:59:43 +02:00
Willy Tarreau
e45997661b [MEDIUM] session: better fix for connection to servers with closed input
The following patch fixed an issue but brought another one :
  296897 [MEDIUM] connect to servers even when the input has already been closed

The new issue is that when a connection is inspected and aborted using
TCP inspect rules, now it is sent to the server before being closed. So
that test is not satisfying. A probably better way is not to prevent a
connection from establishing if only BF_SHUTW_NOW is set but BF_SHUTW
is not. That way, the BF_SHUTW flag is not set if the request has any
data pending, which still fixes the stats issue, but does not let any
empty connection pass through.

Also, as a safety measure, we extend buffer_abort() to automatically
disable the BF_AUTO_CONNECT flag. While it appears to always be OK,
it is by pure luck, so better safe than sorry.
2010-03-21 23:31:42 +01:00
Willy Tarreau
d9b587f260 [STATS] report HTTP requests (total and rate) in frontends
Now that we support keep-alive, it's important to report a separate
counter for requests. Right now it just appears in the CSV output.
2010-02-26 10:05:55 +01:00
Willy Tarreau
b97f199d4b [MEDIUM] http: don't use trash to realign large buffers
The trash buffer may now be smaller than a buffer because we can tune
it at run time. This causes a risk when we're trying to use it as a
temporary buffer to realign unaligned requests, because we may have to
put up to a full buffer into it.

Instead of doing a double copy, we're now relying on an open-coded
bouncing copy algorithm. The principle is that we move one byte at
a time to its final place, and if that place also holds a byte, then
we move it too, and so on. We finish when we've moved all the buffer.
It limits the number of memory accesses, but since it proceeds one
byte at a time and with random walk, it's not cache friendly and
should be slower than a double copy. However, it's only used in
extreme situations and the difference will not be noticeable.

It has been extensively tested and works reliably.
2010-02-25 23:54:31 +01:00
Willy Tarreau
9cc670f7d9 [CLEANUP] config: use build_acl_cond() to simplify http-request ACL parsing
Now that we have this new function to make your life better, use it.
2010-02-01 10:43:44 +01:00
Cyril Bont�
cd19e51b05 [MEDIUM] add a maintenance mode to servers
This is a first attempt to add a maintenance mode on servers, using
the stat socket (in admin level).

It can be done with the following command :
   - disable server <backend>/<server>
   - enable  server <backend>/<server>

In this mode, no more checks will be performed on the server and it
will be marked as a special DOWN state (MAINT).

If some servers were tracking it, they'll go DOWN until the server
leaves the maintenance mode. The stats page and the CSV export also
display this special state.

This can be used to disable the server in haproxy before doing some
operations on this server itself. This is a good complement to the
"http-check disable-on-404" keyword and works in TCP mode.
2010-01-31 23:33:18 +01:00
Krzysztof Piotr Oledzki
8c8bd4593c [MAJOR] use the new auth framework for http stats
Support the new syntax (http-request allow/deny/auth) in
http stats.

Now it is possible to use the same syntax is the same like in
the frontend/backend http-request access control:
 acl src_nagios src 192.168.66.66
 acl stats_auth_ok http_auth(L1)

 stats http-request allow if src_nagios
 stats http-request allow if stats_auth_ok
 stats http-request auth realm LB

The old syntax is still supported, but now it is emulated
via private acls and an aditional userlist.
2010-01-31 19:14:09 +01:00
Krzysztof Piotr Oledzki
f9423ae43a [MINOR] acl: add http_auth and http_auth_group
Add two acls to match http auth data:
 acl <name> http_auth(userlist)
 acl <name> http_auth_hroup(userlist) group1 group2 (...)
2010-01-31 19:14:09 +01:00
Krzysztof Piotr Oledzki
961050465e [MINOR] generic auth support with groups and encrypted passwords
Add generic authentication & authorization support.

Groups are implemented as bitmaps so the count is limited to
sizeof(int)*8 == 32.

Encrypted passwords are supported with libcrypt and crypt(3), so it is
possible to use any method supported by your system. For example modern
Linux/glibc instalations support MD5/SHA-256/SHA-512 and of course classic,
DES-based encryption.
2010-01-31 19:14:07 +01:00
Willy Tarreau
fdb563c06f [MEDIUM] http: add support for conditional response header rewriting
Just as for the req* rules, we can now condition rsp* rules with ACLs.
ACLs match on response, so volatile request information cannot be used.
A warning is emitted if a configuration contains such an anomaly.
2010-01-31 15:43:27 +01:00
Willy Tarreau
6c123b15cb [MEDIUM] http: make the request filter loop check for optional conditions
From now on, if request filters have ACLs defined, these ACLs will be
evaluated to condition the filter. This will be used to conditionally
remove/rewrite headers based on ACLs.
2010-01-28 20:22:06 +01:00
Willy Tarreau
f1e98b8628 [CLEANUP] config: use warnif_cond_requires_resp() to check for bad ACLs
Factor out some repetitive copy-pasted code to check for request ACLs
validity.
2010-01-28 17:59:39 +01:00