RepoMirrors
/
haproxy
mirror of http://git.haproxy.org/git/haproxy.git/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
haproxy/include/proto/stick_table.h

139 lines
4.8 KiB
C
Raw Normal View History

[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
/*
* include/proto/stick_table.h
* Functions for stick tables management.
*
* Copyright (C) 2009-2010 EXCELIANCE, Emeric Brun <ebrun@exceliance.fr>
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
* Copyright (C) 2010 Willy Tarreau <w@1wt.eu>
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef _PROTO_STICK_TABLE_H
#define _PROTO_STICK_TABLE_H
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
#include <common/errors.h>
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
#include <common/ticks.h>
#include <common/time.h>
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
#include <types/stick_table.h>
[MINOR] stick_table: provide functions to return stksess data from a type This function does the indirection job in the table to find the pointer to the real data matching the requested type.
2010-06-06 14:06:52 +00:00
#define stktable_data_size(type) (sizeof(((union stktable_data*)0)->type))
#define stktable_data_cast(ptr, type) ((union stktable_data*)(ptr))->type
MEDIUM: stick-table: allocate the table key of size buffer size Keys are copied from samples to stick_table_key. If a key is larger than the stick_table_key, we have an overflow. In pratice it does not happen because it requires : 1) a configuration with tune.bufsize larger than BUFSIZE (common) 2) a stick-table configured with keys strictly larger than buffers 3) extraction of data larger than BUFSIZE (eg: using payload()) Points 2 and 3 don't make any sense for a real world configuration. That said the issue needs be fixed. The solution consists in allocating it the same size as the global buffer size, just like the samples. This fixes the issue.
2012-10-29 20:56:59 +00:00
extern struct stktable_key *static_table_key;
[MINOR] stick_table: export the stick_table_key This one is huge and will be needed by other portions of code for various data lookups. Let's not have them allocate it in the stack.
2010-06-06 15:39:30 +00:00
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
struct stksess *stksess_new(struct stktable *t, struct stktable_key *key);
[MINOR] stick_table: add support for variable-sized data Right now we're only able to store a server ID in a sticky session. The goal is to be able to store anything whose size is known at startup time. For this, we store the extra data before the stksess pointer, using a negative offset. It will then be easy to cumulate multiple data provided they each have their own offset.
2010-06-06 10:11:37 +00:00
void stksess_setkey(struct stktable *t, struct stksess *ts, struct stktable_key *key);
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
void stksess_free(struct stktable *t, struct stksess *ts);
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
void stksess_kill(struct stktable *t, struct stksess *ts);
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
int stktable_init(struct stktable *t);
int stktable_parse_type(char **args, int *idx, unsigned long *type, size_t *key_size);
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
struct stksess *stktable_get_entry(struct stktable *table, struct stktable_key *key);
[MEDIUM] Create updates tree on stick table to manage sync.
2010-09-23 16:16:52 +00:00
struct stksess *stktable_store(struct stktable *t, struct stksess *ts, int local);
struct stksess *stktable_touch(struct stktable *t, struct stksess *ts, int local);
[MEDIUM] stick_table: don't overwrite data when storing an entry Till now sticky sessions only held server IDs. Now there are other data types so it is not acceptable anymore to overwrite the server ID when writing something. The server ID must then only be written from the caller when appropriate. Doing this has also led to separate lookup and storage.
2010-06-06 13:38:59 +00:00
struct stksess *stktable_lookup(struct stktable *t, struct stksess *ts);
struct stksess *stktable_lookup_key(struct stktable *t, struct stktable_key *key);
[MINOR] stktable: add a stktable_update_key() function This function looks up a key, updates its expiration date, or creates it if it was not found. acl_fetch_src_updt_conn_cnt() was updated to make use of it.
2010-06-20 10:27:21 +00:00
struct stksess *stktable_update_key(struct stktable *table, struct stktable_key *key);
MEDIUM: stick-table: implement lookup from a sample fetch Currently we have stktable_fetch_key() which fetches a sample according to an expression and returns a stick table key, but we also need a function which does only the second half of it from a known sample. So let's cut the function in two and introduce smp_to_stkey() to perform this lookup. The first function was adapted to make use of it in order to avoid code duplication.
2014-07-03 15:02:46 +00:00
struct stktable_key *smp_to_stkey(struct sample *smp, struct stktable *t);
[MEDIUM] Create new protected pattern types CONSTSTRING and CONSTDATA to force memcpy if data from protected areas need to be manipulated. Enhance pattern convs and fetch argument parsing, now fetchs and convs callbacks used typed args. Add more details on error messages on parsing pattern expression function. Update existing pattern convs and fetchs to new proto. Create stick table key type "binary". Manage Truncation and padding if pattern's fetch-converted result don't match table key size.
2010-09-23 16:02:19 +00:00
struct stktable_key *stktable_fetch_key(struct stktable *t, struct proxy *px,
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
struct stream *l4, void *l7, unsigned int opt,
MINOR: stick-table: make stktable_fetch_key() indicate why it failed stktable_fetch_key() does not indicate whether it returns NULL because the input sample was not found or because it's unstable. It causes trouble with track-sc* rules. Just like with sample_fetch_string(), we want it to be able to give more information to the caller about what it found. Thus, now we use the pointer to a sample passed by the caller, and fill it with the information we have about the sample. That way, even if we return NULL, the caller has the ability to check whether a sample was found and if it is still changing or not.
2014-06-25 14:20:53 +00:00
struct sample_expr *expr, struct sample *smp);
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 19:37:17 +00:00
int stktable_compatible_sample(struct sample_expr *expr, unsigned long table_type);
MEDIUM: stick-table: make it easier to register extra data types Some users want to add their own data types to stick tables. We don't want to use a linked list here for performance reasons, so we need to continue to use an indexed array. This patch allows one to reserve a compile-time-defined number of extra data types by setting the new macro STKTABLE_EXTRA_DATA_TYPES to anything greater than zero, keeping in mind that anything larger will slightly inflate the memory consumed by stick tables (not per entry though). Then calling stktable_register_data_store() with the new keyword will either register a new keyword or fail if the desired entry was already taken or the keyword already registered. Note that this patch does not dictate how the data will be used, it only offers the possibility to create new keywords and have an index to reference them in the config and in the tables. The caller will not be able to use stktable_data_cast() and will have to explicitly cast the stable pointers to the expected types. It can be used for experimentation as well.
2014-07-15 14:44:27 +00:00
int stktable_register_data_store(int idx, const char *name, int std_type, int arg_type);
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
int stktable_get_data_type(char *name);
[MINOR] stick-table: provide a table lookup function We'll often need to lookup a table by its name. This will change in the future once we can resolve these names on startup.
2010-06-18 15:26:50 +00:00
struct proxy *find_stktable(const char *name);
MEDIUM: stick-tables: flush old entries upon soft-stop When a process with large stick tables is replaced by a new one and remains present until the last connection finishes, it keeps these data in memory for nothing since they will never be used anymore by incoming connections, except during syncing with the new process. This is especially problematic when dealing with long session protocols such as WebSocket as it becomes possible to stack many processes and eat a lot of memory. So the idea here is to know if a table still needs to be synced or not, and to purge all unused entries once the sync is complete. This means that after a few hundred milliseconds when everything has been synchronized with the new process, only a few entries will remain allocated (only the ones held by sessions during the restart) and all the remaining memory will be freed. Note that we carefully do that only after the grace period is expired so as not to impact a possible proxy that needs to accept a few more connections before leaving. Doing this required to add a sync counter to the stick tables, to know how many peer sync sessions are still in progress in order not to flush the entries until all synchronizations are completed.
2013-09-04 15:54:01 +00:00
int stktable_trash_oldest(struct stktable *t, int to_batch);
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
[MEDIUM] stick-table: make use of generic types for stored data It's a bit cumbersome to have to know all possible storable types from the stats interface. Instead, let's have generic types for all data, which will facilitate their manipulation.
2010-07-18 06:04:30 +00:00
/* return allocation size for standard data type <type> */
static inline int stktable_type_size(int type)
{
switch(type) {
case STD_T_SINT:
case STD_T_UINT:
return sizeof(int);
case STD_T_ULL:
return sizeof(unsigned long long);
case STD_T_FRQP:
return sizeof(struct freq_ctr_period);
}
return 0;
}
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
/* reserve some space for data type <type>, and associate argument at <sa> if
* not NULL. Returns PE_NONE (0) if OK or an error code among :
* - PE_ENUM_OOR if <type> does not exist
* - PE_EXIST if <type> is already registered
[MEDIUM] stick-tables: add stored data argument type checking We're now able to return errors based on the validity of an argument passed to a stick-table store data type. We also support ARG_T_DELAY to pass delays to stored data types (eg: for rate counters).
2010-06-20 08:41:54 +00:00
* - PE_ARG_NOT_USE if <sa> was provided but not expected
* - PE_ARG_MISSING if <sa> was expected but not provided
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
*/
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
static inline int stktable_alloc_data_type(struct stktable *t, int type, const char *sa)
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
{
if (type >= STKTABLE_DATA_TYPES)
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
return PE_ENUM_OOR;
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
if (t->data_ofs[type])
/* already allocated */
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
return PE_EXIST;
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
[MEDIUM] stick-tables: add stored data argument type checking We're now able to return errors based on the validity of an argument passed to a stick-table store data type. We also support ARG_T_DELAY to pass delays to stored data types (eg: for rate counters).
2010-06-20 08:41:54 +00:00
switch (stktable_data_types[type].arg_type) {
case ARG_T_NONE:
if (sa)
return PE_ARG_NOT_USED;
break;
case ARG_T_INT:
if (!sa)
return PE_ARG_MISSING;
t->data_arg[type].i = atoi(sa);
break;
case ARG_T_DELAY:
if (!sa)
return PE_ARG_MISSING;
sa = parse_time_err(sa, &t->data_arg[type].u, TIME_UNIT_MS);
if (sa)
return PE_ARG_INVC; /* invalid char */
break;
}
[MEDIUM] stick-table: make use of generic types for stored data It's a bit cumbersome to have to know all possible storable types from the stats interface. Instead, let's have generic types for all data, which will facilitate their manipulation.
2010-07-18 06:04:30 +00:00
t->data_size += stktable_type_size(stktable_data_types[type].std_type);
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
t->data_ofs[type] = -t->data_size;
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
return PE_NONE;
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
}
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
[MINOR] stick_table: provide functions to return stksess data from a type This function does the indirection job in the table to find the pointer to the real data matching the requested type.
2010-06-06 14:06:52 +00:00
/* return pointer for data type <type> in sticky session <ts> of table <t>, or
* NULL if either <ts> is NULL or the type is not stored.
*/
static inline void *stktable_data_ptr(struct stktable *t, struct stksess *ts, int type)
{
if (type >= STKTABLE_DATA_TYPES)
return NULL;
if (!t->data_ofs[type]) /* type not stored */
return NULL;
if (!ts)
return NULL;
return (void *)ts + t->data_ofs[type];
}
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
/* kill an entry if it's expired and its ref_cnt is zero */
static inline void stksess_kill_if_expired(struct stktable *t, struct stksess *ts)
{
[BUG] stick table entries expire on counters updates/read or show table, even if there is no "expire" parameter
2010-09-23 16:11:05 +00:00
if (t->expire != TICK_ETERNITY && tick_is_expired(ts->expire, now_ms))
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
stksess_kill(t, ts);
}
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
#endif /* _PROTO_STICK_TABLE_H */