RepoMirrors
/
haproxy
mirror of http://git.haproxy.org/git/haproxy.git/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
haproxy/include/proto/stick_table.h

172 lines
6.1 KiB
C
Raw Normal View History

[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
/*
* include/proto/stick_table.h
* Functions for stick tables management.
*
* Copyright (C) 2009-2010 EXCELIANCE, Emeric Brun <ebrun@exceliance.fr>
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
* Copyright (C) 2010 Willy Tarreau <w@1wt.eu>
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef _PROTO_STICK_TABLE_H
#define _PROTO_STICK_TABLE_H
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
#include <common/errors.h>
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
#include <common/ticks.h>
#include <common/time.h>
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
#include <types/stick_table.h>
[MINOR] stick_table: provide functions to return stksess data from a type This function does the indirection job in the table to find the pointer to the real data matching the requested type.
2010-06-06 14:06:52 +00:00
#define stktable_data_size(type) (sizeof(((union stktable_data*)0)->type))
#define stktable_data_cast(ptr, type) ((union stktable_data*)(ptr))->type
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
struct stksess *stksess_new(struct stktable *t, struct stktable_key *key);
[MINOR] stick_table: add support for variable-sized data Right now we're only able to store a server ID in a sticky session. The goal is to be able to store anything whose size is known at startup time. For this, we store the extra data before the stksess pointer, using a negative offset. It will then be easy to cumulate multiple data provided they each have their own offset.
2010-06-06 10:11:37 +00:00
void stksess_setkey(struct stktable *t, struct stksess *ts, struct stktable_key *key);
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
void stksess_free(struct stktable *t, struct stksess *ts);
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
void stksess_kill(struct stktable *t, struct stksess *ts);
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
int stktable_init(struct stktable *t);
int stktable_parse_type(char **args, int *idx, unsigned long *type, size_t *key_size);
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
struct stksess *stktable_get_entry(struct stktable *table, struct stktable_key *key);
[MEDIUM] Create updates tree on stick table to manage sync.
2010-09-23 16:16:52 +00:00
struct stksess *stktable_store(struct stktable *t, struct stksess *ts, int local);
MEDIUM: peers: Fix a peer stick-tables synchronization issue. During the stick-table teaching process which occurs at reloading/restart time, expiration dates of stick-tables entries were not synchronized between peers. This patch adds two new stick-table messages to provide such a synchronization feature. As these new messages are not supported by older haproxy peers protocol versions, this patch increments peers protol version, from 2.0 to 2.1, to help in detecting/supporting such older peers protocol implementations so that new versions might still be able to transparently communicate with a newer one. [wt: technically speaking it would be nice to have this backported into 1.6 as some people who reload often are affected by this design limitation, but it's not a totally transparent change that may make certain users feel reluctant to upgrade older versions. Let's let it cook in 1.7 first and decide later]
2016-10-12 15:30:30 +00:00
struct stksess *stktable_store_with_exp(struct stktable *t, struct stksess *ts,
int local, int expire);
struct stksess *stktable_touch_with_exp(struct stktable *t, struct stksess *ts,
int local, int expire);
[MEDIUM] Create updates tree on stick table to manage sync.
2010-09-23 16:16:52 +00:00
struct stksess *stktable_touch(struct stktable *t, struct stksess *ts, int local);
[MEDIUM] stick_table: don't overwrite data when storing an entry Till now sticky sessions only held server IDs. Now there are other data types so it is not acceptable anymore to overwrite the server ID when writing something. The server ID must then only be written from the caller when appropriate. Doing this has also led to separate lookup and storage.
2010-06-06 13:38:59 +00:00
struct stksess *stktable_lookup(struct stktable *t, struct stksess *ts);
struct stksess *stktable_lookup_key(struct stktable *t, struct stktable_key *key);
[MINOR] stktable: add a stktable_update_key() function This function looks up a key, updates its expiration date, or creates it if it was not found. acl_fetch_src_updt_conn_cnt() was updated to make use of it.
2010-06-20 10:27:21 +00:00
struct stksess *stktable_update_key(struct stktable *table, struct stktable_key *key);
MEDIUM: stick-table: implement lookup from a sample fetch Currently we have stktable_fetch_key() which fetches a sample according to an expression and returns a stick table key, but we also need a function which does only the second half of it from a known sample. So let's cut the function in two and introduce smp_to_stkey() to perform this lookup. The first function was adapted to make use of it in order to avoid code duplication.
2014-07-03 15:02:46 +00:00
struct stktable_key *smp_to_stkey(struct sample *smp, struct stktable *t);
MAJOR: sample: pass a pointer to the session to each sample fetch function Many such function need a session, and till now they used to dereference the stream. Once we remove the stream from the embryonic session, this will not be possible anymore. So as of now, sample fetch functions will be called with this : - sess = NULL, strm = NULL : never - sess = valid, strm = NULL : tcp-req connection - sess = valid, strm = valid, strm->txn = NULL : tcp-req content - sess = valid, strm = valid, strm->txn = valid : http-req / http-res
2015-04-03 23:47:55 +00:00
struct stktable_key *stktable_fetch_key(struct stktable *t, struct proxy *px, struct session *sess,
MAJOR: sample: don't pass l7 anymore to sample fetch functions All of them can now retrieve the HTTP transaction *if it exists* from the stream and be sure to get NULL there when called with an embryonic session. The patch is a bit large because many locations were touched (all fetch functions had to have their prototype adjusted). The opportunity was taken to also uniformize the call names (the stream is now always "strm" instead of "l4") and to fix indent where it was broken. This way when we later introduce the session here there will be less confusion.
2015-04-03 22:52:09 +00:00
struct stream *strm, unsigned int opt,
struct sample_expr *expr, struct sample *smp);
REORG: stkctr: move all the stick counters processing to stick-tables.c Historically we used to have the stick counters processing put into session.c which became stream.c. But a big part of it is now in stick-table.c (eg: converters) but despite this we still have all the sample fetch functions in stream.c These parts do not depend on the stream anymore, so let's move the remaining chunks to stick-table.c and have cleaner files. What remains in stream.c is everything needed to attach/detach trackers to the stream and to update the counters while the stream is being processed.
2016-11-25 15:10:05 +00:00
struct stkctr *smp_fetch_sc_stkctr(struct session *sess, struct stream *strm, const struct arg *args, const char *kw);
struct stkctr *smp_create_src_stkctr(struct session *sess, struct stream *strm, const struct arg *args, const char *kw);
REORG: use the name "sample" instead of "pattern" to designate extracted data This is mainly a massive renaming in the code to get it in line with the calling convention. Next patch will rename a few files to complete this operation.
2012-04-27 19:37:17 +00:00
int stktable_compatible_sample(struct sample_expr *expr, unsigned long table_type);
MEDIUM: stick-table: make it easier to register extra data types Some users want to add their own data types to stick tables. We don't want to use a linked list here for performance reasons, so we need to continue to use an indexed array. This patch allows one to reserve a compile-time-defined number of extra data types by setting the new macro STKTABLE_EXTRA_DATA_TYPES to anything greater than zero, keeping in mind that anything larger will slightly inflate the memory consumed by stick tables (not per entry though). Then calling stktable_register_data_store() with the new keyword will either register a new keyword or fail if the desired entry was already taken or the keyword already registered. Note that this patch does not dictate how the data will be used, it only offers the possibility to create new keywords and have an index to reference them in the config and in the tables. The caller will not be able to use stktable_data_cast() and will have to explicitly cast the stable pointers to the expected types. It can be used for experimentation as well.
2014-07-15 14:44:27 +00:00
int stktable_register_data_store(int idx, const char *name, int std_type, int arg_type);
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
int stktable_get_data_type(char *name);
MEDIUM: stick-tables: flush old entries upon soft-stop When a process with large stick tables is replaced by a new one and remains present until the last connection finishes, it keeps these data in memory for nothing since they will never be used anymore by incoming connections, except during syncing with the new process. This is especially problematic when dealing with long session protocols such as WebSocket as it becomes possible to stack many processes and eat a lot of memory. So the idea here is to know if a table still needs to be synced or not, and to purge all unused entries once the sync is complete. This means that after a few hundred milliseconds when everything has been synchronized with the new process, only a few entries will remain allocated (only the ones held by sessions during the restart) and all the remaining memory will be freed. Note that we carefully do that only after the grace period is expired so as not to impact a possible proxy that needs to accept a few more connections before leaving. Doing this required to add a sync counter to the stick tables, to know how many peer sync sessions are still in progress in order not to flush the entries until all synchronizations are completed.
2013-09-04 15:54:01 +00:00
int stktable_trash_oldest(struct stktable *t, int to_batch);
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
[MEDIUM] stick-table: make use of generic types for stored data It's a bit cumbersome to have to know all possible storable types from the stats interface. Instead, let's have generic types for all data, which will facilitate their manipulation.
2010-07-18 06:04:30 +00:00
/* return allocation size for standard data type <type> */
static inline int stktable_type_size(int type)
{
switch(type) {
case STD_T_SINT:
case STD_T_UINT:
return sizeof(int);
case STD_T_ULL:
return sizeof(unsigned long long);
case STD_T_FRQP:
return sizeof(struct freq_ctr_period);
}
return 0;
}
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
/* reserve some space for data type <type>, and associate argument at <sa> if
* not NULL. Returns PE_NONE (0) if OK or an error code among :
* - PE_ENUM_OOR if <type> does not exist
* - PE_EXIST if <type> is already registered
[MEDIUM] stick-tables: add stored data argument type checking We're now able to return errors based on the validity of an argument passed to a stick-table store data type. We also support ARG_T_DELAY to pass delays to stored data types (eg: for rate counters).
2010-06-20 08:41:54 +00:00
* - PE_ARG_NOT_USE if <sa> was provided but not expected
* - PE_ARG_MISSING if <sa> was expected but not provided
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
*/
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
static inline int stktable_alloc_data_type(struct stktable *t, int type, const char *sa)
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
{
if (type >= STKTABLE_DATA_TYPES)
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
return PE_ENUM_OOR;
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
if (t->data_ofs[type])
/* already allocated */
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
return PE_EXIST;
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
[MEDIUM] stick-tables: add stored data argument type checking We're now able to return errors based on the validity of an argument passed to a stick-table store data type. We also support ARG_T_DELAY to pass delays to stored data types (eg: for rate counters).
2010-06-20 08:41:54 +00:00
switch (stktable_data_types[type].arg_type) {
case ARG_T_NONE:
if (sa)
return PE_ARG_NOT_USED;
break;
case ARG_T_INT:
if (!sa)
return PE_ARG_MISSING;
t->data_arg[type].i = atoi(sa);
break;
case ARG_T_DELAY:
if (!sa)
return PE_ARG_MISSING;
sa = parse_time_err(sa, &t->data_arg[type].u, TIME_UNIT_MS);
if (sa)
return PE_ARG_INVC; /* invalid char */
break;
}
[MEDIUM] stick-table: make use of generic types for stored data It's a bit cumbersome to have to know all possible storable types from the stats interface. Instead, let's have generic types for all data, which will facilitate their manipulation.
2010-07-18 06:04:30 +00:00
t->data_size += stktable_type_size(stktable_data_types[type].std_type);
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
t->data_ofs[type] = -t->data_size;
[MEDIUM] stick-tables: add support for arguments to data_types Some data types will require arguments (eg: period for a rate counter). This patch adds support for such arguments between parenthesis in the "store" directive of the stick-table statement. Right now only integers are supported.
2010-06-20 07:11:39 +00:00
return PE_NONE;
[MEDIUM] stick_table: add room for extra data types The stick_tables will now be able to store extra data for a same key. A limited set of extra data types will be defined and for each of them an offset in the sticky session will be assigned at startup time. All of this information will be stored in the stick table. The extra data types will have to be specified after the new "store" keyword of the "stick-table" directive, which will reserve some space for them.
2010-06-06 11:34:54 +00:00
}
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
[MINOR] stick_table: provide functions to return stksess data from a type This function does the indirection job in the table to find the pointer to the real data matching the requested type.
2010-06-06 14:06:52 +00:00
/* return pointer for data type <type> in sticky session <ts> of table <t>, or
* NULL if either <ts> is NULL or the type is not stored.
*/
static inline void *stktable_data_ptr(struct stktable *t, struct stksess *ts, int type)
{
if (type >= STKTABLE_DATA_TYPES)
return NULL;
if (!t->data_ofs[type]) /* type not stored */
return NULL;
if (!ts)
return NULL;
return (void *)ts + t->data_ofs[type];
}
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
/* kill an entry if it's expired and its ref_cnt is zero */
static inline void stksess_kill_if_expired(struct stktable *t, struct stksess *ts)
{
[BUG] stick table entries expire on counters updates/read or show table, even if there is no "expire" parameter
2010-09-23 16:11:05 +00:00
if (t->expire != TICK_ETERNITY && tick_is_expired(ts->expire, now_ms))
[MEDIUM] session counters: automatically remove expired entries. When a ref_cnt goes down to zero and the entry is expired, remove it.
2010-08-03 18:34:06 +00:00
stksess_kill(t, ts);
}
REORG: stktable: move the stkctr_* functions from stream to sticktable These ones are not stream-specific at all and will be needed outside of stream, so let's move them to stick_tables where struct stkctr is defined.
2015-04-04 14:24:42 +00:00
/* sets the stick counter's entry pointer */
static inline void stkctr_set_entry(struct stkctr *stkctr, struct stksess *entry)
{
stkctr->entry = caddr_from_ptr(entry, 0);
}
/* returns the entry pointer from a stick counter */
static inline struct stksess *stkctr_entry(struct stkctr *stkctr)
{
return caddr_to_ptr(stkctr->entry);
}
/* returns the two flags from a stick counter */
static inline unsigned int stkctr_flags(struct stkctr *stkctr)
{
return caddr_to_data(stkctr->entry);
}
/* sets up to two flags at a time on a composite address */
static inline void stkctr_set_flags(struct stkctr *stkctr, unsigned int flags)
{
stkctr->entry = caddr_set_flags(stkctr->entry, flags);
}
/* returns the two flags from a stick counter */
static inline void stkctr_clr_flags(struct stkctr *stkctr, unsigned int flags)
{
stkctr->entry = caddr_clr_flags(stkctr->entry, flags);
}
[MEDIUM] Add stick table (persistence) management functions and types
2010-01-04 14:23:48 +00:00
#endif /* _PROTO_STICK_TABLE_H */