RepoMirrors
/
haproxy
mirror of http://git.haproxy.org/git/haproxy.git/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
haproxy/include/proto/stream.h

307 lines
9.5 KiB
C
Raw Normal View History

[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
/*
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
* include/proto/stream.h
* This file defines everything related to streams.
[MAJOR] frontend: split accept() into frontend_accept() and session_accept() A new function session_accept() is now called from the lower layer to instanciate a new session. Once the session is instanciated, the upper layer's frontent_accept() is called. This one can be service-dependant. That way, we have a 3-phase accept() sequence : 1) protocol-specific, session-less accept(), which is pointed to by the listener. It defaults to the generic stream_sock_accept(). 2) session_accept() which relies on a frontend but not necessarily for use in a proxy (eg: stats or any future service). 3) frontend_accept() which performs the accept for the service offerred by the frontend. It defaults to frontend_accept() which is really what is used by a proxy. The TCP/HTTP proxies have been moved to this mode so that we can now rely on frontend_accept() for any type of session initialization relying on a frontend. The next step will be to convert the stats to use the same system for the stats.
2010-06-01 15:45:26 +00:00
*
* Copyright (C) 2000-2010 Willy Tarreau - w@1wt.eu
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
#ifndef _PROTO_STREAM_H
#define _PROTO_STREAM_H
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
[CLEANUP] included common/version.h everywhere
2006-06-29 16:54:54 +00:00
#include <common/config.h>
[MAJOR] migrated task, tree64 and session to pool2 task and tree64 are already very close in size and are merged together. Overall performance gained slightly by this simple change.
2007-05-13 17:43:47 +00:00
#include <common/memory.h>
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
#include <types/stream.h>
MAJOR: session: only wake up as many sessions as available buffers permit We've already experimented with three wake up algorithms when releasing buffers : the first naive one used to wake up far too many sessions, causing many of them not to get any buffer. The second approach which was still in use prior to this patch consisted in waking up either 1 or 2 sessions depending on the number of FDs we had released. And this was still inaccurate. The third one tried to cover the accuracy issues of the second and took into consideration the number of FDs the sessions would be willing to use, but most of the time we ended up waking up too many of them for nothing, or deadlocking by lack of buffers. This patch completely removes the need to allocate two buffers at once. Instead it splits allocations into critical and non-critical ones and implements a reserve in the pool for this. The deadlock situation happens when all buffers are be allocated for requests pending in a maxconn-limited server queue, because then there's no more way to allocate buffers for responses, and these responses are critical to release the servers's connection in order to release the pending requests. In fact maxconn on a server creates a dependence between sessions and particularly between oldest session's responses and latest session's requests. Thus, it is mandatory to get a free buffer for a response in order to release a server connection which will permit to release a request buffer. Since we definitely have non-symmetrical buffers, we need to implement this logic in the buffer allocation mechanism. What this commit does is implement a reserve of buffers which can only be allocated for responses and that will never be allocated for requests. This is made possible by the requester indicating how much margin it wants to leave after the allocation succeeds. Thus it is a cooperative allocation mechanism : the requester (process_session() in general) prefers not to get a buffer in order to respect other's need for response buffers. The session management code always knows if a buffer will be used for requests or responses, so that is not difficult : - either there's an applet on the initiator side and we really need the request buffer (since currently the applet is called in the context of the session) - or we have a connection and we really need the response buffer (in order to support building and sending an error message back) This reserve ensures that we don't take all allocatable buffers for requests waiting in a queue. The downside is that all the extra buffers are really allocated to ensure they can be allocated. But with small values it is not an issue. With this change, we don't observe any more deadlocks even when running with maxconn 1 on a server under severely constrained memory conditions. The code becomes a bit tricky, it relies on the scheduler's run queue to estimate how many sessions are already expected to run so that it doesn't wake up everyone with too few resources. A better solution would probably consist in having two queues, one for urgent requests and one for normal requests. A failed allocation for a session dealing with an error, a connection event, or the need for a response (or request when there's an applet on the left) would go to the urgent request queue, while other requests would go to the other queue. Urgent requests would be served from 1 entry in the pool, while the regular ones would be served only according to the reserve. Despite not yet having this, it works remarkably well. This mechanism is quite efficient, we don't perform too many wake up calls anymore. For 1 million sessions elapsed during massive memory contention, we observe about 4.5M calls to process_session() compared to 4.0M without memory constraints. Previously we used to observe up to 16M calls, which rougly means 12M failures. During a test run under high memory constraints (limit enforced to 27 MB instead of the 58 MB normally needed), performance used to drop by 53% prior to this patch. Now with this patch instead it *increases* by about 1.5%. The best effect of this change is that by limiting the memory usage to about 2/3 to 3/4 of what is needed by default, it's possible to increase performance by up to about 18% mainly due to the fact that pools are reused more often and remain hot in the CPU cache (observed on regular HTTP traffic with 20k objects, buffers.limit = maxconn/10, buffers.reserve = limit/2). Below is an example of scenario which used to cause a deadlock previously : - connection is received - two buffers are allocated in process_session() then released - one is allocated when receiving an HTTP request - the second buffer is allocated then released in process_session() for request parsing then connection establishment. - poll() says we can send, so the request buffer is sent and released - process session gets notified that the connection is now established and allocates two buffers then releases them - all other sessions do the same till one cannot get the request buffer without hitting the margin - and now the server responds. stream_interface allocates the response buffer and manages to get it since it's higher priority being for a response. - but process_session() cannot allocate the request buffer anymore => We could end up with all buffers used by responses so that none may be allocated for a request in process_session(). When the applet processing leaves the session context, the test will have to be changed so that we always allocate a response buffer regardless of the left side (eg: H2->H1 gateway). A final improvement would consists in being able to only retry the failed I/O operation without waking up a task, but to date all experiments to achieve this have proven not to be reliable enough.
2014-11-27 00:11:56 +00:00
#include <proto/fd.h>
[MEDIUM] session counters: add conn_rate and sess_rate counters These counters maintain incoming connection rates and session rates in a stick-table, over a period which is defined in the configuration (2 ms to 24 days). They can be used to detect service abuse and enforce a certain accept rate per source address for instance, and block if the rate is passed over. Example : # block if more than 50 requests per 5 seconds from a source. stick-table type ip size 200k expire 1m store conn_rate(5s),sess_rate(5s) tcp-request track-counters src tcp-request reject if { trk_conn_rate gt 50 } # cause a 3 seconds pause to requests from sources in excess of 20 requests/5s tcp-request inspect-delay 3s tcp-request content accept if { trk_sess_rate gt 20 } WAIT_END
2010-06-20 09:19:22 +00:00
#include <proto/freq_ctr.h>
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
#include <proto/stick_table.h>
MAJOR: session: only wake up as many sessions as available buffers permit We've already experimented with three wake up algorithms when releasing buffers : the first naive one used to wake up far too many sessions, causing many of them not to get any buffer. The second approach which was still in use prior to this patch consisted in waking up either 1 or 2 sessions depending on the number of FDs we had released. And this was still inaccurate. The third one tried to cover the accuracy issues of the second and took into consideration the number of FDs the sessions would be willing to use, but most of the time we ended up waking up too many of them for nothing, or deadlocking by lack of buffers. This patch completely removes the need to allocate two buffers at once. Instead it splits allocations into critical and non-critical ones and implements a reserve in the pool for this. The deadlock situation happens when all buffers are be allocated for requests pending in a maxconn-limited server queue, because then there's no more way to allocate buffers for responses, and these responses are critical to release the servers's connection in order to release the pending requests. In fact maxconn on a server creates a dependence between sessions and particularly between oldest session's responses and latest session's requests. Thus, it is mandatory to get a free buffer for a response in order to release a server connection which will permit to release a request buffer. Since we definitely have non-symmetrical buffers, we need to implement this logic in the buffer allocation mechanism. What this commit does is implement a reserve of buffers which can only be allocated for responses and that will never be allocated for requests. This is made possible by the requester indicating how much margin it wants to leave after the allocation succeeds. Thus it is a cooperative allocation mechanism : the requester (process_session() in general) prefers not to get a buffer in order to respect other's need for response buffers. The session management code always knows if a buffer will be used for requests or responses, so that is not difficult : - either there's an applet on the initiator side and we really need the request buffer (since currently the applet is called in the context of the session) - or we have a connection and we really need the response buffer (in order to support building and sending an error message back) This reserve ensures that we don't take all allocatable buffers for requests waiting in a queue. The downside is that all the extra buffers are really allocated to ensure they can be allocated. But with small values it is not an issue. With this change, we don't observe any more deadlocks even when running with maxconn 1 on a server under severely constrained memory conditions. The code becomes a bit tricky, it relies on the scheduler's run queue to estimate how many sessions are already expected to run so that it doesn't wake up everyone with too few resources. A better solution would probably consist in having two queues, one for urgent requests and one for normal requests. A failed allocation for a session dealing with an error, a connection event, or the need for a response (or request when there's an applet on the left) would go to the urgent request queue, while other requests would go to the other queue. Urgent requests would be served from 1 entry in the pool, while the regular ones would be served only according to the reserve. Despite not yet having this, it works remarkably well. This mechanism is quite efficient, we don't perform too many wake up calls anymore. For 1 million sessions elapsed during massive memory contention, we observe about 4.5M calls to process_session() compared to 4.0M without memory constraints. Previously we used to observe up to 16M calls, which rougly means 12M failures. During a test run under high memory constraints (limit enforced to 27 MB instead of the 58 MB normally needed), performance used to drop by 53% prior to this patch. Now with this patch instead it *increases* by about 1.5%. The best effect of this change is that by limiting the memory usage to about 2/3 to 3/4 of what is needed by default, it's possible to increase performance by up to about 18% mainly due to the fact that pools are reused more often and remain hot in the CPU cache (observed on regular HTTP traffic with 20k objects, buffers.limit = maxconn/10, buffers.reserve = limit/2). Below is an example of scenario which used to cause a deadlock previously : - connection is received - two buffers are allocated in process_session() then released - one is allocated when receiving an HTTP request - the second buffer is allocated then released in process_session() for request parsing then connection establishment. - poll() says we can send, so the request buffer is sent and released - process session gets notified that the connection is now established and allocates two buffers then releases them - all other sessions do the same till one cannot get the request buffer without hitting the margin - and now the server responds. stream_interface allocates the response buffer and manages to get it since it's higher priority being for a response. - but process_session() cannot allocate the request buffer anymore => We could end up with all buffers used by responses so that none may be allocated for a request in process_session(). When the applet processing leaves the session context, the test will have to be changed so that we always allocate a response buffer regardless of the left side (eg: H2->H1 gateway). A final improvement would consists in being able to only retry the failed I/O operation without waking up a task, but to date all experiments to achieve this have proven not to be reliable enough.
2014-11-27 00:11:56 +00:00
#include <proto/task.h>
[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
extern struct pool_head *pool2_stream;
extern struct list streams;
MAJOR: session: implement a wait-queue for sessions who need a buffer When a session_alloc_buffers() fails to allocate one or two buffers, it subscribes the session to buffer_wq, and waits for another session to release buffers. It's then removed from the queue and woken up with TASK_WAKE_RES, and can attempt its allocation again. We decide to try to wake as many waiters as we release buffers so that if we release 2 and two waiters need only once, they both have their chance. We must never come to the situation where we don't wake enough tasks up. It's common to release buffers after the completion of an I/O callback, which can happen even if the I/O could not be performed due to half a failure on memory allocation. In this situation, we don't want to move out of the wait queue the session that was just added, otherwise it will never get any buffer. Thus, we only force ourselves out of the queue when freeing the session. Note: at the moment, since session_alloc_buffers() is not used, no task is subscribed to the wait queue.
2014-11-25 20:10:35 +00:00
extern struct list buffer_wq;
[MAJOR] migrated task, tree64 and session to pool2 task and tree64 are already very close in size and are merged together. Overall performance gained slightly by this simple change.
2007-05-13 17:43:47 +00:00
MINOR: cli: report connection status in "show sess xxx" Connection flags, targets and transport layers are now reported in "show sess $PTR", as it is an absolute requirement in debugging.
2012-11-19 15:10:32 +00:00
extern struct data_cb sess_conn_cb;
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
int stream_accept(struct listener *l, int cfd, struct sockaddr_storage *addr);
[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
[MAJOR] migrated task, tree64 and session to pool2 task and tree64 are already very close in size and are merged together. Overall performance gained slightly by this simple change.
2007-05-13 17:43:47 +00:00
/* perform minimal intializations, report 0 in case of error, 1 if OK. */
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
int init_stream();
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
REORG/MEDIUM: stream: rename stream flags from SN_* to SF_* This is in order to keep things consistent.
2015-04-02 23:14:29 +00:00
/* kill a stream and set the termination flags to <why> (one of SF_ERR_*) */
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
void stream_shutdown(struct stream *stream, int why);
[CLEANUP] session.c: Make functions static where possible
2011-06-08 00:19:07 +00:00
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
void stream_process_counters(struct stream *s);
void sess_change_server(struct stream *sess, struct server *newsrv);
struct task *process_stream(struct task *t);
void default_srv_error(struct stream *s, struct stream_interface *si);
MAJOR: sample: pass a pointer to the session to each sample fetch function Many such function need a session, and till now they used to dereference the stream. Once we remove the stream from the embryonic session, this will not be possible anymore. So as of now, sample fetch functions will be called with this : - sess = NULL, strm = NULL : never - sess = valid, strm = NULL : tcp-req connection - sess = valid, strm = valid, strm->txn = NULL : tcp-req content - sess = valid, strm = valid, strm->txn = valid : http-req / http-res
2015-04-03 23:47:55 +00:00
struct stkctr *smp_fetch_sc_stkctr(struct session *sess, struct stream *strm, const struct arg *args, const char *kw);
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
int parse_track_counters(char **args, int *arg,
int section_type, struct proxy *curpx,
struct track_ctr_prm *prm,
MEDIUM: cfgparse: use the new error reporting framework for remaining cfg_keywords All keywords registered using a cfg_kw_list now make use of the new error reporting framework. This allows easier and more precise error reporting without having to deal with complex buffer allocation issues.
2012-05-08 17:47:01 +00:00
struct proxy *defpx, char **err);
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
/* Update the stream's backend and server time stats */
void stream_update_time_stats(struct stream *s);
void __stream_offer_buffers(int rqlimit);
static inline void stream_offer_buffers();
int stream_alloc_work_buffer(struct stream *s);
void stream_release_buffers(struct stream *s);
int stream_alloc_recv_buffer(struct channel *chn);
MEDIUM: session: maintain per-backend and per-server time statistics Using the last rate counters, we now compute the queue, connect, response and total times per server and per backend with a 95% accuracy over the last 1024 samples. The operation is cheap so we don't need to condition it.
2014-06-17 10:19:18 +00:00
MINOR: session: start to reintroduce struct session There is now a pointer to the session in the stream, which is NULL for now. The session pool is created as well. Some parts will move from the stream to the session now.
2015-04-03 11:53:24 +00:00
/* returns the session this stream belongs to */
static inline struct session *strm_sess(const struct stream *strm)
{
return strm->sess;
}
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
/* sets the stick counter's entry pointer */
static inline void stkctr_set_entry(struct stkctr *stkctr, struct stksess *entry)
{
stkctr->entry = caddr_from_ptr(entry, 0);
}
/* returns the entry pointer from a stick counter */
static inline struct stksess *stkctr_entry(struct stkctr *stkctr)
{
return caddr_to_ptr(stkctr->entry);
}
/* returns the two flags from a stick counter */
static inline unsigned int stkctr_flags(struct stkctr *stkctr)
{
return caddr_to_data(stkctr->entry);
}
/* sets up to two flags at a time on a composite address */
static inline void stkctr_set_flags(struct stkctr *stkctr, unsigned int flags)
{
stkctr->entry = caddr_set_flags(stkctr->entry, flags);
}
/* returns the two flags from a stick counter */
static inline void stkctr_clr_flags(struct stkctr *stkctr, unsigned int flags)
{
stkctr->entry = caddr_clr_flags(stkctr->entry, flags);
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
/* Remove the refcount from the stream to the tracked counters, and clear the
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
* pointer to ensure this is only performed once. The caller is responsible for
* ensuring that the pointer is valid first.
*/
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static inline void stream_store_counters(struct stream *s)
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
{
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
void *ptr;
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
int i;
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
MINOR: session: make the number of stick counter entries more configurable In preparation of more flexibility in the stick counters, make their number configurable. It still defaults to 3 which is the minimum accepted value. Changing the value alone is not sufficient to get more counters, some bitfields still need to be updated and the TCP actions need to be updated as well, but this update tries to be easier, which is nice for experimentation purposes.
2013-07-23 17:15:30 +00:00
for (i = 0; i < MAX_SESS_STKCTR; i++) {
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!stkctr_entry(&s->stkctr[i]))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_CONN_CUR);
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
if (ptr)
stktable_data_cast(ptr, conn_cur)--;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
stkctr_entry(&s->stkctr[i])->ref_cnt--;
stksess_kill_if_expired(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]));
stkctr_set_entry(&s->stkctr[i], NULL);
[MEDIUM] session: add concurrent connections counter The new "conn_cur" session counter has been added. It is automatically updated upon "track XXX" directives, and the entry is touched at the moment we increment the value so that we don't consider further counter updates as real updates, otherwise we would end up updating upon completion, which may not be desired. Probably that some other event counters (eg: HTTP requests) will have to be updated upon each event though. This counter can be matched against current session's source address using the "src_conn_cur" ACL.
2010-06-18 14:35:43 +00:00
}
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
/* Remove the refcount from the stream counters tracked at the content level if
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
* any, and clear the pointer to ensure this is only performed once. The caller
* is responsible for ensuring that the pointer is valid first.
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
*/
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static inline void stream_stop_content_counters(struct stream *s)
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
{
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
void *ptr;
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
int i;
[MEDIUM] session-counters: automatically update tracked connection count When a session tracks a counter, automatically increase the cumulated connection count. This makes src_updt_conn_cnt() almost useless. In fact it might still be used to update different tables.
2010-06-18 19:03:20 +00:00
MINOR: session: make the number of stick counter entries more configurable In preparation of more flexibility in the stick counters, make their number configurable. It still defaults to 3 which is the minimum accepted value. Changing the value alone is not sufficient to get more counters, some bitfields still need to be updated and the TCP actions need to be updated as well, but this update tries to be easier, which is nice for experimentation purposes.
2013-07-23 17:15:30 +00:00
for (i = 0; i < MAX_SESS_STKCTR; i++) {
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!stkctr_entry(&s->stkctr[i]))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!(stkctr_flags(&s->stkctr[i]) & STKCTR_TRACK_CONTENT))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_CONN_CUR);
[MEDIUM] session-counters: correctly unbind the counters tracked by the backend In case of HTTP keepalive processing, we want to release the counters tracked by the backend. Till now only the second set of counters was released, while it could have been assigned by the frontend, or the backend could also have assigned the first set. Now we reuse to unused bits of the session flags to mark which stick counters were assigned by the backend and to release them as appropriate.
2010-08-06 18:11:05 +00:00
if (ptr)
stktable_data_cast(ptr, conn_cur)--;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
stkctr_entry(&s->stkctr[i])->ref_cnt--;
stksess_kill_if_expired(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]));
stkctr_set_entry(&s->stkctr[i], NULL);
[MEDIUM] session-counters: correctly unbind the counters tracked by the backend In case of HTTP keepalive processing, we want to release the counters tracked by the backend. Till now only the second set of counters was released, while it could have been assigned by the frontend, or the backend could also have assigned the first set. Now we reuse to unused bits of the session flags to mark which stick counters were assigned by the backend and to release them as appropriate.
2010-08-06 18:11:05 +00:00
}
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
}
[MEDIUM] session-counters: automatically update tracked connection count When a session tracks a counter, automatically increase the cumulated connection count. This makes src_updt_conn_cnt() almost useless. In fact it might still be used to update different tables.
2010-06-18 19:03:20 +00:00
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
/* Increase total and concurrent connection count for stick entry <ts> of table
* <t>. The caller is responsible for ensuring that <t> and <ts> are valid
* pointers, and for calling this only once per connection.
*/
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static inline void stream_start_counters(struct stktable *t, struct stksess *ts)
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
{
void *ptr;
ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_CUR);
if (ptr)
stktable_data_cast(ptr, conn_cur)++;
ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_CNT);
if (ptr)
stktable_data_cast(ptr, conn_cnt)++;
ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_RATE);
if (ptr)
update_freq_ctr_period(&stktable_data_cast(ptr, conn_rate),
t->data_arg[STKTABLE_DT_CONN_RATE].u, 1);
if (tick_isset(t->expire))
ts->expire = tick_add(now_ms, MS_TO_TICKS(t->expire));
}
[MEDIUM] session counters: add conn_rate and sess_rate counters These counters maintain incoming connection rates and session rates in a stick-table, over a period which is defined in the configuration (2 ms to 24 days). They can be used to detect service abuse and enforce a certain accept rate per source address for instance, and block if the rate is passed over. Example : # block if more than 50 requests per 5 seconds from a source. stick-table type ip size 200k expire 1m store conn_rate(5s),sess_rate(5s) tcp-request track-counters src tcp-request reject if { trk_conn_rate gt 50 } # cause a 3 seconds pause to requests from sources in excess of 20 requests/5s tcp-request inspect-delay 3s tcp-request content accept if { trk_sess_rate gt 20 } WAIT_END
2010-06-20 09:19:22 +00:00
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
/* Enable tracking of stream counters as <stkctr> on stksess <ts>. The caller is
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
* responsible for ensuring that <t> and <ts> are valid pointers. Some controls
* are performed to ensure the state can still change.
*/
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static inline void stream_track_stkctr(struct stkctr *ctr, struct stktable *t, struct stksess *ts)
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
{
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (stkctr_entry(ctr))
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
return;
ts->ref_cnt++;
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
ctr->table = t;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
stkctr_set_entry(ctr, ts);
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
stream_start_counters(t, ts);
[MAJOR] session: add track-counters to track counters related to the session This patch adds the ability to set a pointer in the session to an entry in a stick table which holds various counters related to a specific pattern. Right now the syntax matches the target syntax and only the "src" pattern can be specified, to track counters related to the session's IPv4 source address. There is a special function to extract it and convert it to a key. But the goal is to be able to later support as many patterns as for the stick rules, and get rid of the specific function. The "track-counters" directive may only be set in a "tcp-request" statement right now. Only the first one applies. Probably that later we'll support multi-criteria tracking for a single session and that we'll have to name tracking pointers. No counter is updated right now, only the refcount is. Some subsequent patches will have to bring that feature.
2010-06-14 19:04:55 +00:00
}
[MEDIUM] continous statistics By default, counters used for statistics calculation are incremented only when a session finishes. It works quite well when serving small objects, but with big ones (for example large images or archives) or with A/V streaming, a graph generated from haproxy counters looks like a hedgehog. This patch implements a contstats (continous statistics) option. When set counters get incremented continuously, during a whole session. Recounting touches a hotpath directly so it is not enabled by default, as it has small performance impact (~0.5%).
2007-11-24 21:12:47 +00:00
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
/* Increase the number of cumulated HTTP requests in the tracked counters */
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static void inline stream_inc_http_req_ctr(struct stream *s)
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
{
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
void *ptr;
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
int i;
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
MINOR: session: make the number of stick counter entries more configurable In preparation of more flexibility in the stick counters, make their number configurable. It still defaults to 3 which is the minimum accepted value. Changing the value alone is not sufficient to get more counters, some bitfields still need to be updated and the TCP actions need to be updated as well, but this update tries to be easier, which is nice for experimentation purposes.
2013-07-23 17:15:30 +00:00
for (i = 0; i < MAX_SESS_STKCTR; i++) {
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!stkctr_entry(&s->stkctr[i]))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_HTTP_REQ_CNT);
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
if (ptr)
stktable_data_cast(ptr, http_req_cnt)++;
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_HTTP_REQ_RATE);
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
if (ptr)
update_freq_ctr_period(&stktable_data_cast(ptr, http_req_rate),
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
s->stkctr[i].table->data_arg[STKTABLE_DT_HTTP_REQ_RATE].u, 1);
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
}
}
/* Increase the number of cumulated HTTP requests in the backend's tracked counters */
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static void inline stream_inc_be_http_req_ctr(struct stream *s)
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
{
void *ptr;
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
int i;
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
MINOR: session: make the number of stick counter entries more configurable In preparation of more flexibility in the stick counters, make their number configurable. It still defaults to 3 which is the minimum accepted value. Changing the value alone is not sufficient to get more counters, some bitfields still need to be updated and the TCP actions need to be updated as well, but this update tries to be easier, which is nice for experimentation purposes.
2013-07-23 17:15:30 +00:00
for (i = 0; i < MAX_SESS_STKCTR; i++) {
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!stkctr_entry(&s->stkctr[i]))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!(stkctr_flags(&s->stkctr[i]) & STKCTR_TRACK_BACKEND))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_HTTP_REQ_CNT);
MEDIUM: proto_tcp: add support for tracking L7 information Until now it was only possible to use track-sc1/sc2 with "src" which is the IPv4 source address. Now we can use track-sc1/sc2 with any fetch as well as any transformation type. It works just like the "stick" directive. Samples are automatically converted to the correct types for the table. Only "tcp-request content" rules may use L7 information, and such information must already be present when the tracking is set up. For example it becomes possible to track the IP address passed in the X-Forwarded-For header. HTTP request processing now also considers tracking from backend rules because we want to be able to update the counters even when the request was already parsed and tracked. Some more controls need to be performed (eg: samples do not distinguish between L4 and L6).
2012-12-09 11:00:04 +00:00
if (ptr)
stktable_data_cast(ptr, http_req_cnt)++;
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_HTTP_REQ_RATE);
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
if (ptr)
update_freq_ctr_period(&stktable_data_cast(ptr, http_req_rate),
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
s->stkctr[i].table->data_arg[STKTABLE_DT_HTTP_REQ_RATE].u, 1);
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
}
}
/* Increase the number of cumulated failed HTTP requests in the tracked
* counters. Only 4xx requests should be counted here so that we can
* distinguish between errors caused by client behaviour and other ones.
* Note that even 404 are interesting because they're generally caused by
* vulnerability scans.
*/
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static void inline stream_inc_http_err_ctr(struct stream *s)
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
{
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
void *ptr;
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
int i;
[MAJOR] session-counters: split FE and BE track counters Having a single tracking pointer for both frontend and backend counters does not work. Instead let's have one for each. The keyword has changed to "track-be-counters" and "track-fe-counters", and the ACL "trk_*" changed to "trkfe_*" and "trkbe_*".
2010-08-03 14:29:52 +00:00
MINOR: session: make the number of stick counter entries more configurable In preparation of more flexibility in the stick counters, make their number configurable. It still defaults to 3 which is the minimum accepted value. Changing the value alone is not sufficient to get more counters, some bitfields still need to be updated and the TCP actions need to be updated as well, but this update tries to be easier, which is nice for experimentation purposes.
2013-07-23 17:15:30 +00:00
for (i = 0; i < MAX_SESS_STKCTR; i++) {
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
if (!stkctr_entry(&s->stkctr[i]))
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
continue;
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_HTTP_ERR_CNT);
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
if (ptr)
stktable_data_cast(ptr, http_err_cnt)++;
MEDIUM: counters: stop relying on session flags at all Till now, we had one flag per stick counter to indicate if it was tracked in a backend or in a frontend. We just had to add another flag per stick-counter to indicate if it relies on contents or just connection. These flags are quite painful to maintain and tend to easily conflict with other flags if their number is changed. The correct solution consists in moving the flags to the stkctr struct itself, but currently this struct is made of 2 pointers, so adding a new entry there to store only two bits will cause at least 16 more bytes to be eaten per counter due to alignment issues, and we definitely don't want to waste tens to hundreds of bytes per session just for things that most users don't use. Since we only need to store two bits per counter, an intermediate solution consists in replacing the entry pointer with a composite value made of the original entry pointer and the two flags in the 2 unused lower bits. If later a need for other flags arises, we'll have to store them in the struct. A few inline functions have been added to abstract the retrieval and assignment of the pointers and flags, resulting in very few changes. That way there is no more dependence on the number of stick-counters and their position in the session flags.
2014-01-28 22:18:23 +00:00
ptr = stktable_data_ptr(s->stkctr[i].table, stkctr_entry(&s->stkctr[i]), STKTABLE_DT_HTTP_ERR_RATE);
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
if (ptr)
update_freq_ctr_period(&stktable_data_cast(ptr, http_err_rate),
CLEANUP: session: use an array for the stick counters The stick counters were in two distinct sets of struct members, causing some code to be duplicated. Now we use an array, which enables some processing to be performed in loops. This allowed the code to be shrunk by 700 bytes.
2012-12-09 14:55:40 +00:00
s->stkctr[i].table->data_arg[STKTABLE_DT_HTTP_ERR_RATE].u, 1);
[MEDIUM] session-counters: add HTTP req/err tracking This patch adds support for the following session counters : - http_req_cnt : HTTP request count - http_req_rate: HTTP request rate - http_err_cnt : HTTP request error count - http_err_rate: HTTP request error rate The equivalent ACLs have been added to check the tracked counters for the current session or the counters of the current source.
2010-06-23 09:44:09 +00:00
}
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static void inline stream_add_srv_conn(struct stream *sess, struct server *srv)
[MINOR] Add active connection list to server The motivation for this is to allow iteration of all the connections of a server without the expense of iterating over the global list of connections. The first use of this will be to implement an option to close connections associated with a server when is is marked as being down or in maintenance mode.
2011-06-21 05:34:57 +00:00
{
sess->srv_conn = srv;
LIST_ADD(&srv->actconns, &sess->by_srv);
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static void inline stream_del_srv_conn(struct stream *sess)
[MINOR] Add active connection list to server The motivation for this is to allow iteration of all the connections of a server without the expense of iterating over the global list of connections. The first use of this will be to implement an option to close connections associated with a server when is is marked as being down or in maintenance mode.
2011-06-21 05:34:57 +00:00
{
if (!sess->srv_conn)
return;
sess->srv_conn = NULL;
LIST_DEL(&sess->by_srv);
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static void inline stream_init_srv_conn(struct stream *sess)
[BUG] session: risk of crash on out of memory (1.5-dev regression) Patch af5149 introduced an issue which can be detected only on out of memory conditions : a LIST_DEL() may be performed on an uninitialized struct member instead of a LIST_INIT() during the accept() phase, causing crashes and memory corruption to occur. This issue was detected and diagnosed by the Exceliance R&D team. This is 1.5-specific and very recent, so no existing deployment should be impacted.
2011-07-19 22:17:39 +00:00
{
sess->srv_conn = NULL;
LIST_INIT(&sess->by_srv);
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
static inline void stream_offer_buffers()
MAJOR: session: only wake up as many sessions as available buffers permit We've already experimented with three wake up algorithms when releasing buffers : the first naive one used to wake up far too many sessions, causing many of them not to get any buffer. The second approach which was still in use prior to this patch consisted in waking up either 1 or 2 sessions depending on the number of FDs we had released. And this was still inaccurate. The third one tried to cover the accuracy issues of the second and took into consideration the number of FDs the sessions would be willing to use, but most of the time we ended up waking up too many of them for nothing, or deadlocking by lack of buffers. This patch completely removes the need to allocate two buffers at once. Instead it splits allocations into critical and non-critical ones and implements a reserve in the pool for this. The deadlock situation happens when all buffers are be allocated for requests pending in a maxconn-limited server queue, because then there's no more way to allocate buffers for responses, and these responses are critical to release the servers's connection in order to release the pending requests. In fact maxconn on a server creates a dependence between sessions and particularly between oldest session's responses and latest session's requests. Thus, it is mandatory to get a free buffer for a response in order to release a server connection which will permit to release a request buffer. Since we definitely have non-symmetrical buffers, we need to implement this logic in the buffer allocation mechanism. What this commit does is implement a reserve of buffers which can only be allocated for responses and that will never be allocated for requests. This is made possible by the requester indicating how much margin it wants to leave after the allocation succeeds. Thus it is a cooperative allocation mechanism : the requester (process_session() in general) prefers not to get a buffer in order to respect other's need for response buffers. The session management code always knows if a buffer will be used for requests or responses, so that is not difficult : - either there's an applet on the initiator side and we really need the request buffer (since currently the applet is called in the context of the session) - or we have a connection and we really need the response buffer (in order to support building and sending an error message back) This reserve ensures that we don't take all allocatable buffers for requests waiting in a queue. The downside is that all the extra buffers are really allocated to ensure they can be allocated. But with small values it is not an issue. With this change, we don't observe any more deadlocks even when running with maxconn 1 on a server under severely constrained memory conditions. The code becomes a bit tricky, it relies on the scheduler's run queue to estimate how many sessions are already expected to run so that it doesn't wake up everyone with too few resources. A better solution would probably consist in having two queues, one for urgent requests and one for normal requests. A failed allocation for a session dealing with an error, a connection event, or the need for a response (or request when there's an applet on the left) would go to the urgent request queue, while other requests would go to the other queue. Urgent requests would be served from 1 entry in the pool, while the regular ones would be served only according to the reserve. Despite not yet having this, it works remarkably well. This mechanism is quite efficient, we don't perform too many wake up calls anymore. For 1 million sessions elapsed during massive memory contention, we observe about 4.5M calls to process_session() compared to 4.0M without memory constraints. Previously we used to observe up to 16M calls, which rougly means 12M failures. During a test run under high memory constraints (limit enforced to 27 MB instead of the 58 MB normally needed), performance used to drop by 53% prior to this patch. Now with this patch instead it *increases* by about 1.5%. The best effect of this change is that by limiting the memory usage to about 2/3 to 3/4 of what is needed by default, it's possible to increase performance by up to about 18% mainly due to the fact that pools are reused more often and remain hot in the CPU cache (observed on regular HTTP traffic with 20k objects, buffers.limit = maxconn/10, buffers.reserve = limit/2). Below is an example of scenario which used to cause a deadlock previously : - connection is received - two buffers are allocated in process_session() then released - one is allocated when receiving an HTTP request - the second buffer is allocated then released in process_session() for request parsing then connection establishment. - poll() says we can send, so the request buffer is sent and released - process session gets notified that the connection is now established and allocates two buffers then releases them - all other sessions do the same till one cannot get the request buffer without hitting the margin - and now the server responds. stream_interface allocates the response buffer and manages to get it since it's higher priority being for a response. - but process_session() cannot allocate the request buffer anymore => We could end up with all buffers used by responses so that none may be allocated for a request in process_session(). When the applet processing leaves the session context, the test will have to be changed so that we always allocate a response buffer regardless of the left side (eg: H2->H1 gateway). A final improvement would consists in being able to only retry the failed I/O operation without waking up a task, but to date all experiments to achieve this have proven not to be reliable enough.
2014-11-27 00:11:56 +00:00
{
int avail;
if (LIST_ISEMPTY(&buffer_wq))
return;
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
/* all streams will need 1 buffer, so we can stop waking up streams
MAJOR: session: only wake up as many sessions as available buffers permit We've already experimented with three wake up algorithms when releasing buffers : the first naive one used to wake up far too many sessions, causing many of them not to get any buffer. The second approach which was still in use prior to this patch consisted in waking up either 1 or 2 sessions depending on the number of FDs we had released. And this was still inaccurate. The third one tried to cover the accuracy issues of the second and took into consideration the number of FDs the sessions would be willing to use, but most of the time we ended up waking up too many of them for nothing, or deadlocking by lack of buffers. This patch completely removes the need to allocate two buffers at once. Instead it splits allocations into critical and non-critical ones and implements a reserve in the pool for this. The deadlock situation happens when all buffers are be allocated for requests pending in a maxconn-limited server queue, because then there's no more way to allocate buffers for responses, and these responses are critical to release the servers's connection in order to release the pending requests. In fact maxconn on a server creates a dependence between sessions and particularly between oldest session's responses and latest session's requests. Thus, it is mandatory to get a free buffer for a response in order to release a server connection which will permit to release a request buffer. Since we definitely have non-symmetrical buffers, we need to implement this logic in the buffer allocation mechanism. What this commit does is implement a reserve of buffers which can only be allocated for responses and that will never be allocated for requests. This is made possible by the requester indicating how much margin it wants to leave after the allocation succeeds. Thus it is a cooperative allocation mechanism : the requester (process_session() in general) prefers not to get a buffer in order to respect other's need for response buffers. The session management code always knows if a buffer will be used for requests or responses, so that is not difficult : - either there's an applet on the initiator side and we really need the request buffer (since currently the applet is called in the context of the session) - or we have a connection and we really need the response buffer (in order to support building and sending an error message back) This reserve ensures that we don't take all allocatable buffers for requests waiting in a queue. The downside is that all the extra buffers are really allocated to ensure they can be allocated. But with small values it is not an issue. With this change, we don't observe any more deadlocks even when running with maxconn 1 on a server under severely constrained memory conditions. The code becomes a bit tricky, it relies on the scheduler's run queue to estimate how many sessions are already expected to run so that it doesn't wake up everyone with too few resources. A better solution would probably consist in having two queues, one for urgent requests and one for normal requests. A failed allocation for a session dealing with an error, a connection event, or the need for a response (or request when there's an applet on the left) would go to the urgent request queue, while other requests would go to the other queue. Urgent requests would be served from 1 entry in the pool, while the regular ones would be served only according to the reserve. Despite not yet having this, it works remarkably well. This mechanism is quite efficient, we don't perform too many wake up calls anymore. For 1 million sessions elapsed during massive memory contention, we observe about 4.5M calls to process_session() compared to 4.0M without memory constraints. Previously we used to observe up to 16M calls, which rougly means 12M failures. During a test run under high memory constraints (limit enforced to 27 MB instead of the 58 MB normally needed), performance used to drop by 53% prior to this patch. Now with this patch instead it *increases* by about 1.5%. The best effect of this change is that by limiting the memory usage to about 2/3 to 3/4 of what is needed by default, it's possible to increase performance by up to about 18% mainly due to the fact that pools are reused more often and remain hot in the CPU cache (observed on regular HTTP traffic with 20k objects, buffers.limit = maxconn/10, buffers.reserve = limit/2). Below is an example of scenario which used to cause a deadlock previously : - connection is received - two buffers are allocated in process_session() then released - one is allocated when receiving an HTTP request - the second buffer is allocated then released in process_session() for request parsing then connection establishment. - poll() says we can send, so the request buffer is sent and released - process session gets notified that the connection is now established and allocates two buffers then releases them - all other sessions do the same till one cannot get the request buffer without hitting the margin - and now the server responds. stream_interface allocates the response buffer and manages to get it since it's higher priority being for a response. - but process_session() cannot allocate the request buffer anymore => We could end up with all buffers used by responses so that none may be allocated for a request in process_session(). When the applet processing leaves the session context, the test will have to be changed so that we always allocate a response buffer regardless of the left side (eg: H2->H1 gateway). A final improvement would consists in being able to only retry the failed I/O operation without waking up a task, but to date all experiments to achieve this have proven not to be reliable enough.
2014-11-27 00:11:56 +00:00
* once we have enough of them to eat all the buffers. Note that we
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
* don't really know if they are streams or just other tasks, but
MAJOR: session: only wake up as many sessions as available buffers permit We've already experimented with three wake up algorithms when releasing buffers : the first naive one used to wake up far too many sessions, causing many of them not to get any buffer. The second approach which was still in use prior to this patch consisted in waking up either 1 or 2 sessions depending on the number of FDs we had released. And this was still inaccurate. The third one tried to cover the accuracy issues of the second and took into consideration the number of FDs the sessions would be willing to use, but most of the time we ended up waking up too many of them for nothing, or deadlocking by lack of buffers. This patch completely removes the need to allocate two buffers at once. Instead it splits allocations into critical and non-critical ones and implements a reserve in the pool for this. The deadlock situation happens when all buffers are be allocated for requests pending in a maxconn-limited server queue, because then there's no more way to allocate buffers for responses, and these responses are critical to release the servers's connection in order to release the pending requests. In fact maxconn on a server creates a dependence between sessions and particularly between oldest session's responses and latest session's requests. Thus, it is mandatory to get a free buffer for a response in order to release a server connection which will permit to release a request buffer. Since we definitely have non-symmetrical buffers, we need to implement this logic in the buffer allocation mechanism. What this commit does is implement a reserve of buffers which can only be allocated for responses and that will never be allocated for requests. This is made possible by the requester indicating how much margin it wants to leave after the allocation succeeds. Thus it is a cooperative allocation mechanism : the requester (process_session() in general) prefers not to get a buffer in order to respect other's need for response buffers. The session management code always knows if a buffer will be used for requests or responses, so that is not difficult : - either there's an applet on the initiator side and we really need the request buffer (since currently the applet is called in the context of the session) - or we have a connection and we really need the response buffer (in order to support building and sending an error message back) This reserve ensures that we don't take all allocatable buffers for requests waiting in a queue. The downside is that all the extra buffers are really allocated to ensure they can be allocated. But with small values it is not an issue. With this change, we don't observe any more deadlocks even when running with maxconn 1 on a server under severely constrained memory conditions. The code becomes a bit tricky, it relies on the scheduler's run queue to estimate how many sessions are already expected to run so that it doesn't wake up everyone with too few resources. A better solution would probably consist in having two queues, one for urgent requests and one for normal requests. A failed allocation for a session dealing with an error, a connection event, or the need for a response (or request when there's an applet on the left) would go to the urgent request queue, while other requests would go to the other queue. Urgent requests would be served from 1 entry in the pool, while the regular ones would be served only according to the reserve. Despite not yet having this, it works remarkably well. This mechanism is quite efficient, we don't perform too many wake up calls anymore. For 1 million sessions elapsed during massive memory contention, we observe about 4.5M calls to process_session() compared to 4.0M without memory constraints. Previously we used to observe up to 16M calls, which rougly means 12M failures. During a test run under high memory constraints (limit enforced to 27 MB instead of the 58 MB normally needed), performance used to drop by 53% prior to this patch. Now with this patch instead it *increases* by about 1.5%. The best effect of this change is that by limiting the memory usage to about 2/3 to 3/4 of what is needed by default, it's possible to increase performance by up to about 18% mainly due to the fact that pools are reused more often and remain hot in the CPU cache (observed on regular HTTP traffic with 20k objects, buffers.limit = maxconn/10, buffers.reserve = limit/2). Below is an example of scenario which used to cause a deadlock previously : - connection is received - two buffers are allocated in process_session() then released - one is allocated when receiving an HTTP request - the second buffer is allocated then released in process_session() for request parsing then connection establishment. - poll() says we can send, so the request buffer is sent and released - process session gets notified that the connection is now established and allocates two buffers then releases them - all other sessions do the same till one cannot get the request buffer without hitting the margin - and now the server responds. stream_interface allocates the response buffer and manages to get it since it's higher priority being for a response. - but process_session() cannot allocate the request buffer anymore => We could end up with all buffers used by responses so that none may be allocated for a request in process_session(). When the applet processing leaves the session context, the test will have to be changed so that we always allocate a response buffer regardless of the left side (eg: H2->H1 gateway). A final improvement would consists in being able to only retry the failed I/O operation without waking up a task, but to date all experiments to achieve this have proven not to be reliable enough.
2014-11-27 00:11:56 +00:00
* that's a rough estimate. Similarly, for each cached event we'll need
* 1 buffer. If no buffer is currently used, always wake up the number
* of tasks we can offer a buffer based on what is allocated, and in
* any case at least one task per two reserved buffers.
*/
avail = pool2_buffer->allocated - pool2_buffer->used - global.tune.reserved_bufs / 2;
if (avail > (int)run_queue)
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
__stream_offer_buffers(avail);
MAJOR: session: only wake up as many sessions as available buffers permit We've already experimented with three wake up algorithms when releasing buffers : the first naive one used to wake up far too many sessions, causing many of them not to get any buffer. The second approach which was still in use prior to this patch consisted in waking up either 1 or 2 sessions depending on the number of FDs we had released. And this was still inaccurate. The third one tried to cover the accuracy issues of the second and took into consideration the number of FDs the sessions would be willing to use, but most of the time we ended up waking up too many of them for nothing, or deadlocking by lack of buffers. This patch completely removes the need to allocate two buffers at once. Instead it splits allocations into critical and non-critical ones and implements a reserve in the pool for this. The deadlock situation happens when all buffers are be allocated for requests pending in a maxconn-limited server queue, because then there's no more way to allocate buffers for responses, and these responses are critical to release the servers's connection in order to release the pending requests. In fact maxconn on a server creates a dependence between sessions and particularly between oldest session's responses and latest session's requests. Thus, it is mandatory to get a free buffer for a response in order to release a server connection which will permit to release a request buffer. Since we definitely have non-symmetrical buffers, we need to implement this logic in the buffer allocation mechanism. What this commit does is implement a reserve of buffers which can only be allocated for responses and that will never be allocated for requests. This is made possible by the requester indicating how much margin it wants to leave after the allocation succeeds. Thus it is a cooperative allocation mechanism : the requester (process_session() in general) prefers not to get a buffer in order to respect other's need for response buffers. The session management code always knows if a buffer will be used for requests or responses, so that is not difficult : - either there's an applet on the initiator side and we really need the request buffer (since currently the applet is called in the context of the session) - or we have a connection and we really need the response buffer (in order to support building and sending an error message back) This reserve ensures that we don't take all allocatable buffers for requests waiting in a queue. The downside is that all the extra buffers are really allocated to ensure they can be allocated. But with small values it is not an issue. With this change, we don't observe any more deadlocks even when running with maxconn 1 on a server under severely constrained memory conditions. The code becomes a bit tricky, it relies on the scheduler's run queue to estimate how many sessions are already expected to run so that it doesn't wake up everyone with too few resources. A better solution would probably consist in having two queues, one for urgent requests and one for normal requests. A failed allocation for a session dealing with an error, a connection event, or the need for a response (or request when there's an applet on the left) would go to the urgent request queue, while other requests would go to the other queue. Urgent requests would be served from 1 entry in the pool, while the regular ones would be served only according to the reserve. Despite not yet having this, it works remarkably well. This mechanism is quite efficient, we don't perform too many wake up calls anymore. For 1 million sessions elapsed during massive memory contention, we observe about 4.5M calls to process_session() compared to 4.0M without memory constraints. Previously we used to observe up to 16M calls, which rougly means 12M failures. During a test run under high memory constraints (limit enforced to 27 MB instead of the 58 MB normally needed), performance used to drop by 53% prior to this patch. Now with this patch instead it *increases* by about 1.5%. The best effect of this change is that by limiting the memory usage to about 2/3 to 3/4 of what is needed by default, it's possible to increase performance by up to about 18% mainly due to the fact that pools are reused more often and remain hot in the CPU cache (observed on regular HTTP traffic with 20k objects, buffers.limit = maxconn/10, buffers.reserve = limit/2). Below is an example of scenario which used to cause a deadlock previously : - connection is received - two buffers are allocated in process_session() then released - one is allocated when receiving an HTTP request - the second buffer is allocated then released in process_session() for request parsing then connection establishment. - poll() says we can send, so the request buffer is sent and released - process session gets notified that the connection is now established and allocates two buffers then releases them - all other sessions do the same till one cannot get the request buffer without hitting the margin - and now the server responds. stream_interface allocates the response buffer and manages to get it since it's higher priority being for a response. - but process_session() cannot allocate the request buffer anymore => We could end up with all buffers used by responses so that none may be allocated for a request in process_session(). When the applet processing leaves the session context, the test will have to be changed so that we always allocate a response buffer regardless of the left side (eg: H2->H1 gateway). A final improvement would consists in being able to only retry the failed I/O operation without waking up a task, but to date all experiments to achieve this have proven not to be reliable enough.
2014-11-27 00:11:56 +00:00
}
REORG/MAJOR: session: rename the "session" entity to "stream" With HTTP/2, we'll have to support multiplexed streams. A stream is in fact the largest part of what we currently call a session, it has buffers, logs, etc. In order to catch any error, this commit removes any reference to the struct session and tries to rename most "session" occurrences in function names to "stream" and "sess" to "strm" when that's related to a session. The files stream.{c,h} were added and session.{c,h} removed. The session will be reintroduced later and a few parts of the stream will progressively be moved overthere. It will more or less contain only what we need in an embryonic session. Sample fetch functions and converters will have to change a bit so that they'll use an L5 (session) instead of what's currently called "L4" which is in fact L6 for now. Once all changes are completed, we should see approximately this : L7 - http_txn L6 - stream L5 - session L4 - connection | applet There will be at most one http_txn per stream, and a same session will possibly be referenced by multiple streams. A connection will point to a session and to a stream. The session will hold all the information we need to keep even when we don't yet have a stream. Some more cleanup is needed because some code was already far from being clean. The server queue management still refers to sessions at many places while comments talk about connections. This will have to be cleaned up once we have a server-side connection pool manager. Stream flags "SN_*" still need to be renamed, it doesn't seem like any of them will need to move to the session.
2015-04-02 22:22:06 +00:00
#endif /* _PROTO_STREAM_H */
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/