RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2025-02-20 20:57:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

MINOR: config: implement global setting tune.buffers.limit

Browse Source 
This setting is used to limit memory usage without causing the alloc
failures caused by "-m". Unexpectedly, tests have shown a performance
boost of up to about 18% on HTTP traffic when limiting the number of
buffers to about 10% of the amount of concurrent connections.

tune.buffers.limit <number>
  Sets a hard limit on the number of buffers which may be allocated per process.
  The default value is zero which means unlimited. The minimum non-zero value
  will always be greater than "tune.buffers.reserve" and should ideally always
  be about twice as large. Forcing this value can be particularly useful to
  limit the amount of memory a process may take, while retaining a sane
  behaviour. When this limit is reached, sessions which need a buffer wait for
  another one to be released by another session. Since buffers are dynamically
  allocated and released, the waiting time is very short and not perceptible
  provided that limits remain reasonable. In fact sometimes reducing the limit
  may even increase performance by increasing the CPU cache's efficiency. Tests
  have shown good results on average HTTP traffic with a limit to 1/10 of the
  expected global maxconn setting, which also significantly reduces memory
  usage. The memory savings come from the fact that a number of connections
  will not allocate 2*tune.bufsize. It is best not to touch this value unless
  advised to do so by an haproxy core developer.
This commit is contained in:
Willy Tarreau 2014-12-23 22:52:37 +01:00
parent 1058ae73f1
commit 33cb065348
4 changed files with 36 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
doc/configuration.txt
View File

@ -891,6 +891,23 @@ spread-checks <0..50, in percent>
and +/- 50%. A value between 2 and 5 seems to show good results. The
default value remains at 0.
tune.buffers.limit <number>
Sets a hard limit on the number of buffers which may be allocated per process.
The default value is zero which means unlimited. The minimum non-zero value
will always be greater than "tune.buffers.reserve" and should ideally always
be about twice as large. Forcing this value can be particularly useful to
limit the amount of memory a process may take, while retaining a sane
behaviour. When this limit is reached, sessions which need a buffer wait for
another one to be released by another session. Since buffers are dynamically
allocated and released, the waiting time is very short and not perceptible
provided that limits remain reasonable. In fact sometimes reducing the limit
may even increase performance by increasing the CPU cache's efficiency. Tests
have shown good results on average HTTP traffic with a limit to 1/10 of the
expected global maxconn setting, which also significantly reduces memory
usage. The memory savings come from the fact that a number of connections
will not allocate 2*tune.bufsize. It is best not to touch this value unless
advised to do so by an haproxy core developer.
tune.buffers.reserve <number>
Sets the number of buffers which are pre-allocated and reserved for use only
during memory shortage conditions resulting in failed memory allocations. The

1
include/types/global.h
View File

@ -129,6 +129,7 @@ struct global {
int bufsize; /* buffer size in bytes, defaults to BUFSIZE */
int maxrewrite; /* buffer max rewrite size in bytes, defaults to MAXREWRITE */
int reserved_bufs; /* how many buffers can only be allocated for response */
int buf_limit; /* if not null, how many total buffers may only be allocated */
int client_sndbuf; /* set client sndbuf to this value if not null */
int client_rcvbuf; /* set client rcvbuf to this value if not null */
int server_sndbuf; /* set server sndbuf to this value if not null */

2
src/buffer.c
View File

@ -48,6 +48,8 @@ int init_buffer()
* release a server connection).
*/
pool2_buffer->minavail = MAX(global.tune.reserved_bufs, 3);
if (global.tune.buf_limit)
pool2_buffer->limit = global.tune.buf_limit;
buffer = pool_refill_alloc(pool2_buffer, pool2_buffer->minavail - 1);
if (!buffer)

16
src/cfgparse.c
View File

@ -696,6 +696,20 @@ int cfg_parse_global(const char *file, int linenum, char **args, int kwm)
}
}
#endif
else if (!strcmp(args[0], "tune.buffers.limit")) {
if (*(args[1]) == 0) {
Alert("parsing [%s:%d] : '%s' expects an integer argument.\n", file, linenum, args[0]);
err_code |= ERR_ALERT | ERR_FATAL;
goto out;
}
global.tune.buf_limit = atol(args[1]);
if (global.tune.buf_limit) {
if (global.tune.buf_limit < 3)
global.tune.buf_limit = 3;
if (global.tune.buf_limit <= global.tune.reserved_bufs)
global.tune.buf_limit = global.tune.reserved_bufs + 1;
}
}
else if (!strcmp(args[0], "tune.buffers.reserve")) {
if (*(args[1]) == 0) {
Alert("parsing [%s:%d] : '%s' expects an integer argument.\n", file, linenum, args[0]);
@ -705,6 +719,8 @@ int cfg_parse_global(const char *file, int linenum, char **args, int kwm)
global.tune.reserved_bufs = atol(args[1]);
if (global.tune.reserved_bufs < 2)
global.tune.reserved_bufs = 2;
if (global.tune.buf_limit && global.tune.buf_limit <= global.tune.reserved_bufs)
global.tune.buf_limit = global.tune.reserved_bufs + 1;
}
else if (!strcmp(args[0], "tune.bufsize")) {
if (*(args[1]) == 0) {