haproxy/include/common
Willy Tarreau 63a8738724 MEDIUM: pools: directly free objects when pools are too much crowded
During pool_free(), when the ->allocated value is 125% of needed_avg or
more, instead of putting the object back into the pool, it's immediately
freed using free(). By doing this we manage to significantly reduce the
amount of memory pinned in pools after transient traffic spikes.

During a test involving a constant load of 100 concurrent connections
each delivering 100 requests per second, the memory usage was a steady
21 MB RSS. Adding a 1 minute parallel load of 40k connections all looping
on 100kB objects made the memory usage climb to 938 MB before this patch.
With the patch it was only 660 MB. But when this parasit load stopped,
before the patch the RSS would remain at 938 MB while with the patch,
it went down to 480 then 180 MB after a few seconds, to stabilize around
69 MB after about 20 seconds.

This can be particularly important to improve reloads where the memory
has to be shared between the old and new process.

Another improvement would be welcome, we ought to have a periodic task
to check pools usage and continue to free up unused objects regardless
of any call to pool_free(), because the needed_avg value depends on the
past and will not cover recently refilled objects.
2020-05-27 08:32:42 +02:00
..
base64.h
buf.h MINOR: buf: Add function to insert a string at an absolute offset in a buffer 2020-03-06 14:12:59 +01:00
buffer.h MEDIUM: buffer: remove the buffer_wq lock 2020-02-26 10:39:36 +01:00
cfgparse.h Revert "MEDIUM: sink: add global statement to create a new ring (sink buffer)" 2020-05-20 12:06:16 +02:00
chunk.h MINOR: chunk: implement chunk_strncpy() to copy partial strings 2020-02-14 19:02:06 +01:00
compat.h MINOR: debug: use our own backtrace function on clang+x86_64 2020-03-04 12:04:07 +01:00
compiler.h MINOR: debug: add a new DISGUISE() macro to pass a value as identity 2020-03-14 10:52:46 +01:00
config.h CLEANUP: remove THREAD_LOCAL from config.h 2020-05-09 09:08:09 +02:00
debug.h MINOR: debug: consume the write() result in BUG_ON() to silence a warning 2020-03-14 10:58:35 +01:00
defaults.h CLENAUP: config: move CONFIG_HAP_LOCKLESS_POOLS out of config.h 2020-05-09 09:02:35 +02:00
errors.h
fcgi.h MINOR: fcgi: Add function to get the string representation of a record type 2019-10-04 16:12:02 +02:00
h1.h CLEANUP: assorted typo fixes in the code and comments 2020-03-14 09:42:07 +01:00
h2.h MINOR: h2: add a function to report H2 error codes as strings 2019-11-25 11:34:26 +01:00
hash.h BUG/MAJOR: hashes: fix the signedness of the hash inputs 2020-01-16 08:23:42 +01:00
hathreads.h CLEANUP: remove THREAD_LOCAL from config.h 2020-05-09 09:08:09 +02:00
hpack-dec.h BUILD: use inttypes.h instead of stdint.h 2019-04-01 07:44:56 +02:00
hpack-enc.h BUILD: use inttypes.h instead of stdint.h 2019-04-01 07:44:56 +02:00
hpack-huff.h BUILD: use inttypes.h instead of stdint.h 2019-04-01 07:44:56 +02:00
hpack-tbl.h BUILD: hpack: make sure the hpack table can still be built standalone 2020-05-22 12:13:43 +02:00
http-hdr.h BUILD: use inttypes.h instead of stdint.h 2019-04-01 07:44:56 +02:00
http.h CLEANUP: assorted typo fixes in the code and comments 2020-03-14 09:42:07 +01:00
htx.h MINOR: htx: Add a function to copy a buffer in an HTX message 2020-05-20 18:27:14 +02:00
initcall.h MINOR: initcall: Don't forget to define the __start/stop_init_##stg symbols. 2019-04-10 16:33:25 +02:00
ist.h MINOR: ist: add istissame() function 2020-05-05 15:28:59 +02:00
istbuf.h MINOR: istbuf: add ist2buf() function 2020-05-05 15:28:59 +02:00
memory.h MEDIUM: pools: directly free objects when pools are too much crowded 2020-05-27 08:32:42 +02:00
mini-clist.h MINOR: mini-clist: Add functions to iterate backward on a list 2020-04-27 09:39:37 +02:00
namespace.h CLEANUP: build: rename some build macros to use the USE_* ones 2019-05-22 19:47:57 +02:00
net_helper.h CLEANUP: net_helper: Do not negate the result of unlikely 2020-02-25 07:30:49 +01:00
openssl-compat.h BUG/MINOR: sample/ssl: Fix digest converter for openssl < 1.1.0 2020-05-12 16:30:41 +02:00
regex.h CLEANUP: remove THREAD_LOCAL from config.h 2020-05-09 09:08:09 +02:00
standard.h MINOR: standard: Add my_memspn and my_memcspn 2020-04-27 09:39:38 +02:00
template.h
ticks.h
time.h BUILD: remove obsolete support for -mregparm / USE_REGPARM 2020-02-25 07:41:47 +01:00
tools.h BUG/MINOR: ssl: ssl_sock_load_pem_into_ckch is not consistent 2020-01-22 15:55:55 +01:00
uri_auth.h MINOR: stats: replace the ST_* uri_auth flags with STAT_* 2019-10-10 11:30:07 +02:00
version.h MINOR: version: this is development again, update the status 2019-11-25 20:38:32 +01:00
xref.h CLEANUP: assorted typo fixes in the code and comments 2020-03-14 09:42:07 +01:00