RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2025-02-22 13:46:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
51f291c795
haproxy/src/ev_poll.c

349 lines
7.9 KiB
C
Raw Normal View History

[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
* FD polling functions for generic poll()
*
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
* Copyright 2000-2014 Willy Tarreau <w@1wt.eu>
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
*/
OPTIM: poll: enable support for POLLRDHUP On Linux since 2.6.17 poll() supports POLLRDHUP to notify of an upcoming hangup after pending data. Making use of it allows us to avoid a useless recv() after short responses on keep-alive connections. Note that we automatically enable the feature once this flag has been met first in a poll() status. Till now it was only enabled on epoll.
2017-03-13 16:14:51 +00:00
#define _GNU_SOURCE // for POLLRDHUP on Linux
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
#include <unistd.h>
OPTIM: poll: enable support for POLLRDHUP On Linux since 2.6.17 poll() supports POLLRDHUP to notify of an upcoming hangup after pending data. Making use of it allows us to avoid a useless recv() after short responses on keep-alive connections. Note that we automatically enable the feature once this flag has been met first in a poll() status. Till now it was only enabled on epoll.
2017-03-13 16:14:51 +00:00
#include <poll.h>
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
#include <sys/time.h>
#include <sys/types.h>
CLEANUP: include: tree-wide alphabetical sort of include files This patch fixes all the leftovers from the include cleanup campaign. There were not that many (~400 entries in ~150 files) but it was definitely worth doing it as it revealed a few duplicates.
2020-06-09 07:07:15 +00:00
#include <haproxy/activity.h>
REORG: include: update all files to use haproxy/api.h or api-t.h if needed All files that were including one of the following include files have been updated to only include haproxy/api.h or haproxy/api-t.h once instead: - common/config.h - common/compat.h - common/compiler.h - common/defaults.h - common/initcall.h - common/tools.h The choice is simple: if the file only requires type definitions, it includes api-t.h, otherwise it includes the full api.h. In addition, in these files, explicit includes for inttypes.h and limits.h were dropped since these are now covered by api.h and api-t.h. No other change was performed, given that this patch is large and affects 201 files. At least one (tools.h) was already freestanding and didn't get the new one added.
2020-05-27 10:58:42 +00:00
#include <haproxy/api.h>
REORG: time: move time-keeping code and variables to clock.c There is currently a problem related to time keeping. We're mixing the functions to perform calculations with the os-dependent code needed to retrieve and adjust the local time. This patch extracts from time.{c,h} the parts that are solely dedicated to time keeping. These are the "now" or "before_poll" variables for example, as well as the various now_*() functions that make use of gettimeofday() and clock_gettime() to retrieve the current time. The "tv_*" functions moved there were also more appropriately renamed to "clock_*". Other parts used to compute stolen time are in other files, they will have to be picked next.
2021-10-08 07:33:24 +00:00
#include <haproxy/clock.h>
CLEANUP: include: tree-wide alphabetical sort of include files This patch fixes all the leftovers from the include cleanup campaign. There were not that many (~400 entries in ~150 files) but it was definitely worth doing it as it revealed a few duplicates.
2020-06-09 07:07:15 +00:00
#include <haproxy/fd.h>
#include <haproxy/global.h>
BUG/MINOR: signals/poller: set the poller timeout to 0 when there are signals When receiving a signal before entering the poller, and without any activity in the process, the poller will be entered with a timeout calculated without checking the signals. Since commit 4f59d3 ("MINOR: time: increase the minimum wakeup interval to 60s") the issue is much more visible because it could be stuck for 60s. When in mworker mode, if a worker quits and the SIGCHLD signal deliver at the right time to the master, this one could be stuck for the time of the timeout. This should fix issue #1841 Must be backported in every stable version.
2022-09-08 15:46:31 +00:00
#include <haproxy/signal.h>
REORG: sched: move idle time calculation from time.h to task.h time.h is a horrible place to put activity calculation, it's a historical mistake because the functions were there. We already have most of the parts in sched.{c,h} and these ones make an exception in the middle, forcing time.h to include some thread stuff and to access the before/after_poll and idle_pct values. Let's move these 3 functions to task.h with the other ones. They were prefixed with "sched_" instead of the historical "tv_" which already made no sense anymore.
2021-09-30 15:53:22 +00:00
#include <haproxy/task.h>
REORG: include: move common/ticks.h to haproxy/ticks.h Nothing needed to be changed, there are no exported types.
2020-06-02 16:15:32 +00:00
#include <haproxy/ticks.h>
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
OPTIM: poll: enable support for POLLRDHUP On Linux since 2.6.17 poll() supports POLLRDHUP to notify of an upcoming hangup after pending data. Making use of it allows us to avoid a useless recv() after short responses on keep-alive connections. Note that we automatically enable the feature once this flag has been met first in a poll() status. Till now it was only enabled on epoll.
2017-03-13 16:14:51 +00:00
#ifndef POLLRDHUP
/* POLLRDHUP was defined late in libc, and it appeared in kernel 2.6.17 */
#define POLLRDHUP 0
#endif
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
static int maxfd; /* # of the highest fd + 1 */
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
static unsigned int *fd_evts[2];
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/* private data */
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
static THREAD_LOCAL int nbfd = 0;
static THREAD_LOCAL struct pollfd *poll_events = NULL;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
BUILD: remove obsolete support for -mregparm / USE_REGPARM This used to be a minor optimization on ix86 where registers are scarce and the calling convention not very efficient, but this platform is not relevant enough anymore to warrant all this dirt in the code for the sake of saving 1 or 2% of performance. Modern platforms don't use this at all since their calling convention already defaults to using several registers so better get rid of this once for all.
2020-02-25 06:38:05 +00:00
static void __fd_clo(int fd)
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
{
hap_fd_clr(fd, fd_evts[DIR_RD]);
hap_fd_clr(fd, fd_evts[DIR_WR]);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
static void _update_fd(int fd, int *max_add_fd)
{
int en;
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
ulong pr, ps;
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
en = fdtab[fd].state;
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
pr = _HA_ATOMIC_LOAD(&polled_mask[fd].poll_recv);
ps = _HA_ATOMIC_LOAD(&polled_mask[fd].poll_send);
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
/* we have a single state for all threads, which is why we
* don't check the tid_bit. First thread to see the update
* takes it for every other one.
*/
MEDIUM: fd: remove the FD_EV_POLLED status bit Since commit 7ac0e35f2 in 1.9-dev1 ("MAJOR: fd: compute the new fd polling state out of the fd lock") we've started to update the FD POLLED bit a bit more aggressively. Lately with the removal of the FD cache, this bit is always equal to the ACTIVE bit. There's no point continuing to watch it and update it anymore, all it does is create confusion and complicate the code. One interesting side effect is that it now becomes visible that all fd_*_{send,recv}() operations systematically call updt_fd_polling(), except fd_cant_recv()/fd_cant_send() which never saw it change.
2019-09-04 07:52:57 +00:00
if (!(en & FD_EV_ACTIVE_RW)) {
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
if (!(pr | ps)) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
/* fd was not watched, it's still not */
return;
}
/* fd totally removed from poll list */
hap_fd_clr(fd, fd_evts[DIR_RD]);
hap_fd_clr(fd, fd_evts[DIR_WR]);
MEDIUM: pollers: Remember the state for read and write for each threads. In the poller code, instead of just remembering if we're currently polling a fd or not, remember if we're polling it for writing and/or for reading, that way, we can avoid to modify the polling if it's already polled as needed.
2019-07-25 14:00:18 +00:00
_HA_ATOMIC_AND(&polled_mask[fd].poll_recv, 0);
_HA_ATOMIC_AND(&polled_mask[fd].poll_send, 0);
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
}
else {
/* OK fd has to be monitored, it was either added or changed */
MEDIUM: fd: remove the FD_EV_POLLED status bit Since commit 7ac0e35f2 in 1.9-dev1 ("MAJOR: fd: compute the new fd polling state out of the fd lock") we've started to update the FD POLLED bit a bit more aggressively. Lately with the removal of the FD cache, this bit is always equal to the ACTIVE bit. There's no point continuing to watch it and update it anymore, all it does is create confusion and complicate the code. One interesting side effect is that it now becomes visible that all fd_*_{send,recv}() operations systematically call updt_fd_polling(), except fd_cant_recv()/fd_cant_send() which never saw it change.
2019-09-04 07:52:57 +00:00
if (!(en & FD_EV_ACTIVE_R)) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
hap_fd_clr(fd, fd_evts[DIR_RD]);
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
if (pr & ti->ltid_bit)
_HA_ATOMIC_AND(&polled_mask[fd].poll_recv, ~ti->ltid_bit);
MEDIUM: pollers: Remember the state for read and write for each threads. In the poller code, instead of just remembering if we're currently polling a fd or not, remember if we're polling it for writing and/or for reading, that way, we can avoid to modify the polling if it's already polled as needed.
2019-07-25 14:00:18 +00:00
} else {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
hap_fd_set(fd, fd_evts[DIR_RD]);
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
if (!(pr & ti->ltid_bit))
_HA_ATOMIC_OR(&polled_mask[fd].poll_recv, ti->ltid_bit);
MEDIUM: pollers: Remember the state for read and write for each threads. In the poller code, instead of just remembering if we're currently polling a fd or not, remember if we're polling it for writing and/or for reading, that way, we can avoid to modify the polling if it's already polled as needed.
2019-07-25 14:00:18 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
MEDIUM: fd: remove the FD_EV_POLLED status bit Since commit 7ac0e35f2 in 1.9-dev1 ("MAJOR: fd: compute the new fd polling state out of the fd lock") we've started to update the FD POLLED bit a bit more aggressively. Lately with the removal of the FD cache, this bit is always equal to the ACTIVE bit. There's no point continuing to watch it and update it anymore, all it does is create confusion and complicate the code. One interesting side effect is that it now becomes visible that all fd_*_{send,recv}() operations systematically call updt_fd_polling(), except fd_cant_recv()/fd_cant_send() which never saw it change.
2019-09-04 07:52:57 +00:00
if (!(en & FD_EV_ACTIVE_W)) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
hap_fd_clr(fd, fd_evts[DIR_WR]);
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
if (ps & ti->ltid_bit)
_HA_ATOMIC_AND(&polled_mask[fd].poll_send, ~ti->ltid_bit);
} else {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
hap_fd_set(fd, fd_evts[DIR_WR]);
MEDIUM: fd/poller: turn polled_mask to group-local IDs This changes the signification of each bit in the polled_mask so that now each bit represents a local thread ID for the current group instead of a global thread ID. As such, all tests now apply to ltid_bit instead of tid_bit. No particular check was made to verify that the FD's tgid matches the current one because there should be no case where this is not true. A check was added in epoll's __fd_clo() to confirm it never differs unless expected (soft stop under thread isolation, or master in starting mode going to exec mode), but that doesn't prevent from doing the job: it only consists in checking in the group's threads those that are still polling this FD and to remove them. Some atomic loads were added at the various locations, and most repetitive references to polled_mask[fd].xx were turned to a local copy instead making the code much more clear.
2022-07-06 08:37:31 +00:00
if (!(ps & ti->ltid_bit))
_HA_ATOMIC_OR(&polled_mask[fd].poll_send, ti->ltid_bit);
MEDIUM: pollers: Remember the state for read and write for each threads. In the poller code, instead of just remembering if we're currently polling a fd or not, remember if we're polling it for writing and/or for reading, that way, we can avoid to modify the polling if it's already polled as needed.
2019-07-25 14:00:18 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
if (fd > *max_add_fd)
*max_add_fd = fd;
}
}
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
* Poll() poller
*/
BUILD: remove obsolete support for -mregparm / USE_REGPARM This used to be a minor optimization on ix86 where registers are scarce and the calling convention not very efficient, but this platform is not relevant enough anymore to warrant all this dirt in the code for the sake of saving 1 or 2% of performance. Modern platforms don't use this at all since their calling convention already defaults to using several registers so better get rid of this once for all.
2020-02-25 06:38:05 +00:00
static void _do_poll(struct poller *p, int exp, int wake)
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
{
int status;
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
int fd;
[MAJOR] replaced all timeouts with struct timeval The timeout functions were difficult to manipulate because they were rounding results to the millisecond. Thus, it was difficult to compare and to check what expired and what did not. Also, the comparison functions were heavy with multiplies and divides by 1000. Now, all timeouts are stored in timevals, reducing the number of operations for updates and leading to cleaner and more efficient code.
2007-05-12 20:35:00 +00:00
int wait_time;
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
int updt_idx;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
int fds, count;
int sr, sw;
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
int old_maxfd, new_maxfd, max_add_fd;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
unsigned rn, wn; /* read new, write new */
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
int old_fd;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
max_add_fd = -1;
MAJOR: ev_poll: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 16:25:15 +00:00
/* first, scan the update list to find changes */
for (updt_idx = 0; updt_idx < fd_nbupdt; updt_idx++) {
fd = fd_updt[updt_idx];
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
MEDIUM: fd/poller: turn update_mask to group-local IDs From now on, the FD's update_mask only refers to local thread IDs. However, there remains a limitation, in updt_fd_polling(), we temporarily have to check and set shared FDs against .thread_mask, which still contains global ones. As such, nbtgroups > 1 may break (but this is not yet supported without special build options).
2022-07-05 17:21:06 +00:00
_HA_ATOMIC_AND(&fdtab[fd].update_mask, ~ti->ltid_bit);
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
if (!fdtab[fd].owner) {
MINOR: activity: rename confusing poll_* fields in the output We have poll_drop, poll_dead and poll_skip which are confusingly named like their poll_io and poll_exp counterparts except that they are not per poll() call but per-fd. This patch renames them to poll_drop_fd(), poll_dead_fd() and poll_skip_fd() for this reason.
2020-06-17 18:35:33 +00:00
activity[tid].poll_drop_fd++;
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
continue;
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
_update_fd(fd, &max_add_fd);
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
/* Now scan the global update list */
MEDIUM: fd/poller: make the update-list per-group The update-list needs to be per-group because its inspection is based on a mask and we need to be certain when scanning it if a mask is for the same thread or another one. Once per-group there's no doubt about it, even if the FD's polling changes, the entry remains valid. It will be needed to check the tgid though. Note that a soft-stop or pause/resume might not necessarily work here with tgroups>1, because the operation might be delivered to a thread that doesn't belong to the group and whoe update mask will not reflect one that is interesting here. We can't do better at this stage.
2022-07-08 09:33:43 +00:00
for (old_fd = fd = update_list[tgid - 1].first; fd != -1; fd = fdtab[fd].update.next) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
if (fd == -2) {
fd = old_fd;
continue;
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
else if (fd <= -3)
fd = -fd -4;
if (fd == -1)
break;
MEDIUM: fd/poller: turn update_mask to group-local IDs From now on, the FD's update_mask only refers to local thread IDs. However, there remains a limitation, in updt_fd_polling(), we temporarily have to check and set shared FDs against .thread_mask, which still contains global ones. As such, nbtgroups > 1 may break (but this is not yet supported without special build options).
2022-07-05 17:21:06 +00:00
if (fdtab[fd].update_mask & ti->ltid_bit) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
/* Cheat a bit, as the state is global to all pollers
CLEANUP: few extra typo and fixes over last one ("ot" -> "to") As noticed by Tim there were a few incorrect fixes in the previous patch ("ot" -> "to" and not "or").
2021-01-06 16:35:12 +00:00
* we don't need every thread to take care of the
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
* update.
*/
MEDIUM: fd/poller: turn update_mask to group-local IDs From now on, the FD's update_mask only refers to local thread IDs. However, there remains a limitation, in updt_fd_polling(), we temporarily have to check and set shared FDs against .thread_mask, which still contains global ones. As such, nbtgroups > 1 may break (but this is not yet supported without special build options).
2022-07-05 17:21:06 +00:00
_HA_ATOMIC_AND(&fdtab[fd].update_mask, ~tg->threads_enabled);
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
done_update_polling(fd);
} else
continue;
if (!fdtab[fd].owner)
continue;
_update_fd(fd, &max_add_fd);
MAJOR: ev_poll: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 16:25:15 +00:00
}
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
/* maybe we added at least one fd larger than maxfd */
for (old_maxfd = maxfd; old_maxfd <= max_add_fd; ) {
MEDIUM: pollers: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:49:54 +00:00
if (_HA_ATOMIC_CAS(&maxfd, &old_maxfd, max_add_fd + 1))
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
break;
}
/* maxfd doesn't need to be precise but it needs to cover *all* active
* FDs. Thus we only shrink it if we have such an opportunity. The algo
* is simple : look for the previous used place, try to update maxfd to
* point to it, abort if maxfd changed in the mean time.
*/
old_maxfd = maxfd;
do {
new_maxfd = old_maxfd;
while (new_maxfd - 1 >= 0 && !fdtab[new_maxfd - 1].owner)
new_maxfd--;
if (new_maxfd >= old_maxfd)
break;
MEDIUM: pollers: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:49:54 +00:00
} while (!_HA_ATOMIC_CAS(&maxfd, &old_maxfd, new_maxfd));
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
MEDIUM: threads: add a stronger thread_isolate_full() call The current principle of running under isolation was made to access sensitive data while being certain that no other thread was using them in parallel, without necessarily having to place locks everywhere. The main use case are "show sess" and "show fd" which run over long chains of pointers. The thread_isolate() call relies on the "harmless" bit that indicates for a given thread that it's not currently doing such sensitive things, which is advertised using thread_harmless_now() and which ends usings thread_harmless_end(), which also waits for possibly concurrent threads to complete their work if they took this opportunity for starting something tricky. As some system calls were notoriously slow (e.g. mmap()), a bunch of thread_harmless_now() / thread_harmless_end() were placed around them to let waiting threads do their work while such other threads were not able to modify memory contents. But this is not sufficient for performing memory modifications. One such example is the server deletion code. By modifying memory, it not only requires that other threads are not playing with it, but are not either in the process of touching it. The fact that a pool_alloc() or pool_free() on some structure may call thread_harmless_now() and let another thread start to release the same object's memory is not acceptable. This patch introduces the concept of "idle threads". Threads entering the polling loop are idle, as well as those that are waiting for all others to become idle via the new function thread_isolate_full(). Once thread_isolate_full() is granted, the thread is not idle anymore, and it is released using thread_release() just like regular isolation. Its users have to keep in mind that across this call nothing is granted as another thread might have performed shared memory modifications. But such users are extremely rare and are actually expecting this from their peers as well. Note that that in case of backport, this patch depends on previous patch: MINOR: threads: make thread_release() not wait for other ones to complete
2021-08-04 09:44:17 +00:00
thread_idle_now();
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
thread_harmless_now();
MAJOR: ev_poll: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 16:25:15 +00:00
fd_nbupdt = 0;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
nbfd = 0;
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
for (fds = 0; (fds * 8*sizeof(**fd_evts)) < maxfd; fds++) {
rn = fd_evts[DIR_RD][fds];
wn = fd_evts[DIR_WR][fds];
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
if (!(rn|wn))
continue;
for (count = 0, fd = fds * 8*sizeof(**fd_evts); count < 8*sizeof(**fd_evts) && fd < maxfd; count++, fd++) {
sr = (rn >> count) & 1;
sw = (wn >> count) & 1;
if ((sr|sw)) {
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
if (!fdtab[fd].owner) {
/* should normally not happen here except
* due to rare thread concurrency
*/
continue;
}
MEDIUM: fd: make thread_mask now represent group-local IDs With the change that was started on other masks, the thread mask was still not fully converted, sometimes being used as a global mask and sometimes as a local one. This finishes the code modifications so that the mask is always considered as a group-local mask. This doesn't change anything as long as there's a single group, but is necessary for groups 2 and above since it's used against running_mask and so on.
2022-07-07 06:23:03 +00:00
if (!(fdtab[fd].thread_mask & ti->ltid_bit)) {
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
continue;
}
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
poll_events[nbfd].fd = fd;
OPTIM: poll: enable support for POLLRDHUP On Linux since 2.6.17 poll() supports POLLRDHUP to notify of an upcoming hangup after pending data. Making use of it allows us to avoid a useless recv() after short responses on keep-alive connections. Note that we automatically enable the feature once this flag has been met first in a poll() status. Till now it was only enabled on epoll.
2017-03-13 16:14:51 +00:00
poll_events[nbfd].events = (sr ? (POLLIN | POLLRDHUP) : 0) | (sw ? POLLOUT : 0);
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
nbfd++;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
}
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
BUG/MINOR: signals/poller: ensure wakeup from signals Add self-wake in signal_handler() to fix a race condition with a signal coming in between checking signal_queue_len and entering polling sleep. The changes in commit 43c891dda ("BUG/MINOR: signals/poller: set the poller timeout to 0 when there are signals") were insufficient. Move the signal_queue_len check from the poll implementations to run_poll_loop() to keep that logic in one place. The poll loops are terminated either by the parameter wake being set or wake up due to a write to their poller_wr_pipe by wake_thread() in signal_handler(). This fixes issue #1841. Must be backported in every stable version.
2022-09-09 08:21:00 +00:00
/* Now let's wait for polled events. */
wait_time = wake ? 0 : compute_poll_timeout(exp);
REORG: clock: move the updates of cpu/mono time to clock.c The entering_poll/leaving_poll/measure_idle functions that were hard to classify and used to move to various locations have now been placed into clock.c since it's precisely about time-keeping. The functions were renamed to clock_*. The samp_time and idle_time values are now static since there is no reason for them to be read from outside.
2021-10-08 08:43:59 +00:00
clock_entering_poll();
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
status = poll(poll_events, nbfd, wait_time);
REORG: time: move time-keeping code and variables to clock.c There is currently a problem related to time keeping. We're mixing the functions to perform calculations with the os-dependent code needed to retrieve and adjust the local time. This patch extracts from time.{c,h} the parts that are solely dedicated to time keeping. These are the "now" or "before_poll" variables for example, as well as the various now_*() functions that make use of gettimeofday() and clock_gettime() to retrieve the current time. The "tv_*" functions moved there were also more appropriately renamed to "clock_*". Other parts used to compute stolen time are in other files, they will have to be picked next.
2021-10-08 07:33:24 +00:00
clock_update_date(wait_time, status);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
MINOR: poller: centralize poll return handling When returning from the polling syscall, all pollers have a certain dance to follow, made of wall clock updates, thread harmless updates, idle time management and sleeping mask updates. Let's have a centralized function to deal with all of this boring stuff: fd_leaving_poll(), and make all the pollers use it.
2022-06-22 13:21:34 +00:00
fd_leaving_poll(wait_time, status);
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
MINOR: activity: report the number of times poll() reports I/O The "show activity" output mentions a number of indicators to explain wake up reasons but doesn't have the number of times poll() sees some I/O. And given that multiple events can happen simultaneously, it's not always possible to deduce this metric by subtracting. This patch adds a new "poll_io" counter that allows one to see how often poll() returns with at least one active FD. This should help detect stuck events and measure various ratios of poll sub-metrics.
2020-06-17 18:25:18 +00:00
if (status > 0)
activity[tid].poll_io++;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
for (count = 0; status > 0 && count < nbfd; count++) {
MINOR: polling: Use fd_update_events to update events seen for a fd Now, the same function is used by all pollers to update events seen for a fd. This will ease the threads support integration.
2017-08-30 08:34:36 +00:00
unsigned int n;
BUG/MINOR: polling: some events were not set in various pollers fdtab[].ev was only set in ev_sepoll. Unfortunately, some I/O handling functions now rely on this, so depending on the polling mechanism, some useless operations might have been performed, such as performing a useless recv() when a HUP was reported. This is a very old issue, the flags were only added to the fdtab and not propagated into any poller. Then they were used in ev_sepoll which needed them for the cache. It is unsure whether a backport to 1.4 is appropriate or not.
2012-07-06 09:16:01 +00:00
int e = poll_events[count].revents;
MEDIUM: poller: program the update in fd_update_events() for a migrated FD When an FD is migrated, all pollers program an update. That's useless code duplication, and when thread groups will be supported, this will require an extra round of locking just to verify the update_mask on return. Let's just program the update direction from fd_update_events() as it already does for closed FDs, this becomes more logical.
2022-07-09 16:55:37 +00:00
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
fd = poll_events[count].fd;
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
MINOR: poll/epoll: move detection of RDHUP support earlier Let's move the detection of support for RDHUP earlier and out of the FD update chain, as it complicates its simplification.
2021-07-29 14:19:24 +00:00
if ((e & POLLRDHUP) && !(cur_poller.flags & HAP_POLL_F_RDHUP))
_HA_ATOMIC_OR(&cur_poller.flags, HAP_POLL_F_RDHUP);
MINOR: debug: add a new DEBUG_FD build option When DEBUG_FD is set at build time, we'll keep a counter of per-FD events in the fdtab. This counter is reported in "show fd" even for closed FDs if not zero. The purpose is to help spot situations where an apparently closed FD continues to be reported in loops, or where some events are dismissed.
2020-06-23 08:04:54 +00:00
#ifdef DEBUG_FD
CLEANUP: atomic/tree-wide: replace single increments/decrements with inc/dec This patch replaces roughly all occurrences of an HA_ATOMIC_ADD(&foo, 1) or HA_ATOMIC_SUB(&foo, 1) with the equivalent HA_ATOMIC_INC(&foo) and HA_ATOMIC_DEC(&foo) respectively. These are 507 changes over 45 files.
2021-04-06 11:53:36 +00:00
_HA_ATOMIC_INC(&fdtab[fd].event_count);
MINOR: debug: add a new DEBUG_FD build option When DEBUG_FD is set at build time, we'll keep a counter of per-FD events in the fdtab. This counter is reported in "show fd" even for closed FDs if not zero. The purpose is to help spot situations where an apparently closed FD continues to be reported in loops, or where some events are dismissed.
2020-06-23 08:04:54 +00:00
#endif
OPTIM: poll: enable support for POLLRDHUP On Linux since 2.6.17 poll() supports POLLRDHUP to notify of an upcoming hangup after pending data. Making use of it allows us to avoid a useless recv() after short responses on keep-alive connections. Note that we automatically enable the feature once this flag has been met first in a poll() status. Till now it was only enabled on epoll.
2017-03-13 16:14:51 +00:00
if (!(e & ( POLLOUT | POLLIN | POLLERR | POLLHUP | POLLRDHUP )))
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
continue;
CLEANUP: remove the now unused fdtab direct I/O callbacks They were all left to NULL since last commit so we can safely remove them all now and remove the temporary dual polling logic in pollers.
2012-07-06 14:02:29 +00:00
/* ok, we found one active fd */
status--;
MEDIUM: fd: do not use the FD_POLL_* flags in the pollers anymore As mentioned in previous commit, these flags do not map well to modern poller capabilities. Let's use the FD_EV_*_{R,W} flags instead. This first patch only performs a 1-to-1 mapping making sure that the previously reported flags are still reported identically while using the closest possible semantics in the pollers. It's worth noting that kqueue will now support improvements such as returning distinctions between shut and errors on each direction, though this is not exploited for now.
2019-09-06 17:05:50 +00:00
n = ((e & POLLIN) ? FD_EV_READY_R : 0) |
((e & POLLOUT) ? FD_EV_READY_W : 0) |
((e & POLLRDHUP) ? FD_EV_SHUT_R : 0) |
((e & POLLHUP) ? FD_EV_SHUT_RW : 0) |
((e & POLLERR) ? FD_EV_ERR_RW : 0);
BUG/MINOR: polling: some events were not set in various pollers fdtab[].ev was only set in ev_sepoll. Unfortunately, some I/O handling functions now rely on this, so depending on the polling mechanism, some useless operations might have been performed, such as performing a useless recv() when a HUP was reported. This is a very old issue, the flags were only added to the fdtab and not propagated into any poller. Then they were used in ev_sepoll which needed them for the cache. It is unsure whether a backport to 1.4 is appropriate or not.
2012-07-06 09:16:01 +00:00
MEDIUM: poller: program the update in fd_update_events() for a migrated FD When an FD is migrated, all pollers program an update. That's useless code duplication, and when thread groups will be supported, this will require an extra round of locking just to verify the update_mask on return. Let's just program the update direction from fd_update_events() as it already does for closed FDs, this becomes more logical.
2022-07-09 16:55:37 +00:00
fd_update_events(fd, n);
MEDIUM: fd: rely more on fd_update_events() to detect changes This function already performs a number of checks prior to calling the IOCB, and detects the change of thread (FD migration). Half of the controls are still in each poller, and these pollers also maintain activity counters for various cases. Note that the unreliable test on thread_mask was removed so that only the one performed by fd_set_running() is now used, since this one is reliable. Let's centralize all that fd-specific logic into the function and make it return a status among: FD_UPDT_DONE, // update done, nothing else to be done FD_UPDT_DEAD, // FD was already dead, ignore it FD_UPDT_CLOSED, // FD was closed FD_UPDT_MIGRATED, // FD was migrated, ignore it now Some pollers already used to call it last and have nothing to do after it, regardless of the result. epoll has to delete the FD in case a migration is detected. Overall this removes more code than it adds.
2021-07-29 14:57:19 +00:00
}
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
static int init_poll_per_thread()
{
poll_events = calloc(1, sizeof(struct pollfd) * global.maxsock);
if (poll_events == NULL)
return 0;
return 1;
}
static void deinit_poll_per_thread()
{
CLEANUP: tree-wide: replace free(x);x=NULL with ha_free(&x) This makes the code more readable and less prone to copy-paste errors. In addition, it allows to place some __builtin_constant_p() predicates to trigger a link-time error in case the compiler knows that the freed area is constant. It will also produce compile-time error if trying to free something that is not a regular pointer (e.g. a function). The DEBUG_MEM_STATS macro now also defines an instance for ha_free() so that all these calls can be checked. 178 occurrences were converted. The vast majority of them were handled by the following Coccinelle script, some slightly refined to better deal with "&*x" or with long lines: @ rule @ expression E; @@ - free(E); - E = NULL; + ha_free(&E); It was verified that the resulting code is the same, more or less a handful of cases where the compiler optimized slightly differently the temporary variable that holds the copy of the pointer. A non-negligible amount of {free(str);str=NULL;str_len=0;} are still present in the config part (mostly header names in proxies). These ones should also be cleaned for the same reasons, and probably be turned into ist strings.
2021-02-20 09:46:51 +00:00
ha_free(&poll_events);
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
}
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
/*
* Initialization of the poll() poller.
* Returns 0 in case of failure, non-zero in case of success. If it fails, it
* disables the poller by setting its pref to 0.
*/
BUILD: remove obsolete support for -mregparm / USE_REGPARM This used to be a minor optimization on ix86 where registers are scarce and the calling convention not very efficient, but this platform is not relevant enough anymore to warrant all this dirt in the code for the sake of saving 1 or 2% of performance. Modern platforms don't use this at all since their calling convention already defaults to using several registers so better get rid of this once for all.
2020-02-25 06:38:05 +00:00
static int _do_init(struct poller *p)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
{
BUG/MEDIUM: threads: Run the poll loop on the main thread too There was a flaw in the way the threads was created. the main one was just used to create all the others and just wait to exit. Now, it is used to run a poll loop. So we only create nbthread-1 threads. This also fixes a bug about the compression filter when there is only 1 thread (nbthread == 1 or no threads support). The bug was in the way thread-local resources was initialized. per-thread init/deinit callbacks were never called for the main process. So, with nthread set to 1, some buffers remained uninitialized.
2017-10-27 11:53:47 +00:00
__label__ fail_swevt, fail_srevt;
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
int fd_evts_bytes;
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
p->private = NULL;
MEDIUM: poller: disable thread-groups for poll() and select() These old legacy pollers are not designed for this. They're still using a shared list of events for all threads, this will not scale at all, so there's no point in enabling thread-groups there. Modern systems have epoll, kqueue or event ports and do not need these ones. We arrange for failing at boot time, only when thread-groups > 1 so that existing setups will remain unaffected. If there's a compelling reason for supporting thread groups with these pollers in the future, the rework should not be too hard, it would just consume a lot of memory to have an fd_evts[] array per thread, but that is doable.
2022-07-09 21:38:46 +00:00
/* this old poller uses a process-wide FD list that cannot work with
* groups.
*/
if (global.nbtgroups > 1)
goto fail_srevt;
BUG/MINOR: poll: too large size allocation for FD events Commit 80da05a ("MEDIUM: poll: do not use FD_* macros anymore") which appeared in 1.5-dev18 and which was backported to 1.4.23 made explicit use of arrays of FDs mapped to unsigned ints. The problem lies in the allocated size for poll(), as the resulting size is in bits and not bytes, resulting in poll() arrays being 8 times larger than necessary! In practice poll() is not used on highly loaded systems, explaining why nobody noticed. But it definetely has to be addressed. This fix needs to be backported to all stable versions.
2018-01-17 14:48:53 +00:00
fd_evts_bytes = (global.maxsock + sizeof(**fd_evts) * 8 - 1) / (sizeof(**fd_evts) * 8) * sizeof(**fd_evts);
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
if ((fd_evts[DIR_RD] = calloc(1, fd_evts_bytes)) == NULL)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
goto fail_srevt;
MEDIUM: poll: do not use FD_* macros anymore Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. Do not rely on FD_* macros anymore and replace them with bit fields.
2013-03-31 12:06:57 +00:00
if ((fd_evts[DIR_WR] = calloc(1, fd_evts_bytes)) == NULL)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
goto fail_swevt;
BUG/MEDIUM: threads: Run the poll loop on the main thread too There was a flaw in the way the threads was created. the main one was just used to create all the others and just wait to exit. Now, it is used to run a poll loop. So we only create nbthread-1 threads. This also fixes a bug about the compression filter when there is only 1 thread (nbthread == 1 or no threads support). The bug was in the way thread-local resources was initialized. per-thread init/deinit callbacks were never called for the main process. So, with nthread set to 1, some buffers remained uninitialized.
2017-10-27 11:53:47 +00:00
hap_register_per_thread_init(init_poll_per_thread);
hap_register_per_thread_deinit(deinit_poll_per_thread);
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
return 1;
fail_swevt:
free(fd_evts[DIR_RD]);
fail_srevt:
p->pref = 0;
return 0;
}
/*
* Termination of the poll() poller.
* Memory is released and the poller is marked as unselectable.
*/
BUILD: remove obsolete support for -mregparm / USE_REGPARM This used to be a minor optimization on ix86 where registers are scarce and the calling convention not very efficient, but this platform is not relevant enough anymore to warrant all this dirt in the code for the sake of saving 1 or 2% of performance. Modern platforms don't use this at all since their calling convention already defaults to using several registers so better get rid of this once for all.
2020-02-25 06:38:05 +00:00
static void _do_term(struct poller *p)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
{
[CLEANUP] remove 65 useless NULL checks before free C specification clearly states that free(NULL) is a no-op. So remove useless checks before calling free.
2008-08-03 10:19:50 +00:00
free(fd_evts[DIR_WR]);
free(fd_evts[DIR_RD]);
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
p->private = NULL;
p->pref = 0;
}
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
/*
* Check that the poller works.
* Returns 1 if OK, otherwise 0.
*/
BUILD: remove obsolete support for -mregparm / USE_REGPARM This used to be a minor optimization on ix86 where registers are scarce and the calling convention not very efficient, but this platform is not relevant enough anymore to warrant all this dirt in the code for the sake of saving 1 or 2% of performance. Modern platforms don't use this at all since their calling convention already defaults to using several registers so better get rid of this once for all.
2020-02-25 06:38:05 +00:00
static int _do_test(struct poller *p)
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
{
return 1;
}
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
BUILD: pollers: use an initcall to register the pollers Pollers are among the few remaining blocks still using constructors to register themselves. That's not needed anymore since the initcalls so better turn to initcalls.
2022-04-25 17:00:55 +00:00
* Registers the poller.
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
*/
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
static void _do_register(void)
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
{
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
struct poller *p;
if (nbpollers >= MAX_POLLERS)
return;
p = &pollers[nbpollers++];
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
p->name = "poll";
p->pref = 200;
MINOR: pollers: add a new flag to indicate pollers reporting ERR & HUP In practice it's all pollers except select(). It turns out that we're keeping some legacy code only for select and enforcing it on all pollers, let's offer the pollers the ability to declare that they do not need that.
2019-11-28 17:17:33 +00:00
p->flags = HAP_POLL_F_ERRHUP;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
p->private = NULL;
MAJOR: polling: remove unused callbacks from the poller struct Since no poller uses poller->{set,clr,wai,is_set,rem} anymore, let's remove them and remove the associated pointer tests in proto/fd.h.
2012-11-11 20:02:34 +00:00
p->clo = __fd_clo;
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
p->test = _do_test;
p->init = _do_init;
p->term = _do_term;
p->poll = _do_poll;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
BUILD: pollers: use an initcall to register the pollers Pollers are among the few remaining blocks still using constructors to register themselves. That's not needed anymore since the initcalls so better turn to initcalls.
2022-04-25 17:00:55 +00:00
INITCALL0(STG_REGISTER, _do_register);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/
Reference in New Issue Copy Permalink