RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2025-01-20 12:40:46 +00:00
Code Issues Packages Projects Releases Wiki Activity
524344b4e0
haproxy/src/ev_select.c

341 lines
7.7 KiB
C
Raw Normal View History

[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
* FD polling functions for generic select()
*
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
* Copyright 2000-2014 Willy Tarreau <w@1wt.eu>
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
*/
#include <unistd.h>
#include <sys/time.h>
#include <sys/types.h>
#include <common/compat.h>
#include <common/config.h>
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
#include <common/hathreads.h>
[MAJOR] convert all expiration timers from timeval to ticks This is the first attempt at moving all internal parts from using struct timeval to integer ticks. Those provides simpler and faster code due to simplified operations, and this change also saved about 64 bytes per session. A new header file has been added : include/common/ticks.h. It is possible that some functions should finally not be inlined because they're used quite a lot (eg: tick_first, tick_add_ifset and tick_is_expired). More measurements are required in order to decide whether this is interesting or not. Some function and variable names are still subject to change for a better overall logics.
2008-07-06 22:09:58 +00:00
#include <common/ticks.h>
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
#include <common/time.h>
#include <types/global.h>
#include <proto/fd.h>
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
/* private data */
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
static int maxfd; /* # of the highest fd + 1 */
MEDIUM: select: make use of hap_fd_* functions Given that FD_{CLR,SET} are not always guaranteed to be thread safe, let's fall back to using the hap_fd_* functions as we used to till 1.5-dev18 and as poll() continues to use. This will make it easier to remove the poll_lock.
2018-01-25 15:48:46 +00:00
static unsigned int *fd_evts[2];
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
static THREAD_LOCAL fd_set *tmp_evts[2];
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
MAJOR: ev_select: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 15:53:50 +00:00
/* Immediately remove the entry upon close() */
REGPRM1 static void __fd_clo(int fd)
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
{
MEDIUM: select: make use of hap_fd_* functions Given that FD_{CLR,SET} are not always guaranteed to be thread safe, let's fall back to using the hap_fd_* functions as we used to till 1.5-dev18 and as poll() continues to use. This will make it easier to remove the poll_lock.
2018-01-25 15:48:46 +00:00
hap_fd_clr(fd, fd_evts[DIR_RD]);
hap_fd_clr(fd, fd_evts[DIR_WR]);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
static void _update_fd(int fd, int *max_add_fd)
{
int en;
en = fdtab[fd].state;
/* we have a single state for all threads, which is why we
* don't check the tid_bit. First thread to see the update
* takes it for every other one.
*/
if (!(en & FD_EV_POLLED_RW)) {
MINOR: pollers: move polled_mask outside of struct fdtab. The polled_mask is only used in the pollers, and removing it from the struct fdtab makes it fit in one 64B cacheline again, on a 64bits machine, so make it a separate array.
2018-04-26 12:23:07 +00:00
if (!polled_mask[fd]) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
/* fd was not watched, it's still not */
return;
}
/* fd totally removed from poll list */
hap_fd_clr(fd, fd_evts[DIR_RD]);
hap_fd_clr(fd, fd_evts[DIR_WR]);
MINOR: pollers: move polled_mask outside of struct fdtab. The polled_mask is only used in the pollers, and removing it from the struct fdtab makes it fit in one 64B cacheline again, on a 64bits machine, so make it a separate array.
2018-04-26 12:23:07 +00:00
HA_ATOMIC_AND(&polled_mask[fd], 0);
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
}
else {
/* OK fd has to be monitored, it was either added or changed */
if (!(en & FD_EV_POLLED_R))
hap_fd_clr(fd, fd_evts[DIR_RD]);
else
hap_fd_set(fd, fd_evts[DIR_RD]);
if (!(en & FD_EV_POLLED_W))
hap_fd_clr(fd, fd_evts[DIR_WR]);
else
hap_fd_set(fd, fd_evts[DIR_WR]);
MINOR: pollers: move polled_mask outside of struct fdtab. The polled_mask is only used in the pollers, and removing it from the struct fdtab makes it fit in one 64B cacheline again, on a 64bits machine, so make it a separate array.
2018-04-26 12:23:07 +00:00
HA_ATOMIC_OR(&polled_mask[fd], tid_bit);
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
if (fd > *max_add_fd)
*max_add_fd = fd;
}
}
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
* Select() poller
*/
[MAJOR] convert all expiration timers from timeval to ticks This is the first attempt at moving all internal parts from using struct timeval to integer ticks. Those provides simpler and faster code due to simplified operations, and this change also saved about 64 bytes per session. A new header file has been added : include/common/ticks.h. It is possible that some functions should finally not be inlined because they're used quite a lot (eg: tick_first, tick_add_ifset and tick_is_expired). More measurements are required in order to decide whether this is interesting or not. Some function and variable names are still subject to change for a better overall logics.
2008-07-06 22:09:58 +00:00
REGPRM2 static void _do_poll(struct poller *p, int exp)
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
{
int status;
int fd, i;
struct timeval delta;
[MEDIUM] further improve monotonic clock by check forward jumps The first implementation of the monotonic clock did not verify forward jumps. The consequence is that a fast changing time may expire a lot of tasks. While it does seem minor, in fact it is problematic because most machines which boot with a wrong date are in the past and suddenly see their time jump by several years in the future. The solution is to check if we spent more apparent time in a poller than allowed (with a margin applied). The margin is currently set to 1000 ms. It should be large enough for any poll() to complete. Tests with randomly jumping clock show that the result is quite accurate (error less than 1 second at every change of more than one second).
2008-06-23 12:00:57 +00:00
int delta_ms;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
int fds;
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
int updt_idx;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
char count;
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
int readnotnull, writenotnull;
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
int old_maxfd, new_maxfd, max_add_fd;
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
int old_fd;
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
max_add_fd = -1;
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
MAJOR: ev_select: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 15:53:50 +00:00
/* first, scan the update list to find changes */
for (updt_idx = 0; updt_idx < fd_nbupdt; updt_idx++) {
fd = fd_updt[updt_idx];
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
BUG/MINOR: fd: Don't clear the update_mask in fd_insert. Clearing the update_mask bit in fd_insert may lead to duplicate insertion of fd in fd_updt, that could lead to a write past the end of the array. Instead, make sure the update_mask bit is cleared by the pollers no matter what. This should be backported to 1.8. [wt: warning: 1.8 doesn't have the lockless fdcache changes and will require some careful changes in the pollers]
2018-04-03 17:06:18 +00:00
HA_ATOMIC_AND(&fdtab[fd].update_mask, ~tid_bit);
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
if (!fdtab[fd].owner) {
activity[tid].poll_drop++;
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
continue;
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
_update_fd(fd, &max_add_fd);
}
/* Now scan the global update list */
for (old_fd = fd = update_list.first; fd != -1; fd = fdtab[fd].update.next) {
if (fd == -2) {
fd = old_fd;
continue;
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
}
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
else if (fd <= -3)
fd = -fd -4;
if (fd == -1)
break;
if (fdtab[fd].update_mask & tid_bit) {
/* Cheat a bit, as the state is global to all pollers
* we don't need every thread ot take care of the
* update.
*/
HA_ATOMIC_AND(&fdtab[fd].update_mask, ~all_threads_mask);
done_update_polling(fd);
} else
continue;
if (!fdtab[fd].owner)
continue;
_update_fd(fd, &max_add_fd);
MAJOR: ev_select: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 15:53:50 +00:00
}
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
/* maybe we added at least one fd larger than maxfd */
for (old_maxfd = maxfd; old_maxfd <= max_add_fd; ) {
if (HA_ATOMIC_CAS(&maxfd, &old_maxfd, max_add_fd + 1))
break;
}
/* maxfd doesn't need to be precise but it needs to cover *all* active
* FDs. Thus we only shrink it if we have such an opportunity. The algo
* is simple : look for the previous used place, try to update maxfd to
* point to it, abort if maxfd changed in the mean time.
*/
old_maxfd = maxfd;
do {
new_maxfd = old_maxfd;
while (new_maxfd - 1 >= 0 && !fdtab[new_maxfd - 1].owner)
new_maxfd--;
if (new_maxfd >= old_maxfd)
break;
} while (!HA_ATOMIC_CAS(&maxfd, &old_maxfd, new_maxfd));
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
thread_harmless_now();
MAJOR: ev_select: make the poller support speculative events The poller was updated to support speculative events. We'll need this to fully support SSL.
2012-11-11 15:53:50 +00:00
fd_nbupdt = 0;
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
/* let's restore fdset state */
readnotnull = 0; writenotnull = 0;
for (i = 0; i < (maxfd + FD_SETSIZE - 1)/(8*sizeof(int)); i++) {
readnotnull |= (*(((int*)tmp_evts[DIR_RD])+i) = *(((int*)fd_evts[DIR_RD])+i)) != 0;
writenotnull |= (*(((int*)tmp_evts[DIR_WR])+i) = *(((int*)fd_evts[DIR_WR])+i)) != 0;
}
[MAJOR] convert all expiration timers from timeval to ticks This is the first attempt at moving all internal parts from using struct timeval to integer ticks. Those provides simpler and faster code due to simplified operations, and this change also saved about 64 bytes per session. A new header file has been added : include/common/ticks.h. It is possible that some functions should finally not be inlined because they're used quite a lot (eg: tick_first, tick_add_ifset and tick_is_expired). More measurements are required in order to decide whether this is interesting or not. Some function and variable names are still subject to change for a better overall logics.
2008-07-06 22:09:58 +00:00
delta_ms = 0;
delta.tv_sec = 0;
delta.tv_usec = 0;
CLEANUP: poll: move the conditions for waiting out of the poll functions The poll() functions have become a bit dirty because they now check the size of the signal queue, the FD cache and the number of tasks. It's not their job, this must be moved to the caller. In the end it simplifies the code because the expiration date is now set to now_ms if we must not wait, and this achieves in exactly the same result and is cleaner. The change looks large due to the change of indent for blocks which were inside an "if" block.
2015-04-13 18:44:19 +00:00
if (!exp) {
delta_ms = MAX_DELAY_MS;
delta.tv_sec = (MAX_DELAY_MS / 1000);
delta.tv_usec = (MAX_DELAY_MS % 1000) * 1000;
}
else if (!tick_is_expired(exp, now_ms)) {
delta_ms = TICKS_TO_MS(tick_remain(now_ms, exp)) + SCHEDULER_RESOLUTION;
if (delta_ms > MAX_DELAY_MS)
delta_ms = MAX_DELAY_MS;
delta.tv_sec = (delta_ms / 1000);
delta.tv_usec = (delta_ms % 1000) * 1000;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
else
activity[tid].poll_exp++;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
[MEDIUM] poll: add a measurement of idle vs work time We now measure the work and idle times in order to report the idle time in the stats. It's expected that we'll be able to use it at other places later.
2011-09-10 14:56:42 +00:00
gettimeofday(&before_poll, NULL);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
status = select(maxfd,
[MEDIUM] pollers: store the events in arrays Instead of managing StaticReadEvent/StaticWriteEvent, use evts[dir]
2007-04-08 15:42:27 +00:00
readnotnull ? tmp_evts[DIR_RD] : NULL,
writenotnull ? tmp_evts[DIR_WR] : NULL,
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
NULL,
[MEDIUM] further improve monotonic clock by check forward jumps The first implementation of the monotonic clock did not verify forward jumps. The consequence is that a fast changing time may expire a lot of tasks. While it does seem minor, in fact it is problematic because most machines which boot with a wrong date are in the past and suddenly see their time jump by several years in the future. The solution is to check if we spent more apparent time in a poller than allowed (with a margin applied). The margin is currently set to 1000 ms. It should be large enough for any poll() to complete. Tests with randomly jumping clock show that the result is quite accurate (error less than 1 second at every change of more than one second).
2008-06-23 12:00:57 +00:00
&delta);
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
[MEDIUM] further improve monotonic clock by check forward jumps The first implementation of the monotonic clock did not verify forward jumps. The consequence is that a fast changing time may expire a lot of tasks. While it does seem minor, in fact it is problematic because most machines which boot with a wrong date are in the past and suddenly see their time jump by several years in the future. The solution is to check if we spent more apparent time in a poller than allowed (with a margin applied). The margin is currently set to 1000 ms. It should be large enough for any poll() to complete. Tests with randomly jumping clock show that the result is quite accurate (error less than 1 second at every change of more than one second).
2008-06-23 12:00:57 +00:00
tv_update_date(delta_ms, status);
[MEDIUM] poll: add a measurement of idle vs work time We now measure the work and idle times in order to report the idle time in the stats. It's expected that we'll be able to use it at other places later.
2011-09-10 14:56:42 +00:00
measure_idle();
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
thread_harmless_end();
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
if (status <= 0)
return;
[CLEANUP] remove dependency on obsolete INTBITS macro The INTBITS macro was found to be already defined on some platforms, and to equal 32 (while INTBITS was 5 here). Due to pure luck, there was no declaration conflict, but it's nonetheless a problem to fix. Looking at the code showed that this macro was only used for left shifts and nothing else anymore. So the replacement is obvious. The new macro, BITS_PER_INT is more obviously correct.
2008-07-14 22:36:31 +00:00
for (fds = 0; (fds * BITS_PER_INT) < maxfd; fds++) {
[MEDIUM] pollers: store the events in arrays Instead of managing StaticReadEvent/StaticWriteEvent, use evts[dir]
2007-04-08 15:42:27 +00:00
if ((((int *)(tmp_evts[DIR_RD]))[fds] | ((int *)(tmp_evts[DIR_WR]))[fds]) == 0)
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
continue;
[CLEANUP] remove dependency on obsolete INTBITS macro The INTBITS macro was found to be already defined on some platforms, and to equal 32 (while INTBITS was 5 here). Due to pure luck, there was no declaration conflict, but it's nonetheless a problem to fix. Looking at the code showed that this macro was only used for left shifts and nothing else anymore. So the replacement is obvious. The new macro, BITS_PER_INT is more obviously correct.
2008-07-14 22:36:31 +00:00
for (count = BITS_PER_INT, fd = fds * BITS_PER_INT; count && fd < maxfd; count--, fd++) {
MINOR: polling: Use fd_update_events to update events seen for a fd Now, the same function is used by all pollers to update events seen for a fd. This will ease the threads support integration.
2017-08-30 08:34:36 +00:00
unsigned int n = 0;
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
if (!fdtab[fd].owner) {
activity[tid].poll_dead++;
continue;
}
if (!(fdtab[fd].thread_mask & tid_bit)) {
activity[tid].poll_skip++;
CLEANUP: remove the now unused fdtab direct I/O callbacks They were all left to NULL since last commit so we can safely remove them all now and remove the temporary dual polling logic in pollers.
2012-07-06 14:02:29 +00:00
continue;
MINOR: global: add some global activity counters to help debugging A number of counters have been added at special places helping better understanding certain bug reports. These counters are maintained per thread and are shown using "show activity" on the CLI. The "clear counters" commands also reset these counters. The output is sent as a single write(), which currently produces up to about 7 kB of data for 64 threads. If more counters are added, it may be necessary to write into multiple buffers, or to reset the counters. To backport to 1.8 to help collect more detailed bug reports.
2018-01-20 18:30:13 +00:00
}
CLEANUP: remove the now unused fdtab direct I/O callbacks They were all left to NULL since last commit so we can safely remove them all now and remove the temporary dual polling logic in pollers.
2012-07-06 14:02:29 +00:00
if (FD_ISSET(fd, tmp_evts[DIR_RD]))
MINOR: polling: Use fd_update_events to update events seen for a fd Now, the same function is used by all pollers to update events seen for a fd. This will ease the threads support integration.
2017-08-30 08:34:36 +00:00
n |= FD_POLL_IN;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
CLEANUP: remove the now unused fdtab direct I/O callbacks They were all left to NULL since last commit so we can safely remove them all now and remove the temporary dual polling logic in pollers.
2012-07-06 14:02:29 +00:00
if (FD_ISSET(fd, tmp_evts[DIR_WR]))
MINOR: polling: Use fd_update_events to update events seen for a fd Now, the same function is used by all pollers to update events seen for a fd. This will ease the threads support integration.
2017-08-30 08:34:36 +00:00
n |= FD_POLL_OUT;
MAJOR: polling: centralize calls to I/O callbacks In order for HTTP/2 not to eat too much memory, we'll have to support on-the-fly buffer allocation, since most streams will have an empty request buffer at some point. Supporting allocation on the fly means being able to sleep inside I/O callbacks if a buffer is not available. Till now, the I/O callbacks were called from two locations : - when processing the cached events - when processing the polled events from the poller This change cleans up the design a bit further than what was started in 1.5. It now ensures that we never call any iocb from the poller itself and that instead, events learned by the poller are put into the cache. The benefit is important in terms of stability : we don't have to care anymore about the risk that new events are added into the poller while processing its events, and we're certain that updates are processed at a single location. To achieve this, we now modify all the fd_* functions so that instead of creating updates, they add/remove the fd to/from the cache depending on its state, and only create an update when the polling status reaches a state where it will have to change. Since the pollers make use of these functions to notify readiness (using fd_may_recv/fd_may_send), the cache is always up to date with the poller. Creating updates only when the polling status needs to change saves a significant amount of work for the pollers : a benchmark showed that on a typical TCP proxy test, the amount of updates per connection dropped from 11 to 1 on average. This also means that the update list is smaller and has more chances of not thrashing too many CPU cache lines. The first observed benefit is a net 2% performance gain on the connection rate. A second benefit is that when a connection is accepted, it's only when we're processing the cache, and the recv event is automatically added into the cache *after* the current one, resulting in this event to be processed immediately during the same loop. Previously we used to have a second run over the updates to detect if new events were added to catch them before waking up tasks. The next gain will be offered by the next steps on this subject consisting in implementing an I/O queue containing all cached events ordered by priority just like the run queue, and to be able to leave some events pending there as long as needed. That will allow us *not* to perform some FD processing if it's not the proper time for this (typically keep waiting for a buffer to be allocated if none is available for an recv()). And by only processing a small bunch of them, we'll allow priorities to take place even at the I/O level. As a result of this change, functions fd_alloc_or_release_cache_entry() and fd_process_polled_events() have disappeared, and the code dedicated to checking for new fd events after the callback during the poll() loop was removed as well. Despite the patch looking large, it's mostly a change of what function is falled upon fd_*() and almost nothing was added.
2014-11-19 18:43:05 +00:00
MINOR: polling: Use fd_update_events to update events seen for a fd Now, the same function is used by all pollers to update events seen for a fd. This will ease the threads support integration.
2017-08-30 08:34:36 +00:00
fd_update_events(fd, n);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
}
}
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
static int init_select_per_thread()
{
int fd_set_bytes;
fd_set_bytes = sizeof(fd_set) * (global.maxsock + FD_SETSIZE - 1) / FD_SETSIZE;
if ((tmp_evts[DIR_RD] = (fd_set *)calloc(1, fd_set_bytes)) == NULL)
goto fail;
if ((tmp_evts[DIR_WR] = (fd_set *)calloc(1, fd_set_bytes)) == NULL)
goto fail;
return 1;
fail:
free(tmp_evts[DIR_RD]);
free(tmp_evts[DIR_WR]);
return 0;
}
static void deinit_select_per_thread()
{
BUG/MEDIUM: threads: Run the poll loop on the main thread too There was a flaw in the way the threads was created. the main one was just used to create all the others and just wait to exit. Now, it is used to run a poll loop. So we only create nbthread-1 threads. This also fixes a bug about the compression filter when there is only 1 thread (nbthread == 1 or no threads support). The bug was in the way thread-local resources was initialized. per-thread init/deinit callbacks were never called for the main process. So, with nthread set to 1, some buffers remained uninitialized.
2017-10-27 11:53:47 +00:00
free(tmp_evts[DIR_WR]); tmp_evts[DIR_WR] = NULL;
free(tmp_evts[DIR_RD]); tmp_evts[DIR_RD] = NULL;
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
}
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
/*
* Initialization of the select() poller.
* Returns 0 in case of failure, non-zero in case of success. If it fails, it
* disables the poller by setting its pref to 0.
*/
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
REGPRM1 static int _do_init(struct poller *p)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
{
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
__label__ fail_swevt, fail_srevt, fail_revt;
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
int fd_set_bytes;
p->private = NULL;
BUG/MAJOR: ev_select: disable the select() poller if maxsock > FD_SETSIZE Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. For this reason, we now control the compatibility between global.maxsock and FD_SETSIZE, and refuse to use select() if there too many FDs are expected to be used. Note that on Solaris, FD_SETSIZE is already forced to 65536, and that FreeBSD and OpenBSD allow it to be redefined, though this is not needed thanks to kqueue which is much more efficient. In practice, since poll() is enabled on all targets, it should not cause any problem, unless it is explicitly disabled. This change must be backported to 1.4 because the crashes caused by glibc have already been reported on this version.
2013-03-31 12:41:15 +00:00
if (global.maxsock > FD_SETSIZE)
goto fail_revt;
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
fd_set_bytes = sizeof(fd_set) * (global.maxsock + FD_SETSIZE - 1) / FD_SETSIZE;
MEDIUM: select: make use of hap_fd_* functions Given that FD_{CLR,SET} are not always guaranteed to be thread safe, let's fall back to using the hap_fd_* functions as we used to till 1.5-dev18 and as poll() continues to use. This will make it easier to remove the poll_lock.
2018-01-25 15:48:46 +00:00
if ((fd_evts[DIR_RD] = calloc(1, fd_set_bytes)) == NULL)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
goto fail_srevt;
MEDIUM: select: make use of hap_fd_* functions Given that FD_{CLR,SET} are not always guaranteed to be thread safe, let's fall back to using the hap_fd_* functions as we used to till 1.5-dev18 and as poll() continues to use. This will make it easier to remove the poll_lock.
2018-01-25 15:48:46 +00:00
if ((fd_evts[DIR_WR] = calloc(1, fd_set_bytes)) == NULL)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
goto fail_swevt;
BUG/MEDIUM: threads: Run the poll loop on the main thread too There was a flaw in the way the threads was created. the main one was just used to create all the others and just wait to exit. Now, it is used to run a poll loop. So we only create nbthread-1 threads. This also fixes a bug about the compression filter when there is only 1 thread (nbthread == 1 or no threads support). The bug was in the way thread-local resources was initialized. per-thread init/deinit callbacks were never called for the main process. So, with nthread set to 1, some buffers remained uninitialized.
2017-10-27 11:53:47 +00:00
hap_register_per_thread_init(init_select_per_thread);
hap_register_per_thread_deinit(deinit_select_per_thread);
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
return 1;
fail_swevt:
free(fd_evts[DIR_RD]);
fail_srevt:
free(tmp_evts[DIR_WR]);
free(tmp_evts[DIR_RD]);
fail_revt:
p->pref = 0;
return 0;
}
/*
* Termination of the select() poller.
* Memory is released and the poller is marked as unselectable.
*/
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
REGPRM1 static void _do_term(struct poller *p)
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
{
[CLEANUP] remove 65 useless NULL checks before free C specification clearly states that free(NULL) is a no-op. So remove useless checks before calling free.
2008-08-03 10:19:50 +00:00
free(fd_evts[DIR_WR]);
free(fd_evts[DIR_RD]);
[MINOR] ev_* : moved the poll function closer to fd_*
2007-04-09 07:23:31 +00:00
p->private = NULL;
p->pref = 0;
}
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
/*
* Check that the poller works.
* Returns 1 if OK, otherwise 0.
*/
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
REGPRM1 static int _do_test(struct poller *p)
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
{
BUG/MAJOR: ev_select: disable the select() poller if maxsock > FD_SETSIZE Some recent glibc updates have added controls on FD_SET/FD_CLR/FD_ISSET that crash the program if it tries to use a file descriptor larger than FD_SETSIZE. For this reason, we now control the compatibility between global.maxsock and FD_SETSIZE, and refuse to use select() if there too many FDs are expected to be used. Note that on Solaris, FD_SETSIZE is already forced to 65536, and that FreeBSD and OpenBSD allow it to be redefined, though this is not needed thanks to kqueue which is much more efficient. In practice, since poll() is enabled on all targets, it should not cause any problem, unless it is explicitly disabled. This change must be backported to 1.4 because the crashes caused by glibc have already been reported on this version.
2013-03-31 12:41:15 +00:00
if (global.maxsock > FD_SETSIZE)
return 0;
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
return 1;
}
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
* It is a constructor, which means that it will automatically be called before
* main(). This is GCC-specific but it works at least since 2.95.
* Special care must be taken so that it does not need any uninitialized data.
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
*/
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
__attribute__((constructor))
static void _do_register(void)
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
{
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
struct poller *p;
if (nbpollers >= MAX_POLLERS)
return;
p = &pollers[nbpollers++];
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
p->name = "select";
p->pref = 150;
MINOR: fd: add a new flag HAP_POLL_F_RDHUP to struct poller We'll need to differenciate between pollers which can report hangup at the same time as read (POLL_RDHUP) from the other ones, because only these ones may benefit from the fd_done_recv() optimization. Epoll has had support for EPOLLRDHUP since Linux 2.6.17 and has always been used this way in haproxy, so now we only set the flag once we've observed it once in a response. It means that some initial requests may try to perform a second recv() call, but after the first closed connection it will be enough to know that the second call is not needed anymore. Later we may extend these flags to designate event-triggered pollers.
2017-03-13 10:38:28 +00:00
p->flags = 0;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
p->private = NULL;
MAJOR: polling: remove unused callbacks from the poller struct Since no poller uses poller->{set,clr,wai,is_set,rem} anymore, let's remove them and remove the associated pointer tests in proto/fd.h.
2012-11-11 20:02:34 +00:00
p->clo = __fd_clo;
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
p->test = _do_test;
p->init = _do_init;
p->term = _do_term;
p->poll = _do_poll;
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
}
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/
Reference in New Issue Copy Permalink