RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-24 13:42:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
b75e828b29
haproxy/include/proto/fd.h

563 lines
16 KiB
C
Raw Normal View History

[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
/*
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
* include/proto/fd.h
* File descriptors states.
*
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
* Copyright (C) 2000-2014 Willy Tarreau - w@1wt.eu
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
#ifndef _PROTO_FD_H
#define _PROTO_FD_H
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
#include <stdio.h>
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>
[CLEANUP] included common/version.h everywhere
2006-06-29 16:54:54 +00:00
#include <common/config.h>
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
#include <types/fd.h>
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
/* public variables */
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
MAJOR: fd/threads: Make the fdcache mostly lockless. Create a local, per-thread, fdcache, for file descriptors that only belongs to one thread, and make the global fd cache mostly lockless, as we can get a lot of contention on the fd cache lock.
2018-01-24 17:17:56 +00:00
extern volatile struct fdlist fd_cache;
extern volatile struct fdlist fd_cache_local[MAX_THREADS];
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
extern volatile struct fdlist update_list;
MINOR: pollers: move polled_mask outside of struct fdtab. The polled_mask is only used in the pollers, and removing it from the struct fdtab makes it fit in one 64B cacheline again, on a 64bits machine, so make it a separate array.
2018-04-26 12:23:07 +00:00
extern unsigned long *polled_mask;
MINOR: threads/fd: Use a bitfield to know if there are FDs for a thread in the FD cache A bitfield has been added to know if there are some FDs processable by a specific thread in the FD cache. When a FD is inserted in the FD cache, the bits corresponding to its thread_mask are set. On each thread, the bitfield is updated when the FD cache is processed. If there is no FD processed, the thread is removed from the bitfield by unsetting its tid_bit. Note that this bitfield is updated but not checked in fd_process_cached_events. So, when this function is called, the FDs cache is always processed. [wt: should be backported to 1.8 as it will help fix a design limitation]
2018-01-15 10:57:03 +00:00
extern unsigned long fd_cache_mask; // Mask of threads with events in the cache
MAJOR: threads/fd: Make fd stuffs thread-safe Many changes have been made to do so. First, the fd_updt array, where all pending FDs for polling are stored, is now a thread-local array. Then 3 locks have been added to protect, respectively, the fdtab array, the fd_cache array and poll information. In addition, a lock for each entry in the fdtab array has been added to protect all accesses to a specific FD or its information. For pollers, according to the poller, the way to manage the concurrency is different. There is a poller loop on each thread. So the set of monitored FDs may need to be protected. epoll and kqueue are thread-safe per-se, so there few things to do to protect these pollers. This is not possible with select and poll, so there is no sharing between the threads. The poller on each thread is independant from others. Finally, per-thread init/deinit functions are used for each pollers and for FD part for manage thread-local ressources. Now, you must be carefull when a FD is created during the HAProxy startup. All update on the FD state must be made in the threads context and never before their creation. This is mandatory because fd_updt array is thread-local and initialized only for threads. Because there is no pollers for the main one, this array remains uninitialized in this context. For this reason, listeners are now enabled in run_thread_poll_loop function, just like the worker pipe.
2017-05-29 08:40:41 +00:00
extern THREAD_LOCAL int *fd_updt; // FD updates list
extern THREAD_LOCAL int fd_nbupdt; // number of updates in the list
MINOR: pollers: Add a way to wake a thread sleeping in the poller. Add a new pipe, one per thread, so that we can write on it to wake a thread sleeping in a poller, and use it to wake threads supposed to take care of a task, if they are all sleeping.
2018-07-26 15:55:11 +00:00
extern int poller_wr_pipe[MAX_THREADS];
MINOR: fd: cache-align fdtab and fdcache locks These locks are highly contended, let's not make them share cache lines.
2017-11-26 10:07:34 +00:00
__decl_hathreads(extern HA_RWLOCK_T __attribute__((aligned(64))) fdcache_lock); /* global lock to protect fd_cache array */
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
/* Deletes an FD from the fdsets.
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
* The file descriptor is also closed.
*/
void fd_delete(int fd);
[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
MEDIUM: polling: start to move maxfd computation to the pollers Since only select() and poll() still make use of maxfd, let's move its computation right there in the pollers themselves, and only during each fd update pass. The computation doesn't need a lock anymore, only a few atomic ops. It will be accurate, be done much less often and will not be required anymore in the FD's fast patch. This provides a small performance increase of about 1% in connection rate when using epoll since we get rid of this computation which was performed under a lock.
2018-01-26 20:48:23 +00:00
/* Deletes an FD from the fdsets.
MINOR: proxy: Don't close FDs if not our proxy. When running with multiple process, if some proxies are just assigned to some processes, the other processes will just close the file descriptors for the listening sockets. However, we may still have to provide those sockets when reloading, so instead we just try hard to pretend those proxies are dead, while keeping the sockets opened. A new global option, no-reused-socket", has been added, to restore the old behavior of closing the sockets not bound to this process.
2017-04-05 23:05:05 +00:00
* The file descriptor is kept open.
*/
void fd_remove(int fd);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/* disable the specified poller */
void disable_poller(const char *poller_name);
[LICENSE] licensing clarifications
2006-06-15 19:48:13 +00:00
MINOR: pollers: Add a way to wake a thread sleeping in the poller. Add a new pipe, one per thread, so that we can write on it to wake a thread sleeping in a poller, and use it to wake threads supposed to take care of a task, if they are all sleeping.
2018-07-26 15:55:11 +00:00
void poller_pipe_io_handler(int fd);
[MINOR] turn every FD_* into functions On recent CPUs, functions are about twice as fast as inline FD_*, so there is now a #define CONFIG_HAP_INLINE_FD_SET to choose between the two modes.
2006-10-15 12:52:29 +00:00
/*
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
* Initialize the pollers till the best one is found.
* If none works, returns 0, otherwise 1.
[MAJOR] auto-registering of pollers at load time Gcc provides __attribute__((constructor)) which is very convenient to execute functions at startup right before main(). All the pollers have been converted to have their register() function declared like this, so that it is not necessary anymore to call them from a centralized file.
2007-04-15 22:25:25 +00:00
* The pollers register themselves just before main() is called.
[MINOR] turn every FD_* into functions On recent CPUs, functions are about twice as fast as inline FD_*, so there is now a #define CONFIG_HAP_INLINE_FD_SET to choose between the two modes.
2006-10-15 12:52:29 +00:00
*/
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
int init_pollers();
[MINOR] turn every FD_* into functions On recent CPUs, functions are about twice as fast as inline FD_*, so there is now a #define CONFIG_HAP_INLINE_FD_SET to choose between the two modes.
2006-10-15 12:52:29 +00:00
[MEDIUM] Fix memory freeing at exit New functions implemented: - deinit_pollers: called at the end of deinit()) - prune_acl: called via list_for_each_entry_safe Add missing pool_destroy2 calls: - p->hdr_idx_pool - pool2_tree64 Implement all task stopping: - health-check: needs new "struct task" in the struct server - queue processing: queue_mgt - appsess_refresh: appsession_refresh before (idle system): ==6079== LEAK SUMMARY: ==6079== definitely lost: 1,112 bytes in 75 blocks. ==6079== indirectly lost: 53,356 bytes in 2,090 blocks. ==6079== possibly lost: 52 bytes in 1 blocks. ==6079== still reachable: 150,996 bytes in 504 blocks. ==6079== suppressed: 0 bytes in 0 blocks. after (idle system): ==6945== LEAK SUMMARY: ==6945== definitely lost: 7,644 bytes in 137 blocks. ==6945== indirectly lost: 9,913 bytes in 587 blocks. ==6945== possibly lost: 0 bytes in 0 blocks. ==6945== still reachable: 0 bytes in 0 blocks. ==6945== suppressed: 0 bytes in 0 blocks. before (running system for ~2m): ==9343== LEAK SUMMARY: ==9343== definitely lost: 1,112 bytes in 75 blocks. ==9343== indirectly lost: 54,199 bytes in 2,122 blocks. ==9343== possibly lost: 52 bytes in 1 blocks. ==9343== still reachable: 151,128 bytes in 509 blocks. ==9343== suppressed: 0 bytes in 0 blocks. after (running system for ~2m): ==11616== LEAK SUMMARY: ==11616== definitely lost: 7,644 bytes in 137 blocks. ==11616== indirectly lost: 9,981 bytes in 591 blocks. ==11616== possibly lost: 0 bytes in 0 blocks. ==11616== still reachable: 4 bytes in 1 blocks. ==11616== suppressed: 0 bytes in 0 blocks. Still not perfect but significant improvement.
2008-05-29 21:53:44 +00:00
/*
* Deinitialize the pollers.
*/
void deinit_pollers();
[MAJOR] delay registering of listener sockets at startup Some pollers such as kqueue lose their FD across fork(), meaning that the registered file descriptors are lost too. Now when the proxies are started by start_proxies(), the file descriptors are not registered yet, leaving enough time for the fork() to take place and to get a new pollfd. It will be the first call to maintain_proxies that will register them.
2007-04-09 17:29:56 +00:00
/*
* Some pollers may lose their connection after a fork(). It may be necessary
* to create initialize part of them again. Returns 0 in case of failure,
* otherwise 1. The fork() function may be NULL if unused. In case of error,
* the the current poller is destroyed and the caller is responsible for trying
* another one by calling init_pollers() again.
*/
int fork_poller();
/*
* Lists the known pollers on <out>.
* Should be performed only before initialization.
*/
int list_pollers(FILE *out);
[MAJOR] modularize the polling mechanisms select, poll and epoll now have their dedicated functions and have been split into distinct files. Several FD manipulation primitives have been provided with each poller. The rest of the code needs to be cleaned to remove traces of StaticReadEvent/StaticWriteEvent. A trick involving a macro has temporarily been used right now. Some work needs to be done to factorize tests and sets everywhere.
2007-04-08 14:39:58 +00:00
/*
* Runs the polling loop
*/
void run_poller();
REORG: polling: rename "fd_process_spec_events()" to "fd_process_cached_events()" This is in order to be coherent with the rest.
2014-01-25 18:24:15 +00:00
/* Scan and process the cached events. This should be called right after
REORG: fd: centralize the processing of speculative events Speculative events are independant on the poller, so they can be centralized in fd.c.
2012-11-11 15:43:45 +00:00
* the poller.
*/
REORG: polling: rename "fd_process_spec_events()" to "fd_process_cached_events()" This is in order to be coherent with the rest.
2014-01-25 18:24:15 +00:00
void fd_process_cached_events();
REORG: fd: centralize the processing of speculative events Speculative events are independant on the poller, so they can be centralized in fd.c.
2012-11-11 15:43:45 +00:00
MINOR: fd: Make the lockless fd list work with multiple lists. Modify fd_add_to_fd_list() and fd_rm_from_fd_list() so that they take an offset in the fdtab to the list entry, instead of hardcoding the fd cache, so we can use them with other lists.
2018-04-25 13:10:30 +00:00
void fd_add_to_fd_list(volatile struct fdlist *list, int fd, int off);
void fd_rm_from_fd_list(volatile struct fdlist *list, int fd, int off);
MINOR: fd: move the fd_{add_to,rm_from}_fdlist functions to fd.c There's not point inlining these huge functions, better move them to real functions in fd.c.
2018-02-05 16:14:55 +00:00
MAJOR: polling: centralize calls to I/O callbacks In order for HTTP/2 not to eat too much memory, we'll have to support on-the-fly buffer allocation, since most streams will have an empty request buffer at some point. Supporting allocation on the fly means being able to sleep inside I/O callbacks if a buffer is not available. Till now, the I/O callbacks were called from two locations : - when processing the cached events - when processing the polled events from the poller This change cleans up the design a bit further than what was started in 1.5. It now ensures that we never call any iocb from the poller itself and that instead, events learned by the poller are put into the cache. The benefit is important in terms of stability : we don't have to care anymore about the risk that new events are added into the poller while processing its events, and we're certain that updates are processed at a single location. To achieve this, we now modify all the fd_* functions so that instead of creating updates, they add/remove the fd to/from the cache depending on its state, and only create an update when the polling status reaches a state where it will have to change. Since the pollers make use of these functions to notify readiness (using fd_may_recv/fd_may_send), the cache is always up to date with the poller. Creating updates only when the polling status needs to change saves a significant amount of work for the pollers : a benchmark showed that on a typical TCP proxy test, the amount of updates per connection dropped from 11 to 1 on average. This also means that the update list is smaller and has more chances of not thrashing too many CPU cache lines. The first observed benefit is a net 2% performance gain on the connection rate. A second benefit is that when a connection is accepted, it's only when we're processing the cache, and the recv event is automatically added into the cache *after* the current one, resulting in this event to be processed immediately during the same loop. Previously we used to have a second run over the updates to detect if new events were added to catch them before waking up tasks. The next gain will be offered by the next steps on this subject consisting in implementing an I/O queue containing all cached events ordered by priority just like the run queue, and to be able to leave some events pending there as long as needed. That will allow us *not* to perform some FD processing if it's not the proper time for this (typically keep waiting for a buffer to be allocated if none is available for an recv()). And by only processing a small bunch of them, we'll allow priorities to take place even at the I/O level. As a result of this change, functions fd_alloc_or_release_cache_entry() and fd_process_polled_events() have disappeared, and the code dedicated to checking for new fd events after the callback during the poll() loop was removed as well. Despite the patch looking large, it's mostly a change of what function is falled upon fd_*() and almost nothing was added.
2014-11-19 18:43:05 +00:00
/* Mark fd <fd> as updated for polling and allocate an entry in the update list
* for this if it was not already there. This can be done at any time.
MEDIUM: polling: centralize polled events processing Currently, each poll loop handles the polled events the same way, resulting in a lot of duplicated, complex code. Additionally, epoll was the only one to handle newly created FDs immediately. So instead, let's move that code to fd.c in a new function dedicated to this task : fd_process_polled_events(). All pollers now use this function.
2014-01-25 08:58:06 +00:00
*/
MAJOR: polling: centralize calls to I/O callbacks In order for HTTP/2 not to eat too much memory, we'll have to support on-the-fly buffer allocation, since most streams will have an empty request buffer at some point. Supporting allocation on the fly means being able to sleep inside I/O callbacks if a buffer is not available. Till now, the I/O callbacks were called from two locations : - when processing the cached events - when processing the polled events from the poller This change cleans up the design a bit further than what was started in 1.5. It now ensures that we never call any iocb from the poller itself and that instead, events learned by the poller are put into the cache. The benefit is important in terms of stability : we don't have to care anymore about the risk that new events are added into the poller while processing its events, and we're certain that updates are processed at a single location. To achieve this, we now modify all the fd_* functions so that instead of creating updates, they add/remove the fd to/from the cache depending on its state, and only create an update when the polling status reaches a state where it will have to change. Since the pollers make use of these functions to notify readiness (using fd_may_recv/fd_may_send), the cache is always up to date with the poller. Creating updates only when the polling status needs to change saves a significant amount of work for the pollers : a benchmark showed that on a typical TCP proxy test, the amount of updates per connection dropped from 11 to 1 on average. This also means that the update list is smaller and has more chances of not thrashing too many CPU cache lines. The first observed benefit is a net 2% performance gain on the connection rate. A second benefit is that when a connection is accepted, it's only when we're processing the cache, and the recv event is automatically added into the cache *after* the current one, resulting in this event to be processed immediately during the same loop. Previously we used to have a second run over the updates to detect if new events were added to catch them before waking up tasks. The next gain will be offered by the next steps on this subject consisting in implementing an I/O queue containing all cached events ordered by priority just like the run queue, and to be able to leave some events pending there as long as needed. That will allow us *not* to perform some FD processing if it's not the proper time for this (typically keep waiting for a buffer to be allocated if none is available for an recv()). And by only processing a small bunch of them, we'll allow priorities to take place even at the I/O level. As a result of this change, functions fd_alloc_or_release_cache_entry() and fd_process_polled_events() have disappeared, and the code dedicated to checking for new fd events after the callback during the poll() loop was removed as well. Despite the patch looking large, it's mostly a change of what function is falled upon fd_*() and almost nothing was added.
2014-11-19 18:43:05 +00:00
static inline void updt_fd_polling(const int fd)
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
{
MINOR: fd cache: And the thread_mask with all_threads_mask. When we choose to insert a fd in either the global or the local fd update list, and the thread_mask against all_threads_mask before checking if it's tid_bit, that way, if we run with nbthreads==1, we will always use the local list, which is cheaper than the global one.
2018-08-17 11:37:59 +00:00
if ((fdtab[fd].thread_mask & all_threads_mask) == tid_bit) {
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
unsigned int oldupdt;
/* note: we don't have a test-and-set yet in hathreads */
if (HA_ATOMIC_BTS(&fdtab[fd].update_mask, tid))
return;
oldupdt = HA_ATOMIC_ADD(&fd_nbupdt, 1) - 1;
fd_updt[oldupdt] = fd;
} else {
unsigned long update_mask = fdtab[fd].update_mask;
do {
if (update_mask == fdtab[fd].thread_mask)
return;
} while (!HA_ATOMIC_CAS(&fdtab[fd].update_mask, &update_mask,
fdtab[fd].thread_mask));
fd_add_to_fd_list(&update_list, fd, offsetof(struct fdtab, update));
}
MEDIUM: fd: make updt_fd_polling() use atomics It only needed a test-and-set and an atomic increment so we can take it out of the fd lock now.
2018-01-17 21:57:54 +00:00
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
}
MEDIUM: fd: make updt_fd_polling() use atomics It only needed a test-and-set and an atomic increment so we can take it out of the fd lock now.
2018-01-17 21:57:54 +00:00
BUG/MEDIUM: pollers: Use a global list for fd shared between threads. With the old model, any fd shared by multiple threads, such as listeners or dns sockets, would only be updated on one threads, so that could lead to missed event, or spurious wakeups. To avoid this, add a global list for fd that are shared, using the same implementation as the fd cache, and only remove entries from this list when every thread as updated its poller. [wt: this will need to be backported to 1.8 but differently so this patch must not be backported as-is]
2018-04-25 14:58:25 +00:00
/* Called from the poller to acknoledge we read an entry from the global
* update list, to remove our bit from the update_mask, and remove it from
* the list if we were the last one.
*/
static inline void done_update_polling(int fd)
{
unsigned long update_mask;
update_mask = HA_ATOMIC_AND(&fdtab[fd].update_mask, ~tid_bit);
while ((update_mask & all_threads_mask)== 0) {
/* If we were the last one that had to update that entry, remove it from the list */
fd_rm_from_fd_list(&update_list, fd, offsetof(struct fdtab, update));
if (update_list.first == fd)
abort();
update_mask = (volatile unsigned long)fdtab[fd].update_mask;
if ((update_mask & all_threads_mask) != 0) {
/* Maybe it's been re-updated in the meanwhile, and we
* wrongly removed it from the list, if so, re-add it
*/
fd_add_to_fd_list(&update_list, fd, offsetof(struct fdtab, update));
update_mask = (volatile unsigned long)(fdtab[fd].update_mask);
/* And then check again, just in case after all it
* should be removed, even if it's very unlikely, given
* the current thread wouldn't have been able to take
* care of it yet */
} else
break;
}
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
}
REORG: polling: rename the cache allocation functions - alloc_spec_entry() becomes fd_alloc_cache_entry() - release_spec_entry() becomes fd_release_cache_entry()
2014-01-25 18:20:35 +00:00
/* Allocates a cache entry for a file descriptor if it does not yet have one.
* This can be done at any time.
*/
static inline void fd_alloc_cache_entry(const int fd)
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
{
BUG/MEDIUM: fd/threads: ensure the fdcache_mask always reflects the cache contents Commit 4815c8c ("MAJOR: fd/threads: Make the fdcache mostly lockless.") made the fd cache lockless, but after a few iterations, a subtle part was lost, consisting in setting the bit on the fd_cache_mask immediately when adding an event. Now it was done only when the cache started to process events, but the problem it causes is that fd_cache_mask isn't reliable anymore as an indicator of presence of events to be processed with no delay outside of fd_process_cached_events(). This results in some spurious delays when processing inter-thread wakeups between tasks. Just restoring the flag when the event is added is enough to fix the problem. Kudos to Christopher for spotting this one! No backport is needed as this is only in the development version.
2018-03-20 18:06:52 +00:00
HA_ATOMIC_OR(&fd_cache_mask, fdtab[fd].thread_mask);
MAJOR: fd/threads: Make the fdcache mostly lockless. Create a local, per-thread, fdcache, for file descriptors that only belongs to one thread, and make the global fd cache mostly lockless, as we can get a lot of contention on the fd cache lock.
2018-01-24 17:17:56 +00:00
if (!(fdtab[fd].thread_mask & (fdtab[fd].thread_mask - 1)))
MINOR: fd: Make the lockless fd list work with multiple lists. Modify fd_add_to_fd_list() and fd_rm_from_fd_list() so that they take an offset in the fdtab to the list entry, instead of hardcoding the fd cache, so we can use them with other lists.
2018-04-25 13:10:30 +00:00
fd_add_to_fd_list(&fd_cache_local[my_ffsl(fdtab[fd].thread_mask) - 1], fd, offsetof(struct fdtab, cache));
MAJOR: fd/threads: Make the fdcache mostly lockless. Create a local, per-thread, fdcache, for file descriptors that only belongs to one thread, and make the global fd cache mostly lockless, as we can get a lot of contention on the fd cache lock.
2018-01-24 17:17:56 +00:00
else
MINOR: fd: Make the lockless fd list work with multiple lists. Modify fd_add_to_fd_list() and fd_rm_from_fd_list() so that they take an offset in the fdtab to the list entry, instead of hardcoding the fd cache, so we can use them with other lists.
2018-04-25 13:10:30 +00:00
fd_add_to_fd_list(&fd_cache, fd, offsetof(struct fdtab, cache));
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
}
REORG: polling: rename the cache allocation functions - alloc_spec_entry() becomes fd_alloc_cache_entry() - release_spec_entry() becomes fd_release_cache_entry()
2014-01-25 18:20:35 +00:00
/* Removes entry used by fd <fd> from the FD cache and replaces it with the
MAJOR: fd/threads: Make the fdcache mostly lockless. Create a local, per-thread, fdcache, for file descriptors that only belongs to one thread, and make the global fd cache mostly lockless, as we can get a lot of contention on the fd cache lock.
2018-01-24 17:17:56 +00:00
* last one.
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
* If the fd has no entry assigned, return immediately.
*/
MINOR: fd: move the fd_{add_to,rm_from}_fdlist functions to fd.c There's not point inlining these huge functions, better move them to real functions in fd.c.
2018-02-05 16:14:55 +00:00
static inline void fd_release_cache_entry(const int fd)
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
{
MAJOR: fd/threads: Make the fdcache mostly lockless. Create a local, per-thread, fdcache, for file descriptors that only belongs to one thread, and make the global fd cache mostly lockless, as we can get a lot of contention on the fd cache lock.
2018-01-24 17:17:56 +00:00
if (!(fdtab[fd].thread_mask & (fdtab[fd].thread_mask - 1)))
MINOR: fd: Make the lockless fd list work with multiple lists. Modify fd_add_to_fd_list() and fd_rm_from_fd_list() so that they take an offset in the fdtab to the list entry, instead of hardcoding the fd cache, so we can use them with other lists.
2018-04-25 13:10:30 +00:00
fd_rm_from_fd_list(&fd_cache_local[my_ffsl(fdtab[fd].thread_mask) - 1], fd, offsetof(struct fdtab, cache));
MAJOR: fd/threads: Make the fdcache mostly lockless. Create a local, per-thread, fdcache, for file descriptors that only belongs to one thread, and make the global fd cache mostly lockless, as we can get a lot of contention on the fd cache lock.
2018-01-24 17:17:56 +00:00
else
MINOR: fd: Make the lockless fd list work with multiple lists. Modify fd_add_to_fd_list() and fd_rm_from_fd_list() so that they take an offset in the fdtab to the list entry, instead of hardcoding the fd cache, so we can use them with other lists.
2018-04-25 13:10:30 +00:00
fd_rm_from_fd_list(&fd_cache, fd, offsetof(struct fdtab, cache));
REORG: fd: move the speculative I/O management from ev_sepoll The speculative I/O will need to be ported to all pollers, so move this to fd.c.
2012-11-11 14:02:54 +00:00
}
[MINOR] turn every FD_* into functions On recent CPUs, functions are about twice as fast as inline FD_*, so there is now a #define CONFIG_HAP_INLINE_FD_SET to choose between the two modes.
2006-10-15 12:52:29 +00:00
MAJOR: polling: centralize calls to I/O callbacks In order for HTTP/2 not to eat too much memory, we'll have to support on-the-fly buffer allocation, since most streams will have an empty request buffer at some point. Supporting allocation on the fly means being able to sleep inside I/O callbacks if a buffer is not available. Till now, the I/O callbacks were called from two locations : - when processing the cached events - when processing the polled events from the poller This change cleans up the design a bit further than what was started in 1.5. It now ensures that we never call any iocb from the poller itself and that instead, events learned by the poller are put into the cache. The benefit is important in terms of stability : we don't have to care anymore about the risk that new events are added into the poller while processing its events, and we're certain that updates are processed at a single location. To achieve this, we now modify all the fd_* functions so that instead of creating updates, they add/remove the fd to/from the cache depending on its state, and only create an update when the polling status reaches a state where it will have to change. Since the pollers make use of these functions to notify readiness (using fd_may_recv/fd_may_send), the cache is always up to date with the poller. Creating updates only when the polling status needs to change saves a significant amount of work for the pollers : a benchmark showed that on a typical TCP proxy test, the amount of updates per connection dropped from 11 to 1 on average. This also means that the update list is smaller and has more chances of not thrashing too many CPU cache lines. The first observed benefit is a net 2% performance gain on the connection rate. A second benefit is that when a connection is accepted, it's only when we're processing the cache, and the recv event is automatically added into the cache *after* the current one, resulting in this event to be processed immediately during the same loop. Previously we used to have a second run over the updates to detect if new events were added to catch them before waking up tasks. The next gain will be offered by the next steps on this subject consisting in implementing an I/O queue containing all cached events ordered by priority just like the run queue, and to be able to leave some events pending there as long as needed. That will allow us *not* to perform some FD processing if it's not the proper time for this (typically keep waiting for a buffer to be allocated if none is available for an recv()). And by only processing a small bunch of them, we'll allow priorities to take place even at the I/O level. As a result of this change, functions fd_alloc_or_release_cache_entry() and fd_process_polled_events() have disappeared, and the code dedicated to checking for new fd events after the callback during the poll() loop was removed as well. Despite the patch looking large, it's mostly a change of what function is falled upon fd_*() and almost nothing was added.
2014-11-19 18:43:05 +00:00
/* This function automatically enables/disables caching for an entry depending
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
* on its state. It is only called on state changes.
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
*/
MAJOR: polling: centralize calls to I/O callbacks In order for HTTP/2 not to eat too much memory, we'll have to support on-the-fly buffer allocation, since most streams will have an empty request buffer at some point. Supporting allocation on the fly means being able to sleep inside I/O callbacks if a buffer is not available. Till now, the I/O callbacks were called from two locations : - when processing the cached events - when processing the polled events from the poller This change cleans up the design a bit further than what was started in 1.5. It now ensures that we never call any iocb from the poller itself and that instead, events learned by the poller are put into the cache. The benefit is important in terms of stability : we don't have to care anymore about the risk that new events are added into the poller while processing its events, and we're certain that updates are processed at a single location. To achieve this, we now modify all the fd_* functions so that instead of creating updates, they add/remove the fd to/from the cache depending on its state, and only create an update when the polling status reaches a state where it will have to change. Since the pollers make use of these functions to notify readiness (using fd_may_recv/fd_may_send), the cache is always up to date with the poller. Creating updates only when the polling status needs to change saves a significant amount of work for the pollers : a benchmark showed that on a typical TCP proxy test, the amount of updates per connection dropped from 11 to 1 on average. This also means that the update list is smaller and has more chances of not thrashing too many CPU cache lines. The first observed benefit is a net 2% performance gain on the connection rate. A second benefit is that when a connection is accepted, it's only when we're processing the cache, and the recv event is automatically added into the cache *after* the current one, resulting in this event to be processed immediately during the same loop. Previously we used to have a second run over the updates to detect if new events were added to catch them before waking up tasks. The next gain will be offered by the next steps on this subject consisting in implementing an I/O queue containing all cached events ordered by priority just like the run queue, and to be able to leave some events pending there as long as needed. That will allow us *not* to perform some FD processing if it's not the proper time for this (typically keep waiting for a buffer to be allocated if none is available for an recv()). And by only processing a small bunch of them, we'll allow priorities to take place even at the I/O level. As a result of this change, functions fd_alloc_or_release_cache_entry() and fd_process_polled_events() have disappeared, and the code dedicated to checking for new fd events after the callback during the poll() loop was removed as well. Despite the patch looking large, it's mostly a change of what function is falled upon fd_*() and almost nothing was added.
2014-11-19 18:43:05 +00:00
static inline void fd_update_cache(int fd)
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
{
MAJOR: polling: centralize calls to I/O callbacks In order for HTTP/2 not to eat too much memory, we'll have to support on-the-fly buffer allocation, since most streams will have an empty request buffer at some point. Supporting allocation on the fly means being able to sleep inside I/O callbacks if a buffer is not available. Till now, the I/O callbacks were called from two locations : - when processing the cached events - when processing the polled events from the poller This change cleans up the design a bit further than what was started in 1.5. It now ensures that we never call any iocb from the poller itself and that instead, events learned by the poller are put into the cache. The benefit is important in terms of stability : we don't have to care anymore about the risk that new events are added into the poller while processing its events, and we're certain that updates are processed at a single location. To achieve this, we now modify all the fd_* functions so that instead of creating updates, they add/remove the fd to/from the cache depending on its state, and only create an update when the polling status reaches a state where it will have to change. Since the pollers make use of these functions to notify readiness (using fd_may_recv/fd_may_send), the cache is always up to date with the poller. Creating updates only when the polling status needs to change saves a significant amount of work for the pollers : a benchmark showed that on a typical TCP proxy test, the amount of updates per connection dropped from 11 to 1 on average. This also means that the update list is smaller and has more chances of not thrashing too many CPU cache lines. The first observed benefit is a net 2% performance gain on the connection rate. A second benefit is that when a connection is accepted, it's only when we're processing the cache, and the recv event is automatically added into the cache *after* the current one, resulting in this event to be processed immediately during the same loop. Previously we used to have a second run over the updates to detect if new events were added to catch them before waking up tasks. The next gain will be offered by the next steps on this subject consisting in implementing an I/O queue containing all cached events ordered by priority just like the run queue, and to be able to leave some events pending there as long as needed. That will allow us *not* to perform some FD processing if it's not the proper time for this (typically keep waiting for a buffer to be allocated if none is available for an recv()). And by only processing a small bunch of them, we'll allow priorities to take place even at the I/O level. As a result of this change, functions fd_alloc_or_release_cache_entry() and fd_process_polled_events() have disappeared, and the code dedicated to checking for new fd events after the callback during the poll() loop was removed as well. Despite the patch looking large, it's mostly a change of what function is falled upon fd_*() and almost nothing was added.
2014-11-19 18:43:05 +00:00
/* only READY and ACTIVE states (the two with both flags set) require a cache entry */
if (((fdtab[fd].state & (FD_EV_READY_R | FD_EV_ACTIVE_R)) == (FD_EV_READY_R | FD_EV_ACTIVE_R)) ||
((fdtab[fd].state & (FD_EV_READY_W | FD_EV_ACTIVE_W)) == (FD_EV_READY_W | FD_EV_ACTIVE_W))) {
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
fd_alloc_cache_entry(fd);
}
else {
fd_release_cache_entry(fd);
}
}
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
/*
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
* returns the FD's recv state (FD_EV_*)
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
*/
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
static inline int fd_recv_state(const int fd)
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
{
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
return ((unsigned)fdtab[fd].state >> (4 * DIR_RD)) & FD_EV_STATUS;
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/*
* returns true if the FD is active for recv
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
*/
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
static inline int fd_recv_active(const int fd)
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
{
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
return (unsigned)fdtab[fd].state & FD_EV_ACTIVE_R;
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/*
* returns true if the FD is ready for recv
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
*/
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
static inline int fd_recv_ready(const int fd)
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
{
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
return (unsigned)fdtab[fd].state & FD_EV_READY_R;
}
/*
* returns true if the FD is polled for recv
*/
static inline int fd_recv_polled(const int fd)
{
return (unsigned)fdtab[fd].state & FD_EV_POLLED_R;
}
/*
* returns the FD's send state (FD_EV_*)
*/
static inline int fd_send_state(const int fd)
{
return ((unsigned)fdtab[fd].state >> (4 * DIR_WR)) & FD_EV_STATUS;
}
/*
* returns true if the FD is active for send
*/
static inline int fd_send_active(const int fd)
{
return (unsigned)fdtab[fd].state & FD_EV_ACTIVE_W;
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/*
* returns true if the FD is ready for send
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
*/
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
static inline int fd_send_ready(const int fd)
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
{
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
return (unsigned)fdtab[fd].state & FD_EV_READY_W;
}
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/*
* returns true if the FD is polled for send
*/
static inline int fd_send_polled(const int fd)
{
return (unsigned)fdtab[fd].state & FD_EV_POLLED_W;
}
MINOR: fd: Add fd_active function This inlined function is used to check if a fd is active for receive or send. It will ease threads support integration.
2017-08-30 07:59:38 +00:00
/*
* returns true if the FD is active for recv or send
*/
static inline int fd_active(const int fd)
{
return (unsigned)fdtab[fd].state & FD_EV_ACTIVE_RW;
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Disable processing recv events on fd <fd> */
static inline void fd_stop_recv(int fd)
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (!(old & FD_EV_ACTIVE_R))
return;
new = old & ~FD_EV_ACTIVE_R;
new &= ~FD_EV_POLLED_R;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_R)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Disable processing send events on fd <fd> */
static inline void fd_stop_send(int fd)
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (!(old & FD_EV_ACTIVE_W))
return;
new = old & ~FD_EV_ACTIVE_W;
new &= ~FD_EV_POLLED_W;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_W)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
REORG: fd: move the fd state management from ev_sepoll ev_sepoll already provides everything needed to manage FD events by only manipulating the speculative I/O list. Nothing there is sepoll-specific so move all this to fd.
2012-11-11 15:05:19 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Disable processing of events on fd <fd> for both directions. */
static inline void fd_stop_both(int fd)
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (!(old & FD_EV_ACTIVE_RW))
return;
new = old & ~FD_EV_ACTIVE_RW;
new &= ~FD_EV_POLLED_RW;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_RW)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Report that FD <fd> cannot receive anymore without polling (EAGAIN detected). */
static inline void fd_cant_recv(const int fd)
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (!(old & FD_EV_READY_R))
return;
new = old & ~FD_EV_READY_R;
if (new & FD_EV_ACTIVE_R)
new |= FD_EV_POLLED_R;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_R)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Report that FD <fd> can receive anymore without polling. */
static inline void fd_may_recv(const int fd)
MEDIUM: fd: add fd_poll_{recv,send} for use when explicit polling is required The old EV_FD_SET() macro was confusing, as it would enable receipt but there was no way to indicate that EAGAIN was received, hence the recently added FD_WAIT_* flags. They're not enough as we're still facing a conflict between EV_FD_* and FD_WAIT_*. So let's offer I/O functions what they need to explicitly request polling.
2012-08-09 10:14:03 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
/* marking ready never changes polled status */
HA_ATOMIC_OR(&fdtab[fd].state, FD_EV_READY_R);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MEDIUM: fd: add fd_poll_{recv,send} for use when explicit polling is required The old EV_FD_SET() macro was confusing, as it would enable receipt but there was no way to indicate that EAGAIN was received, hence the recently added FD_WAIT_* flags. They're not enough as we're still facing a conflict between EV_FD_* and FD_WAIT_*. So let's offer I/O functions what they need to explicitly request polling.
2012-08-09 10:14:03 +00:00
}
OPTIM: raw-sock: don't speculate after a short read if polling is enabled This is the reimplementation of the "done" action : when we experience a short read, we're almost certain that we've exhausted the system's buffers and that we'll meet an EAGAIN if we attempt to read again. If the FD is not yet polled, the stream interface already takes care of stopping the speculative read. When the FD is already being polled, we have two options : - either we're running from a level-triggered poller, in which case we'd rather report that we've reached the end so that we don't speculate over the poller and let it report next time data are available ; - or we're running from an edge-triggered poller in which case we have no choice and have to see the EAGAIN to re-enable events. At the moment we don't have any edge-triggered poller, so it's desirable to avoid speculative I/O that we know will fail. Note that this must not be ported to SSL since SSL hides the real readiness of the file descriptor. Thanks to this change, we observe no EAGAIN anymore during keep-alive transfers, and failed recvfrom() are reduced by half in http-server-close mode (the client-facing side is always being polled and the second recv can be avoided). Doing so results in about 5% performance increase in keep-alive mode. Similarly, we used to have up to about 1.6% of EAGAIN on accept() (1/maxaccept), and these have completely disappeared under high loads.
2014-01-23 23:54:27 +00:00
/* Disable readiness when polled. This is useful to interrupt reading when it
* is suspected that the end of data might have been reached (eg: short read).
* This can only be done using level-triggered pollers, so if any edge-triggered
* is ever implemented, a test will have to be added here.
*/
static inline void fd_done_recv(const int fd)
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if ((old & (FD_EV_POLLED_R|FD_EV_READY_R)) != (FD_EV_POLLED_R|FD_EV_READY_R))
return;
new = old & ~FD_EV_READY_R;
if (new & FD_EV_ACTIVE_R)
new |= FD_EV_POLLED_R;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_R)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
OPTIM: raw-sock: don't speculate after a short read if polling is enabled This is the reimplementation of the "done" action : when we experience a short read, we're almost certain that we've exhausted the system's buffers and that we'll meet an EAGAIN if we attempt to read again. If the FD is not yet polled, the stream interface already takes care of stopping the speculative read. When the FD is already being polled, we have two options : - either we're running from a level-triggered poller, in which case we'd rather report that we've reached the end so that we don't speculate over the poller and let it report next time data are available ; - or we're running from an edge-triggered poller in which case we have no choice and have to see the EAGAIN to re-enable events. At the moment we don't have any edge-triggered poller, so it's desirable to avoid speculative I/O that we know will fail. Note that this must not be ported to SSL since SSL hides the real readiness of the file descriptor. Thanks to this change, we observe no EAGAIN anymore during keep-alive transfers, and failed recvfrom() are reduced by half in http-server-close mode (the client-facing side is always being polled and the second recv can be avoided). Doing so results in about 5% performance increase in keep-alive mode. Similarly, we used to have up to about 1.6% of EAGAIN on accept() (1/maxaccept), and these have completely disappeared under high loads.
2014-01-23 23:54:27 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Report that FD <fd> cannot send anymore without polling (EAGAIN detected). */
static inline void fd_cant_send(const int fd)
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (!(old & FD_EV_READY_W))
return;
new = old & ~FD_EV_READY_W;
if (new & FD_EV_ACTIVE_W)
new |= FD_EV_POLLED_W;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_W)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Report that FD <fd> can send anymore without polling (EAGAIN detected). */
static inline void fd_may_send(const int fd)
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
/* marking ready never changes polled status */
HA_ATOMIC_OR(&fdtab[fd].state, FD_EV_READY_W);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Prepare FD <fd> to try to receive */
static inline void fd_want_recv(int fd)
MEDIUM: fd: add fd_poll_{recv,send} for use when explicit polling is required The old EV_FD_SET() macro was confusing, as it would enable receipt but there was no way to indicate that EAGAIN was received, hence the recently added FD_WAIT_* flags. They're not enough as we're still facing a conflict between EV_FD_* and FD_WAIT_*. So let's offer I/O functions what they need to explicitly request polling.
2012-08-09 10:14:03 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (old & FD_EV_ACTIVE_R)
return;
new = old | FD_EV_ACTIVE_R;
if (!(new & FD_EV_READY_R))
new |= FD_EV_POLLED_R;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_R)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MEDIUM: fd: add fd_poll_{recv,send} for use when explicit polling is required The old EV_FD_SET() macro was confusing, as it would enable receipt but there was no way to indicate that EAGAIN was received, hence the recently added FD_WAIT_* flags. They're not enough as we're still facing a conflict between EV_FD_* and FD_WAIT_*. So let's offer I/O functions what they need to explicitly request polling.
2012-08-09 10:14:03 +00:00
}
MAJOR: polling: rework the whole polling system This commit heavily changes the polling system in order to definitely fix the frequent breakage of SSL which needs to remember the last EAGAIN before deciding whether to poll or not. Now we have a state per direction for each FD, as opposed to a previous and current state previously. An FD can have up to 8 different states for each direction, each of which being the result of a 3-bit combination. These 3 bits indicate a wish to access the FD, the readiness of the FD and the subscription of the FD to the polling system. This means that it will now be possible to remember the state of a file descriptor across disable/enable sequences that generally happen during forwarding, where enabling reading on a previously disabled FD would result in forgetting the EAGAIN flag it met last time. Several new state manipulation functions have been introduced or adapted : - fd_want_{recv,send} : enable receiving/sending on the FD regardless of its state (sets the ACTIVE flag) ; - fd_stop_{recv,send} : stop receiving/sending on the FD regardless of its state (clears the ACTIVE flag) ; - fd_cant_{recv,send} : report a failure to receive/send on the FD corresponding to EAGAIN (clears the READY flag) ; - fd_may_{recv,send} : report the ability to receive/send on the FD as reported by poll() (sets the READY flag) ; Some functions are used to report the current FD status : - fd_{recv,send}_active - fd_{recv,send}_ready - fd_{recv,send}_polled Some functions were removed : - fd_ev_clr(), fd_ev_set(), fd_ev_rem(), fd_ev_wai() The POLLHUP/POLLERR flags are now reported as ready so that the I/O layers knows it can try to access the file descriptor to get this information. In order to simplify the conditions to add/remove cache entries, a new function fd_alloc_or_release_cache_entry() was created to be used from pollers while scanning for updates. The following pollers have been updated : ev_select() : done, built, tested on Linux 3.10 ev_poll() : done, built, tested on Linux 3.10 ev_epoll() : done, built, tested on Linux 3.10 & 3.13 ev_kqueue() : done, built, tested on OpenBSD 5.2
2014-01-10 15:58:45 +00:00
/* Prepare FD <fd> to try to send */
static inline void fd_want_send(int fd)
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
{
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
unsigned char old, new;
old = fdtab[fd].state;
do {
if (old & FD_EV_ACTIVE_W)
return;
new = old | FD_EV_ACTIVE_W;
if (!(new & FD_EV_READY_W))
new |= FD_EV_POLLED_W;
} while (unlikely(!HA_ATOMIC_CAS(&fdtab[fd].state, &old, new)));
if ((old ^ new) & FD_EV_POLLED_W)
updt_fd_polling(fd);
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: compute the new fd polling state out of the fd lock Each fd_{may|cant|stop|want}_{recv|send} function sets or resets a single bit at once, then recomputes the need for updates, and then the new cache state. Later, pollers will compute the new polling state based on the resulting operations here. In fact the conditions are so simple that they can be performed by a single "if", or sometimes even optimized away. This means that in practice a simple compare-and-swap operation if often enough to set the new value inluding the new polling state, and that only the cache and fdupdt have to be performed under the lock. Better, for the most common operations (fd_may_{recv,send}, used by the pollers), a simple atomic OR is needed. This patch does this for the fd_* functions above and it doesn't yet remove the now useless fd_compute_new_polling_status() because it's still used by other pollers. A pure connection rate test shows a 1% performance increase.
2018-01-17 20:25:57 +00:00
fd_update_cache(fd); /* need an update entry to change the state */
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MAJOR: fd: replace all EV_FD_* macros with new fd_*_* inline calls These functions have a more explicity meaning and will offer provisions for explicit polling. EV_FD_ISSET() has been left for now as it is still in use in checks.
2012-08-09 10:11:58 +00:00
}
[MINOR] turn every FD_* into functions On recent CPUs, functions are about twice as fast as inline FD_*, so there is now a #define CONFIG_HAP_INLINE_FD_SET to choose between the two modes.
2006-10-15 12:52:29 +00:00
MINOR: fd: Add fd_update_events function This function should be called by the poller to set FD_POLL_* flags on an FD and update its state if needed. This function has been added to ease threads support integration.
2017-08-30 08:30:04 +00:00
/* Update events seen for FD <fd> and its state if needed. This should be called
* by the poller to set FD_POLL_* flags. */
static inline void fd_update_events(int fd, int evts)
{
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MINOR: fd: Add fd_update_events function This function should be called by the poller to set FD_POLL_* flags on an FD and update its state if needed. This function has been added to ease threads support integration.
2017-08-30 08:30:04 +00:00
fdtab[fd].ev &= FD_POLL_STICKY;
fdtab[fd].ev |= evts;
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(fdtab[fd].thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
MINOR: fd: Add fd_update_events function This function should be called by the poller to set FD_POLL_* flags on an FD and update its state if needed. This function has been added to ease threads support integration.
2017-08-30 08:30:04 +00:00
if (fdtab[fd].ev & (FD_POLL_IN | FD_POLL_HUP | FD_POLL_ERR))
fd_may_recv(fd);
if (fdtab[fd].ev & (FD_POLL_OUT | FD_POLL_ERR))
fd_may_send(fd);
}
[BUG] fix truncated responses with sepoll Due to the way Linux delivers EPOLLIN and EPOLLHUP, a closed connection received after some server data sometimes results in truncated responses if the client disconnects before server starts to respond. The reason is that the EPOLLHUP flag is processed as an indication of end of transfer while some data may remain in the system's socket buffers. This problem could only be triggered with sepoll, although nothing should prevent it from happening with normal epoll. In fact, the work factoring performed by sepoll increases the risk that this bug appears. The fix consists in making FD_POLL_HUP and FD_POLL_ERR sticky and that they are only checked if FD_POLL_IN is not set, meaning that we have read all pending data. That way, the problem is definitely fixed and sepoll still remains about 17% faster than epoll since it can take into account all information returned by the kernel.
2008-01-18 16:20:13 +00:00
/* Prepares <fd> for being polled */
MINOR: fd: pass the iocb and owner to fd_insert() fd_insert() is currently called just after setting the owner and iocb, but proceeding like this prevents the operation from being atomic and requires a lock to protect the maxfd computation in another thread from meeting an incompletely initialized FD and computing a wrong maxfd. Fortunately for now all fdtab[].owner are set before calling fd_insert(), and the first lock in fd_insert() enforces a memory barrier so the code is safe. This patch moves the initialization of the owner and iocb to fd_insert() so that the function will be able to properly arrange its operations and remain safe even when modified to become lockless. There's no other change beyond the internal API.
2018-01-25 06:22:13 +00:00
static inline void fd_insert(int fd, void *owner, void (*iocb)(int fd), unsigned long thread_mask)
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
{
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(thread_mask))
HA_SPIN_LOCK(FD_LOCK, &fdtab[fd].lock);
MINOR: fd: pass the iocb and owner to fd_insert() fd_insert() is currently called just after setting the owner and iocb, but proceeding like this prevents the operation from being atomic and requires a lock to protect the maxfd computation in another thread from meeting an incompletely initialized FD and computing a wrong maxfd. Fortunately for now all fdtab[].owner are set before calling fd_insert(), and the first lock in fd_insert() enforces a memory barrier so the code is safe. This patch moves the initialization of the owner and iocb to fd_insert() so that the function will be able to properly arrange its operations and remain safe even when modified to become lockless. There's no other change beyond the internal API.
2018-01-25 06:22:13 +00:00
fdtab[fd].owner = owner;
fdtab[fd].iocb = iocb;
[BUG] fix truncated responses with sepoll Due to the way Linux delivers EPOLLIN and EPOLLHUP, a closed connection received after some server data sometimes results in truncated responses if the client disconnects before server starts to respond. The reason is that the EPOLLHUP flag is processed as an indication of end of transfer while some data may remain in the system's socket buffers. This problem could only be triggered with sepoll, although nothing should prevent it from happening with normal epoll. In fact, the work factoring performed by sepoll increases the risk that this bug appears. The fix consists in making FD_POLL_HUP and FD_POLL_ERR sticky and that they are only checked if FD_POLL_IN is not set, meaning that we have read all pending data. That way, the problem is definitely fixed and sepoll still remains about 17% faster than epoll since it can take into account all information returned by the kernel.
2008-01-18 16:20:13 +00:00
fdtab[fd].ev = 0;
MEDIUM: connection: centralize handling of nolinger in fd management Right now we see many places doing their own setsockopt(SO_LINGER). Better only do it just before the close() in fd_delete(). For this we add a new flag on the file descriptor, indicating if it's safe or not to linger. If not (eg: after a connect()), then the setsockopt() call is automatically performed before a close(). The flag automatically turns to safe when receiving a read0.
2013-12-15 13:19:38 +00:00
fdtab[fd].linger_risk = 0;
BUG/MEDIUM: polling: fix possible CPU hogging of worker processes after receiving SIGUSR1. When run in daemon mode (i.e. with at least one forked process) and using the epoll poller, sending USR1 (graceful shutdown) to the worker processes can cause some workers to start running at 100% CPU. Precondition is having an established HTTP keep-alive connection when the signal is received. The cloned (during fork) listening sockets do not get closed in the parent process, thus they do not get removed from the epoll set automatically (see man 7 epoll). This can lead to the process receiving epoll events that it doesn't feel responsible for, resulting in an endless loop around epoll_wait() delivering these events. The solution is to explicitly remove these file descriptors from the epoll set. To not degrade performance, care was taken to only do this when neccessary, i.e. when the file descriptor was cloned during fork. Signed-off-by: Conrad Hoffmann <conrad@soundcloud.com> [wt: a backport to 1.4 could be studied though chances to catch the bug are low]
2014-05-20 12:28:24 +00:00
fdtab[fd].cloned = 0;
CLEANUP: threads: rename process_mask to thread_mask It was a leftover from the last cleaning session; this mask applies to threads and calling it process_mask is a bit confusing. It's the same in fd, task and applets.
2017-10-31 15:06:06 +00:00
fdtab[fd].thread_mask = thread_mask;
MINOR: fd: add a bitmask to indicate that an FD is known by the poller Some pollers like epoll() need to know if the fd is already known or not in order to compute the operation to perform (add, mod, del). For now this is performed based on the difference between the previous FD state and the new state but this will not be usable anymore once threads become responsible for their own polling. Here we come with a different approach : a bitmask is stored with the fd to indicate which pollers already know it, and the pollers will be able to simply perform the add/mod/del operations based on this bit combined with the new state. This patch only adds the bitmask declaration and initialization, it is it not yet used. It will be needed by the next two fixes and will need to be backported to 1.8.
2018-01-17 17:44:46 +00:00
/* note: do not reset polled_mask here as it indicates which poller
* still knows this FD from a possible previous round.
*/
MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread The vast majority of FDs are only seen by one thread. Currently the lock on FDs costs a lot because it's touched often, though there should be very little contention. This patch ensures that the lock is only grabbed if the FD is shared by more than one thread, since otherwise the situation is safe. Doing so resulted in a 15% performance boost on a 12-threads test.
2018-10-15 07:44:46 +00:00
if (atleast2(thread_mask))
HA_SPIN_UNLOCK(FD_LOCK, &fdtab[fd].lock);
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
}
MINOR: fd: move the hap_fd_{clr,set,isset} functions to fd.h These functions were created for poll() in 1.5-dev18 (commit 80da05a4) to replace the previous FD_{CLR,SET,ISSET} that were shared with select() because some libcs enforce a limit on FD_SET. But FD_SET doesn't seem to be universally MT-safe, requiring locks in the select() code that are not needed in the poll code. So let's move back to the initial situation where we used to only use bit fields, since that has been in use since day one without a problem, and let's use these hap_fd_* functions instead of FD_*. This patch only moves the functions to fd.h and revives hap_fd_isset() that was recently removed to kill an "unused" warning.
2018-01-25 15:37:04 +00:00
/* These are replacements for FD_SET, FD_CLR, FD_ISSET, working on uints */
static inline void hap_fd_set(int fd, unsigned int *evts)
{
MEDIUM: fd: use atomic ops for hap_fd_{clr,set} and remove poll_lock Now that we can use atomic ops to set/clear an fd occurrence in an fd_set, we don't need the poll_lock anymore. Let's remove it.
2018-01-25 15:59:09 +00:00
HA_ATOMIC_OR(&evts[fd / (8*sizeof(*evts))], 1U << (fd & (8*sizeof(*evts) - 1)));
MINOR: fd: move the hap_fd_{clr,set,isset} functions to fd.h These functions were created for poll() in 1.5-dev18 (commit 80da05a4) to replace the previous FD_{CLR,SET,ISSET} that were shared with select() because some libcs enforce a limit on FD_SET. But FD_SET doesn't seem to be universally MT-safe, requiring locks in the select() code that are not needed in the poll code. So let's move back to the initial situation where we used to only use bit fields, since that has been in use since day one without a problem, and let's use these hap_fd_* functions instead of FD_*. This patch only moves the functions to fd.h and revives hap_fd_isset() that was recently removed to kill an "unused" warning.
2018-01-25 15:37:04 +00:00
}
static inline void hap_fd_clr(int fd, unsigned int *evts)
{
MEDIUM: fd: use atomic ops for hap_fd_{clr,set} and remove poll_lock Now that we can use atomic ops to set/clear an fd occurrence in an fd_set, we don't need the poll_lock anymore. Let's remove it.
2018-01-25 15:59:09 +00:00
HA_ATOMIC_AND(&evts[fd / (8*sizeof(*evts))], ~(1U << (fd & (8*sizeof(*evts) - 1))));
MINOR: fd: move the hap_fd_{clr,set,isset} functions to fd.h These functions were created for poll() in 1.5-dev18 (commit 80da05a4) to replace the previous FD_{CLR,SET,ISSET} that were shared with select() because some libcs enforce a limit on FD_SET. But FD_SET doesn't seem to be universally MT-safe, requiring locks in the select() code that are not needed in the poll code. So let's move back to the initial situation where we used to only use bit fields, since that has been in use since day one without a problem, and let's use these hap_fd_* functions instead of FD_*. This patch only moves the functions to fd.h and revives hap_fd_isset() that was recently removed to kill an "unused" warning.
2018-01-25 15:37:04 +00:00
}
static inline unsigned int hap_fd_isset(int fd, unsigned int *evts)
{
return evts[fd / (8*sizeof(*evts))] & (1U << (fd & (8*sizeof(*evts) - 1)));
}
MINOR: pollers: Add a way to wake a thread sleeping in the poller. Add a new pipe, one per thread, so that we can write on it to wake a thread sleeping in a poller, and use it to wake threads supposed to take care of a task, if they are all sleeping.
2018-07-26 15:55:11 +00:00
static inline void wake_thread(int tid)
{
char c = 'c';
shut_your_big_mouth_gcc(write(poller_wr_pipe[tid], &c, 1));
}
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
#endif /* _PROTO_FD_H */
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/
Reference in New Issue Copy Permalink