RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-15 07:54:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
c5bd311b2a
haproxy/include/proto/task.h

577 lines
18 KiB
C
Raw Normal View History

[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
/*
[MEDIUM] signals: add support for registering functions and tasks The two new functions below make it possible to register any number of functions or tasks to a system signal. They will be called in the registration order when the signal is received. struct sig_handler *signal_register_fct(int sig, void (*fct)(struct sig_handler *), int arg); struct sig_handler *signal_register_task(int sig, struct task *task, int reason);
2010-08-27 15:56:48 +00:00
* include/proto/task.h
* Functions for task management.
*
* Copyright (C) 2000-2010 Willy Tarreau - w@1wt.eu
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
#ifndef _PROTO_TASK_H
#define _PROTO_TASK_H
#include <sys/time.h>
[CLEANUP] included common/version.h everywhere
2006-06-29 16:54:54 +00:00
#include <common/config.h>
[CLEANUP] renamed include/haproxy to include/common
2006-06-29 15:53:05 +00:00
#include <common/memory.h>
[MAJOR] replaced rbtree with ul2tree. The rbtree-based wait queue consumes a lot of CPU. Use the ul2tree instead. Lots of cleanups and code reorganizations made it possible to reduce the task struct and simplify the code a bit.
2007-04-29 08:41:56 +00:00
#include <common/mini-clist.h>
#include <common/standard.h>
[CLEANUP] task: distinguish between clock ticks and timers Timers are unsigned and used as tree positions. Ticks are signed and used as absolute date within current time frame. While the two are normally equal (except zero), it's important not to confuse them in the code as they are not interchangeable. We add two inline functions to turn each one into the other. The comments have also been moved to the proper location, as it was not easy to understand what was a tick and what was a timer unit.
2009-03-08 14:53:06 +00:00
#include <common/ticks.h>
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
#include <common/hathreads.h>
MAJOR: task: make use of the scope-aware ebtree functions Currently the task scheduler suffers from an O(n) lookup when skipping tasks that are not for the current thread. The reason is that eb32_lookup_ge() has no information about the current thread so it always revisits many tasks for other threads before finding its own tasks. This is particularly visible with HTTP/2 since the number of concurrent streams created at once causes long series of tasks for the same stream in the scheduler. With only 10 connections and 100 streams each, by running on two threads, the performance drops from 640kreq/s to 11.2kreq/s! Lookup metrics show that for only 200000 task lookups, 430 million skips had to be performed, which means that on average, each lookup leads to 2150 nodes to be visited. This commit backports the principle of scope lookups for ebtrees from the ebtree_v7 development tree. The idea is that each node contains a mask indicating the union of the scopes for the nodes below it, which is fed during insertion, and used during lookups. Then during lookups, branches that do not contain any leaf matching the requested scope are simply ignored. This perfectly matches a thread mask, allowing a thread to only extract the tasks it cares about from the run queue, and to always find them in O(log(n)) instead of O(n). Thus the scheduler uses tid_bit and task->thread_mask as the ebtree scope here. Doing this has recovered most of the performance, as can be seen on the test below with two threads, 10 connections, 100 streams each, and 1 million requests total : Before After Gain test duration : 89.6s 4.73s x19 HTTP requests/s (DEBUG) : 11200 211300 x19 HTTP requests/s (PROD) : 15900 447000 x28 spin_lock time : 85.2s 0.46s /185 time per lookup : 13us 40ns /325 Even when going to 6 threads (on 3 hyperthreaded CPU cores), the performance stays around 284000 req/s, showing that the contention is much lower. A test showed that there's no benefit in using this for the wait queue though.
2017-11-05 12:34:20 +00:00
#include <eb32sctree.h>
[MEDIUM] build: switch ebtree users to use new ebtree version All files referencing the previous ebtree code were changed to point to the new one in the ebtree directory. A makefile variable (EBTREE_DIR) is also available to use files from another directory. The ability to build the libebtree library temporarily remains disabled because it can have an impact on some existing toolchains and does not appear worth it in the medium term if we add support for multi-criteria stickiness for instance.
2009-10-26 20:10:04 +00:00
#include <eb32tree.h>
[MAJOR] replaced rbtree with ul2tree. The rbtree-based wait queue consumes a lot of CPU. Use the ul2tree instead. Lots of cleanups and code reorganizations made it possible to reduce the task struct and simplify the code a bit.
2007-04-29 08:41:56 +00:00
MINOR: task: release the task pool when stopping When we're stopping, we're not going to create new tasks anymore, so let's release the task pool upon each task_free() in order to reduce memory fragmentation.
2014-11-13 15:57:19 +00:00
#include <types/global.h>
[CLEANUP] included common/version.h everywhere
2006-06-29 16:54:54 +00:00
#include <types/task.h>
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
[CLEANUP] task: distinguish between clock ticks and timers Timers are unsigned and used as tree positions. Ticks are signed and used as absolute date within current time frame. While the two are normally equal (except zero), it's important not to confuse them in the code as they are not interchangeable. We add two inline functions to turn each one into the other. The comments have also been moved to the proper location, as it was not easy to understand what was a tick and what was a timer unit.
2009-03-08 14:53:06 +00:00
/* Principle of the wait queue.
*
* We want to be able to tell whether an expiration date is before of after the
* current time <now>. We KNOW that expiration dates are never too far apart,
* because they are measured in ticks (milliseconds). We also know that almost
* all dates will be in the future, and that a very small part of them will be
* in the past, they are the ones which have expired since last time we checked
* them. Using ticks, we know if a date is in the future or in the past, but we
* cannot use that to store sorted information because that reference changes
* all the time.
*
[MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1 Since we're now able to search from a precise expiration date in the timer tree using ebtree 4.1, we don't need to maintain 4 trees anymore. Not only does this simplify the code a lot, but it also ensures that we can always look 24 days back and ahead, which doubles the ability of the previous scheduler. Indeed, while based on absolute values, the timer tree is now relative to <now> as we can always search from <now>-31 bits. The run queue uses the exact same principle now, and is now simpler and a bit faster to process. With these changes alone, an overall 0.5% performance gain was observed. Tests were performed on the few wrapping cases and everything works as expected.
2009-03-21 09:01:42 +00:00
* We'll use the fact that the time wraps to sort timers. Timers above <now>
* are in the future, timers below <now> are in the past. Here, "above" and
* "below" are to be considered modulo 2^31.
[CLEANUP] task: distinguish between clock ticks and timers Timers are unsigned and used as tree positions. Ticks are signed and used as absolute date within current time frame. While the two are normally equal (except zero), it's important not to confuse them in the code as they are not interchangeable. We add two inline functions to turn each one into the other. The comments have also been moved to the proper location, as it was not easy to understand what was a tick and what was a timer unit.
2009-03-08 14:53:06 +00:00
*
[MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1 Since we're now able to search from a precise expiration date in the timer tree using ebtree 4.1, we don't need to maintain 4 trees anymore. Not only does this simplify the code a lot, but it also ensures that we can always look 24 days back and ahead, which doubles the ability of the previous scheduler. Indeed, while based on absolute values, the timer tree is now relative to <now> as we can always search from <now>-31 bits. The run queue uses the exact same principle now, and is now simpler and a bit faster to process. With these changes alone, an overall 0.5% performance gain was observed. Tests were performed on the few wrapping cases and everything works as expected.
2009-03-21 09:01:42 +00:00
* Timers are stored sorted in an ebtree. We use the new ability for ebtrees to
* lookup values starting from X to only expire tasks between <now> - 2^31 and
* <now>. If the end of the tree is reached while walking over it, we simply
* loop back to the beginning. That way, we have no problem keeping sorted
* wrapping timers in a tree, between (now - 24 days) and (now + 24 days). The
* keys in the tree always reflect their real position, none can be infinite.
* This reduces the number of checks to be performed.
[CLEANUP] task: distinguish between clock ticks and timers Timers are unsigned and used as tree positions. Ticks are signed and used as absolute date within current time frame. While the two are normally equal (except zero), it's important not to confuse them in the code as they are not interchangeable. We add two inline functions to turn each one into the other. The comments have also been moved to the proper location, as it was not easy to understand what was a tick and what was a timer unit.
2009-03-08 14:53:06 +00:00
*
* Another nice optimisation is to allow a timer to stay at an old place in the
* queue as long as it's not further than the real expiration date. That way,
* we use the tree as a place holder for a minorant of the real expiration
* date. Since we have a very low chance of hitting a timeout anyway, we can
* bounce the nodes to their right place when we scan the tree if we encounter
* a misplaced node once in a while. This even allows us not to remove the
* infinite timers from the wait queue.
*
* So, to summarize, we have :
* - node->key always defines current position in the wait queue
* - timer is the real expiration date (possibly infinite)
[MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1 Since we're now able to search from a precise expiration date in the timer tree using ebtree 4.1, we don't need to maintain 4 trees anymore. Not only does this simplify the code a lot, but it also ensures that we can always look 24 days back and ahead, which doubles the ability of the previous scheduler. Indeed, while based on absolute values, the timer tree is now relative to <now> as we can always search from <now>-31 bits. The run queue uses the exact same principle now, and is now simpler and a bit faster to process. With these changes alone, an overall 0.5% performance gain was observed. Tests were performed on the few wrapping cases and everything works as expected.
2009-03-21 09:01:42 +00:00
* - node->key is always before or equal to timer
[CLEANUP] task: distinguish between clock ticks and timers Timers are unsigned and used as tree positions. Ticks are signed and used as absolute date within current time frame. While the two are normally equal (except zero), it's important not to confuse them in the code as they are not interchangeable. We add two inline functions to turn each one into the other. The comments have also been moved to the proper location, as it was not easy to understand what was a tick and what was a timer unit.
2009-03-08 14:53:06 +00:00
*
* The run queue works similarly to the wait queue except that the current date
* is replaced by an insertion counter which can also wrap without any problem.
*/
[MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1 Since we're now able to search from a precise expiration date in the timer tree using ebtree 4.1, we don't need to maintain 4 trees anymore. Not only does this simplify the code a lot, but it also ensures that we can always look 24 days back and ahead, which doubles the ability of the previous scheduler. Indeed, while based on absolute values, the timer tree is now relative to <now> as we can always search from <now>-31 bits. The run queue uses the exact same principle now, and is now simpler and a bit faster to process. With these changes alone, an overall 0.5% performance gain was observed. Tests were performed on the few wrapping cases and everything works as expected.
2009-03-21 09:01:42 +00:00
/* The farthest we can look back in a timer tree */
#define TIMER_LOOK_BACK (1U << 31)
[CLEANUP] task: distinguish between clock ticks and timers Timers are unsigned and used as tree positions. Ticks are signed and used as absolute date within current time frame. While the two are normally equal (except zero), it's important not to confuse them in the code as they are not interchangeable. We add two inline functions to turn each one into the other. The comments have also been moved to the proper location, as it was not easy to understand what was a tick and what was a timer unit.
2009-03-08 14:53:06 +00:00
/* a few exported variables */
[MINOR] task: keep a task count and clean up task creators It's sometimes useful at least for statistics to keep a task count. It's easy to do by forcing the rare task creators to always use the same functions to create/destroy a task.
2009-03-21 17:13:21 +00:00
extern unsigned int nb_tasks; /* total number of tasks */
MINOR: tasks: Make active_tasks_mask volatile. To be sure we have the relevant informations, make active_tasks_mask volatile
2018-07-26 16:45:22 +00:00
extern volatile unsigned long active_tasks_mask; /* Mask of threads with active tasks */
MINOR: task: Rename run_queue and run_queue_cur counters <run_queue> is used to track the number of task in the run queue and <run_queue_cur> is a copy used for the reporting purpose. These counters has been renamed, respectively, <tasks_run_queue> and <tasks_run_queue_cur>. So the naming is consistent between tasks and applets. [wt: needed for next fixes, backport to 1.7 and 1.6]
2016-12-06 08:15:30 +00:00
extern unsigned int tasks_run_queue; /* run queue size */
extern unsigned int tasks_run_queue_cur;
[MINOR] stats: report number of tasks (active and running) It may be useful for statistics purposes to report the number of tasks.
2009-03-21 17:33:52 +00:00
extern unsigned int nb_tasks_cur;
[MEDIUM] introduce task->nice and boot access to statistics The run queue scheduler now considers task->nice to queue a task and to pick a task out of the queue. This makes it possible to boost the access to statistics (both via HTTP and UNIX socket). The UNIX socket receives twice as much a boost as the HTTP socket because it is more sensible.
2008-06-30 05:51:00 +00:00
extern unsigned int niced_tasks; /* number of niced tasks in the run queue */
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
extern struct pool_head *pool_head_task;
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
extern struct pool_head *pool_head_tasklet;
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
extern struct pool_head *pool_head_notification;
BUG/MEDIUM: task: Don't free a task that is about to be run. While running a task, we may try to delete and free a task that is about to be run, because it's part of the local tasks list, or because rq_next points to it. So flag any task that is in the local tasks list to be deleted, instead of run, by setting t->process to NULL, and re-make rq_next a global, thread-local variable, that is modified if we attempt to delete that task. Many thanks to PiBa-NL for reporting this and analysing the problem. This should be backported to 1.8.
2018-05-04 13:46:16 +00:00
extern THREAD_LOCAL struct task *curr_task; /* task currently running or NULL */
extern THREAD_LOCAL struct eb32sc_node *rq_next; /* Next task to be potentially run */
MINOR: tasks: Don't define rqueue if we're building without threads. To make sure we don't inadvertently insert task in the global runqueue, while only the local runqueue is used without threads, make its definition and usage conditional on USE_THREAD.
2018-06-06 12:22:03 +00:00
#ifdef USE_THREAD
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
extern struct eb_root timers; /* sorted timers tree, global */
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
extern struct eb_root rqueue; /* tree constituting the run queue */
BUG/MEDIUM: tasks: make __task_unlink_rq responsible for the rqueue size. As __task_wakeup() is responsible for increasing rqueue_local[tid]/global_rqueue_size, make __task_unlink_rq responsible for decreasing it, as process_runnable_tasks() isn't the only one that removes tasks from runqueues.
2018-07-26 13:59:38 +00:00
extern int global_rqueue_size; /* Number of element sin the global runqueue */
MINOR: tasks: Don't define rqueue if we're building without threads. To make sure we don't inadvertently insert task in the global runqueue, while only the local runqueue is used without threads, make its definition and usage conditional on USE_THREAD.
2018-06-06 12:22:03 +00:00
#endif
BUG/MEDIUM: tasks: make __task_unlink_rq responsible for the rqueue size. As __task_wakeup() is responsible for increasing rqueue_local[tid]/global_rqueue_size, make __task_unlink_rq responsible for decreasing it, as process_runnable_tasks() isn't the only one that removes tasks from runqueues.
2018-07-26 13:59:38 +00:00
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
/* force to split per-thread stuff into separate cache lines */
struct task_per_thread {
struct eb_root timers; /* tree constituting the per-thread wait queue */
struct eb_root rqueue; /* tree constituting the per-thread run queue */
struct list task_list; /* List of tasks to be run, mixing tasks and tasklets */
int task_list_size; /* Number of tasks in the task_list */
int rqueue_size; /* Number of elements in the per-thread run queue */
__attribute__((aligned(64))) char end[0];
};
extern struct task_per_thread task_per_thread[MAX_THREADS];
MINOR: threads: Use __decl_hathreads to declare locks This macro should be used to declare variables or struct members depending on the USE_THREAD compile option. It avoids the encapsulation of such declarations between #ifdef/#endif. It is used to declare all lock variables.
2017-11-13 09:34:01 +00:00
__decl_hathreads(extern HA_SPINLOCK_T rq_lock); /* spin lock related to run queue */
__decl_hathreads(extern HA_SPINLOCK_T wq_lock); /* spin lock related to wait queue */
[MAJOR] migrated task, tree64 and session to pool2 task and tree64 are already very close in size and are merged together. Overall performance gained slightly by this simple change.
2007-05-13 17:43:47 +00:00
MINOR: tasks: Allow tasklet_wakeup() to wakeup a task. Modify tasklet_wakeup() so that it handles a task as well, and inserts it directly into the tasklet list, making it effectively a tasklet. This should make future developments easier.
2018-08-01 13:58:44 +00:00
static inline void task_insert_into_tasklet_list(struct task *t);
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
/* return 0 if task is in run queue, otherwise non-zero */
static inline int task_in_rq(struct task *t)
{
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
/* Check if leaf_p is NULL, in case he's not in the runqueue, and if
* it's not 0x1, which would mean it's in the tasklet list.
*/
return t->rq.node.leaf_p != NULL && t->rq.node.leaf_p != (void *)0x1;
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
}
/* return 0 if task is in wait queue, otherwise non-zero */
static inline int task_in_wq(struct task *t)
{
return t->wq.node.leaf_p != NULL;
}
[MEDIUM] indicate a reason for a task wakeup It's very frequent to require some information about the reason why a task is running. Some flags have been added so that a task now knows if it got woken up due to I/O completion, timeout, etc...
2008-08-29 16:19:04 +00:00
/* puts the task <t> in run queue with reason flags <f>, and returns <t> */
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
/* This will put the task in the local runqueue if the task is only runnable
* by the current thread, in the global runqueue otherwies.
*/
void __task_wakeup(struct task *t, struct eb_root *);
static inline void task_wakeup(struct task *t, unsigned int f)
[OPTIM] reduce the number of calls to task_wakeup() A test has shown that more than 16% of the calls to task_wakeup() could be avoided because the task is already woken up. So make it inline and move the test to the inline part.
2008-08-29 13:26:14 +00:00
{
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
unsigned short state;
#ifdef USE_THREAD
struct eb_root *root;
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
if (t->thread_mask == tid_bit || global.nbthread == 1)
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
root = &task_per_thread[tid].rqueue;
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
else
root = &rqueue;
#else
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
struct eb_root *root = &task_per_thread[tid].rqueue;
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
#endif
BUG/MINOR: task: fix possibly missed event in inter-thread wakeups There's a very small but existing uncertainty window when waking another thread up where it is possible for task_wakeup() not to wake the other task up because it's still running while this once is in the process of finishing and loses its TASK_RUNNING flag. In this case the wakeup will be missed. The problem is that we have a single flag to store 3 states, since the transition from running to sleeping isn't atomic. Thus we need to have another flag to cover this part. This patch introduces TASK_QUEUED to mention that the task is already in the run queue, running or not. This bit will be removed while TASK_RUNNING is kept once dequeued, and will be used when removing TASK_RUNNING to check if the task has been requeued. It might be possible to slightly improve this but the occurrence rate is quite low and we don't really need to complexify the scheduler to optimize for a rare case. The impact with the current code is very low since we have few inter- thread wakeups. Most of them are caused by checks killing sessions. This must be backported to 1.9.
2019-01-27 16:41:27 +00:00
f |= TASK_QUEUED;
state = t->state;
while (!HA_ATOMIC_CAS(&t->state, &state, state | f))
;
if (!(state & TASK_QUEUED))
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
__task_wakeup(t, root);
[OPTIM] reduce the number of calls to task_wakeup() A test has shown that more than 16% of the calls to task_wakeup() could be avoided because the task is already woken up. So make it inline and move the test to the inline part.
2008-08-29 13:26:14 +00:00
}
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
CLEANUP: threads: rename process_mask to thread_mask It was a leftover from the last cleaning session; this mask applies to threads and calling it process_mask is a bit confusing. It's the same in fd, task and applets.
2017-10-31 15:06:06 +00:00
/* change the thread affinity of a task to <thread_mask> */
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
static inline void task_set_affinity(struct task *t, unsigned long thread_mask)
{
CLEANUP: threads: rename process_mask to thread_mask It was a leftover from the last cleaning session; this mask applies to threads and calling it process_mask is a bit confusing. It's the same in fd, task and applets.
2017-10-31 15:06:06 +00:00
t->thread_mask = thread_mask;
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
}
CLEANUP: threads: rename process_mask to thread_mask It was a leftover from the last cleaning session; this mask applies to threads and calling it process_mask is a bit confusing. It's the same in fd, task and applets.
2017-10-31 15:06:06 +00:00
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
/*
* Unlink the task from the wait queue, and possibly update the last_timer
* pointer. A pointer to the task itself is returned. The task *must* already
* be in the wait queue before calling this function. If unsure, use the safer
* task_unlink_wq() function.
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
*/
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
static inline struct task *__task_unlink_wq(struct task *t)
{
eb32_delete(&t->wq);
return t;
}
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
/* remove a task from its wait queue. It may either be the local wait queue if
* the task is bound to a single thread (in which case there's no locking
* involved) or the global queue, with locking.
*/
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
static inline struct task *task_unlink_wq(struct task *t)
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
{
BUG/MAJOR: fd/threads, task/threads: ensure all spin locks are unlocked Calculate if the fd or task should be locked once, before locking, and reuse the calculation when determing when to unlock. Fixes a race condition added in 87d54a9a for fds, and b20aa9ee for tasks, released in 1.9-dev4. When one thread modifies thread_mask to be a single thread for a task or fd while a second thread has locked or is waiting on a lock for that task or fd, the second thread will not unlock it. For FDs, this is observable when a listener is polled by multiple threads, and is closed while those threads have events pending. For tasks, this seems possible, where task_set_affinity is called, but I did not observe it. This must be backported to 1.9.
2019-02-20 20:43:45 +00:00
unsigned long locked;
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
if (likely(task_in_wq(t))) {
BUG/MAJOR: fd/threads, task/threads: ensure all spin locks are unlocked Calculate if the fd or task should be locked once, before locking, and reuse the calculation when determing when to unlock. Fixes a race condition added in 87d54a9a for fds, and b20aa9ee for tasks, released in 1.9-dev4. When one thread modifies thread_mask to be a single thread for a task or fd while a second thread has locked or is waiting on a lock for that task or fd, the second thread will not unlock it. For FDs, this is observable when a listener is polled by multiple threads, and is closed while those threads have events pending. For tasks, this seems possible, where task_set_affinity is called, but I did not observe it. This must be backported to 1.9.
2019-02-20 20:43:45 +00:00
locked = atleast2(t->thread_mask);
if (locked)
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
HA_SPIN_LOCK(TASK_WQ_LOCK, &wq_lock);
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
__task_unlink_wq(t);
BUG/MAJOR: fd/threads, task/threads: ensure all spin locks are unlocked Calculate if the fd or task should be locked once, before locking, and reuse the calculation when determing when to unlock. Fixes a race condition added in 87d54a9a for fds, and b20aa9ee for tasks, released in 1.9-dev4. When one thread modifies thread_mask to be a single thread for a task or fd while a second thread has locked or is waiting on a lock for that task or fd, the second thread will not unlock it. For FDs, this is observable when a listener is polled by multiple threads, and is closed while those threads have events pending. For tasks, this seems possible, where task_set_affinity is called, but I did not observe it. This must be backported to 1.9.
2019-02-20 20:43:45 +00:00
if (locked)
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
HA_SPIN_UNLOCK(TASK_WQ_LOCK, &wq_lock);
}
[MAJOR] replaced rbtree with ul2tree. The rbtree-based wait queue consumes a lot of CPU. Use the ul2tree instead. Lots of cleanups and code reorganizations made it possible to reduce the task struct and simplify the code a bit.
2007-04-29 08:41:56 +00:00
return t;
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
}
/*
MINOR: task: Rename run_queue and run_queue_cur counters <run_queue> is used to track the number of task in the run queue and <run_queue_cur> is a copy used for the reporting purpose. These counters has been renamed, respectively, <tasks_run_queue> and <tasks_run_queue_cur>. So the naming is consistent between tasks and applets. [wt: needed for next fixes, backport to 1.7 and 1.6]
2016-12-06 08:15:30 +00:00
* Unlink the task from the run queue. The tasks_run_queue size and number of
* niced tasks are updated too. A pointer to the task itself is returned. The
* task *must* already be in the run queue before calling this function. If
* unsure, use the safer task_unlink_rq() function. Note that the pointer to the
* next run queue entry is neither checked nor updated.
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
*/
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
static inline struct task *__task_unlink_rq(struct task *t)
{
BUG/MEDIUM: tasks: Don't forget to increase/decrease tasks_run_queue. Don't forget to increase tasks_run_queue when we're adding a task to the tasklet list, and to decrease it when we remove a task from a runqueue, or its value won't be accurate, and could lead to tasks not being executed when put in the global run queue. 1.9-dev only, no backport is needed.
2018-05-28 11:51:06 +00:00
HA_ATOMIC_SUB(&tasks_run_queue, 1);
BUG/MEDIUM: tasks: make __task_unlink_rq responsible for the rqueue size. As __task_wakeup() is responsible for increasing rqueue_local[tid]/global_rqueue_size, make __task_unlink_rq responsible for decreasing it, as process_runnable_tasks() isn't the only one that removes tasks from runqueues.
2018-07-26 13:59:38 +00:00
#ifdef USE_THREAD
if (t->state & TASK_GLOBAL) {
MINOR: tasks: Add a flag that tells if we're in the global runqueue. How that we have bits available in task->state, add a flag that tells if we're in the global runqueue or not.
2018-07-26 14:19:58 +00:00
HA_ATOMIC_AND(&t->state, ~TASK_GLOBAL);
BUG/MEDIUM: tasks: make __task_unlink_rq responsible for the rqueue size. As __task_wakeup() is responsible for increasing rqueue_local[tid]/global_rqueue_size, make __task_unlink_rq responsible for decreasing it, as process_runnable_tasks() isn't the only one that removes tasks from runqueues.
2018-07-26 13:59:38 +00:00
global_rqueue_size--;
} else
#endif
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
task_per_thread[tid].rqueue_size--;
BUG/MEDIUM: tasks: make __task_unlink_rq responsible for the rqueue size. As __task_wakeup() is responsible for increasing rqueue_local[tid]/global_rqueue_size, make __task_unlink_rq responsible for decreasing it, as process_runnable_tasks() isn't the only one that removes tasks from runqueues.
2018-07-26 13:59:38 +00:00
eb32sc_delete(&t->rq);
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
if (likely(t->nice))
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
HA_ATOMIC_SUB(&niced_tasks, 1);
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
return t;
}
MEDIUM: task: always ensure that the run queue is consistent As found by Thierry Fournier, if a task manages to kill another one and if this other task is the next one in the run queue, we can do whatever including crashing, because the scheduler restarts from the saved next task. For now, there is no such concept of a task killing another one, but with Lua it will come. A solution consists in always performing the lookup of the first task in the scheduler's loop, but it's expensive and costs around 2% of the performance. Another solution consists in keeping a global next run queue node and ensuring that when this task gets removed, it updates this pointer to the next one. This allows to simplify the code a bit and in the end to slightly increase the performance (0.3-0.5%). The mechanism might still be usable if we later migrate to a multi-threaded scheduler.
2015-02-23 15:07:01 +00:00
/* This function unlinks task <t> from the run queue if it is in it. It also
* takes care of updating the next run queue task if it was this task.
*/
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
static inline struct task *task_unlink_rq(struct task *t)
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
{
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
if (t->thread_mask != tid_bit)
HA_SPIN_LOCK(TASK_RQ_LOCK, &rq_lock);
BUG/MEDIUM: task: Don't free a task that is about to be run. While running a task, we may try to delete and free a task that is about to be run, because it's part of the local tasks list, or because rq_next points to it. So flag any task that is in the local tasks list to be deleted, instead of run, by setting t->process to NULL, and re-make rq_next a global, thread-local variable, that is modified if we attempt to delete that task. Many thanks to PiBa-NL for reporting this and analysing the problem. This should be backported to 1.8.
2018-05-04 13:46:16 +00:00
if (likely(task_in_rq(t))) {
if (&t->rq == rq_next)
rq_next = eb32sc_next(rq_next, tid_bit);
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
__task_unlink_rq(t);
BUG/MEDIUM: task: Don't free a task that is about to be run. While running a task, we may try to delete and free a task that is about to be run, because it's part of the local tasks list, or because rq_next points to it. So flag any task that is in the local tasks list to be deleted, instead of run, by setting t->process to NULL, and re-make rq_next a global, thread-local variable, that is modified if we attempt to delete that task. Many thanks to PiBa-NL for reporting this and analysing the problem. This should be backported to 1.8.
2018-05-04 13:46:16 +00:00
}
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
if (t->thread_mask != tid_bit)
HA_SPIN_UNLOCK(TASK_RQ_LOCK, &rq_lock);
[OPTIM] task_queue: assume most consecutive timers are equal When queuing a timer, it's very likely that an expiration date is equal to that of the previously queued timer, due to time rounding to the millisecond. Optimizing for this case provides a noticeable 1% performance boost.
2008-07-05 16:16:19 +00:00
return t;
}
[MAJOR] replace ultree with ebtree in wait-queues The ultree code has been removed in favor of a simpler and cleaner ebtree implementation. The eternity queue does not need to exist anymore, and the pool_tree64 has been removed. The ebtree node is stored in the task itself. The qlist list header is still used by the run-queue, but will be able to disappear once the run-queue uses ebtree too.
2008-06-24 06:17:16 +00:00
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
static inline void tasklet_wakeup(struct tasklet *tl)
{
MINOR: tasks: Allow tasklet_wakeup() to wakeup a task. Modify tasklet_wakeup() so that it handles a task as well, and inserts it directly into the tasklet list, making it effectively a tasklet. This should make future developments easier.
2018-08-01 13:58:44 +00:00
if (!TASK_IS_TASKLET(tl)) {
task_insert_into_tasklet_list((struct task *)tl);
return;
}
MINOR: tasklets: Don't attempt to add a tasklet in the list twice. Don't try to add a tasklet to the run queue if it's already in there, or we might get an infinite loop.
2018-07-17 16:29:22 +00:00
if (!LIST_ISEMPTY(&tl->list))
return;
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
LIST_ADDQ(&task_per_thread[tid].task_list, &tl->list);
task_per_thread[tid].task_list_size++;
BUG/MEDIUM: tasklets: Add the thread as active when waking a tasklet. Set the flag for the current thread in active_threads_mask when waking a tasklet, or we will never run it if no tasks are available. This is 1.9-specific, no backport is needed.
2018-08-17 16:57:51 +00:00
HA_ATOMIC_OR(&active_tasks_mask, tid_bit);
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
HA_ATOMIC_ADD(&tasks_run_queue, 1);
}
static inline void task_insert_into_tasklet_list(struct task *t)
{
struct tasklet *tl;
void *expected = NULL;
/* Protect ourself against anybody trying to insert the task into
* another runqueue. We set leaf_p to 0x1 to indicate that the node is
* not in a tree but that it's in the tasklet list. See task_in_rq().
*/
MINOR: task: Fix a compiler warning by adding a cast. When calling HA_ATOMIC_CAS with a pointer as the target, the compiler expects a pointer as the new value, so give it one by casting 0x1 to (void *).
2018-06-01 12:32:39 +00:00
if (unlikely(!HA_ATOMIC_CAS(&t->rq.node.leaf_p, &expected, (void *)0x1)))
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
return;
BUG/MEDIUM: tasks: Don't forget to increase/decrease tasks_run_queue. Don't forget to increase tasks_run_queue when we're adding a task to the tasklet list, and to decrease it when we remove a task from a runqueue, or its value won't be accurate, and could lead to tasks not being executed when put in the global run queue. 1.9-dev only, no backport is needed.
2018-05-28 11:51:06 +00:00
HA_ATOMIC_ADD(&tasks_run_queue, 1);
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
task_per_thread[tid].task_list_size++;
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
tl = (struct tasklet *)t;
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
LIST_ADDQ(&task_per_thread[tid].task_list, &tl->list);
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
}
static inline void task_remove_from_task_list(struct task *t)
{
LIST_DEL(&((struct tasklet *)t)->list);
MINOR: tasks: Make sure we correctly init and deinit a tasklet. Up until now, a tasklet couldn't be free'd while it was in the list, it is no longer the case, so make sure we remove it from the list before freeing it. To do so, we have to make sure we correctly initialize it, so use LIST_INIT, instead of setting the pointers to NULL.
2018-06-08 15:08:19 +00:00
LIST_INIT(&((struct tasklet *)t)->list);
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
task_per_thread[tid].task_list_size--;
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
HA_ATOMIC_SUB(&tasks_run_queue, 1);
if (!TASK_IS_TASKLET(t)) {
t->rq.node.leaf_p = NULL; // was 0x1
__ha_barrier_store();
}
}
[OPTIM] task_queue: assume most consecutive timers are equal When queuing a timer, it's very likely that an expiration date is equal to that of the previously queued timer, due to time rounding to the millisecond. Optimizing for this case provides a noticeable 1% performance boost.
2008-07-05 16:16:19 +00:00
/*
* Unlinks the task and adjusts run queue stats.
* A pointer to the task itself is returned.
*/
static inline struct task *task_delete(struct task *t)
{
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
task_unlink_wq(t);
task_unlink_rq(t);
[MAJOR] replace ultree with ebtree in wait-queues The ultree code has been removed in favor of a simpler and cleaner ebtree implementation. The eternity queue does not need to exist anymore, and the pool_tree64 has been removed. The ebtree node is stored in the task itself. The qlist list header is still used by the run-queue, but will be able to disappear once the run-queue uses ebtree too.
2008-06-24 06:17:16 +00:00
return t;
}
/*
[MINOR] task: keep a task count and clean up task creators It's sometimes useful at least for statistics to keep a task count. It's easy to do by forcing the rare task creators to always use the same functions to create/destroy a task.
2009-03-21 17:13:21 +00:00
* Initialize a new task. The bare minimum is performed (queue pointers and
* state). The task is returned. This function should not be used outside of
* task_new().
[MAJOR] replace ultree with ebtree in wait-queues The ultree code has been removed in favor of a simpler and cleaner ebtree implementation. The eternity queue does not need to exist anymore, and the pool_tree64 has been removed. The ebtree node is stored in the task itself. The qlist list header is still used by the run-queue, but will be able to disappear once the run-queue uses ebtree too.
2008-06-24 06:17:16 +00:00
*/
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
static inline struct task *task_init(struct task *t, unsigned long thread_mask)
[MAJOR] replace ultree with ebtree in wait-queues The ultree code has been removed in favor of a simpler and cleaner ebtree implementation. The eternity queue does not need to exist anymore, and the pool_tree64 has been removed. The ebtree node is stored in the task itself. The qlist list header is still used by the run-queue, but will be able to disappear once the run-queue uses ebtree too.
2008-06-24 06:17:16 +00:00
{
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
t->wq.node.leaf_p = NULL;
t->rq.node.leaf_p = NULL;
MAJOR: tasks: Create a per-thread runqueue. A lot of tasks are run on one thread only, so instead of having them all in the global runqueue, create a per-thread runqueue which doesn't require any locking, and add all tasks belonging to only one thread to the corresponding runqueue. The global runqueue is still used for non-local tasks, and is visited by each thread when checking its own runqueue. The nice parameter is thus used both in the global runqueue and in the local ones. The rare tasks that are bound to multiple threads will have their nice value used twice (once for the global queue, once for the thread-local one).
2018-05-18 16:38:23 +00:00
t->state = TASK_SLEEPING;
CLEANUP: threads: rename process_mask to thread_mask It was a leftover from the last cleaning session; this mask applies to threads and calling it process_mask is a bit confusing. It's the same in fd, task and applets.
2017-10-31 15:06:06 +00:00
t->thread_mask = thread_mask;
[MEDIUM] introduce task->nice and boot access to statistics The run queue scheduler now considers task->nice to queue a task and to pick a task out of the queue. This makes it possible to boost the access to statistics (both via HTTP and UNIX socket). The UNIX socket receives twice as much a boost as the HTTP socket because it is more sensible.
2008-06-30 05:51:00 +00:00
t->nice = 0;
[MINOR] show sess: report number of calls to each task For debugging purposes, it can be useful to know how many times each task has been called.
2009-03-28 16:54:35 +00:00
t->calls = 0;
MEDIUM: tasks: collect per-task CPU time and latency Right now we measure for each task the cumulated time spent waiting for the CPU and using it. The timestamp uses a 64-bit integer to report a nanosecond-level date. This is only enabled when "profiling.tasks" is enabled, and consumes less than 1% extra CPU on x86_64 when enabled. The cumulated processing time and wait time are reported in "show sess". The task's counters are also reset when an HTTP transaction is reset since the HTTP part pretends to restart on a fresh new stream. This will make sure we always report correct numbers for each request in the logs.
2018-05-31 12:48:54 +00:00
t->call_date = 0;
t->cpu_time = 0;
t->lat_time = 0;
MINOR: task: always preinitialize the task's timeout in task_init() task_init() is called exclusively by task_new() which is the only way to create a task. Most callers set t->expire to TICK_ETERNITY, some set it to another value and a few like Lua don't set it at all as they don't need a timeout, causing random values to be used in case the task gets queued. Let's always set t->expire to TICK_ETERNITY in task_init() so that all tasks are now initialized in a clean state. This patch can be backported as it will definitely make the code more robust (at least the Lua code, possibly other places).
2017-07-24 15:52:58 +00:00
t->expire = TICK_ETERNITY;
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
return t;
}
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
static inline void tasklet_init(struct tasklet *t)
{
t->nice = -32768;
t->calls = 0;
t->state = 0;
MINOR: tasklet: Set process to NULL. Some consumers expect the process to be NULL when a tasklet it created, so do so.
2018-07-19 14:02:16 +00:00
t->process = NULL;
MINOR: tasks: Make sure we correctly init and deinit a tasklet. Up until now, a tasklet couldn't be free'd while it was in the list, it is no longer the case, so make sure we remove it from the list before freeing it. To do so, we have to make sure we correctly initialize it, so use LIST_INIT, instead of setting the pointers to NULL.
2018-06-08 15:08:19 +00:00
LIST_INIT(&t->list);
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
}
static inline struct tasklet *tasklet_new(void)
{
struct tasklet *t = pool_alloc(pool_head_tasklet);
if (t) {
tasklet_init(t);
}
return t;
}
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
/*
[MINOR] task: keep a task count and clean up task creators It's sometimes useful at least for statistics to keep a task count. It's easy to do by forcing the rare task creators to always use the same functions to create/destroy a task.
2009-03-21 17:13:21 +00:00
* Allocate and initialise a new task. The new task is returned, or NULL in
* case of lack of memory. The task count is incremented. Tasks should only
* be allocated this way, and must be freed using task_free().
*/
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
static inline struct task *task_new(unsigned long thread_mask)
[MINOR] task: keep a task count and clean up task creators It's sometimes useful at least for statistics to keep a task count. It's easy to do by forcing the rare task creators to always use the same functions to create/destroy a task.
2009-03-21 17:13:21 +00:00
{
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
struct task *t = pool_alloc(pool_head_task);
[MINOR] task: keep a task count and clean up task creators It's sometimes useful at least for statistics to keep a task count. It's easy to do by forcing the rare task creators to always use the same functions to create/destroy a task.
2009-03-21 17:13:21 +00:00
if (t) {
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
HA_ATOMIC_ADD(&nb_tasks, 1);
task_init(t, thread_mask);
[MINOR] task: keep a task count and clean up task creators It's sometimes useful at least for statistics to keep a task count. It's easy to do by forcing the rare task creators to always use the same functions to create/destroy a task.
2009-03-21 17:13:21 +00:00
}
return t;
}
/*
* Free a task. Its context must have been freed since it will be lost.
* The task count is decremented.
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
*/
BUG/MEDIUM: task: Don't free a task that is about to be run. While running a task, we may try to delete and free a task that is about to be run, because it's part of the local tasks list, or because rq_next points to it. So flag any task that is in the local tasks list to be deleted, instead of run, by setting t->process to NULL, and re-make rq_next a global, thread-local variable, that is modified if we attempt to delete that task. Many thanks to PiBa-NL for reporting this and analysing the problem. This should be backported to 1.8.
2018-05-04 13:46:16 +00:00
static inline void __task_free(struct task *t)
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
{
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
pool_free(pool_head_task, t);
MINOR: task: release the task pool when stopping When we're stopping, we're not going to create new tasks anymore, so let's release the task pool upon each task_free() in order to reduce memory fragmentation.
2014-11-13 15:57:19 +00:00
if (unlikely(stopping))
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
pool_flush(pool_head_task);
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
HA_ATOMIC_SUB(&nb_tasks, 1);
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
}
BUG/MEDIUM: task: Don't free a task that is about to be run. While running a task, we may try to delete and free a task that is about to be run, because it's part of the local tasks list, or because rq_next points to it. So flag any task that is in the local tasks list to be deleted, instead of run, by setting t->process to NULL, and re-make rq_next a global, thread-local variable, that is modified if we attempt to delete that task. Many thanks to PiBa-NL for reporting this and analysing the problem. This should be backported to 1.8.
2018-05-04 13:46:16 +00:00
static inline void task_free(struct task *t)
{
/* There's no need to protect t->state with a lock, as the task
* has to run on the current thread.
*/
if (t == curr_task || !(t->state & TASK_RUNNING))
__task_free(t);
else
t->process = NULL;
}
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
static inline void tasklet_free(struct tasklet *tl)
{
BUG/MEDIUM: tasks: Decrement tasks_run_queue in tasklet_free(). If the tasklet is in the list, don't forget to decrement tasks_run_queue in tasklet_free(). This should be backported to 1.9.
2018-12-24 13:03:10 +00:00
if (!LIST_ISEMPTY(&tl->list)) {
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
task_per_thread[tid].task_list_size--;
BUG/MEDIUM: tasks: Decrement tasks_run_queue in tasklet_free(). If the tasklet is in the list, don't forget to decrement tasks_run_queue in tasklet_free(). This should be backported to 1.9.
2018-12-24 13:03:10 +00:00
HA_ATOMIC_SUB(&tasks_run_queue, 1);
}
MINOR: tasks: Make sure we correctly init and deinit a tasklet. Up until now, a tasklet couldn't be free'd while it was in the list, it is no longer the case, so make sure we remove it from the list before freeing it. To do so, we have to make sure we correctly initialize it, so use LIST_INIT, instead of setting the pointers to NULL.
2018-06-08 15:08:19 +00:00
LIST_DEL(&tl->list);
MAJOR: tasks: Introduce tasklets. Introduce tasklets, lightweight tasks. They have no notion of priority, they are just run as soon as possible, and will probably be used for I/O later. For the moment they're used to replace the temporary thread-local list that was used in the scheduler. The first part of the struct is common with tasks so that tasks can be cast to tasklets and queued in this list. Once a task is in the tasklet list, it has its leaf_p set to 0x1 so that it cannot accidently be confused as not in the queue. Pure tasklets are identifiable by their nice value of -32768 (which is normally not possible).
2018-05-18 16:45:28 +00:00
pool_free(pool_head_tasklet, tl);
if (unlikely(stopping))
pool_flush(pool_head_tasklet);
}
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
void __task_queue(struct task *task, struct eb_root *wq);
[OPTIM] task: don't unlink a task from a wait queue when waking it up In many situations, we wake a task on an I/O event, then queue it exactly where it was. This is a real waste because we delete/insert tasks into the wait queue for nothing. The only reason for this is that there was only one tree node in the task struct. By adding another tree node, we can have one tree for the timers (wait queue) and one tree for the priority (run queue). That way, we can have a task both in the run queue and wait queue at the same time. The wait queue now really holds timers, which is what it was designed for. The net gain is at least 1 delete/insert cycle per session, and up to 2-3 depending on the workload, since we save one cycle each time the expiration date is not changed during a wake up.
2009-03-07 16:25:21 +00:00
/* Place <task> into the wait queue, where it may already be. If the expiration
[OPTIM] task: reduce the number of calls to task_queue() Most of the time, task_queue() will immediately return. By extracting the preliminary checks and putting them in an inline function, we can significantly reduce the number of calls to the function itself, and most of the tests can be optimized away due to the caller's context. Another minor improvement in process_runnable_tasks() consisted in taking benefit from the processor's branch prediction unit by making a special case of the process_session() callback which is by far the most common one. All this improved performance by about 1%, mainly during the call from process_runnable_tasks().
2009-03-08 15:35:27 +00:00
* timer is infinite, do nothing and rely on wake_expired_task to clean up.
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
* If the task is bound to a single thread, it's assumed to be bound to the
* current thread's queue and is queued without locking. Otherwise it's queued
* into the global wait queue, protected by locks.
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
*/
[OPTIM] task: reduce the number of calls to task_queue() Most of the time, task_queue() will immediately return. By extracting the preliminary checks and putting them in an inline function, we can significantly reduce the number of calls to the function itself, and most of the tests can be optimized away due to the caller's context. Another minor improvement in process_runnable_tasks() consisted in taking benefit from the processor's branch prediction unit by making a special case of the process_session() callback which is by far the most common one. All this improved performance by about 1%, mainly during the call from process_runnable_tasks().
2009-03-08 15:35:27 +00:00
static inline void task_queue(struct task *task)
{
/* If we already have a place in the wait queue no later than the
* timeout we're trying to set, we'll stay there, because it is very
* unlikely that we will reach the timeout anyway. If the timeout
* has been disabled, it's useless to leave the queue as well. We'll
* rely on wake_expired_tasks() to catch the node and move it to the
* proper place should it ever happen. Finally we only add the task
* to the queue if it was not there or if it was further than what
* we want.
*/
if (!tick_isset(task->expire))
return;
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
#ifdef USE_THREAD
if (atleast2(task->thread_mask)) {
HA_SPIN_LOCK(TASK_WQ_LOCK, &wq_lock);
if (!task_in_wq(task) || tick_is_lt(task->expire, task->wq.key))
__task_queue(task, &timers);
HA_SPIN_UNLOCK(TASK_WQ_LOCK, &wq_lock);
} else
#endif
{
if (!task_in_wq(task) || tick_is_lt(task->expire, task->wq.key))
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
__task_queue(task, &task_per_thread[tid].timers);
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
}
[OPTIM] task: reduce the number of calls to task_queue() Most of the time, task_queue() will immediately return. By extracting the preliminary checks and putting them in an inline function, we can significantly reduce the number of calls to the function itself, and most of the tests can be optimized away due to the caller's context. Another minor improvement in process_runnable_tasks() consisted in taking benefit from the processor's branch prediction unit by making a special case of the process_session() callback which is by far the most common one. All this improved performance by about 1%, mainly during the call from process_runnable_tasks().
2009-03-08 15:35:27 +00:00
}
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
[MINOR] task: new function task_schedule() to schedule a wake up This function is used when a task should be woken up at most at a given date. This will be used with rate shapers.
2011-07-25 12:30:42 +00:00
/* Ensure <task> will be woken up at most at <when>. If the task is already in
* the run queue (but not running), nothing is done. It may be used that way
* with a delay : task_schedule(task, tick_add(now_ms, delay));
*/
static inline void task_schedule(struct task *task, int when)
{
MAJOR: threads/task: handle multithread on task scheduler 2 global locks have been added to protect, respectively, the run queue and the wait queue. And a process mask has been added on each task. Like for FDs, this mask is used to know which threads are allowed to process a task. For many tasks, all threads are granted. And this must be your first intension when you create a new task, else you have a good reason to make a task sticky on some threads. This is then the responsibility to the process callback to lock what have to be locked in the task context. Nevertheless, all tasks linked to a session must be sticky on the thread creating the session. It is important that I/O handlers processing session FDs and these tasks run on the same thread to avoid conflicts.
2017-09-27 12:59:38 +00:00
/* TODO: mthread, check if there is no tisk with this test */
[MINOR] task: new function task_schedule() to schedule a wake up This function is used when a task should be woken up at most at a given date. This will be used with rate shapers.
2011-07-25 12:30:42 +00:00
if (task_in_rq(task))
return;
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
#ifdef USE_THREAD
if (atleast2(task->thread_mask)) {
HA_SPIN_LOCK(TASK_WQ_LOCK, &wq_lock);
if (task_in_wq(task))
when = tick_first(when, task->expire);
task->expire = when;
if (!task_in_wq(task) || tick_is_lt(task->expire, task->wq.key))
__task_queue(task, &timers);
HA_SPIN_UNLOCK(TASK_WQ_LOCK, &wq_lock);
} else
#endif
{
if (task_in_wq(task))
when = tick_first(when, task->expire);
[MINOR] task: new function task_schedule() to schedule a wake up This function is used when a task should be woken up at most at a given date. This will be used with rate shapers.
2011-07-25 12:30:42 +00:00
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
task->expire = when;
if (!task_in_wq(task) || tick_is_lt(task->expire, task->wq.key))
OPTIM: tasks: group all tree roots per cache line Currently we have per-thread arrays of trees and counts, but these ones unfortunately share cache lines and are accessed very often. This patch moves the task-specific stuff into a structure taking a multiple of a cache line, and has one such per thread. Just doing this has reduced the cache miss ratio from 19.2% to 18.7% and increased the 12-thread test performance by 3%. It starts to become visible that we really need a process-wide per-thread storage area that would cover more than just these parts of the tasks. The code was arranged so that it's easy to move the pieces elsewhere if needed.
2018-10-15 14:12:48 +00:00
__task_queue(task, &task_per_thread[tid].timers);
MAJOR: tasks: create per-thread wait queues Now we still have a main contention point with the timers in the main wait queue, but the vast majority of the tasks are pinned to a single thread. This patch creates a per-thread wait queue and queues a task to the local wait queue without any locking if the task is bound to a single thread (the current one) otherwise to the shared queue using locking. This significantly reduces contention on the wait queue. A test with 12 threads showed 11 ms spent in the WQ lock compared to 4.7 seconds in the same test without this change. The cache miss ratio decreased from 19.7% to 19.2% on the 12-thread test, and its performance increased by 1.5%. Another indirect benefit is that the average queue size is divided by the number of threads, which roughly removes log(nbthreads) levels in the tree and further speeds up lookups.
2018-10-15 12:52:21 +00:00
}
[MINOR] task: new function task_schedule() to schedule a wake up This function is used when a task should be woken up at most at a given date. This will be used with rate shapers.
2011-07-25 12:30:42 +00:00
}
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
/* This function register a new signal. "lua" is the current lua
* execution context. It contains a pointer to the associated task.
* "link" is a list head attached to an other task that must be wake
* the lua task if an event occurs. This is useful with external
* events like TCP I/O or sleep functions. This funcion allocate
* memory for the signal.
*/
static inline struct notification *notification_new(struct list *purge, struct list *event, struct task *wakeup)
{
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
struct notification *com = pool_alloc(pool_head_notification);
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
if (!com)
return NULL;
LIST_ADDQ(purge, &com->purge_me);
LIST_ADDQ(event, &com->wake_me);
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_INIT(&com->lock);
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
com->task = wakeup;
return com;
}
/* This function purge all the pending signals when the LUA execution
* is finished. This prevent than a coprocess try to wake a deleted
* task. This function remove the memory associated to the signal.
DOC: notifications: add precisions about thread usage Precise the terms of use the notification functions.
2017-12-10 16:14:07 +00:00
* The purge list is not locked because it is owned by only one
* process. before browsing this list, the caller must ensure to be
* the only one browser.
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
*/
static inline void notification_purge(struct list *purge)
{
struct notification *com, *back;
/* Delete all pending communication signals. */
list_for_each_entry_safe(com, back, purge, purge_me) {
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_LOCK(NOTIF_LOCK, &com->lock);
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
LIST_DEL(&com->purge_me);
MEDIUM: threads/tasks: Add lock around notifications This patch add lock around some notification calls
2017-07-16 22:14:07 +00:00
if (!com->task) {
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_UNLOCK(NOTIF_LOCK, &com->lock);
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
pool_free(pool_head_notification, com);
MEDIUM: threads/tasks: Add lock around notifications This patch add lock around some notification calls
2017-07-16 22:14:07 +00:00
continue;
}
com->task = NULL;
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_UNLOCK(NOTIF_LOCK, &com->lock);
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
}
}
BUG/MEDIUM: lua/notification: memory leak The thread patches adds refcount for notifications. The notifications are used with the Lua cosocket. These refcount free the notifications when the session is cleared. In the Lua task case, it not have sessions, so the nofications are never cleraed. This patch adds a garbage collector for signals. The garbage collector just clean the notifications for which the end point is disconnected. This patch should be backported in 1.8
2017-12-10 16:10:57 +00:00
/* In some cases, the disconnected notifications must be cleared.
* This function just release memory blocs. The purge list is not
* locked because it is owned by only one process. Before browsing
* this list, the caller must ensure to be the only one browser.
* The "com" is not locked because when com->task is NULL, the
* notification is no longer used.
*/
static inline void notification_gc(struct list *purge)
{
struct notification *com, *back;
/* Delete all pending communication signals. */
list_for_each_entry_safe (com, back, purge, purge_me) {
if (com->task)
continue;
LIST_DEL(&com->purge_me);
pool_free(pool_head_notification, com);
}
}
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
/* This function sends signals. It wakes all the tasks attached
* to a list head, and remove the signal, and free the used
DOC: notifications: add precisions about thread usage Precise the terms of use the notification functions.
2017-12-10 16:14:07 +00:00
* memory. The wake list is not locked because it is owned by
* only one process. before browsing this list, the caller must
* ensure to be the only one browser.
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
*/
static inline void notification_wake(struct list *wake)
{
struct notification *com, *back;
/* Wake task and delete all pending communication signals. */
list_for_each_entry_safe(com, back, wake, wake_me) {
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_LOCK(NOTIF_LOCK, &com->lock);
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
LIST_DEL(&com->wake_me);
MEDIUM: threads/tasks: Add lock around notifications This patch add lock around some notification calls
2017-07-16 22:14:07 +00:00
if (!com->task) {
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_UNLOCK(NOTIF_LOCK, &com->lock);
CLEANUP: pools: rename all pool functions and pointers to remove this "2" During the migration to the second version of the pools, the new functions and pool pointers were all called "pool_something2()" and "pool2_something". Now there's no more pool v1 code and it's a real pain to still have to deal with this. Let's clean this up now by removing the "2" everywhere, and by renaming the pool heads "pool_head_something".
2017-11-24 16:34:44 +00:00
pool_free(pool_head_notification, com);
MEDIUM: threads/tasks: Add lock around notifications This patch add lock around some notification calls
2017-07-16 22:14:07 +00:00
continue;
}
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
task_wakeup(com->task, TASK_WOKEN_MSG);
MEDIUM: threads/tasks: Add lock around notifications This patch add lock around some notification calls
2017-07-16 22:14:07 +00:00
com->task = NULL;
BUILD: threads: Rename SPIN/RWLOCK macros using HA_ prefix This remove any name conflicts, especially on Solaris.
2017-11-07 09:42:54 +00:00
HA_SPIN_UNLOCK(NOTIF_LOCK, &com->lock);
MINOR: tasks: Move Lua notification from Lua to tasks These notification management function and structs are generic and it will be better to move in common parts. The notification management functions and structs have names containing some "lua" references because it was written for the Lua. This patch removes also these references.
2017-07-12 12:31:10 +00:00
}
}
MINOR: task/notification: Is notifications registered ? This function returns true is some notifications are registered. This function is usefull for the following patch BUG/MEDIUM: lua/socket: Sheduling error on write: may dead-lock It should be backported in 1.6, 1.7 and 1.8
2018-05-30 09:40:08 +00:00
/* This function returns true is some notification are pending
*/
static inline int notification_registered(struct list *wake)
{
return !LIST_ISEMPTY(wake);
}
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
/*
[MAJOR] proxy: finally get rid of maintain_proxies() This function is finally not needed anymore, as it has been replaced with a per-proxy task that is scheduled when some limits are encountered on incoming connections or when the process is stopping. The savings should be noticeable on configs with a large number of proxies. The most important point is that the rate limiting is now enforced in a clean and solid way.
2011-07-25 14:33:49 +00:00
* This does 3 things :
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
* - wake up all expired tasks
* - call all runnable tasks
[MAJOR] replaced all timeouts with struct timeval The timeout functions were difficult to manipulate because they were rounding results to the millisecond. Thus, it was difficult to compare and to check what expired and what did not. Also, the comparison functions were heavy with multiplies and divides by 1000. Now, all timeouts are stored in timevals, reducing the number of operations for updates and leading to cleaner and more efficient code.
2007-05-12 20:35:00 +00:00
* - return the date of next event in <next> or eternity.
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
*/
MAJOR: poll: only rely on wake_expired_tasks() to compute the wait delay Actually, HAProxy uses the function "process_runnable_tasks" and "wake_expired_tasks" to get the next task which can expires. If a task is added with "task_schedule" or other method during the execution of an other task, the expiration of this new task is not taken into account, and the execution of this task can be too late. Actualy, HAProxy seems to be no sensitive to this bug. This fix moves the call to process_runnable_tasks() before the timeout calculation and ensures that all wakeups are processed together. Only wake_expired_tasks() needs to return a timeout now.
2014-12-15 12:26:01 +00:00
void process_runnable_tasks();
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
[MAJOR] use an ebtree instead of a list for the run queue We now insert tasks in a certain sequence in the run queue. The sorting key currently is the arrival order. It will now be possible to apply a "nice" value to any task so that it goes forwards or backwards in the run queue. The calls to wake_expired_tasks() and maintain_proxies() have been moved to the main run_poll_loop(), because they had nothing to do in process_runnable_tasks(). The task_wakeup() function is not inlined anymore, as it was only used at one place. The qlist member of the task structure has been removed now. The run_queue list has been replaced for an integer indicating the number of tasks in the run queue.
2008-06-29 20:40:23 +00:00
/*
* Extract all expired timers from the timer queue, and wakes up all
* associated tasks. Returns the date of next event (or eternity).
*/
MAJOR: poll: only rely on wake_expired_tasks() to compute the wait delay Actually, HAProxy uses the function "process_runnable_tasks" and "wake_expired_tasks" to get the next task which can expires. If a task is added with "task_schedule" or other method during the execution of an other task, the expiration of this new task is not taken into account, and the execution of this task can be too late. Actualy, HAProxy seems to be no sensitive to this bug. This fix moves the call to process_runnable_tasks() before the timeout calculation and ensures that all wakeups are processed together. Only wake_expired_tasks() needs to return a timeout now.
2014-12-15 12:26:01 +00:00
int wake_expired_tasks();
[MAJOR] use an ebtree instead of a list for the run queue We now insert tasks in a certain sequence in the run queue. The sorting key currently is the arrival order. It will now be possible to apply a "nice" value to any task so that it goes forwards or backwards in the run queue. The calls to wake_expired_tasks() and maintain_proxies() have been moved to the main run_poll_loop(), because they had nothing to do in process_runnable_tasks(). The task_wakeup() function is not inlined anymore, as it was only used at one place. The qlist member of the task structure has been removed now. The run_queue list has been replaced for an integer indicating the number of tasks in the run queue.
2008-06-29 20:40:23 +00:00
BUG/MEDIUM: mworker: stop every tasks in the master The master is not supposed to run (at the moment) any task before the polling loop, the created tasks should be run only in the workers but in the master they should be disabled or removed. No backport needed.
2018-12-06 13:05:20 +00:00
/*
* Delete every tasks before running the master polling loop
*/
void mworker_cleantasks();
[BIGMOVE] exploded the monolithic haproxy.c file into multiple files. The files are now stored under : - include/haproxy for the generic includes - include/types.h for the structures needed within prototypes - include/proto.h for function prototypes and inline functions - src/*.c for the C files Most include files are now covered by LGPL. A last move still needs to be done to put inline functions under GPL and not LGPL. Version has been set to 1.3.0 in the code but some control still needs to be done before releasing.
2006-06-26 00:48:02 +00:00
#endif /* _PROTO_TASK_H */
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/
Reference in New Issue Copy Permalink