RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-18 09:24:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
5775d0964a
haproxy/src/hathreads.c

252 lines
7.0 KiB
C
Raw Normal View History

MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
/*
* functions about threads.
*
* Copyright (C) 2017 Christopher Fauet - cfaulet@haproxy.com
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*
*/
MAJOR: threads: enable one thread per CPU by default Threads have long matured by now, still for most users their usage is not trivial. It's about time to enable them by default on platforms where we know the number of CPUs bound. This patch does this, it counts the number of CPUs the process is bound to upon startup, and enables as many threads by default. Of course, "nbthread" still overrides this, but if it's not set the default behaviour is to start one thread per CPU. The default number of threads is reported in "haproxy -vv". Simply using "taskset -c" is now enough to adjust this number of threads so that there is no more need for playing with cpu-map. And thanks to the previous patches on the listener, the vast majority of configurations will not need to duplicate "bind" lines with the "process x/y" statement anymore either, so a simple config will automatically adapt to the number of processors available.
2019-01-26 13:27:06 +00:00
#define _GNU_SOURCE
MEDIUM: threads: Adds a set of functions to handle sync-point A sync-point is a protected area where you have the warranty that no concurrency access is possible. It is implementated as a thread barrier to enter in the sync-point and another one to exit from it. Inside the sync-point, all threads that must do some syncrhonous processing will be called one after the other while all other threads will wait. All threads will then exit from the sync-point at the same time. A sync-point will be evaluated only when necessary because it is a costly operation. To limit the waiting time of each threads, we must have a mechanism to wakeup all threads. This is done with a pipe shared by all threads. By writting in this pipe, we will interrupt all threads blocked on a poller. The pipe is then flushed before exiting from the sync-point.
2017-10-19 09:59:15 +00:00
#include <unistd.h>
MINOR: threads: move "nbthread" parsing to hathreads.c The purpose is to make sure that all variables which directly depend on this nbthread argument are set at the right moment. For now only all_threads_mask needs to be set. It used to be set while calling thread_sync_init() which is called too late for certain checks. The same function handles threads and non-threads, which removes the need for some thread-specific knowledge from cfgparse.c.
2018-07-30 08:34:35 +00:00
#include <stdlib.h>
MEDIUM: threads: Adds a set of functions to handle sync-point A sync-point is a protected area where you have the warranty that no concurrency access is possible. It is implementated as a thread barrier to enter in the sync-point and another one to exit from it. Inside the sync-point, all threads that must do some syncrhonous processing will be called one after the other while all other threads will wait. All threads will then exit from the sync-point at the same time. A sync-point will be evaluated only when necessary because it is a costly operation. To limit the waiting time of each threads, we must have a mechanism to wakeup all threads. This is done with a pipe shared by all threads. By writting in this pipe, we will interrupt all threads blocked on a poller. The pipe is then flushed before exiting from the sync-point.
2017-10-19 09:59:15 +00:00
#include <fcntl.h>
MAJOR: threads: enable one thread per CPU by default Threads have long matured by now, still for most users their usage is not trivial. It's about time to enable them by default on platforms where we know the number of CPUs bound. This patch does this, it counts the number of CPUs the process is bound to upon startup, and enables as many threads by default. Of course, "nbthread" still overrides this, but if it's not set the default behaviour is to start one thread per CPU. The default number of threads is reported in "haproxy -vv". Simply using "taskset -c" is now enough to adjust this number of threads so that there is no more need for playing with cpu-map. And thanks to the previous patches on the listener, the vast majority of configurations will not need to duplicate "bind" lines with the "process x/y" statement anymore either, so a simple config will automatically adapt to the number of processors available.
2019-01-26 13:27:06 +00:00
#ifdef USE_CPU_AFFINITY
#include <sched.h>
#endif
MINOR: threads: Implement thread_cpus_enabled() for FreeBSD. Use cpuset_getaffinity() to implement thread_cpus_enabled() on FreeBSD, so that we can know the number of CPUs available, and automatically launch as much threads if nbthread isn't specified.
2019-04-10 22:06:47 +00:00
#ifdef __FreeBSD__
#include <sys/cpuset.h>
#endif
MINOR: thread: report multi-thread support in haproxy -vv Otherwise it's hard to know if it was enabled or not.
2017-11-03 22:39:25 +00:00
#include <common/cfgparse.h>
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
#include <common/hathreads.h>
MEDIUM: threads: Adds a set of functions to handle sync-point A sync-point is a protected area where you have the warranty that no concurrency access is possible. It is implementated as a thread barrier to enter in the sync-point and another one to exit from it. Inside the sync-point, all threads that must do some syncrhonous processing will be called one after the other while all other threads will wait. All threads will then exit from the sync-point at the same time. A sync-point will be evaluated only when necessary because it is a costly operation. To limit the waiting time of each threads, we must have a mechanism to wakeup all threads. This is done with a pipe shared by all threads. By writting in this pipe, we will interrupt all threads blocked on a poller. The pipe is then flushed before exiting from the sync-point.
2017-10-19 09:59:15 +00:00
#include <common/standard.h>
MINOR: initcall: apply initcall to all register_build_opts() calls Most register_build_opts() calls use static strings. These ones were replaced with a trivial REGISTER_BUILD_OPTS() statement adding the string and its call to the STG_REGISTER section. A dedicated section could be made for this if needed, but there are very few such calls for this to be worth it. The calls made with computed strings however, like those which retrieve OpenSSL's version or zlib's version, were moved to a dedicated function to guarantee they are called late in the process. For example, the SSL call probably requires that SSL_library_init() has been called first.
2018-11-26 09:19:54 +00:00
#include <types/global.h>
MEDIUM: threads: Adds a set of functions to handle sync-point A sync-point is a protected area where you have the warranty that no concurrency access is possible. It is implementated as a thread barrier to enter in the sync-point and another one to exit from it. Inside the sync-point, all threads that must do some syncrhonous processing will be called one after the other while all other threads will wait. All threads will then exit from the sync-point at the same time. A sync-point will be evaluated only when necessary because it is a costly operation. To limit the waiting time of each threads, we must have a mechanism to wakeup all threads. This is done with a pipe shared by all threads. By writting in this pipe, we will interrupt all threads blocked on a poller. The pipe is then flushed before exiting from the sync-point.
2017-10-19 09:59:15 +00:00
#include <proto/fd.h>
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
BUILD/MEDIUM: threads: rename thread_info struct to ha_thread_info On Darwin, the thread_info name exists as a standard function thus we need to rename our array to ha_thread_info to fix this conflict.
2019-09-13 04:03:12 +00:00
struct thread_info ha_thread_info[MAX_THREADS] = { };
THREAD_LOCAL struct thread_info *ti = &ha_thread_info[0];
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
#ifdef USE_THREAD
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
volatile unsigned long threads_want_rdv_mask = 0;
volatile unsigned long threads_harmless_mask = 0;
MEDIUM: threads: add thread_sync_release() to synchronize steps This function provides an alternate way to leave a critical section run under thread_isolate(). Currently, a thread may remain in thread_release() without having the time to notice that the rdv mask was released and taken again by another thread entering thread_isolate() (often the same that just released it). This is because threads wait in harmless mode in the loop, which is compatible with the conditions to enter thread_isolate(). It's not possible to make them wait with the harmless bit off or we cannot know when the job is finished for the next thread to start in thread_isolate(), and if we don't clear the rdv bit when going there, we create another race on the start point of thread_isolate(). This new synchronous variant of thread_release() makes use of an extra mask to indicate the threads that want to be synchronously released. In this case, they will be marked harmless before releasing their sync bit, and will wait for others to release their bit as well, guaranteeing that thread_isolate() cannot be started by any of them before they all left thread_sync_release(). This allows to construct synchronized blocks like this : thread_isolate() /* optionally do something alone here */ thread_sync_release() /* do something together here */ thread_isolate() /* optionally do something alone here */ thread_sync_release() And so on. This is particularly useful during initialization where several steps have to be respected and no thread must start a step before the previous one is completed by other threads. This one must not be placed after any call to thread_release() or it would risk to block an earlier call to thread_isolate() which the current thread managed to leave without waiting for others to complete, and end up here with the thread's harmless bit cleared, blocking others. This might be improved in the future.
2019-06-09 10:20:02 +00:00
volatile unsigned long threads_sync_mask = 0;
MINOR: threads: move "nbthread" parsing to hathreads.c The purpose is to make sure that all variables which directly depend on this nbthread argument are set at the right moment. For now only all_threads_mask needs to be set. It used to be set while calling thread_sync_init() which is called too late for certain checks. The same function handles threads and non-threads, which removes the need for some thread-specific knowledge from cfgparse.c.
2018-07-30 08:34:35 +00:00
volatile unsigned long all_threads_mask = 1; // nbthread 1 assumed by default
MINOR: threads: add more consistency between certain variables in no-thread case When threads are disabled, some variables such as tid and tid_bit are still checked everywhere, the MAX_THREADS_MASK macro is ~0UL while MAX_THREADS is 1, and the all_threads_mask variable is replaced with a macro forced to zero. The compiler cannot optimize away all this code involving checks on tid and tid_bit, and we end up in special cases where all_threads_mask has to be specifically tested for being zero or not. It is not even certain the code paths are always equivalent when testing without threads and with nbthread 1. Let's change this to make sure we always present a single thread when threads are disabled, and have the relevant values declared as constants so that the compiler can optimize all the tests away. Now we have MAX_THREADS_MASK set to 1, all_threads_mask set to 1, tid set to zero and tid_bit set to 1. Doing just this has removed 4 kB of code in the no-thread case. A few checks for all_threads_mask==0 have been removed since it never happens anymore.
2018-08-01 17:12:20 +00:00
THREAD_LOCAL unsigned int tid = 0;
THREAD_LOCAL unsigned long tid_bit = (1UL << 0);
MAJOR: threads: enable one thread per CPU by default Threads have long matured by now, still for most users their usage is not trivial. It's about time to enable them by default on platforms where we know the number of CPUs bound. This patch does this, it counts the number of CPUs the process is bound to upon startup, and enables as many threads by default. Of course, "nbthread" still overrides this, but if it's not set the default behaviour is to start one thread per CPU. The default number of threads is reported in "haproxy -vv". Simply using "taskset -c" is now enough to adjust this number of threads so that there is no more need for playing with cpu-map. And thanks to the previous patches on the listener, the vast majority of configurations will not need to duplicate "bind" lines with the "process x/y" statement anymore either, so a simple config will automatically adapt to the number of processors available.
2019-01-26 13:27:06 +00:00
int thread_cpus_enabled_at_boot = 1;
MINOR: threads: add more consistency between certain variables in no-thread case When threads are disabled, some variables such as tid and tid_bit are still checked everywhere, the MAX_THREADS_MASK macro is ~0UL while MAX_THREADS is 1, and the all_threads_mask variable is replaced with a macro forced to zero. The compiler cannot optimize away all this code involving checks on tid and tid_bit, and we end up in special cases where all_threads_mask has to be specifically tested for being zero or not. It is not even certain the code paths are always equivalent when testing without threads and with nbthread 1. Let's change this to make sure we always present a single thread when threads are disabled, and have the relevant values declared as constants so that the compiler can optimize all the tests away. Now we have MAX_THREADS_MASK set to 1, all_threads_mask set to 1, tid set to zero and tid_bit set to 1. Doing just this has removed 4 kB of code in the no-thread case. A few checks for all_threads_mask==0 have been removed since it never happens anymore.
2018-08-01 17:12:20 +00:00
MEDIUM: threads: Adds a set of functions to handle sync-point A sync-point is a protected area where you have the warranty that no concurrency access is possible. It is implementated as a thread barrier to enter in the sync-point and another one to exit from it. Inside the sync-point, all threads that must do some syncrhonous processing will be called one after the other while all other threads will wait. All threads will then exit from the sync-point at the same time. A sync-point will be evaluated only when necessary because it is a costly operation. To limit the waiting time of each threads, we must have a mechanism to wakeup all threads. This is done with a pipe shared by all threads. By writting in this pipe, we will interrupt all threads blocked on a poller. The pipe is then flushed before exiting from the sync-point.
2017-10-19 09:59:15 +00:00
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
#if defined(DEBUG_THREAD) || defined(DEBUG_FULL)
struct lock_stat lock_stats[LOCK_LABELS];
#endif
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
/* Marks the thread as harmless until the last thread using the rendez-vous
* point quits. Given that we can wait for a long time, sched_yield() is used
* when available to offer the CPU resources to competing threads if needed.
*/
void thread_harmless_till_end()
{
MEDIUM: threads: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:51:17 +00:00
_HA_ATOMIC_OR(&threads_harmless_mask, tid_bit);
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
while (threads_want_rdv_mask & all_threads_mask) {
MINOR: thread: implement ha_thread_relax() At some places we're using a painful ifdef to decide whether to use sched_yield() or pl_cpu_relax() to relax in loops, this is hardly exportable. Let's move this to ha_thread_relax() instead and une this one only.
2019-05-17 14:33:13 +00:00
ha_thread_relax();
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
}
}
/* Isolates the current thread : request the ability to work while all other
* threads are harmless. Only returns once all of them are harmless, with the
* current thread's bit in threads_harmless_mask cleared. Needs to be completed
* using thread_release().
*/
void thread_isolate()
{
unsigned long old;
MEDIUM: threads: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:51:17 +00:00
_HA_ATOMIC_OR(&threads_harmless_mask, tid_bit);
__ha_barrier_atomic_store();
_HA_ATOMIC_OR(&threads_want_rdv_mask, tid_bit);
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
/* wait for all threads to become harmless */
old = threads_harmless_mask;
while (1) {
if (unlikely((old & all_threads_mask) != all_threads_mask))
old = threads_harmless_mask;
MEDIUM: threads: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:51:17 +00:00
else if (_HA_ATOMIC_CAS(&threads_harmless_mask, &old, old & ~tid_bit))
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
break;
MINOR: thread: implement ha_thread_relax() At some places we're using a painful ifdef to decide whether to use sched_yield() or pl_cpu_relax() to relax in loops, this is hardly exportable. Let's move this to ha_thread_relax() instead and une this one only.
2019-05-17 14:33:13 +00:00
ha_thread_relax();
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
}
/* one thread gets released at a time here, with its harmess bit off.
* The loss of this bit makes the other one continue to spin while the
* thread is working alone.
*/
}
/* Cancels the effect of thread_isolate() by releasing the current thread's bit
* in threads_want_rdv_mask and by marking this thread as harmless until the
* last worker finishes.
*/
void thread_release()
{
MEDIUM: threads: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:51:17 +00:00
_HA_ATOMIC_AND(&threads_want_rdv_mask, ~tid_bit);
MINOR: threads: avoid clearing harmless twice in thread_release() thread_release() is to be called after thread_isolate(), i.e. when the thread already has its harmless bit cleared. No need to clear it twice, thus avoid calling thread_harmless_end() and directly check the rdv bits then loop on them.
2019-06-09 06:44:19 +00:00
while (threads_want_rdv_mask & all_threads_mask) {
_HA_ATOMIC_OR(&threads_harmless_mask, tid_bit);
while (threads_want_rdv_mask & all_threads_mask)
ha_thread_relax();
HA_ATOMIC_AND(&threads_harmless_mask, ~tid_bit);
}
MEDIUM: hathreads: implement a more flexible rendez-vous point The current synchronization point enforces certain restrictions which are hard to workaround in certain areas of the code. The fact that the critical code can only be called from the sync point itself is a problem for some callback-driven parts. The "show fd" command for example is fragile regarding this. Also it is expensive in terms of CPU usage because it wakes every other thread just to be sure all of them join to the rendez-vous point. It's a problem because the sleeping threads would not need to be woken up just to know they're doing nothing. Here we implement a different approach. We keep track of harmless threads, which are defined as those either doing nothing, or doing harmless things. The rendez-vous is used "for others" as a way for a thread to isolate itself. A thread then requests to be alone using thread_isolate() when approaching the dangerous area, and then waits until all other threads are either doing the same or are doing something harmless (typically polling). The function only returns once the thread is guaranteed to be alone, and the critical section is terminated using thread_release().
2018-08-02 08:16:17 +00:00
}
MEDIUM: threads: Adds a set of functions to handle sync-point A sync-point is a protected area where you have the warranty that no concurrency access is possible. It is implementated as a thread barrier to enter in the sync-point and another one to exit from it. Inside the sync-point, all threads that must do some syncrhonous processing will be called one after the other while all other threads will wait. All threads will then exit from the sync-point at the same time. A sync-point will be evaluated only when necessary because it is a costly operation. To limit the waiting time of each threads, we must have a mechanism to wakeup all threads. This is done with a pipe shared by all threads. By writting in this pipe, we will interrupt all threads blocked on a poller. The pipe is then flushed before exiting from the sync-point.
2017-10-19 09:59:15 +00:00
MEDIUM: threads: add thread_sync_release() to synchronize steps This function provides an alternate way to leave a critical section run under thread_isolate(). Currently, a thread may remain in thread_release() without having the time to notice that the rdv mask was released and taken again by another thread entering thread_isolate() (often the same that just released it). This is because threads wait in harmless mode in the loop, which is compatible with the conditions to enter thread_isolate(). It's not possible to make them wait with the harmless bit off or we cannot know when the job is finished for the next thread to start in thread_isolate(), and if we don't clear the rdv bit when going there, we create another race on the start point of thread_isolate(). This new synchronous variant of thread_release() makes use of an extra mask to indicate the threads that want to be synchronously released. In this case, they will be marked harmless before releasing their sync bit, and will wait for others to release their bit as well, guaranteeing that thread_isolate() cannot be started by any of them before they all left thread_sync_release(). This allows to construct synchronized blocks like this : thread_isolate() /* optionally do something alone here */ thread_sync_release() /* do something together here */ thread_isolate() /* optionally do something alone here */ thread_sync_release() And so on. This is particularly useful during initialization where several steps have to be respected and no thread must start a step before the previous one is completed by other threads. This one must not be placed after any call to thread_release() or it would risk to block an earlier call to thread_isolate() which the current thread managed to leave without waiting for others to complete, and end up here with the thread's harmless bit cleared, blocking others. This might be improved in the future.
2019-06-09 10:20:02 +00:00
/* Cancels the effect of thread_isolate() by releasing the current thread's bit
* in threads_want_rdv_mask and by marking this thread as harmless until the
* last worker finishes. The difference with thread_release() is that this one
* will not leave the function before others are notified to do the same, so it
* guarantees that the current thread will not pass through a subsequent call
* to thread_isolate() before others finish.
*/
void thread_sync_release()
{
_HA_ATOMIC_OR(&threads_sync_mask, tid_bit);
__ha_barrier_atomic_store();
_HA_ATOMIC_AND(&threads_want_rdv_mask, ~tid_bit);
while (threads_want_rdv_mask & all_threads_mask) {
_HA_ATOMIC_OR(&threads_harmless_mask, tid_bit);
while (threads_want_rdv_mask & all_threads_mask)
ha_thread_relax();
HA_ATOMIC_AND(&threads_harmless_mask, ~tid_bit);
}
/* the current thread is not harmless anymore, thread_isolate()
* is forced to wait till all waiters finish.
*/
_HA_ATOMIC_AND(&threads_sync_mask, ~tid_bit);
while (threads_sync_mask & all_threads_mask)
ha_thread_relax();
}
MINOR: threads: implement ha_tkill() and ha_tkillall() These functions are used respectively to signal one thread or all threads. When multithreading is disabled, it's always the current thread which is signaled.
2019-05-22 06:43:34 +00:00
/* send signal <sig> to thread <thr> */
void ha_tkill(unsigned int thr, int sig)
{
BUILD/MEDIUM: threads: rename thread_info struct to ha_thread_info On Darwin, the thread_info name exists as a standard function thus we need to rename our array to ha_thread_info to fix this conflict.
2019-09-13 04:03:12 +00:00
pthread_kill(ha_thread_info[thr].pthread, sig);
MINOR: threads: implement ha_tkill() and ha_tkillall() These functions are used respectively to signal one thread or all threads. When multithreading is disabled, it's always the current thread which is signaled.
2019-05-22 06:43:34 +00:00
}
/* send signal <sig> to all threads. The calling thread is signaled last in
* order to allow all threads to synchronize in the handler.
*/
void ha_tkillall(int sig)
{
unsigned int thr;
for (thr = 0; thr < global.nbthread; thr++) {
if (!(all_threads_mask & (1UL << thr)))
continue;
if (thr == tid)
continue;
BUILD/MEDIUM: threads: rename thread_info struct to ha_thread_info On Darwin, the thread_info name exists as a standard function thus we need to rename our array to ha_thread_info to fix this conflict.
2019-09-13 04:03:12 +00:00
pthread_kill(ha_thread_info[thr].pthread, sig);
MINOR: threads: implement ha_tkill() and ha_tkillall() These functions are used respectively to signal one thread or all threads. When multithreading is disabled, it's always the current thread which is signaled.
2019-05-22 06:43:34 +00:00
}
raise(sig);
}
MINOR: thread: provide a set of lock initialisers This patch adds ha_spin_init() and ha_rwlock_init() which are used as a callback to initialise locks at boot time. They perform exactly the same as HA_SPIN_INIT() or HA_RWLOCK_INIT() but from within a real function.
2018-11-25 18:28:23 +00:00
/* these calls are used as callbacks at init time */
void ha_spin_init(HA_SPINLOCK_T *l)
{
HA_SPIN_INIT(l);
}
/* these calls are used as callbacks at init time */
void ha_rwlock_init(HA_RWLOCK_T *l)
{
HA_RWLOCK_INIT(l);
}
MAJOR: threads: enable one thread per CPU by default Threads have long matured by now, still for most users their usage is not trivial. It's about time to enable them by default on platforms where we know the number of CPUs bound. This patch does this, it counts the number of CPUs the process is bound to upon startup, and enables as many threads by default. Of course, "nbthread" still overrides this, but if it's not set the default behaviour is to start one thread per CPU. The default number of threads is reported in "haproxy -vv". Simply using "taskset -c" is now enough to adjust this number of threads so that there is no more need for playing with cpu-map. And thanks to the previous patches on the listener, the vast majority of configurations will not need to duplicate "bind" lines with the "process x/y" statement anymore either, so a simple config will automatically adapt to the number of processors available.
2019-01-26 13:27:06 +00:00
/* returns the number of CPUs the current process is enabled to run on */
static int thread_cpus_enabled()
{
int ret = 1;
#ifdef USE_CPU_AFFINITY
#if defined(__linux__) && defined(CPU_COUNT)
cpu_set_t mask;
if (sched_getaffinity(0, sizeof(mask), &mask) == 0)
ret = CPU_COUNT(&mask);
MINOR: threads: Implement thread_cpus_enabled() for FreeBSD. Use cpuset_getaffinity() to implement thread_cpus_enabled() on FreeBSD, so that we can know the number of CPUs available, and automatically launch as much threads if nbthread isn't specified.
2019-04-10 22:06:47 +00:00
#elif defined(__FreeBSD__) && defined(USE_CPU_AFFINITY)
cpuset_t cpuset;
if (cpuset_getaffinity(CPU_LEVEL_CPUSET, CPU_WHICH_PID, -1,
sizeof(cpuset), &cpuset) == 0)
ret = CPU_COUNT(&cpuset);
MAJOR: threads: enable one thread per CPU by default Threads have long matured by now, still for most users their usage is not trivial. It's about time to enable them by default on platforms where we know the number of CPUs bound. This patch does this, it counts the number of CPUs the process is bound to upon startup, and enables as many threads by default. Of course, "nbthread" still overrides this, but if it's not set the default behaviour is to start one thread per CPU. The default number of threads is reported in "haproxy -vv". Simply using "taskset -c" is now enough to adjust this number of threads so that there is no more need for playing with cpu-map. And thanks to the previous patches on the listener, the vast majority of configurations will not need to duplicate "bind" lines with the "process x/y" statement anymore either, so a simple config will automatically adapt to the number of processors available.
2019-01-26 13:27:06 +00:00
#endif
#endif
ret = MAX(ret, 1);
ret = MIN(ret, MAX_THREADS);
return ret;
}
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
__attribute__((constructor))
static void __hathreads_init(void)
{
MINOR: threads: make MAX_THREADS configurable at build time There's some value in being able to limit MAX_THREADS, either to save precious resources in embedded environments, or to protect certain deployments against accidently incorrect settings. With this patch, if MAX_THREADS is defined at build time, it will be used. However, given that LONGBITS is not a macro but is defined according to sizeof(long), we can't check the value range at build time and instead we need to perform the check at early boot time. However, the compiler is able to optimize away the constant comparisons and doesn't even emit the check code when values are correct. The output message regarding threading support was improved to report the number of threads.
2019-01-26 12:35:03 +00:00
char *ptr = NULL;
if (MAX_THREADS < 1 || MAX_THREADS > LONGBITS) {
ha_alert("MAX_THREADS value must be between 1 and %d inclusive; "
"HAProxy was built with value %d, please fix it and rebuild.\n",
LONGBITS, MAX_THREADS);
exit(1);
}
MAJOR: threads: enable one thread per CPU by default Threads have long matured by now, still for most users their usage is not trivial. It's about time to enable them by default on platforms where we know the number of CPUs bound. This patch does this, it counts the number of CPUs the process is bound to upon startup, and enables as many threads by default. Of course, "nbthread" still overrides this, but if it's not set the default behaviour is to start one thread per CPU. The default number of threads is reported in "haproxy -vv". Simply using "taskset -c" is now enough to adjust this number of threads so that there is no more need for playing with cpu-map. And thanks to the previous patches on the listener, the vast majority of configurations will not need to duplicate "bind" lines with the "process x/y" statement anymore either, so a simple config will automatically adapt to the number of processors available.
2019-01-26 13:27:06 +00:00
thread_cpus_enabled_at_boot = thread_cpus_enabled();
memprintf(&ptr, "Built with multi-threading support (MAX_THREADS=%d, default=%d).",
MAX_THREADS, thread_cpus_enabled_at_boot);
MINOR: threads: make MAX_THREADS configurable at build time There's some value in being able to limit MAX_THREADS, either to save precious resources in embedded environments, or to protect certain deployments against accidently incorrect settings. With this patch, if MAX_THREADS is defined at build time, it will be used. However, given that LONGBITS is not a macro but is defined according to sizeof(long), we can't check the value range at build time and instead we need to perform the check at early boot time. However, the compiler is able to optimize away the constant comparisons and doesn't even emit the check code when values are correct. The output message regarding threading support was improved to report the number of threads.
2019-01-26 12:35:03 +00:00
hap_register_build_opts(ptr, 1);
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
#if defined(DEBUG_THREAD) || defined(DEBUG_FULL)
memset(lock_stats, 0, sizeof(lock_stats));
#endif
}
BUILD: thread: properly report multi-thread support When refactoring the build option strings in 1.9, the thread support was placed outside of the ifdef block resulting in threads always being mentioned even if that was not true. Let's fix this and also mention when threads are disabled to help troubleshooting.
2018-12-15 15:48:14 +00:00
#else
REGISTER_BUILD_OPTS("Built without multi-threading support (USE_THREAD not set).");
MINOR: threads: move "nbthread" parsing to hathreads.c The purpose is to make sure that all variables which directly depend on this nbthread argument are set at the right moment. For now only all_threads_mask needs to be set. It used to be set while calling thread_sync_init() which is called too late for certain checks. The same function handles threads and non-threads, which removes the need for some thread-specific knowledge from cfgparse.c.
2018-07-30 08:34:35 +00:00
#endif // USE_THREAD
/* Parse the number of threads in argument <arg>, returns it and adjusts a few
* internal variables accordingly, or fails and returns zero with an error
* reason in <errmsg>. May be called multiple times while parsing.
*/
int parse_nbthread(const char *arg, char **err)
{
long nbthread;
char *errptr;
nbthread = strtol(arg, &errptr, 10);
if (!*arg || *errptr) {
memprintf(err, "passed a missing or unparsable integer value in '%s'", arg);
return 0;
}
#ifndef USE_THREAD
if (nbthread != 1) {
memprintf(err, "specified with a value other than 1 while HAProxy is not compiled with threads support. Please check build options for USE_THREAD");
return 0;
}
#else
if (nbthread < 1 || nbthread > MAX_THREADS) {
memprintf(err, "value must be between 1 and %d (was %ld)", MAX_THREADS, nbthread);
return 0;
}
CLEANUP: threads: use nbits to calculate the thread mask It's pointless to do arithmetics by hand, we have a function for this.
2019-02-02 16:05:03 +00:00
all_threads_mask = nbits(nbthread);
MEDIUM: threads: Add hathreads header file This file contains all functions and macros used to deal with concurrency in HAProxy. It contains all high-level function to do atomic operation (HA_ATOMIC_*). Note, for now, we rely on "__atomic" GCC builtins to do atomic operation. So HAProxy can be compiled with the thread support iff these builtins are available. It also contains wrappers around plocks to use spin or read/write locks. These wrappers are used to abstract the internal representation of the locking system and to add information to help debugging, when compiled with suitable options. To add extra info on locks, you need to add DEBUG=-DDEBUG_THREAD or DEBUG=-DDEBUG_FULL compilation option. In addition to timing info on locks, we keep info on where a lock was acquired the last time (function name, file and line). There are also the thread id and a flag to know if it is still locked or not. This will be useful to debug deadlocks.
2017-10-12 14:09:09 +00:00
#endif
MINOR: threads: move "nbthread" parsing to hathreads.c The purpose is to make sure that all variables which directly depend on this nbthread argument are set at the right moment. For now only all_threads_mask needs to be set. It used to be set while calling thread_sync_init() which is called too late for certain checks. The same function handles threads and non-threads, which removes the need for some thread-specific knowledge from cfgparse.c.
2018-07-30 08:34:35 +00:00
return nbthread;
}
Reference in New Issue Copy Permalink