RepoMirrors
/
haproxy
mirror of http://git.haproxy.org/git/haproxy.git/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
haproxy/include/proto/applet.h

127 lines
3.7 KiB
C
Raw Normal View History

REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
/*
* include/proto/applet.h
* This file contains applet function prototypes
*
* Copyright (C) 2000-2015 Willy Tarreau - w@1wt.eu
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef _PROTO_APPLET_H
#define _PROTO_APPLET_H
#include <stdlib.h>
#include <common/config.h>
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
#include <common/memory.h>
MEDIUM: applet: add basic support for an applet run queue This will be needed so that we can schedule applets out of the streams. For now nothing calls the queue yet.
2015-04-13 15:11:11 +00:00
#include <common/mini-clist.h>
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
#include <types/applet.h>
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
#include <proto/task.h>
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
MINOR: applet: Count number of (active) applets As for tasks, 2 counters has been added to track : * the total number of applets : nb_applets * the number of active applets : applets_active_queue [wt: needed for next fixes, to backport to 1.7 and 1.6]
2016-12-06 08:13:22 +00:00
extern unsigned int nb_applets;
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
extern struct pool_head *pool_head_appctx;
MEDIUM: applet: add basic support for an applet run queue This will be needed so that we can schedule applets out of the streams. For now nothing calls the queue yet.
2015-04-13 15:11:11 +00:00
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
struct task *task_run_applet(struct task *t, void *context, unsigned short state);
MEDIUM: applet: implement a run queue for active appctx The new function is called for each round of polling in order to call any active appctx. For now we pick the stream interface from the appctx's owner. At the moment there's no appctx queued yet, but we have everything needed to queue them and remove them.
2015-04-19 07:59:31 +00:00
MEDIUM: appctx: check for allocation attempts in buffer allocation callbacks The buffer allocation callback appctx_res_wakeup() used to rely on old tricks to detect if a buffer was already granted to an appctx, namely by checking the task's state. Not only this test is not valid anymore, but it's inaccurate. Let's solely on SI_FL_WAIT_ROOM that is now set on allocation failure by the functions trying to allocate a buffer. The buffer is now allocated on the fly and the flag removed so that the consistency between the two remains granted. The patch also fixes minor issues such as the function being improperly declared inline(!) and the fact that using appctx_wakeup() sets the wakeup reason to TASK_WOKEN_OTHER while we try to use TASK_WOKEN_RES when waking up consecutive to a ressource allocation such as a buffer.
2018-11-06 16:32:37 +00:00
int appctx_buf_available(void *arg);
BUG/MAJOR: Fix how the list of entities waiting for a buffer is handled When an entity tries to get a buffer, if it cannot be allocted, for example because the number of buffers which may be allocated per process is limited, this entity is added in a list (called <buffer_wq>) and wait for an available buffer. Historically, the <buffer_wq> list was logically attached to streams because it were the only entities likely to be added in it. Now, applets can also be waiting for a free buffer. And with filters, we could imagine to have more other entities waiting for a buffer. So it make sense to have a generic list. Anyway, with the current design there is a bug. When an applet failed to get a buffer, it will wait. But we add the stream attached to the applet in <buffer_wq>, instead of the applet itself. So when a buffer is available, we wake up the stream and not the waiting applet. So, it is possible to have waiting applets and never awakened. So, now, <buffer_wq> is independant from streams. And we really add the waiting entity in <buffer_wq>. To be generic, the entity is responsible to define the callback used to awaken it. In addition, applets will still request an input buffer when they become active. But they will not be sleeped anymore if no buffer are available. So this is the responsibility to the applet I/O handler to check if this buffer is allocated or not. This way, an applet can decide if this buffer is required or not and can do additional processing if not. [wt: backport to 1.7 and 1.6]
2016-12-09 16:30:18 +00:00
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
/* Initializes all required fields for a new appctx. Note that it does the
* minimum acceptable initialization for an appctx. This means only the
MEDIUM: cli: Add payload support In order to use arbitrary data in the CLI (multiple lines or group of words that must be considered as a whole, for example), it is now possible to add a payload to the commands. To do so, the first line needs to end with a special pattern: <<\n. Everything that follows will be left untouched by the CLI parser and will be passed to the commands parsers. Per-command support will need to be added to take advantage of this feature. Signed-off-by: Aurlien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-18 11:26:46 +00:00
* 3 integer states st0, st1, st2 and the chunk used to gather unfinished
* commands are zeroed
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
*/
MAJOR: threads/applet: Handle multithreading for applets A global lock has been added to protect accesses to the list of active applets. A process mask has also been added on each applet. Like for FDs and tasks, it is used to know which threads are allowed to process an applet. Because applets are, most of time, linked to a session, it should be sticky on the same thread. But in all cases, it is the responsibility of the applet handler to lock what have to be protected in the applet context.
2017-06-19 10:38:55 +00:00
static inline void appctx_init(struct appctx *appctx, unsigned long thread_mask)
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
{
appctx->st0 = appctx->st1 = appctx->st2 = 0;
MEDIUM: cli: Add payload support In order to use arbitrary data in the CLI (multiple lines or group of words that must be considered as a whole, for example), it is now possible to add a payload to the commands. To do so, the first line needs to end with a special pattern: <<\n. Everything that follows will be left untouched by the CLI parser and will be passed to the commands parsers. Per-command support will need to be added to take advantage of this feature. Signed-off-by: Aurlien Nephtali <aurelien.nephtali@corp.ovh.com>
2018-04-18 11:26:46 +00:00
appctx->chunk = NULL;
MINOR: cli: add private pointer and release function This pointer will be used for storing private context. With this, the same executed function can handle more than one keyword. This will be very useful for creation Lua cli bindings. The release function is called when the command is terminated (give back the hand to the prompt) or when the session is broken (timeout or client closed).
2016-11-12 09:51:33 +00:00
appctx->io_release = NULL;
CLEANUP: threads: rename process_mask to thread_mask It was a leftover from the last cleaning session; this mask applies to threads and calling it process_mask is a bit confusing. It's the same in fd, task and applets.
2017-10-31 15:06:06 +00:00
appctx->thread_mask = thread_mask;
MINOR: applet: measure and report an appctx's call rate in "show sess" Very similarly to previous commit doing the same for streams, we now measure and report an appctx's call rate. This will help catch applets which do not consume all their data and/or which do not properly report that they're waiting for something else. Some of them like peers might theorically be able to exhibit some occasional peeks when teaching a full table to a nearby peer (e.g. the new replacement process), but nothing close to what a bogus service can do so there is no risk of confusion.
2019-04-24 06:41:29 +00:00
appctx->call_rate.curr_sec = 0;
appctx->call_rate.curr_ctr = 0;
appctx->call_rate.prev_ctr = 0;
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
appctx->state = 0;
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
}
/* Tries to allocate a new appctx and initialize its main fields. The appctx
* is returned on success, NULL on failure. The appctx must be released using
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
* appctx_free(). <applet> is assigned as the applet, but it can be NULL.
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
*/
MAJOR: threads/applet: Handle multithreading for applets A global lock has been added to protect accesses to the list of active applets. A process mask has also been added on each applet. Like for FDs and tasks, it is used to know which threads are allowed to process an applet. Because applets are, most of time, linked to a session, it should be sticky on the same thread. But in all cases, it is the responsibility of the applet handler to lock what have to be protected in the applet context.
2017-06-19 10:38:55 +00:00
static inline struct appctx *appctx_new(struct applet *applet, unsigned long thread_mask)
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
{
struct appctx *appctx;
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
appctx = pool_alloc(pool_head_appctx);
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
if (likely(appctx != NULL)) {
appctx->obj_type = OBJ_TYPE_APPCTX;
appctx->applet = applet;
MAJOR: threads/applet: Handle multithreading for applets A global lock has been added to protect accesses to the list of active applets. A process mask has also been added on each applet. Like for FDs and tasks, it is used to know which threads are allowed to process an applet. Because applets are, most of time, linked to a session, it should be sticky on the same thread. But in all cases, it is the responsibility of the applet handler to lock what have to be protected in the applet context.
2017-06-19 10:38:55 +00:00
appctx_init(appctx, thread_mask);
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
appctx->t = task_new(thread_mask);
if (unlikely(appctx->t == NULL)) {
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
pool_free(pool_head_appctx, appctx);
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
return NULL;
}
appctx->t->process = task_run_applet;
appctx->t->context = appctx;
MEDIUM: buffer: remove the buffer_wq lock This lock was only needed to protect the buffer_wq list, but now we have the mt_list for this. This patch simply turns the buffer_wq list to an mt_list and gets rid of the lock. It's worth noting that the whole buffer_wait thing still looks totally wrong especially in a threaded context: the wakeup_cb() callback is called synchronously from any thread and may end up calling some connection code that was not expected to run on a given thread. The whole thing should probably be reworked to use tasklets instead and be a bit more centralized.
2020-02-26 09:39:36 +00:00
MT_LIST_INIT(&appctx->buffer_wait.list);
BUG/MAJOR: Fix how the list of entities waiting for a buffer is handled When an entity tries to get a buffer, if it cannot be allocted, for example because the number of buffers which may be allocated per process is limited, this entity is added in a list (called <buffer_wq>) and wait for an available buffer. Historically, the <buffer_wq> list was logically attached to streams because it were the only entities likely to be added in it. Now, applets can also be waiting for a free buffer. And with filters, we could imagine to have more other entities waiting for a buffer. So it make sense to have a generic list. Anyway, with the current design there is a bug. When an applet failed to get a buffer, it will wait. But we add the stream attached to the applet in <buffer_wq>, instead of the applet itself. So when a buffer is available, we wake up the stream and not the waiting applet. So, it is possible to have waiting applets and never awakened. So, now, <buffer_wq> is independant from streams. And we really add the waiting entity in <buffer_wq>. To be generic, the entity is responsible to define the callback used to awaken it. In addition, applets will still request an input buffer when they become active. But they will not be sleeped anymore if no buffer are available. So this is the responsibility to the applet I/O handler to check if this buffer is allocated or not. This way, an applet can decide if this buffer is required or not and can do additional processing if not. [wt: backport to 1.7 and 1.6]
2016-12-09 16:30:18 +00:00
appctx->buffer_wait.target = appctx;
MEDIUM: appctx: check for allocation attempts in buffer allocation callbacks The buffer allocation callback appctx_res_wakeup() used to rely on old tricks to detect if a buffer was already granted to an appctx, namely by checking the task's state. Not only this test is not valid anymore, but it's inaccurate. Let's solely on SI_FL_WAIT_ROOM that is now set on allocation failure by the functions trying to allocate a buffer. The buffer is now allocated on the fly and the flag removed so that the consistency between the two remains granted. The patch also fixes minor issues such as the function being improperly declared inline(!) and the fact that using appctx_wakeup() sets the wakeup reason to TASK_WOKEN_OTHER while we try to use TASK_WOKEN_RES when waking up consecutive to a ressource allocation such as a buffer.
2018-11-06 16:32:37 +00:00
appctx->buffer_wait.wakeup_cb = appctx_buf_available;
MEDIUM: applets: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:46:48 +00:00
_HA_ATOMIC_ADD(&nb_applets, 1);
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
}
return appctx;
}
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
/* Releases an appctx previously allocated by appctx_new(). */
MAJOR: applet: applet scheduler rework. In order to authorize call of appctx_wakeup on running task: - from within the task handler itself. - in futur, from another thread. The appctx is considered paused as default after running the handler. The handler should explicitly call appctx_wakeup to be re-called. When the appctx_free is called on a running handler. The real free is postponed at the end of the handler process.
2017-06-26 14:36:53 +00:00
static inline void __appctx_free(struct appctx *appctx)
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
{
MEDIUM: tasks: Merge task_delete() and task_free() into task_destroy(). task_delete() was never used without calling task_free() just after, and task_free() was only used on error pathes to destroy a just-created task, so merge them into task_destroy(), that will remove the task from the wait queue, and make sure the task is either destroyed immediately if it's not in the run queue, or destroyed when it's supposed to run.
2019-04-17 20:51:06 +00:00
task_destroy(appctx->t);
MEDIUM: buffer: remove the buffer_wq lock This lock was only needed to protect the buffer_wq list, but now we have the mt_list for this. This patch simply turns the buffer_wq list to an mt_list and gets rid of the lock. It's worth noting that the whole buffer_wait thing still looks totally wrong especially in a threaded context: the wakeup_cb() callback is called synchronously from any thread and may end up calling some connection code that was not expected to run on a given thread. The whole thing should probably be reworked to use tasklets instead and be a bit more centralized.
2020-02-26 09:39:36 +00:00
if (MT_LIST_ADDED(&appctx->buffer_wait.list))
MT_LIST_DEL(&appctx->buffer_wait.list);
MAJOR: threads/applet: Handle multithreading for applets A global lock has been added to protect accesses to the list of active applets. A process mask has also been added on each applet. Like for FDs and tasks, it is used to know which threads are allowed to process an applet. Because applets are, most of time, linked to a session, it should be sticky on the same thread. But in all cases, it is the responsibility of the applet handler to lock what have to be protected in the applet context.
2017-06-19 10:38:55 +00:00
MINOR: applet: make appctx use their own pool A long time ago, applets were seen as an alternative to connections, and since their respective sizes were roughly equal it appeared wise to share the same pool. Nowadays, connections got significantly larger but applets are not that often used, except for the cache. However applets are mostly complementary and not alternatives anymore, as it's very possible not to have a back connection or to share one with other streams. The connections will soon lose their addresses and their size will shrink so much that appctx won't fit anymore. Given that the old benefits of sharing these pools have long disappeared, let's stop doing this and have a dedicated pool for appctx.
2019-07-18 08:41:36 +00:00
pool_free(pool_head_appctx, appctx);
MEDIUM: applets: Use the new _HA_ATOMIC_* macros. Use the new _HA_ATOMIC_* macros and add barriers where needed.
2019-03-08 17:46:48 +00:00
_HA_ATOMIC_SUB(&nb_applets, 1);
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
}
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
MAJOR: applet: applet scheduler rework. In order to authorize call of appctx_wakeup on running task: - from within the task handler itself. - in futur, from another thread. The appctx is considered paused as default after running the handler. The handler should explicitly call appctx_wakeup to be re-called. When the appctx_free is called on a running handler. The real free is postponed at the end of the handler process.
2017-06-26 14:36:53 +00:00
static inline void appctx_free(struct appctx *appctx)
{
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
/* The task is supposed to be run on this thread, so we can just
* check if it's running already (or about to run) or not
*/
BUG/MAJOR: task: make sure never to delete a queued task Commit 0c7a4b6 ("MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet list.") revealed a hole in the way tasks may be freed : they could be removed while in the run queue when the TASK_QUEUED flag was present but not the TASK_RUNNING one. But it seems the issue was emphasized by commit cde7902 ("MEDIUM: tasks: improve fairness between the local and global queues") though the code it replaces was already affected given how late the TASK_RUNNING flag was set after removal from the global queue. At the moment the task is picked from the global run queue, if it is the last one, the global run queue lock is dropped, and then the TASK_RUNNING flag was added. In the mean time another thread might have performed a task_free(), and immediately after, the TASK_RUNNING flag was re-added to the task, which was then added to the tasklet list. The unprotected window was extremely faint but does definitely exist and inconsistent task lists have been observed a few times during very intensive tests over the last few days. From this point various options are possible, the task might have been re-allocated while running, and assigned state 0 and/or state QUEUED while it was still running, resulting in the tast not being put back into the tree. This commit simply makes sure that tests on TASK_RUNNING before removing the task also cover TASK_QUEUED. It must be backported to 1.9 along with the previous ones touching that area.
2019-04-17 19:58:23 +00:00
if (!(appctx->t->state & (TASK_QUEUED | TASK_RUNNING)))
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
__appctx_free(appctx);
else {
/* if it's running, or about to run, defer the freeing
* until the callback is called.
*/
MAJOR: applet: applet scheduler rework. In order to authorize call of appctx_wakeup on running task: - from within the task handler itself. - in futur, from another thread. The appctx is considered paused as default after running the handler. The handler should explicitly call appctx_wakeup to be re-called. When the appctx_free is called on a running handler. The real free is postponed at the end of the handler process.
2017-06-26 14:36:53 +00:00
appctx->state |= APPLET_WANT_DIE;
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
task_wakeup(appctx->t, TASK_WOKEN_OTHER);
MAJOR: applet: applet scheduler rework. In order to authorize call of appctx_wakeup on running task: - from within the task handler itself. - in futur, from another thread. The appctx is considered paused as default after running the handler. The handler should explicitly call appctx_wakeup to be re-called. When the appctx_free is called on a running handler. The real free is postponed at the end of the handler process.
2017-06-26 14:36:53 +00:00
}
}
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
MEDIUM: applet: add basic support for an applet run queue This will be needed so that we can schedule applets out of the streams. For now nothing calls the queue yet.
2015-04-13 15:11:11 +00:00
/* wakes up an applet when conditions have changed */
MAJOR: applet: applet scheduler rework. In order to authorize call of appctx_wakeup on running task: - from within the task handler itself. - in futur, from another thread. The appctx is considered paused as default after running the handler. The handler should explicitly call appctx_wakeup to be re-called. When the appctx_free is called on a running handler. The real free is postponed at the end of the handler process.
2017-06-26 14:36:53 +00:00
static inline void appctx_wakeup(struct appctx *appctx)
MEDIUM: applet: implement a run queue for active appctx The new function is called for each round of polling in order to call any active appctx. For now we pick the stream interface from the appctx's owner. At the moment there's no appctx queued yet, but we have everything needed to queue them and remove them.
2015-04-19 07:59:31 +00:00
{
MAJOR: applets: Use tasks, instead of rolling our own scheduler. There's no real reason to have a specific scheduler for applets anymore, so nuke it and just use tasks. This comes with some benefits, the first one being that applets cannot induce high latencies anymore since they share nice values with other tasks. Later it will be possible to configure the applets' nice value. The second benefit is that the applet scheduler was not very thread-friendly, having a big lock around it in prevision of this change. Thus applet-intensive workloads should now scale much better with threads. Some more improvement is possible now : some applets also use a task to handle timers and timeouts. These ones could now be simplified to use only one task.
2018-05-25 14:58:52 +00:00
task_wakeup(appctx->t, TASK_WOKEN_OTHER);
MEDIUM: applet: implement a run queue for active appctx The new function is called for each round of polling in order to call any active appctx. For now we pick the stream interface from the appctx's owner. At the moment there's no appctx queued yet, but we have everything needed to queue them and remove them.
2015-04-19 07:59:31 +00:00
}
REORG: applet: move the applet definitions out of stream_interface We're tidying the definitions so that appctx lives on its own. A new set of applet.h files has been added for this purpose.
2015-04-13 11:24:54 +00:00
#endif /* _PROTO_APPLET_H */
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/