haproxy/include/proto/queue.h
Christopher Faulet 5cd4bbd7ab BUG/MAJOR: threads/queue: Fix thread-safety issues on the queues management
The management of the servers and the proxies queues was not thread-safe at
all. First, the accesses to <strm>->pend_pos were not protected. So it was
possible to release it on a thread (for instance because the stream is released)
and to use it in same time on another one (because we redispatch pending
connections for a server). Then, the accesses to stream's information (flags and
target) from anywhere is forbidden. To be safe, The stream's state must always
be updated in the context of process_stream.

So to fix these issues, the queue module has been refactored. A lock has been
added in the pendconn structure. And now, when we try to dequeue a pending
connection, we start by unlinking it from the server/proxy queue and we wake up
the stream. Then, it is the stream reponsibility to really dequeue it (or
release it). This way, we are sure that only the stream can create and release
its <pend_pos> field.

However, be careful. This new implementation should be thread-safe
(hopefully...). But it is not optimal and in some situations, it could be really
slower in multi-threaded mode than in single-threaded one. The problem is that,
when we try to dequeue pending connections, we process it from the older one to
the newer one independently to the thread's affinity. So we need to wait the
other threads' wakeup to really process them. If threads are blocked in the
poller, this will add a significant latency. This problem happens when maxconn
values are very low.

This patch must be backported in 1.8.
2018-03-19 10:03:06 +01:00

70 lines
2.2 KiB
C

/*
* include/proto/queue.h
* This file defines everything related to queues.
*
* Copyright (C) 2000-2012 Willy Tarreau - w@1wt.eu
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation, version 2.1
* exclusively.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef _PROTO_QUEUE_H
#define _PROTO_QUEUE_H
#include <common/config.h>
#include <common/memory.h>
#include <common/mini-clist.h>
#include <types/proxy.h>
#include <types/queue.h>
#include <types/stream.h>
#include <types/server.h>
#include <types/task.h>
#include <proto/backend.h>
extern struct pool_head *pool_head_pendconn;
int init_pendconn();
struct pendconn *pendconn_add(struct stream *strm);
int pendconn_dequeue(struct stream *strm);
void pendconn_free(struct pendconn *p);
void process_srv_queue(struct server *s);
unsigned int srv_dynamic_maxconn(const struct server *s);
int pendconn_redistribute(struct server *s);
int pendconn_grab_from_px(struct server *s);
/* Returns 0 if all slots are full on a server, or 1 if there are slots available. */
static inline int server_has_room(const struct server *s) {
return !s->maxconn || s->cur_sess < srv_dynamic_maxconn(s);
}
/* returns 0 if nothing has to be done for server <s> regarding queued connections,
* and non-zero otherwise. If the server is down, we only check its own queue. Suited
* for and if/else usage.
*/
static inline int may_dequeue_tasks(const struct server *s, const struct proxy *p) {
return (s && (s->nbpend || (p->nbpend && srv_currently_usable(s))) &&
(!s->maxconn || s->cur_sess < srv_dynamic_maxconn(s)));
}
#endif /* _PROTO_QUEUE_H */
/*
* Local variables:
* c-indent-level: 8
* c-basic-offset: 8
* End:
*/