Commit Graph

23 Commits

Author SHA1 Message Date
Willy Tarreau
967de20a43 BUG/MEDIUM: list: fix again LIST_ADDQ_LOCKED
Well, that's becoming embarrassing. Now this fixes commit 4ef6801c
("BUG/MEDIUM: list: correct fix for LIST_POP_LOCKED's removal of last
element") which itself tried to fix commit 285192564. This fix only
works under low contention and was tested with the listener's queue.
With the idle conns it's obvious that it's still wrong since adding
more than one element to the list leaves a LLIST_BUSY pointer into
the list's head. This was visible when accumulating idle connections
in a server's list.

This new version of the fix almost goes back to the original code,
except that since then we addressed issues with expectedly idempotent
operations that were not. Now the code has been verified on paper again
and has survived 300 million connections spread over 4 threads.

This will have to be backported if the commit above is backported.
2019-03-04 14:09:22 +01:00
Willy Tarreau
4ef6801cd4 BUG/MEDIUM: list: correct fix for LIST_POP_LOCKED's removal of last element
As seen with Olivier, in the end the fix in commit 285192564 ("BUG/MEDIUM:
list: fix LIST_POP_LOCKED's removal of the last pointer") is wrong,
the code there was right but the bug was triggered by another bug in
LIST_ADDQ_LOCKED() which doesn't properly update the list's head by
inserting in the wrong order.

This will have to be backported if the commit above is backported.
2019-02-28 16:51:28 +01:00
Willy Tarreau
4c747e86cd MINOR: list: make the delete and pop operations idempotent
These operations previously used to return a "locked" element, which is
a constraint when multiple threads try to delete the same element, because
the second one will block indefinitely. Instead, let's make sure that both
LIST_DEL_LOCKED() and LIST_POP_LOCKED() always reinitialize the element
after deleting it. This ensures that the second thread will immediately
unblock and succeed with the removal. It also secures the pop vs delete
competition that may happen when trying to remove an element that's about
to be dequeued.
2019-02-28 16:03:29 +01:00
Willy Tarreau
690d2ad4d2 BUG/MEDIUM: list: add missing store barriers when updating elements and head
Commit a8434ec14 ("MINOR: lists: Implement locked variations.")
introduced locked lists which use the elements pointers as locks
for concurrent operations. Under heavy stress the lists occasionally
fail. The cause is a missing barrier at some points when updating
the list element and the head : nothing prevents the compiler (or
CPU) from updating the list head first before updating the element,
making another thread jump to a wrong location. This patch simply
adds the missing barriers before these two opeations.

This will have to be backported if the commit above is backported.
2019-02-28 15:59:31 +01:00
Willy Tarreau
285192564d BUG/MEDIUM: list: fix LIST_POP_LOCKED's removal of the last pointer
There was a typo making the last updated pointer be the pre-last element's
prev instead of the last's prev element. It didn't show up during early
tests because the contention is very rare on this one  and it's implicitly
recovered when updating the pointers to go to the next element, but it was
clearly visible in the listener_accept() tests by having all threads block
on LIST_POP_LOCKED() with n==p==LLIST_BUSY.

This will have to be backported if commit a8434ec14 ("MINOR: lists:
Implement locked variations.") is backported.
2019-02-28 15:59:31 +01:00
Willy Tarreau
bd20ad5874 BUG/MEDIUM: list: fix the rollback on addq in the locked liss
Commit a8434ec14 ("MINOR: lists: Implement locked variations.")
introduced locked lists which use the elements pointers as locks
for concurrent operations. A copy-paste typo in LIST_ADDQ_LOCKED()
causes corruption in the list in case the next pointer is already
held, as it restores the previous pointer into the next one. It
may impact the server pools.

This will have to be backported if the commit above is backported.
2019-02-28 15:10:15 +01:00
Olivier Houchard
db64489aac BUG/MEDIUM: lists: Properly handle the case we're removing the first elt.
In LIST_DEL_LOCKED(), initialize p2 to NULL, and only attempt to set it back
to its previous value if we had a previous element, and thus p2 is non-NULL.
2019-02-26 18:47:59 +01:00
Olivier Houchard
a8434ec146 MINOR: lists: Implement locked variations.
Implement LIST_ADD_LOCKED(), LIST_ADDQ_LOCKED(), LIST_DEL_LOCKED() and
LIST_POP_LOCKED().

LIST_ADD_LOCKED, LIST_ADDQ_LOCKED and LIST_DEL_LOCKED work the same as
LIST_ADD, LIST_ADDQ and LIST_DEL, except before any manipulation it locks
the relevant elements of the list, so it's safe to manipulate the list
with multiple threads.
LIST_POP_LOCKED() removes the first element from the list, and returns its
data.
2019-02-26 18:17:32 +01:00
Joseph Herlant
41abef77cb CLEANUP: Fix a typo in the mini-clist header
Fixes a typo in the code comments of the mini-clist header.
2018-12-02 18:38:15 +01:00
William Lallemand
83215a44b8 MEDIUM: lists: list_for_each_entry{_safe}_from functions
Add list_for_each_entry_from and list_for_each_entry_safe_from which
allows to iterate in a list starting from a specific item.
2017-10-31 03:44:11 +01:00
Thierry FOURNIER
1db96672c4 BUILD: freebsd: double declaration
On freebsd, the macro LIST_PREV already exists in the header file
<sys/queue.h>, and this makes a build error.

This patch removes the macros before declaring it. This ensure
that the error doesn't occurs.
2015-11-06 01:15:02 +01:00
Willy Tarreau
3dd717cd5d CLEANUP: lists: remove dead code
Remove the code dealing with the old dual-linked lists imported from
librt that has remained unused for the last 8 years. Now everything
uses the linux-like circular lists instead.
2014-12-24 23:47:31 +01:00
Willy Tarreau
dc13c11c1e BUG/MEDIUM: prevent gcc from moving empty keywords lists into BSS
Benoit Dolez reported a failure to start haproxy 1.5-dev19. The
process would immediately report an internal error with missing
fetches from some crap instead of ACL names.

The cause is that some versions of gcc seem to trim static structs
containing a variable array when moving them to BSS, and only keep
the fixed size, which is just a list head for all ACL and sample
fetch keywords. This was confirmed at least with gcc 3.4.6. And we
can't move these structs to const because they contain a list element
which is needed to link all of them together during the parsing.

The bug indeed appeared with 1.5-dev19 because it's the first one
to have some empty ACL keyword lists.

One solution is to impose -fno-zero-initialized-in-bss to everyone
but this is not really nice. Another solution consists in ensuring
the struct is never empty so that it does not move there. The easy
solution consists in having a non-null list head since it's not yet
initialized.

A new "ILH" list head type was thus created for this purpose : create
an Initialized List Head so that gcc cannot move the struct to BSS.
This fixes the issue for this version of gcc and does not create any
burden for the declarations.
2013-06-21 23:29:02 +02:00
Willy Tarreau
f4f04125d4 [MINOR] prepare req_*/rsp_* to receive a condition
It will be very handy to be able to pass conditions to req_* and rsp_*.
For now, we just add the pointer to the condition in the affected
structs.
2010-01-28 18:10:50 +01:00
Willy Tarreau
deb9ed8f60 [MEDIUM] config: remove the limitation of 10 reqadd/rspadd statements
Now we use a linked list, there is no limit anymore.
2010-01-03 21:22:14 +01:00
Willy Tarreau
bc04ce7cd9 [MINOR] add a new back-reference type : struct bref
This type will be used to maintain back-references to items which
are subject to move between accesses. Typical usage includes session
removal during a listing.
2008-12-07 20:00:15 +01:00
Willy Tarreau
47d940485a [OPTIM] add branch prediction hints in list manipulations
GCC does not do very clever things with the ifs in list manipulation
macros. Adding a 'likely' helps it remove expensive jumps and useless
code.
2008-06-23 22:39:37 +02:00
Willy Tarreau
bd578bbe1a [CLEANUP] silent warning about LIST_* being redefined on OpenBSD
Building ev_kqueue on OpenBSD causes some warnings to occur,
because OpenBSD also uses LIST_* macros in sys/queue.h, included
from sys/event.h. Simply undefine those macros since we don't
need them.
2007-10-28 11:41:06 +01:00
Willy Tarreau
2b1dccd501 [MINOR] add LIST_HEAD_INIT to the list management macros
It is useful to have the ability to initialize a list while declaring it.
2007-05-08 14:07:44 +02:00
Willy Tarreau
b9c62b90da [MEDIUM] add better list management macros
The FOREACH_ITEM and FOREACH_ITEM_SAFE macros are nasty, they
require too many arguments. Let's port those from linux instead.
2007-05-08 14:07:39 +02:00
Willy Tarreau
40cf67da89 [MINOR] added support for dual-linked lists
Imported dual-linked lists management macros from the old scheduler project,
and fixed a long-standing nasty bug in the DLIST_ADD() macro.
2007-04-29 13:40:50 +02:00
Willy Tarreau
e3ba5f0aaa [CLEANUP] included common/version.h everywhere 2006-06-29 18:54:54 +02:00
Willy Tarreau
2dd0d4799e [CLEANUP] renamed include/haproxy to include/common 2006-06-29 17:53:05 +02:00