RepoMirrors
/
haproxy
mirror of http://git.haproxy.org/git/haproxy.git/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
haproxy public development tree
10,987 Commits 20 Branches 367 Tags 199 MiB
C 98.1%
Shell 0.8%
Makefile 0.6%
Python 0.2%
Lua 0.1%
Go to file
Willy Tarreau dd0e89a084 BUG/MAJOR: task: add a new TASK_SHARED_WQ flag to fix foreing requeuing 
Since 1.9 with commit b20aa9eef3 ("MAJOR: tasks: create per-thread wait
queues") a task bound to a single thread will not use locks when being
queued or dequeued because the wait queue is assumed to be the owner
thread's.

But there exists a rare situation where this is not true: the health
check tasks may be running on one thread waiting for a response, and
may in parallel be requeued by another thread calling health_adjust()
after a detecting a response error in traffic when "observe l7" is set,
and "fastinter" is lower than "inter", requiring to shorten the running
check's timeout. In this case, the task being requeued was present in
another thread's wait queue, thus opening a race during task_unlink_wq(),
and gets requeued into the calling thread's wait queue instead of the
running one's, opening a second race here.

This patch aims at protecting against the risk of calling task_unlink_wq()
from one thread while the task is queued on another thread, hence unlocked,
by introducing a new TASK_SHARED_WQ flag.

This new flag indicates that a task's position in the wait queue may be
adjusted by other threads than then one currently executing it. This means
that such WQ manipulations must be performed under a lock. There are two
types of such tasks:
  - the global ones, using the global wait queue (technically speaking,
    those whose thread_mask has at least 2 bits set).
  - some local ones, which for now will be placed into the global wait
    queue as well in order to benefit from its lock.

The flag is automatically set on initialization if the task's thread mask
indicates more than one thread. The caller must also set it if it intends
to let other threads update the task's expiration delay (e.g. delegated
I/Os), or if it intends to change the task's affinity over time as this
could lead to the same situation.

Right now only the situation described above seems to be affected by this
issue, and it is very difficult to trigger, and even then, will often have
no visible effect beyond stopping the checks for example once the race is
met. On my laptop it is feasible with the following config, chained to
httpterm:

    global
        maxconn 400 # provoke FD errors, calling health_adjust()

    defaults
        mode http
        timeout client 10s
        timeout server 10s
        timeout connect 10s

    listen px
        bind :8001
        option httpchk /?t=50
        server sback 127.0.0.1:8000 backup
        server-template s 0-999 127.0.0.1:8000 check port 8001 inter 100 fastinter 10 observe layer7

This patch will automatically address the case for the checks because
check tasks are created with multiple threads bound and will get the
TASK_SHARED_WQ flag set.

If in the future more tasks need to rely on this (multi-threaded muxes
for example) and the use of the global wait queue becomes a bottleneck
again, then it should not be too difficult to place locks on the local
wait queues and queue the task on its bound thread.

This patch needs to be backported to 2.1, 2.0 and 1.9. It depends on
previous patch "MINOR: task: only check TASK_WOKEN_ANY to decide to
requeue a task".

Many thanks to William Dauchy for providing detailed traces allowing to
spot the problem.
 		2019-12-19 14:42:22 +01:00
.github/ISSUE_TEMPLATE DOC: Add GitHub issue config.yml 2019-11-03 15:36:06 +01:00
contrib BUG/MINOR: contrib/prometheus-exporter: decode parameter and value only 2019-11-27 11:51:35 +01:00
doc MINOR: http: add a new "replace-path" action 2019-12-19 09:24:57 +01:00
ebtree BUILD: ebtree: make eb_is_empty() and eb_is_dup() take a const 2019-10-02 15:24:19 +02:00
examples CLEANUP: removed obsolete examples an move a few to better places 2019-06-15 21:25:06 +02:00
include BUG/MAJOR: task: add a new TASK_SHARED_WQ flag to fix foreing requeuing 2019-12-19 14:42:22 +01:00
reg-tests REGTEST: run-regtests: implement #REQUIRE_BINARIES 2019-12-19 14:36:46 +01:00
scripts REGTEST: run-regtests: implement #REQUIRE_BINARIES 2019-12-19 14:36:46 +01:00
src BUG/MAJOR: task: add a new TASK_SHARED_WQ flag to fix foreing requeuing 2019-12-19 14:42:22 +01:00
tests TESTS: Add a stress-test for mt_lists. 2019-09-23 18:16:08 +02:00
.cirrus.yml BUILD: CI: comment out cygwin build, upgrade various ssl libraries 2019-10-29 06:27:50 +01:00
.gitignore DOC: create a BRANCHES file to explain the life cycle 2019-06-15 22:00:14 +02:00
.travis.yml BUILD: CI: comment out cygwin build, upgrade various ssl libraries 2019-10-29 06:27:50 +01:00
BRANCHES DOC: create a BRANCHES file to explain the life cycle 2019-06-15 22:00:14 +02:00
CHANGELOG [RELEASE] Released version 2.2-dev0 2019-11-25 20:36:16 +01:00
CONTRIBUTING DOC: improve the wording in CONTRIBUTING about how to document a bug fix 2019-07-26 15:46:21 +02:00
INSTALL DOC: this is development again 2019-11-25 20:37:49 +01:00
LICENSE
…
MAINTAINERS
…
Makefile BUILD: reorder the objects in the makefile 2019-11-25 19:47:23 +01:00
README DOC: create a BRANCHES file to explain the life cycle 2019-06-15 22:00:14 +02:00
ROADMAP DOC: update the outdated ROADMAP file 2019-06-15 21:59:54 +02:00
SUBVERS
…
VERDATE [RELEASE] Released version 2.1.0 2019-11-25 19:47:40 +01:00
VERSION [RELEASE] Released version 2.2-dev0 2019-11-25 20:36:16 +01:00

README 

The HAProxy documentation has been split into a number of different files for
ease of use.

Please refer to the following files depending on what you're looking for :

  - INSTALL for instructions on how to build and install HAProxy
  - BRANCHES to understand the project's life cycle and what version to use
  - LICENSE for the project's license
  - CONTRIBUTING for the process to follow to submit contributions

The more detailed documentation is located into the doc/ directory :

  - doc/intro.txt for a quick introduction on HAProxy
  - doc/configuration.txt for the configuration's reference manual
  - doc/lua.txt for the Lua's reference manual
  - doc/SPOE.txt for how to use the SPOE engine
  - doc/network-namespaces.txt for how to use network namespaces under Linux
  - doc/management.txt for the management guide
  - doc/regression-testing.txt for how to use the regression testing suite
  - doc/peers.txt for the peers protocol reference
  - doc/coding-style.txt for how to adopt HAProxy's coding style
  - doc/internals for developer-specific documentation (not all up to date)