RepoMirrors/haproxy
1
0
Fork 0
mirror of http://git.haproxy.org/git/haproxy.git/ synced 2024-12-18 01:14:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

EXAMPLES: fix race condition in lua mailers script

Browse Source 
Christopher reported a rare race condition involving 'healthcheckmail.vtc'

The regtest would randomly FAIL with this kind of error:

    **   S1    === expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Health check for server b...
    **** S1    EXPECT MATCH ~ "[^:\[ ]\[581669\]: Health check for server be1/srv1 failed.+check duration: [[:digit:]]+ms.+status: 0/1 DOWN."
    **   S1    === recv info
    **** S1    syslog|<25>May 11 15:38:46 haproxy[581669]: Server be1/srv1 is DOWN. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
    **** S1    syslog|<24>May 11 15:38:46 haproxy[581669]: backend be1 has no server available!

It turns out that this it due to the recent commit 7963fb5 ("REGTESTS: use
lua mailer script for mailers tests") in which we tell the regtest to use
the new lua mailers instead of the legacy mailers API.

However, in the lua mailers script, due to the event subscriptions being
performed from a lua task, it is possible that the subscription may be
delayed during startup. Indeed lua tasks relie on the scheduler which runs
tasks with no ordering guarantees. Thus early tasks, including server
checks which are used in the regtest are competing during startup.

As such, we may end up with some events that are generated right before
the lua mailers script starts subscribing to events (because the lua task
is scheduled but started yet), resulting in events loss from lua point of
view.

To fix this and to make lua mailers more reliable during startup, we now
perform the events subscription from an init function instead of an
asynchronous task. (The init function is called synchronously during
haproxy post_init, and exclusively runs before the scheduler starts)

This should be enough to prevent healthcheckmail.vtc from randomly failing
This commit is contained in:
Aurelien DARRAGON 2023-05-11 17:46:42 +02:00 committed by Christopher Faulet
parent 2a29571c11
commit 4cc2714ae2
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
examples/lua/mailers.lua
View File

@ -362,9 +362,9 @@ end
-- disable legacy email-alerts since email-alerts will be sent from lua directly
core.disable_legacy_mailers()
-- event subscription are purposely performed in a task to prevent email
-- alerts from being generated too early (when process is starting up)
core.register_task(function()
-- event subscriptions are purposely performed in an init function to prevent
-- email alerts from being generated too early (when process is starting up)
core.register_init(function()
-- do nothing if not on primary thread
-- this prevents emails from being sent multiple times when