mirror of
http://git.haproxy.org/git/haproxy.git/
synced 2024-12-15 07:54:33 +00:00
2ea15a0804
This test is inherently racy. It regularly pops up on the CI, and I've spent one hour chasing a bug that apparently doesn't exist, just because I'm running it 10 times in a row and it reports from 4 to 8 failures when built at -O2 and generally even more at -O0. The logs are very confusing, often reporting that it failed with status 0, with nothing else wrong. I suspect it might sometimes be the shell command that fails if it executes faster than haproxy finishes to start up, which would also explain the relation with the optimization level. E.g: > Testing with haproxy version: 2.2.0 > # top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (3.006) exit=2 > # top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (3.006) exit=2 > # top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (3.009) exit=2 > # top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (3.008) exit=2 > # top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (3.007) exit=2 > # top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (3.007) exit=2 > 6 tests failed, 0 tests skipped, 4 tests passed Some of the failures include this, suggesting that some barriers could help: ---- h1 haproxy h1 PID file check failed: Could not read PID file '/tmp/haregtests-2020-10-09_11-19-40.kgsDB4/vtc.30539.04dbea7f/h1/pid Since it has been causing false positives and consumed way more troubleshooting time than it saved, let's mark it as broken so that it doesn't waste more time. We can bring it back when someone manages to figure what the problem is. |
||
---|---|---|
.. | ||
balance | ||
cache | ||
checks | ||
compression | ||
connection | ||
converter | ||
http-capture | ||
http-cookies | ||
http-errorfiles | ||
http-messaging | ||
http-rules | ||
log | ||
lua | ||
mailers | ||
mcli | ||
peers | ||
sample_fetches | ||
seamless-reload | ||
server | ||
spoe | ||
ssl | ||
stick-table | ||
stickiness | ||
stream | ||
webstats | ||
README |
* Regression testing for HAProxy with VTest * This little README file is about how to compile and run vtest test case files (VTC files) to test HAProxy for any regression. To do so, you will have to compile vtest program sources which depends on Varnish cache application sources. vtest, formerly varnishtest, is a very useful program which has been developed to test Varnish cache application. vtest has been modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support HAProxy in addition to Varnish cache. See also: doc/regression-testing.txt * vtest compilation * $ git clone https://github.com/vtest/VTest $ cd VTest $ make vtest Then vtest program may be found at the root directory of vtest sources directory. The Varnish cache manuals are located in 'man' directory of Varnish cache sources directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to use vtest. Some information may also be found in doc/regression-testing.txt in HAProxy sources. Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory of Varnish cache sources directory which may be found here: https://github.com/varnishcache/varnish-cache * vtest execution * You must set HAPROXY_PROGRAM environment variable to give the location of the HAProxy program to test to vtest: $ HAPROXY_PROGRAM=<my haproxy program> vtest [-Dno-htx=] ... The HAProxy VTC files found in HAProxy sources may be run with the reg-tests Makefile target. You must set the VTEST_PROGRAM environment variable to give the location of the vtest program which has been previously compiled. $ VTEST_PROGRAM=<my vtest program> make reg-tests "reg-tests" Makefile target run scripts/run-regtest.sh script. To get more information about this script run it with --help option. Note that vtest is run with -t10 and -l option. -l option is to keep keep vtest temporary directory in case of failed test cases. core files may be found in this directory (if enabled by ulimit). * vtest patches for HAProxy VTC files * When producing a patch to add a VTC regression testing file to reg-tests directory, please follow these simple rules: - If your VTC file needs others files, if possible, use the same basename as that of the VTC file, - Put these files in a directory with the same name as the code area concerned by the bug ('peers', 'lua', 'acl' etc).