mirror of
http://git.haproxy.org/git/haproxy.git/
synced 2024-12-24 21:52:17 +00:00
f673923629
With the CI occasionally slowing down, we're starting to see again some spurious failures despite the long 1-second timeouts. This reports false positives that are disturbing and doesn't provide as much value as this could. However at this delay it already becomes a pain for developers to wait for the tests to complete. This commit adds support for the new environment variable HAPROXY_TEST_TIMEOUT that will allow anyone to modify the connect, client and server timeouts. It was set to 5 seconds by default, which should be plenty for quite some time in the CI. All relevant values that were 200ms or above were replaced by this one. A few larger values were left as they are special. One test for the set-timeout action that used to rely on a fixed 1-sec value was extended to a fixed 5-sec, as the timeout is normally not reached, but it needs to be known to compare the old and new values.
72 lines
2.8 KiB
Plaintext
72 lines
2.8 KiB
Plaintext
* Regression testing for HAProxy with VTest *
|
|
|
|
|
|
This little README file is about how to compile and run vtest test case files (VTC files)
|
|
to test HAProxy for any regression.
|
|
|
|
To do so, you will have to compile vtest program sources which depends on
|
|
Varnish cache application sources. vtest, formerly varnishtest, is a very useful
|
|
program which has been developed to test Varnish cache application. vtest has been
|
|
modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support
|
|
HAProxy in addition to Varnish cache.
|
|
|
|
See also: doc/regression-testing.txt
|
|
|
|
* vtest compilation *
|
|
|
|
$ git clone https://github.com/vtest/VTest
|
|
|
|
$ cd VTest
|
|
|
|
$ make vtest
|
|
|
|
Then vtest program may be found at the root directory of vtest sources directory.
|
|
The Varnish cache manuals are located in 'man' directory of Varnish cache sources
|
|
directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to
|
|
use vtest.
|
|
|
|
Some information may also be found in doc/regression-testing.txt in HAProxy
|
|
sources.
|
|
|
|
Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory
|
|
of Varnish cache sources directory which may be found here:
|
|
https://github.com/varnishcache/varnish-cache
|
|
|
|
|
|
* vtest execution *
|
|
|
|
You must set HAPROXY_PROGRAM environment variable to give the location
|
|
of the HAProxy program to test to vtest:
|
|
|
|
$ HAPROXY_PROGRAM=<my haproxy program> vtest ...
|
|
|
|
The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
|
|
Makefile target. You must set the VTEST_PROGRAM environment variable to
|
|
give the location of the vtest program which has been previously compiled.
|
|
|
|
$ VTEST_PROGRAM=<my vtest program> make reg-tests
|
|
|
|
"reg-tests" Makefile target run scripts/run-regtest.sh script.
|
|
To get more information about this script run it with --help option.
|
|
|
|
Note that vtest is run with -t10 and -l option. -l option is to keep
|
|
keep vtest temporary directory in case of failed test cases. core files
|
|
may be found in this directory (if enabled by ulimit).
|
|
|
|
|
|
* vtest patches for HAProxy VTC files *
|
|
|
|
When producing a patch to add a VTC regression testing file to reg-tests directory,
|
|
please follow these simple rules:
|
|
|
|
- If your VTC file needs others files, if possible, use the same basename as that
|
|
of the VTC file,
|
|
- Put these files in a directory with the same name as the code area concerned
|
|
by the bug ('peers', 'lua', 'acl' etc).
|
|
|
|
Please note that most tests use a common set of timeouts defined by the
|
|
environment variable HAPROXY_TEST_TIMEOUT. As much as possible, for regular I/O
|
|
(i.e. not errors), please try to reuse that setting so that the value may
|
|
easily be adjusted when running in some particularly slow environments, or be
|
|
shortened to fail faster on developers' machines.
|