It is now possible to specify a list of "alternatives" for a required
option. This must be done by separating options by a pipe ('|'). A test will be
executed if at least one of them is available. For instance:
#REQUIRED_OPTIONS=ZLIB|SLZ,LUA,OPENSSL
The function _findtest() has also been sligthly simplified.
A check is done on HAProxy version to be sure it is supported. Internally, by
default, the macro 'no-htx' is set to "#". So in regtests, we can prefix
http-use-htx option line with this macro to inhibit the HTX. Concretly, this
line can be added in the HAProxy configuration:
${no-htx} option http-use-htx
When the option --use-htx is used, the macro 'no-htx' is unset.
By default a log directory is kept only if the test fails. With this option it
is possible to always keep it. If this option is used, the result of all tests
are displayed (and not only the failing ones).
Running regtests several times leaves many direcotries. It is painful to remove
them by hand. This command do it for you. It ask a confirmation to proceed to be
sure. The template used to create log direcotries has been changed. The prefix
'haregtests-' has been added to help the cleanup function to find existing log
direcotries.
Instead of directly executing varnishtest command, we use a variable to build
the command line and we execute it with the 'eval' builtin. This way it is
possible to have quoted parameters. For instance:
> ./scripts/run-regtests.sh --varnishtestparams "-Dmacro='some value' -n 10"
And the variable 'varnishtestparams' is also move at the end of the command
line, just before the list of test files. So it is possible to override all
default varnish options set by the script.
The help message is now in the function _help(). It exits the script with the
status 0 to not run regtests when the help is displayed. So it is also handy to
process script arguments before anything else. This avoids to start printing
messages related to regtests execution when it is not appropriate. Another
change, when it detects an invalid argument, the script exits with an error.
Add a new target to the Makefile named "reg-tests-help" to have an idea
about how to run the reg tests from haproxy Makefile.
Handle list of levels and lists of level range passed to make with LEVEL variable.
New supported syntax:
LEVEL=1,4 make reg-tests
LEVEL=1-2,5-6 make reg-tests
Add two new levels 5 and 6. 5 is for broken script, 6 for experimental scripts.
Signed-off-by: Frédéric Lécaille <flecaille@haproxy.com>
Some commands chain grep and sed while sed already does grep by
default, let's simply use sed. In addition to being more intuitive,
it saves up to 150ms per run on the 13 tests covered by level 4.
Use #!bin/sh more portable shebang.
Support filenames with spaces.
Set HAPROXY_PROGRAM environment variable value to ${PWD}/haproxy.
exit(1) if we could not creat the higher level temporary working directory
or create its sub-directory with mktemp utility.
As defined by POSIX, use six characters for the mktemp template.
Some tests require a minimal haproxy version or compilation options to be
able to run successfully. This script allows to add 'requirements' to tests
to check so they will automatically be skipped if a requirement is not met.
The script supports several parameters to slightly modify its behavior
including the directories to search for tests.
Also some features are not available for certain OS's these can also
be 'excluded', this should allow for the complete set of test cases to be
run on any OS against any haproxy release without 'expected failures'.
The test .vtc files will need to be modified to include their 'requirements'
by listing including text options as shown below:
#EXCLUDE_TARGETS=dos,freebsd,windows
#REQUIRE_OPTIONS=ZLIB,OPENSSL,LUA
#REQUIRE_VERSION=0.0
#REQUIRE_VERSION_BELOW=99.9,
When excluding a OS by its TARGET, please do make a comment why the test
can not succeed on that TARGET.
