Commit Graph

25 Commits

Author SHA1 Message Date
Ilya Shipitsin
856aabcda5 CLEANUP: assorted typo fixes in the code and comments
This is 8th iteration of typo fixes
2020-04-17 09:37:36 +02:00
William Lallemand
d5b464bfee REGTEST: run-regtests: implement #REQUIRE_BINARIES
Implement #REQUIRE_BINARIES for vtc files.

The run-regtests.sh script will check if the binary is available in the
environment, if not, it wil disable the vtc.
2019-12-19 14:36:46 +01:00
Christopher Faulet
1eee6ca89e REGTEST: Add an HTX reg-test to check an edge case
This test checks that an HTTP message is properly processed when we failed to
add the HTX EOM block in an HTX message during the parsing because the buffer is
full. Some space must be released in the buffer to make it possible. This
requires an extra pass in the H1 multiplexer. Here, we must be sure the mux is
called while there is no more incoming data.

It is a "devel" test because conditions to run the test successfully is highly
dependent on the implementation. So if it fail, it is not necessarily a bug. It
may be due of an internal change. It relies on internal HTX sample fetches.
2019-12-11 16:46:16 +01:00
Bertrand Jacquin
7e30b50417 DOC: fix "successful" typo
s/succesfull/successful at a few places
2019-05-18 08:25:29 +02:00
Frédéric Lécaille
85a7ea0740 REGTEST: Add a new reg test for log load-balancing feature.
This is a reg test for the log load-balancing feature implemented by
these commits:
  MINOR: log: Add "sample" new keyword to "log" lines
  MINOR: log: Enable the log sampling and load-balancing feature

The size of the logging buffer for vtest has been doubled to support this script.
2019-04-30 09:25:09 +02:00
Willy Tarreau
ca8df4c074 REGTEST: make the "run-regtests" script search for tests in reg-tests by default
It happens almost daily to me that make regtests fails because the script
found a temporary, old, or broken VTC file that was lying in my work dir,
leaving me no place to hide it. This is a real pain as some tests take ages
to fail, so let's make this script only look up for tests where they are
expected to be stored, under reg-tests only. It remains possible to force
the location on the command line though.
2019-04-23 16:09:50 +02:00
Frédéric Lécaille
dc1a3bd999 REGTEST: replace LEVEL option by a more human readable one.
This patch replaces LEVEL variable by REGTESTS_TYPES variable which is more
mnemonic and human readable. It is uses as a filter to run the reg tests scripts
where a commented #REGTEST_TYPE may be defined to designate their types.
Running the following command:

    $ REGTESTS_TYPES=slow,default

will start all the reg tests where REGTEST_TYPE is defines as 'slow' or 'default'.
Note that 'default' is also the default value of REGTEST_TYPE when not specified
dedicated to run all the current h*.vtc files. When REGTESTS_TYPES is not specified
there is no filter at all. All the tests are run.

This patches also defines REGTEST_TYPE with 'slow' value for all the s*.vtc files,
'bug' value for al the b*.vtc files, 'broken' value for all the k*.vtc files.
2019-04-23 15:14:52 +02:00
Christopher Faulet
78f31bf07a REGTEST: Use HTX by default and add '--no-htx' option to disable it
Because the HTX is now the default mode in HAProxy, it is also enabled by
default in reg-tests. The option '--use-htx' is still available, but deprecated
and have concretly no effect. To run reg-tests with the legacy HTTP mode, you
should use the option '--no-htx'.
2019-04-12 22:06:53 +02:00
Christopher Faulet
1d2b586cdd MAJOR: htx: Enable the HTX mode by default for all proxies
The legacy HTTP mode is no more the default one. So now, by default, without any
option in your configuration, all proxies will use the HTX mode. The line
"option http-use-htx" in proxy sections are now useless, except to cancel the
legacy HTTP mode. To fallback on legacy HTTP mode, you should use the line "no
option http-use-htx" explicitly.

Note that the reg-tests still work by default on legacy HTTP mode. The HTX will
be enabled by default in a futur commit.
2019-04-12 22:06:53 +02:00
Willy Tarreau
ad25c2658e REGTEST: script: remove platform-specific assigments of OPTIONS
We don't use OPTIONS anymore, let's simply remove all the code that sets
this variable. It was not viable anyway to keep this one in sync with
the makefile.
2019-03-27 14:47:55 +01:00
Willy Tarreau
87586e1595 REGTEST: script: make the script use the new features list
Currently the script only sees options that differ from the default,
which makes it fail on default options that are disabled (such as
threads on the relevant platforms). Let's make it instead extract
the newly introduced feature list and search for an explicit "+" in
front of the desiered feature. This one is known for always being
valid.
2019-03-27 14:47:55 +01:00
Frédéric Lécaille
4336123b5c REGTEST: Switch to vtest.
This patch replace the usage of the formerly varnish cache reg
testing program, name varnishtest by the new standalone one: vtest.
2019-01-14 14:21:13 +01:00
Frédéric Lécaille
f9a48ef791 REGTEST: Add some informatoin to test results.
When the reg tests fail, it may be useful to display additional information
coming from varnishtest, especially when this latter aborts.
In such case, the test output may be made of lines prefixed by "*     diag"
string.
2019-01-08 11:43:35 +01:00
Christopher Faulet
7e245dfaca REGTEST: script: Add support of alternatives in requited options list
It is now possible to specify a list of "alternatives" for a required
option. This must be done by separating options by a pipe ('|'). A test will be
executed if at least one of them is available. For instance:

  #REQUIRED_OPTIONS=ZLIB|SLZ,LUA,OPENSSL

The function _findtest() has also been sligthly simplified.
2018-12-21 15:33:26 +01:00
Christopher Faulet
6bd82cdbe6 REGTEST: script: Print only errors in the results report 2018-12-20 10:37:32 +01:00
Christopher Faulet
fa6798f97c REGTEST: script: Add the option --use-htx to enable the HTX in regtests
A check is done on HAProxy version to be sure it is supported. Internally, by
default, the macro 'no-htx' is set to "#". So in regtests, we can prefix
http-use-htx option line with this macro to inhibit the HTX. Concretly, this
line can be added in the HAProxy configuration:

   ${no-htx} option http-use-htx

When the option --use-htx is used, the macro 'no-htx' is unset.
2018-12-20 10:37:32 +01:00
Christopher Faulet
9c6df5ecb4 REGTEST: script: Add the option --keep-logs to keep all log directories
By default a log directory is kept only if the test fails. With this option it
is possible to always keep it. If this option is used, the result of all tests
are displayed (and not only the failing ones).
2018-12-20 10:37:24 +01:00
Christopher Faulet
2a7cf922c1 REGTEST: script: Add the option --debug to show logs on standard ouput
It implies the option --v. It is handy when you run the same test during a debug
session.
2018-12-20 10:33:18 +01:00
Christopher Faulet
8d0fdf5ef2 REGTEST: script: Add the option --clean to remove previous log direcotries
Running regtests several times leaves many direcotries. It is painful to remove
them by hand. This command do it for you. It ask a confirmation to proceed to be
sure. The template used to create log direcotries has been changed. The prefix
'haregtests-' has been added to help the cleanup function to find existing log
direcotries.
2018-12-20 10:33:18 +01:00
Christopher Faulet
1ecf0ea70a REGTEST: script: Evaluate the varnishtest command to allow quoted parameters
Instead of directly executing varnishtest command, we use a variable to build
the command line and we execute it with the 'eval' builtin. This way it is
possible to have quoted parameters. For instance:

  > ./scripts/run-regtests.sh --varnishtestparams "-Dmacro='some value' -n 10"

And the variable 'varnishtestparams' is also move at the end of the command
line, just before the list of test files. So it is possible to override all
default varnish options set by the script.
2018-12-20 10:33:18 +01:00
Christopher Faulet
8d67cf8187 REGTEST: script: Process script arguments before everything else
The help message is now in the function _help(). It exits the script with the
status 0 to not run regtests when the help is displayed. So it is also handy to
process script arguments before anything else. This avoids to start printing
messages related to regtests execution when it is not appropriate. Another
change, when it detects an invalid argument, the script exits with an error.
2018-12-20 10:33:18 +01:00
Frederic Lecaille
d4f36e3eaa REGTEST: Reg testing improvements.
Add a new target to the Makefile named "reg-tests-help" to have an idea
about how to run the reg tests from haproxy Makefile.
Handle list of levels and lists of level range passed to make with LEVEL variable.
New supported syntax:
    LEVEL=1,4     make reg-tests
    LEVEL=1-2,5-6 make reg-tests
Add two new levels 5 and 6. 5 is for broken script, 6 for experimental scripts.

Signed-off-by: Frédéric Lécaille <flecaille@haproxy.com>
2018-12-14 06:50:45 +01:00
Willy Tarreau
939193a1a1 SCRIPTS/REGTEST: merge grep+sed into sed in run-regtests
Some commands chain grep and sed while sed already does grep by
default, let's simply use sed. In addition to being more intuitive,
it saves up to 150ms per run on the 13 tests covered by level 4.
2018-12-06 15:51:35 +01:00
Frédéric Lécaille
51e01b56ec REGTEST: Fix several issues.
Use #!bin/sh more portable shebang.
Support filenames with spaces.
Set HAPROXY_PROGRAM environment variable value to ${PWD}/haproxy.
exit(1) if we could not creat the higher level temporary working directory
or create its sub-directory with mktemp utility.
As defined by POSIX, use six characters for the mktemp template.
2018-11-30 05:58:45 +01:00
PiBa-NL
7250404b71 REGTEST/MINOR: script: add run-regtests.sh script
Some tests require a minimal haproxy version or compilation options to be
able to run successfully. This script allows to add 'requirements' to tests
to check so they will automatically be skipped if a requirement is not met.
The script supports several parameters to slightly modify its behavior
including the directories to search for tests.

Also some features are not available for certain OS's these can also
be 'excluded', this should allow for the complete set of test cases to be
run on any OS against any haproxy release without 'expected failures'.

The test .vtc files will need to be modified to include their 'requirements'
by listing including text options as shown below:
    #EXCLUDE_TARGETS=dos,freebsd,windows
    #REQUIRE_OPTIONS=ZLIB,OPENSSL,LUA
    #REQUIRE_VERSION=0.0
    #REQUIRE_VERSION_BELOW=99.9,
When excluding a OS by its TARGET, please do make a comment why the test
can not succeed on that TARGET.
2018-11-29 04:54:45 +01:00