REGTEST: Adapt reg test doc files to vtest.
This is a first patch to switch from varnishtest to new standalone varnish cache reg testing program: vtest. More information may be found here: https://github.com/vtest/VTest https://varnish-cache.org/docs/trunk/reference/varnishtest.html https://varnish-cache.org/docs/trunk/reference/vtc.html
This commit is contained in:
Frédéric Lécaille
Willy Tarreau
parent
4336123b5c
commit
617d4f5276
|
|
@ -1,74 +1,82 @@
|
|||
+---------------------------------------------+
|
||||
| HAProxy regression testing with varnishtest |
|
||||
+---------------------------------------------+
|
||||
+---------------------------------------+
|
||||
| HAProxy regression testing with vtest |
|
||||
+---------------------------------------+
|
||||
|
||||
|
||||
The information found in this file are a short starting guide to help you to
|
||||
write VTC (Varnish Test Case) scripts (or VTC files) for haproxy regression testing.
|
||||
Such VTC files are currently used to test Varnish cache application developed by
|
||||
Poul-Henning Kamp. A very big thanks you to him for having helped you to add
|
||||
our haproxy C modules to varnishtest tool.
|
||||
our haproxy C modules to vtest tool. Note that vtest was formally developed for
|
||||
varnish cache reg testing and was named varnishtest. vtest is an haproxy specific
|
||||
version of varnishtest program which reuses the non varnish cache specific code.
|
||||
|
||||
A lot of general information about how to write VTC files may be found in 'man/vtc.7'
|
||||
manual. It is *highly* recommended to read this manual before asking. This
|
||||
documentation only deals with the varnishtest support for haproxy.
|
||||
manual of varnish cache sources directory or directly on the web here:
|
||||
|
||||
https://varnish-cache.org/docs/trunk/reference/vtc.html
|
||||
|
||||
It is *highly* recommended to read this manual before asking to haproxy ML. This
|
||||
documentation only deals with the vtest support for haproxy.
|
||||
|
||||
|
||||
varnishtest installation
|
||||
vtest installation
|
||||
------------------------
|
||||
|
||||
To use varnishtest you will have to download and compile the recent Varnish cache
|
||||
sources found at https://github.com/varnishcache/varnish-cache.
|
||||
To use vtest you will have to download and compile the recent vtest
|
||||
sources found at https://github.com/vtest/VTest.
|
||||
|
||||
To compile Varnish cache :
|
||||
To compile vtest:
|
||||
|
||||
$ ./autogen.sh
|
||||
$ ./configure
|
||||
$ make
|
||||
$ cd VTest
|
||||
$ make vtest
|
||||
|
||||
The varnishtest sources may be found in 'bin/varnishtest' directory.
|
||||
'bin/varnishtest/tests' is plenty of VTC files for Varnish cache. After having
|
||||
compiled these sources, the varnishtest executable location is
|
||||
'bin/varnishtest/varnishtest'.
|
||||
Note that varnishtest may be also compiled but not without the varnish cache
|
||||
sources already compiled:
|
||||
|
||||
varnishtest is able to search for the haproxy executable file it is supposed to
|
||||
launch in the PATH environment variable. To force the executable to be used by
|
||||
varnishtest, the HAPROXY_PROGRAM environment variable for varnishtest may be
|
||||
typically set as follows:
|
||||
$ VARNISH_SRC=<...> make varnishtest
|
||||
|
||||
$ HAPROXY_PROGRAM=~/srcs/haproxy/haproxy varnishtest ...
|
||||
After having compiled these sources, the vtest executable location is at the
|
||||
root of the vtest sources directory.
|
||||
|
||||
|
||||
varnistest exectution
|
||||
vtest exectution
|
||||
---------------------
|
||||
|
||||
varnishtest program comes with interesting options. The most interesting are:
|
||||
vtest is able to search for the haproxy executable file it is supposed to
|
||||
launch thanks to the PATH environment variable. To force the executable to be used by
|
||||
vtest, the HAPROXY_PROGRAM environment variable for vtest may be
|
||||
typically set as follows:
|
||||
|
||||
$ HAPROXY_PROGRAM=~/srcs/haproxy/haproxy vtest ...
|
||||
|
||||
vtest program comes with interesting options. The most interesting are:
|
||||
|
||||
-t Timeout in seconds to abort the test if some launched program
|
||||
-v By default, varnishtest does not dump the outputs of processus it launched
|
||||
-v By default, vtest does not dump the outputs of processus it launched
|
||||
when the test passes. With this option the outputs are dumped even
|
||||
when the test passes.
|
||||
-L to always keep the temporary VTC directories.
|
||||
-l to keep the temporary VTC directories only when the test fails.
|
||||
|
||||
About haproxy when launched by varnishtest, -d option is enabled by default.
|
||||
About haproxy, when launched by vtest, -d option is enabled by default.
|
||||
|
||||
|
||||
How to write VTC files
|
||||
----------------------
|
||||
|
||||
A VTC file must start with a "varnishtest" command line followed by a descriptive
|
||||
line enclosed by double quotes. This is not specific to the VTC files for haproxy.
|
||||
A VTC file must start with a "varnishtest" or "vtest" command line followed by a
|
||||
descriptive line enclosed by double quotes. This is not specific to the VTC files
|
||||
for haproxy.
|
||||
|
||||
The VTC files for haproxy must also contain a "feature ignore_unknown_macro" line
|
||||
if any macro is used for haproxy in this file. This is due to the fact that
|
||||
varnishtest parser code for haproxy commands generates macros the varnishtest
|
||||
parser code for varnish has no knowledge of. This line prevents varnishtest from
|
||||
failing in such cases. Since bbc34e2 varnish commit, as a "cli" macro is
|
||||
automatically generated, this "feature ignore_unknown_macro" is mandatory
|
||||
for each VTC file for haproxy.
|
||||
vtest parser code for haproxy commands generates macros the vtest
|
||||
parser code for varnish cache has no knowledge of. This line prevents vtest from
|
||||
failing in such cases. As a "cli" macro automatically generated, this
|
||||
"feature ignore_unknown_macro" is mandatory for each VTC file for haproxy.
|
||||
|
||||
To make varnishtest capable of testing haproxy, two new VTC commands have been
|
||||
To make vtest capable of testing haproxy, two new VTC commands have been
|
||||
implemented: "haproxy" and "syslog". "haproxy" is used to start haproxy processus.
|
||||
"syslog" is used to start syslog servers (at this time, only used by haproxy).
|
||||
|
||||
|
|
@ -99,7 +107,7 @@ the configuration file is temporarily written.
|
|||
A 'stats.sock' UNIX socket is also created in this directory. There is no need
|
||||
to declare such stats sockets in the -conf {...} section. The name of the parent
|
||||
directory of the haproxy instances working directories is stored in 'tmpdir'. In
|
||||
fact this is the working directory of the current varnishtest processus.
|
||||
fact this is the working directory of the current vtest processus.
|
||||
|
||||
There also exists 'testdir' macro which is the parent directory of the VTC file.
|
||||
It may be useful to use other files located in the same directory than the current
|
||||
|
|
|
|||
|
|
@ -1,63 +1,65 @@
|
|||
* Regression testing for HAProxy with varnishtest *
|
||||
* Regression testing for HAProxy with VTest *
|
||||
|
||||
|
||||
This little README file is about how to compile and run varnishtest test case files (VTC files) to test HAProxy for any regression.
|
||||
This little README file is about how to compile and run vtest test case files (VTC files)
|
||||
to test HAProxy for any regression.
|
||||
|
||||
To do so, you will have to compile vtest program sources which depends on
|
||||
Varnish cache application sources. vtest, formerly varnishtest, is a very useful
|
||||
program which has been developed to test Varnish cache application. vtest has been
|
||||
modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support
|
||||
HAProxy in addition to Varnish cache.
|
||||
|
||||
To do so, you will have to compile varnishtest program sources which comes with
|
||||
Varnish cache application. varnishtest is a very useful program which has been
|
||||
developed to test Varnish cache application. varnishtest has been modified in
|
||||
collaboration with Varnish cache conceptor Poul-Henning Kamp to support HAProxy in
|
||||
addition to Varnish cache.
|
||||
See also: doc/regression-testing.txt
|
||||
|
||||
* varnishtest compilation *
|
||||
* vtest compilation *
|
||||
|
||||
- Clone recent Varnish cache sources which may be found in this GitHub repository
|
||||
https://github.com/varnishcache/varnish-cache:
|
||||
$ git clone https://github.com/vtest/VTest
|
||||
|
||||
$ git clone https://github.com/varnishcache/varnish-cache
|
||||
$ cd VTest
|
||||
|
||||
- Compile Varnish cache sources:
|
||||
$ make vtest
|
||||
|
||||
$ cd varnish-cache && ./autogen.sh && ./configure && make
|
||||
|
||||
Then varnishtest program may be found in bin/varnishtest directory.
|
||||
VTC files for Varnish cache may be found in bin/varnishtest/tests directory.
|
||||
The Varnish cache manuals are located in 'man' directory. You will have to
|
||||
have a look at varnishtest(7) and vtc(7) manuals.
|
||||
Then vtest program may be found at the root directory of vtest sources directory.
|
||||
The Varnish cache manuals are located in 'man' directory of Varnish cache sources
|
||||
directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to
|
||||
use vtest.
|
||||
|
||||
Some information may also be found in doc/regression-testing.txt in HAProxy
|
||||
sources.
|
||||
|
||||
Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory
|
||||
of Varnish cache sources directory which may be found here:
|
||||
https://github.com/varnishcache/varnish-cache
|
||||
|
||||
* varnishtest execution *
|
||||
|
||||
* vtest execution *
|
||||
|
||||
You must set HAPROXY_PROGRAM environment variable to give the location
|
||||
of the HAProxy program to test to varnishtest:
|
||||
of the HAProxy program to test to vtest:
|
||||
|
||||
$ HAPROXY_PROGRAM=<my haproxy program> varnishtest [-Dno-htx=] ...
|
||||
$ HAPROXY_PROGRAM=<my haproxy program> vtest [-Dno-htx=] ...
|
||||
|
||||
The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
|
||||
Makefile target. You must set the VARNISHTEST_PROGRAM environment variable to
|
||||
give the location of the varnishtest program which has been previously compiled.
|
||||
Makefile target. You must set the VTEST_PROGRAM environment variable to
|
||||
give the location of the vtest program which has been previously compiled.
|
||||
|
||||
$ VARNISHTEST_PROGRAM=<my varnishtest program> HAPROXY_PROGRAM=<my haproxy program> make reg-tests
|
||||
$ VTEST_PROGRAM=<my vtest program> make reg-tests
|
||||
|
||||
Note that varnishtest is run with -t5 and -l option. -l option is to keep
|
||||
keep varnishtest temporary directory in case of failed test cases. core files
|
||||
"reg-tests" Makefile target run scripts/run-regtest.sh script.
|
||||
To get more information about this script run it with --help option.
|
||||
|
||||
Note that vtest is run with -t10 and -l option. -l option is to keep
|
||||
keep vtest temporary directory in case of failed test cases. core files
|
||||
may be found in this directory (if enabled by ulimit).
|
||||
|
||||
|
||||
* varnishtest patches for HAProxy VTC files *
|
||||
* vtest patches for HAProxy VTC files *
|
||||
|
||||
When producing a patch to add a VCT regression testing file to reg-tests directory,
|
||||
When producing a patch to add a VTC regression testing file to reg-tests directory,
|
||||
please follow these simple rules:
|
||||
|
||||
- Add the commit number on the first line.
|
||||
- Then, copy and paste the commit log.
|
||||
- If your VTC file needs others files, use the same basename as that of the VTC
|
||||
file.
|
||||
- If your VTC file needs others files, if possible, use the same basename as that
|
||||
of the VTC file,
|
||||
- Put these files in a directory with the same name as the code area concerned
|
||||
by the bug ('peers', 'lua', 'acl' etc).
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue