haproxy/reg-tests
Willy Tarreau 3a4bedccc6 MEDIUM: vars: replace the global name index with a hash
The global table of known variables names can only grow and was designed
for static names that are registered at boot. Nowadays it's possible to
set dynamic variable names from Lua or from the CLI, which causes a real
problem that was partially addressed in 2.2 with commit 4e172c93f
("MEDIUM: lua: Add `ifexist` parameter to `set_var`"). Please see github
issue #624 for more context.

This patch simplifies all this by removing the need for a central
registry of known names, and storing 64-bit hashes instead. This is
highly sufficient given the low number of variables in each context.
The hash is calculated using XXH64() which is bijective over the 64-bit
space thus is guaranteed collision-free for 1..8 chars. Above that the
risk remains around 1/2^64 per extra 8 chars so in practice this is
highly sufficient for our usage. A random seed is used at boot to seed
the hash so that it's not attackable from Lua for example.

There's one particular nit though. The "ifexist" hack mentioned above
is now limited to variables of scope "proc" only, and will only match
variables that were already created or declared, but will now verify
the scope as well. This may affect some bogus Lua scripts and SPOE
agents which used to accidentally work because a similarly named
variable used to exist in a different scope. These ones may need to be
fixed to comply with the doc.

Now we can sum up the situation as this one:
  - ephemeral variables (scopes sess, txn, req, res) will always be
    usable, regardless of any prior declaration. This effectively
    addresses the most problematic change from the commit above that
    in order to work well could have required some script auditing ;

  - process-wide variables (scope proc) that are mentioned in the
    configuration, referenced in a "register-var-names" SPOE directive,
    or created via "set-var" in the global section or the CLI, are
    permanent and will always accept to be set, with or without the
    "ifexist" restriction (SPOE uses this internally as well).

  - process-wide variables (scope proc) that are only created via a
    set-var() tcp/http action, via Lua's set_var() calls, or via an
    SPOE with the "force-set-var" directive), will not be permanent
    but will always accept to be replaced once they are created, even
    if "ifexist" is present

  - process-wide variables (scope proc) that do not exist will only
    support being created via the set-var() tcp/http action, Lua's
    set_var() calls without "ifexist", or an SPOE declared with
    "force-set-var".

This means that non-proc variables do not care about "ifexist" nor
prior declaration, and that using "ifexist" should most often be
reliable in Lua and that SPOE should most often work without any
prior declaration. It may be doable to turn "ifexist" to 1 by default
in Lua to further ease the transition. Note: regtests were adjusted.

Cc: Tim Düsterhus <tim@bastelstu.be>
2021-09-08 15:06:11 +02:00
..
balance REGTEST: fix host part in balance-uri-path-only.vtc 2020-09-29 10:52:27 +02:00
cache BUG/MINOR: cache: Correctly handle existing-but-empty 'accept-encoding' header 2021-06-18 15:48:20 +02:00
checks REGTESTS: fix maxconn update with agent-check 2021-06-22 16:34:23 +02:00
compression REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests 2021-06-11 19:21:28 +02:00
connection REGTESTS: http_upgrade: fix incorrect expectation on TCP->H1->H2 2021-08-20 11:02:28 +02:00
contrib REGTESTS: contrib/prometheus-exporter: test well known labels 2021-02-19 18:03:59 +01:00
converter MINOR: sample: Add be2hex converter 2021-08-26 19:48:34 +02:00
filters CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
http-capture CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
http-cookies CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
http-errorfiles BUG/MINOR: http-rules: Fix ACLs parsing for http deny rules 2020-06-30 09:32:03 +02:00
http-messaging REGTESTS: Use `feature cmd` for 2.5+ tests 2021-08-25 21:38:38 +02:00
http-rules MINOR: http-rules: add a new "ignore-empty" option to redirects. 2021-09-02 17:06:18 +02:00
http-set-timeout REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
log REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
lua MEDIUM: vars: replace the global name index with a hash 2021-09-08 15:06:11 +02:00
mailers REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests 2021-06-11 19:21:28 +02:00
mcli REGTESTS: Replace REQUIRE_BINARIES with 'command -v' 2021-06-17 14:59:55 +02:00
peers REGTESTS: Send valid URIs in peers reg-tests and fix HA config to avoid warnings 2019-10-14 22:28:50 +02:00
sample_fetches MINOR: vars: make the vars() sample fetch function support a default value 2021-09-03 12:08:54 +02:00
seamless-reload CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
server REGTESTS: Use `feature cmd` for 2.5+ tests 2021-08-25 21:38:38 +02:00
spoe CLEANUP: assorted typo fixes in the code and comments 2020-03-14 09:42:07 +01:00
ssl REGTESTS: ssl: Add tests for bc_conn_err and ssl_bc_hsk_err sample fetches 2021-09-01 22:55:56 +02:00
startup REGTESTS: add more complex check conditions to check_conditions.vtc 2021-07-17 11:01:47 +02:00
stick-table CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
stickiness CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
stream REGTEST: make the unique-id test depend on version 2.0 2020-04-01 16:08:43 +02:00
webstats REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests 2021-06-11 19:21:28 +02:00
README CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00

README

                 * Regression testing for HAProxy with VTest *


This little README file is about how to compile and run vtest test case files (VTC files)
to test HAProxy for any regression.

To do so, you will have to compile vtest program sources which depends on
Varnish cache application sources. vtest, formerly varnishtest, is a very useful
program which has been developed to test Varnish cache application. vtest has been
modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support
HAProxy in addition to Varnish cache.

See also: doc/regression-testing.txt

* vtest compilation *

    $ git clone https://github.com/vtest/VTest

    $ cd VTest

    $ make vtest

  Then vtest program may be found at the root directory of vtest sources directory.
  The Varnish cache manuals are located in 'man' directory of Varnish cache sources
  directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to
  use vtest.

  Some information may also be found in doc/regression-testing.txt in HAProxy
  sources.

  Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory
  of Varnish cache sources directory which may be found here:
  https://github.com/varnishcache/varnish-cache


* vtest execution *

  You must set HAPROXY_PROGRAM environment variable to give the location
  of the HAProxy program to test to vtest:

    $ HAPROXY_PROGRAM=<my haproxy program> vtest ...

  The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
  Makefile target. You must set the VTEST_PROGRAM environment variable to
  give the location of the vtest program which has been previously compiled.

    $ VTEST_PROGRAM=<my vtest program> make reg-tests

  "reg-tests" Makefile target run scripts/run-regtest.sh script.
  To get more information about this script run it with --help option.

  Note that vtest is run with -t10 and -l option. -l option is to keep
  keep vtest temporary directory in case of failed test cases. core files
  may be found in this directory (if enabled by ulimit).


* vtest patches for HAProxy VTC files *

  When producing a patch to add a VTC regression testing file to reg-tests directory,
  please follow these simple rules:

    - If your VTC file needs others files, if possible, use the same basename as that
      of the VTC file,
    - Put these files in a directory with the same name as the code area concerned
      by the bug ('peers', 'lua', 'acl' etc).