haproxy/reg-tests
Willy Tarreau 44c5ff69ac MEDIUM: vars: make the var() sample fetch function really return type ANY
A long-standing issue was reported in issue #1215.

In short, var() was initially internally declared as returning a string
because it was not possible by then to return "any type". As such, users
regularly get trapped thinking that when they're storing an integer there,
then the integer matching method automatically applies. Except that this
is not possible since this is related to the config parser and is decided
at boot time where the variable's type is not known yet.

As such, what is done is that the output being declared as type string,
the string match will automatically apply, and any value will first be
converted to a string. This results in several issues like:

    http-request set-var(txn.foo) int(-1)
    http-request deny if { var(txn.foo) lt 0 }

not working. This is because the string match on the second line will in
fact compare the string representation of the variable against strings
"lt" and "0", none of which matches.

The doc says that the matching method is mandatory, though that's not
the case in the code due to that default string type being permissive.
There's not even a warning when no explicit match is placed, because
this happens very deep in the expression evaluator and making a special
case just for "var" can reveal very complicated.

The set-var() converter already mandates a matching method, as the
following will be rejected:

    ... if { int(12),set-var(txn.truc) 12 }

  while this one will work:

    ... if { int(12),set-var(txn.truc) -m int 12 }

As such, this patch this modifies var() to match the doc, returning the
type "any", and mandating the matching method, implying that this bogus
config which does not work:

    http-request set-var(txn.foo) int(-1)
    http-request deny if { var(txn.foo) lt 0 }

  will need to be written like this:

    http-request set-var(txn.foo) int(-1)
    http-request deny if { var(txn.foo) -m int lt 0 }

This *will* break some configs (and even 3 of our regtests relied on
this), but except those which already match string exclusively, all
other ones are already broken and silently fail (and one of the 3
regtests, the one on FIX, was bogus regarding this).

In order to fix existing configs, one can simply append "-m str"
after a "var()" in an ACL or "if" expression:

    http-request deny unless { var(txn.jwt_alg) "ES" }

  must become:

    http-request deny unless { var(txn.jwt_alg) -m str "ES" }

Most commonly, patterns such as "le", "lt", "ge", "gt", "eq", "ne" in
front of a number indicate that the intent was to match an integer,
and in this case "-m int" would be desired:

    tcp-response content reject if ! { var(res.size) gt 3800 }

  ought to become:

    tcp-response content reject if ! { var(res.size) -m int gt 3800 }

This must not be backported, but if a solution is found to at least
detect this exact condition in the generic expression parser and
emit a warning, this could probably help spot configuration bugs.

Link: https://www.mail-archive.com/haproxy@formilux.org/msg41341.html
Cc: Christopher Faulet <cfaulet@haproxy.com>
Cc: Tim Düsterhus <tim@bastelstu.be>
2021-11-02 17:28:43 +01:00
..
balance
…
cache BUG/MINOR: cache: Correctly handle existing-but-empty 'accept-encoding' header 2021-06-18 15:48:20 +02:00
checks BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check 2021-09-16 17:24:50 +02:00
compression REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests 2021-06-11 19:21:28 +02:00
connection REGTESTS: Add script to test client src/dst manipulation at different levels 2021-10-27 11:35:59 +02:00
contrib
…
converter MEDIUM: vars: make the var() sample fetch function really return type ANY 2021-11-02 17:28:43 +01:00
filters BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the release 2021-10-04 08:28:44 +02:00
http-capture CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
http-cookies CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
http-errorfiles
…
http-messaging CLEANUP: assorted typo fixes in the code and comments 2021-10-18 07:26:19 +02:00
http-rules REGTESTS: Add scripts to test support of TCP/HTTP rules in defaults sections 2021-10-15 14:12:19 +02:00
http-set-timeout
…
jwt MEDIUM: vars: make the var() sample fetch function really return type ANY 2021-11-02 17:28:43 +01:00
log
…
lua REGTESTS: lua: test httpclient with body streaming 2021-10-28 16:26:47 +02:00
mailers REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests 2021-06-11 19:21:28 +02:00
mcli REGTESTS: Replace REQUIRE_BINARIES with 'command -v' 2021-06-17 14:59:55 +02:00
peers
…
sample_fetches MINOR: vars: make the vars() sample fetch function support a default value 2021-09-03 12:08:54 +02:00
seamless-reload CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
server REGTESTS: Use `feature cmd` for 2.5+ tests 2021-08-25 21:38:38 +02:00
spoe
…
ssl CLEANUP: assorted typo fixes in the code and comments 2021-10-18 07:26:19 +02:00
startup REGTESTS: Add scripts to test support of TCP/HTTP rules in defaults sections 2021-10-15 14:12:19 +02:00
stick-table CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
stickiness CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00
stream
…
tcp-rules REGTESTS: Add scripts to test support of TCP/HTTP rules in defaults sections 2021-10-15 14:12:19 +02:00
webstats REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests 2021-06-11 19:21:28 +02:00
README CLEANUP: reg-tests: Remove obsolete no-htx parameter for reg-tests 2021-06-04 15:41:21 +02:00

README

                 * Regression testing for HAProxy with VTest *


This little README file is about how to compile and run vtest test case files (VTC files)
to test HAProxy for any regression.

To do so, you will have to compile vtest program sources which depends on
Varnish cache application sources. vtest, formerly varnishtest, is a very useful
program which has been developed to test Varnish cache application. vtest has been
modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support
HAProxy in addition to Varnish cache.

See also: doc/regression-testing.txt

* vtest compilation *

    $ git clone https://github.com/vtest/VTest

    $ cd VTest

    $ make vtest

  Then vtest program may be found at the root directory of vtest sources directory.
  The Varnish cache manuals are located in 'man' directory of Varnish cache sources
  directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to
  use vtest.

  Some information may also be found in doc/regression-testing.txt in HAProxy
  sources.

  Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory
  of Varnish cache sources directory which may be found here:
  https://github.com/varnishcache/varnish-cache


* vtest execution *

  You must set HAPROXY_PROGRAM environment variable to give the location
  of the HAProxy program to test to vtest:

    $ HAPROXY_PROGRAM=<my haproxy program> vtest ...

  The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
  Makefile target. You must set the VTEST_PROGRAM environment variable to
  give the location of the vtest program which has been previously compiled.

    $ VTEST_PROGRAM=<my vtest program> make reg-tests

  "reg-tests" Makefile target run scripts/run-regtest.sh script.
  To get more information about this script run it with --help option.

  Note that vtest is run with -t10 and -l option. -l option is to keep
  keep vtest temporary directory in case of failed test cases. core files
  may be found in this directory (if enabled by ulimit).


* vtest patches for HAProxy VTC files *

  When producing a patch to add a VTC regression testing file to reg-tests directory,
  please follow these simple rules:

    - If your VTC file needs others files, if possible, use the same basename as that
      of the VTC file,
    - Put these files in a directory with the same name as the code area concerned
      by the bug ('peers', 'lua', 'acl' etc).