haproxy/reg-tests
Willy Tarreau 826f3ab5e6 MINOR: stick-tables/counters: add http_fail_cnt and http_fail_rate data types
Historically we've been counting lots of client-triggered events in stick
tables to help detect misbehaving ones, but we've been missing the same on
the server side, and there's been repeated requests for being able to count
the server errors per URL in order to precisely monitor the quality of
service or even to avoid routing requests to certain dead services, which
is also called "circuit breaking" nowadays.

This commit introduces http_fail_cnt and http_fail_rate, which work like
http_err_cnt and http_err_rate in that they respectively count events and
their frequency, but they only consider server-side issues such as network
errors, unparsable and truncated responses, and 5xx status codes other
than 501 and 505 (since these ones are usually triggered by the client).
Note that retryable errors are purposely not accounted for, so that only
what the client really sees is considered.

With this it becomes very simple to put some protective measures in place
to perform a redirect or return an excuse page when the error rate goes
beyond a certain threshold for a given URL, and give more chances to the
server to recover from this condition. Typically it could look like this
to bypass a URL causing more than 10 requests per second:

  stick-table type string len 80 size 4k expire 1m store http_fail_rate(1m)
  http-request track-sc0 base       # track host+path, ignore query string
  http-request return status 503 content-type text/html \
      lf-file excuse.html if { sc0_http_fail_rate gt 10 }

A more advanced mechanism using gpt0 could even implement high/low rates
to disable/enable the service.

Reg-test converteers_ref_cnt_never_dec.vtc was updated to test it.
2021-02-10 12:27:01 +01:00
..
balance REGTEST: fix host part in balance-uri-path-only.vtc 2020-09-29 10:52:27 +02:00
cache REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
checks REGTESTS: unbreak http-check-send.vtc 2021-02-05 10:13:15 +01:00
compression CLEANUP: assorted typo fixes in the code and comments 2021-01-06 16:26:50 +01:00
connection MINOR: reg-tests: add http-reuse test 2021-01-26 14:48:39 +01:00
contrib MEDIUM: contrib/prometheus-exporter: export base stick table stats 2021-02-08 10:49:08 +01:00
converter REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
filters BUG/MEDIUM: filters/htx: Fix data forwarding when payload length is unknown 2021-01-26 09:53:52 +01:00
http-capture
http-cookies MEDIUM: proxy: remove start_proxies() 2020-10-09 11:27:30 +02:00
http-errorfiles BUG/MINOR: http-rules: Fix ACLs parsing for http deny rules 2020-06-30 09:32:03 +02:00
http-messaging REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
http-rules MINOR: http_act: Add -m flag for del-header name matching method 2020-11-21 15:54:30 +01:00
http-set-timeout REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
log REGTESTS: Fix required versions for several scripts 2021-01-28 16:37:14 +01:00
lua REGTESTS: add a test for the threaded Lua code 2020-12-02 21:53:16 +01:00
mailers
mcli
peers
sample_fetches REGTESTS: mark sample_fetches/hashes.vtc as 2.4-only 2021-02-04 18:07:59 +01:00
seamless-reload REGTESTS: mark the abns test as broken again 2020-11-17 11:45:10 +01:00
server REGTEST: server/cli_set_ssl.vtc requires OpenSSL 2020-11-18 17:41:28 +01:00
spoe
ssl MINOR: ssl/cli: flush the server session cache upon 'commit ssl cert' 2021-02-03 18:51:01 +01:00
stick-table MINOR: stick-tables/counters: add http_fail_cnt and http_fail_rate data types 2021-02-10 12:27:01 +01:00
stickiness REGTESTS: add unresolvable servers to srvkey-addr 2021-01-06 09:20:22 +01:00
stream
webstats
README

README

                 * Regression testing for HAProxy with VTest *


This little README file is about how to compile and run vtest test case files (VTC files)
to test HAProxy for any regression.

To do so, you will have to compile vtest program sources which depends on
Varnish cache application sources. vtest, formerly varnishtest, is a very useful
program which has been developed to test Varnish cache application. vtest has been
modified in collaboration with Varnish cache conceptor Poul-Henning Kamp to support
HAProxy in addition to Varnish cache.

See also: doc/regression-testing.txt

* vtest compilation *

    $ git clone https://github.com/vtest/VTest

    $ cd VTest

    $ make vtest

  Then vtest program may be found at the root directory of vtest sources directory.
  The Varnish cache manuals are located in 'man' directory of Varnish cache sources
  directory. You will have to have a look at varnishtest(7) and vtc(7) manuals to
  use vtest.

  Some information may also be found in doc/regression-testing.txt in HAProxy
  sources.

  Note that VTC files for Varnish cache may be found in bin/varnishtest/tests directory
  of Varnish cache sources directory which may be found here:
  https://github.com/varnishcache/varnish-cache


* vtest execution *

  You must set HAPROXY_PROGRAM environment variable to give the location
  of the HAProxy program to test to vtest:

    $ HAPROXY_PROGRAM=<my haproxy program> vtest [-Dno-htx=] ...

  The HAProxy VTC files found in HAProxy sources may be run with the reg-tests
  Makefile target. You must set the VTEST_PROGRAM environment variable to
  give the location of the vtest program which has been previously compiled.

    $ VTEST_PROGRAM=<my vtest program> make reg-tests

  "reg-tests" Makefile target run scripts/run-regtest.sh script.
  To get more information about this script run it with --help option.

  Note that vtest is run with -t10 and -l option. -l option is to keep
  keep vtest temporary directory in case of failed test cases. core files
  may be found in this directory (if enabled by ulimit).


* vtest patches for HAProxy VTC files *

  When producing a patch to add a VTC regression testing file to reg-tests directory,
  please follow these simple rules:

    - If your VTC file needs others files, if possible, use the same basename as that
      of the VTC file,
    - Put these files in a directory with the same name as the code area concerned
      by the bug ('peers', 'lua', 'acl' etc).