Ilya reported that the "which" utility is not that much portable and is
absent from Fedora. "type -p" is not portable either, and the correct
solution appears to be "command -v", so let's use this for now, we can
change it again in the future in case of problems.
Link: https://www.mail-archive.com/haproxy@formilux.org/msg36332.html
A reg test has been added to ensure the evaluation of http-after-responses rules
is functionnal for all kind of responses (server, applet and internal
responses).
2 reg tests have been added to ensure the HTTP return action is functionnal. A
reg test is about returning error files. The other one is about returning
default responses and responses based on string or file payloads.
2 reg tests are added. The first one ensures the declaration of errors in a
proxy is fonctionnal. It declares http-errors sections and declare error files
using the errorfile and the errorfiles directives, both in the default section
and the frontend sections. The second one ensures it is possible to use a custom
error file for an HTTP deny rule.
With this new reg test we ensure the strict rewriting mode of HTTP rules is
functional. The mode is tested for request and response rules. The default mode
(strict), the swtich off and the reset on new ruleset are tested for both.
First, concat() is a converter, not a sample fetch. So use str() sample fetch
with no string and call concat on it. Then, the argument of the set-uri rule
must be a log format string. So it must be inside %[] to be evaluated.
This regtest validates all hashes that we support, on all input bytes from
0x00 to 0xFF. Those supporting avalanche are tested as well. It also tests
len(), hex() and base64(). It purposely does not enable sha2() because this
one relies on OpenSSL and there's no point in validating that OpenSSL knows
how to hash, what matters is that we can test our hashing functions in all
cases. However since the tests were written, they're still present and
commented out in case that helps.
It may be backported to supported versions, possibly dropping a few algos
that were not supported (e.g. crc32c requires 1.9 minimum).
Note that this test will fail on crc32/djb2/sdbm/wt6 unless patches
"BUG/MINOR: stream: init variables when the list is empty" and
"BUG/MAJOR: hashes: fix the signedness of the hash inputs" are included.
This regtest tests the issue #446 by starting 2 programs and checking if
they exist in the "show proc" of the master CLI.
Should be backported as far as 2.0.
Implement #REQUIRE_BINARIES for vtc files.
The run-regtests.sh script will check if the binary is available in the
environment, if not, it wil disable the vtc.
Add a reg-test which test the update of a certificate over the CLI. This
test requires socat and curl.
This commit also adds an ECDSA certificate in the ssl directory.
Instead of failing the conversion when an invalid number of bits is
given the sha2 converter now fails with an appropriate error message
during startup.
The sha2 converter was introduced in d437630237,
which is in 2.1 and higher.
This test checks that an HTTP message is properly processed when we failed to
add the HTX EOM block in an HTX message during the parsing because the buffer is
full. Some space must be released in the buffer to make it possible. This
requires an extra pass in the H1 multiplexer. Here, we must be sure the mux is
called while there is no more incoming data.
It is a "devel" test because conditions to run the test successfully is highly
dependent on the implementation. So if it fail, it is not necessarily a bug. It
may be due of an internal change. It relies on internal HTX sample fetches.
The sample fetche can get srv_name without foreach
`core.backends["bk"].servers`.
Then we can get Server class quickly via
`core.backends[txn.f:be_name()].servers[txn.f:srv_name()]`.
Issue#342
VTest can now enable mworker and mcli with separate flags so lets
update vtc files that need it. This also allows to revert the change
made with 1545a59c ("REGTESTS: make seamless-reload depend on 1.9
and above").
Since latest updates, vtest requires the master CLI when running in
master-worker mode, and this one is only available starting with 1.9.
The seamless reload test is the only one depending on this and now
fails on 1.8, so let's adjust it accordingly.
Previously an expression like:
path,field(2,/) -m found
always returned `true`.
Bug exists since the `field` converter exists. That is:
f399b0debf
The fix should be backported to 1.6+.
This test launches a HAProxy process in master worker with 'nbproc 4'.
It sends a "show info" to the process 3 and verify that the right
process replied.
This regtest depends on the support of the master CLI for VTest.
Absolute path must be used, otherwise, the requests are rejected by HAProxy
because of the recent changes. In addition, the configuration has been slightly
updated to remove warnings at startup.
This test occasionally fails on the Travis CI tests because the
"in progress" bit is sometimes still set (or set again) in the show
servers state output and is not expected in all regexes (some do
already cover it), like in this one :
https://travis-ci.com/haproxy/haproxy/jobs/221324920
Let's extend the remaining ones to accept this as well. Other tests
do not seem affected as they only expect sequences of digits there.
This patch excludes freebsd, osx and generic targets for this vtc.
Basic tcp checks performed by haproxy on a linux system leverage the
TCP_QUICKACK option which implies that the connection is never
established from the perspective of the backend server. On other systems
a regular tcp 3 way handshake is performed immediately followed by a
reset, which from the perspective of the server is an aborted connection.
When we run this regtest on FreeBSD (or anything other than linux) there
is a race condition in the server_thread() function of the vtc_server.c
file. If we receive the reset when we are in accept() then fd is -1 and
vtest calls vtc_fatal, failing the test.
Other checks specific reg-tests were excluded on FreeBSD, osx and
generic for the same reason, but were at the time documented as being
disabled because they used TCP_DEFER_ACCEPT. These commits are
15685c791 ("REGTEST: Exclude freebsd target for some reg tests") and
03c6ab0cb ("REGTEST: exclude osx and generic targets for
40be_2srv_odd_health_checks")
This test uses two sets of tcp-check connect port rules, with one
of the two ports being closed and expects the check to fail for both
backends at different steps. It aims at detecting regressions such as
the one fixed by 7df8ca62 (BUG/MEDIUM: tcp-check: unbreak multiple
connect rules again).
Make HAProxy set the `Vary: Accept-Encoding` response header if it compressed
the server response.
Technically the `Vary` header SHOULD also be set for responses that would
normally be compressed based off the current configuration, but are not due
to a missing or invalid `Accept-Encoding` request header or due to the
maximum compression rate being exceeded.
Not setting the header in these cases does no real harm, though: An
uncompressed response might be returned by a Cache, even if a compressed
one could be retrieved from HAProxy. This increases the traffic to the end
user if the cache is unable to compress itself, but it saves another
roundtrip to HAProxy.
see the discussion on the mailing list: https://www.mail-archive.com/haproxy@formilux.org/msg34221.html
Message-ID: 20190617121708.GA2964@1wt.eu
A small issue remains: The User-Agent is not added to the `Vary` header,
despite being relevant to the response. Adding the User-Agent header would
make responses effectively uncacheable and it's unlikely to see a Mozilla/4
in the wild in 2019.
Add a reg-test to ensure the behaviour as described in this commit message.
see issue #121
Should be backported to all branches with compression (i.e. 1.6+).
This adds a converter for the SHA-2 family, supporting SHA-224, SHA-256
SHA-384 and SHA-512.
The converter relies on the OpenSSL implementation, thus only being available
when HAProxy is compiled with USE_OPENSSL.
See GitHub issue #123. The hypothetical `ssl_?_sha256` fetch can then be
simulated using `ssl_?_der,sha2(256)`:
http-response set-header Server-Cert-FP %[ssl_f_der,sha2(256),hex]
With this new reg test we ensure the server by names stickiness is functional
between servers organized differently (with identical names, but with different IDs)
among two haproxy processes backends.
Even after commit 1bbc74b55 ("REGTEST: fix tls_health_checks random
failures on MacOS in Travis-CI") which extended it to 100ms, it still
randomly fails on Travis, so let's push it to 500ms and mark it slow.
Since commit 2eb1c79df ("REGTEST: make the tls_health_checks test much
faster") the build tests randomly fail on MacOS on Travis-CI. Each time
this test is reponsible for the failure, showing huge response times
possibly indicating that the VMs running the tests are sometimes
overloaded. Since this delay directly impacts the whole regtest execution
time everywhere, it's important not to inflate it too much. It was bumped
to 100ms instead of 40, that doesn't add significantly to the perceived
execution time and should be enough for Travis since test reports have
shown around 60-70 ms.
In this reg test, as the client connection is not supposed to receive any
server response, we should try to "rxresp" but we should expect the client
connection to be closed by haproxy. This is done replacing "rxresp" by
"expect_close". Furthermore since dbb75ee3 vtest commit, calling "rxresp"
expects at least to receive a HTTP header as shown by Travis build
here: https://travis-ci.com/haproxy/haproxy/jobs/198126488.
Fix a wrong reg test file renaming which came with d7a8f14 commit
(REGTEST: rename the reg test files). This prevented
reg-tests/log/wrong_ip_port_logging.vtc with "bug" as reg test type
from being run.
This test relies on a server timeout and was using the default 2s check
interval with a full 1s server timeout, thus adding a whole second to the
test series by itself. Let's shrink the server timeout to 20ms which is
way enough to properly trigger a timeout, and set the check interval to
the double of this, or 40ms.
This is a reg test for the log load-balancing feature implemented by
these commits:
MINOR: log: Add "sample" new keyword to "log" lines
MINOR: log: Enable the log sampling and load-balancing feature
The size of the logging buffer for vtest has been doubled to support this script.
In Travis build https://travis-ci.com/haproxy/haproxy/jobs/195477767 we
can see that OSX tends to pad zeroes at a different position than Linux
in compact IPv6 addresses, resulting in a failure in the checks which
were developped on Linux. This patch uses [0:]* in holes and [0:]+ at the
end of addresses to allow the different variants. It will unfortunately
also accept impossible addresses but there is no reason that we have to
care about for such crap to be emitted.
As explained in the commit below, this test relies on TCP_DEFER_ACCEPT
which is not available everywhere, and as such fails on OSX as well :
15685c791 ("REGTEST: Exclude freebsd target for some reg tests.")
Some reg tests and their dependencies have been renamed. They may be
referenced by the .vtc files. So, this patch modifies also the references
to these dependencies.
This patch replaces LEVEL variable by REGTESTS_TYPES variable which is more
mnemonic and human readable. It is uses as a filter to run the reg tests scripts
where a commented #REGTEST_TYPE may be defined to designate their types.
Running the following command:
$ REGTESTS_TYPES=slow,default
will start all the reg tests where REGTEST_TYPE is defines as 'slow' or 'default'.
Note that 'default' is also the default value of REGTEST_TYPE when not specified
dedicated to run all the current h*.vtc files. When REGTESTS_TYPES is not specified
there is no filter at all. All the tests are run.
This patches also defines REGTEST_TYPE with 'slow' value for all the s*.vtc files,
'bug' value for al the b*.vtc files, 'broken' value for all the k*.vtc files.
checks/s00001.vtc needs support for "srvrecord" which came with 1.8 version.
peers/s_basic_sync.vtc and s_tls_basic_sync.vtc need support for "server"
keyword usage in "peers" section which came with 2.0 version.
To do so, the server does not send anything. Instead it waits 2ms before
closing. The client, on its side, will wait for a response. So it will be
blocked. Becauase the client timeout is set to 1ms, HAProxy should always close
the client connection because it times out.
Because HAProxy may decide to close 301 responses, as others internal responses,
it is safer to use a different client for these requests. This is not the
purpose of this test to verify the keep-alive in such cases.
The way unexpected bodies are handled for responses to HEAD requests differs from
the legacy HTTP to the HTX. While it is dropped wih the legacy HTTP, in HTX, it
is parsed as the response to the next request. So, in HTX, a 502 error is
returned to the client and the connexion is closed.
This test has been modified to pass in both mode.
Size reported in logs may differ between legacy HTTP and HTX, at least for
now. So in the regtest http-capture/h00000.vtc, we need to relax the regex
matching the log message.
Lua test files 2 and 3 fail when threads are disabled because of a
"nbthread" statement that seems to be a leftover from an ancient
configuration. One of them even mentions a commit message showing
a reproducer not involving threads. Let's clean this up so that
running the tests without threads also works.
This should be backported to 1.9 as the problem also exists there.
These reg tests have been disabled because they required a version of vtest
including a bug fix supposed to make these ones work without breaking others.
But reg-tests for compression were broken.
This issue has been fixed by 525ef0f vtest commit. So, to make all the
reg tests work you must update your vtest program to include 525ef0f commit.
(see https://github.com/vtest/VTest/commit/525ef0f for more information.
This reverts commit 47e4e13c01.
It's a temporary revert. This commit suggested to update to vtest
commit 4e43cc1 to fix handling of HEAD requests, but the compression
was broken two commits before, leaving us with no single version of
vtest being able to run all tests anymore.
Let's temporary disable HEAD again in the tests so that we can use
any version up to and including a2e82a8 for the time it takes vtest
to fix the compression.
This patch enables the part of this reg test which could not work due to a vtest
(formerly varnishtest) bug.
NOTE: You must have a vtest version with 4e43cc1 commit for this bug fix to make this
script succeed (see 4e43cc1fec
for more information).
Commit 26f6ae12c ("MAJOR: config: disable support for nbproc and nbthread
in parallel") revealed that there was accidently nbproc+nbthread in this
test while nbproc is the one expected. This likely is a leftover from a
previous attempt at reproducing the issue.
RFC 7232 section 2.3.3 states:
> Note: Content codings are a property of the representation data,
> so a strong entity-tag for a content-encoded representation has to
> be distinct from the entity tag of an unencoded representation to
> prevent potential conflicts during cache updates and range
> requests. In contrast, transfer codings (Section 4 of [RFC7230])
> apply only during message transfer and do not result in distinct
> entity-tags.
Thus a strong ETag must be changed when compressing. Usually this is done
by converting it into a weak ETag, which represents a semantically, but not
byte-by-byte identical response. A conversion to a weak ETag still allows
If-None-Match to work.
This should be backported to 1.9 and might be backported to every supported
branch with compression.
This is mandated by RFC7541#8.1.2.6. Till now we didn't have a copy of
the content-length header field. But now that it's already parsed, it's
easy to add the check.
The reg-test was updated to match the new behaviour as the previous one
expected unadvertised data to be silently discarded.
This should be backported to 1.9 along with previous patch (MEDIUM: h2:
always parse and deduplicate the content-length header) after it has got
a bit more exposure.
This regtest verifies that the stats webpage can be used to change a
server state to maintenance or drain, and that filtering the page scope
will result in a filtered page.
This script runs two tests. One with "httpchk" over SSL/TLS and another
one with "check-ssl" option. As varnishtest does not support SSL/TLS
we use two haproxy processes to run these tests. h2 haproxy process
be2 and be4 backends declare one server each wich are the frontend
of h1 haproxy process. We check the layer6/7 checks thanks to syslog
messages.
Signed-off-by: Frdric Lcaille <flecaille@haproxy.com>
This test verifies the mailers section works properly by checking that
it sends the proper amount of mails when health-checks are changing and
or marking a server up/down
The test currently fails on all versions of haproxy i tried with varying
results:
- 1.9.0 produces thousands of mails.
- 1.8.14 only sends 1 mail, needs a 200ms 'timeout mail' to succeed
- 1.7.11 only sends 1 mail, needs a 200ms 'timeout mail' to succeed
- 1.6 only sends 1 mail, (does not have the 'timeout mail' setting implemented)
These ones are not needed anymore since commit 97aaa67 ("MINOR: mux-h2:
only increase the connection window with the first update"). The tests
should now be more reliable. It might be worth simply removing all the
explicit handshake though it doesn't hurt and still serves as documentation.
This script tests the "cookie <name> insert indirect" directive with
header checks on server and client side. syslog messages are also
checked, especially --II (invalid, insert) flags logging.
Signed-off-by: Frdric Lcaille <flecaille@haproxy.com>
These tests upload contents and randomly make the server start to
respond before the client finishes to upload data, making the test
occasionally fail. Waiting for a body in the server doesn't always
work, depending on the method or how the data are advertised. Thus,
let's ask haproxy to wait for the request using the aforementioned
option, it guarantees that the DATA frame is sent before the response
HEADERS frame is delivered.
These tests send GET/HEAD/POST requests in H1 and H2, with and without
HTX, with and without a body, and verify that the behaviour is the expected
one. For now HEAD requests have been commented out because in H1 they are
not really testable as varnishtest expects to read a body, and in H2 the
behaviour depends on HTX/legacy, indicating a bug in haproxy (it looks
like we can deliver some data in response to HEAD in legacy mode).
With this test we check that the health-checks do not consume any connection on
the backend side.
Signed-off-by: Frdric Lcaille <flecaille@haproxy.com>
Varnishtest is not happy to see the window update come before the
settings ACK, as by default it expects exactly tx/rx/txack/rxack.
One workaround could consist in making haproxy send the WU after
the settings ACK but this would be a real hack as the preface is
already finished when sending this ack. Instead, let's make the
initial sequence explicit in the tests.
fix http-rules/h00000.vtc / http-rules/h00000.vtc as both 'bodylen' and
'body' are specified, these settings conflict with each other as they
both generate/present the body to send.
The HTTP rules test now runs an H1 and an H2 client. Since the H2 one
requires the "proto" directive on the bind line, a new file has been
created requiring version 1.9 and the previous one was marked as usable
below 1.9 so that it's skipped by default but still usable when testing
backports.
There is always a risk of breaking HTTP processing when performing certain
code changes. This test modifies a request's start line, uses variables,
adds and modifies headers, interleaves them with the start-line changes,
and makes use of different header formats involving duplicated names,
duplicated values, empty fields and spaces around values. These operations
are performed both in the frontend and in the backend, for both the request
and the response. A CRC is computed on the concatenation of all the values,
and the concatenations are sent as individual header fields as well to help
debugging when the test fails.
The test reliably works since 1.6, implying that the HTTP processing did
not change. It currently fails on HTX.
