Commit Graph

78 Commits

Author SHA1 Message Date
William Dauchy
4488434c97 BUG/MINOR: reg-tests: fix service dependency script
I badly tested my previous patch forgetting to remove the "+" testing
present in options, and not in services; the list of services do not
have any "+" at the beginning of each service

this patch is fixing commit aabde71332 ("MINOR:
reg-tests: add a way to add service dependency")

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-11 14:16:06 +01:00
William Dauchy
aabde71332 MINOR: reg-tests: add a way to add service dependency
I was looking at writing a simple first test for prometheus but I
realised there is no proper way to exclude it if haproxy was not built
with prometheus plugin.

Today we have `REQUIRE_OPTIONS` in reg-tests which is based on `Feature
list` from `haproxy -vv`. Those options are coming from the Makefile
itself.

A plugin is build this way:
  EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"

It does register service actions through `service_keywords_register`.
Those are listed through `list_services` in `haproxy -vv`.
To facilitate parsing, I slightly changed the output to a single line
and integrate it in regtests shell script so that we can now specify a
dependency while writing a reg-test for prometheus, e.g:

  #REQUIRE_SERVICE=prometheus-exporter
  #REQUIRE_SERVICES=prometheus-exporter,foo

There might be other ways to handle this, but that's the cleanest I
found; I understand people might be concerned by this output change in
`haproxy -vv` which goes from:

  Available services :
          foo
          bar

to:

  Available services : foo bar

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-10 07:42:33 +01:00
Thayne McCombs
478e5ddc8d SCRIPTS: announce-release: fix typo in help message
s/relase/release in -p help message.
2021-01-08 14:58:22 +01:00
Willy Tarreau
96d5368202 SCRIPTS: make announce release support preparing announces before tag exists
It takes so much time to write an announce message that sometimes it's
annoying not being able to start the work while a fix is being finished.
With the new "-p" argument, announce-release will allow to prepare the
announce message for the current HEAD and with no tag yet. It will
restart from the last tag and automatically increment the version using
the same algorithm as create-release so that everything is accurate. It
should then be easier at the last moment to just include the final entry
by hand when the last fix finally arrives. For convenience, this argument
also allows to create an announce from another branch than master.
2021-01-06 15:49:44 +01:00
Willy Tarreau
0c612936b2 SCRIPTS: improve announce-release to support different tag and versions
By having three variables it will be easier to preset the version and
the tag separately. One contains the announced version, another one the
associated tag and the last one the final commit ID (used as the ending
point before the release). This initially allows to check for the HEAD
matching the tag only when the version was not forced, hence re-announce
already tagged versions after some extra commits were added for example.
2021-01-06 15:45:15 +01:00
Willy Tarreau
a4009cd610 REGTESTS: make use of HAPROXY_ARGS and pass -dM by default
Enabling memory poisonning is often pretty effective for detecting
uninitialized structure fields. Let's enable it by default and let
the user change the arguments at will (e.g. forcing some memory limits
or disabling a poller). This will work with the latest vtest version
to date (02a9bc1).
2020-12-16 21:43:50 +01:00
Ilya Shipitsin
6b736b4476 CI: travis-ci: replace not defined SSL_LIB, SSL_INC for BotringSSL builds
after 73b520b958 variables SSL_LIB, SSL_INC
are not set, but still used by BoringSSL builds. That leads to error
(I wish we could stop on such errors) and using stock openssl instead
of boringssl
2020-10-11 21:12:33 +02:00
Ilya Shipitsin
7faeea9750 REGTESTS: use "command" instead of "which" for better POSIX compatibility
for example, "which" is not installed by default in Fedora docker image.
2020-09-26 10:12:48 +02:00
Willy Tarreau
1f927d1bc2 SCRIPTS: git-show-backports: emit the shell command to backport a commit
It's cumbersome to copy-paste a commit ID into another window after having
typed "git cherry-pick -sx", so let's have the suggested output format of
git-show prepare this line just before the subject line, it remains at a
stable position on the terminal when searching for "/^commit". One just
has to copy-paste the line into another terminal will result in the commit
being properly picked.
2020-07-31 16:57:35 +02:00
Willy Tarreau
f456f6f2a3 SCRIPTS: git-show-backports: make -m most only show the left branch
We've never used the output of the rightmost branch with this tool,
and it systematically causes two identical outputs making the job
harder during backport sessions. Let's simply remove the right part
when it's identical to the left one. This also adds a few line feeds
to make the output more readable.
2020-07-31 16:57:09 +02:00
Willy Tarreau
be789dfc5d SCRIPTS: announce-release: add the link to the wiki in the announce messages
Let's add the link to the wiki to the announce messages, plenty of
users don't even know it exists.
2020-07-30 17:41:42 +02:00
Ilya Shipitsin
aaa34ea622 CI: travis-ci: switch BoringSSL builds to ninja
using ninja instead of make speed up build by 40 sec
2020-06-26 11:26:26 +02:00
Willy Tarreau
d678805783 REORG: include: move version.h to haproxy/
Few files were affected. The release scripts was updated.
2020-06-11 10:18:56 +02:00
Willy Tarreau
6fab3e6d91 SCRIPTS: publish-release: pass -n to gzip to remove timestamp
It just appeared that the tar.gz we put online are not reproducible
because a timestamp is put by default into the archive. Passing "-n"
to gzip is sufficient to remove this timestamp, so let's do it, and
also make the gzip command configurable for more flexibility. Now
issuing the commands multiple times finally results in the same
archives being produced.

This should be backported to supported stable branches.
2020-05-30 06:59:07 +02:00
Ilya Shipitsin
856aabcda5 CLEANUP: assorted typo fixes in the code and comments
This is 8th iteration of typo fixes
2020-04-17 09:37:36 +02:00
Ilya Shipitsin
6e18f92d4f CI: adopt openssl download script to download all versions
with recent change, OpenSSL download URL was changed in
incompatiable way. i.e. only the most recent openssl version
might be downloaded using previous script.

older versions are available under different URLs. as we need
several openssl versions, let us adopt script accordingly.

bug was caught after travis-ci cache was purged for some reason.
2020-04-07 22:02:41 +02:00
Willy Tarreau
1392d029e6 SCRIPTS: announce-release: use mutt -H instead of -i to include the draft
Commit 0f5ce6014a ("SCRIPTS: announce-release: place the send command
in the mail's header") broke the announce-release script: by not having
to edit the message at all anymore, mutt does nothing when sending, but
it still does if the message is edited (which was the case before). With
some testing, it appears that mutt -H does work when there's no change,
so let's use this instead. This should be backported till 1.7.
2020-02-15 15:24:28 +01:00
Willy Tarreau
332ded5f3a SCRIPTS: make announce-release executable again
I managed to mess up with the file's permission while using a temporary
one during last release, and to backport the non-exec version everywhere.
This can be backported as far as 1.7 now.
2020-02-12 18:21:11 +01:00
Ilya Shipitsin
47e09dd034 BUILD: scripts/build-ssl.sh: use "uname" instead of ${TRAVIS_OS_NAME}
it is also useful for local builds, when ${TRAVIS_OS_NAME} is not set
2020-02-12 15:42:44 +01:00
Willy Tarreau
f9beea52da SCRIPTS: backport: fix the master branch detection
The condition was inverted. When the branch was the master, it was
harmless because it caused an extra "checkout master", but when it
was not the master, the commit could be applied to the wrong branch
and it could even possibly not match the name to stop on.
2020-02-07 08:26:49 +01:00
Willy Tarreau
3823408b60 SCRIPTS: announce-release: allow the user to force to overwrite old files
When starting the script multiple times, one had to remove the previous
files by hand. Now with -f it's not needed anymore, they get removed.
2020-02-07 08:11:45 +01:00
Willy Tarreau
0f5ce6014a SCRIPTS: announce-release: place the send command in the mail's header
I'm fed up with having to scroll my terminals trying to look for the
mail send command printed 30 minutes before the release, let's have
it copied into the e-mail template itself, and replace the old headers
that used to be duplicated there and that are not needed anymore.
2020-02-07 08:10:06 +01:00
Willy Tarreau
499b298ce0 SCRIPTS: backport: use short revs and resolve the initial commit
I find myself often getting trapped into calling "backport 2.0 HEAD" which
doesn't work because "HEAD" is passed as the argument to cherry-pick in
other repos. Let's resolve it first. And also let's shorten the commit IDs
to make the error messages more readable and to ease copy-paste.
2020-02-06 18:38:19 +01:00
Willy Tarreau
be9b00f992 SCRIPTS: use /usr/bin/env bash instead of /bin/bash for scripts
Given that some OSes have bash in /usr/local/bin and in order not to
give too easy an excuse to Olivier for not backporting fixes, let's
make a few scripts rely on /usr/bin/env bash instead of /bin/bash :-)
2020-02-05 04:49:07 +01:00
Willy Tarreau
e77a13aa3f SCRIPTS: add a new "backport" script to simplify long series of backports
The script is simply called from the repository holding the patch to
backport, with the last branch number and the commit(s) ID(s) to send
there and it then follows the chain of "down" repos to go down one step
until it meets the indicated last one. It basically automates what we do
by hand. Example:

   ./scripts/backport 1.9 1c7c0d6b97

Note that it does *not* push, which still has to be done by hand after
building and testing.
2020-02-04 13:50:36 +01:00
William Lallemand
d5b464bfee REGTEST: run-regtests: implement #REQUIRE_BINARIES
Implement #REQUIRE_BINARIES for vtc files.

The run-regtests.sh script will check if the binary is available in the
environment, if not, it wil disable the vtc.
2019-12-19 14:36:46 +01:00
Christopher Faulet
1eee6ca89e REGTEST: Add an HTX reg-test to check an edge case
This test checks that an HTTP message is properly processed when we failed to
add the HTX EOM block in an HTX message during the parsing because the buffer is
full. Some space must be released in the buffer to make it possible. This
requires an extra pass in the H1 multiplexer. Here, we must be sure the mux is
called while there is no more incoming data.

It is a "devel" test because conditions to run the test successfully is highly
dependent on the implementation. So if it fail, it is not necessarily a bug. It
may be due of an internal change. It relies on internal HTX sample fetches.
2019-12-11 16:46:16 +01:00
Willy Tarreau
7e8c016506 SCRIPTS: update create-release to fix the changelog on new branches
The changelog is empty when creating a dev0 version and this confuses
the commit message, let's clearly mention the exact copy when there are
no changes.
2019-11-25 20:40:52 +01:00
Willy Tarreau
1a3af78302 SCRIPTS: git-show-backports: add "-s" to proposed cherry-pick commands
Since we're using signed-off-by tags for backports, let's add -s to
the command so that we can finally copy-paste it!
2019-11-25 15:51:47 +01:00
Willy Tarreau
ff0c8424c8 SCRIPTS: create-release: show the correct origin name in suggested commands
create-release shows the next steps at the end and suggest to use
"git push origin master" but on my machine it's not "origin" so let's
determine it using git config and only use origin as a fall back.
2019-11-25 15:49:31 +01:00
Ilya Shipitsin
8abf026aff BUILD: CI: install golang-1.13 when building BoringSSL 2019-09-17 13:52:39 +02:00
Willy Tarreau
a8ee4b199f CLEANUP: removed obsolete examples an move a few to better places
The following example files awere removed as irrelevant by this
time :
  auth.cfg check.conf ssl.cfg haproxy.spec

The following scripts were removed as having been unused for more
than a decade :
  debug2ansi debug2html debugfind check init.haproxy stats_haproxy.sh

seemless_reload.txt was moved to doc/ where it's more suitable.

haproxy.vim was moved to contrib/syntax-highlight/

scripts/create-release was updated not to try to update haproxy.spec
anymore.
2019-06-15 21:25:06 +02:00
Ilya Shipitsin
a088d3dea9 BUILD: travis-ci improvements
full list:

update LibreSSL to 2.9.2
speed up build by using "make -j3"
cache BoringSSL checkout
build prometeus exporter
add basic cygwin build
add USE_TFO=1, USE_SYSTEMD=1 to linux builds
2019-06-05 10:02:42 +02:00
Bertrand Jacquin
7e30b50417 DOC: fix "successful" typo
s/succesfull/successful at a few places
2019-05-18 08:25:29 +02:00
Ilya Shipitsin
35d20afe6f BUILD: add BoringSSL to travis-ci build matrix 2019-05-10 10:39:29 +02:00
Ilya Shipitsin
db6e4beea7 BUILD: remove "build_libressl" duplicate declaration 2019-05-06 07:25:51 +02:00
Ilya Shipitsin
054a5b82c1 BUILD: extend travis-ci matrix
added openssl-1.0.2, 1.1.0, 1.1.1, libressl-2.7.5, 2.8.3, 2.9.1
added linux-ppc64le image
2019-05-05 10:16:13 +02:00
Frédéric Lécaille
85a7ea0740 REGTEST: Add a new reg test for log load-balancing feature.
This is a reg test for the log load-balancing feature implemented by
these commits:
  MINOR: log: Add "sample" new keyword to "log" lines
  MINOR: log: Enable the log sampling and load-balancing feature

The size of the logging buffer for vtest has been doubled to support this script.
2019-04-30 09:25:09 +02:00
Willy Tarreau
ca8df4c074 REGTEST: make the "run-regtests" script search for tests in reg-tests by default
It happens almost daily to me that make regtests fails because the script
found a temporary, old, or broken VTC file that was lying in my work dir,
leaving me no place to hide it. This is a real pain as some tests take ages
to fail, so let's make this script only look up for tests where they are
expected to be stored, under reg-tests only. It remains possible to force
the location on the command line though.
2019-04-23 16:09:50 +02:00
Frédéric Lécaille
dc1a3bd999 REGTEST: replace LEVEL option by a more human readable one.
This patch replaces LEVEL variable by REGTESTS_TYPES variable which is more
mnemonic and human readable. It is uses as a filter to run the reg tests scripts
where a commented #REGTEST_TYPE may be defined to designate their types.
Running the following command:

    $ REGTESTS_TYPES=slow,default

will start all the reg tests where REGTEST_TYPE is defines as 'slow' or 'default'.
Note that 'default' is also the default value of REGTEST_TYPE when not specified
dedicated to run all the current h*.vtc files. When REGTESTS_TYPES is not specified
there is no filter at all. All the tests are run.

This patches also defines REGTEST_TYPE with 'slow' value for all the s*.vtc files,
'bug' value for al the b*.vtc files, 'broken' value for all the k*.vtc files.
2019-04-23 15:14:52 +02:00
Christopher Faulet
78f31bf07a REGTEST: Use HTX by default and add '--no-htx' option to disable it
Because the HTX is now the default mode in HAProxy, it is also enabled by
default in reg-tests. The option '--use-htx' is still available, but deprecated
and have concretly no effect. To run reg-tests with the legacy HTTP mode, you
should use the option '--no-htx'.
2019-04-12 22:06:53 +02:00
Christopher Faulet
1d2b586cdd MAJOR: htx: Enable the HTX mode by default for all proxies
The legacy HTTP mode is no more the default one. So now, by default, without any
option in your configuration, all proxies will use the HTX mode. The line
"option http-use-htx" in proxy sections are now useless, except to cancel the
legacy HTTP mode. To fallback on legacy HTTP mode, you should use the line "no
option http-use-htx" explicitly.

Note that the reg-tests still work by default on legacy HTTP mode. The HTX will
be enabled by default in a futur commit.
2019-04-12 22:06:53 +02:00
Willy Tarreau
ad25c2658e REGTEST: script: remove platform-specific assigments of OPTIONS
We don't use OPTIONS anymore, let's simply remove all the code that sets
this variable. It was not viable anyway to keep this one in sync with
the makefile.
2019-03-27 14:47:55 +01:00
Willy Tarreau
87586e1595 REGTEST: script: make the script use the new features list
Currently the script only sees options that differ from the default,
which makes it fail on default options that are disabled (such as
threads on the relevant platforms). Let's make it instead extract
the newly introduced feature list and search for an explicit "+" in
front of the desiered feature. This one is known for always being
valid.
2019-03-27 14:47:55 +01:00
Willy Tarreau
9589c3bce7 SCRIPTS: add the issue tracker URL to the announce script
This way it's easier for users to follow the status of pending issues
with each release.
2019-01-29 06:51:16 +01:00
Frédéric Lécaille
4336123b5c REGTEST: Switch to vtest.
This patch replace the usage of the formerly varnish cache reg
testing program, name varnishtest by the new standalone one: vtest.
2019-01-14 14:21:13 +01:00
Frédéric Lécaille
f9a48ef791 REGTEST: Add some informatoin to test results.
When the reg tests fail, it may be useful to display additional information
coming from varnishtest, especially when this latter aborts.
In such case, the test output may be made of lines prefixed by "*     diag"
string.
2019-01-08 11:43:35 +01:00
Christopher Faulet
7e245dfaca REGTEST: script: Add support of alternatives in requited options list
It is now possible to specify a list of "alternatives" for a required
option. This must be done by separating options by a pipe ('|'). A test will be
executed if at least one of them is available. For instance:

  #REQUIRED_OPTIONS=ZLIB|SLZ,LUA,OPENSSL

The function _findtest() has also been sligthly simplified.
2018-12-21 15:33:26 +01:00
Christopher Faulet
6bd82cdbe6 REGTEST: script: Print only errors in the results report 2018-12-20 10:37:32 +01:00
Christopher Faulet
fa6798f97c REGTEST: script: Add the option --use-htx to enable the HTX in regtests
A check is done on HAProxy version to be sure it is supported. Internally, by
default, the macro 'no-htx' is set to "#". So in regtests, we can prefix
http-use-htx option line with this macro to inhibit the HTX. Concretly, this
line can be added in the HAProxy configuration:

   ${no-htx} option http-use-htx

When the option --use-htx is used, the macro 'no-htx' is unset.
2018-12-20 10:37:32 +01:00