Commit Graph

47 Commits

Author SHA1 Message Date
Ilia Shipitsin ab23d7eb69 CI: speedup apt package install
we are fine to skip some repos like languages and translations.
this drops number of repos twice
2024-06-03 11:59:07 +02:00
Ilia Shipitsin f3e6dfdc92 CI: VTest: accelerate package install a bit
let's check and install only package is required
2024-05-30 17:04:08 +02:00
Ilia Shipitsin 11c088e203 CI: switch to lua 5.4
current release is 5.4, let's switch to it
2024-05-29 22:37:26 +02:00
Ilia Shipitsin 01c213a4bb CI: use "--no-install-recommends" for apt-get
this reduces number of packages installed by 1
2024-05-29 22:37:26 +02:00
Ilia Shipitsin cbe78c0281 CI: drop asan.log umbrella completely
asan.log redirection appeared to work poorly, let's cease that practice
for good.

ML: https://www.mail-archive.com/haproxy@formilux.org/msg44844.html
2024-05-13 11:36:36 +02:00
William Lallemand 378d500caf REGTESTS: use -dI for insecure fork by default in the regtest scripts
Let's remove the CI HAPROXY_ARGS setting and set -dI for anything run
with the run-regtests.sh.
2024-04-22 16:15:57 +02:00
Ilya Shipitsin 2bb9e3bd06 CI: reduce ASAN log redirection umbrella size
previously ASAN_OPTIONS=log_path=asan.log was intended for VTest
execution only, it should not affect "haproxy -vv" and hsproxy
config smoke testing
2024-04-17 11:14:44 +02:00
Ilya Shipitsin b5dd8b466e CI: revert kernel addr randomization introduced in 3a0fc864
It has been resolved on image generation side:
https://github.com/actions/runner-images/issues/9491

It is no harm to keep it on our side as well, but we can drop it.
2024-04-13 14:43:36 +02:00
Willy Tarreau 6b6a6ad431 CI: update the build options to get rid of unneeded DEBUG options
Now that DEBUG_STRICT and DEBUG_MEMORY_POOLS are the default, we can
drop them from the build options.
2024-04-11 17:25:45 +02:00
Ilya Shipitsin 9dd9289cec CI: vtest: show coredumps if any
if any coredump	is found, it is	passed to gdb with
'thread apply all bt full'
2024-04-02 07:29:08 +02:00
Ilia Shipitsin 3a0fc8641b CI: temporarily adjust kernel entropy to work with ASAN/clang
clang runtime (shipped with clang14) is not compatible with recent
Ubuntu kernels

more details: https://github.com/actions/runner-images/issues/9491
2024-03-18 19:54:33 +01:00
William Lallemand 9c2e900a9b CI: github: add -dI to haproxy arguments
-dI is useful when running with ASAN, allow to fork addr2line
2024-03-13 11:23:14 +01:00
Ilya Shipitsin 6b54a7b65e CI: run more smoke tests on config syntax to check memory related issues
config syntax check seems add a value on testing code path not
covered by VTest, also checks are very fast
2024-03-05 11:50:34 +01:00
Tim Duesterhus 4559470728 CI: Update to actions/cache@v4
No functional change, but this upgrade is required, due to the v3 runtime being
deprecated:

> Node.js 16 actions are deprecated. Please update the following actions to use
> Node.js 20: actions/cache@v3. For more information see:
> https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
2024-02-10 05:12:20 +01:00
Tim Duesterhus 5c923f1869 CI: Update to actions/checkout@v4
No functional change, but we should keep this current.

see 5f4ddb54b0
2023-09-06 17:46:02 +02:00
Ilya Shipitsin f7dcceccc9 CI: explicitely highlight VTest result section if there's something
it turned out that people miss VTest result section because it is not highlighted,
let us fix that
2023-07-17 15:56:53 +02:00
Ilya Shipitsin 7e6e0388d6 CI: run smoke tests on config syntax to check memory related issues
config syntax check seems add a value on testing code path not
covered by VTest, also checks are very fast
2023-04-01 18:30:52 +02:00
Tim Duesterhus 96e03e1a89 CI: Unify the `GITHUB_TOKEN` name across matrix.py and vtest.yml
This makes naming a little clearer in matrix.py, because the name matches the
name of the actual secret.
2023-01-03 16:28:34 +01:00
William Lallemand 61839b098d CI: github: use the GITHUB_TOKEN instead of a manually generated token
Github allows to use a auto generated GITHUB_TOKEN so we don't need to
handle the token in the secret configuration.

https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret
2022-12-23 15:19:25 +01:00
William Lallemand 2cb1493748 CI: github: set ulimit -n to a greater value
Set ulimit -n to 65536 to limit less the maxconn computation.

Could be backported at least to 2.5.
2022-12-14 00:31:19 +01:00
Ilya Shipitsin 4a04cd35ae CI: github: split ssl lib selection based on git branch
when *SSL_VERSION="latest" behaviour was introduced, it seems to be fine
for development branches, but too intrusive for stable branches.

let us limit "latest" semantic only for development builds, if branch name
contains "haproxy-" it is supposed to be stable branch, no latest openssl
should be taken

[wla: must be backported as far as 2.6]
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2022-12-12 16:20:48 +01:00
Willy Tarreau a051816c03 CI: emit the compiler's version in the build reports
Some occasional builds fail only on a specific platform and being able
to figure the exact compiler version used there is crucial. It's not
easy to guess from the rest of the output, so let's add it before the
platform-specific defines, which suit the same needs.
2022-11-14 11:14:02 +01:00
Tim Duesterhus b87ecbb179 CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in workflow definition
See "CI: Replace the deprecated `::set-output` command by writing to
$GITHUB_OUTPUT in matrix.py" for the reasoning behind this commit.
2022-10-14 19:50:34 +02:00
Tim Duesterhus 538d8fe8b7 CI: Update to actions/cache@v3
No functional changes for our use case, but we should keep this current.
2022-04-11 07:13:24 +02:00
Tim Duesterhus 5f4ddb54b0 CI: Update to actions/checkout@v3
No functional change, but we should keep this current.
2022-04-11 07:13:24 +02:00
Willy Tarreau a0a6911bde CI: github: enable pool debugging by default
This enables DEBUG_MEMORY_POOLS and DEBUG_POOL_INTEGRITY so that by
default the tests run under stricter checks, which are likely to
catch more bugs. Note that these ones are permanently used in prod
on haproxy.org.
2022-02-23 18:14:49 +01:00
Ilya Shipitsin 27df87cc63 CI: github actions: use cache for SSL libs
we have two kinds of SSL libs built - git based and version based.
this commit introduces caching for version based SSL libs.
2022-01-25 12:02:08 +01:00
Ilya Shipitsin e9efc3a5be CI: refactor OpenTracing build script
re-use scripts/build-ot.sh in CI again. Bump opentracing-cpp to 1.6.0
2022-01-19 07:37:40 +01:00
Ilya Shipitsin b9e3fb7315 CI: github actions: use cache for OpenTracing
this caches OpenTracing libs between builds, should save couple of minutes
for each build.
2022-01-13 13:37:34 +01:00
Ilya Shipitsin 65eab587a2 CI: github actions: clean default step conditions
step condition "if: ${{ !failure() }}" was added in 2ef4c7c843
during my experiments. As Tim Düsterhus mentioned, that condition is default and may be omitted.
2022-01-11 15:00:55 +01:00
Ilya Shipitsin 2ef4c7c843 CI: Github Actions: do not show VTest failures if build failed
this is mostly cleanup, issue is minor. If build failed, VTest execution
tried to be performed as well as VTest result show. This change ignores
those steps if build failed.
2021-12-25 15:09:06 +01:00
Willy Tarreau 4673c5e2c8 CI: github actions: add the output of $CC -dM -E-
Sometimes figuring what differs between platforms is useful to fix
build issues, to decide what ifdef to add for example. Let's always
call $CC -dM -E- before starting make.
2021-11-26 17:58:42 +01:00
Tim Duesterhus 89c9d0a169 CI: Add `permissions` to GitHub Actions
This change locks down the permissions of the access token in GitHub Actions to
only allow reading the repository contents and nothing else.

see https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
2021-10-18 07:17:04 +02:00
Tim Duesterhus 8a91fa4219 CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'
Using 'sudo' required quite a few workarounds in various places. Setting an
explicit 'ulimit -n' removes the requirement for 'sudo', resulting in a cleaner
workflow configuration.
2021-06-17 14:57:43 +02:00
Ilya Shipitsin 75c95aa649 CI: github actions: add OpenTracing builds
let us add OpenTracing module to "all features" builds
2021-05-18 18:37:37 +02:00
Ilya Shipitsin 0b03895620 CI: introduce scripts/build-vtest.sh for installing VTest
we install VTest for all CI systems, let us unify instalation
2021-05-18 10:48:30 +02:00
Tim Duesterhus b38b5c3d0d Revert "CI: Build VTest with clang"
The issue with VTest not building properly in gcc is fixed since commit
vtest/VTest@0730540c43. Revert the patch to keep
the CI configuration simple.

This reverts commit e61f53eb44.
2021-05-12 21:28:08 +02:00
Tim Duesterhus e61f53eb44 CI: Build VTest with clang
Current VTest master fails to build using gcc, see vtest/VTest#27.

This patch is to be reverted once VTest is fixed.
2021-05-10 23:00:59 +02:00
Willy Tarreau aca389a483 CI: github: do not build libslz any more
As hinted by Tim, it's not needed any more since it's now integrated,
let's get rid of this step.
2021-04-22 16:10:32 +02:00
Willy Tarreau 57610c694e CONTRIB: move src/wurfl.c and contrib/wurfl to addons/wurfl
Both the source file and the dummy library are now at the same place.
Maybe the build howto could be moved there as well to make things even
cleaner.

The Makefile, MAINTAINERS, doc, and vtest matrix were updated.
2021-04-02 17:48:42 +02:00
Tim Duesterhus 2ca0bbc7d8 Revert "CI: Pin VTest to a known good commit"
The issue with VTest now is fixed in VTest master since commit
vtest/VTest@040bb6737a.

This reverts commit 24105300b8.
2021-03-03 05:07:10 +01:00
Tim Duesterhus 24105300b8 CI: Pin VTest to a known good commit
As of January, 11th the macOS builds fail due to regression introduced in
VTest. This patch pins VTest to the newest good commit.

This patch should be reverted once VTest's 'master' is stable again.

see vtest/VTest#26
2021-01-20 19:23:57 +01:00
Tim Duesterhus 9fee7e02d1 CI: Set DEBUG=-DDEBUG_STRICT=1 in GitHub Actions
This was missing when migrating from Travis.
2020-11-21 18:27:33 +01:00
Tim Duesterhus 8d173e17c0 CI: Pass the github.event_name to matrix.py
This is a preparation to later run some matrix entries on schedule only.

Within the matrix.py script it can now be detected whether the workflow is
running on schedule by using:

    if build_type == "schedule":
        matrix.append(...)
2020-11-21 11:05:16 +01:00
Ilya Shipitsin e48853aaf4 CI: Github Action: run "apt-get update" before packages restore
ubuntu somehow needs it, no idea why does not apt-get do it itself.
2020-11-21 10:44:16 +01:00
Tim Duesterhus fa1747dc7d CI: Stop hijacking the hosts file
vtest/VTest#24 is merged now. This step is no longer required.
2020-11-17 11:55:07 +01:00
Tim Duesterhus 288c0772e6 CI: Expand use of GitHub Actions for CI
Travis is becoming overall increasingly unreliable lately. We've already
seen that the timing sensitive tests regularly fail and thus they were
disabled.

Additionally they recently announced a new pricing model that caps the number
of minutes for Open Source projects:
https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

GitHub Actions VMs are working well, possibly allowing to use custom runners
for special tasks in the future.

In addition to this better performance its workflow configuration language
is more expressive compared to the Travis CI one. Specifically the build
matrix does not need to be specified in YAML. Instead it can be generated
ad-hoc using a script. This allows us to cleanly define the various build
configurations without having an unreadable 80 line mess where the flags
are inconsistently activated. As an example in the current Travis CI
configuration the prometheus exporter is tested together with LibreSSL 2.9.2
for whatever reason.

In addition to all the previous points the UI of Travis is not that nice.
On GitHub you are just seeing that "Travis failed" without any details which
exact job failed. This requires you to visit the slow Travis page and look
up the details there. GitHub Actions creates a single entry for each
configuration that is tested, allowing you to see the details without needing
to leave GitHub.

This new GitHub Actions workflow aims to reproduce the configurations tested
in Travis. It comes close, but is not completely there, yet. Consider this
patch a proof of concept that will evolve in the future, ideally with Ilya's
expertise.

The current configurations are as follows. Each one is tested with both gcc
and clang.
- All features disabled (no USE flags)
- All features enabled (all USE flags)
- Standalone test of each of the supported compression libraries:
  - USE_ZLIB=1
  - USE_SLZ=1
- Standalone test of various SSL libraries:
  - stock (the SSL installed by default on the VM)
  - OpenSSL 1.0.2u
  - LibreSSL 2.9.2, 3.0.2, 3.1.1
- All features enabled with ASAN (clang only)

Future additions of new tests should take care to not test unrelated stuff.
Instead a distinct configuration should be added.

Additionally there is a Mac OS test with clang and all features disabled.

Known issues:
- Apparently the git commit is not properly detected during build. The HEAD
  currently shows as 2.4-dev0.
2020-11-10 10:15:17 +01:00