Commit Graph

55 Commits

Author SHA1 Message Date
David Sterba 71d4768ad0 btrfs-progs: ci: run lowmem check tests in CI images
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:34:00 +02:00
David Sterba e8542843ae btrfs-progs: ci: add pull request worfklows to cleanup list
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:57 +02:00
Josef Bacik 8c2e4578cd btrfs-progs: ci: add dwarves to the package list for docker images
We run pahole for the ioctl-test, which requires the dwarves package.
Add that to the docker images.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:56 +02:00
David Sterba b2c4a9ea75 btrfs-progs: ci: add awk to Tumbleweed image
The CI image build workflow started to fail due to missing 'awk',
install it.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:46 +02:00
David Sterba 55bed0a8db btrfs-progs: ci: add cleanup script for coverage tests
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba b28ce934b0 btrfs-progs: ci: update maintenance scripts
- use proper json array filtering
- use long parameter names
- add comments
- add cleaner for coverage runs
- keep last run for branches with badges at README

Signed-off-by: David Sterba <dsterba@suse.com>
2023-07-27 15:42:40 +02:00
David Sterba ae1bf6d881 btrfs-progs: ci: add github actions admin scripts
Add helper scripts to build static binaries manually or clean old runs.
Requires the 'gh' command line tool and write access the git repository.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-26 18:02:27 +02:00
David Sterba ec841d1451 btrfs-progs: ci: add 32bit build test on alpine/musl
Add new CI image target based on 32bit Alpine with musl. It's a copy of
musl and requires docker's buildx extension to properly select the
platform.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-05-24 19:58:50 +02:00
David Sterba 7f0d756083 btrfs-progs: ci: remove travis CI scripts
The free tier of travis-ci has ended on 05/2021, remove the scripts.
Nowadays zstd has made it to all distros so we don't need to build
it from sources, reiserfs support is fading out so we don't need to
build it either. The integration with coverity scan needs to be done
again. The rest has been moved to new github actions CI.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-17 19:28:06 +02:00
David Sterba 4efcbf1781 btrfs-progs: ci: remove gitlab CI scripts
The gitlab CI was experimentally used for btrfs-progs testsuite in
nested VM with recent kernel (as the base CI image was too old). This
unfortunately did not work due to time limit for the CI run, the build
and tests haven't finished.

Gitlab supports self-hosted runners so this can be integrated
eventually.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-17 19:28:06 +02:00
David Sterba 71decdc5eb btrfs-progs: README: update CI status and links
Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-13 16:51:00 +02:00
David Sterba 5eb5ff2bdf btrfs-progs: ci: check if docker run has a terminal
Using 'docker run -it' works for interactive sessions but not inside the
CI environment. We want both so make it optional by detecting if stdin
is a terminal.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-12 18:43:15 +02:00
David Sterba cb0bf52edc btrfs-progs: ci: fix parameter parsing in image running scripts
There's an unconditional 'shift' in the scripts that will lead to an
error "cannot shift" and this would fail inside CI. We want an implicit
run on current HEAD so shift only when there are more parameters.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-04-12 18:27:19 +02:00
David Sterba b79b330f62 btrfs-progs: ci: enable crypto backends on alpine/musl image
Add the libaries and update script so additional configure parameters
can be passed. Also enable backtrace so it's closer to the defaults.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-03-01 16:58:29 +01:00
David Sterba 9981a6767d btrfs-progs: ci: enable crypto backends on Centos images
Add the libaries and update script so additional configure parameters
can be passed. Also enable backtrace so it's closer to the defaults.

Only the libsodium is enabled and verified to build, libgcrypt has
sufficient version on 8 but configure does not detect it for some
reason. Libkca is old on both.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-03-01 16:51:27 +01:00
David Sterba 223fa4ca01 btrfs-progs: ci: enable crypto backends on Leap image
Add the libaries and update script so additional configure parameters
can be passed. Also enable backtrace and libudev so it's closer to the
defaults.

Leap 15.3 properly builds only with libsodium, 15.4 with libgcrypt and
libsodium. Libkcapi is old on both.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-03-01 16:33:12 +01:00
David Sterba d8b66fe2e2 btrfs-progs: ci: enable crypto backends on Tumbleweed image
Add the libaries and update script so additional configure parameters
can be passed. Also enable backtrace and libudev so it's closer to the
defaults.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-03-01 16:10:53 +01:00
David Sterba 0a464d4155 btrfs-progs: ci: enable gcc 13 on tumbleweed
GCC 13 is now available, add it as default compiler to tumbleweed image
to catch new warnings.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba f629b5041b btrfs-progs: ci: update documentation
Write a quick start howto and point to the admin scripts.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba 256e64dfa8 btrfs-progs: ci: update and sync scripts
The main script ci/images/test-build should be most up to date and
copied to all docker templates, sync them. Fix descriptions that were
copy&pasted from musl. Add missing test-build script from musl image.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba cfc093bb6a btrfs-progs: ci: fix Centos 8 package urls
A fresh build of Centos 8 fails due to changes in upstream repositoris:

"Error: Failed to download metadata for repo ‘appstream’: Cannot prepare
internal mirrorlist: No URLs in mirrorlist"

There are repositories no longer updated (and missing some packages like
reiserfs-utils) but for our CI purposes it's sufficient so make it work.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba b473169d2d btrfs-progs: ci: drop reiserfs support from Tumbleweed image
The devel package for reiserfs is no longer available in openSUSE
Tumbleweed. Build auto-detects the support for convert.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba d4bc1bf290 btrfs-progs: ci: build test programs
The test programs are not build by default but we want at least the
build coverage, so add them. Also make the build parallel.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba 466e025c74 btrfs-progs: ci: fix image updater script
There's a typo in the way how parameter is passed and is a syntax error
for docker.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba b788f1dc58 btrfs-progs: ci: add helpers to update base images
Add convenience build scripts for updating all CI images.

    $ images-base-update
    $ images-build-all

Signed-off-by: David Sterba <dsterba@suse.com>
2022-02-01 18:41:40 +01:00
David Sterba b8f31fabfc btrfs-progs: ci: add Leap 15.4 for testing
Reuse 15.2 for 15.4.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-02-01 18:41:37 +01:00
David Sterba ad528264b8 btrfs-progs: ci: add run scripts for more targets
Add some scripts for convenience, so far there was only one for musl as
it usually breaks first, but we've had some problems on centos due to
old kernel headers and potential breakage when changing kerncpomat.h.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-02-01 18:41:35 +01:00
David Sterba e1377c073e btrfs-progs: ci: add missing docker scripts for musl image
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:56 +02:00
David Sterba 463ef8515c btrfs-progs: ci: add openSUSE Leap 15.3 image
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:53 +02:00
David Sterba 35d9b3266d btrfs-progs: ci: update base images with libudev-devel
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:51 +02:00
David Sterba 27615e18bc btrfs-progs: ci: disable libudev for musl build
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:50 +02:00
David Sterba 1907cd64db btrfs-progs: ci: add script to do build test on musl
Run ci/ci-build-musl to verify build of current branch works in
environment with musl libc. Also works for a given branch name.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-26 13:45:35 +02:00
David Sterba a8bc071d9e btrfs-progs: ci: list hosted CI requirements
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-12 13:03:55 +02:00
David Sterba 900d2ccfe4 btrfs-progs: ci: install static libs to Tumbleweed image
All the static versions of libraries are available on Tumbleweed, let's
add them to the image.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba d92d7bcd70 btrfs-progs: ci: install clang on all images
Extend build coverage. The versions are different on all images and can
be run as:

  $ ./docker-run --env CC=clang

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba b02d151bd1 btrfs-progs: ci: fix docker-run argument parsing
The runner script allows to pass arguments to docker and the final
command, using the -- separator. This did not work as expected, the
arguments got concatenated to the first member, not all of them passed.
The following now works:

  $ ./docker-run --env CC=clang
  $ ./docker-run --env CC=clang --
  $ ./docker-run --env CC=clang -- /bin/bash

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba e15526653d btrfs-progs: ci: fix package ordering for tumbleweed image
Move installation of gzip before autotools as it would otherwise pull
busybox-gzip (and busybox) and that causes problems later.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba c1da8b0ed5 btrfs-progs: ci: disable travis-ci.org
The free travis-ci.org service is going to be discontinued. The
replacement travis-ci.com could be used instead but is not exactly the
same.

The images provided by the service contained old kernel and this hinders
testing of new features, tests were failing and the coverage was
incomplete. The docker images will be used to do build coverage in
another way.  A hosted CI is still desired so the search continues.

Issue: #171
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba 3886c81fb5 btrfs-progs: ci: add docker build and run script
Add scripts to build and run the musl image, no runtime tests, we want
to verify just the build.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba adeeb80dfe btrfs-progs: ci: disable zoned mode where not working
The support for zoned mode is incomplete and won't change so we can
disable it on Leap 15.2 and Centos 8.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:21 +02:00
David Sterba 66931220e2 btrfs-progs: ci: fix symlink paths of helper scripts
Switch absolute symlink path to relative.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-15 14:59:42 +01:00
David Sterba ee59a58112 btrfs-progs: ci: add README
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-04 21:14:41 +01:00
David Sterba 1304eb9e12 btrfs-progs: ci: add CentOS 8 image
Build tests inside CentOS 8, long term support distro.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 16:47:14 +01:00
David Sterba 287512dc20 btrfs-progs: ci: add CentOS 7 image
Build tests inside CentOS 7, long term support distro.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 16:46:53 +01:00
David Sterba cb419f18fc btrfs-progs: ci: add openSUSE Leap 15.2 image
Build tests inside openSUSE Leap 15.2, long term support distro.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 16:46:28 +01:00
David Sterba ba9ad9939c btrfs-progs: ci: add openSUSE Tumbleweed image
Build tests inside openSUSE Tumbleweed, rolling distro, most up to date.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 16:45:56 +01:00
David Sterba e61eb57b6b btrfs-progs: ci: add docker build and test scripts
Add scripts that can be used to build docker images and executed from
inside docker containers to verify build or run the testsuite.
Some tweaks are needed at each step to make things work.

- docker-build - build the image
- docker-run - run the default command (test-build)
- run-tests - run the testsuite

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 15:46:34 +01:00
David Sterba a7043efdf7 btrfs-progs: ci: check for local copy of branch tar before downloading
The ci/test-build script unconditionally downloads the latest devel
snapshot. This is not practical for local development. Add a conditional
check for a file named devel.tar.gz, either it's missing or empty, then
download.

The empty file is also considered because this allows to use a docker
image that does not support conditional contents, so a stub file is a
fallback.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-24 16:40:20 +01:00
David Sterba 65058429a5 btrfs-progs: ci: temporarily disable gitlab CI
The jobs has been failing for some time due the time limit 1h:

+ qemu-system-x86_64 -m 512 -nographic -kernel /repo/bzImage -drive
  file=/repo/qemu-image.img,index=0,media=disk,format=raw -fsdev
  local,id=btrfs-progs,path=/repo,security_model=mapped -device
  virtio-9p-pci,fsdev=btrfs-progs,mount_tag=btrfs-progs -append
  'console=tty1 root=/dev/sda rw'
  main-loop: WARNING: I/O thread spun for 1000 iterations
  ERROR: Job failed: execution took longer than 1h0m0s seconds

We'd still like to use the qemu test as it could pull the recent
development kernel that the base image does not provide. However the
overall performance is too bad and it does not make sense to waste
GitLab resources.

Also remove the build status badge from README as it changed at some
point and does not render as a SVG image anymore.

Issue: #171
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-18 20:53:23 +01:00
David Sterba f43df8a863 btrfs-progs: ci: add sanitizer coverage
Add builds with ASAN and UBSAN for both gcc. Clang + asan fails at
some tests due to wrong asan runtime, clang + ubsan fails at
libbtrfsutil/btrfsutil_internal.h:71:20:

	memcpy(&tmp, &sh->type, sizeof(__u32));

which seems to be too strict.

    [TEST/cli]   008-subvolume-get-set-default
subvolume get-default: default id is not 256, but
libbtrfsutil/btrfsutil_internal.h:71:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00
             ^

libbtrfsutil/btrfsutil_internal.h:64:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00
             ^

libbtrfsutil/btrfsutil_internal.h:78:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00