Commit Graph

80 Commits

Author SHA1 Message Date
Christian Marangi
6f89a0ca20
CI: use openwrt official tools container by default
Use openwrt official tools container by default.
Fork will use openwrt tools container by default.

This can be disabled by setting the option use_openwrt_container to
false for the build.yml and check-kernel-patches.yml.

The push-containers workflow is disabled on forks. The workflow can be
reenabled by commenting the condition in push-containers.yml.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-31 16:36:57 +01:00
Petr Štetiar
ba3aa6233d ci: allow custom kernel and target jobs based on labels
Current job triggers based on matching of changed paths is quite
limited, so lets make it possible to additionally trigger manual CI jobs
by adding CI specific pull request build labels:

 * `ci:target:x86:64` label is going to trigger CI target check jobs for
    x86/64 (sub)target.

 * `ci:kernel:x86:64` label is going to trigger CI kernel check jobs for
    x86/64 (sub)target.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-01-27 11:04:07 +01:00
Christian Marangi
d5943ffed8
CI: push-containers: limit to one concurrent run
We may find in a situation where due the queue an old run finish after
the last run, resulting in the containers getting overwritten with an
old version.

Limit the push-containers workflow to one concurrent run and cancel any
run in progress.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-24 23:23:56 +01:00
Christian Marangi
d40f59825a
CI: tools: directly copy prebuilt tools in container
Directly copy prebuilt tools in container instead of creating an
archieve and extracting it later in other workflows.

Update build workflow to support this new implementation.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:06 +01:00
Christian Marangi
7fe3d3670b
CI: tools: skip including dl dir in prebuilt tools tar
We can now drop the dl dir in the prebuilt tools tar as package archieve
is not a requirement anymore and won't trigger a package recompile.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-23 19:18:06 +01:00
Rosen Penev
183bad6641
CI: tools: add gnu-getopt to macOS CI
This used to be implicit. No longer for some reason.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-23 15:48:07 +01:00
Christian Marangi
6ca347379a
CI: kernel: enable target testing also for pull request event
Testing target changes was only set for push events. Enable this also
for pull request events to enable testing pr making specific target
changes.

Fixes: 57a02cbbff ("CI: kernel: test each target with additional changes than target/linux")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-20 15:14:53 +01:00
Robert Marko
d9b80ba8b8 CI: labeler: add ipq807x target
Add support for ipq807x target to the labeler.

Signed-off-by: Robert Marko <robimarko@gmail.com>
2023-01-16 12:42:23 +01:00
Rosen Penev
0a35d3f992
CI: remove pcre from macOS
ff02e1561f added a host version of pcre
for packages that need it.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-11 22:55:43 +01:00
Rosen Penev
dd357409f6
CI: remove already installed packages
The GitHub image already includes these.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-11 22:55:42 +01:00
Rosen Penev
11627f3ac1
CI: remove various tools from macOS
zstd, openssl, and quilt are already built in tools/. No need to install
them.

The rest are unused.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2023-01-11 22:55:42 +01:00
Christian Marangi
57a02cbbff
CI: kernel: test each target with additional changes than target/linux
Test each target if there are additional changes than target/linux.
This is needed to do wide test with changes to kmods, include/kernel and
changes to the workflow files.

While at it also cleanup and rework the code to drop duplication.
Also drop since_last_remote_commit to better track changes.

Fixes: 04ada8bc41 ("CI: kernel: build only changed targets")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-11 17:46:53 +01:00
Christian Marangi
cb679adf18
CI: add concurrency limits for pr test
Add concurrency limits for pull request test so that on pull request
refresh old jobs are cancelled.

The group is created based on the github ref + workflow name and the
workflow is cancelled only it it comes from a pull_request event.
Push events are not affected by this limit.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-11 14:31:57 +01:00
Christian Marangi
04ada8bc41
CI: kernel: build only changed targets
Detect changes in commit and build only changed targets.
If a change is related to the generic target, build test each target.

The matrix json is split. For target check patch only the first
subtarget is selected, for build test each target subtarget is built.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-11 14:28:41 +01:00
Christian Marangi
895f38ca1e
CI: build: fallback to compile toolchain if external toolchain fail
If for whatever reason external toolchain can't be found or downloaded,
fallback to internal toolchain build.

This can be useful when new target are introduced and external toolchain
are not present in openwrt fileserver.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-11 00:05:18 +01:00
Petr Štetiar
f9653cbaac ci: show_build_failures: fix missing output for configure failures
We're currently missing log output in cases where `configure` fails
which returns 77 as its error code:

 make[3]: *** [Makefile:118: elfutils-0.188/.configured_889556d2f423f99e091beece9c8d870a] Error 77

So lets adjust the regexps so they can handle multiple digits.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2023-01-09 11:22:10 +01:00
Christian Schmidbauer
169878a190
github: add command for device
This adds a command to the issue template to simplify the "device" name reporting.

Signed-off-by: Christian Schmidbauer <github@grische.xyz>
2023-01-06 15:28:59 +01:00
Christian Marangi
f655923b36
CI: build: fix external toolchain use with release tag tests
When a new tag for a release is created, the just checkout repo from
github actions will already have such tag locally created.

This will result in git fetch --tags failing with error rejecting the
remote tag with (would clobber existing tag).

Add -f option to overwrite any local tags and always fetch them from
remote.

Fixes: e24a1e6f6d ("CI: build: add support for external toolchains from stable branch")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-01-04 19:26:16 +01:00
Hauke Mehrtens
b6c7f90b17 CI: labeler: Add target label to target packages
This adds the target label also to changes in packages which are target
specific like the boot loader of a target or some drivers which are only
use on one target.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-12-30 15:10:18 +01:00
Christian Marangi
af44174187
CI: tools: reogranize and split workflow
Generilize tools workflow for future usage in shared workflow for tools
build.

Split tools workflow to tools and push-containers:
- tools just execute build test
- push-containers build and push prebuilt containers

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-24 12:09:23 +01:00
Stijn Tintel
9b6dd57a33 github: point to forum feature request category
This should have been amended in the previous commit.

Fixes: 65e72f8e380e ("github: add issue template")
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2022-12-21 17:24:12 +02:00
Stijn Tintel
65e72f8e38 github: add issue template
Add an issue template with required fields, instructions how to easily
get some of that required data from the device. Aside from that, also
add some links to the OpenWrt contact page, and for feature requests,
link to the forum.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2022-12-21 17:20:15 +02:00
Christian Marangi
1b8c8864c5
CI: introduce target toolchain test
Add test to build test toolchain for each target.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-16 16:29:37 +01:00
Christian Marangi
dcdb0b064a
CI: build: make kernel build configurable
Make kernel build configurable to permit to introduce toolchain testing.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-16 16:29:36 +01:00
Christian Marangi
819b676af0
CI: improve build naming for shared workflow
Impove build naming for build shared workflow to better understand what
is being test.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-16 16:29:32 +01:00
Christian Marangi
925e2a155e
CI: kernel: don't checkout and install feeds
We don't need to checkout feed and install feeds for kernel tests. This
saves up to 2 minutes for each target kernel build test.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-07 18:12:31 +01:00
Christian Marangi
99eaedfe39
CI: build: skip sdk adapt to external toolchain on cache hit
On cache hit, skip sdk adapt to external toolchain. This is needed because we
cache the already extracted sdk and that is already adapted to be used
as external toolchain.

Rerunning the adap step will result in the test to fail for missing file
as the file are already got wrapped to the external toolchain format.

Fixes: 42f0ab028e2e ("CI: build: fix use of sdk as toolchain")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-07 18:09:18 +01:00
Christian Marangi
42f0ab028e
CI: build: fix use of sdk as toolchain
The toolchain included in a sdk have a different format than an external
toolchain tar.

Since sdk is a more integrated setup doesn't use and include wrapper bin
that use the external toolchain config and use an alternative and more
standard way to include all the toolchain headers.

External toolchain use wrapper.sh to append the configured include
header when each tool is called.

Fix the sdk toolchain by reverting their own sdk wrapper scripts and to
simulate an external toolchain build copying what is done in the
toolchain target makefile.

This handle compilation error and warning caused by not using fortify
header on building packages.

Fixes: 006e52545d14 ("CI: build: add support to fallback to sdk for external toolchain")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-06 23:46:13 +01:00
Christian Marangi
644175c29c
CI: trigger check also on build and check-kernel-patches workflow change
Since kernel and packages workflow now use a shared build workflow, they
also need to react on changes on these shared workflow.

Fix this and add these shared workflow to the event paths to check.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 20:58:11 +01:00
Christian Marangi
e3cf2b84e5
CI: build: fix matching for openwrt release branch for toolchain parsing
The current match logic doesn't handle test for push events related to
stable release (example openwrt-22.03) but only fork with the related
prefix (example openwrt-22.03-fixup)

Fix wrong matching and while at it also add extra checks to other
matching (check if the branch name actually start with the requested
prefix)

Fixes: e24a1e6f6d7f ("CI: build: add support for external toolchains from stable branch")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 20:56:33 +01:00
Christian Marangi
65c3d19c4b
CI: fix matching for openwrt release branch for container selection
The current match logic doesn't handle test for push events related to
stable release (example openwrt-22.03) but only fork with the related
prefix (example openwrt-22.03-fixup)

Fix wrong matching and while at it also add extra checks to other
matching (check if the branch name actually start with the requested
prefix)

Fixes: abe8a4824210 ("CI: build: add support for per branch tools container")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 20:37:58 +01:00
Christian Marangi
b59ac2a7d0
CI: build: add support to fallback to sdk for external toolchain
Add support to use sdk as external toolchain if the packaged external
toolchain tar is not found on openwrt servers for build shared workflow.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 16:04:27 +01:00
Christian Marangi
e24a1e6f6d
CI: build: add support for external toolchains from stable branch
Add support to use external toolchains from stable branch if we are
testing commit targeting stable openwrt branch in kernel and packages
workflow.

With pr the target branch is parsed and the right toolchain is used.

To use the stable toolchain for local testing the branch needs to have
the prefix openwrt-[0-9][0-9].[0-9][0-9]- (example openwrt-21.02-fixup)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 16:04:27 +01:00
Christian Marangi
abe8a48242
CI: build: add support for per branch tools container
Add support in build shared workflow for per branch tools container.

With pr the target branch is parsed and the right container is used.

To use the stable container for local testing the branch needs to have
the prefix openwrt-[0-9][0-9].[0-9][0-9]- (example openwrt-21.02-fixup)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 16:03:23 +01:00
Christian Marangi
75550771ae
CI: tools: support per branch tools container
Add support to push per branch container tools.
For anything not official stick to latest tag that correspond to test
run from master.

If we are testing something for one of the openwrt stable branch, parse
the branch name or the tag and push dedicated tools containers.

To use the stable container for local testing the branch needs to have
the prefix openwrt-[0-9][0-9].[0-9][0-9] (example openwrt-21.02-fixup)

Any branch that will match this pattern openwrt-[0-9][0-9].[0-9][0-9]
will refresh the tools container with the matching tag.
(example branch openwrt-22.03 -> tools:openwrt-22.03)
(example branch openwrt-22.03-test -> tools:openwrt-22.03)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-04 16:03:22 +01:00
Hauke Mehrtens
cf361b8509 CI: Build all boards and testing kernel
This adds options to build all boards of a selected target and an
additional option to build the testing kernel instead of the normal
kernel. This can be used by other trigger work flows.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-12-03 03:10:10 +01:00
Hauke Mehrtens
08f5283392 CI: Allow building with internal toolchain
This adds an option to build with internal toolchain. This can be used
to build targets which are currently not build by the OpenWrt build bots
and which needs their own toolchain build for every build.

Building the toolchain takes about 30 minutes compared to using the
external toolchain which takes some seconds.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-12-03 03:10:06 +01:00
Hauke Mehrtens
7c406a5f08 CI: Extract the OpenWrt building to own sub workflow
Extract the building of OpenWrt into an own workflow which is then
triggered by the kernel.yml and packages.yml workflow with different
inputs. This allows us to share much of the code of the workflow.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-12-03 03:10:03 +01:00
Hauke Mehrtens
ce343653c2 CI: Simplify if conditions
There is no need to put a ${{ }} around the if conditions.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-12-03 02:57:30 +01:00
Christian Marangi
af8bc8e51b
CI: labeler: fix wrong label for pr targeting stable branch
The label used for stable branch is in the form of
release/[0-9][0-9].[0-9][0-9]
Currently we apply the name of the target branch as the label, fix this
and correctly use the current label.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-01 01:51:58 +01:00
Christian Marangi
b67d284e93
CI: add support to tag pr targeting stable branch
Add support to tag pr targeting stable branch matching the simple regex
of openwrt-[0-9][0-9].[0-9][0-9]. The tag that will be added will match
the pr target branch.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-12-01 01:28:40 +01:00
Hauke Mehrtens
6645a019f8 CI: packages.yml: Fix usage of pre-build tools
Activate CONFIG_AUTOREMOVE to match the settings used to build the
pre-build tools. This has to match the pre-build tools to not rebuild
them.

This prevents the tools being rebuild in packages.yml.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-11-06 00:08:09 +01:00
Hauke Mehrtens
b99d377886 CI: packages: Add github CI job to build all packages
This will build OpenWrt for MIPS malta BE and x86 64 Bit with all
packages and kernel modules activated. It is triggered when something
changes in the build system or when a package definition is changed.
This task probably needs 90 minutes to execute, but I hope that it
will find build problems in pull requests early.

This intentionally does not activate the feeds, because building them
too would take too long. We only build x86/64 and malta/be to save
resources.

I would like to detect build problems when a package is changed. We
often had build breaks when a package version was increased sometime
even in other packages which used it as a dependency.

This is based on the .github/workflows/packages.yml workflow.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2022-10-31 00:46:55 +01:00
Christian Marangi
6d4bcadaa3
CI: kernel: fix deprecation of set-output
From [0], github deprecated set-output with a better approach of
appending variables to $GITHUB_OUTPUT

[0] https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-21 16:09:19 +02:00
Petr Štetiar
5e31c82bb5 ci: kernel: trigger build check on changes in kernel.mk as well
So we can QA more parts of kernel build process.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2022-10-20 09:53:23 +02:00
Christian Marangi
689cfaeb7c
CI: kernel: check if patch are refreshed for each target
Enforce refreshed patch for each target with kernel pr tests.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-17 00:40:52 +02:00
Christian Marangi
5fb7232bc0
CI: labeler: target major version of labeler action
Target major version of labeler to include minor fixes and use always
the latest major version with included fixes.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-12 16:49:46 +02:00
Christian Marangi
77b24012db
CI: bump actions/download,upload-artifact action to v3
Bump actions/download,upload-artifact action to v3 on every workflow
to mute node deprecation warning.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-12 16:48:46 +02:00
Christian Marangi
87c69d73bb
CI: bump actions/checkout action to v3
Bump actions/checkout action to v3 on every workflow to mute node
deprecation warning.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-12 16:47:46 +02:00
Christian Marangi
23e946d1ad
CI: kernel: generate ccache cache on kernel push
To actually use ccache cache on kernel test from pr, the kernel workflow
has to be run first from a push action.

This will permit as a side effect to test merged commits and catch commit
that may cause regression in kernel compilation even outside the github
system.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-10-11 16:47:24 +02:00