Trying to use 'package_whatdepends' feature of the ImageBuilder with OPKG
will currently fail as OPKG does not support "list --depends" call at all,
it seems that this is a mixup from the original APK support commit.
So, lets restore 'package_whatdepends' support for OPKG by calling
"whatdepends -A" instead as we used to before APK support.
Fixes: d788ab376f ("build: add APK package build capabilities")
Link: https://github.com/openwrt/openwrt/pull/17022
Signed-off-by: Robert Marko <robimarko@gmail.com>
Currently trying to use the 'package_depends' feature of ImageBuilder will
result in it trying to call OPKG:
make package_depends PACKAGE="usbutils"
OK: 26 MiB in 142 packages
bash: line 1: openwrt-imagebuilder-qualcommax-ipq807x.Linux-x86_64/staging_dir/host/bin/opkg: No such file or directory
make: *** [Makefile:363: package_depends] Error 127
It looks like a copy/paste error when APK support was being added, so lets
fix it by calling APK and its "info --depends" command.
Fixes: #16996
Fixes: d788ab376f ("build: add APK package build capabilities")
Link: https://github.com/openwrt/openwrt/pull/17022
Signed-off-by: Robert Marko <robimarko@gmail.com>
Currently, when "apk add --initdb" is called it will populate the
/etc/apk/arch file with the architecture of the host instead of the target
package architecture and this will then make it impossible for new packages
to be installed on the device as the target architecture will not match
after APK is updated to include [1].
So, lets use override the architecture by passing the target package one
by using the "--arch" argument to "apk add".
[1] c1a3e69f24Fixes: #16953
Link: https://github.com/openwrt/openwrt/pull/17015
Signed-off-by: Robert Marko <robimarko@gmail.com>
DEFAULT_PACKAGES handling was moved in commit 40be892a02
("imagebuilder: move handling of DEFAULT_PACKAGES into shareable place")
to `include/default-packages.mk`, but they weren't removed from
ImageBuilder's Makefile, so lets remove it now.
Once removed, I've noticed, that it stopped working as there is
target.mk included later in that file, overriding the DEFAULT_PACKAGES
again, so moved it after this target.mk include.
Fixes: 40be892a02 ("imagebuilder: move handling of DEFAULT_PACKAGES into shareable place")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
It seems, that handling of DEFAULT_PACKAGES is needed in more places, so
lets move it into dedicated include file so it can be easily shared.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Link: https://github.com/openwrt/openwrt/pull/16986
Signed-off-by: Robert Marko <robimarko@gmail.com>
Make keys directory for APK instead of OPKG while adding local key.
Signed-off-by: Kuan-Yi Li <kyli@abysm.org>
Link: https://github.com/openwrt/openwrt/pull/16942
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Since the image builder pulls package lists from metadata directly,
add procd and busybox as depdendencies to base-files.
As for the package manager itself, since it can be disabled it needs
to be added directly in the image builder makefile
Fixes: 44598c233d ("build: remove broken dependency of metadata on toplevel .config variables")
Signed-off-by: Felix Fietkau <nbd@nbd.name>
In 451e2ce006 the code would only move packages separated by dashes,
however OPKG uses underscores to split between package name and version.
Result was the lack of such packages in the packed ImageBuilder.
Use dashes or underscores depending on the select package manger.
Fixes: 451e2ce006 "imagebuilder: fix package inclusion and APK index"
Signed-off-by: Paul Spooren <mail@aparcar.org>
The `--initdb` command creates basic folders required by APK,
previoiusly it would only run a single time when package_index is
actually called. Since the function isn't called if nothing changes,
`--initdb` doesn't initialize the rootfs again.
This commit moves it to package_reload, which runs every time.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Previously three different ways to include packages in an ImageBuilder
existed:
* buildbot: include libc, kernel (and base-files) in $(IB_LDIR)
* not buildbot, standalone: include all packages in ./packages/
* not buildbot, not standalone: include libc, kernel (and base-files) in
./packages/
First of, the separation between *buildbot* and *not buildbot, not
standalone* is not required, we can just always copy packages to
./packages instead of ever using the special place $(IB_LDIR).
Doing so drops the need to handle the extra case and also allows to
clean up the OPKG package installation, which no longer requries the
`firstword` logic, things are now always at ./packages.
Signed-off-by: Paul Spooren <mail@aparcar.org>
We need to exclude packages from the APK index which must not be
upgraded. To do so, the packages `libc`, `kernel` and `base-files` are
excluded to APK never suggestes them as upgradable.
The previous logic would however match packages like `libcomerr0`,
causing build failures. Make the copying and excluding logic more
precise by adding a single dash.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This commit solves multiple issues. First of just install the three
special packages base-files, libc and kernel directly from the index. In
upstream indexes, those will never appear to prevent accidental upgrades
may breaking the system.
Next, enable caching for the ImageBuilder, which speeds up consecutive
builds from ~33 seconds to ~5 seconds. Using cache however makes APK
create the folder `/var/cache/apk/` which conflicts with the base-files
installation, which ships a symlink from `/var` to `/tmp`, so specify
`--no-cache` for the rootfs initialization.
Lastly, drop the use of `apk update` since APK automatically does that.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Those packages were not copied due to OPKG using an underscore while APK
uses dashes. Remove that char to copy kernel/libc for either APK/OPKG.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Complete support for local signing keys for APK.
A local key will be always generated, mkndx is always called with
--allow-untrusted as it needs to replace the sign key with the new local
one.
With CONFIG_SIGNATURE_CHECK the local index is signed with the local
key. Local public key is added with the ADD_LOCAL_KEY option.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
ImageBuilder compiled by buildbot doesn't have any package in the
packages directory. Package needs to be downloaded instead.
This works by calling update to the package manage to download the
remove index and download the file.
Fix missing support for this with APK, by configuring the
--repositories-file option and calling the APK update.
Also move the apk add --initdb to package_index.
If CONFIG_SIGNATURE_CHECK is not enabled, the signature is not checked.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Correctly export PACKAGE_DIR and PACKAGE_DIR_ALL so that they won't be
reset on internal call of rules.mk
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Also remove vmlinuz-initramfs files from final imagebuilder image as
these file are not needed.
Fixes: c85348d9ab ("imagebuilder: remove initramfs image files")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Initramfs images are not supported by imagebuilder. With recent changes
to support Per Device Rootfs, we now generate an image and a vmlinux for
each Rootfs and these additional files are all shipped in the
imagebuilder tar.
Drop these new file and any vmlinux-initramfs as they are not used and
increase the final size of the imagebuilder archive.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Fix broken IB_STANDALONE option for OPKG due to an error in ifdef logic
where we weren't adding the required entry to reference the local files
in repositories.conf
Rework the ifdef to more explicit and restore original functionality of
this option.
While at it also provide different README for APK or OPKG.
Fixes: d788ab376f ("build: add APK package build capabilities")
Link: https://github.com/openwrt/openwrt/pull/15599
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Fix multiple issue with manifest handling where APK was hardcoded
and fix a logic error where (TODO) APK _check_keys was called for the
OPKG codepath instead of correctly calling for the APK codepath.
Fixes: d788ab376f ("build: add APK package build capabilities")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
A new option called `USE_APK` is added which generated APK packages
(.apk) instead of OPKG packages (.ipk).
Some features like fstools `snapshot` command are not yet ported
Signed-off-by: Paul Spooren <mail@aparcar.org>
Historically it's possible to leave the `SUBTARGETS` undefined and
automatically fallback to a "generic" subtarget. This however breaks
various downstream scripts which may have expectations around filenames:
While some targets with an explicit generic subtarget contain `generic`
in the filenames of artifacts, implicit "subtargets" don't.
Right now this breaks the CI[1], possibly also scripts using the ImageBuilders.
This commit removes all code that support implicit handling of
subtargets and instead requires every target to define "SUBTARGETS".
[1]: https://github.com/openwrt/openwrt/actions/runs/8592821105/job/23548273630
Signed-off-by: Paul Spooren <mail@aparcar.org>
The image generation would fail, if the target is included from a feed.
To fix this, check if targets is found in the feed directory.
Signed-off-by: Florian Eckert <fe@dev.tdt.de>
Tested-by: Thomas Richard <thomas.richard@bootlin.com>
Fixes the regression so that targets that were installed via a feed can
also be build again with the Image Builder.
Fixes: 84ec8c4 ("imagebuilder: copy from buildroot only target/linux")
Signed-off-by: Florian Eckert <fe@dev.tdt.de>
Tested-by: Thomas Richard <thomas.richard@bootlin.com>
This reverts commit 131e41614d.
Sadly it makes menuconfig fail with
tmp/.config-package.in:171: glob failed: No files found "feeds/base/utils/busybox/Config.in"
make: *** [/usr/src/openwrt/include/toplevel.mk:136: menuconfig] Error 1
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Building a package in the build system or the SDK results in different
values for the `SOURCE` property, it's either `packages/<package name>`
or `feeds/base/<package name>`. The reason is that the SDK handles
`openwrt.git` as an external feed called while the build system contains
the *base* packages directly.
Since packages created with either method are (ideally) the same (bit
for bit), align the content of SOURCE. To do so this commit creates a
symlink from `feeds/base` to `$(TOPDIR)/package` and adopts the SOURCE
when building from inside the build system.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This reduces the size of a single imagebuilder by about 40MB
In example for the target ath79 it would be the sum of generic and <target> directories, so about 16MB,
instead of the whole size of the target directory, about 53MB:
11M target/linux/generic/
3.9M target/linux/ath79/
Signed-off-by: a-gave <agave@dracaena.it>
Some device recipes remove default target packages. If user tries to add
them back they will be ignored, since packages list is processed in one
go. Process the device recipe packages first and do user ones later, so
additions won't get filtered out.
Signed-off-by: Tomasz Maciej Nowak <tmn505@gmail.com>
Add a variable that stores the original value of $PATH
in the host system's shell, before Make alters it.
This can be useful for when it is necessary
to ignore symlinks and programs made by the build system.
Define this new variable before all instances of
'export PATH:=' or similar.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Setting this options modifies the rootfs size of created images. When
installing a large number of packages it may become necessary to
increase the size to have enough storage.
This option is only useful for supported devices, i.e. with an attached
SD Card or installed on a hard drive.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This commits adds the makefile targets `depends` this wrapper is a call
to `opkg depends`. This command shows which runtime dependencies exist
if this package is installed into the image.
Signed-off-by: Florian Eckert <fe@dev.tdt.de>
Export SOURCE_DATE_EPOCH to environment so filesystem and image
creation tools will make use of it.
Fixes reproducibility of images generated with the ImageBuilder.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
When using external targets there is a symlink being created for the
target under target/linux which then becomes dangling under Image
Builder. Fix it by dereferencing the possible symlink.
Tested on IB with external target, ipq40xx and mvebu.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
This commit fixes commit "2999f810ff: build,IB: include kmods only in
local builds" which cause the local packages/ folder only to be added
for local builds but no longer for ImageBuilder created by the Buildbot.
The commits intention was to use remote kmods repositories rather than
storing them locally. Accidentally the entire handling of the local
`packages/` was removed.
Re-add the folder and include a README describing what it can be used
for.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Using `make info` show the current target, revision, default packages
and available profiles. This commits adds the used architecture.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Using these config-options to customize the folders used at build-time
makes these folder settings appear in generated archive. This causes the
imagebuilder to be not portable, as it's going to use the build-time folders
on the new systems. Errors look like:
mkdir: cannot create directory '/mnt/build': Permission denied
Makefile:116: recipe for target '_call_image' failed
make[2]: *** [_call_image] Error 1
Makefile:241: recipe for target 'image' failed
make[1]: *** [image] Error 2
The build-time settings of these folders are passed into the archives via
.config file.
The expected behavior is that after unpacking the imagebuilder acts like
these settings have their defaults, using intree folders. So unset the
build-time settings.
Signed-off-by: Sven Roederer <devel-sven@geroedel.de>
The ImageBuilder `make manifest` prints all installed packages. This
function can be used to create a list of package and corresponding
package versions before attempting image creation.
When called with `--strip-abi` OPKG can automatically strip attached
ABIVersions from package names. Make this function accessible for the
ImageBuilder by adding a `STRIP_ABI` variable.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Invoke bundle-libraries.sh with any buildroot related directory entries
removed from $PATH to avoid picking up cross versions of utilities like
ldd which will not properly work when used against host executables.
This should fix executable bundling for glibc-target imagebuilders.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
When building images with the imagebuilder, the partition signature
never changes. The signature is generated by hashing SOURCE_DATE_EPOCH
and LINUX_VERMAGIC which are undefined. Prepopulate these variables, as
done by the SDK.
Signed-off-by: Matthew Gyurgyik <matthew@gyurgyik.io>
The ImageBuilder downloads pre-built packages and adds them to images.
This process uses `opkg` which has the capability to verify package list
signatures via `usign`, as enabled per default on running OpenWrt
devices.
Until now this was disabled for ImageBuilders because neither the `opkg`
keys nor the `opkg-add` script was present during first packagelist
update.
To harden the ImageBuilder against *drive-by-download-attacks* both keys
and verification script are added to the ImageBuilder allowing `opkg` to
verify downloaded package indices.
This commit adds `opkg-add` to the ImageBuilder scripts folder. The keys
folder is added to ImageBuilder $TOPDIR to have an obvious place for users to
store their own keys. The `option check_signature` is appended to the
repositories.conf file. All of the above only happens if the Buildbot
runs with the SIGNATURE_CHECK option.
The keys stored in the ImageBuilder keys/ are the same as included in
the openwrt-keyring package. To avoid the chicken-egg problem of
downloading and verifying a package, containing signing keys, the keys
are added during the ImageBuilder generation. They are same as in
shipped images (stored at `/etc/opkg/keys/`).
To allow a local package feed in which the user can add additional
packages, a local set of `usign` and `ucert` keys is generated, same as
building OpenWrt from source. The private key signs the local repository
inside the packages/ folder. The local public key is added to the keys/
folder to be considered by `opkg` when updating repositories. This way a
local package feed can be modified while requiring `opkg` to check
signatures for remote feed, making HTTPS optional.
The new option `ADD_LOCAL_KEY` allows to add the local key inside the
created images, adding the advantage that sysupgrades can validate the
ImageBuilders local key.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Without an absolute path to staging_dir/host/bin/sstrip the Makefile
tries to run a host installed version of sstrip, which is likely not
available.
Signed-off-by: Paul Spooren <mail@aparcar.org>
With the fix of external kmod feeds it is possible to ship the
ImageBuilder without any packages except the pseudo packages kernel and
libc. Therefore the local package feeds becomes optional.
This commit adds a check to the package_reload function to only run if
the local feed is existing.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The buildbots generate a kmod archive which should be used instead of a
local copy. This is possible due to the introduction of a kernelversion
specific feed.
This commit adds the ability of using only signed package feeds.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Robert Marko
Petr Štetiar
Kuan-Yi Li
Felix Fietkau
Paul Spooren
Christian Marangi
Florian Eckert
Daniel Golle
a-gave
Tomasz Maciej Nowak
Michael Pratt
Sven Roederer
Jo-Philipp Wich
Matthew Gyurgyik
Paulo Machado