While experimenting with the AUTOREMOVE option in search of a way to use
prebuilt host tools in different buildroot, it was discovered that the
md5 generated by find_md5 in depends.mk is not reproducible.
Currently the hash is generated by the path of the file in addition to
the file mod time. Out of confusion, probably, there was an idea that
such command was used on the package build_dir. Reality is that this
command is run on the package files. (Makefile, patches, src)
This is problematic because the package Makefile (for example) change at
each git clone and base the hash on the Makefile mtime doesn't really
reflect if the Makefile actually changes across a buildroot or not.
A better approach is to generate an hash of each file and then generate
an hash on the sort hash list. This way we remove the problem of git
clone setting a wrong mtime while keeping the integrity of checking if a
file changed for the package as any change will result in a different
hash.
Introduce a new kind of find_md5 function, find_md5_reproducible that
apply this new logic and limit it only with AUTOREMOVE option set to
prevent any kind of slowdown due to additional hash generation.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
HOST_LOADLIBES was renamed to KBUILD_HOSTLDLIBS in kernel 4.19. As the
oldest kernel version we support is 5.10, cleanup HOST_LOADLIBES use.
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
When a target configuration has unser Kconfig symbols, the build will
fail when OpenWrt is compiled with V=s and stdin is connected to a tty.
In case OpenWrt is compiled without either of these preconditions, the
build will uscceed with the symbols in question being unset.
Modify the kernel configuration in a way it fails on unset symbols
regardless of the aformentioned preconditions.
Signed-off-by: David Bauer <mail@david-bauer.net>
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
While parsing the nm output, we need to account for the fact that 64-bit kernels
have 64-bit wide addresses. While at it, replace the grep | sed combo with a
single awk invocation and a stronger regex.
Fixes: 2ef0acc5fc "kernel-build: fix
STRIP_KERNEL_EXPORTS for recent kernels"
Signed-off-by: Rui Salvaterra <rsalvaterra@gmail.com>
Commit 5d76065 moved the creation of the symvers directory to
include/kernel-build.mk. This is fine when building from scratch. But
when unpacking an SDK the directory doesn't exist and because the kernel
won't be built (again) this directory will not be created by the build
system, causing build failure if make tries to copy files into it.
This moves the creation of the symvers directory back into
include/kernel.mk so that the directory is created in any case.
Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Define wildcard patterns for filtering in target/linux/generic/config-filter
Preparation for supporting newer kernels
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This replaces the previous (deprecated) method of collecting symvers data
in $(PKG_BUILD_DIR)/Module.symvers, which does not work on newer kernels
Signed-off-by: Felix Fietkau <nbd@nbd.name>
zstd with its default settings (compression level -3) compresses better
than bzip2 -9 (which is the default setting), and is an order of magnitude
faster.
I made the following measurements for the most common compression tools
(all standard Debian Buster versions, default flags unless noted
otherwise), using the debug information of a large x86-64 kernel with
ALL_KMODS:
* kernel-debug.tar: 376M
* kernel-debug.tar.gz: 101M, compressed in ~12s
* kernel-debug.tar.bz2: 91M, compressed in ~15s
* kernel-debug.tar.xz: 57M, compressed in ~101s
* kernel-debug.tar.zst: 86M, compressed in ~1s
With zstd, there is still some room for improvement by increasing the
compression, but the slight increase in compression ratio
(22.83% -> 19.46%) does not justify the significant increase in
compression time (about 5 times on my machine) in my opinion.
Note that multithreaded compression (-T argument) does not affect
reproducibility with zstd.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Don't put CC_HAS_ASM_GOTO, IS_GCC, IS_CLANG and GCC_VERSION runtime
config options into the kernel configs during reconfiguration as it
makes no sense, since these options should be set at runtime.
Fixes: FS#2588
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Commit 965f341aa9fd ("build: fix host menu config targets using
ncurses") has moved host's path with pkg-config (usually /usr/bin) at
the first place in PATH variable, which is now causing issues with bison
as BISON_PKGDATADIR points into STAGING_DIR_HOST, but the actual bison
used is the one under host PATH (usually /usr/bin/bison), leading to the
following strange failures:
$ make target/linux/clean kernel_menuconfig V=sc
export MAKEFLAGS= ;make -C /somewhere/linux-4.19.81 menuconfig
make -f ./scripts/Makefile.build obj=scripts/kconfig menuconfig
...
bison -oscripts/kconfig/zconf.tab.c -t -l scripts/kconfig/zconf.y
staging_dir/host/bin/m4: cannot open `staging_dir/host/share/bison/bison.m4': No such file or directory
staging_dir/host/bin/m4: cannot open `staging_dir/host/share/bison/c-skel.m4': No such file or directory
...
gcc -Wp,-MD,scripts/kconfig/.zconf.tab.o.d <...snip...> -o scripts/kconfig/zconf.tab.o scripts/kconfig/zconf.tab.c
gcc: error: scripts/kconfig/zconf.tab.c: No such file or directory
gcc: fatal error: no input files
Fix this by forcing usage of bison under STAGING_DIR_HOST/bin via YACC
make variable.
Cc: Thomas Albers <thomas.gameiro@gmail.com>
Cc: Stijn Tintel <stijn@linux-ipv6.be>
Cc: Eneas U de Queiroz <cotequeiroz@gmail.com>
Tested-by: Ivan Revyakin <LovingFox@GMail.com>
Tested-by: Thomas Albers <thomas.gameiro@googlemail.com>
Ref: https://forum.openwrt.org/t/bpi-r64-kernel-4-19-kernel-menuconfig-error
Fixes: 965f341aa9fd ("build: fix host menu config targets using ncurses")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Pass suitable pkg-config overrides to the kernel build process in
order to let our pkg-config wrapper discover libraries provided
by tools/.
This mainly affects the use of libelf which is required for the
CONFIG_STACK_VALIDATION features. So far, the build system either
silently used host system libraries or kbuild simply disabled the
feature due to the lack of a suitable libelf.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
An incorrect variable name was referenced in KERNEL_FILE_DEPENDS, leading
to the omission of the backport-* patch dirs in the generation of the
prepared stamp name.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
The Download/git rule will do a `git checkout <git-ref>`.
So, we can use any ref we want.
No need to limit just to branches.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Commit 86c966a8ae caused HOST_LOADLIBES to
include -lncurses. This was added for fixing build issues on macOS.
This introduces issues on Linux when wide-character ncurses is being
used for compiling, but the non-wide-character version is linked in.
Fix this by adding the extra override for HOST_LOADLIBES only on macOS.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This allows packages to use kernel make options without the forced
-C $(LINUX_DIR). It also makes it more clear that it to be called from
kernel module packages directly.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Before SUBDIR was set to $(PATCHVER) which may
or may not include the minor version number of
the linux kernel version. Usually it doesn't.
So the git-clone'd linux kernel was packed without
the minor version number taken into account, which
broke further processing, as it expected the
extracted dir being named linux-$(LINUX_VERSION)
(=with minor version) rather than linux-$(PATCHVER)
(=without minor version).
Changing SUBDIR to $(LINUX_VERSION) creates
consistent behaviour here.
Signed-off-by: Mirko Vogt <mirko-openwrt@nanl.de>
Utilize the existing git download logic from include/download.mk and migrate
the kernel download over to it. This avoids repeatedly cloning kernel sources
after a make target/linux/clean for instance.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [fix build error]
The build system overrides HOST_LOADLIBES to add the staging dir to the
library search path. menuconfig needs -lncurses, add another override
for it.
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Pass KERNEL_FILE_DEPENDS to rdep instead of PKG_FILE_DEPENDS, which is
empty. Also don't pass $(CURDIR) as the directory to timestamp, as it
would also pick up non kernel related changes like image building code.
Should fix kernel being rebuild for unrelated changes, as well as not
being rebuild for changes in target/linux/generic.
Fixes: 22ef1c83b35c ("kernel: make the kernel build auto-clean the build dir like package build")
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
The --mtime argument to 'tar' sets the modification time for all files within
the archive, which determines the timestamp files will get when they are
extracted. In this case, rootfs and other tarballs will get mtimes which
correspond to the last commit timestamp of the build system, as reported by
git/subversion.
This is a step towards reproducible image builds.
Signed-off-by: bryan newbold <bnewbold@robocracy.org>
Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
SVN-Revision: 48586
Recent kernels started to mark exported symbols as global.
Adapt expressions in kernel-build.mk to also match global symbols
when grep'ing through nm output.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
SVN-Revision: 42555
When using the options EXTERNAL_KERNEL_TREE or KERNEL_GIT_CLONE_URI,
the command "make downloads" fails as it tries to download the kernel
tarball despite the option. This doesn't happen during a regular build
as in that case, the dependency is conditionned through the LINUX_SITE
variable, which is not set in these cases.
Below is a snapshot of the error for an target using a 3.14 kernel:
make[3]: *** No rule to make target `.../dl/linux-3.14.tar.xz',
needed by `download'. Stop.
Change-Id: I1244969c1bbf9c81a6a64d68ae88ac58b0f8e79e
Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com>
SVN-Revision: 40913
GNU grep has a high per-expression setup overhead when compiling regular
expressions. Use -F to force it to interpret the input as fixed strings,
which is much faster (fraction of a second instead of multiple minutes).
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
SVN-Revision: 39049
If initramfs is enabled, build a second pass kernel containing the CPIO
rootfs, preliminary work to get non exclusive enabling of initramfs.
Signed-off-by: Florian Fainelli <florian@openwrt.org>
SVN-Revision: 37047
