Commit Graph

22 Commits

Author SHA1 Message Date
Christian Marangi 53a08e3743
build: make find_md5 reproducible with AUTOREMOVE
While experimenting with the AUTOREMOVE option in search of a way to use
prebuilt host tools in different buildroot, it was discovered that the
md5 generated by find_md5 in depends.mk is not reproducible.

Currently the hash is generated by the path of the file in addition to
the file mod time. Out of confusion, probably, there was an idea that
such command was used on the package build_dir. Reality is that this
command is run on the package files. (Makefile, patches, src)

This is problematic because the package Makefile (for example) change at
each git clone and base the hash on the Makefile mtime doesn't really
reflect if the Makefile actually changes across a buildroot or not.

A better approach is to generate an hash of each file and then generate
an hash on the sort hash list. This way we remove the problem of git
clone setting a wrong mtime while keeping the integrity of checking if a
file changed for the package as any change will result in a different
hash.

Introduce a new kind of find_md5 function, find_md5_reproducible that
apply this new logic and limit it only with AUTOREMOVE option set to
prevent any kind of slowdown due to additional hash generation.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2022-09-30 22:26:51 +02:00
Michael Pratt f34690e487
depends.mk: fix typo in rdep function
Just like later in the file,
the precursor to <stampfile>_check
<stampfile>_check.1 is supposed to be moved to
<stampfile>_check before it is touched.

This line would error if it was ever run.

Signed-off-by: Michael Pratt <mcpratt@pm.me>
2022-09-11 11:07:53 +02:00
Leonardo Mörlein b993b68b6c build: introduce $(MKHASH)
Before this commit, it was assumed that mkhash is in the PATH. While
this was fine for the normal build workflow, this led to some issues if

    make TOPDIR="$(pwd)" -C "$pkgdir" compile

was called manually. In most of the cases, I just saw warnings like this:

    make: Entering directory '/home/.../package/gluon-status-page'
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    bash: line 1: mkhash: command not found
    [...]

While these were only warnings and the package still compiled sucessfully,
I also observed that some package even fail to build because of this.

After applying this commit, the variable $(MKHASH) is introduced. This
variable points to $(STAGING_DIR_HOST)/bin/mkhash, which is always the
correct path.

Signed-off-by: Leonardo Mörlein <me@irrelefant.net>
2021-05-13 15:13:15 +02:00
Paul Spooren a17b8eaa2e build: use SPDX license tags
The license folder is a core part of OpenWrt and all GPL-2.0 licensed.
Use SPDX license tags to allow machines to check licenses.

Signed-off-by: Paul Spooren <mail@aparcar.org>
[rebase, keep some Copyright lines, sharpen commit message]
Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
2021-02-05 14:54:47 +01:00
John Beckett 36e0268aa6 build: find_md5 list with mod time and sorted
It was observed that the MD5 would not change after source files had been
modified, looking deeper into the build process it was discovered that
find_md5 build function makes a list of the files being built and then
passes the list to a summing utility on stdin.  The resultant MD5 is of
the file list, not the contents of the files.

The MD5 would change if the ordering of the list changed, or items were
removed or deleted.

The proposed fix is to add the modification time after the filename and
then sort the list to prevent find returning files in a different order
falsely re-triggering a rebuild. The MD5 will now change when a file is
modified or files are added/removed from the list.

Using 'T@' to show time in epoch for timezone independent behaviour.

Signed-off-by: John Beckett <john.beckett@net2edge.com>
2020-12-07 10:55:59 -10:00
Felix Fietkau 40fb293f24 build: add missing wildcard for ignoring .pkgdir in dependency checks
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-05 22:17:30 +01:00
Felix Fietkau 87d9e3462d build: stage unstripped packages inside PKG_BUILD_DIR
This makes the build slightly more efficient by avoiding the need to
re-stage packages on every full build run.

It is also necessary for the upcoming CONFIG_AUTOREMOVE feature

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:08 +01:00
Felix Fietkau 9228d1c066 build: introduce extra targets that contain only proper dependencies
This can be used to check if targets like prepare or compile are up to date

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-18 23:57:08 +01:00
Felix Fietkau 84bd74057f build: use mkhash to replace various quirky md5sum/openssl calls
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-05 11:09:12 +01:00
Yousong Zhou f66cb6f810 build: find_md5: ignore non-existent files or directories
Targets like malta can have no patches/ directory available and this
commit tries quash "no such file or directory" messages from `find`

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2016-11-24 12:53:19 +01:00
Luka Perkov 7002f6326e include: remove trailing whitespaces
Signed-off-by: Luka Perkov <luka@openwrt.org>

SVN-Revision: 45127
2015-03-29 07:29:18 +00:00
Felix Fietkau 695a1268e1 change the recursive dependency template to use more make evaluation - the shell command got so long that it triggered "Argument list too long" on some systems (#8231)
SVN-Revision: 24131
2010-11-24 14:07:12 +00:00
Jo-Philipp Wich 6734c999af ignore vim .swp files when scanning for updates (#7867)
SVN-Revision: 22880
2010-09-03 12:31:18 +00:00
Felix Fietkau a9136c0892 speed up the build system by including include/shell.sh on shell commands only where necessary
SVN-Revision: 22720
2010-08-19 12:49:37 +00:00
Felix Fietkau 091934e7fb fix timestamp checks for build system paths which have '.svn' in their directory name
SVN-Revision: 20856
2010-04-14 22:21:15 +00:00
Felix Fietkau af03fdb6c5 add autorebuild check for menuconfig
SVN-Revision: 9301
2007-10-14 02:28:34 +00:00
Felix Fietkau 5f748a6c24 clean up recursive dependency handling, use timestamp.pl again, because it saves memory and execution time
SVN-Revision: 8558
2007-08-30 21:12:39 +00:00
Felix Fietkau 23f0017829 fix a rebuild bug related to quilt
SVN-Revision: 8255
2007-07-30 23:52:17 +00:00
Felix Fietkau 109f15b54c ignore filenames that can cause problems for the recursive dependency handling
SVN-Revision: 8171
2007-07-26 11:32:13 +00:00
Felix Fietkau 56cdcf23a3 fix recursive dependencies on build directories - should lead to fewer spontaneous rebuilds
SVN-Revision: 7151
2007-05-09 22:47:01 +00:00
Felix Fietkau a3a9e039d8 ignore errors in the find command for dep checks
SVN-Revision: 6997
2007-04-18 10:12:50 +00:00
Felix Fietkau 449a4f4079 clean up dependency handling for autorebuilds
SVN-Revision: 6956
2007-04-15 18:28:24 +00:00