This keeps the configuration, like the size of the cache, and the
statistics intact. Move the removal of the cache directory to the
distclean target, but only delete the .ccache directory inside of our
build tree, as we should not mess with a user-configured external ccache
directory this destructively.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
`which` utility is not shipped by default for example on recent Arch
Linux and then any steps relying on its presence fails, like for example
following Python3 prereq build check:
$ python3 --version
Python 3.9.1
$ make
/bin/sh: line 1: which: command not found
/bin/sh: line 1: which: command not found
/bin/sh: line 1: which: command not found
...
Checking 'python3'... failed.
...
Fix this by switching to Bash builtin `command` which should provide
same functionality.
Fixes: FS#3525
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR).
This allows to do clean and dirclean. Cache hit rate for test build
after dirclean is ~65%.
If CCACHE is enabled stats are printed out at the end of building process.
CCACHE_DIR config variable allows to override default, which could be useful
when sharing cache with many builds.
cacheclean make target allows to clean the cache.
Changes from v1:
- remove ccache directory using CCACHE_DIR variable
- remove ccache leftovers from sdk and toolchain make files
- introduce CONFIG_CCACHE_DIR variable
- introduce cacheclean make target
Signed-off-by: Roman Yeryomin <roman@advem.lv>
JSON info files contain machine readable information of built profiles
and resulting images. These files were added in commit 881ed09ee6
("build: create JSON files containing image info").
They are useful for firmware wizards and script checking for
reproducibility.
Currently all JSON files are stored next to the built images, resulting
in up to 168 individual files for the ath79/generic target.
This patch refactors the JSON creation to store individual per image
(not per profile) files in $(BUILD_DIR)/json_info_files and create an
single overview file called `profiles.json` in the target directory.
Storing per image files and not per profile solves the problem of
parallel file writes. If a profiles sysupgrade and factory image are
finished at the same time both processes would write to the same JSON
file, resulting in randomly broken outputs.
Some target like x86/64 do not use the image code yet, resulting in
missing JSON files. If no JSON info files were created, no
`profiles.json` files is created as it would be empty anyway.
As before, this creation is enabled by default only if `BUILDBOT` is set.
Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64.
Signed-off-by: Paul Spooren <mail@aparcar.org>
[json_info_files dir handling in Make, if case refactoring]
Signed-off-by: Petr Štetiar <ynezz@true.cz>
When sharing a common build directory between different build platforms
eg. macos v docker based linux v virtual machine, a 'make dirclean'
isn't quite enough to clean all the platform related binaries. The
'conf' and 'mconf' aka 'make menuconfig/defconfig & friends' utilities
remain.
Clean those as part of 'dirclean' so they get rebuilt for the current
platform on the next 'make'
Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
The prepare target was added some 11 years ago to build tools and
toolchain and was recently extended to create buildinfo files for
reproducibility, meaning {feeds,version,config}.buildinfo.
As the buildbot workflow is more complex than the single prepare (kmod
feed insertion), prepare is only used to create those buildinfo files.
Running prepare however runs `target/compile` as well, taking time even
everything is already compiled.
Splitting this allows the buildbot to run only the `buildinfo` target
while others can still use the convenience feature `prepare`.
Signed-off-by: Paul Spooren <mail@aparcar.org>
On a recent Gentoo Linux installation, invoking `make menuconfig`, `make
kernel_menuconfig` or `make kernel_nconfig` in the build system fails,
whereas for example `make menuconfig` in the kernel tree alone works as
expected.
This is happening because STAGING_PREFIX is not defined when kernel's
{menu,n}config target calls pkg-config from the toolchain/host and thus
pkg-config returns an empty value, and the fallback values in the kernel
config script are applied but those are off and the linking fails.
Solution is to use system's pkg-config for all ncurses based menu config
targets in order to provide proper compiler/linker flags.
Ref: FS#2423
Cc: Thomas Albers <thomas.gameiro@gmail.com>
Tested-by: Stijn Tintel <stijn@linux-ipv6.be>
Tested-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
generate feeds.buildinfo and version.buildinfo in build dir after
containing the feed revisions (via ./scripts/feeds list -sf) as well as
the current revision of buildroot (via ./scripts/getver.sh).
With this information it should be possible to reproduce any build,
especially the release builds.
Usage would be to move feeds.buildinfo to feeds.conf and git checkout the
revision hash of version.buildinfo.
Content of feeds.buildinfo would look similar to this:
src-git routing https://git.openwrt.org/feed/routing.git^bf475d6
src-git telephony https://git.openwrt.org/feed/telephony.git^470eb8e
...
Content of version.buildinfo would look similar to this:
r10203+1-c12bd3a21b
Without the exact feed revision it is not possible to determine
installed package versions.
Also rename config.seed to config.buildinfo to follow the recommended
style of https://reproducible-builds.org/docs/recording/
Signed-off-by: Paul Spooren <mail@aparcar.org>
Revert "mac80211: add new minstrel_ht patches to improve probing on mt76x2" (9861050b85)
Revert "kernel: use bulk free in kfree_skb_list to improve performance" (98b654de2e)
Revert "ramips: add preliminary support for WIO ONE" (085141dc5b)
Revert "ramips: add preliminary support for SGE AP-MTKH7-0006 developer board" (b1db6d0539)
Revert "build: use config.site generated by autoconf-lean, drop hardcoded sitefiles" (363ce4329d)
Revert "toolchain: add autoconf-lean" (fdb30eed03)
Revert "build: allow overriding the filename on the remote server when downloading" (6fa0e07758)
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This reverts commit fe928c7a81 as it seems
to introduce build breakage when local.mk doesn't exist.
make --trace
toplevel.mk:216: target 'local.mk' does not exist
Signed-off-by: Petr Štetiar <ynezz@true.cz>
I tend to automate most of the boring and repetitive tasks like firmware
flashing, build config reconfiguration etc., so I always end up adding
`include local.mk` as a last line in my Makefile, where local.mk is
usually symlink to some other place, just to not accidentally delete it
during `git clean`.
Carrying this single uncommited modification along in the development
process is quite PITA, because it's causing problems during Git
workflow, while rebasing etc.
I hope, that I'm not alone using similar workflow, so I believe, that
this modification might be useful for others as well.
Signed-off-by: Petr Štetiar <ynezz@true.cz>
In some places the output of commands, which include "cd" are used.
In case of CDPATH the new path is printed, which might not be expected.
Disable the variable to avoid these problem.
When CDPATH was set by the user to some value like "export CDPATH=."
the git checkout done by the build system did not work anymore, the
git cloning aborted with such an error message for example:
....
Packing checkout...
tar: /disk/fs1/tmp2/mehrtens/pon-ugw/ugw-haps/openwrt/tmp/dl/ppa-drv-1.0\n@1534240258: Cannot stat: No such file or directory
tar: Date sample file not found
Try 'tar --help' or 'tar --usage' for more information.
.....
To avoid this, this patch makes the build system unset CDPATH inside
the build system, so the build system will still work even when the
user set this variable in his local environment.
Signed-off-by: Thomas Langer <thomas.langer@intel.com>
Signed-off-by: Hauke Mehrtens <hauke.mehrtens@intel.com>
Acked-by: Hans Dedecker <dedeckeh@gmail.com>
In order to be able to better compare files to sync in the future, include
all BIN_DIR subdirectories in the checksum calculation.
To not break existing applications, restrict the recursive checksumming to
CONFIG_BUILDBOT for now.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Using a single host package staging dir (and build dir) significantly speeds up
builds when multiple targets are built in succession, especially for large host
packages like NodeJS.
$(STAGING_DIR)/host is kept in addition to $(STAGING_DIR_HOSTPKG) in most
places; it is still used as destination for host files in Build/InstallDev.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Ensure that BIN_DIR exists when the diffconfig target needs it.
Otherwise 'make diffconfig' fails after 'make clean'
Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
Add a "diffconfig" build target which stores the output of
"scripts/diffconfig.sh" as "config.seed" in the image output directory and
invoke that target by default.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This is intended to be used for a wide array of package sanity checks.
The first check that is implemented is for the hash of downloaded files.
It checks:
- Missing hash
- Use of SHA256 instead of MD5
- dl/<file> hash not matching hash in makefile
- deprecated MD5SUM variable
The deprecated MD5SUM variable check is skipped for feeds/ until OpenWrt
is updated as well
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Add a new "checksum" make target which generates an sha256sums file over the
image files produced in bin/targets/ and automatically call it during make
world after the package index generation.
The advantage of this new target is that it is guaranteed to run after the
images, the SDK and the ImageBuilder archives have been generated to ensure
that they all end up in the checksum file. Fixes FS#51.
Uses sed to postprocess the OpenSSL digest output into an sha256sum command
compatible format.
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Currently "make clean" only clears the build_dir/target*, but leaves
staging_dir/target* intact. "make clean" should also clean the
staging_dir/target* directories, as in the current situation some old
packages or libraries may be linked into the firmware from staging_dir
despite a "make clean".
The patch reorganises clean / dirclean functionality so that
* "make clean" also clears the staging_dir/target* in addition to
build_dir/target*.
* "make dirclean" clears toolchain and host(=tools) directories from both
build_dir and staging_dir
signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
SVN-Revision: 45973
Makefile: remove non-existent STAGING_DIR_TOOLCHAIN from dirclean
Openwrt's top level Makefile uses STAGING_DIR_TOOLCHAIN in the make dirclean
statement.
https://dev.openwrt.org/browser/trunk/Makefile#L55
rm -rf $(STAGING_DIR) $(STAGING_DIR_HOST) $(STAGING_DIR_TOOLCHAIN)
$(TOOLCHAIN_DIR) $(BUILD_DIR_HOST) $(BUILD_DIR_TOOLCHAIN)
As far as I can determine, no such variable has been defined. I made a search
in Openwrt source repository and the one line in Makefile's dirclean command
is the only place where that variable exists.
The item has been introduced to Makefile by r8362, but even at that time
neither Makefile nor rules.mk defined such a variable. Most likely the goal
has been to set both staging_dir/toolchain and build_dir/toolchain to be
cleaned, but one of the variables has been erroneous. The correct variable
for build_dir/toolchain has been then added by r13494.
References:
https://dev.openwrt.org/changeset/8362/https://dev.openwrt.org/browser/trunk/Makefile?rev=8362https://dev.openwrt.org/browser/trunk/rules.mk?rev=8362https://lists.openwrt.org/pipermail/openwrt-devel/2007-August/001159.htmlhttps://dev.openwrt.org/changeset/13494
In current code,
TOOLCHAIN_DIR = $(TOPDIR)/staging_dir/$(TOOLCHAIN_DIR_NAME)
BUILD_DIR_TOOLCHAIN = $(TOPDIR)/build_dir/$(TOOLCHAIN_DIR_NAME)
so the item STAGING_DIR_TOOLCHAIN in the rm command is unnecessary.
signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
SVN-Revision: 45736
Most of the time, we want to make sure OpenWrt has been configured and
setup before start running make. However, in case of package/symlinks,
forcing prereq as a dependency creates multiple issues:
*when executed on a clean workspace, it will prompt for user input
and open a menuconfig window before executing the feeds command
*the only way around that is to provide a .config. However, the "prereq"
target would then run a "make defconfig", which will remove all the
packages in the .config but from external feeds, as feeds have not been
installed yet.
The only way to currently work around this, is to generate a fake config
by running "make defconfig", then "make package/symlinks", copy the real
config (which at this point disregards the previously generated config),
and run make defconfig again. Something like this:
make defconfig
make package/symlinks
cp real.config .config
make defconfig
This change is removing the need for the first defconfig, making the
process more logical for OpenWrt users using the package/symlinks target.
Signed-off-by: Mathieu Olivari <mathieu@qca.qualcomm.com>
SVN-Revision: 45657
When using GREP_OPTIONS to supply default options to grep, the buildsystem might get broken (For example adding --color=always breaks it)
This patch will empty the GREP_OPTIONS to prevent the described (and any other) problems related to GREP_OPTIONS
Signed-off-by: Maarten Bezemer <m.m.bezemer@utwente.nl>
SVN-Revision: 22443
