Now that util-linux is building libuuid we can simply use that instead
of manually pointing to e2fsprogs libuuid so we can disable building
libuuid in e2fsprogs.
Link: https://github.com/openwrt/openwrt/pull/15806
Signed-off-by: Robert Marko <robimarko@gmail.com>
We are now using util-linux to provide libuuid so disable building it
in e2fsprogs and simply use the util-linux provided one.
Disabling libuuid removes the need for custom install recipe.
Link: https://github.com/openwrt/openwrt/pull/15806
Signed-off-by: Robert Marko <robimarko@gmail.com>
util-linux supports passing --disable-all-programs configure flag to
disable building anything that isnt then manually enabled.
So, lets switch to using that instead of manually having to disable all
tools we dont need.
However, current drawback is that there is no upstream support for enabling
building hexdump so I included a patch that is pending upstream[0].
[0] https://github.com/util-linux/util-linux/pull/3101
Link: https://github.com/openwrt/openwrt/pull/15806
Signed-off-by: Robert Marko <robimarko@gmail.com>
Since the Yafut tool is now used for both updating the kernel on
MikroTik devices with NAND flash and preparing firmware images for
MikroTik devices with NOR flash, remove the kernel2minor utility from
the tree as it is no longer used for anything.
Signed-off-by: Michał Kępień <openwrt@kempniu.pl>
Link: https://github.com/openwrt/openwrt/pull/13453
Signed-off-by: Robert Marko <robimarko@gmail.com>
The Yafut tool now has limited capabilities for working on filesystem
images stored in regular files. This enables preparing Yaffs2 images
for devices with NOR flash using upstream Yaffs2 filesystem code instead
of the custom kernel2minor tool.
Since minimizing the size of the resulting filesystem image size is
important and upstream Yaffs2 code requires two allocator reserve blocks
to be available when writing a file to the filesystem, a trick is
employed while preparing an OpenWRT image: the blank filesystem image
that Yafut operates on initially contains two extra erase blocks that
are chopped off after the kernel file is written. This is safe to do
because Yaffs2 has a true log structure and therefore only ever writes
sequentially (and the size of the kernel file is known beforehand).
While the two extra erase blocks are necessary for writes, Yaffs2 code
seems to be perfectly capable of reading back files from a "truncated"
filesystem that does not contain these extra erase blocks.
In terms of image size, this new approach is only marginally worse than
the current kernel2minor-based one: specifically, upstream Yaffs2 code
needs to write three object headers (each of which takes up an entire
data chunk) when the kernel file is written to the filesystem:
- an object header for the kernel file when it is created,
- an object header for the root directory when the kernel file is
created,
- an updated object header for the kernel file when the latter is
fully written (so that its new size can be recorded).
kernel2minor only writes two of these headers, which is the absolute
minimum required for reading the file back. This means that the
Yafut-based approach causes firmware images to be at most one erase
block (64 kB) larger than those created using kernel2minor, but only in
the very unfortunate scenario where the size of the kernel file is
really close to a multiple of the erase block size.
The rest of the calculations performed when the empty filesystem image
is first prepared stems from the Yaffs2 layout used by MikroTik NOR
devices: each 65,536-byte erase block contains 63 chunks, each of which
consists of 1024 bytes of data followed by 16-byte Yaffs tags without
ECC data; each such group of 63 chunks is then followed by 16 bytes of
padding, which translates to "-C 1040 -B 64k -E" in the Yafut
invocation. Yaffs2 checkpoints and summaries are disabled (using
Yafut's -P and -S switches, respectively) as they are merely performance
optimizations that require extra storage space. The -L and -M switches
are used to force little-endian or big-endian byte order (respectively)
in the resulting filesystem image, no matter what byte order the build
host uses. The tr invocation is used to ensure that the filesystem
image is initialized with 0xFF bytes (which are an indicator of unused
space for Yaffs2 code).
Signed-off-by: Michał Kępień <openwrt@kempniu.pl>
Link: https://github.com/openwrt/openwrt/pull/13453
Signed-off-by: Robert Marko <robimarko@gmail.com>
ZSTD and libdeflate do not depend on SED nor flock, so instead of the whole
for loop that will filter 2 out of 4 core packages just specify that patch
and tar depend on sed explicitly.
ZSTD now depends on libdeflate since libdeflate-gzip will then be used to
unpack ZSTD as well as most tool archives.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: Robert Marko <robimarko@gmail.com>
Convert to make and drop meson dependency since it's not a core tools
and can't depend on advanced build system like cmake or meson.
On top of this make is the official build support and cmake/meson are
supported by 3rd parties.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
since kernel 6.4, commit bca2f3a9406b ("efi/zboot: Add BSS padding
before compression") introduces the use of hexdump to padding the
EFI kernel binary before compression.
util-linux which containing hexdump should then be compiled as a host
tool to guarantee not breaking the kernel build process.
Signed-off-by: Weijie Gao <hackpascal@gmail.com>
mold is a faster drop-in replacement for existing Unix linkers.
A single binary is able to link various targets, which is why this lives
in tools/.
All toolchain builds then just need to copy the linker over, hence avoiding
multiple builds with the same outcome.
Signed-off-by: Andre Heider <a.heider@gmail.com>
Using the local gnulib source during bootstrap
allows for fine-grained control over the macros
and source files for use with coreutils
but part of gnulib instead of coreutils,
without having to wait for a release
or deal with gnulib as a git submodule.
In this case, the execution of autotools
must be skipped by force.
Autoconf and Automake during bootstrap on coreutils
only works right when using directly checked-out source.
There is a symbol in gnulib, @GNULIB_TIME@
that is not yet defined in coreutils source,
so we use the backup of lib/time.in.h instead
of the one provided by gnulib source.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
Using the local gnulib source during bootstrap
allows for fine-grained control over the macros
and source files for use with libtool
but part of gnulib instead of libtool,
without having to wait for a release
or deal with gnulib as a git submodule.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
By having a local copy of gnulib, we can:
import the latest macro fixes into any package,
get rid of some statically stored macros that were otherwise missing,
bootstrap GNU tools with the latest relevant source
without having to wait for a release or rely on git submodules,
and possibly more...
The patch assists in bootstrapping by ignoring
the building of po files using gettext,
and also to allow a user-defined path to a program
to include parameters.
Signed-off-by: Michael Pratt <mcpratt@pm.me>
`bzip2` is the standard executable for bzip2 compression
this includes development includes and both static and shared libs
(libbz2) which can be used by other packages
the initramfs generator offers the BZIP2 option but there was no
executable to support it, and worked only via side effect of having a
system-installed version of bzip2, which could be less predictable
Signed-off-by: Tony Butler <spudz76@gmail.com>
[ remove unintended change ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This reverts commit 394d7134ec.
The commit has unintentded change that cause compilation error with SDK
or LZO compression.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
`bzip2` is the standard executable for bzip2 compression
this includes development includes and both static and shared libs
(libbz2) which can be used by other packages
the initramfs generator offers the BZIP2 option but there was no
executable to support it, and worked only via side effect of having a
system-installed version of bzip2, which could be less predictable
Signed-off-by: Tony Butler <spudz76@gmail.com>
squashfs tool is finally reborn and correctly maintained.
Introduce the new version as a replacement for squasfs4kit as it was a
fork and also abandoned.
Add additional patch to add the missing feature present in squashfskit4
but still missing on this new project.
Backport each required patch that fix compilation error on macos.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
The name of squashfs is confusing since in reality it's a really old
version using an old lzma library. This tools is used for old ath79
netgear target and to produde a fake squasfs3 image needed for some
specific bootloader from some OEM (AVM for example)
Rename squashfs tool to squasfs3-lzma to better describe it.
Rename the installed bin from mksquashfs-lzma to mksquashfs3-lzma.
Use tar transform to migrate the root directory in tar to the new
naming.
Drop redundant PKG_CAT variable not needed anymore.
Also update any user of this tool.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
CMake depends on (libdeflate-)gunzip, libdeflate depends on Cmake, so we
can't win.
Luckily libdeflate is _very_ easy to build, without any build system, so
lets just manually compile it and be done with it.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Like with commit ae614fb397 ("tools: Improve diffability/maintainability")
we also want tools-core to be easy to maintain. While a smaller target,
it's still usefull and makes things nice and consistent.
To avoid duplicating any tools in the comment, simplify the comment
instead.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Depends: `tools/liblzo`
`lzop` is the standard executable for LZO compression
the initramfs generator offers the LZO option but there was no
executable to support it actually working
Signed-off-by: Tony Butler <spudz76@gmail.com>
prerequisite of upcoming `tools/lzop` addition, and subsequent initramfs
and squashfs cleanups
same as `packages/lzo` modified to be a HOST/tools type build,
and should always be the same version and sources when either one is
bumped
because this (and `packages/lzo`) only provide liblzo and no
executables, use the clearer name `tools/liblzo`
Signed-off-by: Tony Butler <spudz76@gmail.com>
same as `packages/liblz4` modified to be a HOST/tools type build with
unified liblz4 (this is also the dev package for liblz4)
the image initramfs generator offers the LZ4 option but there was no
executable to support it actually working
Signed-off-by: Tony Butler <spudz76@gmail.com>
Several devices provide U-Boot versions with only gzip compressed kernel
support (e.g. Realtek switches). This compression method produces larger
images than lzma. To save space on flash and avoid going the hard way with
lzma-loader we can make use of enhanced gzip tool based on libdeflate
compression library from https://github.com/ebiggers/libdeflate. It
keeps 100% deflate/gzip compatibility while improving compression ratio.
The image can be unpacked by the default inflate routines inside U-Boot.
Signed-off-by: Markus Stockhausen <markus.stockhausen@gmx.de>
[Switched to v1.15 and made it work with cmake]
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Reviewed-by: Robert Marko <robimarko@gmail.com>
Reviewed-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Sander Vanheule <sander@svanheule.net>
There's no purpose to squish multiple tools into a single line (and
spread those out over multiple lines). It might look 'nice' in certain
conditions, but it's annoying to maintain.
For example, but not limited to:
* adding/removing tools, causes hard to read diffs
* Duplicates are harder to spot
* Sorting can not be (easily?) automated
With this proposed change, the above annoyances go away. Inserting a new
tool can be done with a single line-change-diff, sorting can be done by
any editor (in vi, select, :sort for example) and dupes are much easier
to spot.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Get rid of drivers that are either limited to 802.11b/g or don't even support
cfg80211/mac80211. Most of these are either limited to boards that we don't even
support anymore because of firmware size, or were only used for custom hacks by
a really small number of users in the past.
Let's get rid of those to reduce the maintenance effort and the number of useless
packages
Signed-off-by: Felix Fietkau <nbd@nbd.name>
This fixes the following build error:
./autogen.sh: line 13: aclocal: command not found
Fixes: b6d29af947 ("tools/genext2fs: update to 1.5.0")
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
Add option to compile all host tools even if not needed.
This can be useful to prepare a universal precompiled host tools
archive to use in another buildroot and speedup compilation.
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Saves a little bit of time when compiling cmake.
Added patches to fix searching liblzma and zlib. The issue is that
because pkgconfig is not used, the system libraries get used.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
It shouldn't be needed anymore as we've now `scripts/xxdi.pl`, which
should be self contained and fully compatible `xxd -i` replacement.
Fixes: #10555
Signed-off-by: Petr Štetiar <ynezz@true.cz>
Add the 7zr command line tool, which is a version of the 7z application
that only supports 7z archives.
7z is one of the two compression formats supported in H3C firmware
images (the alternative would be ARJ).
(Alternatively, the 7zr command line tool could also be built from a
current version of the public-domain LZMA SDK. That would require
repackaging the source package, as it is only provided in 7z format.)
Signed-off-by: Jan Hoffmann <jan@3e8.eu>
Some buildbots fail to build elfutils due to m4 being missing. Add m4 as
a dependency for elfutils to fix this.
Fixes: ad79b92719 ("elfutils: move host build to tools")
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
dwarves is a set of tools that use the debugging information inserted in
ELF binaries by compilers such as GCC. Utilities in the dwarves suite
include pahole, which can be used to find alignment holes in structs and
classes, and also extracts other information such as CPU cacheline
alignment, helping pack those structures to achieve more cache hits.
These tools are also used to encode and read the BTF type information
format used with the bpf syscall, making this a Linux build dependency
when using kernel BTF information.
Signed-off-by: Tony Ambardar <itugrok@yahoo.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
[bump to 1.23, add elfutils dep, drop host lib usage, drop cmake release
target, use RM macro]
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
The upcoming dwarves host package requires elfutils. As dependencies for
tools must exist in tools, we need to move elfutils host build there.
As there is at least one package that depends on this, and there is no
proper way to create such dependency in the build system, build it
unconditionally when not building on macOS.
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
Switched to CMake for faster compilation and greater parallel
friendliness.
Added CMake options from the packages feed.
This release fixes various CVEs.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Switched to building with meson as it's faster and does not need a
dependency on cmake, which takes a long time to build.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Switched to building with meson as it's faster and does not need a
dependency on cmake, which takes a long time to build.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
On macOS, system binaries silently drop the environment variables for injecting
extra shared libraries (used by fakeroot). This is done for security reasons.
Work around this by building bash from source, so that it gets an ad-hoc signature
and does not have these restrictions
Signed-off-by: Felix Fietkau <nbd@nbd.name>
Robert Marko
Michał Kępień
Rosen Penev
Christian Marangi
Weijie Gao
Felix Fietkau
Andre Heider
Michael Pratt
Tony Butler
Olliver Schinagl
Markus Stockhausen
Hauke Mehrtens
Petr Štetiar
Jan Hoffmann
Stijn Tintel
Tony Ambardar
Huangbin Zhan
Jo-Philipp Wich