Since https://gitlab.com/kernel-firmware/linux-firmware/-/merge_requests/319
and https://gitlab.com/kernel-firmware/linux-firmware/-/merge_requests/331 ,
doing `make COPYOPTS="blahblah" install` does not fail, but does
not do what it did before. This is a surprising and possibly
undesired result (in Fedora, it led to our firmware suddenly
taking up 500MB more space on disk, and that led to a bunch of
live image builds failing).
To make the change more apparent, let's just fail out with an
explanatory message if invoked with COPYOPTS set.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Co-authored by: Emil Velikov <emil.l.velikov@gmail.com>
For better or worse some distributions cannot have the firmware
de-duplication happen automatically.
In addition, when it was introduced it changed the default behaviour
leaving people with no firmware in their systems.
Revert to the original behaviour: in the worst case, people will have a
few MB extra of duplicate firmware, yet their systems will continue to
work.
To make things stand out, we print a message at the end of install so
that everyone can opt-in as needed.
In addition, I've went ahead and opened tentative MR/PR with Alpine,
Arch, CentOS Stream, Debian, Gentoo, informing them of this change.
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Flesh out the de-duplication logic in separate script. The copy-firmware.sh is
already complex enough and de-duplication doesn't really fit in there.
In the process we migrate away from the open-coded `ln --relative`. We also
avoid touching symlinks, which are not created by rdfind. Otherwise we end up
"fixing" the folder to folder symlinks (created earlier in the process) and
things explode.
As result we also get a few bonuses:
- the COPYOPTS shell injection is gone - the variable was never used
- people can dedup as separate step if/when they choose to do so
Aside: based on the noise in git log and around distros ... I'm wondering if
having the de-duplication as opt-in, would have been better. Is it too late to
change or the ship has sailed?
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The copy-firmware.sh script continues to grow new options and adding
bespoke make targets for them all isn't really sustainable. This
introduces a COPYOPTS variable that can be set to pass arguments
directly to the script via a make invocation. For now we just
include it on the 'install' target to preserve the existing behavior
of the rest.
Signed-off-by: Josh Boyer <jwboyer@kernel.org>
A number of distributions don't include rdfind by default. This is an
optimization, so make it optional via a --ignore-duplicates option and a
new make target.
Distributions that include rdfind should add it as a dependency for
their linux-firmware package. Those that don't should use the new
target or option.
Signed-off-by: Josh Boyer <jwboyer@kernel.org>
The "which" utility is not guaranteed to be installed either, and if it
is, its behavior is not portable either. This means that when rdfind /
pre-commit are installed, the `which` check will report a fatal error
because the which tool did not exist and the shell returned a nonzero
status when attempting to fork+exec. If it did exist, it might not be an
implementation of `which` that returns nonzero when commands do not
exist.
Conversely, the "command -v" shell builtin is required to exist in all
POSIX 2008 compliant shells, and is thus guaranteed to work everywhere.
For some in-depth discussions on the topic, see:
- https://mywiki.wooledge.org/BashFAQ/081
- https://unix.stackexchange.com/questions/85249/why-not-use-which-what-to-use-then/85250#85250
Examples of open-source shells likely to be installed as /bin/sh on
Linux, which implement the 15-year-old standard: ash, bash, busybox,
dash, ksh, mksh and zsh.
A side benefit of using the POSIX portable option is that it requires
neither an external disk executable, nor (because unlike "which", the
exit code is reliable) a subshell fork. This therefore represents a mild
speedup.
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
The package would put all files into /lib/firmware/updates to avoid
conflicting with distro packages.
The package is also intentionally named to avoid conflicts with
distro packages.
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
This target produces a tarball following the most recent git tag.
The tarball contains firmware binaries that have been de-duped.
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
The kernel has supported compressed firmware for quite some time. So
let's add a couple of targets to produce that. In practical terms this
means it we'll use ~5x times less space on disk.
Reportedly the amd ucode, needs to be uncompressed _within_ the
initrd in order to work. Using compressed ucode in late load just works.
Ideally this will be addressed by the initrd generators, but considering
the files are tiny in size let's skip the compression.
v2
- commit message, skip compression for files annotated as Raw
v3
- rebase
[Drop extra verbose statement in zstd case, Josh Boyer
<jwboyer@kernel.org>]
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Josh Boyer <jwboyer@kernel.org>
mkdir -p creates paths that are bound to user's settings and therefore
can lead to different file mode bits of the base paths accross different
machines.
Use install instead, as this tool is not prone to such behavior.
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Josh Boyer <jwboyer@kernel.org>
Silence the make echo'ing, so the output is cleaner.
Add an exit code to check_whence.py, so we can get a real failure for
bad WHENCE files.
Signed-off-by: Brian Norris <briannorris@chromium.org>
Signed-off-by: Josh Boyer <jwboyer@kernel.org>
The current make-install procedure leaves lots of garbage files that
aren't really firmware files in /lib/firmware.
Instead of copy-all-and-prune approach, copy only the listed files and
links in WHENCE by make-install for assuring only the proper firmware
files.
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Signed-off-by: Josh Boyer <jwboyer@kernel.org>
The script compares the files listed in WHENCE (or otherwise expected)
and the files known to git, and reports all differences as errors.
Add a 'check' rule to the Makefile that runs this.
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
One more step on the road to entirely eliminating the concept of
packages. I adapted the installation rules from the Fedora
spec file.
They should clearly be less lame, but to do so we'd need more formal
rules about which files should be installed. Maybe just limit it to
things ending in ".bin" or ".fw"?
See http://people.gnome.org/~walters/docs/build-api.txt
Signed-off-by: Colin Walters <walters@verbum.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Adam Williamson
Emil Velikov
Josh Boyer
Eli Schwartz
Mario Limonciello
Konrad Weihmann
Brian Norris
Takashi Iwai
Ben Hutchings
Kyle McMartin
Colin Walters