We can't run mkusers inside the chroot because of bwrap peculiarities.
Presently, rootbld does therefore not work at all for APKBUILDs which
use $pkggroups/$pkguser. While not polluting the host is a noble goal
it isn't really useful if it causes rootbld to be incapable of building
certain packages (i.e. those using $pkggroups/$pkguser with groups
not existent on the host).
This commit therefore restores the original behavior for now. While at
it, I also added a comment to the mkusers invocation.
See: https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10094
This reverts commit 84d7b7693d.
In abuild's getopts loop, option variables such as "keep", "verbose",
etc are only set if the corresponding option is found. If such an option
is *not* found, any environment variable with the same name will leak
in, instead. Prevent this by explicitly unsetting almost all of them.
the previous implementation used -regex, which is subtly different between busybox and findutils
[0-9]\+ matches on busybox, but doesn't match with gnu findutils
[0-9]+ matches with findutils, but doesn't match on busybox
this means python deps were subtly broken when findutils was installed
(sometimes pulled via makedeps) vs not
Similar to suid binaries, abuild will now error out if the package
includes binaries with setcap(8) capabilities but doesn't have `setcap`
in `$options`. This eases identifying package which ship binaries
with extra capabilities.
Furthermore, if these binaries are executable by others a warning is
emitted. This warning could be changed to an error in the future.
The recommendation is to make such binaries only executable by owner
and group, thereby requiring the system administrator to explicitly
add users to a specific group in order to give them accesses to these
capabilities.
See: https://gitlab.alpinelinux.org/alpine/tsc/-/issues/45
Discussion: This change requires abuild to depend on the `libcap`
package for the `getcap` binary. It does not seem to be possible
at the moment to use scanelf(1) to identify these binaries.
strictly speaking, it is possible for an x86_64 cpu to run 32-bit
userspace binaries without qemu emulation. it is also possible for an
aarch64 cpu to run armhf/armv7 binaries (as long as the cpu implements
it, most do). rather than check for every possible combination of when
this is allowed (host cpu + emulated target, does cpu support it, ...),
just downgrade this case to a warning, to permit non-emulated use.
ref https://gitlab.alpinelinux.org/alpine/abuild/-/merge_requests/117#note_255174
otherwise every package prints
>>> gotosocial-openrc*: Running postcheck for gotosocial-openrc
find: /builds/raspbeguy/aports/testing/gotosocial/pkg/gotosocial-openrc/usr/lib/python*: No such file or directory
the other postchecks already conditionalise on if [ -d, but we use a wildcard here
no actual package change
Python by default pre-compiles cache files in __pycache__ directories,
which we currently happily install along in python packages.
Theses .pyc files are rather big and the time/space tradeoff could be
left to users if we just split these out to a -pyc subpackage.
With this default_pyc helper, one can add $pkgname-pyc to their
package's subpackages and it will automatically split off the pyc files
in a package that will be automatically installed if the virtual 'pyc'
package is installed.
Note that this does not work so easily if there already were python
subpackages, the function could be adjusted to strip off the last dash
if required but that seems rather rare.
Random data, sizes:
- python3: currently 47MiB, split into 23M (main package) / 24M (pyc)
- py3-markdown: currently 700KiB, 368K (main) / 288K (pyc)
Random benchmark, with python3-pyc:
hyperfine --warmup 5 -m 100 \
"python3 -c 'import time; print(time.strftime(\"%T\"))'"
Time (mean ± σ): 24.5 ms ± 2.5 ms [User: 18.4 ms, System: 6.0 ms]
Range (min … max): 19.4 ms … 28.9 ms 148 runs
without python3-pyc (same as user without root permissions, root would
generate files on first root, for reference this command generates 184KB
of pyc files):
hyperfine --warmup 5 -m 100 \
-p 'rm -rf /usr/lib/python3.10/__pycache__ /usr/lib/python3.10/encodings/__pycache__' \
"python3 -c 'import time; print(time.strftime(\"%T\"))'"
Time (mean ± σ): 53.7 ms ± 4.3 ms [User: 39.3 ms, System: 14.3 ms]
Range (min … max): 47.0 ms … 65.6 ms 100 runs
Link: https://gitlab.alpinelinux.org/alpine/aports/-/issues/11906
Suggested-by: Alex Xu (Hello71) <alex_y_xu@yahoo.ca>
Packages installing python3 site packages for python3 in version 3.x.y
depend on python3~3.x. This automatically adds the required
dependencies.
Unit test cases have been added by reusing the `py3-foo-and-bar` test
package. However, the path of that has been renamed to contain spaces
to be extra sure the logic is safe in regrade to spaces in path
names.
Otherwise, if a different REPODEST is being used (e.g. due to
`buildrepo -d <repo-dest>`) then the abuild invocation in the
created chroot will not write packages to the correct REPODEST.
Therefore, `buildrepo -R -d` does presently not work correctly.
This commit fixes this by also copying the REPODEST value from
the environment.
This fixes a regression introduced in 1582617eb8ba3df4752f8050f0412c0353c33fdf.
since 1582617eb8ba3df4752f8050f0412c0353c33fdf, it's not passed,
so it's not possible to skip tests for a run via a
$ ABUILD_BOOTSTRAP=1 abuild rootbld
This reverts commit 489fc06e40.
this needs way more thought to work, see
32f314f8076d509bd4c541b1d250b3744947867f in aports
we should probably just create *-pyc splits instead. these
won't reduce mirror size, but at least allow easily uninstalling the cache.
Otherwise, user-set environment variables can leak into the container
and cause spurious build/test failures. A common example is the value of
the SHELL environment variable which is used by a lot of software.
Outside of the bwrap container I use ksh and my SHELL environment
variable points to /bin/ksh, however, inside the container /bin/ksh is
not available and hence software relying on $SHELL doesn't work
properly. This can cause annoying to debug test failures, e.g. !43430.
apk 2.14 now warns on missing deps in the same repo even with --quiet:
WARNING: No provider for the dependencies:
/bin/sh aardvark-dns abseil-cpp-dev acl acl-dev alsa-lib-dev android-tools aom-dev apache2 at-spi2-core at-spi2-core-dev atomicparsley attr attr-dev
audacious autoconf avahi avahi-dev aws-c-cal-dev aws-c-compression-dev aws-checksums-dev baloo-dev bash bc binutils binutils-dev black blas-dev bluez
