Python by default pre-compiles cache files in __pycache__ directories,
which we currently happily install along in python packages.
Theses .pyc files are rather big and the time/space tradeoff could be
left to users if we just split these out to a -pyc subpackage.
With this default_pyc helper, one can add $pkgname-pyc to their
package's subpackages and it will automatically split off the pyc files
in a package that will be automatically installed if the virtual 'pyc'
package is installed.
Note that this does not work so easily if there already were python
subpackages, the function could be adjusted to strip off the last dash
if required but that seems rather rare.
Random data, sizes:
- python3: currently 47MiB, split into 23M (main package) / 24M (pyc)
- py3-markdown: currently 700KiB, 368K (main) / 288K (pyc)
Random benchmark, with python3-pyc:
hyperfine --warmup 5 -m 100 \
"python3 -c 'import time; print(time.strftime(\"%T\"))'"
Time (mean ± σ): 24.5 ms ± 2.5 ms [User: 18.4 ms, System: 6.0 ms]
Range (min … max): 19.4 ms … 28.9 ms 148 runs
without python3-pyc (same as user without root permissions, root would
generate files on first root, for reference this command generates 184KB
of pyc files):
hyperfine --warmup 5 -m 100 \
-p 'rm -rf /usr/lib/python3.10/__pycache__ /usr/lib/python3.10/encodings/__pycache__' \
"python3 -c 'import time; print(time.strftime(\"%T\"))'"
Time (mean ± σ): 53.7 ms ± 4.3 ms [User: 39.3 ms, System: 14.3 ms]
Range (min … max): 47.0 ms … 65.6 ms 100 runs
Link: https://gitlab.alpinelinux.org/alpine/aports/-/issues/11906
Suggested-by: Alex Xu (Hello71) <alex_y_xu@yahoo.ca>
Packages installing python3 site packages for python3 in version 3.x.y
depend on python3~3.x. This automatically adds the required
dependencies.
Unit test cases have been added by reusing the `py3-foo-and-bar` test
package. However, the path of that has been renamed to contain spaces
to be extra sure the logic is safe in regrade to spaces in path
names.
Otherwise, if a different REPODEST is being used (e.g. due to
`buildrepo -d <repo-dest>`) then the abuild invocation in the
created chroot will not write packages to the correct REPODEST.
Therefore, `buildrepo -R -d` does presently not work correctly.
This commit fixes this by also copying the REPODEST value from
the environment.
This fixes a regression introduced in 1582617eb8.
This reverts commit 489fc06e40.
this needs way more thought to work, see
32f314f8076d509bd4c541b1d250b3744947867f in aports
we should probably just create *-pyc splits instead. these
won't reduce mirror size, but at least allow easily uninstalling the cache.
Otherwise, user-set environment variables can leak into the container
and cause spurious build/test failures. A common example is the value of
the SHELL environment variable which is used by a lot of software.
Outside of the bwrap container I use ksh and my SHELL environment
variable points to /bin/ksh, however, inside the container /bin/ksh is
not available and hence software relying on $SHELL doesn't work
properly. This can cause annoying to debug test failures, e.g. !43430.
apk 2.14 now warns on missing deps in the same repo even with --quiet:
WARNING: No provider for the dependencies:
/bin/sh aardvark-dns abseil-cpp-dev acl acl-dev alsa-lib-dev android-tools aom-dev apache2 at-spi2-core at-spi2-core-dev atomicparsley attr attr-dev
audacious autoconf avahi avahi-dev aws-c-cal-dev aws-c-compression-dev aws-checksums-dev baloo-dev bash bc binutils binutils-dev black blas-dev bluez
Some languages (such as Hare) do not support position-independent
executables. Such cases require extra care and scrutiny, but should not
be entirely disallowed.
Currently, abuild stores the license variable ad-verbatim in the PKGINFO
file. This causes many downstream systems to fail. For example, `apk
info` won't show anything after the newline, our webview that lists
licenses do not show these either.
Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>
Otherwise apk will rely on defaults 'etc/apk/cache' and 'var/apk/cache'
based on new chroot directory and will delete everything when rootbld finishes.
The default provider priority is 0, which disables the normal way apk
selects an unversioned virtual dependency. Instead, version the
provider so that cmd: dependencies can be robustly used like normal
dependencies.
git log may return color codes if users colored output forcibly enabled
with:
[color]
ui = always
Use `git rev-list` instead of `git log` to make sure that we don't get
any color codes.
fixes https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10042
Now all python packages that install python modules under
/usr/lib/pythonX.Y/site-packages will have a provider that indicates
their MAJOR (X) and MINOR (Y) versions.
pyX maps to the MAJOR version of the package, its objective is to allow
users to quickly install a python module without having to search around
for the correct package, doing `apk add py3.9:foo` will install whatever
packages provides the foo module.
The directories checked only go one level deep, so
'/usr/lib/python3.9/site-packages/date' will generate
py3:date=$pkgver-r$pkgrel.
files ending with .py also count and are added with their .py prefix
stripped away. so '/usr/lib/python3.9/site-packages/six.py' will
generate py3:six=$pkgver-r$pkgrel.
The reason for doing this is the same as creating pc:, so: and cmd:, it
is more reliable and robust to depened on what we known of what the package
provides than to try to guess via the pkgname.
Co-authored-by: Chloe Kudryavtsev <toast@toast.cafe>
If there are only empty files in the pkgdir, on some filesystems
(discovered on btrfs), du might return 0 for the sum size of the files.
But apk-tools considers packages with size = 0 to be virtual and skips
extraction of any files contained.
To work around that (until it is resolved in apk-tools 3), settings the
size to 1 when it is zero AND some files are present should work fine.
In order to make diffs more tidy and the APKBUILD overall more visually
pleasing, new lines are added after opening and before closing quote,
turning
sha512sums="HASH foo
HASH bar"
into
sha512sums="
HASH foo
HASH bar
"
therefore keeping alignment of file names.
By default, fakeroot forwards chown system calls to the libc and ignores
EPERM errors. Unfortunately, when fakeroot is used inside a restricted
bubblewrap environment, bubblewrap intercepts these system calls and
returns EINVAL. The EINVAL return value is not masked by fakeroot and
returned to the caller, thereby causing failures of mv(1), install(1),
chown(1), … which use the chown system calls internally.
Setting the FAKEROOTDONTTRYCHOWN environment variable prevents the chown
systemcalls to be performed in the first place. This variable is only
set in the environment created through bubblewrap by abuild rootbld.
Fixes#10021
the SRCDEST may be shared by multiple builders and apparently cargo does
not seem to handle that well. There are also no good way to do clean-up
of the cache for purging things that is no longer in use.
This reverts commit af0c88e6ab.
Store exit code in the variable 'e', then return it, if there are no
errors then it will return 0 as that is the default value, otherwise it
will return 1 and error out as previously
This commit broke abuild-rmtemp which has a prefix check to ensure it
doesn't remove non-abuild directories. Revert the commit instead of
adjusting the prefix check for now.
This reverts commit d0828c0655.
This is the only appearance of `apk` in the source. Appears accidental.
Fixes: 41343329 ("abuild: fix dependency tracing for cross builds")
Signed-off-by: A. Wilcox <AWilcox@Wilcox-Tech.com>
This will warn then paths for certain shells that have completions are
found and tell the user to add a subpackage for it.
In a future date we also want to warn the user to move certain
directories where packages have completions to those directories so our
default_ functions can move them without problems
CARGO_HOME tells cargo where to store installed dependencies, save it to
a directory in $SRCDEST so we don't need to download all dependencies
again when compiling a rust package.
This makes it easier to figure out to which build the directory belongs
to. Occasionally, I have many failed abuild rootblds in my /var/tmp and
including the $pkgname in the directory would help me associating the
directories with failed builds I recently executed.
Fixes the error:
df: .: can't find mount point
When running abuild inside a chroot when the root file system
mountpoint information is not necessarily directly available.
Without this change abuild rootbld would fail with:
touch: invalid date '@'
Because SOURCE_DATE_EPOCH wasn't set when abuild rootbld was used. This
is a bug introduced in 71d9d5233b. Instead
of reverting the aforementioned commit move the SOURCE_DATE_EPOCH
initialization to a custom function and also call it from the abuild
rootbld function.
Fixes#9978
Natanael Copa
Dominique Martinet
Marian Buschsieweke
Sören Tempel
psykose
ptrcnull
Jakub Jirutka
Ariadne Conill
swiftlebottomE
Rohit Lodha
Drew DeVault
xdavidwu
Nathan Rennie-Waldock
Olliver Schinagl
donoban
Ross Younger
Alex Xu (Hello71)
Alex D.
Samanta Navarro
Kaarle Ritvanen
Leo
Wolf
Rasmus Thomsen
Minecrell
A. Wilcox
Wictor Lund
Timo Teräs
kpcyrd
Bart Ribbers