There are no functional changes, only cleanup of header files. This
could lead to build failures in case the headers were used as a
convenience outside of scope of libbtrfs just because of the kernel
compatibility.
- removed various definitions of variables, types, helpers and macros
from kerncompat.h that are neither used nor needed for libbtrfs code
- file list.h no longer shipped
- file rbtree.h no longer shipped
Signed-off-by: David Sterba <dsterba@suse.com>
Commit [1] 164bc10d "btrfs-progs: add musl compatibility for printf
format %pV" added a logic to detect the presence of the glibc
<printf.h> header, and if present, to use the
register_printf_specifier() and register_printf_modifier() functions.
The original intent (as the commit log suggests), was to support the
musl libc, which does not provides this <printf.h> header.
When compiling with another libc, such as uClibc-ng, btrfs-progs fail
to build with error:
common/messages.c: In function 'print_va_format':
common/messages.c:51:19: error: 'const struct printf_info' has no member named 'user'
51 | if (!(info->user & va_modifier))
| ^~
common/messages.c: In function 'btrfs_no_printk':
common/messages.c:76:17: warning: implicit declaration of function 'register_printf_specifier'; did you mean 'register_printf_function'? [-Wimplicit-function-declaration]
76 | register_printf_specifier('V', print_va_format,
| ^~~~~~~~~~~~~~~~~~~~~~~~~
| register_printf_function
common/messages.c:78:31: warning: implicit declaration of function 'register_printf_modifier'; did you mean 'register_printf_function'? [-Wimplicit-function-declaration]
78 | va_modifier = register_printf_modifier(L"p");
| ^~~~~~~~~~~~~~~~~~~~~~~~
| register_printf_function
This is because uClibc-ng provides a <printf.h> header, but not the
register_printf_specifier() and register_printf_modifier() functions.
See [2]. It mainly includes register_printf_function(). uClibc-ng
emulates an older glibc behavior. Glibc added support for printf user
elements in commit [3] (first included in glibc-2.10, in 2009). Checking
only the <printf.h> is not sufficient.
This commit fixes this build issue by refining the detection logic of
the <printf.h> functions required by btrfs-progs.
[1] 164bc10dfc
[2] https://gogs.waldemar-brodkorb.de/oss/uclibc-ng/src/v1.0.49/include/printf.h
[3] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=9d26efa90c6dcbcd6b3e586c9927b6058ef4d529
Pull-request: #843
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
There's no obvious reason why there's -O1 instead -O2 which is commonly
used on distro builds. -O1 was enabled in c1690a3832 ("Switch to -O1
for optimizations to enable FORTIFY_SOURCE") for the fortify checks.
Signed-off-by: David Sterba <dsterba@suse.com>
Fix parsing of send stream, properly access potentially unaligned data.
This can happen on hosts with strict alignment (ARM v5 or v6).
Related issue: #770
Signed-off-by: David Sterba <dsterba@suse.com>
There is a new release candidate for e2fsprogs https://github.com/tytso/e2fsprogs/releases/tag/v1.47.1-rc2
Linking btrfs-progs v6.8 against this version of e2fsprogs leads to the following compile error:
convert/source-ext2.c: In function 'ext4_copy_inode_timespec_extra':
convert/source-ext2.c:733:13: warning: implicit declaration of function 'inode_includes' [-Wimplicit-function-declaration]
733 | if (inode_includes(inode_size, i_ ## xtime ## _extra)) { \
| ^~~~~~~~~~~~~~
convert/source-ext2.c:769:9: note: in expansion of macro 'EXT4_COPY_XTIME'
769 | EXT4_COPY_XTIME(atime, dst, tv_sec, tv_nsec);
| ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: error: 'i_atime_extra' undeclared (first use in this function)
733 | if (inode_includes(inode_size, i_ ## xtime ## _extra)) { \
| ^~
convert/source-ext2.c:769:9: note: in expansion of macro 'EXT4_COPY_XTIME'
769 | EXT4_COPY_XTIME(atime, dst, tv_sec, tv_nsec);
| ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: note: each undeclared identifier is reported only once for each function it appears in
733 | if (inode_includes(inode_size, i_ ## xtime ## _extra)) { \
| ^~
convert/source-ext2.c:769:9: note: in expansion of macro 'EXT4_COPY_XTIME'
769 | EXT4_COPY_XTIME(atime, dst, tv_sec, tv_nsec);
| ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: error: 'i_mtime_extra' undeclared (first use in this function)
733 | if (inode_includes(inode_size, i_ ## xtime ## _extra)) { \
| ^~
convert/source-ext2.c:770:9: note: in expansion of macro 'EXT4_COPY_XTIME'
770 | EXT4_COPY_XTIME(mtime, dst, tv_sec, tv_nsec);
| ^~~~~~~~~~~~~~~
convert/source-ext2.c:733:40: error: 'i_ctime_extra' undeclared (first use in this function)
733 | if (inode_includes(inode_size, i_ ## xtime ## _extra)) { \
| ^~
convert/source-ext2.c:771:9: note: in expansion of macro 'EXT4_COPY_XTIME'
771 | EXT4_COPY_XTIME(ctime, dst, tv_sec, tv_nsec);
| ^~~~~~~~~~~~~~~
convert/source-ext2.c:774:40: error: 'i_crtime_extra' undeclared (first use in this function)
774 | if (inode_includes(inode_size, i_crtime_extra)) {
| ^~~~~~~~~~~~~~
from tytso/e2fsprogs@ca8bc92
Fix inode_includes() macro to properly wrap "inode" parameter,
and rename to ext2fs_inode_includes() to avoid potential name
clashes. Use this to check inode field inclusion in debugfs
instead of bare constants for inode field offsets.
To fix that use the new prefixed macro and add backward compatibility that
would still use inode_includes().
Issue: #785
Signed-off-by: David Sterba <dsterba@suse.com>
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
The "=" operator should be used as configure may be run by a POSIX shell
at /bin/sh (like dash). Bash recognises "=" too so this retains
compatibility with it.
Pull-request: #721
Signed-off-by: Sam James <sam@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Compiling with clang on aarch64 leads to an error when detecting the
SIMD instruction support. Gcc ignores the arch/feature mismatch.
Conditionally detect the -m flags only on x86_64.
Issue: #712
Signed-off-by: David Sterba <dsterba@suse.com>
https://www.openssl.org/ Is a well known cryptography library and since
freshly released version 3.2 it also supports variable digest size of
blake2b, so we can now add it among the crypto providers.
Configure with --with-crypto=openssl.
Signed-off-by: David Sterba <dsterba@suse.com>
https://botan.randombit.net/ Botan is a cryptography library with C
bindings and provides what we need (sha256 and blake2b), among many
others. Add it to the list of crypto backends if somebody wants to use
it.
Currently the version 2.19 is the latest one. Botan3 3.2.0 exists but
does not seem to be widely available in distros yet.
Configure with --with-crypto=botan.
Signed-off-by: David Sterba <dsterba@suse.com>
The accelerated crc32c needs to check for two CPU features, the crc32c
instructions is in SSE 4.2 and 'pclmulqdq' is a separate. There's still
old hardware used that does not have the PCLMUL instructions. Detect it
and make it the condition.
The pclmul is not supported on old compilers so also add a
configure-time detection and leave the SSE 4.2 only implementation as
the accelerated one if possible.
Issue: #676
Signed-off-by: David Sterba <dsterba@suse.com>
Copy faster implementation of crc32c from linux kernel as of 6.5-rc7
(x86_64, arch/x86/crypto/crc32c-pcl-intel-asm_64.S). This needs
assembler build support, so detect target architecture so
cross-compilation still works.
Add a special CPU flag so the old and new implementations can be
benchmarked and verified separately.
Sample benchmark:
CPU flags: 0x1ff
CPU features: SSE2 SSSE3 SSE41 SSE42 SHA AVX AVX2 CRC32C_PCL
Block size: 4096
Iterations: 1000000
Implementation: builtin
Units: CPU cycles
NULL-NOP: cycles: 77177218, cycles/i 77
NULL-MEMCPY: cycles: 226313072, cycles/i 226, 62133.395 MiB/s
CRC32C-ref: cycles: 24418596066, cycles/i 24418, 575.859 MiB/s
CRC32C-NI: cycles: 1188335920, cycles/i 1188, 11833.073 MiB/s
CRC32C-PCL: cycles: 463193456, cycles/i 463, 30358.037 MiB/s
XXHASH: cycles: 851606646, cycles/i 851, 16511.916 MiB/s
SHA256-ref: cycles: 74476234956, cycles/i 74476, 188.808 MiB/s
SHA256-NI: cycles: 34198637428, cycles/i 34198, 411.177 MiB/s
BLAKE2-ref: cycles: 14761411664, cycles/i 14761, 952.597 MiB/s
BLAKE2-SSE2: cycles: 18101896796, cycles/i 18101, 776.807 MiB/s
BLAKE2-SSE41: cycles: 12599091062, cycles/i 12599, 1116.087 MiB/s
BLAKE2-AVX2: cycles: 9668247506, cycles/i 9668, 1454.418 MiB/s
The new implementation is about 2.5x faster.
Note: there new version does not work on musl because of linkage
problems (relocations in .rodata), so it's still using the old
implementation.
Signed-off-by: David Sterba <dsterba@suse.com>
Move files from m4/ to config that is also used for build and we can
reduce the number of toplevel directories.
Signed-off-by: David Sterba <dsterba@suse.com>
If I run autoreconf -fi in btrfs-progs repo in VirtualBox, it fails with
possibly undefined macro AC_DEFINE. To fix this, I added m4 as the macro
directory in configure.ac as follows. Adding that line fixes the error
when running autoreconf.
Versions:
autoreconf 2.69
automake 1.15.1
btrfs-progs v6.3-fba31d63
openSUSE 15.5
VirtualBox 6.1.40
configure.ac:7: error: possibly undefined macro: AC_DEFINE
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
configure.ac:63: error: possibly undefined macro: AC_MSG_ERROR
configure.ac:276: error: possibly undefined macro: AC_MSG_WARN
autoreconf: /usr/bin/autoconf failed with exit status: 1
Issue: #632
Signed-off-by: David Sterba <dsterba@suse.com>
Glibc provides an interface to extend the printf formats but this is not
standardized and does not work on musl. The code brought from kernel
uses %pV for varargs and also has own implementation of printk.
As a workaround for musl expand the pV value to a string and then
simply print it. The details are hidden behind macros:
- DECLARE_PV(vaf)
- PV_ASSIGN(vaf, format, args)
- PV_FMT in printf string
- PV_VAL in arguments
Signed-off-by: David Sterba <dsterba@suse.com>
The wiki has been archived so remove the links from manual page
footers. Also replace the wiki link by RTD site in configure and
libbtrfsutil.
Signed-off-by: David Sterba <dsterba@suse.com>
Change what hash-speedtest benchmarks according to the
--with-crypto=backend option. Until now it would run the same version
under different names inherited from the builting.
At configure time detect availability of all backends and define all
macros.
Signed-off-by: David Sterba <dsterba@suse.com>
The build fails with crypto backends other than builtin, the
initializers cannot be reached as they're ifdef-ed out. Move
hash_init_accel under the right condition and delete the
algorithm-specific initializers as they're used only by the hash test
and that can simply call hash_init_accel to set the implementation.
All the -m flags need to be detected at configure time and the flag used
for ifdef (HAVE_CFLAG_m*), not the actual feature defined by compiler as
the dispatcher function is not built with the -m flags.
The uname check for x86_64 must be dropped so on i386/i586 we can still
build accelerated version.
Signed-off-by: David Sterba <dsterba@suse.com>
Commit 0d5767e664 ("btrfs-progs: build: use -std=gnu11") was supposed
to set C level to gnu11 to match kernel but it was done only in the C=1
(checker) branch. The std used was still gnu90 that can be verified by
running with V=1.
Set the std option to gnu11 unconditionally and delete the m4 macros
that were detecting support on older gcc (4.5). The reference build on
lowest supported distro (Centos 7) compiles fine.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy sha256-x86.c from https://github.com/noloader/SHA-Intrinsics, that
uses the compiler intrinsics to implement the update step with the
native x86_64 instructions.
To avoid dependencies of the reference code and the x86 version, check
runtime support only if the compiler also supports -msha.
Signed-off-by: David Sterba <dsterba@suse.com>
Process an enable_verity cmd by running the enable verity ioctl on the
file. Since enabling verity denies write access to the file, it is
important that we don't have any open write file descriptors.
This also revs the send stream format to version 3 with no format
changes besides the new commands and attributes. This version is not
finalized and commands may change, also this needs to be synchronized
with any kernel changes.
Note: the build is conditional on the header linux/fsverity.h
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Use the autoconf archive macros for gcc builtin detection and add the
overflow from recently added from kernel.
New:
__builtin_add_overflow
__builtin_sub_overflow
__builtin_mul_overflow
Signed-off-by: David Sterba <dsterba@suse.com>
Now that LZO and ZSTD are optional for not just restore, rename the
build variables to a more generic name and update configure summary.
Signed-off-by: David Sterba <dsterba@suse.com>
LZO as a compression format is pretty archaic these days, there are
better algorithms in all metrics for compression and decompression, and
lzo hasn't had a new release since 2017.
Add an option to disable LZO (defaulting to enabled), and respect it in
cmds/restore.c.
NOTE: disabling support for LZO will make make it impossible to restore
data from filesystems where the compression has ever been used. It's not
recommended to build without the support in general.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There was a workaround for asciidoc/xmlto build because page btrfs.5 and
btrfs.8 used the same intermediate file. To avoid clash for the section
5 page it is part of the name, this is still needed, but for sphinx we
can use the final name as it will get the right suffix. This affects
only the generated manual pages.
Signed-off-by: David Sterba <dsterba@suse.com>
Update sphinx build so that by default it now builds the manual pages,
in quiet mode. 'make html' builds the html, other sphinx targets are
available, see 'make help'. Installation now works as well.
Note: sphinx is still conditional and must be selected by
ASCIIDOOC_TOOL=sphinx ./configure
Signed-off-by: David Sterba <dsterba@suse.com>
The zoned support requires a header file and some structures for full
support. There are distros that have only part of that and the
autodetection at configure time does not handle that properly, assuming
the user requested the support.
Check if there was any of the --*able-zoned options and if not detect
the support level and fallback to no zoned eventually, not requiring
users to specify --disable-zoned as before.
Issue: #425
Signed-off-by: David Sterba <dsterba@suse.com>
Make it possible to select sphinx doc generator instead of asciidoc so
we don't have two makefiles for that. It's still a bit crude and does
not support installing the files.
The required package is python-Sphinx (or similar name), built by
'sphinx-build'.
Configure:
$ ASCIIDOC_TOOL=sphinx ./configure
...
doc generator: sphinx
...
Generate:
$ cd Documentation/
$ make man
$ make html
$ make info # yes we can have info pages too
There are several more targets provided by sphinx, run 'make' to list
them.
Signed-off-by: David Sterba <dsterba@suse.com>
Previously the build would be optional based on what is installed on the
system. Add an option to make this selectable by user, ie. to allow
build without libudev even if it is installed on the system.
For most users and distros libudev dependency should be ok, but we've
had requests to disable various features (embedded, systems with
non-systemd user space base system) so let's do the same.
Signed-off-by: David Sterba <dsterba@suse.com>
This is needed for future code which will make btrfs-progs' device
scanning logic a little smarter by filtering out path device in
multipath setups. libudev is added as an optional dependency since the
library doesn't have a static version so making it a hard dependency
means forfeiting static build support. To alleviate this a fallback code
will be added for the static build case which doesn't rely on libudev.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Running convert-tests.sh Reported that the 019-ext4-copy-timestamps test
failed:
...
mount -o loop -t ext4 btrfs-progs/tests/test.img btrfs-progs/tests/mnt
====== RUN CHECK touch btrfs-progs/tests/mnt/file
====== RUN CHECK stat btrfs-progs/tests/mnt/file
File: 'btrfs-progs/tests/mnt/file'
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 700h/1792d Inode: 13 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: unconfined_u:object_r:unlabeled_t:s0
Access: 2021-08-24 22:10:21.999209679 +0800
Modify: 2021-08-24 22:10:21.999209679 +0800
Change: 2021-08-24 22:10:21.999209679 +0800
...
btrfs-progs/btrfs-convert btrfs-progs/tests/test.img
...
====== RUN CHECK mount -t btrfs -o loop btrfs-progs/tests/test.img btrfs-progs/tests/mnt
====== RUN CHECK stat btrfs-progs/tests/mnt/file
File: 'btrfs-progs/tests/mnt/file'
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: 2ch/44d Inode: 267 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: unconfined_u:object_r:unlabeled_t:s0
Access: 2021-08-24 22:10:21.000000000 +0800
Modify: 2021-08-24 22:10:21.000000000 +0800
Change: 2021-08-24 22:10:21.000000000 +0800
...
atime on converted inode does not match
test failed for case 019-ext4-copy-timestamps
Obviously, the log says that btrfs-convert does not support nanoseconds.
I looked at the source code and found that only if ext2_fs.h defines
EXT4_EPOCH_MASK btrfs-convert to support nanoseconds. But in e2fsprogs,
EXT4_EPOCH_MASK was introduced in v1.43, but in some older versions,
such as v1.40, e2fsprogs actually supports nanoseconds. It seems that if
struct ext2_inode_large contains the i_atime_extra member, ext4 is
supports nanoseconds, so I updated the logic to determine whether the
current ext4 file system supports nanosecond precision. In addition, I
imported some definitions to encode and decode tv_nsec (copied from
e2fsprogs source code).
Author: Li Zhang <zhanglikernel@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add --enable-experimental configure option that allows to merge unstable
features or partially implemented features. This is supposed to help
features that need time to settle, tweak output or formatting and would
require constant rebases and would have limited exposure to users that
could provide feedback.
If this is enabled, the following may change without notice:
- the whole feature may disappear in the future
- new command names could change or relocate to other subcommands
- parameter names
- output formatting
- json output
Signed-off-by: David Sterba <dsterba@suse.com>
The build now fails on older distros that have incomplete support for
zoned mode. Lack of blkzoned.h will automatically skip it. The
member blk_zone.capacity has been added in 5.9 and this would fail the
build, similarly BLKGETZONESZ.
Check each of them separately and fail the build unless --disable-zoned
is set. Build verified on Leap 15.2, Centos 7/8.
Signed-off-by: David Sterba <dsterba@suse.com>
If the kernel supports zoned block devices, the file
/usr/include/linux/blkzoned.h will be present. Check this and define
BTRFS_ZONED if the file is present.
If it present, enables ZONED feature, if not disable it.
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Autoheader uses the AC_DEFINE macros (and a few others) to populate
the config.h.in file. The autotools documentation does not tell
what happens if AC_DEFINE is used twice for the same identifier.
This patch prevents using AC_DEFINE twice for
HAVE_OWN_FIEMAP_EXTENT_DEFINE, preserving the logic (using the
fact that an undefined identifier in a preprocessor directive is
taken as zero).
Signed-off-by: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
Commit b3df561fbf ("btrfs-progs: convert: copy extra timespec on
ext4") has introduced the ability to convert extended inode time
precision on ext4, but this breaks builds on older distros, where ext4
does not have the nsec time precision.
Commit c615287cc0 ("btrfs-progs: a bunch of typo fixes") tried to fix
that by testing the availability of the EXT4_EPOCH_MASK macro, but the
test is not complete.
This patch aims at fixing the macro test, and changes the
name of the associated HAVE_ macro, since the logic is reverted.
This fixes#353 when ext4 has nsec time precision. Note that the test
convert/019-ext4-copy-timestamps fails when ext4 does not have the nsec
time precision and needs to check for the support.
Issue: #353
Signed-off-by: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
Hard-coding the pkg-config executable might result in build errors
on system and cross environments that have prefixed toolchains. The
PKG_CONFIG variable already holds the proper one and is already used
in a few other places.
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: Heiko Becker <heirecka@exherbo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
In 5.10 the convert gained support for extended inode time precision,
but this is not available on older distros and breaks build. Add a
configure-time check for the EXT4_EPOCH_MASK macro and add a stub in
case it's not detected.
This means that the 64bit timestamps will not be transferred from the
original filesystem in such environment, at least a warning is printed.
Issue: #344
Signed-off-by: David Sterba <dsterba@suse.com>
This reverts commit 61ecaff036.
The libmount functionality is not used anymore, we can remove it
entirely.
Signed-off-by: David Sterba <dsterba@suse.com>
API provided by libmount allows to read various information from /proc
files about mount paths.
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add .in substitutions for main package version, split and compound
without the 'v' prefix. BTRFS_VERSION_PATCHLEVEL could be empty.
Signed-off-by: David Sterba <dsterba@suse.com>
Add pkg-config file for libbtrfsutil which can be used to set up
compilation and link flags.
The paths in pc file depend on prefix variable but ignore DESTDIR.
DESTDIR is used for packaging and it should not affect the paths in pc
file.
Issue: #285
Signed-off-by: Sheng Mao <shngmao@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba
Julien Olivain
Sam James
Nathan Mills
Boris Burkov
Ross Burton
Nikolay Borisov
Li Zhang
Naohiro Aota
Pierre Labastie
Heiko Becker
Marcos Paulo de Souza
Sheng Mao