This is a bit larger than the previous syncs, because we use
extent_io_tree's everywhere. There's a lot of stuff added to
kerncompat.h, and then I went through and cleaned up all the API
changes, which were
- extent_io_tree_init takes an fs_info and an owner now.
- extent_io_tree_cleanup is now extent_io_tree_release.
- set_extent_dirty takes a gfpmask.
- clear_extent_dirty takes a cached_state.
- find_first_extent_bit takes a cached_state.
The diffstat looks insane for this, but keep in mind extent-io-tree.c
and extent-io-tree.h are ~2000 loc just by themselves.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We won't actually use the async code in progs, however we call the
helpers and such all over the normal code, so sync this into btrfs-progs
to make syncing other parts of the kernel easier.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This syncs accessors.[ch] from the kernel. For the most part
accessors.h will remain the same, there's just some helpers that need to
be adjusted for eb->data instead of eb->pages. Additionally accessors.c
needed to be completely updated to deal with this as well.
This is a set of files where we will likely only sync the header going
forward, and leave the C file in place as it needs to be specific to
btrfs-progs.
This forced a few "unrelated" changes
- Using btrfs_dir_item_ftype() instead of btrfs_dir_item_type(). This
is due to the encryption changes, and was simpler to just do in this
patch.
- Adjusting some of the print tree code to use the actual helpers and
not the btrfs-progs ones.
A local definition of static_assert is used to avoid compilation
failures on older gcc (< 9) where the 2nd parameter is mandatory.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These are the printk helpers from the kernel. There were a few
modifications, the hi-lights are
- We do not have fs_info::fs_state, so that needed to be removed.
- We do not have discard.h sync'ed yet, so that dependency was dropped.
- Anything related to struct super_block was commented out.
- The transaction abort had to be modified to fit with the current
btrfs-progs code.
- Added a btrfs_no_printk() helper to common/messages.* so that the
print statements still worked.
- The 32bit limit checkers are not needed so are behind __KERNEL__
Additionally there were kerncompat.h changes that needed to be made to
handle the dependencies properly. Those are easier to spot.
Any function that needed to be modified has a MODIFIED tag in the
comment section with a list of things that were changed.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We want to keep this file locally as we want to be uptodate with
upstream, so we can build btrfs-progs regardless of which kernel is
currently installed. Sync this with the upstream version and put it in
kernel-shared/uapi to maintain some semblance of where this file comes
from.
There are some changes that need to be synced back to kernel. A local
definition of static_assert is used to avoid compilation problems on gcc
(< 9) due to mandatory 2nd parameter.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
From the very beginning of free-space-tree feature, we allow mount
option "space_cache=v2" to convert the filesystem to the new feature.
But this is not the proper practice for new features (no matter if it's
incompat or compat_ro).
This is already making the clear_cache/space_cache mount option more
complex.
Thus this patch introduces the proper way to enable free-space-tree, and
I hope one day we can deprecate the "space_cache=" mount option.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The build fails with crypto backends other than builtin, the
initializers cannot be reached as they're ifdef-ed out. Move
hash_init_accel under the right condition and delete the
algorithm-specific initializers as they're used only by the hash test
and that can simply call hash_init_accel to set the implementation.
All the -m flags need to be detected at configure time and the flag used
for ifdef (HAVE_CFLAG_m*), not the actual feature defined by compiler as
the dispatcher function is not built with the -m flags.
The uname check for x86_64 must be dropped so on i386/i586 we can still
build accelerated version.
Signed-off-by: David Sterba <dsterba@suse.com>
The crc32c.c is going to be updated with CPU feature detection from
common/ which is not suitable for libbtrfs, so make own copy of the
file.
Signed-off-by: David Sterba <dsterba@suse.com>
Commit 0d5767e664 ("btrfs-progs: build: use -std=gnu11") was supposed
to set C level to gnu11 to match kernel but it was done only in the C=1
(checker) branch. The std used was still gnu90 that can be verified by
running with V=1.
Set the std option to gnu11 unconditionally and delete the m4 macros
that were detecting support on older gcc (4.5). The reference build on
lowest supported distro (Centos 7) compiles fine.
Signed-off-by: David Sterba <dsterba@suse.com>
The messages of ioctl-test related build is not aligned with other
commands:
[LD] hash-speedtest
[LD] hash-vectest
[CC] ioctl-test.o
[LD] ioctl-test
Signed-off-by: David Sterba <dsterba@suse.com>
Copy sha256-x86.c from https://github.com/noloader/SHA-Intrinsics, that
uses the compiler intrinsics to implement the update step with the
native x86_64 instructions.
To avoid dependencies of the reference code and the x86 version, check
runtime support only if the compiler also supports -msha.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy AVX2 implementation from https://github.com/sneves/blake2-avx2 .
Though this is marked experimental, libsodium uses this version.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy implementation from https://github.com/BLAKE2/BLAKE2, add runtime
detection of SSE2 and add the switch function.
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for run-time detection of CPU features on x86_64 to allow
selection of accelerated implementations of hash algorithms.
When possible use the compiler builtin (works on gcc and clang).
The SHA extensions can't be detected by __builtin_cpu_supports and the
__cpuid/__cpuidex macros are not consistently provided in all supported
gcc and clang versions. Copy the __cpuidex and call it manually for the
SHA extensions. Complete list https://en.wikipedia.org/wiki/CPUID .
Signed-off-by: David Sterba <dsterba@suse.com>
Move the source file to own directory so it can be further split and
refactored. File needs to be renamed to main.c so the build magic works.
Signed-off-by: David Sterba <dsterba@suse.com>
Concrete use-case:
- I want a static build to run in a small test VM.
- RedHat regrettably does not provide util-linux static libraries
- like libuuid.a and libblkid.a.
- So, one has to build them from source.
- Installing these to /usr is lame.
- This change lets me do make btrfs.static EXTRA_LDFLAGS=-L/path/to/util-linux-2.38/.libs
Pull-request: #563
Author: Alexey <snarkmaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a copy from v6.0.2 of kerncompat.h for libbtrfs to avoid accidental
build breakage as it's a public header and until now also shared with
the rest of btrfs-progs code.
Signed-off-by: David Sterba <dsterba@suse.com>
Add initial reflink group with example command 'clone' to test the
interface. Work in progress, experimental build needed.
Issue: #396
Signed-off-by: David Sterba <dsterba@suse.com>
We're going to sync btrfs.h into btrfs-progs from the kernel, however
libbtrfs still needs ioctl.h. To deal with this copy ioctl.h into
libbtrfs, and update that code to use the local copy and update the
libbtrfs headers list to use this copy.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The kernel switched to this recently, switch btrfs-progs to this as well
to avoid issues with syncing the kernel code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We were not clearing the .o files for btrfs-convert as we had the wrong
directory, which meant I missed a compile error that happened when I was
messing with kernel-shared. Fix this by making sure we clear the .o
files for convert properly.
Fixes: 753baf2443 ("btrfs-progs: build: redirect dependency files files to .deps")
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In converting some of our helpers to take new args I would miss some
locations because we don't stop on any warning, and I would miss the
warning in the scrollback.
Note: the Centos7 build is not yet warning-free so we can't enable
-Werror by default. It can be used as:
make EXTRA_CFLAGS=-Werror
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
[ add note ]
Signed-off-by: David Sterba <dsterba@suse.com>
The dependency files clutter the source directories. Generate them in
the .deps directory, similar what automake uses.
Signed-off-by: David Sterba <dsterba@suse.com>
The radix-tree is not used in userspace code. In kernel it's for
tracking unpersisted and in-memory structures and has been replaced by
the xarray.
Signed-off-by: David Sterba <dsterba@suse.com>
All files include the <btrfsutil.h> which could be confused with the
system-wide installation. Drop the -I path from build and use full path
for any libbtrfsutil headers.
Signed-off-by: David Sterba <dsterba@suse.com>
The header dependency rules generated as .o.d files are sometimes stale
and fail the build. Add a rule to clean them if needed, otherwise
they're also cleaned by 'make clean'.
Signed-off-by: David Sterba <dsterba@suse.com>
Now that LZO and ZSTD are optional for not just restore, rename the
build variables to a more generic name and update configure summary.
Signed-off-by: David Sterba <dsterba@suse.com>
There are build-time options for LZO and ZSTD support, the stream v2+
supports compression. The help text lists what has been compiled in,
similar to what 'restore' does, with a similar limitation that a stream
with compressed data cannot be processed if any of the extents is
compressed.
Signed-off-by: David Sterba <dsterba@suse.com>
Let libbtrfs use own copy of the exported header files to avoid
potential breakage when syncing with kernel headers and also to remove
declarations that are not used by userspace. The send.h is frozen to
support protocol v1.
Signed-off-by: David Sterba <dsterba@suse.com>
LZO as a compression format is pretty archaic these days, there are
better algorithms in all metrics for compression and decompression, and
lzo hasn't had a new release since 2017.
Add an option to disable LZO (defaulting to enabled), and respect it in
cmds/restore.c.
NOTE: disabling support for LZO will make make it impossible to restore
data from filesystems where the compression has ever been used. It's not
recommended to build without the support in general.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In order to use rb_root_cached we need to sync with kernel sources. Copy
the file from linux.git/include/linux/rbtree_types.h and update so it's
C++ protected for inclusion to libbtrfs and remove duplicate
definitions.
Signed-off-by: David Sterba <dsterba@suse.com>
The fsstress tool is a useful file generator, pull it from fstests as
it's not packaged as a standalone tool anywhere and the LTP version is
out of date.
The file has been modified to build, some xfs-specific ioctls are not
supported.
Signed-off-by: David Sterba <dsterba@suse.com>
As zoned btrfs uses regular writes for metadata, it needs zone write
locking in the IO scheduler. Add a udev rule that configures an IO
scheduler doing zone write locking.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
