[BUG]
We got some test failures related to btrfs-convert with subpage, e.g.
btrfs/012, the failure would cause the following dmesg:
BTRFS warning (device nvme0n1p7): v1 space cache is not supported for page size 16384 with sectorsize 4096
BTRFS error (device nvme0n1p7): open_ctree failed
[CAUSE]
v1 space cache has tons of hard coded PAGE_SIZE usage, and considering
v2 space cache is going to replace it (which is already the new default
since v5.15 btrfs-progs), thus for btrfs subpage support, we just simply
reject the v1 space cache, and utilize v2 space cache when possible.
But there is special catch in btrfs-convert, although we're specifying
v2 space cache as the new default for btrfs-convert, it doesn't really
follow the specification at all.
Thus the converted filesystem will still go v1 space cache.
[FIX]
It can be a huge change to btrfs-convert to make the initial btrfs image
to support v2 cache.
Thus this patch would change the fs at the final stage, just before we
finalize the btrfs.
This patch would drop all the v1 cache created, then call
btrfs_create_free_space_tree() to populate the free space tree and
commit the superblock with needed compat_ro flags.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Move files from m4/ to config that is also used for build and we can
reduce the number of toplevel directories.
Signed-off-by: David Sterba <dsterba@suse.com>
More complete test coverage:
- json an string table formatters
- fuzz tests (no mount)
- libbtrfs build test
- libbtrfsutil python test
- ioctl build test
- hash tests
Signed-off-by: David Sterba <dsterba@suse.com>
With all the known warnings fixed, we can enable -Wmissing-prototypes
and prevent such warnings from happening.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This syncs tree-checker.c from the kernel. The main modification was to
add a open ctree flag to skip the deeper leaf checks, and plumbing this
through tree-checker.c. We need this for things like fsck or
btrfs-image that need to work with slightly corrupted file systems, and
these checks simply make us unable to look at the corrupted blocks.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We want locking.h to have all the definitions that get used throughout
the codebase, however we don't want to actually use any of the actual
locking. This sync's the bulk of locking.h, and then stubs out all of
the definitions. We need a locking.c for the root lock helpers that
return the extent buffer, but everything else can simply be inlined out.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is a bit larger than the previous syncs, because we use
extent_io_tree's everywhere. There's a lot of stuff added to
kerncompat.h, and then I went through and cleaned up all the API
changes, which were
- extent_io_tree_init takes an fs_info and an owner now.
- extent_io_tree_cleanup is now extent_io_tree_release.
- set_extent_dirty takes a gfpmask.
- clear_extent_dirty takes a cached_state.
- find_first_extent_bit takes a cached_state.
The diffstat looks insane for this, but keep in mind extent-io-tree.c
and extent-io-tree.h are ~2000 loc just by themselves.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We won't actually use the async code in progs, however we call the
helpers and such all over the normal code, so sync this into btrfs-progs
to make syncing other parts of the kernel easier.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This syncs accessors.[ch] from the kernel. For the most part
accessors.h will remain the same, there's just some helpers that need to
be adjusted for eb->data instead of eb->pages. Additionally accessors.c
needed to be completely updated to deal with this as well.
This is a set of files where we will likely only sync the header going
forward, and leave the C file in place as it needs to be specific to
btrfs-progs.
This forced a few "unrelated" changes
- Using btrfs_dir_item_ftype() instead of btrfs_dir_item_type(). This
is due to the encryption changes, and was simpler to just do in this
patch.
- Adjusting some of the print tree code to use the actual helpers and
not the btrfs-progs ones.
A local definition of static_assert is used to avoid compilation
failures on older gcc (< 9) where the 2nd parameter is mandatory.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These are the printk helpers from the kernel. There were a few
modifications, the hi-lights are
- We do not have fs_info::fs_state, so that needed to be removed.
- We do not have discard.h sync'ed yet, so that dependency was dropped.
- Anything related to struct super_block was commented out.
- The transaction abort had to be modified to fit with the current
btrfs-progs code.
- Added a btrfs_no_printk() helper to common/messages.* so that the
print statements still worked.
- The 32bit limit checkers are not needed so are behind __KERNEL__
Additionally there were kerncompat.h changes that needed to be made to
handle the dependencies properly. Those are easier to spot.
Any function that needed to be modified has a MODIFIED tag in the
comment section with a list of things that were changed.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We want to keep this file locally as we want to be uptodate with
upstream, so we can build btrfs-progs regardless of which kernel is
currently installed. Sync this with the upstream version and put it in
kernel-shared/uapi to maintain some semblance of where this file comes
from.
There are some changes that need to be synced back to kernel. A local
definition of static_assert is used to avoid compilation problems on gcc
(< 9) due to mandatory 2nd parameter.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
From the very beginning of free-space-tree feature, we allow mount
option "space_cache=v2" to convert the filesystem to the new feature.
But this is not the proper practice for new features (no matter if it's
incompat or compat_ro).
This is already making the clear_cache/space_cache mount option more
complex.
Thus this patch introduces the proper way to enable free-space-tree, and
I hope one day we can deprecate the "space_cache=" mount option.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The build fails with crypto backends other than builtin, the
initializers cannot be reached as they're ifdef-ed out. Move
hash_init_accel under the right condition and delete the
algorithm-specific initializers as they're used only by the hash test
and that can simply call hash_init_accel to set the implementation.
All the -m flags need to be detected at configure time and the flag used
for ifdef (HAVE_CFLAG_m*), not the actual feature defined by compiler as
the dispatcher function is not built with the -m flags.
The uname check for x86_64 must be dropped so on i386/i586 we can still
build accelerated version.
Signed-off-by: David Sterba <dsterba@suse.com>
The crc32c.c is going to be updated with CPU feature detection from
common/ which is not suitable for libbtrfs, so make own copy of the
file.
Signed-off-by: David Sterba <dsterba@suse.com>
Commit 0d5767e664 ("btrfs-progs: build: use -std=gnu11") was supposed
to set C level to gnu11 to match kernel but it was done only in the C=1
(checker) branch. The std used was still gnu90 that can be verified by
running with V=1.
Set the std option to gnu11 unconditionally and delete the m4 macros
that were detecting support on older gcc (4.5). The reference build on
lowest supported distro (Centos 7) compiles fine.
Signed-off-by: David Sterba <dsterba@suse.com>
The messages of ioctl-test related build is not aligned with other
commands:
[LD] hash-speedtest
[LD] hash-vectest
[CC] ioctl-test.o
[LD] ioctl-test
Signed-off-by: David Sterba <dsterba@suse.com>
Copy sha256-x86.c from https://github.com/noloader/SHA-Intrinsics, that
uses the compiler intrinsics to implement the update step with the
native x86_64 instructions.
To avoid dependencies of the reference code and the x86 version, check
runtime support only if the compiler also supports -msha.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy AVX2 implementation from https://github.com/sneves/blake2-avx2 .
Though this is marked experimental, libsodium uses this version.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy implementation from https://github.com/BLAKE2/BLAKE2, add runtime
detection of SSE2 and add the switch function.
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for run-time detection of CPU features on x86_64 to allow
selection of accelerated implementations of hash algorithms.
When possible use the compiler builtin (works on gcc and clang).
The SHA extensions can't be detected by __builtin_cpu_supports and the
__cpuid/__cpuidex macros are not consistently provided in all supported
gcc and clang versions. Copy the __cpuidex and call it manually for the
SHA extensions. Complete list https://en.wikipedia.org/wiki/CPUID .
Signed-off-by: David Sterba <dsterba@suse.com>
Move the source file to own directory so it can be further split and
refactored. File needs to be renamed to main.c so the build magic works.
Signed-off-by: David Sterba <dsterba@suse.com>
Concrete use-case:
- I want a static build to run in a small test VM.
- RedHat regrettably does not provide util-linux static libraries
- like libuuid.a and libblkid.a.
- So, one has to build them from source.
- Installing these to /usr is lame.
- This change lets me do make btrfs.static EXTRA_LDFLAGS=-L/path/to/util-linux-2.38/.libs
Pull-request: #563
Author: Alexey <snarkmaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a copy from v6.0.2 of kerncompat.h for libbtrfs to avoid accidental
build breakage as it's a public header and until now also shared with
the rest of btrfs-progs code.
Signed-off-by: David Sterba <dsterba@suse.com>
Add initial reflink group with example command 'clone' to test the
interface. Work in progress, experimental build needed.
Issue: #396
Signed-off-by: David Sterba <dsterba@suse.com>
We're going to sync btrfs.h into btrfs-progs from the kernel, however
libbtrfs still needs ioctl.h. To deal with this copy ioctl.h into
libbtrfs, and update that code to use the local copy and update the
libbtrfs headers list to use this copy.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The kernel switched to this recently, switch btrfs-progs to this as well
to avoid issues with syncing the kernel code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We were not clearing the .o files for btrfs-convert as we had the wrong
directory, which meant I missed a compile error that happened when I was
messing with kernel-shared. Fix this by making sure we clear the .o
files for convert properly.
Fixes: 753baf2443 ("btrfs-progs: build: redirect dependency files files to .deps")
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In converting some of our helpers to take new args I would miss some
locations because we don't stop on any warning, and I would miss the
warning in the scrollback.
Note: the Centos7 build is not yet warning-free so we can't enable
-Werror by default. It can be used as:
make EXTRA_CFLAGS=-Werror
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
[ add note ]
Signed-off-by: David Sterba <dsterba@suse.com>
The dependency files clutter the source directories. Generate them in
the .deps directory, similar what automake uses.
Signed-off-by: David Sterba <dsterba@suse.com>
The radix-tree is not used in userspace code. In kernel it's for
tracking unpersisted and in-memory structures and has been replaced by
the xarray.
Signed-off-by: David Sterba <dsterba@suse.com>
All files include the <btrfsutil.h> which could be confused with the
system-wide installation. Drop the -I path from build and use full path
for any libbtrfsutil headers.
Signed-off-by: David Sterba <dsterba@suse.com>
The header dependency rules generated as .o.d files are sometimes stale
and fail the build. Add a rule to clean them if needed, otherwise
they're also cleaned by 'make clean'.
Signed-off-by: David Sterba <dsterba@suse.com>
Now that LZO and ZSTD are optional for not just restore, rename the
build variables to a more generic name and update configure summary.
Signed-off-by: David Sterba <dsterba@suse.com>
There are build-time options for LZO and ZSTD support, the stream v2+
supports compression. The help text lists what has been compiled in,
similar to what 'restore' does, with a similar limitation that a stream
with compressed data cannot be processed if any of the extents is
compressed.
Signed-off-by: David Sterba <dsterba@suse.com>
