Currently, to fix device following the write failure of one or more devices
during btrfstune -m|M, we rely on the kernel's ability to reassemble devices,
even when they possess distinct fsids.
Kernel hinges combinations of metadata_uuid and generation number, with
additional cues taken from the fsid and the BTRFS_SUPER_FLAG_CHANGING_FSID_V2
flag. This patch adds this logic to btrfs-progs.
In complex scenarios (such as multiple fsids with the same metadata_uuid and
matching generation), user intervention becomes necessary to resolve the
situations which btrfs-progs can do better.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Most of the code and functions in this patch is copied from the kernel.
Now, with this patch applied, there is no need to mount the device to
complete the incomplete 'btrfstune -m|M' command (CHANING_FSID_V2 flag).
Instead, the same command could be run, which will successfully complete
the operation.
Currently, the 'tests/misc-tests/034-metadata-uuid' tests the kernel using
four sets of disk images with CHANING_FSID_V2. Now, this test case has been
updated (as in the next patch) to test the progs part.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently, btrfstune relies on the superblock of the device specified
in the btrfstune argument for fs_info::super_copy. However, it should
use fs_devices::latest_bdev, as it points to the device with the highest
fs_devices::generation number. This will contain the superblock updates
that other devices may have missed and we can now support reuniting
devices following failures of btrfstune -m|M|u|U as in the patches:
btrfs-progs: add support to fix superblock with CHANGING_FSID_V2 flag
btrfs-progs: recover from the failed btrfstune -m|M
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the new OWNER_REF inline items to the tree-checker extent item
checking code. We could somehow validate the root id for being a valid
fstree id, but just skipping it seems fine as well.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new option --simple to 'btrfs quota enable'. If set, this enables
simple quotas instead of full qgroups by using the new ioctl command
value.
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the ability to enable simple quotas on an existing file system at
rest with btrfstune.
This is similar to the functionality in mkfs, except it must also find
all the roots for which it must create qgroups. Note that this *does
not* retroactively compute usage for existing extents as that is
impossible for data. This is consistent with the behavior of the live
enable ioctl.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the ability to enable simple quotas from mkfs with '-O squota'
There is some complication around handling enable_gen while still
counting the root node of an fs. To handle this, employ a hack of doing
a no-op write on the root node to bump its generation up above that of
the qgroup enable generation, which results in counting it properly.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Add simple quotas checks to btrfs check.
Like the kernel feature, these checks bypass most of the backref walking
in the qgroups check. Instead, they enforce the invariant behind the
design of simple quotas by scanning the extent tree and determining the
owner of each extent:
Data: reading the owner ref inline item
Metadata: reading the tree block and reading its btrfs_header's owner
This gives us the expected count from squotas which we check against the
on-disk state of the qgroup items.
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Add support to btrfs inspect-internal dump-super and dump-tree for the
new structures and feature flags introduced by squota.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Copy over structs, accessors, and constants for simple quotas
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Document the new options in btrfs quota and mkfs.btrfs. Also, add a
section to the long form qgroups document about squota.
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
For single device btrfs filesystem, allow duplicate fsid to be created.
This should be used with caution as more devices with the same uuid
could be confused with each other.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is another variable shadowing problem which can only be exposed
if experimental features are enabled.
Inside the branch of BTRFS_PRINT_TREE_CSUM_HEADERS, we declare another
local variable @csum, shadowing the @csum of print_header_info(), which
is only declared when experimental features are enabled.
Just rename the @csum to @tree_csum to avoid the problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The variable @e is only utilized to record the errno from ioctl() call,
and is only for the error message.
We can go with "%m" to replace the usage of variable @e, and remove the
variable shadowing, as later we will declare a local variable @e with a
different type.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The option -t recognizes tree names in various names and the stem must
not contain the "_TREE", which the bgt had.
Signed-off-by: David Sterba <dsterba@suse.com>
Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree
is used.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for the RAID stripe tree to btrfs inspect-internal dump-tree.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When encountering a filesystem formatted with the raid stripe tree
feature, read it from disk.
Also add the incompat declaration to the tree printer.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the definitions for the on-disk format of the raid stripe tree.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We're getting more features and the string size limit will not be
sufficient, so extend enough that we won't have to care for some time.
Signed-off-by: David Sterba <dsterba@suse.com>
We can speed up the CI tests by running them as separate build jobs,
they only need to checkout the sources.
Signed-off-by: David Sterba <dsterba@suse.com>
The release-test branch is used for pre-release checks so don't pollute
the workflows list with unverified builds. The static build checks will
be done within the other workflows but no binaries need to be published.
Signed-off-by: David Sterba <dsterba@suse.com>
The accelerated crc32c needs to check for two CPU features, the crc32c
instructions is in SSE 4.2 and 'pclmulqdq' is a separate. There's still
old hardware used that does not have the PCLMUL instructions. Detect it
and make it the condition.
The pclmul is not supported on old compilers so also add a
configure-time detection and leave the SSE 4.2 only implementation as
the accelerated one if possible.
Issue: #676
Signed-off-by: David Sterba <dsterba@suse.com>
Some errors may be reported in the logs only, scan the file each time
there are fresh results. The scanning script will return error that is
supposed to be caught by the testsuite environment.
Signed-off-by: David Sterba <dsterba@suse.com>
Scanning all results is meant for the whole testsuite, we'd like to make
it more fine grained to verify them once a test is run via the test
running wrappers.
Signed-off-by: David Sterba <dsterba@suse.com>
The test markers have changed in 4.17 so the result scanner does not
print the test names (but still detects the errors).
Signed-off-by: David Sterba <dsterba@suse.com>
For a skinny metadata item in the extent tree, the key offset represents
the level of the tree it points to. This adds a check that these values
match, as otherwise it can cause a volume to go readonly when deleting a
large number of inodes.
See https://github.com/maharmstone/ntfs2btrfs/issues/51
Pull-request: #623
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Mark Harmstone <mark@harmstone.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Without this, the btrfs programs fail to link when build with -Wl,-z,relro,
due to the jumptable containing relocations.
Pull-request: #675
Signed-off-by: Ariadne Conill <ariadne@dereferenced.org>
Signed-off-by: David Sterba <dsterba@suse.com>
This reverts commit 83ab92512e.
The commit removed some old code that seemed to be unused but this
actually broke snapper. Revert the changes completely to the v6.3 ABI
level.
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1212217
Issue: #672
Signed-off-by: David Sterba <dsterba@suse.com>
[PROBLEM]
Btrfstune allows multiple different options to be executed in one go,
some options are completely fine, like no-holes along with extref, but
with more and more options, we need more exclusive checks.
In fact a lot of new options are already not following the old
success/total checks.
[ENHANCEMENT]
There is really no need to allow multiple features to be set in one go.
So this patch introduces an array which groups all the compatible
options into following categories:
- Extent tree
This includes converting to/from extent and block group tree.
- Space cache
This includes converting to v2 space cache.
- Metadata UUID
This includes changing metadata uuid.
- FSID change
This includes the slower full fs fsid rewrites.
- Csum change
This includes the csum rewrites.
- Seed devices
This includes changing the device seed flag.
- Legacy options
This includes no-holes/extref/skinny-metadata features, which are
already default mkfs features.
Now we only allow options inside the same group to be specified.
E.g. "btrfstune -r -S 1" would fail as it includes both legacy and seed
groups.
Meanwhile "btrfstune -r -n" would still be allowed.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add appropriate prefix to the error messages to make it easier to track
down which case failed.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The btrfs-fragments utility generates a html page with a set of images
representing block groups and visualises fragmentation. Though this
might be useful, the build has been broken since commit 877f512c55
("btrfs-progs: sync block group item accessors from kernel"), version
5.7. Also it's been out of date regarding newly added trees or data
structure updates and does not work correctly anymore.
Signed-off-by: David Sterba <dsterba@suse.com>
Aligning progs's struct btrfs_fs_devices with the kernel rename
btrfs_fs_devices::latest_trans to btrfs_fs_devices::latest_generation.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Aligning with the kernel's struct btrfs_fs_devices:fs_list, rename
btrfs_fs_devices::list to btrfs_fs_devices::fs_list.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a flag to copy the superblock of the latest device to the
fs_info::super_copy for the commit process, rather than using the
superblock from the device specified in the argument.
This serves as groundwork to enable recovery from an incomplete
btrfstune -M|m|u|U operation.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Anand Jain
Boris Burkov
Qu Wenruo
David Sterba
Johannes Thumshirn
Mark Harmstone
Ariadne Conill