Commit Graph

6739 Commits

Author SHA1 Message Date
Boris Burkov
55611db353 btrfs-progs: print-tree: add support for squota structures
Add support to btrfs inspect-internal dump-super and dump-tree for the
new structures and feature flags introduced by squota.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:54 +02:00
Boris Burkov
2c729bbe91 btrfs-progs: add squota kernel definitions
Copy over structs, accessors, and constants for simple quotas

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:54 +02:00
Boris Burkov
c59fc1d89c btrfs-progs: docs: document squota
Document the new options in btrfs quota and mkfs.btrfs. Also, add a
section to the long form qgroups document about squota.

Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:54 +02:00
Anand Jain
ff4c4a3a00 btrfs-progs: allow duplicate fsid for single device filesystems
For single device btrfs filesystem, allow duplicate fsid to be created.
This should be used with caution as more devices with the same uuid
could be confused with each other.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Qu Wenruo
93e8b661f6 btrfs-progs: fix a variable shadowing when enabling experimental features
There is another variable shadowing problem which can only be exposed
if experimental features are enabled.

Inside the branch of BTRFS_PRINT_TREE_CSUM_HEADERS, we declare another
local variable @csum, shadowing the @csum of print_header_info(), which
is only declared when experimental features are enabled.

Just rename the @csum to @tree_csum to avoid the problem.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Qu Wenruo
c94965198a btrfs-progs: replace errno with %m in cmd_inspect_list_chunks()
The variable @e is only utilized to record the errno from ioctl() call,
and is only for the error message.

We can go with "%m" to replace the usage of variable @e, and remove the
variable shadowing, as later we will declare a local variable @e with a
different type.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba
553273a834 btrfs-progs: dump-tree: fix block group tree short name
The option -t recognizes tree names in various names and the stem must
not contain the "_TREE", which the bgt had.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
a9fd50f85f btrfs-progs: read stripe tree when mapping blocks
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
fff57d3774 btrfs-progs: load zone info for all zoned devices
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
b4ab282686 btrfs-progs: allow zoned RAID
Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree
is used.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
81d6ecd5c2 btrfs-progs: add dump tree support for the raid stripe tree
Add support for the RAID stripe tree to btrfs inspect-internal dump-tree.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
b6490733a8 btrfs-progs: read fs with stripe tree from disk
When encountering a filesystem formatted with the raid stripe tree
feature, read it from disk.

Also add the incompat declaration to the tree printer.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
Johannes Thumshirn
2eb351210f btrfs-progs: add raid stripe tree definitions
Add the definitions for the on-disk format of the raid stripe tree.

Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba
8d3193da89 btrfs-progs: increase size of BTRFS_FEATURE_STRING_BUF_SIZE
We're getting more features and the string size limit will not be
sufficient, so extend enough that we won't have to care for some time.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:41:08 +02:00
David Sterba
7db445a3de btrfs-progs: crypto: unify naming of crc32c implementations
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:05:38 +02:00
David Sterba
78c227e7d2 btrfs-progs: ci: run CI image tests in parallel
We can speed up the CI tests by running them as separate build jobs,
they only need to checkout the sources.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-02 18:05:38 +02:00
David Sterba
92d04d4780
Btrfs progs v6.5.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 19:47:53 +02:00
David Sterba
2c574064b5 btrfs-progs: update CHANGES for 6.5.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 19:47:10 +02:00
David Sterba
d0bf814100 btrfs-progs: ci: build static binaries only for latest release
The release-test branch is used for pre-release checks so don't pollute
the workflows list with unverified builds. The static build checks will
be done within the other workflows but no binaries need to be published.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 19:43:26 +02:00
David Sterba
03f41ac508 btrfs-progs: detect PCLMUL CPU support for accelerated crc32c
The accelerated crc32c needs to check for two CPU features, the crc32c
instructions is in SSE 4.2 and 'pclmulqdq' is a separate. There's still
old hardware used that does not have the PCLMUL instructions. Detect it
and make it the condition.

The pclmul is not supported on old compilers so also add a
configure-time detection and leave the SSE 4.2 only implementation as
the accelerated one if possible.

Issue: #676
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-13 00:38:50 +02:00
David Sterba
b40943dea4 btrfs-progs: docs: updates
- group features on status page
- update developer docs
- add cross references

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba
2726a83952 btrfs-progs: tests: scan test results
Some errors may be reported in the logs only, scan the file each time
there are fresh results. The scanning script will return error that is
supposed to be caught by the testsuite environment.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba
7a7df06f98 btrfs-progs: tests: extend scan-results.sh to scan only a given file
Scanning all results is meant for the whole testsuite, we'd like to make
it more fine grained to verify them once a test is run via the test
running wrappers.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba
e0263f2b31 btrfs-progs: tests: fix test case name detection scan-results.sh
The test markers have changed in 4.17 so the result scanner does not
print the test names (but still detects the errors).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
David Sterba
634e9a4792 btrfs-progs: tests: fix build -Wmissing-prototypes warnings
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
Mark Harmstone
d8317a445c btrfs-progs: check: add check for metadata level mismatch
For a skinny metadata item in the extent tree, the key offset represents
the level of the tree it points to. This adds a check that these values
match, as otherwise it can cause a volume to go readonly when deleting a
large number of inodes.

See https://github.com/maharmstone/ntfs2btrfs/issues/51

Pull-request: #623
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Mark Harmstone <mark@harmstone.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
Ariadne Conill
b390c276c2 btrfs-progs: crypto: fix readonly relocation of the jumptable
Without this, the btrfs programs fail to link when build with -Wl,-z,relro,
due to the jumptable containing relocations.

Pull-request: #675
Signed-off-by: Ariadne Conill <ariadne@dereferenced.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 22:08:02 +02:00
David Sterba
f7ecc34555
Btrfs progs v6.5
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 14:41:37 +02:00
David Sterba
226f3a97e7 btrfs-progs: update CHANGES for 6.5
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba
f9b0da8e78 Revert "libbtrfs: remove the support for fs without uuid tree"
This reverts commit 83ab92512e.

The commit removed some old code that seemed to be unused but this
actually broke snapper. Revert the changes completely to the v6.3 ABI
level.

Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1212217
Issue: #672
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
Qu Wenruo
7a0da24f52 btrfs-progs: tune: do not allow multiple incompatible features to be set
[PROBLEM]
Btrfstune allows multiple different options to be executed in one go,
some options are completely fine, like no-holes along with extref, but
with more and more options, we need more exclusive checks.

In fact a lot of new options are already not following the old
success/total checks.

[ENHANCEMENT]
There is really no need to allow multiple features to be set in one go.

So this patch introduces an array which groups all the compatible
options into following categories:

- Extent tree
  This includes converting to/from extent and block group tree.

- Space cache
  This includes converting to v2 space cache.

- Metadata UUID
  This includes changing metadata uuid.

- FSID change
  This includes the slower full fs fsid rewrites.

- Csum change
  This includes the csum rewrites.

- Seed devices
  This includes changing the device seed flag.

- Legacy options
  This includes no-holes/extref/skinny-metadata features, which are
  already default mkfs features.

Now we only allow options inside the same group to be specified.
E.g. "btrfstune -r -S 1" would fail as it includes both legacy and seed
groups.
Meanwhile "btrfstune -r -n" would still be allowed.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
Anand Jain
e8e5537d8c btrfs-progs: tests: add prefixes to error messages in misc-test/034
Add appropriate prefix to the error messages to make it easier to track
down which case failed.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba
bdfb83e7d3 btrfs-progs: README: pull request workflow, minor fixes
Mention the basic pull request workflow, the rest are style fixes.
[skip ci]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba
008ca33f3e btrfs-progs: docs: add 6.5 kernel development statistics
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:41 +02:00
David Sterba
4c24d928d4 btrfs-progs: ci: build static binaries on oldest supported x86-64 architecture
As mentioned on https://wiki.tnonline.net/w/Btrfs/Statically_built_btrfs-progs
the static binaries could be built on with flags so they're available on
most x86-64 models (intel, AMD).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:59 +02:00
David Sterba
4167c86123 btrfs-progs: remove btrfs-fragments
The btrfs-fragments utility generates a html page with a set of images
representing block groups and visualises fragmentation. Though this
might be useful, the build has been broken since commit 877f512c55
("btrfs-progs: sync block group item accessors from kernel"), version
5.7. Also it's been out of date regarding newly added trees or data
structure updates and does not work correctly anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:59 +02:00
David Sterba
b2c4a9ea75 btrfs-progs: ci: add awk to Tumbleweed image
The CI image build workflow started to fail due to missing 'awk',
install it.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 12:11:46 +02:00
David Sterba
f58694e87a btrfs-progs: docs: update Interoperability
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
David Sterba
65306da75c btrfs-progs: ci: add workflow for pull-request
Reduced version of devel checks, unrestricted.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
09351e3831 btrfs-progs: rename fs_devices::latest_trans to match the kernel
Aligning progs's struct btrfs_fs_devices with the kernel rename
btrfs_fs_devices::latest_trans to btrfs_fs_devices::latest_generation.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
a11d468e98 btrfs-progs: rename fs_devices::list to match the kernel
Aligning with the kernel's struct btrfs_fs_devices:fs_list, rename
btrfs_fs_devices::list to btrfs_fs_devices::fs_list.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
66c4c9632f btrfs-progs: prepare the latest device's superblock for commit
Add a flag to copy the superblock of the latest device to the
fs_info::super_copy for the commit process, rather than using the
superblock from the device specified in the argument.

This serves as groundwork to enable recovery from an incomplete
btrfstune -M|m|u|U operation.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
9f6e0ab2a4 btrfs-progs: tune: fix return without flag reset commit
In the function set_metadata_uuid(), we set the flag
BTRFS_SUPER_FLAG_CHANGING_FSID_V2 in step 1 at line 71 as shown below:

   71         super_flags |= BTRFS_SUPER_FLAG_CHANGING_FSID_V2;
   72         btrfs_set_super_flags(disk_super, super_flags);
   73         ret = btrfs_commit_transaction(trans, root);

However, we fail to reset this flag if there is no change in the fsid on
the incoming disks, as we return too early.

  105       } else {
  106               /* Setting the same fsid as current, do nothing */
  107               return 0;

Fix this by allowing the thread to pass through the step 2, where we
reset the flag.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
fd3c4c4df1 btrfs-progs: tune: pass metadata_uuid in check_unfinished_fsid_change
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
rename %unused2 to %metadata_uuid as the function
check_unfinished_fsid_change() write the metadata_uuid from the ctree to
it.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
aff1f16deb btrfs-progs: tune: pass fsid in check_unfinished_fsid_change arg2
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
delete unused1 argument instead reuse %fsid as the function
check_unfinished_fsid_change() returns the fsid.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
70bedfa68a btrfs-progs: tune: rename uuid_changed to fsid_changed in set_metadata_uuid
We never change the metadata_uuid; we only change the fsid.  So
'%fsid_changed' flows more appropriately than '%uuid_changed'.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
c6ecbe54bf btrfs-progs: tune: rename new_uuid to new_fsid in set_metadata_uuid
%new_uuid is being used to say there is a new fsid. So why not just call
it %new_fsid.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
21440a64f7 btrfs-progs: tune: rename new_fsid to fsid in set_metadata_uuid
The %new_fsid is not only new it can be the fsid from the passed disk
so just rename it to %fsid. Also, in the next patch the %new_fsid will
be a bool variable to indicate if the %fsid is new from the fsid in the
disk.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
9e9ac97902 btrfs-progs: tune: rename arg to new_fsid_str in set_metadata_uuid
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
%uuid_string arg is actually carries new fsid to be used. So just name
it to %new_fsid_str.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
3aa3342959 btrfs-progs: tune: cache local variable of fs_info
Since the root pointer dereferences for the fs_info several times,
it is rational to save the fs_info.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:56 +02:00