Commit Graph

231 Commits

Author SHA1 Message Date
David Sterba
a7fa81f296 btrfs-progs: open code print_usage where applicable
After previous change to usage() that now has the return code, there's
no purpose of the print_usage() wrapper so it can be removed.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
Qu Wenruo
f61b90aff9 btrfs-progs: make usage call properly return an exit value
[BUG]
Currently cli/009 test case failed with different exit number:

  ====== RUN CHECK /home/adam/btrfs-progs/btrfstune --help
  usage: btrfstune [options] device
  [...]
  failed: /home/adam/btrfs-progs/btrfstune --help
  test failed for case 009-btrfstune

[CAUSE]
In tune/main.c, we have the following call on usage():

  static void print_usage(int ret)
  {
	usage(&tune_cmd);
	exit(ret);
  }

However usage() itself would always call exit(1):

  void usage(const struct cmd_struct *cmd)
  {
	usage_command_usagestr(cmd->usagestr, NULL, 0, true, true);
	exit(1);
  }

This makes prevents any caller of usage() to modify its exit number.

[FIX]
Add a new argument @error for print_usage(), so we can properly return 0
for -h/--help usage.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
24ec095295 btrfs-progs: crypto: add common function for accelerated initialization
Prepare a single location that will detect or set accelerated versions
of hash algorithms. Right now it's the crc32c, blake2 and sha256 do
an if-else switch while crc32c sets a function pointer.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:31 +01:00
Qu Wenruo
f914949b1a btrfs-progs: fix set but not used variables
[WARNING]
Clang 15.0.7 warns about several unused variables:

  kernel-shared/zoned.c:829:6: warning: variable 'num_sequential' set but not used [-Wunused-but-set-variable]
          u32 num_sequential = 0, num_conventional = 0;
              ^
  cmds/scrub.c:1174:6: warning: variable 'n_skip' set but not used [-Wunused-but-set-variable]
          int n_skip = 0;
              ^
  mkfs/main.c:493:6: warning: variable 'total_block_count' set but not used [-Wunused-but-set-variable]
          u64 total_block_count = 0;
              ^
  image/main.c:2246:6: warning: variable 'bytenr' set but not used [-Wunused-but-set-variable]
          u64 bytenr = 0;
              ^

[CAUSE]
Most of them are just straightforward set but not used variables.

The only exception is total_block_count, which has commented out code
relying on it.

[FIX]
Just remove those variables, and for @total_block_count, also remove the
comments.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-18 17:44:03 +01:00
David Sterba
347c8209e8 btrfs-progs: mkfs: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-25 19:55:47 +01:00
David Sterba
dfd58c294b btrfs-progs: mkfs: use help and cmd_struct for printing help text
Unify the mkfs help text so it uses the help framework. The cmd struct
is set up only partially.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-25 19:55:47 +01:00
Naohiro Aota
d8c6021727 btrfs-progs: mkfs: check blkid version on zoned filesystems
Prior to version 2.38, libblkid fails to detect zoned mode's superblock
location resulting in blkid failing to detect btrfs on zoned block
devices. This patch suggest to the user to upgrade libblkid if it
detects a version lower then 2.38.

Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-25 16:19:55 +01:00
David Sterba
f5e07cc60a btrfs-progs: warn when an experimental functionality is used
Print warning when one of the following is requested by some command
line option:

- btrfstune -b: conversion to block-group-tree
- mkfs.btrfs --num-global-roots: extent-tree-v2
- btrfs-image -d: dump image with data

Issue: #523
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-20 16:39:11 +02:00
Qu Wenruo
d8f1bd519f btrfs-progs: mkfs: fix a stack over-flow when features string are too long
[BUG]
Even with chunk_objectid bug fixed, mkfs.btrfs can still caused stack
overflow when enabling extent-tree-v2 feature (need experimental
features enabled):

  # ./mkfs.btrfs  -f -O extent-tree-v2 ~/test.img
  btrfs-progs v5.19.1
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: superblock magic doesn't match
  NOTE: several default settings have changed in version 5.15, please make sure
        this does not affect your deployments:
        - DUP for metadata (-m dup)
        - enabled no-holes (-O no-holes)
        - enabled free-space-tree (-R free-space-tree)

  Label:              (null)
  UUID:               205c61e7-f58e-4e8f-9dc2-38724f5c554b
  Node size:          16384
  Sector size:        4096
  Filesystem size:    512.00MiB
  Block group profiles:
    Data:             single            8.00MiB
    Metadata:         DUP              32.00MiB
    System:           DUP               8.00MiB
  SSD detected:       no
  Zoned device:       no
  =================================================================
  [... Skip full ASAN output ...]
  ==65655==ABORTING

[CAUSE]
For experimental build, we have unified feature output, but the old
buffer size is only 64 bytes, which is too small to cover the new full
feature string:

  extref, skinny-metadata, no-holes, free-space-tree, block-group-tree, extent-tree-v2

Above feature string is already 84 bytes, over the 64 on-stack memory
size.

This can also be proved by the ASAN output:

  ==65655==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffc4e03b1d0 at pc 0x7ff0fc05fafe bp 0x7ffc4e03ac60 sp 0x7ffc4e03a408
  WRITE of size 17 at 0x7ffc4e03b1d0 thread T0
      #0 0x7ff0fc05fafd in __interceptor_strcat /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:377
      #1 0x55cdb7b06ca5 in parse_features_to_string common/fsfeatures.c:316
      #2 0x55cdb7b06ce1 in btrfs_parse_fs_features_to_string common/fsfeatures.c:324
      #3 0x55cdb7a37226 in main mkfs/main.c:1783
      #4 0x7ff0fbe3c28f  (/usr/lib/libc.so.6+0x2328f)
      #5 0x7ff0fbe3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
      #6 0x55cdb7a2cb34 in _start ../sysdeps/x86_64/start.S:115

[FIX]
Introduce a new macro, BTRFS_FEATURE_STRING_BUF_SIZE, along with a new
sanity check helper, btrfs_assert_feature_buf_size().

The problem is I can not find a build time method to verify
BTRFS_FEATURE_STRING_BUF_SIZE is large enough to contain all feature
names, thus have to go the runtime function to do the BUG_ON() to verify
the macro size.

Now the minimal buffer size for experimental build is 138 bytes, just
bump it to 160 for future expansion.

And if further features go beyond that number, mkfs.btrfs/btrfs-convert
will immediately crash at that BUG_ON(), so we can definitely detect it.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Tested-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:12 +02:00
Qu Wenruo
56e75c9f75 btrfs-progs: mkfs: fix a crash when enabling extent-tree-v2
[BUG]
When enabling extent-tree-v2 feature at mkfs time (need to enable
experimental features), mkfs.btrfs will crash:

  # ./mkfs.btrfs  -f -O extent-tree-v2 ~/test.img
  btrfs-progs v5.19.1
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: superblock magic doesn't match
  NOTE: several default settings have changed in version 5.15, please make sure
        this does not affect your deployments:
        - DUP for metadata (-m dup)
        - enabled no-holes (-O no-holes)
        - enabled free-space-tree (-R free-space-tree)

  Segmentation fault (core dumped)

[CAUSE]
The block group tree looks like this after make_btrfs() call:

  (gdb) call btrfs_print_tree(root->fs_info->block_group_root->node, 0)
  leaf 1163264 items 1 free space 16234 generation 1 owner BLOCK_GROUP_TREE
  leaf 1163264 flags 0x0() backref revision 1
  checksum stored f137c1ac
  checksum calced f137c1ac
  fs uuid 450d4b15-4954-4574-9801-8c6d248aaec6
  chunk uuid 4c4cc54d-f240-4aa4-b88b-bd487db43444
	item 0 key (1048576 BLOCK_GROUP_ITEM 4194304) itemoff 16259 itemsize 24
		block group used 131072 chunk_objectid 256 flags SYSTEM|single
						       ^^^

This looks completely sane, but notice that chunk_objectid 256.
That 256 value is the expected one for regular non-extent-tree-v2 btrfs,
but for extent-tree-v2, chunk_objectid is reused as the global id of
extent tree where the block group belongs to.

With the old 256 value as chunk_objectid, btrfs will not find an extent
tree root for the block group, and return NULL for btrfs_extent_root()
call, and trigger segfault.

This is a regression caused by commit 1430b41427 ("btrfs-progs:
separate block group tree from extent tree v2"), which doesn't take
extent-tree-v2 on-disk format into consideration.

[FIX]
For the initial btrfs created by make_btrfs(), all block group items
will be in extent-tree global id 0, thus we can reset chunk_objectid to
0, if and only if extent-tree-v2 is enabled.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Tested-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:12 +02:00
Qu Wenruo
bed70b939f btrfs-progs: fsfeatures: properly merge -O and -R options
[BUG]
Commit "btrfs-progs: prepare merging compat feature lists" tries to
merged "-O" and "-R" options, as they don't correctly represents
btrfs features.

But that commit caused the following bug during mkfs for experimental
build:

  $ mkfs.btrfs -f -O block-group-tree  /dev/nvme0n1
  btrfs-progs v5.19.1
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: superblock magic doesn't match
  ERROR: illegal nodesize 16384 (not equal to 4096 for mixed block group)

[CAUSE]
Currently btrfs_parse_fs_features() will return a u64, and reuse the
same u64 for both incompat and compat RO flags for experimental branch.

This can easily leads to conflicts, as
BTRFS_FEATURE_INCOMPAT_MIXED_BLOCK_GROUP and
BTRFS_FEATURE_COMPAT_RO_BLOCK_GROUP_TREE both share the same bit
(1 << 2).

Thus for above case, mkfs.btrfs believe it has set MIXED_BLOCK_GROUP
feature, but what we really want is BLOCK_GROUP_TREE.

[FIX]
Instead of incorrectly re-using the same bits in btrfs_feature, split
the old flags into 3 flags:

- incompat_flag
- compat_ro_flag
- runtime_flag

The first two flags are easy to understand, the corresponding flag of
each feature.
The last runtime_flag is to compensate features which doesn't have any
on-disk flag set, like QUOTA and LIST_ALL.

And since we're no longer using a single u64 as features, we have to
introduce a new structure, btrfs_mkfs_features, to contain above 3
flags.

This also mean, things like default mkfs features must be converted to
use the new structure, thus those old macros are all converted to
const static structures:

- BTRFS_MKFS_DEFAULT_FEATURES + BTRFS_MKFS_DEFAULT_RUNTIME_FEATURES
  -> btrfs_mkfs_default_features

- BTRFS_CONVERT_ALLOWED_FEATURES -> btrfs_convert_allowed_features

And since we're using a structure, it's not longer as easy to implement
a disallowed mask.

Thus functions with @mask_disallowed are all changed to using
an @allowed structure pointer (which can be NULL).

Finally if we have experimental features enabled, all features can be
specified by -O options, and we can output a unified feature list,
instead of the old split ones.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:11 +02:00
Qu Wenruo
2cdc8dddbf btrfs-progs: mkfs: offset inode numbers of the source filesystem
[BUG]
When running mkfs tests on a newly rebooted minimal system, it can cause
mkfs/009 to fail.

The reproduce steps requires /tmp to has minimal files in the first
place.

  # mkdir /tmp/rootdir
  # xfs_io -f -c "pwrite 0 16k" /tmp/rootdir
  # mkfs.btrfs --rootdir /tmp/rootdir -f $dev
  # btrfs check $dev
  Opening filesystem to check...
  Checking filesystem on /dev/test/scratch1
  UUID: 6821b3db-f056-4c18-b797-32679dcd4272
  [1/7] checking root items
  [2/7] checking extents
  data backref 13631488 root 5 owner 170 offset 0 num_refs 0 not found in extent tree
  incorrect local backref count on 13631488 root 5 owner 170 offset 0 found 1 wanted 0 back 0x55ff6cd72260
  backref 13631488 root 5 not referenced back 0x55ff6cd4c1f0
  incorrect global backref count on 13631488 found 2 wanted 1
  backpointer mismatch on [13631488 16384]
  ERROR: errors found in extent allocation tree or chunk allocation

[CAUSE]
The extent tree has the following weird item:

	item 0 key (13631488 EXTENT_ITEM 16384) itemoff 16250 itemsize 33
		refs 1 gen 0 flags DATA
		tree block backref root FS_TREE

This is an extent item for data, thus it should not have an inline tree
backref.

Then checking the fs tree:

	item 0 key (170 INODE_ITEM 0) itemoff 16123 itemsize 160
		generation 7 transid 0 size 16384 nbytes 16384
		block group 0 mode 100600 links 1 uid 1000 gid 1000 rdev 0
		sequence 0 flags 0x0(none)
		atime 1664866393.0 (2022-10-04 14:53:13)
		ctime 1664863510.0 (2022-10-04 14:05:10)
		mtime 1664863455.0 (2022-10-04 14:04:15)
		otime 0.0 (1970-01-01 08:00:00)

There is an inode item before the root dir inode.

And that inode number 170 is causing the problem.

In traverse_directory(), we use the inode number reported from stat()
directly as btrfs inode number, and pass it to
btrfs_record_file_extent(), which finally calls btrfs_inc_extent_ref(),
with above 170 passed as @owner parameter.

But inside btrfs_inc_extent_ref() we use that @owner value to determine
if it's a data backref.
Since we got a smaller than BTRFS_FIRST_FREE_OBJECTID, btrfs treats it
as tree block, and cause the above problem.

[FIX]
As a quick fix, always add BTRFS_FIRST_FREE_OBJECTID to all inode number
directly grabbed from stat().

And add an ASSERT() in __btrfs_record_file_extent() to catch unexpected
objectid.

This is not a perfect solution, as the resulted fs will has a huge gap
in its inodes:

	item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
	item 4 key (426 INODE_ITEM 0) itemoff 15883 itemsize 160

For a proper fix, we should allocate new btrfs inode numbers in a
sequential order, but that would be another series of patches.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba
ccb2d4aa45 btrfs-progs: device-utils: rename btrfs_device_size
There's a group of helpers to read device size, the btrfs_device_size
should be one of them. Rename it and so minor cleanup.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba
ea0b894967 btrfs-progs: mkfs: do proper error handling
Replace BUG_ON after transaction start failures, all the functions
already handle errors and return them to the caller. The other error
handling is for impossible conditions.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba
a827bb2db8 btrfs-progs: use template for transaction commit error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba
8fcafae04a btrfs-progs: use template for transaction start error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:10 +02:00
David Sterba
c2be0e2ce0 btrfs-progs: use template for out of memory error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
David Sterba
f7a768d624 btrfs-progs: mkfs: remove support for option --leafsize
The leafsize has never been different from nodesize and since 4.0 (2015)
it's been alias for nodesize. This should be enough time for everybody
to update so the support is removed.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
David Sterba
48c5740e87 btrfs-progs: docs: clarify meaning of mkfs --byte-count
The meaning of the -b/--byte-count option is different than what the
help text says. Historically it was used to set the filesystem size but
with multiple devices it sets the size on each device:

  $ mkfs.btrfs /dev/sdx[1234]
  ...
  Number of devices:  4
  Devices:
     ID        SIZE  PATH
      1     2.00GiB  /dev/sdx1
      2     2.00GiB  /dev/sdx2
      3     2.00GiB  /dev/sdx3
      4     2.00GiB  /dev/sdx4

And when set to 1G:

  $ mkfs.btrfs -b 1G /dev/sdx[1234]
  ...
  Number of devices:  4
  Devices:
     ID        SIZE  PATH
      1     1.00GiB  /dev/sdx1
      2     1.00GiB  /dev/sdx2
      3     1.00GiB  /dev/sdx3
      4     1.00GiB  /dev/sdx4

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
David Sterba
6743a47b35 btrfs-progs: mkfs: rename dev_cnt to device_count
The variable name could expanded, it's not necessary to be abbreviated.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
David Sterba
971bee96c2 btrfs-progs: mkfs: use _set suffix for option tracking
Unify naming of variables that track if the command option is set to use
the _set suffix.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
David Sterba
c85bb9b5bf btrfs-progs: mkfs: group feature option declarations
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
David Sterba
b73a29936a btrfs-progs: remove unnecessary casts for u64
The (unsigned long long) type casts can be dropped, printf understands
%llu and u64 and does not warn. In cases where the type is not u64 keep
the cast.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:09 +02:00
Li Zhang
bb2eed3aa5 btrfs-progs: mkfs: run device preparation in parallel
When devices are formatted as btrfs, btrfs_prepare_device is called
sequentially for each device, which takes too much time.

Put each btrfs_prepare_device into a thread, wait for the first thread
to complete to mkfs.btrfs, and wait for other threads to complete before
adding other devices to the file system.

During the preparation it's either trim/discard or zone reset.

This was tested with TCMU emulation with two zoned devices.  Each device
is 2000G (about 19.53 TiB), the region size is 4MB, Use the following
parameters for targetcli:

  create name=zbc0 size=20000G cfgstring=model-HM/zsize-4/conv-100@~/zbc0.raw

Call difftime to calculate the running time of the function
btrfs_prepare_device.  Calculate the time from thread creation to
completion of all threads after patching:

  $ lsscsi -p
  [10:0:1:0]   (0x14)  LIO-ORG  TCMU ZBC device  0002  /dev/sdb   -          none
  [11:0:1:0]   (0x14)  LIO-ORG  TCMU ZBC device  0002  /dev/sdc   -          none

  $ sudo mkfs.btrfs -d single -m single -O zoned /dev/sdc /dev/sdb -f
  ....
  time for prepare devices:4.000000.
  ....

  $ sudo mkfs.btrfs -d single -m single -O zoned /dev/sdc /dev/sdb -f
  ...
  time for prepare devices:2.000000.
  ...

Issue: #496
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Li Zhang <zhanglikernel@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
David Sterba
b19ab3ba9f btrfs-progs: mkfs: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba
6edd4b2121 btrfs-progs: factor string helpers out of utils.c
Utils is the catch-all file, we can now separate some string utility
functions.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba
0f03da53cc btrfs-progs: mkfs: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba
61de520917 btrfs-progs: mkfs: reorder includes
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00
David Sterba
c33c2c66b3 btrfs-progs: mkfs: duplicate argument for --rootdir path
The source dir points to the argv data, we should make a copy to be sure
it won't change due to further processing.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00
David Sterba
54fe8e648a btrfs-progs: mkfs: open code label parsing helper
The helper parse_label is used only once and is trivial. Open code it in
the argument parsing, also to make the exit() is more visible.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00
David Sterba
74be9f5115 btrfs-progs: mkfs: open code profile parsing helper
There's a helper to parse profile name and exits on error. As this is a
trivial helper we can open code it and adapt the error message to be
more specific what failed.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00
David Sterba
e331037d40 btrfs-progs: factor out and export rotational/ssd device helper
The helper belongs to device utils, move it from the mkfs.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00
Qu Wenruo
1c414061ed btrfs-progs: mkfs: add artificial dependency for block group tree
To reduce the test matrix and to follow the kernel behavior, make sure
for block-group-tree feature, we have no-holes and free-space-tree
features enabled.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-09-12 18:25:32 +02:00
Qu Wenruo
1430b41427 btrfs-progs: separate block group tree from extent tree v2
Block group tree feature is completely a standalone feature, and it has
been over 5 years before the initial introduction to solve the long
mount time.

I don't really want to waste another 5 years waiting for a feature which
may or may not work, but definitely not properly reviewed for its
preparation patches.

So this patch will separate the block group tree feature into a
standalone compat RO feature.

There is a catch, in mkfs create_block_group_tree(), current
tree-checker only accepts block group item with valid chunk_objectid,
but the existing code from extent-tree-v2 didn't properly initialize it.

This patch will also fix above mentioned problem so kernel can mount it
correctly.

Now mkfs/fsck should be able to handle the fs with block group tree.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-09-12 18:25:32 +02:00
Qu Wenruo
c5a21a7814 btrfs-progs: don't save block group root into super block
The extent tree v2 (thankfully not yet fully materialized) needs a
new root for storing all block group items.

My initial proposal years ago just added a new tree rootid, and load it
from tree root, just like what we did for quota/free space tree/uuid/extent
roots.

But the extent tree v2 patches introduced a completely new (and to me,
wasteful) way to store block group tree root into super block.

Currently there are only 3 trees stored in super blocks, and they all
have their valid reasons:

- Chunk root
  Needed for bootstrap.

- Tree root
  Really the entrance of all trees.

- Log root
  This is special as log root has to be updated out of existing
  transaction mechanism.

There is not even any reason to put block group root into super blocks,
the block group tree is updated at the same timing as old extent tree,
no need for extra bootstrap/out-of-transaction update.

So just move block group root from super block into tree root.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-09-12 15:31:27 +02:00
Qu Wenruo
cc4a249e99 btrfs-progs: mkfs: dynamically modify mkfs blocks array
In mkfs_btrfs(), we have a btrfs_mkfs_block array to store how many tree
blocks we need to reserve for the initial btrfs image.

Currently we have two very similar arrays, extent_tree_v1_blocks and
extent_tree_v2_blocks.

The only difference is just v2 has an extra block for block group tree.

This patch will add two helpers, mkfs_blocks_add() and
mkfs_blocks_remove() to properly add/remove one block dynamically from
the array.

This allows 3 things:

- Merge extent_tree_v1_blocks and extent_tree_v2_blocks into one array
  The new array will be the same as extent_tree_v1_blocks.
  For extent-tree-v2, we just dynamically add MKFS_BLOCK_GROUP_TREE.

- Remove free space tree block on-demand
  This only works for extent-tree-v1 case, as v2 has a hard requirement
  on free space tree.
  But this still make code much cleaner, not doing any special hacks.

- Allow future expansion without introduce new array
  I strongly doubt why this is not properly done in extent-tree-v2
  preparation patches.
  We should not allow bad practice to sneak in just because it's some
  preparation patches for a larger feature.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-09-12 15:31:26 +02:00
David Sterba
a33af50c52 btrfs-progs: add constant for initial getopt values
Add constant for initial value to avoid unexpected clashes with user
defined getopt values and shift the common size getopt values.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-08-16 15:18:11 +02:00
Qu Wenruo
007c799ca8 btrfs-progs: mkfs: use sectorsize as nodesize fallback for mixed profiles
[BUG]
When running btrfs/011 with subpage case, even with RAID56 support, it
still fails with the following error:

 QA output created by 011
 *** test btrfs replace
 mkfs failed
 (see /home/adam/xfstests-dev/results//btrfs/011.full for details)

The full log shows:

  ---------workout "-m single -d single -M" 1 no 64-----------
  ERROR: illegal nodesize 65536 (not equal to 4096 for mixed block group)
  mkfs failed

This is a critical error, making test case to be aborted, without
checking the rest profiles.

[CAUSE]
Mkfs.btrfs always uses the maximum value between sectorsize and page
size for its mixed profile nodesize.

For subpage case, it means we always go PAGE_SIZE, no matter whatever
the sectorsize is passed in.

[FIX]
Just get rid of the direct PAGE_SIZE usage when determining nodesize for
mixed profiles.
And use sectorsize directly (either passed in by the user, or
determined from page size).

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-04-26 01:14:48 +02:00
Naohiro Aota
e273f9ebbd btrfs-progs: zoned: fix initial system BG location
Currently, we create the initial system block group in the zone 2. That
will create the BG at 64MB when the zone size is 32 MB, which collides with
the regular superblock location. It results in mount failure with:

  BTRFS info (device nullb0): zoned mode enabled with zone size 33554432
  BTRFS error (device nullb0): zoned: block group 67108864 must not contain super block
  BTRFS error (device nullb0): failed to read block groups: -117
  BTRFS error (device nullb0): open_ctree failed

Fix that by calculating the proper location of the initial system BG. It
avoids using zones reserved for zoned superblock logging and the zones
where a regular superblock resides.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-04-08 23:17:35 +02:00
Josef Bacik
659f041537 btrfs-progs: mkfs: create the global root's
Now that we have all of the supporting code, add the ability to create
all of the global roots for an extent tree v2 fs.  This will default to
nr_cpu's, but also allow the user to specify how many global roots they
would like.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 18:07:26 +01:00
Josef Bacik
1f89a5d461 btrfs-progs: mkfs: set chunk_item_objectid properly for extent tree v2
Our initial block group will use global root id 0 with extent tree v2,
so adjust the helper to take the chunk_objectid as an argument, as we'll
set this to 0 for extent tree v2 and then
BTRFS_FIRST_CHUNK_TREE_OBJECTID for extent tree v1.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 18:07:24 +01:00
Josef Bacik
02fb308bdc btrfs-progs: make btrfs_create_tree take a key for the root key
We're going to start create global roots from mkfs, and we need to have
a offset set for the root key.  Make the btrfs_create_tree() take a key
for the root_key instead of just the objectid so we can setup these new
style roots properly.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 18:07:22 +01:00
Josef Bacik
27eaa3b514 btrfs-progs: set the number of global roots in the super block
In order to make sure the file system is consistent we need to record
the number of global roots we should have in the super block.  We could
infer this from the number of global roots we find, however this could
lead to interesting fuzzing problems, so add a source of truth to the
super block in order to make it easier to verify the file system is
consistent.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 18:07:14 +01:00
Josef Bacik
7e3bf7fc44 btrfs-progs: mkfs: add support for the block group tree
Add the extent tree v2 table with the block group tree as a root, and
then create the empty root and use the proper root for cleanup up the
temporary block groups.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 18:07:02 +01:00
Josef Bacik
360103e610 btrfs-progs: mkfs: use the btrfs_block_group_root helper
Instead of accessing the extent root directory for modifying block
groups, use the helper which will do the correct thing based on the
flags of the file system.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 18:06:56 +01:00
Josef Bacik
5dc3964aaa btrfs-progs: remove the _nr from the item helpers
Now that all callers are using the _nr variations we can simply rename
these helpers to btrfs_item_##member/btrfs_set_item_##member and change
the actual item SETGET funcs to raw_item_##member/set_raw_item_##member
and then change all callers to drop the _nr part.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 15:13:13 +01:00
Josef Bacik
04ffea07e4 btrfs-progs: add btrfs_set_item_*_nr() helpers
We have a lot of the following patterns

	item = btrfs_item_nr(nr);
	btrfs_set_item_*(eb, item, val);

	btrfs_set_item_*(eb, btrfs_item_nr(nr), val);

in a lot of places in our code.  Instead add _nr variations of these
helpers and convert all of the users to this new helper.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 15:13:13 +01:00
Josef Bacik
ed098523dc btrfs-progs: reduce usage of __BTRFS_LEAF_DATA_SIZE
This helper only takes the nodesize, but in the future it'll take a bool
to indicate if we're extent tree v2.  The remaining users are all where
we only have extent_buffer, but we should always have a valid
eb->fs_info in these cases, so add BUG_ON()'s for the !eb->fs_info case
and then convert these callers to use BTRFS_LEAF_DATA_SIZE which takes
the fs_info.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 15:13:13 +01:00
Josef Bacik
7300aeecff btrfs-progs: store LEAF_DATA_SIZE in the mkfs_config
We use __BTRFS_LEAF_DATA_SIZE() in a few places for mkfs.  With extent
tree v2 we'll be increasing the size of btrfs_header, so it'll be kind
of annoying to add flags to all callers of __BTRFS_LEAF_DATA_SIZE, so
simply calculate it once and put it in the mkfs_config and use that.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-03-09 15:13:12 +01:00
Johannes Thumshirn
89191f8c12 btrfs-progs: pass in block-group type to zoned_profile_supported
Pass BTRFS_BLOCK_GROUP_DATA and BTRFS_BLOCK_GROUP_METADATA to
zoned_profile_supported(), so we can actually distinguish if it is a data
or a meta-data block group.

Fixes: 8f914d518a46 ("btrfs-progs: zoned support DUP on metadata block groups")
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-02-16 22:48:01 +01:00