mkfs_main() is a main-like function, meaning that return and exit are
equivalent. Deduplicate our cleanup code by moving the error label.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently mkfs uses its own create_uuid_tree(), but that function is
only handling FS_TREE. This means for btrfs-convert we do not generate
the uuid tree, nor add the UUID of the image subvolume. This can be a
problem if we're going to support multiple subvolumes during mkfs time.
To address this, introduce a new helper, btrfs_rebuild_uuid_tree():
- Create a new uuid tree if there is not one
- Remove all the existing items from uuid tree
- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree
By this, this new helper can handle all the uuid tree generation needs for:
- Current mkfs
Only one uuid entry for FS_TREE
- Current btrfs-convert
Only FS_TREE and the image subvolume
- Future multi-subvolume mkfs
As we do the scan for all subvolumes.
- Future "btrfs rescue rebuild-uuid-tree"
Signed-off-by: Qu Wenruo <wqu@suse.com>
The modification is minimal:
- Replace WARN_ON() with UASSERT()
- Remove the @trans parameter for btrfs_extend_item() and
btrfs_mark_buffer_dirty()
As progs version doesn't need a transaction handler.
- Remove the btrfs_uuid_tree_add() in mkfs/main.c
Signed-off-by: Qu Wenruo <wqu@suse.com>
Currently we already have a kernel-shared/uuid-tree.c, which is mostly
shared with kernel.
Kernel also has a uuid-tree.h, but we are still using ctree.h for the
header.
Move all the uuid-tree related definitions to kernel-shared/uuid-tree.h,
making future code sync easier.
Signed-off-by: Qu Wenruo <wqu@suse.com>
There are two different subvolume/data reloc tree creation routines:
- create_subvol() from convert/main.c
* calls btrfs_copy_root() to create an empty root
This is not safe, as it relies on the source root to be empty.
* calls btrfs_read_fs_root() to add it to the cache and trace it
properly
* calls btrfs_make_root_dir() to initialize the empty new root
- create_data_reloc_tree() from mkfs/main.c
* calls btrfs_create_tree() to create an empty root
* Manually add the root to fs_root cache
This is only safe for data reloc tree as it's never updated
inside btrfs-progs.
But not safe for other subvolume trees.
* manually setup the root dir
Both have their good and bad aspects, so here we introduce a new helper,
btrfs_make_subvolume():
- Calls btrfs_create_tree() to create an empty root
- Calls btrfs_read_fs_root() to setup the cache and tracking properly
- Calls btrfs_make_root_dir() to initialize the root dir
- Calls btrfs_update_root() to reflect the rootdir change
So this new helper can replace both create_subvol() and
create_data_reloc_tree().
Signed-off-by: Qu Wenruo <wqu@suse.com>
Use the safe version of strncpy that makes sure the string is
terminated.
To be noted:
- the conversion in scrub path handling was skipped
- sizes of device paths in some ioctl related structures is
BTRFS_DEVICE_PATH_NAME_MAX + 1
Recently gcc 13.3 started to detect problems with our use of strncpy
potentially lacking the null terminator, warnings like:
cmds/inspect.c: In function ‘cmd_inspect_logical_resolve’:
cmds/inspect.c:294:33: warning: ‘__builtin_strncpy’ specified bound 4096 equals destination size [-Wstringop-truncation]
294 | strncpy(mount_path, mounted, PATH_MAX);
| ^
Signed-off-by: David Sterba <dsterba@suse.com>
Now that there's only __strncpy_null we can drop the underscore and move
it to string-utils as it's a generic string function rather than
something for paths.
Signed-off-by: David Sterba <dsterba@suse.com>
The label is of a fixed size 256 bytes and expects the zero terminator.
Using __strncpy_null is correct as it makes sure there's always the zero
termination but the argument passed in skips the last character.
Signed-off-by: David Sterba <dsterba@suse.com>
Currently raid-stripe-tree feature is still experimental as it requires
a BTRFS_DEBUG kernel to recognize it. To avoid confusion move it back
to experimental so regular users won't incorrectly set it.
And since RST is no longer supported by default, also change the RST
profile detection so that for non-experimental build we won't enable RST
according to the data profiles.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
While "byte_count" is eventually rounded down to sectorsize at make_btrfs()
or btrfs_add_to_fs_id(), it would be better round it down first and do the
size checks not to confuse the things.
Also, on a zoned device, creating a filesystem whose size is not aligned
to the zone boundary can be confusing. Round it down further to the zone
boundary.
The size calculation with a source directory is also tweaked to be aligned.
device_get_partition_size_fd_stat() must be aligned down not to exceed the
device size. And, btrfs_mkfs_size_dir() should have return sectorsize aligned
size. So, add an UASSERT for it.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently, we check if a device is larger than 5 zones to determine we can
create btrfs on the device or not. Actually, we need more zones to create
DUP block groups, so it fails with "ERROR: not enough free space to
allocate chunk". Implement proper support for non-SINGLE profile.
Also, current code does not ensure we can create tree-log BG and data
relocation BG, which are essential for the real usage. Count them as
requirement too.
The calculation for a regular btrfs is also adjusted to use dev_stripes
style.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
We are going to implement a better minimum size calculation for the zoned
mode. Move the current logic to btrfs_min_dev_size() and unify the size
checking path.
Also, convert "int mixed" to "bool mixed" while at it.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
test_minimum_size() already checks if each device can host the initial
block groups. There is no need to check if the first device can host the
initial system chunk again.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
block_count and dev_block_count are counting the size in bytes. And,
comparing them with e.g, "min_dev_size" is confusing. Rename them to
represent the unit better.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a bug report that, with very weird mount status, there can be
some mount source which can not be accessed:
/path/dev/exports fs 500G 57G 444G 12% /path/dev/exports
Strace shows we can not access the above mount source:
131065 stat("/path/dev/exports", 0x7ffed17b8e20) = -1 EACCES (Permission denied)
And lead to failed mount check:
131065 write(2, "ERROR: ", 7) = 7
131065 write(2, "cannot check mount status of /de"..., 56) = 56
131065 write(2, "\n", 1) = 1
[CAUSE]
The mount check is based on libblkid, which gives the mount source, and
for non-btrfs mounts, we call path_is_reg_or_block_device() to check if
we even need to continue checking.
But in above case, the mount source is another fs, and we can not access
the source.
So we error out causing the check_mounted() to return error.
[FIX]
There is never any guarantee we can access the mount source, but on the
other hand, I do not want to ignore all access failure for the mount
source.
Let test_status_for_mkfs() to only skip check_mounted() error if
@force_overwrite is true.
This would still keep the old strict checks on whether the target is
already mounted, but if the end user really knows that certain mount
source do not need to be checked, they can always pass "-f" option to
skip the false alerts.
Link: https://bugzilla.suse.com/show_bug.cgi?id=1223799
Reported-by: Jiri Belka <jiri.belka@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use the objectid, type, offset natural order as it's more readable and
we're used to read keys like that.
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a bug report that mkfs.btrfs can not specify block-group-tree
feature along with zoned devices:
# mkfs.btrfs /dev/nullb0 -O block-group-tree,zoned
btrfs-progs v6.7.1
See https://btrfs.readthedocs.io for more information.
Resetting device zones /dev/nullb0 (40 zones) ...
NOTE: several default settings have changed in version 5.15, please make sure
this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)
ERROR: error during mkfs: Invalid argument
[CAUSE]
During mkfs, we need to write all the 7 or 8 tree blocks into the
metadata zone, and since it's zoned device, we need to fulfill all the
requirement for zoned writes, including:
- All writes must be in sequential bytenr
- Buffer must be aligned to sector size
The sequential bytenr requirement is already met by the mkfs design, but
the second requirement on memory alignment is never met for metadata, as
we put the contents of a leaf in extent_buffer::data[], which is after a
lot of small members.
Thus metadata IO buffer would never be aligned to sector size (normally
4K).
And we require btrfs_pwrite() and btrfs_pread() to handle the memory
alignment for us.
However in create_block_group_tree() we didn't use btrfs_pwrite(), but
plain pwrite() call directly, which would lead to -EINVAL error due to
memory alignment problem.
[FIX]
Just call btrfs_pwrite() instead of the plain pwrite() in
create_block_group_tree().
Issue: #765
Pull-request: #767
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use the templated error message for transaction failures, use the same
pattern assigning the ret and errno.
Signed-off-by: David Sterba <dsterba@suse.com>
Sync a few more file on the source level with kernel 6.8.
- type cleanups
- defines and enums
- comments
- parameter updates
- error handling
Signed-off-by: David Sterba <dsterba@suse.com>
We've deprecated the -R option and runtime feature distinction, there's
only one option -O recommended so let it also print on the same line.
Incompat/compat/runtime status of a feature shall be consulted with the
documentation.
Signed-off-by: David Sterba <dsterba@suse.com>
Be verbose about the potential compatibility problems with the
sectorsize and page size. Also print the page size on the overview.
Signed-off-by: David Sterba <dsterba@suse.com>
We have had working subpage support in Btrfs for many cycles now.
Generally, we do not want people creating filesystems by default
with non-4k sectorsizes since it creates portability problems.
As the subpage has stabilized it seems to be safe to do the switch.
This may still affect users that relying on the previous behaviour.
Issue: #604
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Eric Curtin <ecurtin@redhat.com>
Signed-off-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch introduces a new parser helper, parse_u64_with_suffix(),
which has a better error handling, following all the parse_*()
helpers to return non-zero value for errors.
This new helper is going to replace parse_size_from_string(), which
would directly call exit(1) to stop the whole program.
Furthermore most callers of parse_size_from_string() are expecting
exit(1) for error, so that they can skip the error handling.
For those call sites, introduce a wrapper, arg_strtou64_with_suffix(),
to do that. The only disadvantage is a little less detailed error
report for why the parse failed, but for most cases the generic error
string should be enough.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In zoned mode print zone count for each device, the zone size must be
the same so it's sufficient to print it in the summary.
$ mkfs.btrfs -O zoned /dev/nullb[0-3]
...
Zoned device: yes
Zone size: 16.00MiB
...
Devices:
ID SIZE ZONES PATH
1 512.00MiB 32 /dev/nullb0
2 256.00MiB 16 /dev/nullb1
3 1.00GiB 64 /dev/nullb2
4 2.00GiB 128 /dev/nullb3
Issue: #693
Signed-off-by: David Sterba <dsterba@suse.com>
In experimental build, read global '--param zone-size=SIZE' and use it
as emulated zone size. This is for testing only, will be promoted to a
proper option in the future.
Signed-off-by: David Sterba <dsterba@suse.com>
We need to validate the device uuid the same way as the fsid:
$ ./mkfs.btrfs --device-uuid 18eabcf0-6766-4fbf-b366-71b4ae725b2- img
btrfs-progs v6.5.2
See https://btrfs.readthedocs.io for more information.
ERROR: could not parse device UUID: 18eabcf0-6766-4fbf-b366-71b4ae725b2-
Signed-off-by: David Sterba <dsterba@suse.com>
Print the device uuid in the summary in case it's specified on the
command line, not always as it would be confusing and is not usually
needed. Can be found in 'btrfs inspect-internal dump-super' as
device_item.uuid .
Signed-off-by: David Sterba <dsterba@suse.com>
Add option --device-uuid that will set the device uuid item in super
block.
This is useful for creating a filesystem with a specific device uuid,
namely for testing.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The commit ("btrfs-progs: allow duplicate fsid for single device
filesystems") lets the duplicate fsid used for a new mkfs document this.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When using "mkfs.btrfs" with "--rootdir" option, the top level inode
(rootdir) will not get the same xattr from the source dir:
mkdir -p source_dir/
touch source_dir/file
setfattr -n user.rootdir_xattr source_dir/
setfattr -n user.regular_xattr source_dir/file
mkfs.btrfs -f --rootdir source_dir $dev
mount $dev $mnt
getfattr $mnt
# Nothing <<<
getfattr $mnt/file
# file: $mnt/file
user.regular_xattr <<<
[CAUSE]
In function traverse_directory(), we only call add_xattr_item() for all
the child inodes, not really for the rootdir inode itself, leading to
the missing xattr items.
Not only xattr, in fact we also miss the uid/gid/timestamps/mode for the
rootdir inode.
[FIX]
Extract a dedicated function, copy_rootdir_inode(), to handle every
needed attributes for the rootdir inode, including:
- xattr
- uid
- gid
- mode
- timestamps
Issue: #688
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When running mkfs.btrfs with --rootdir on a block device, and the source
directory contains a sparse file, whose size is larger than the block
size, then mkfs.btrfs would fail:
# lsblk /dev/test/test
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
test-test 253:0 0 10G 0 lvm
# mkdir -p /tmp/output
# truncate -s 20G /tmp/output/file
# mkfs.btrfs -f --rootdir /tmp/output /dev/test/test
# sudo mkfs.btrfs -f /dev/test/scratch1 --rootdir /tmp/output/
btrfs-progs v6.3.3
See https://btrfs.readthedocs.io for more information.
ERROR: unable to zero the output file
[CAUSE]
Mkfs.btrfs would try to zero out the target file according to the total
size of the directory.
However the directory size is calculated using the file size, not the
real bytes taken by the file, thus for such sparse file with holes only,
it would still take 20G.
Then we would use that 20G size to zero out the target file, but if the
target file is a block device, we would fail as we can not enlarge a block
device.
[FIX]
When zeroing the file, we only enlarge it if the target is a regular
file.
Otherwise we warn about the size and continue.
Please note that, since "mkfs.btrfs --rootdir" doesn't handle sparse
file any differently from regular file, above case would still fail due
to ENOSPC, as will write zeros into the target file inside the fs.
Proper handling for sparse files would need a new series of patch to
address.
Issue: #653
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The kernel patches for RST and squota are queued for 6.7, we need to be
able to test the features so it's not necessary to hide the mkfs support
under experimental build. The kernel may still need debug build to
enable mount.
Signed-off-by: David Sterba <dsterba@suse.com>
This function is only used in mkfs, and doesn't exist in the kernel in
ctree.c. Additionally we have a uuid lookup function to see if the uuid
exists in the tree, which for mkfs it won't because we just created the
tree. Move btrfs_uuid_tree_add into mkfs, and remove the lookup
function as it's not needed.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function and it's related functions only exist for the utilities
that populate existing file systems, and do not exist in the upstream
kernel. Move this function and the related function into it's own
common source file and out of the kernel-shared sources, and then update
all of the users to include the new location of this code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This simply zero's out the path, and this is used everywhere we use a
stack path. Drop this usage and simply init the path's to empty instead
of using a function to do the memset.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the ability to enable simple quotas from mkfs with '-O squota'
There is some complication around handling enable_gen while still
counting the root node of an fs. To handle this, employ a hack of doing
a no-op write on the root node to bump its generation up above that of
the qgroup enable generation, which results in counting it properly.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
For single device btrfs filesystem, allow duplicate fsid to be created.
This should be used with caution as more devices with the same uuid
could be confused with each other.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Allow for RAID levels 0, 1 and 10 on zoned devices if the RAID stripe tree
is used.
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The struct open_ctree_flags currently holds arguments for
open_ctree_fs_info(), it can be confusing when mixed with a local variable
named open_ctree_flags as below in the function cmd_inspect_dump_tree().
cmd_inspect_dump_tree()
::
struct open_ctree_flags ocf = { 0 };
::
unsigned open_ctree_flags;
So rename struct open_ctree_flags to struct open_ctree_args.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
mkfs.btrfs help message for --quiet is 'no message except errors' so
we probably ought to silence this as well in the quiet case.
Author: Dominique Martinet <dominique.martinet@atmark-techno.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Print the source directory for --rootdir and if --shrink is used. With
-vv then print the individual files as added:
$ mkfs.btrfs --rootdir dir --shrink -vv img
...
Rootdir from: Documentation
ADD: /btrfs-progs/Documentation/btrfs-check.rst
...
ADD: /btrfs-progs/Documentation/btrfs-send.rst
Shrink: yes
Label: (null)
UUID: 40d3a16f-02d8-40d7-824b-239cee528093
...
The 'Rootdir from' is printed before the files are added so there's now
message before the files are added which could take some time.
Issue: #627
Signed-off-by: David Sterba <dsterba@suse.com>
This patch would modify btrfs_csum_file_block() to handle csum type
other than the one used in the current fs.
The new data checksum would use a different objectid (-13) to
distinguish with the existing one (-10).
This needs to change tree-checker to skip the item size checks,
since new csum can be larger than the original csum.
After this stage, the resulted csum tree would look like this:
item 0 key (CSUM_CHANGE EXTENT_CSUM 13631488) itemoff 8091 itemsize 8192
range start 13631488 end 22020096 length 8388608
item 1 key (EXTENT_CSUM EXTENT_CSUM 13631488) itemoff 7067 itemsize 1024
range start 13631488 end 14680064 length 1048576
Note the itemsize is 8 times the original one, as the original csum is
CRC32, while target csum is SHA256, which is 8 times the size.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The function write_and_map_eb() is quite abused as a way to write any
generic buffer back to disk.
But we have a more suitable function already, write_data_to_disk().
This patch would remove the abused write_data_to_disk() calls, and
convert the only three valid call sites to write_data_to_disk() instead.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Mark Harmstone
Qu Wenruo
Yaroslav Halchenko
David Sterba
Naohiro Aota
Neal Gompa
Anand Jain
Josef Bacik
Boris Burkov
Johannes Thumshirn
Dominique Martinet