Commit Graph

7256 Commits

Author SHA1 Message Date
David Sterba 39470a52b5 btrfs-progs: docs: clarify swapfile an multi-device filesystem
It is possible to create swapfile on a multi-device filesystem but it's
not reliable. The check that verifies that in kernel:

10698                 } else if (device != map->stripes[0].dev) {
10699                         btrfs_warn(fs_info, "swapfile must be on one device");
10700                         ret = -EINVAL;
10701                         goto out;
10702                 }

This does not count devices but rather the actual placement of the
swapfile extents, so multi-device filesystem with single profile can
create it as long as there's enough space and the allocator decides to
place it properly.

[ci skip]

Pull-request: #839
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 20:05:11 +02:00
David Sterba 704437e022 btrfs-progs: ci: enable thread sanitizer for sanitizer workflow
The thread sanitizer finds race conditions and in the past did find
some bugs. There's not much threaded code, it's namely the progress
tracking in btrfs-convert so the coverage is slightly redundant. Add it
just in case.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 20:05:11 +02:00
Qu Wenruo 5aed7ea89a btrfs-progs: qgroup clear-stale: fix memory leak
[BUG]
ASAN test fails at misc/055 with the following leak:

Qgroupid    Referenced    Exclusive   Path
--------    ----------    ---------   ----
0/5           16.00KiB     16.00KiB   <toplevel>
0/256         16.00KiB     16.00KiB   <stale>
====== RUN CHECK /home/runner/work/btrfs-progs/btrfs-progs/btrfs qgroup clear-stale /home/runner/work/btrfs-progs/btrfs-progs/tests/mnt

=================================================================
==102571==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 4096 byte(s) in 1 object(s) allocated from:
    #0 0x7fd1c98fbb37 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x55aa2f8953f8 in btrfs_util_subvolume_path_fd libbtrfsutil/subvolume.c:178
    #2 0x55aa2f8fa2a6 in get_or_add_qgroup cmds/qgroup.c:837
    #3 0x55aa2f8fa7e9 in update_qgroup_info cmds/qgroup.c:883
    #4 0x55aa2f8fd912 in __qgroups_search cmds/qgroup.c:1385
    #5 0x55aa2f8fe196 in qgroups_search_all cmds/qgroup.c:1453
    #6 0x55aa2f902a7c in cmd_qgroup_clear_stale cmds/qgroup.c:2281
    #7 0x55aa2f73425b in cmd_execute cmds/commands.h:126
    #8 0x55aa2f734bcc in handle_command_group /home/runner/work/btrfs-progs/btrfs-progs/btrfs.c:177
    #9 0x55aa2f73425b in cmd_execute cmds/commands.h:126
    #10 0x55aa2f735a96 in main /home/runner/work/btrfs-progs/btrfs-progs/btrfs.c:518
    #11 0x7fd1c942a1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 08134323d00289185684a4cd177d202f39c2a5f3)
    #12 0x7fd1c942a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 08134323d00289185684a4cd177d202f39c2a5f3)
    #13 0x55aa2f734144 in _start (/home/runner/work/btrfs-progs/btrfs-progs/btrfs+0x84144) (BuildId: 56f3dd838e1ae189c142c5d27fac025cd46deddb)

Indirect leak of 432 byte(s) in 2 object(s) allocated from:
    #0 0x7fd1c98fb4d0 in calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x55aa2f8fa1a1 in get_or_add_qgroup cmds/qgroup.c:822
    #2 0x55aa2f8fa7e9 in update_qgroup_info cmds/qgroup.c:883
    #3 0x55aa2f8fd912 in __qgroups_search cmds/qgroup.c:1385
    #4 0x55aa2f8fe196 in qgroups_search_all cmds/qgroup.c:1453
    #5 0x55aa2f902a7c in cmd_qgroup_clear_stale cmds/qgroup.c:2281
    #6 0x55aa2f73425b in cmd_execute cmds/commands.h:126
    #7 0x55aa2f734bcc in handle_command_group /home/runner/work/btrfs-progs/btrfs-progs/btrfs.c:177
    #8 0x55aa2f73425b in cmd_execute cmds/commands.h:126
    #9 0x55aa2f735a96 in main /home/runner/work/btrfs-progs/btrfs-progs/btrfs.c:518
    #10 0x7fd1c942a1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 08134323d00289185684a4cd177d202f39c2a5f3)
    #11 0x7fd1c942a28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 08134323d00289185684a4cd177d202f39c2a5f3)
    #12 0x55aa2f734144 in _start (/home/runner/work/btrfs-progs/btrfs-progs/btrfs+0x84144) (BuildId: 56f3dd838e1ae189c142c5d27fac025cd46deddb)

[CAUSE]
Above leaks are caused by two btrfs_qgroup structures and one path for
toplevel qgroup.

It's caused by the fact that we called qgroups_search_all() but didn't
do any cleanup.

[FIX]
Call __free_all_qgroups() inside cmd_qgroup_clear_stale() to properly
free the qgroups.

Fixes: 701ab151c2 ("btrfs-progs: qgroup: new command to delete stale qgroups")
Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:04:50 +02:00
David Sterba 8d0bfe4c01 btrfs-progs: use proper path buffer in __ino_to_path_fd()
Commit d7492ec59e ("btrfs-progs: use on-stack buffer in
__ino_to_path_fd") was supposed to switch path buffer from dynamic
allocation to on-stack but it was done wrong. The btrfs_data_container
is a flexible array so it needs to be explicitly allocated to the right
size.

The conversion turned it to an array. Gcc 13.x started to warn about
access to fspath->val[i] being out of bounds. Fortunately overall size
was 65536 and used only first 4096 bytes.

cmds/inspect.c: In function ‘__ino_to_path_fd’:
cmds/inspect.c:86:35: warning: array subscript i is outside array bounds of ‘__u64[]’ {aka ‘long long unsigned int[]’} [-Warray-bounds=]
   86 |                 ptr += fspath->val[i];
      |                        ~~~~~~~~~~~^~~
In file included from ./kernel-shared/accessors.h:11,
                 from cmds/inspect.c:35:
./kernel-shared/uapi/btrfs.h:724:17: note: while referencing ‘val’
  724 |         __u64   val[];          /* out */

Add an on-stack buffer and map it over fspath, similar to the previous
dynamic array.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 20:04:38 +02:00
Mark Harmstone e8c729743a btrfs-progs: mkfs: simplify mkfs_main() cleanup
mkfs_main() is a main-like function, meaning that return and exit are
equivalent. Deduplicate our cleanup code by moving the error label.

Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 20:04:38 +02:00
David Sterba 7e6e0417d1 libbtrfsutil: comment out fd_converter()
Recent changes to the python code reworked path_converter() so that it
does not use fd_converter() anymore. Assuming it may be used in the
future again comment it out.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 20:04:36 +02:00
Qu Wenruo d3cf350e21 btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert
Currently mkfs uses its own create_uuid_tree(), but that function is
only handling FS_TREE.  This means for btrfs-convert we do not generate
the uuid tree, nor add the UUID of the image subvolume.  This can be a
problem if we're going to support multiple subvolumes during mkfs time.

To address this, introduce a new helper, btrfs_rebuild_uuid_tree():

- Create a new uuid tree if there is not one

- Remove all the existing items from uuid tree

- Iterate through all subvolumes
  * If the subvolume has no valid UUID, regenerate one
  * Add the uuid entry for the subvolume UUID
  * If the subvolume has received UUID, also add it to UUID tree

By this, this new helper can handle all the uuid tree generation needs for:

- Current mkfs
  Only one uuid entry for FS_TREE

- Current btrfs-convert
  Only FS_TREE and the image subvolume

- Future multi-subvolume mkfs
  As we do the scan for all subvolumes.

- Future "btrfs rescue rebuild-uuid-tree"

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:02:56 +02:00
Qu Wenruo 99dc37bcfe btrfs-progs: cross-port btrfs_uuid_tree_add() from kernel
The modification is minimal:

- Replace WARN_ON() with UASSERT()

- Remove the @trans parameter for btrfs_extend_item() and
  btrfs_mark_buffer_dirty()
  As progs version doesn't need a transaction handler.

- Remove the btrfs_uuid_tree_add() in mkfs/main.c

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:02:42 +02:00
Qu Wenruo 8efa8092aa btrfs-progs: move uuid-tree definitions to kernel-shared/uuid-tree.h
Currently we already have a kernel-shared/uuid-tree.c, which is mostly
shared with kernel.

Kernel also has a uuid-tree.h, but we are still using ctree.h for the
header.

Move all the uuid-tree related definitions to kernel-shared/uuid-tree.h,
making future code sync easier.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:01:59 +02:00
Mark Harmstone 32ab0e6328 btrfs-progs: set transid in btrfs_insert_dir_item
btrfs_insert_dir_item wasn't setting the transid field in
btrfs_dir_item. Set it to the current transaction ID rather than writing
uninitialized memory to disk.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:01:38 +02:00
Omar Sandoval d0ab8f55a5 libbtrfsutil: python: fix build on Python 3.13
Python 3.13, currently in beta, removed the internal
_PyObject_LookupSpecial function. The libbtrfsutil Python bindings use
it in the path_converter() function because it was based on internal
path_converter() function in CPython [1]. This is causing build failures
on Fedora Rawhide [2] and Gentoo [3]. Replace path_converter() with a
version that only uses public functions based on the one in drgn [4].

1: d9efa45d74/Modules/posixmodule.c (L1253)
2: https://bugzilla.redhat.com/show_bug.cgi?id=2245650
3: https://github.com/kdave/btrfs-progs/issues/838
4: 9ad29fd864/libdrgn/python/util.c (L81)

Issue: #838
Reported-by: Neal Gompa <neal@gompa.dev>
Reported-by: Sam James <sam@gentoo.org>
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 20:01:38 +02:00
Qu Wenruo c5ee9301c6 btrfs-progs: tests: use feature output from "btrfs --version"
With the new feature description output in "btrfs --version" there is no
need to do the config.h hack to determine if we have certain feature.
This provides a more reliable way to detect features.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:01:05 +02:00
Qu Wenruo fdbab93ca5 btrfs-progs: tune csum-change: add leaf based threshold
For "btrfstune --csum", currently we do the following operations in just
one transaction for each:

- Delete old data csums
- Change new data csums objectid

Both operation can modify up to GiB or even TiB level of metadata, doing
them in just one transaction is definitely going to cause problems.

This patch adds a leaf number based threshold (32 leaves), after
modifying/deleting this many leaves, we commit a transaction to avoid
huge amount of dirty leaves piling up.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 20:00:13 +02:00
Qu Wenruo 6f99038a5f btrfs-progs: tests: add test case to verify convert rollback output
The new new test case is to make sure the rollback output for a fixed
content converted fs contains the string "ext2_saved/image".

As we have a bug in the past where after the string "ext2_saved", we can
have some unterminated garbage.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:59:46 +02:00
Qu Wenruo a927cb1b0a btrfs-progs: convert: fix the filename output when rolling back
[BUG]
When rolling back a converted btrfs, the filename output is corrupted:

  $ btrfs-convert -r  ~/test.img
  btrfs-convert from btrfs-progs v6.9.2

  Open filesystem for rollback:
    Label:
    UUID:            df54baf3-c91e-4956-96f9-99413a857576
    Restoring from:  ext2_saved0ƨy/image
                               ^^^ Corruption
  Rollback succeeded

[CAUSE]
The error is in how we handle the filename.  In btrfs all our strings
are not '\0' terminated, but with explicit length.

But in C, most strings are '\0' terminated, so after reading a filename
from btrfs, we need to manually terminate the string.

However the code adding the terminating '\0' looks like this:

	/* Get the filename length. */
	name_len = btrfs_root_ref_name_len(path.nodes[0], root_ref_item);

	/*
	 * This should not happen, but as an extra handling for possible
	 * corrupted btrfs.
	 */
	if (name_len > sizeof(dir_name))
		name_len = sizeof(dir_name) - 1;
	/* Got the real filename into our buffer. */
 	read_extent_buffer(path.nodes[0], dir_name, (unsigned long)(root_ref_item + 1), name_len);

	/* Terminate the string. */
	dir_name[sizeof(dir_name) - 1] = 0;

The problem is, the final termination is totally wrong, it always make
the last buffer char '\0', not using the @name_len we read before.

[FIX]
Use @name_len to terminate the string, as we have already updated it to
handle buffer overflow, it can handle both the regular and corrupted
case.

Fixes: dc29a5c51d ("btrfs-progs: convert: update default output")
Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:59:07 +02:00
Qu Wenruo fbdb8dfdee btrfs-progs: tests: add a test case for filename sanitization
This test case checks:

- If a regular btrfs-image dump has the unsanitized filenames
- If a sanitized btrfs-image dump has filenames properly censored

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:58:43 +02:00
Qu Wenruo 031282b85a btrfs-progs: tests: add a basic resume test using error injection
The new test case does:

- Make sure the build has error injection support
  This is done by checking "btrfs --version" output.

- Inject error at the last commit transaction of new data csum
  generation

- Resume the csum conversion and make sure it works

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:58:11 +02:00
Qu Wenruo a2c6c59333 btrfs-progs: image: fix the bug that filename sanitization not working
[BUG]
There is a bug report that image dump taken by "btrfs-image -s" doesn't
really sanitize the filenames:

  # truncates -s 1G source.raw
  # mkfs.btrfs -f source.raw
  # mount source.raw $mnt
  # touch $mnt/top_secret_filename
  # touch $mnt/secret_filename
  # umount $mnt
  # btrfs-image -s source.raw dump.img
  # string dump.img | grep filename
  top_secret_filename
  secret_filename
  top_secret_filename
  secret_filename
  top_secret_filename

[CAUSE]
Using above image to store the fs, and we got the following result in fs
tree:

	item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
		generation 3 transid 7 size 68 nbytes 16384
		block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0
		sequence 2 flags 0x0(none)
	item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12
		index 0 namelen 2 name: ..
	item 2 key (256 DIR_ITEM 439756795) itemoff 16062 itemsize 49
		location key (257 INODE_ITEM 0) type FILE
		transid 7 data_len 0 name_len 19
		name: top_secret_filename
	item 3 key (256 DIR_ITEM 693462946) itemoff 16017 itemsize 45
		location key (258 INODE_ITEM 0) type FILE
		transid 7 data_len 0 name_len 15
		name: secret_filename
	item 4 key (256 DIR_INDEX 2) itemoff 15968 itemsize 49
		location key (257 INODE_ITEM 0) type FILE
		transid 7 data_len 0 name_len 19
		name: top_secret_filename
	item 5 key (256 DIR_INDEX 3) itemoff 15923 itemsize 45
		location key (258 INODE_ITEM 0) type FILE
		transid 7 data_len 0 name_len 15
		name: secret_filename
	item 6 key (257 INODE_ITEM 0) itemoff 15763 itemsize 160
		generation 7 transid 7 size 0 nbytes 0
		block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0
		sequence 1 flags 0x0(none)
	item 7 key (257 INODE_REF 256) itemoff 15734 itemsize 29
		index 2 namelen 19 name: top_secret_filename
	item 8 key (258 INODE_ITEM 0) itemoff 15574 itemsize 160
		generation 7 transid 7 size 0 nbytes 0
		block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0
		sequence 1 flags 0x0(none)
	item 9 key (258 INODE_REF 256) itemoff 15549 itemsize 25
		index 3 namelen 15 name: 1���'�gc*&R

The result shows, only the last INODE_REF got sanitized, all the
remaining are not touched at all.

This is caused by how we sanitize the filenames:

 copy_buffer()
 |- memcpy(dst, src->data, src->len);
 |  This means we copy the whole eb into our buffer already.
 |
 |- zero_items()
    |- sanitize_name()
       |- eb = alloc_dummy_eb();
       |- memcpy(eb->data, src->data, src->len);
       |  This means we generate a dummy eb with the same contents of
       |  the source eb.
       |
       |- sanitize_dir_item();
       |  We override the dir item of the given item (specified by the
       |  slot number) inside our dummy eb.
       |
       |- memcpy(dst, eb->data, eb->lem);

The last one copy the dummy eb into our buffer, with only the slot
corrupted.

But when the whole work flow hits the next slot, we only corrupt the
next slot, but still copy the whole dummy eb back to buffer.

This means the previous slot would be overwritten by the old unsanitized
data.

Resulting only the last slot is corrupted.

[FIX]
Fix the bug by only copying back the corrupted item to the buffer.
So that other slots won't be overwritten by unsanitized data.

Reported-by: Andrea Gelmini <andrea.gelmini@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:57:06 +02:00
Qu Wenruo c58266504e btrfs-progs: docs: add warning for -s option of btrfs-image
The filename sanitization is not recommended as it introduces mismatches
between DIR_ITEM and INODE_REF.

Even hash collision mode (double "-s" option) is not ensured to always
find a hash collision, and when fails to find one, a mismatch happens.

And when a mismatch happens, the kernel will not resolve the path
correctly since kernel uses the hash from DIR_ITEM to lookup the child
inode.

So add a warning into the "-s" option of btrfs-image.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:56:41 +02:00
David Sterba 3461f5aa95 btrfs-progs: ci: add run-name for codespell workflow
Print the workflow name on the Actions page instead of the last commit
subject.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Yaroslav Halchenko 16a7cbca91 btrfs-progs: run codespell throughout fixing typos automagically
Spell checking can now run in automated mode.

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "codespell -w",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Yaroslav Halchenko 5b3bb3973a btrfs-progs: do interactive fixing of some ambigous typos
Typos fixed manually using the following:

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "codespell -w -i 3 -C 2",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Yaroslav Halchenko 1c710603b3 btrfs-progs: add codespell exceptions to ignore
Add exceptions that should not be reported as typos for a reason (names,
abbreviations, preferred other spelling).

Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Yaroslav Halchenko baedb21313 btrfs-progs: add rudimentary codespell config
Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Yaroslav Halchenko 17a37d3983 btrfs-progs: ci: add action to codespell devel on push and PRs
Run spellchecking on devel branch on push or on a pull-request.

Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Yaroslav Halchenko c4190ca330 btrfs-progs: ci: do apt-get update before trying to install
To ensure that package indexes are up to date.

That should help to avoid recent failed CI runs, which failed to install
certain packages as local cache is out-of-date and remote mirrors no
longer provide that specific (and out-of-date) version of package:

 E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/s/systemd/libudev-dev_255.4-1ubuntu8.1_amd64.deb  404  Not Found [IP: 52.147.219.192 80]

Signed-off-by: Yaroslav Halchenko <debian@onerussian.com>
[ Minor modification on the commit message. ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ Move cache update to a separate command. ]
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:56:08 +02:00
Qu Wenruo 9ad15a7301 btrfs-progs: use btrfs_link_subvolume() to replace btrfs_mksubvol()
The function btrfs_mksubvol() is very different between btrfs-progs and
kernel, the former version is really just linking a subvolume to another
directory inode, but the kernel version is really to make a completely
new subvolume.

Instead of same-named function, introduce btrfs_link_subvolume() and use
it to replace the old btrfs_mksubvol().

This is done by:

- Introduce btrfs_link_subvolume()
  Which does extra checks before doing any modification:
  * Make sure the target inode is a directory
  * Make sure no filename conflict

  Then do the linkage:
  * Add the dir_item/dir_index into the parent inode
  * Add the forward and backward root refs into tree root

- Introduce link_image_subvolume() helper
  Currently btrfs_mksubvol() has a dedicated convert filename retry
  behavior, which is unnecessary and should be done by the convert code.

  Now move the filename retry behavior into the helper.

- Remove btrfs_mksubvol()
  Since there is only one caller utilizing btrfs_mksubvol(), and it's
  now gone, we can remove the old btrfs_mksubvol().

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:54:50 +02:00
Qu Wenruo 3c555beabf btrfs-progs: introduce btrfs_make_subvolume()
There are two different subvolume/data reloc tree creation routines:

- create_subvol() from convert/main.c
  * calls btrfs_copy_root() to create an empty root
    This is not safe, as it relies on the source root to be empty.
  * calls btrfs_read_fs_root() to add it to the cache and trace it
    properly
  * calls btrfs_make_root_dir() to initialize the empty new root

- create_data_reloc_tree() from mkfs/main.c
  * calls btrfs_create_tree() to create an empty root
  * Manually add the root to fs_root cache
    This is only safe for data reloc tree as it's never updated
    inside btrfs-progs.
    But not safe for other subvolume trees.
  * manually setup the root dir

Both have their good and bad aspects, so here we introduce a new helper,
btrfs_make_subvolume():

- Calls btrfs_create_tree() to create an empty root
- Calls btrfs_read_fs_root() to setup the cache and tracking properly
- Calls btrfs_make_root_dir() to initialize the root dir
- Calls btrfs_update_root() to reflect the rootdir change

So this new helper can replace both create_subvol() and
create_data_reloc_tree().

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:54:04 +02:00
Qu Wenruo 9b74d80919 btrfs-progs: remove fs_info parameter from btrfs_create_tree()
The @fs_info parameter can be easily extracted from @trans, and kernel
has already remove the parameter.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2024-07-30 19:54:00 +02:00
David Sterba 64e272e74f btrfs-progs: inspect list-chunks: move it out of experimental
The list-chunk command is deemed to be reasonably complete so make it
visible in the default build. The output can be tweaked later.

Issue: #559
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba 4a1acc205a btrfs-progs: list-chunks: update help and documentation
Sync help text with current implementation (sorting, no usage nor empty
options).

Issue: #559
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba 44e190d25e btrfs-progs: tree-stats: unify setting raw/bytes options
The long options now allow to pass the unit mode in the usual way, drop
the local variable for raw byte values.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba 0eb8a65844 btrfs-progs: tree-stats: add options for size output units
Add the usual long options for all byte size related values in the
output.

Issue: #268
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba ef73193623 btrfs-progs: dump-tree: escape special characters in paths or xattrs
Filenames can contain a newline (or other funny characters), this makes
the dump-tree output confusing, same for xattr names or values that can
binary data.  Encode the special characters in the C-style ('\e' ->
"\e", or \NNN if there's no single letter representation). This is based
on the isprint() as it's espected either on a terminal or in a dump
file.

Issue: #350
Issue: #407
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba 3d2e879463 btrfs-progs: factor string escaping helpers from receive dump
The string escaping functionality is more generic and can be used in
other commands (e.g. in dump-tree). Move it to the string utils.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
Yaroslav Halchenko 1de113eb0c btrfs-progs: tests: fix filename typo in fsck/052 compex -> complex
Codespell discovered a typo in tests that's actually a bug in filename
that is supposed to be created and written to.

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "git-sedi compex complex",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

Pull-request: #846
Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba 5308564ca7 btrfs-progs: docs: document github PR workflow
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
Julien Olivain 9fdb8d7469 btrfs-progs: add uClibc-ng compatibility for printf format %pV
Commit [1] 164bc10d "btrfs-progs: add musl compatibility for printf
format %pV" added a logic to detect the presence of the glibc
<printf.h> header, and if present, to use the
register_printf_specifier() and register_printf_modifier() functions.

The original intent (as the commit log suggests), was to support the
musl libc, which does not provides this <printf.h> header.

When compiling with another libc, such as uClibc-ng, btrfs-progs fail
to build with error:

    common/messages.c: In function 'print_va_format':
    common/messages.c:51:19: error: 'const struct printf_info' has no member named 'user'
       51 |         if (!(info->user & va_modifier))
          |                   ^~
    common/messages.c: In function 'btrfs_no_printk':
    common/messages.c:76:17: warning: implicit declaration of function 'register_printf_specifier'; did you mean 'register_printf_function'? [-Wimplicit-function-declaration]
       76 |                 register_printf_specifier('V', print_va_format,
          |                 ^~~~~~~~~~~~~~~~~~~~~~~~~
          |                 register_printf_function
    common/messages.c:78:31: warning: implicit declaration of function 'register_printf_modifier'; did you mean 'register_printf_function'? [-Wimplicit-function-declaration]
       78 |                 va_modifier = register_printf_modifier(L"p");
          |                               ^~~~~~~~~~~~~~~~~~~~~~~~
          |                               register_printf_function

This is because uClibc-ng provides a <printf.h> header, but not the
register_printf_specifier() and register_printf_modifier() functions.
See [2]. It mainly includes register_printf_function(). uClibc-ng
emulates an older glibc behavior. Glibc added support for printf user
elements in commit [3] (first included in glibc-2.10, in 2009). Checking
only the <printf.h> is not sufficient.

This commit fixes this build issue by refining the detection logic of
the <printf.h> functions required by btrfs-progs.

[1] 164bc10dfc
[2] https://gogs.waldemar-brodkorb.de/oss/uclibc-ng/src/v1.0.49/include/printf.h
[3] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=9d26efa90c6dcbcd6b3e586c9927b6058ef4d529

Pull-request: #843
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
Julien Olivain 61e7ade17e btrfs-progs: kerncompat: fix fallthrough definition for gcc 5.x and 6.x.
Commit [1] 3a1d4aa089 "btrfs-progs: fix fallthrough cases with proper
attributes" introduced a macro "fallthrough" to better handle compiler
warnings of fallthrough situations.

This macro is defined using the "__has_attribute" built-in
function-like macro, which was introduced in GCC 5. See [2]. It then
test for the "__fallthrough__" attribute, which was introduced in
GCC 7. See [3].

When compiling with a gcc version which supports "__has_attribute" and
not the "__fallthrough__" attribute, compilation fails with error
message:

    common/format-output.c: In function 'print_escaped':
    common/format-output.c:78:4: error: 'fallthrough' undeclared (first use in this function)
        fallthrough;
        ^
btrfs-progs claim to support gcc at minimal version 4.8 in [4].

This commit fixes this issue by adding the missing definition.

The definition of the unsupported case is duplicated, because testing
for "__has_attribute" and an attribute at the same time is not
portable. See the cpp "__has_attribute" documentation [5].

Note: the issue was found with Buildroot Linux [6], while testing with
the command "utils/test-pkg -a -p btrfs-progs".

[1] 3a1d4aa089
[2] https://gcc.gnu.org/gcc-5/changes.html
[3] https://gcc.gnu.org/gcc-7/changes.html
[4] https://github.com/kdave/btrfs-progs/tree/v6.9.2#build-compatibility
[5] https://gcc.gnu.org/onlinedocs/cpp/_005f_005fhas_005fattribute.html
[6] https://buildroot.org/

Pull-request: #842
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba 023b20421c btrfs-progs: docs: add headings for 6.9 and 6.10 changes
Add headings for now, contents to be added later.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba f68dc323da btrfs-progs: docs: update 6.10 contribution graphs
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:33 +02:00
David Sterba a1945b28fb btrfs-progs: docs: add 6.10 kernel development statistics
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
David Sterba fb9c0feec8 btrfs-progs: ci: build html manual page previews if source changed
Similar to the manual page on terminal preview, also do the html output
that will be on the RTD page. Due to the way how it's displayed in the
CI action summary the CSS is missing and there are some visual
artifacts, e.g. in the option lists or special characters.

Issue: #824
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
Chung-Chiang Cheng ef8e050d64 btrfs-progs: inspect tree-stats: support to show a specified tree
tree-stats currently displays only some global trees and fs-tree 5. Add
support to show the stats of a specified tree.

Issue: #268
Signed-off-by: Chung-Chiang Cheng <cccheng@synology.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
David Sterba 2eadd15e6b btrfs-progs: ci: build manual page previews if source changed
Extend CI workflow of devel branch to generate manual page preview as it
would be rendered in a terminal. The output is in the workflow summary
page, for each file changed (if any).

Issue: #824
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
Ivan Kozik ec3c842859 btrfs-progs: scrub status: with --si, show rate in metric units
This makes btrfs scrub status --si show the Rate in metric units as well.

Before:

Total to scrub:   877.65GB
Rate:             609.22MiB/s

After:

Total to scrub:   877.65GB
Rate:             638.81MB/s

Pull-request: #832
Author: Ivan Kozik <ivan@ludios.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
David Sterba 5c32511a5c btrfs-progs: image: fix declaration of current_version
Gcc 13.3 reports:

In file included from image/main.c:41:
./image/metadump.h:47:1: warning: ‘extern’ is not at beginning of declaration [-Wold-style-declaration]
   47 | const extern struct dump_version *current_version;
      | ^~~~~

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
David Sterba 6954981f16 btrfs-progs: fix warnings reported by -Wold-style-definition
Gcc 13.3 reports:

check/mode-common.c: In function ‘reset_cached_block_groups’:
check/mode-common.c:590:6: warning: old-style function definition [-Wold-style-definition]
  590 | void reset_cached_block_groups()
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
David Sterba 2c3a2f1437 btrfs-progs: print optional features in btrfs version
There's a number of optionally built features, print the list on the 2nd
line of 'btrfs version'. The format is same as for e.g. systemd:

$ btrfs version
btrfs-progs v6.9.2
+EXPERIMENTAL -INJECT -STATIC +LZO +ZSTD +UDEV +FSVERITY +ZONED CRYPTO=builtin

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00
David Sterba 84841dc8b4 btrfs-progs: ci: promote fast CI image build to be the default one
Promote the fast workflow so it's in the place of ci-build-test.yml,
make it run for most branches.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-07-30 19:53:32 +02:00