The current print-tree can not handle unsupported inode flags, e.g.
created by Synology's out-of-tree btrfs implementation.
The existing one just checks all the supported flags, and if no flag
hits, it will output "none" no matter if there is any unsupported one.
Fix this by implementing sprint_readable_flag(), and use the same
handling of print_readable_flag().
Although for inode flag, adds one extra handling to output "none" if no
flag hit at all.
Signed-off-by: Qu Wenruo <wqu@suse.com>
This includes:
- Remove the "__" prefix
Now the "__" is no longer recommended, and there is no function taking
the "print_readable_flag" in the first place.
- Move the supported flags calculation into print_readable_flag()
Since all callers are doing the same work before calling the function.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Added DEV_ITEM object id to the reserved object id list. It's historical reason
to let both of DEV_ITEM and ROOT_TREE have same object id. Developers should
be aware of it.
Signed-off-by: HAN Yuwei <hrx@bupt.moe>
[ Replace immediate number with key names ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
This "stil" -> "still" typo is causing the latest CI spellchecks to fail.
Fix that so we can get a good CI run.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Fixes false positive in btrfs check that was causing btrfs/192 to fail.
It looks like there's circumstances in which btrfs_log_changed_extents
can also log unmodified extents, but not their csums. If that happens,
check the main csum root as well before showing an error.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Use the correct address and size when looking for the csums for a
compressed extent in the tree log.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Fix a false positive in btrfs check, where we were returning an error
because an explicit hole in the log tree had no associated csum entry.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
The slides for the talk "Kernel maintainership: an oral tradition",
linked to in the documentation, seem to have gone from the Linux
Foundation website. Change to the version on bootlin.com.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
On systems with glibc 2.34 and 2.39, the following warning appears when
building the binary:
[CC] common/help.o
common/help.c: In function ‘usage’:
common/help.c:315:58: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
315 | fprintf(outf, "No short description for '%s'\n", token);
| ^~
common/help.c:312:46: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
312 | fprintf(outf, "No usage for '%s'\n", token);
| ^~
This happens for usage() which passes NULL pointer as token. Normally
this is fine, as fprintf() will output "(null)" for the NULL pointer,
but it's still not ideal.
Fix the warning by changing the token to "" if it's NULL.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Adds tests to mkfs-tests/036-rootdir-subvol for the modifiers to
mkfs.btrfs --subvol: ro, rw, default, and default-ro.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Reworks mkfs.btrfs --subvol so that dir and full_path in struct
rootdir_subvol are stored as arrays rather than pointers.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Change mkfs.btrfs --subvol so that instead of being of the form --subvol
DIR:FLAGS, it's instead --subvol MODIFIER:DIR, with MODIFIER being ro,
rw, default, or ro-default.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
With --chroot, the receive subcommand unconditionally sent a non-error
status message to stderr, e.g.:
$ btrfs --quiet receive --chroot /some/path
Chroot to /some/path
Signed-off-by: Sebastian Hamann <code@ares-macrotechnology.com>
Instead of copying the file during custom build commands, just use a
soft link to re-use the existing README.d from libbtrfsutil.
Issue: #310
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
Currently with python3.12, the python bindding will always result the
following warning:
[PY] libbtrfsutil
/usr/lib/python3.12/site-packages/setuptools/_distutils/extension.py:134: UserWarning: Unknown Extension options: 'headers'
warnings.warn(msg)
[CAUSE]
In the setup.py which specifies the files to be included into the package,
we use setuptools::Extension to specify the file lists and include paths.
But there is no handling of Extension::headers member, thus resulting the
above warning.
[FIX]
According to the docs of setuptools, MANIFEST.in is the file controlling
what files should be included.
So instead of the non-supported headers, use MANIFEST.in to include the
needed headers.
Signed-off-by: Qu Wenruo <wqu@suse.com>
This introduces two new cases:
- 3 hardlinks without any subvolume
This should results 3 hard links inside the btrfs.
- 3 hardlinks, but a subvolume will split 2 of them
Then the 2 inside the same subvolume should still report 2 nlinks,
but the lone one inside the new subvolume can only report 1 nlink.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The new hard link detection and creation support is done by maintaining
an rb tree with the following members:
- st_ino, st_dev
This is to record the stat() report from the host fs.
With this two, we can detect if it's really a hard link (st_dev
determines one filesystem/subvolume, and st_ino determines the inode
number inside the fs).
- root
This is btrfs root pointer. This a special requirement for the recent
introduced "--subvol" option.
As we can have the following corner case:
rootdir/
|- foobar_hardlink1
|- foobar_hardlink2
|- subv/ <- To be a subvolume inside btrfs
|- foobar_hardlink3
In above case, on the host fs, `subv/` directory is just a regular
directory, but in the new btrfs it will be a subvolume.
In that case, `foobar_hardlink3` cannot be created as a hard link,
but a new inode.
- st_nlink and found_nlink
Records the original reported number of links, and the nlinks we
created inside btrfs.
This is recorded in case we created all hard links and can remove
the entry early.
- btrfs_ino
This is the inode number inside btrfs.
And since we can handle hard links safely, remove all the related
warnings, and add a new note for `--subvol` option, warning about the
case where we need to split hard links due to subvolume boundary.
Pull-request: #873
Signed-off-by: Qu Wenruo <wqu@suse.com>
Remove functions that after the previous two patches are no longer
referenced.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Co-authored-by: Omar Sandoval <osandov@fb.com>
Call btrfs_util_subvolume_snapshot in cmd_subvolume_snapshot rather than
calling the ioctl directly.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Co-authored-by: Omar Sandoval <osandov@fb.com>
Call btrfs_util_subvolume_create in create_one_subvolume rather than
calling the ioctl directly.
Pull-request: #878
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Co-authored-by: Omar Sandoval <osandov@fb.com>
Change --subvol that it can accept flags, and add a "default" flag that
allows you to mark a subvolume as the default.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Add a new option --subvol, which tells mkfs.btrfs to create the
specified directories as subvolumes when used with --rootdir.
Given a populated directory dir, the command
$ mkfs.btrfs --rootdir dir --subvol usr --subvol home --subvol home/username img
will create subvolumes 'usr' and 'home' within the toplevel subvolume,
and subvolume 'username' within the 'home' subvolume. It will fail if
any of the directories do not yet exist.
Pull-request: #868
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new option --recursive 'btrfs subvol delete', causing it to pass the
BTRFS_UTIL_DELETE_SUBVOLUME_RECURSIVE flag through to libbtrfsutil.
This can work in two modes, depending on the user:
- regular user - this will skip subvolumes that are not accessible
- root (CAP_SYS_ADMIN) - no limitations
Pull-request: #861
Signed-off-by: Mark Harmstone <maharmstone@meta.com>
Co-authored-by: Omar Sandoval <osandov@osandov.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
[ Add details to man page, fix indent in the doc. ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Explain the difference, in case somebody want's to use it as a source to
correct that on Wikipedia.
[ ci skip ]
Signed-off-by: David Sterba <dsterba@suse.com>
When searching the extent tree for the target extent item, we can miss it
if the extent item is the first item in a leaf and if there is a previous
leaf in the extent tree.
For example, if we call btrfs-map-logical like this:
$ btrfs-map-logical -l 5382144 /dev/sdc
And we have the following extent tree layout:
leaf 5386240 items 26 free space 2505 generation 7 owner EXTENT_TREE
leaf 5386240 flags 0x1(WRITTEN) backref revision 1
(...)
item 25 key (5373952 METADATA_ITEM 0) itemoff 3155 itemsize 33
refs 1 gen 7 flags TREE_BLOCK
tree block skinny level 0
(176 0x5) tree block backref root FS_TREE
leaf 5480448 items 56 free space 276 generation 7 owner EXTENT_TREE
leaf 5480448 flags 0x1(WRITTEN) backref revision 1
(...)
item 0 key (5382144 METADATA_ITEM 0) itemoff 3962 itemsize 33
refs 1 gen 7 flags TREE_BLOCK
tree block skinny level 0
(176 0x7) tree block backref root CSUM_TREE
(...)
Then the following happens:
1) We enter map_one_extent() with search_forward == 0 and
*logical_ret == 5382144;
2) We search for the key (5382144 0 0) which leaves us with a path
pointing to leaf 5386240 at slot 26 - one slot beyond the last item;
3) We then call:
btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0])
Which is not valid since there's no item at that slot, but since the
area of the leaf where an item at that slot should be is zeroed out,
we end up getting a key of (0 0 0);
4) We then enter the "if" statement bellow, since key.type is 0, and call
btrfs_previous_extent_item(), which leaves at slot 25 of leaf 5386240,
point to the extent item of the extent 5373952.
The requested extent, 5382144, is the first item of the next leaf
(5480448), but we totally miss it;
5) We return to the caller, the main() function, with 'cur_logical'
pointing to the metadata extent at 5373952, and not to the requested
one at 5382144.
In the last while loop of main() we have 'cur_logical' == 5373952,
which makes the loop have no iterations and therefore the local
variable 'found' remains with a value of 0, and then the program fails
like this:
$ btrfs-map-logical -l 5382144 /dev/sdc
ERROR: no extent found at range [5382144,5386240)
Fix this by never accessing beyond the last slot of a leaf. If we ever end
up at a slot beyond the last item in a leaf, just call btrfs_next_leaf()
and process the first item in the returned path.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The inode_cache and involved on-disk formats are deprecated and will
have no effect since v5.11 kernel.
And in v6.11 kernel, new tree-checker will even reject data extents
belonging to those deprecated inode cache.
Lowmem check can detect such deprecated inode cache from the beginning.
This images are generated by 5.10 LTS kernels with inode cache.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
There are reports about deprecated inode cache causing newer kernels to
rejecting them.
Such inode cache is rarely utilized and already fully deprecated since
v5.11, and newer kernel will reject data extents of inode cache since
v6.11.
But original mode btrfs check won't detect nor report them as error.
Meanwhile lowmem mode can properly detect and report them:
ERROR: root 5 INODE[18446744073709551604] nlink(1) not equal to inode_refs(0)
ERROR: invalid imode mode bits: 00
ERROR: invalid inode generation 18446744073709551604 or transid 1 for ino 18446744073709551605, expect [0, 72)
ERROR: root 5 INODE[18446744073709551605] is orphan item
Since those inode cache paid no attention to properly maintain all the
numbers, they are easy targets for more recent lowmem mode.
[CAUSE]
For original mode, it has extra hardcoded hacks to avoid nlink checks
for inode cache inode.
Furthermore original mode doesn't check the mode bits nor its
generation.
[FIX]
For original mode, remove the hack for inode cache so that the
deprecated inode cache can be reported as an error.
For both modes, add extra global message to direct the affected users to
use 'btrfs rescue clear-ino-cache' to clear the deprecated cache.
Pull-request: #891
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
There is one report about `btrfs rescue clear-ino-cache` failed with
tree block level mismatch:
# btrfs rescue clear-ino-cache /dev/mapper/rootext
Successfully cleaned up ino cache for root id: 5
Successfully cleaned up ino cache for root id: 257
Successfully cleaned up ino cache for root id: 258
corrupt node: root=7 block=647369064448 slot=0, invalid level for leaf, have 1 expect 0
node 647369064448 level 1 items 252 free space 241 generation 6065173 owner CSUM_TREE
node 647369064448 flags 0x1(WRITTEN) backref revision 1
fs uuid e6614f01-6f56-4776-8b0a-c260089c35e7
chunk uuid f665f535-4cfd-49e0-8be9-7f94bf59b75d
key (EXTENT_CSUM EXTENT_CSUM 3714473984) block 677126111232 gen 6065002
[...]
key (EXTENT_CSUM EXTENT_CSUM 6192357376) block 646396493824 gen 6065032
ERROR: failed to clear ino cache: Input/output error
[CAUSE]
During `btrfs rescue clear-ino-cache`, btrfs-progs will iterate through
all the subvolumes, and clear the inode cache inode from each subvolume.
The problem is in how we iterate the subvolumes.
We hold a path of tree root, and go modifiy the fs for each found
subvolume, then call btrfs_next_item().
This is not safe, because the path to tree root is not longer reliable
if we modified the fs.
So the btrfs_next_item() call will fail because the fs is modified
halfway, resulting the above problem.
[FIX]
Instead of holding a path to a subvolume root item, and modify the fs
halfway, here introduce a helper, find_next_root(), to locate the root
item whose objectid >= our target rootid, and return the found item key.
The path to root tree is only hold then released inside
find_next_root().
By this, we won't hold any unrelated path while modifying the
filesystem.
And since we're here, also adding back the missing new line when all ino
cache is cleared.
Pull-request: #890
Reported-by: Archange <archange@archlinux.org>
Link: https://lore.kernel.org/linux-btrfs/4803f696-2dc5-4987-a353-fce1272e93e7@archlinux.org/
Signed-off-by: Qu Wenruo <wqu@suse.com>
There is an internal report that, during btrfs-convert to block-group
tree, by accident some systemd events triggered the mount of the target
fs.
This leads to double mount (one by kernel and one by the btrfs-progs),
which seems to cause quite some problems.
To avoid such accident, exclusively opens all devices if btrfs-progs is
doing write operations.
Pull-request: #888
Reported-by: pandada8 <pandada8@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
btrfs-ioctl.rst was laid out like it should be a man page, including
having a section number, but it wasn't getting installed because there
was not enough content.
Pull-request: #892
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case will:
- Create a symbolic which contains a 4095 bytes sized target on ext4
- Convert the ext4 to btrfs
- Make sure we can still read the symbolic link
For unpatched btrfs-convert, the resulted symbolic link will be rejected
by kernel and fail.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
There is a recent bug that btrfs/012 fails and kernel rejects to read a
symbolic link which is backed by a regular extent.
Furthremore in that case, "btrfs check --mode=lowmem" doesn't detect such
problem at all.
[CAUSE]
For symbolic links, we only allow inline extents, and this means we should
only have a symbolic link target which is smaller than 4K.
But lowmem mode btrfs check doesn't handle symbolic link inodes any
differently, thus it doesn't check if the file extents are inlined or not,
nor reporting this problem as an error.
[FIX]
When processing data extents, if we find the owning inode is a symbolic
link, and the file extent is regular/preallocated, report an error for
the bad file extent item.
Signed-off-by: Qu Wenruo <wqu@suse.com>
symbolic links
[BUG]
There is a recent bug that btrfs/012 fails and kernel rejects to read a
symbolic link which is backed by a regular extent.
Furthremore in that case, "btrfs check" doesn't detect such problem at
all.
[CAUSE]
For symbolic links, we only allow inline file extents, and this means we
should only have a symbolic link target which is smaller than 4K.
But btrfs check doesn't handle symbolic link inodes any differently, thus
it doesn't check if the file extents are inlined or not, nor reporting
this problem as an error.
[FIX]
When processing data extents, if we find the owning inode is a symbolic
link, and the file extent is regular/preallocated, mark the inode with
I_ERR_FILE_EXTENT_TOO_LARGE error.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
Sometimes test case btrfs/012 fails randomly, with the failure to read a
symlink:
QA output created by 012
Checking converted btrfs against the original one:
-OK
+readlink: Structure needs cleaning
Checking saved ext2 image against the original one:
OK
Furthermore, this will trigger a kernel error message:
BTRFS critical (device dm-2): regular/prealloc extent found for non-regular inode 133081
[CAUSE]
For that specific inode 133081, the tree dump looks like this:
item 127 key (133081 INODE_ITEM 0) itemoff 40984 itemsize 160
generation 1 transid 1 size 4095 nbytes 4096
block group 0 mode 120777 links 1 uid 0 gid 0 rdev 0
sequence 0 flags 0x0(none)
item 128 key (133081 INODE_REF 133080) itemoff 40972 itemsize 12
index 2 namelen 2 name: l3
item 129 key (133081 EXTENT_DATA 0) itemoff 40919 itemsize 53
generation 4 type 1 (regular)
extent data disk byte 2147483648 nr 38080512
extent data offset 37974016 nr 4096 ram 38080512
extent compression 0 (none)
Note that, the symlink inode size is 4095 at the max size (PATH_MAX,
removing the terminating NUL).
But the nbytes is 4096, exactly matching the sector size of the btrfs.
Thus it results the creation of a regular extent, but for btrfs we do
not accept a symlink with a regular/preallocated extent, thus kernel
rejects such read and failed the readlink call.
The root cause is in the convert code, where for symlinks we always
create a data extent with its size + 1, causing the above problem.
I guess the original code is to handle the terminating NUL, but in btrfs
we never need to store the terminating NUL for inline extents nor
file names.
Thus this pitfall in btrfs-convert leads to the above invalid data
extent and fail the test case.
[FIX]
- Fix the ext2 and reiserfs symbolic link creation code
To remove the terminating NUL.
- Add extra checks for the size of a symbolic link
Btrfs has extra limits on the size of a symbolic link, as btrfs must
store symbolic link targets as inlined extents.
This means for 4K node sized btrfs, the size limit is smaller than the
usual PATH_MAX - 1 (only around 4000 bytes instead of 4095).
So for certain nodesize, some filesystems can not be converted to
btrfs.
(this should be rare, because the default nodesize is 16K already)
- Split the symbolic link and inline data extent size checks
For symbolic links the real limit is PATH_MAX - 1 (removing the
terminating NUL), but for inline data extents the limit is
sectorsize - 1, which can be different from 4096 - 1 (e.g. 64K sector
size).
Pull-request: #884
Signed-off-by: Qu Wenruo <wqu@suse.com>
