Pre-release build checks are not yet done in the CI, so add a workflow
for that and only for the release-test branch as this usually does not
break and can take a long time due to the number of options (even with
ccache).
Signed-off-by: David Sterba <dsterba@suse.com>
Update the build test script with LZO optional build and add combination
of all existing options with the experimental build (this effectively
doubles the number of build runs).
Signed-off-by: David Sterba <dsterba@suse.com>
The unified naming API of libbtrfsutil is exported in the library, we
should use it in the internal code as well. In the future the old naming
scheme will be deprecated and remove.
Signed-off-by: David Sterba <dsterba@suse.com>
Create a source for --rootdir and then use it for mkfs with compression.
Try a few levels, nothing special.
Signed-off-by: David Sterba <dsterba@suse.com>
The templates need to be renamed, so make it more suggestive by
prepending the expected name of the script test.sh. Also fix the
permissions and make it 755 so it's not missed later as this would not
execute the script.
Signed-off-by: David Sterba <dsterba@suse.com>
When simple quotas is enabled, every new data extent gets a special
inline OWNER_REF item that identifies the owning subvolume. This makes
simple quotas backwards incompatible with kernels older than v6.7. Even
if you disable quotas on the filesystem, the OWNER_REF items are
sprinkled throughout the extent tree and older kernels are unable to
parse them.
However, it is relatively easy to simply walk the extent tree and remove
these inline ref items. This gives squota adopters the option to *fully*
disable squotas on their system and un-set the incompat bit. Add this
capability to btrfstune, which requires only a little tricky btrfs item
data shifting.
This functionality was tested with a new unit test, as well as a similar
but more thorough integration test in fstests
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Adds tests to mkfs-tests/036-rootdir-subvol for the modifiers to
mkfs.btrfs --subvol: ro, rw, default, and default-ro.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
This introduces two new cases:
- 3 hardlinks without any subvolume
This should results 3 hard links inside the btrfs.
- 3 hardlinks, but a subvolume will split 2 of them
Then the 2 inside the same subvolume should still report 2 nlinks,
but the lone one inside the new subvolume can only report 1 nlink.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Add a new option --subvol, which tells mkfs.btrfs to create the
specified directories as subvolumes when used with --rootdir.
Given a populated directory dir, the command
$ mkfs.btrfs --rootdir dir --subvol usr --subvol home --subvol home/username img
will create subvolumes 'usr' and 'home' within the toplevel subvolume,
and subvolume 'username' within the 'home' subvolume. It will fail if
any of the directories do not yet exist.
Pull-request: #868
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The inode_cache and involved on-disk formats are deprecated and will
have no effect since v5.11 kernel.
And in v6.11 kernel, new tree-checker will even reject data extents
belonging to those deprecated inode cache.
Lowmem check can detect such deprecated inode cache from the beginning.
This images are generated by 5.10 LTS kernels with inode cache.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The new test case will:
- Create a symbolic which contains a 4095 bytes sized target on ext4
- Convert the ext4 to btrfs
- Make sure we can still read the symbolic link
For unpatched btrfs-convert, the resulted symbolic link will be rejected
by kernel and fail.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Currently the transaction log is more or less ignored by btrfs check,
meaning that it's possible for a FS with a corrupt log to pass btrfs
check, but be immediately corrupted by the kernel when it's mounted.
Adds a check that if there's an inode in the log, any pending
non-inlined csumed writes also have corresponding csum entries.
Pull-request: #879
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
[ Small commit message update. ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
There's a report that newly added --rootdir print too many warnings for
hardlinks, which is maybe not that uncommon. We still want to let the
user know about that so print it just once and count how many were
found:
$ mkfs.btrfs --rootdir ...
WARNING: '/tmp/btrfs-progs-mkfs-rootdir-hardlinks.7RcdfR/rootdir/inside_link' has extra hardlinks, they will be converted into new inodes
WARNING: 12 hardlinks were detected in /tmp/btrfs-progs-mkfs-rootdir-hardlinks.7RcdfR/rootdir, all converted to new inodes
Link: https://github.com/kdave/btrfs-progs/pull/872#issuecomment-2289096125
Signed-off-by: David Sterba <dsterba@suse.com>
process_clone() only searches the received_uuid, but could exist in an
earlier uuid that isn't the received_uuid. Mirror what process_snapshot
does and search both the received_uuid and if that fails look up by
normal uuid.
Fixes: https://github.com/kdave/btrfs-progs/issues/606
Issue: #606
Pull-request: #643
Pull-request: #862
Signed-off-by: Arsenii Skvortsov <ettavolt@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
None of the public API uses the rb-tree code besides definitions, so
change the includes in ctree.h and drop rbtree.h, this is used only by
internal implementation in send-utils.c. We could remove it in the
future but last time it was not possible due to 3rd party code depending
on it.
Removed in 83ab92512e ("libbtrfs: remove the support for fs without
uuid tree") and reverted again in f9b0da8e78 ("Revert "libbtrfs:
remove the support for fs without uuid tree"")
Signed-off-by: David Sterba <dsterba@suse.com>
The list_head is used in struct definitions but otherwise not at all as
it was copied from kernel code. For ctree.h add stub definition that
won't change the containing structure size.
Drop list.h from libbtrfs. This may break some builds if they used the
header, though this was never meant to be exported.
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case creates a special layout like this:
rootdir/ (fs1 ino=256)
|- dir1/ (fs1 ino=257)
| |- dir1/ (fs2 ino=257)
| |- dir2/ (fs2 ino=258)
| |- file1 (fs2 ino=259)
| |- file2 (fs2 ino=260)
|- dir2/ (fs1 ino=258)
|- file1 (fs1 ino=259)
|- file2 (fs2 ino=259)
This layout intentionally creates inode number conflicts, which will
make the old "mkfs.btrfs --rootdir" to fail.
But newer reworked one will successfully handle them, just leave a test
case to avoid to hit the old bugs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The test case will create the following directory layout:
.
|- rootdir/
| |- inside_link
|- outside_link
Both inside_link and outside_link are hard links of each other.
And use rootdir/ as the rootdir for mkfs.
This is to ensure the nlink of inside_link is correctly set to 1.
Inspired by the recent rework which fixes the handling of hard links.
Signed-off-by: Qu Wenruo <wqu@suse.com>
With the new feature description output in "btrfs --version" there is no
need to do the config.h hack to determine if we have certain feature.
This provides a more reliable way to detect features.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The new new test case is to make sure the rollback output for a fixed
content converted fs contains the string "ext2_saved/image".
As we have a bug in the past where after the string "ext2_saved", we can
have some unterminated garbage.
Signed-off-by: Qu Wenruo <wqu@suse.com>
This test case checks:
- If a regular btrfs-image dump has the unsanitized filenames
- If a sanitized btrfs-image dump has filenames properly censored
Signed-off-by: Qu Wenruo <wqu@suse.com>
The new test case does:
- Make sure the build has error injection support
This is done by checking "btrfs --version" output.
- Inject error at the last commit transaction of new data csum
generation
- Resume the csum conversion and make sure it works
Signed-off-by: Qu Wenruo <wqu@suse.com>
Codespell discovered a typo in tests that's actually a bug in filename
that is supposed to be created and written to.
=== Do not change lines below ===
{
"chain": [],
"cmd": "git-sedi compex complex",
"exit": 0,
"extra_inputs": [],
"inputs": [],
"outputs": [],
"pwd": "."
}
^^^ Do not change lines above ^^^
Pull-request: #846
Author: Yaroslav Halchenko <debian@onerussian.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch reverts the following 3 commits:
d0cc40d23a ("btrfs-progs: tests: add test case for ram_bytes detection and repair")
7313573c19 ("btrfs-progs: check: original, detect and repair ram_bytes mismatch")
97bf7a5969 ("btrfs-progs: check: lowmem, detect and repair mismatched ram_bytes")
The problem with the ram_bytes check is, kernel can handle it without
any problem, and the original objective for this is to detect such
problem as I immaturelly believe the problem is fixed.
But it turns out to be incorrect and this check is already causing
problems.
Fix it by doing a full revert for now.
Pull-request: #828
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Free memory after errors in sum(), this is reported by gcc 13 on the CI.
This was reproduced by:
$ make D=asan TEST=019-receive-clones-on-mounted-subvol test-misc
Signed-off-by: David Sterba <dsterba@suse.com>
Test the following:
- Create a btrfs with crc32c csum
- Populate the filesystem
- Convert the filesystem to the following csums:
* xxhash
* blake2
* sha256
* crc32c
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This test case will check the following behviour:
- Regular zoned supported data/metadata profiles
Should success
- RST with zoned feature
Should fail for non-experimental builds
- zoned with data profiles that only RST supports
Should fail for non-experimental builds.
Meanwhile for experimental builds it should success and enable RST
feature automatically (verified in mkfs/030)
Pull-request: #813
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently we unconditionally run mkfs/029 and mkfs/030 as we export
RST feature through mkfs.btrfs as supported features.
But considering the mkfs.btrfs features don't match kernel support (only
a BTRFS_DEBUG kernel can support that), we're going to hide RST behind
experimental builds.
In that case RST related tests cases also need to be behind experimental
builds as regular builds of mkfs.btrfs will no longer support RST
feature.
Introduce two helpers:
- _test_config()
To verify if certain config is set to 1
- check_experimental_build()
A wrapper of "_test_config EXPERIMENTAL", and skip the test if
btrfs-progs has no experimental features.
So that test cases can be skipped for non-experimental builds.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The raid-stripe-tree (RST) feature is for zoned devices to support extra
data profiles, and is not yet a stable feature (still requires
CONFIG_BTRFS_DEBUG enabled kernel to support it).
Furthermore the supported filesystems (ext*, reiserfs and ntfs) don't
even support zoned devices, and even we force RST support for
btrfs-convert, we would only create an empty tree for RST, as btrfs
convert would only result SINGLE data profile with SINGLE/DUP metadata
profile, neither needs RST at all.
Enabling RST for btrfs-convert would only cause problems for false test
failures as we incorrectly allow RST feature for btrfs-convert.
Fixes the problem by removing raid-stripe-tree support from
btrfs-convert and remove the test cases support for RST.
This patch is mostly reverting commit 346a381923 ("btrfs-progs:
convert: add raid-stripe-tree to allowed features"), but keeps the test
infrastructure to support bgt features for convert.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a random failure for misc/028 in github CI, but unable to
reproduce locally.
[EXTRA DUMP]
The failure is at mount time, which is mostly out of our control (kernel
is provided by the CI image), but at least we can do extra kernel dump
if a mount fails.
This dump is done inside run_check() where if we detects there is a
"mount" word inside the command string.
The detection is not the best, it may detect other commands that
contains the word "mount", but it should be good enough to detect real
mount failure.
Pull-request: #815
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a random chance that misc/055 would fail with stale qgroups
detected.
[CAUSE]
Commit 82f7d6c1d7 ("btrfs-progs: qgroup: handle stale qgroup deletion
more accurately") changed the behavior of "btrfs qgroup clear-stale"
that it will no longer try to remove qgroup under deletion.
And the test case itself relies on the return value of "btrfs qgroup
clear-stale" to do the extra wait for subvolume deletion.
This means after that commit, the test case would skip the wait if there
is a subvolume waiting for cleanup, resulting in a race window where the
subvolume can be dropped and become stale, and eventually trigger the
stale qgroup detection and cause false alerts.
[FIX]
Fix the test case by always wait for the subvolume to be dropped, so
that later "btrfs qgroup clear-stale" can properly remove all stale
qgroups.
Pull-request: #814
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test check/037-freespacetree-repair sometimes fails (does not in the
CI though) that the last unmount does not work due to -EBUSY. This
depends on timing, but what is obviously wrong is the 'rm' before
umount. The quoting of "*" does not remove all the files before
fallocate but is otherwise silent due to '-f'.
Add the run_check so it's recorded in the logs taht it happens and alos
add sudo so it can actually work. This makes the test work more
reliably.
Signed-off-by: David Sterba <dsterba@suse.com>
Fix the following issues in the test suite:
- lack of quoting for variables
- declare function variables local when missing (prevent accidental
overwrite of global variables)
- for variables with underscore in the name use plain "$VAR_NAME"
instead of { } (unless necessary)
- minor style adjustments like moving quotes to the end of the same
string
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a long existing failure in my local VM that with any newer
kernel (6.x) the test case misc/004 would fail with ENOSPC during
balance:
[TEST] misc-tests.sh
[TEST/misc] 004-shrink-fs
failed: /home/adam/btrfs-progs/btrfs balance start -mconvert=single -sconvert=single -f /home/adam/btrfs-progs/tests/mnt
test failed for case 004-shrink-fs
make: *** [Makefile:547: test-misc] Error 1
[CAUSE]
With more testing, it turns out that just before the balance, the
filesystem still have several empty data block groups.
The reason is the new default discard=async behavior, as it also changes
the empty block groups to be async, this leave the empty block groups
there, resulting no extra space for the convert balance.
[FIX]
I do not understand why for loop block devices we also enable
discard, but at least disable discard for the test case so that we can
ensure the empty block groups get cleaned up properly.
Pull-request: #809
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
