Fix the following issues in the test suite:
- lack of quoting for variables
- declare function variables local when missing (prevent accidental
overwrite of global variables)
- for variables with underscore in the name use plain "$VAR_NAME"
instead of { } (unless necessary)
- minor style adjustments like moving quotes to the end of the same
string
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a long existing failure in my local VM that with any newer
kernel (6.x) the test case misc/004 would fail with ENOSPC during
balance:
[TEST] misc-tests.sh
[TEST/misc] 004-shrink-fs
failed: /home/adam/btrfs-progs/btrfs balance start -mconvert=single -sconvert=single -f /home/adam/btrfs-progs/tests/mnt
test failed for case 004-shrink-fs
make: *** [Makefile:547: test-misc] Error 1
[CAUSE]
With more testing, it turns out that just before the balance, the
filesystem still have several empty data block groups.
The reason is the new default discard=async behavior, as it also changes
the empty block groups to be async, this leave the empty block groups
there, resulting no extra space for the convert balance.
[FIX]
I do not understand why for loop block devices we also enable
discard, but at least disable discard for the test case so that we can
ensure the empty block groups get cleaned up properly.
Pull-request: #809
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case always fails in my VM, with the following error:
$ sudo TEST=038\* make test-misc
[TEST] misc-tests.sh
[TEST/misc] 038-backup-root-corruption
Backup 2 not overwritten
test failed for case 038-backup-root-corruption
After more debugging, it turns out that there is nothing wrong except
the final check:
[ "$main_root_ptr" -ne "$backup_new_root_ptr" ] || _fail "Backup 2 not overwritten"
The _fail() is only triggered if the previous check returns false, which
is completely the opposite.
Furthermore on the github CI, the kernel commits 2 instead of 1
transaction, resulting the next slot never to match the current
generation/tree root.
The two bugs combined, github CI always passses the test case, while
for my VM which does the expected one transaction, it would always fail.
Fix it by:
- Use a proper "if [] then; fi" block to check the tree root bytenr
- Use the generation diff to calculate the expected backup root slot
- Log the full super block dump for debug usage
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Executing the script inside the directories as './test.sh' is not
supposed to work but could happen accidentally. With an exit after
attempting to source the we can fix that. Not all cases have been fixed
in f6bbe06c08 ("btrfs-progs: tests: add protection against running
out of test suite").
Signed-off-by: David Sterba <dsterba@suse.com>
The test case verifies behavior of ext4 unwritten extents:
- Create a unwritten (preallocated) extent on ext4
- Fill the on-disk extent with random garbage
This is to make sure if btrfs tries to read the on-disk data, it would
definitely get some garbage.
As I found sometimes mkfs.ext4 can fill the unused bg with zeros.
- Fill the preallocated file range with some data
This is to make sure btrfs-convert can handle mixed written and
unwritten ranges.
- Save the checksum of the file.
- Convert the fs
- Verify the checksum
For older btrfs-convert, there would be only one regular file extent,
and reading the file would read out some garbage and cause checksum to
mismatch.
For the fixed btrfs-convert, we punch holes for unwritten extents,
thus only the written part would be read out and match the checksum.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With the change of minimal number of zones, mkfs-tests/030-zoned-rst now
fails because the loopback device is 2GB and can contain 8x 256MB zones.
Use the nullb helpers to choose a smaller zone size.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test for mkfs.btrfs zone reset behavior to check if
- it resets all the zones without "-b" option
- it detects an active zone outside of the FS range
- it do not reset a zone outside of the range
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
After patch "btrfs-progs: qgroup: handle stale qgroup deletion more
accurately" cleaning stale qgroups may not happen if the subvolume
cleaning is still in progress. Update the test so it's able to handle
that.
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case would contain a file system image, with the following
file:
item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160
generation 7 transid 8 size 16384 nbytes 16384
block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0
sequence 258 flags 0x0(none)
atime 1714635006.328482575 (2024-05-02 17:00:06)
ctime 1714635013.394980640 (2024-05-02 17:00:13)
mtime 1714635013.394980640 (2024-05-02 17:00:13)
otime 1714635006.328482575 (2024-05-02 17:00:06)
item 5 key (257 INODE_REF 256) itemoff 15869 itemsize 14
index 2 namelen 4 name: file
item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
generation 7 type 1 (regular)
extent data disk byte 13631488 nr 1048576
extent data offset 0 nr 16384 ram 16384
extent compression 0 (none)
Note the ram bytes, which should be 1048576.
Furthermore, the inode size is truncated to 16K (originally 1M), so that
offset + num_bytes would still be no larger than ram_bytes.
So the only error is the mismatch between ram_bytes and disk_num_bytes
for the non-compressed data extent.
The image is hand crafted for now, as btrfs-corrupt-block doesn't not yet
support corrupting the ram_bytes field.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case relies on `--delete-qgroup` option which has been removed.
The feature needs to be implemented in kernel as it's more complex than
just calling an ioctl. The test case does not take the complexity of
subvolume dropping into consideration and only tested the simplest
cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new test case to make sure:
- btrfstune can convert a zoned btrfs with extent tree to bgt
- btrfstune can convert a zoned btrfs with bgt back to extent tree
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
There are now two tests with 060, we'd like to be that a unique number,
rename the one that was added more recently.
Signed-off-by: David Sterba <dsterba@suse.com>
The raid-stripe-tree can be enabled for convert, though it's still
considered incomplete and slightly experimental. Due to that the tests
need to be adjusted to check for support and skip mount eventually.
Possible remaining options to add: quota, squota
Issue: #694
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance the conversion macro to recognize the filesystem type in case
the ntfs-specific converter utility has to be used. Its current feature
set does not match btrfs-convert.
Signed-off-by: David Sterba <dsterba@suse.com>
The test does not pass in environments where /bin/sh is not bash (like
github CI). Unify the initial setup like the other tests do: source,
compatibility, prerequisities, the rest.
Signed-off-by: David Sterba <dsterba@suse.com>
The strict check is enabled for both check_image() and
convert_test_do_convert() to detect chunk alignment related problems.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case would check if "btrfs subvolume create":
- Report error on an existing path
- Still report error if mulitple paths are given and one of them already
exists
- For above case, still created a subvolume for the good parameter
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
https://www.openssl.org/ Is a well known cryptography library and since
freshly released version 3.2 it also supports variable digest size of
blake2b, so we can now add it among the crypto providers.
Configure with --with-crypto=openssl.
Signed-off-by: David Sterba <dsterba@suse.com>
https://botan.randombit.net/ Botan is a cryptography library with C
bindings and provides what we need (sha256 and blake2b), among many
others. Add it to the list of crypto backends if somebody wants to use
it.
Currently the version 2.19 is the latest one. Botan3 3.2.0 exists but
does not seem to be widely available in distros yet.
Configure with --with-crypto=botan.
Signed-off-by: David Sterba <dsterba@suse.com>
The test case would verify the following behaviors:
- Partial failure
Should return 1, but the remaining valid destinations would still be
created.
- All success
That's as usual.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With the kernel commit 070bb0011ccf ("btrfs: sysfs: show if ACL
support has been compiled in") we can now check if ACL is compiled
without requiring a btrfs device. Retain older method for older
kernels.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The zone size may not be parsed correctly in 'nullb create' due to
copy&paste error from the 'size'. This is already fixed upstream.
Signed-off-by: David Sterba <dsterba@suse.com>
Since a few days the CI started to fail randomly when there were loop
devices used in the tests. The mount fails because some device is
reported to be missing:
$ losetup --show --find
/dev/loop3
...
$ mkfs ...
ERROR: device scan failed on '/dev/loop3': No such file or directory
...
$ mount
mount: /home/runner/work/btrfs-progs/btrfs-progs/tests/mnt: wrong fs
type, bad option, bad superblock on /dev/loop3, missing codepage or
helper program, or other error.
$ dmesg
...
BTRFS error (device loop0): devid 3 uuid 11d9c345-9527-433e-a024-7102659fa0ee is missing
BTRFS error (device loop0): failed to read the system array: -2
BTRFS error (device loop0): open_ctree failed
This was reproducible in the "cli" tests, but also happened on a local
machine.
To fix that wait for all loop devices before mount, the command
'btrfs device ready' should block until that. The convenience helper
does that, for any standalone 'mount' used with loop devices this must
be done manually.
Signed-off-by: David Sterba <dsterba@suse.com>
The test names 011 and 012 are the same but 012 is actually testing
recursion of the 'btrfs fi du' command.
Signed-off-by: David Sterba <dsterba@suse.com>
There was a support for a short syntax of 'btrfs balance' that accepted
a path where normally would be the mandatory subcommand. This was a
heuristic and nowadays everybody should be using the
'btrfs balance action' syntax. The warning was in place for a year, it's
time to remove the short syntax completely.
Issue: #517
Signed-off-by: David Sterba <dsterba@suse.com>
The most common sector sizes are 4k, 16k and 64k, we don't really need
to test that with convert, this takes a long time for little benefit as
the node size is only for metadata, while the rest remains the same.
Signed-off-by: David Sterba <dsterba@suse.com>
The ntfs2btrfs tool recently found a bug in 'check', add the conversion
test support to our testsuite. It's optional and depends on kernel
support of the 'ntfs3' driver and the availability of mkntfs and
ntfs2btrfs, which might not be available everywhere.
Signed-off-by: David Sterba <dsterba@suse.com>
Minor refactoring, check arguments and print parameters. This can be
used to stress btree allocation by changing run_size.
Signed-off-by: David Sterba <dsterba@suse.com>
There's an old test for btree code that could be used in the testsuite.
Still needs some polishing and review what tests make sense.
Signed-off-by: David Sterba <dsterba@suse.com>
Previously the XFS-specific code was commented out so we don't need the
headers for building fsstress, this changes how the utility behaves
compared to the one in fstests, e.g. randomness or additional open/close
operations. Add enough code from xfsprogs to make it compile and enable
the #if0-ed code.
Signed-off-by: David Sterba <dsterba@suse.com>
Issue #622 reported a case where ntfs2btrfs can generate out-of-order
inline backref items, which can lead to kernel transaction abort, but
not detected by btrfs-check.
This patch would add such image, whose extent tree looks like this for
the only data extent:
item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16172 itemsize 111
refs 3 gen 7 flags DATA
(178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
(178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
(178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1
While the original good base image has the following backrefs for the
same data extent:
item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16172 itemsize 111
refs 3 gen 7 flags DATA
(178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
(178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
(178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1
Notice the sequence (the 2nd number in the round brackets) should be
descending. (Meanwhile type should be ascending, but it's way harder to
create.)
For now we don't have a way to fix it, but at least we should detect it.
Issue: #622
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case would:
- Prepare two loopback devices
One is for storing the source directory (on a btrfs).
This is to ensure we can set xattr for the directory, as filesystems
like tmpfs (mostly utilized by mktemp) is not supporting xattr.
The other one is the real target fs where we call
"mkfs.btrfs --rootdir" on.
- Create the source directory with the following contents:
* rootdir inode attributs:
# mode (750)
# uid (1000)
# gid (1000)
# xattr (user.rootdir)
* one regular file, with attributes:
# xattr (user.foorbar)
- Execute "mkfs.btrfs --rootdir" and mount the new fs
- Verify the above attributes
The target fs should have the same attributes, especially for the
rootdir inode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The convert tests take a long time and are not necessary for quick test
rounds under the convenience target 'make test'. We still run the tests
in CI.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba
Qu Wenruo
Naohiro Aota
Anand Jain