When simple quotas is enabled, every new data extent gets a special
inline OWNER_REF item that identifies the owning subvolume. This makes
simple quotas backwards incompatible with kernels older than v6.7. Even
if you disable quotas on the filesystem, the OWNER_REF items are
sprinkled throughout the extent tree and older kernels are unable to
parse them.
However, it is relatively easy to simply walk the extent tree and remove
these inline ref items. This gives squota adopters the option to *fully*
disable squotas on their system and un-set the incompat bit. Add this
capability to btrfstune, which requires only a little tricky btrfs item
data shifting.
This functionality was tested with a new unit test, as well as a similar
but more thorough integration test in fstests
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
process_clone() only searches the received_uuid, but could exist in an
earlier uuid that isn't the received_uuid. Mirror what process_snapshot
does and search both the received_uuid and if that fails look up by
normal uuid.
Fixes: https://github.com/kdave/btrfs-progs/issues/606
Issue: #606
Pull-request: #643
Pull-request: #862
Signed-off-by: Arsenii Skvortsov <ettavolt@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This test case checks:
- If a regular btrfs-image dump has the unsanitized filenames
- If a sanitized btrfs-image dump has filenames properly censored
Signed-off-by: Qu Wenruo <wqu@suse.com>
The new test case does:
- Make sure the build has error injection support
This is done by checking "btrfs --version" output.
- Inject error at the last commit transaction of new data csum
generation
- Resume the csum conversion and make sure it works
Signed-off-by: Qu Wenruo <wqu@suse.com>
Test the following:
- Create a btrfs with crc32c csum
- Populate the filesystem
- Convert the filesystem to the following csums:
* xxhash
* blake2
* sha256
* crc32c
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a random chance that misc/055 would fail with stale qgroups
detected.
[CAUSE]
Commit 82f7d6c1d7 ("btrfs-progs: qgroup: handle stale qgroup deletion
more accurately") changed the behavior of "btrfs qgroup clear-stale"
that it will no longer try to remove qgroup under deletion.
And the test case itself relies on the return value of "btrfs qgroup
clear-stale" to do the extra wait for subvolume deletion.
This means after that commit, the test case would skip the wait if there
is a subvolume waiting for cleanup, resulting in a race window where the
subvolume can be dropped and become stale, and eventually trigger the
stale qgroup detection and cause false alerts.
[FIX]
Fix the test case by always wait for the subvolume to be dropped, so
that later "btrfs qgroup clear-stale" can properly remove all stale
qgroups.
Pull-request: #814
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Fix the following issues in the test suite:
- lack of quoting for variables
- declare function variables local when missing (prevent accidental
overwrite of global variables)
- for variables with underscore in the name use plain "$VAR_NAME"
instead of { } (unless necessary)
- minor style adjustments like moving quotes to the end of the same
string
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a long existing failure in my local VM that with any newer
kernel (6.x) the test case misc/004 would fail with ENOSPC during
balance:
[TEST] misc-tests.sh
[TEST/misc] 004-shrink-fs
failed: /home/adam/btrfs-progs/btrfs balance start -mconvert=single -sconvert=single -f /home/adam/btrfs-progs/tests/mnt
test failed for case 004-shrink-fs
make: *** [Makefile:547: test-misc] Error 1
[CAUSE]
With more testing, it turns out that just before the balance, the
filesystem still have several empty data block groups.
The reason is the new default discard=async behavior, as it also changes
the empty block groups to be async, this leave the empty block groups
there, resulting no extra space for the convert balance.
[FIX]
I do not understand why for loop block devices we also enable
discard, but at least disable discard for the test case so that we can
ensure the empty block groups get cleaned up properly.
Pull-request: #809
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case always fails in my VM, with the following error:
$ sudo TEST=038\* make test-misc
[TEST] misc-tests.sh
[TEST/misc] 038-backup-root-corruption
Backup 2 not overwritten
test failed for case 038-backup-root-corruption
After more debugging, it turns out that there is nothing wrong except
the final check:
[ "$main_root_ptr" -ne "$backup_new_root_ptr" ] || _fail "Backup 2 not overwritten"
The _fail() is only triggered if the previous check returns false, which
is completely the opposite.
Furthermore on the github CI, the kernel commits 2 instead of 1
transaction, resulting the next slot never to match the current
generation/tree root.
The two bugs combined, github CI always passses the test case, while
for my VM which does the expected one transaction, it would always fail.
Fix it by:
- Use a proper "if [] then; fi" block to check the tree root bytenr
- Use the generation diff to calculate the expected backup root slot
- Log the full super block dump for debug usage
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Executing the script inside the directories as './test.sh' is not
supposed to work but could happen accidentally. With an exit after
attempting to source the we can fix that. Not all cases have been fixed
in f6bbe06c08 ("btrfs-progs: tests: add protection against running
out of test suite").
Signed-off-by: David Sterba <dsterba@suse.com>
After patch "btrfs-progs: qgroup: handle stale qgroup deletion more
accurately" cleaning stale qgroups may not happen if the subvolume
cleaning is still in progress. Update the test so it's able to handle
that.
Signed-off-by: David Sterba <dsterba@suse.com>
The test case relies on `--delete-qgroup` option which has been removed.
The feature needs to be implemented in kernel as it's more complex than
just calling an ioctl. The test case does not take the complexity of
subvolume dropping into consideration and only tested the simplest
cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new test case to make sure:
- btrfstune can convert a zoned btrfs with extent tree to bgt
- btrfstune can convert a zoned btrfs with bgt back to extent tree
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are now two tests with 060, we'd like to be that a unique number,
rename the one that was added more recently.
Signed-off-by: David Sterba <dsterba@suse.com>
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since a few days the CI started to fail randomly when there were loop
devices used in the tests. The mount fails because some device is
reported to be missing:
$ losetup --show --find
/dev/loop3
...
$ mkfs ...
ERROR: device scan failed on '/dev/loop3': No such file or directory
...
$ mount
mount: /home/runner/work/btrfs-progs/btrfs-progs/tests/mnt: wrong fs
type, bad option, bad superblock on /dev/loop3, missing codepage or
helper program, or other error.
$ dmesg
...
BTRFS error (device loop0): devid 3 uuid 11d9c345-9527-433e-a024-7102659fa0ee is missing
BTRFS error (device loop0): failed to read the system array: -2
BTRFS error (device loop0): open_ctree failed
This was reproducible in the "cli" tests, but also happened on a local
machine.
To fix that wait for all loop devices before mount, the command
'btrfs device ready' should block until that. The convenience helper
does that, for any standalone 'mount' used with loop devices this must
be done manually.
Signed-off-by: David Sterba <dsterba@suse.com>
Create subvolumes and delete every N-th to generate output where we can
also see how the stale qgroups are placed (at the end).
Issue: #687
Signed-off-by: David Sterba <dsterba@suse.com>
The kernel patch, ("btrfs: reject device with CHANGING_FSID_V2 flag"),
removes kernel support for the CHANGING_FSID_V2 flag. So, drop its
related testcase.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The misc-test/034-metadata_uuid test case, has four sets of disk images
to simulate failed writes during btrfstune -m|M operations. As of now,
this tests kernel only. Update the test case to verify btrfstune -m|M's
functionality to recover from the same scenarios.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add appropriate prefix to the error messages to make it easier to track
down which case failed.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Previous commit "btrfs-progs: dump-super: print actual metadata_uuid
value" changed the value of the super_block::metadata_uuid to be printed
as it is, without tweaking it depending on the METADATA_UUID flag.
Apply similar tweak in the common helper functions used to read the
metadata_uuid so that test-cases still be successful.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Test case misc/046-seed-multi-mount would always fail with the following
error:
[TEST] misc-tests.sh
[TEST/misc] 046-seed-multi-mount
unexpected success: writable file despite read-only mount
test failed for case 046-seed-multi-mount
[CAUSE]
Although mounting seed device is indeed read-only, sprouting it with a
new device would always make it read-write by itself.
The behavior is already there for a long time, thus expecting a new
behavior (not changing the read-only flag) is a little weird.
[FIX]
Instead of doing the write check after the sprout, do it before the
sprout.
This looks more correct, and would not rely on the kernel behavior
change (if we determine to go that path).
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When I was testing misc/058, the fs still has around 7GiB free space,
but during that test case, btrfs kernel module reports write failures
and even git commands failed inside that fs.
And obviously the test case failed.
[CAUSE]
It turns out that, the test case itself would require 6GiB (4 data
disks) + 1.5GiB x 2 (the two replace target), thus it requires 9 GiB
free space.
And obviously my partition is not that large and failed.
[FIX]
In fact, we really don't need that much space at all.
The test verifies that two consecutive replace operations can be started
and enqueued, the sleep of 1 second is not strictly necessary as the
first command should start the replace right away.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[PROBLEM]
Since we have migrated to default v2 cache, the test case
misc/030-missing-device-image is no longer executed:
[TEST/misc] 030-missing-device-image
[NOTRUN] unable to create v1 space cache
[CAUSE]
The test case itself is trying its best to cover all paths, including
the data extent read path.
Thus the test case is requiring v1 cache, as that's the only way to
cover the data read path.
[FIX]
Just remove the v1 space cache requirement, it's still better to run the
test even it only exercises the metadata read path.
The good news is, after commit 3ff9d35257 ("btrfs-progs: use
read_data_from_disk() to replace read_extent_from_disk() and replace
read_extent_data()"), all data/metadata read paths are unified.
They only difference is the verification part.
Thus even if we didn't fully exercise the data read path, we didn't lose
much coverage anyway.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test misc/058 does not properly filter out the output due to -s that
only ignores non-existent files and was there due to previous changes.
We need to use -q.
Signed-off-by: David Sterba <dsterba@suse.com>
Fix failures caused by the lack of ACL support in btrfs. For example:
$ make test
::
[TEST/misc] 057-btrfstune-free-space-tree
failed: setfacl -m u:root:x /Volumes/ws/btrfs-progs/tests/mnt/acls/acls.1
test failed for case 057-btrfstune-free-space-tree
make: *** [Makefile:493: test-misc] Error 1
Similar failures occurred in the test cases convert/001-ext2-basic,
convert/003-ext4-basic, convert/005-delete-all-rollback, and
convert/006-large-hole-extent.
Resolve it by adding a check for ACL support using the
check_kernel_support_acl() helper function. It gracefully handles the case
when ACL support is not compiled by calling _not_run().
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case would create a fs without free space tree, then
populate it, convert to free-space-tree feature, and make sure
everything is fine.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Executing the script inside the directories as './test.sh' is not
supposed to work but could happen accidentally. With an exit after
attempting to source the we can fix that.
Signed-off-by: David Sterba <dsterba@suse.com>
There are some helpers that invoke setup_root_helper internally so it's
not needed to run test.sh without root, but it should be there in case
the test calls SUDO_HELPER so it's always paired.
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Test case misc/034 can fail like this:
====== RUN CHECK mount /dev/loop16 /home/adam/btrfs-progs/tests/mnt
mount: /home/adam/btrfs-progs/tests/mnt: wrong fs type, bad option, bad superblock on /dev/loop16, missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
failed: mount /dev/loop16 /home/adam/btrfs-progs/tests/mnt
And the dmesg looks like this:
loop16: detected capacity change from 0 to 1024000
loop17: detected capacity change from 0 to 1024000
BTRFS: device fsid 593e23af-a7e6-4360-b16a-229f415de697 devid 1 transid 6 /dev/loop16 scanned by mount (79348)
BTRFS info (device loop16): using crc32c (crc32c-intel) checksum algorithm
BTRFS info (device loop16): found metadata UUID change in progress flag, clearing
BTRFS info (device loop16): disk space caching is enabled
BTRFS error (device loop16): devid 2 uuid cde07de6-db7e-4b34-909e-d3db6e7c0b06 is missing
BTRFS error (device loop16): failed to read the system array: -2
BTRFS error (device loop16): open_ctree failed
[CAUSE]
From the dmesg, it shows that although both loopback devices are
properly registered, only one is properly scanned by mount.
Thus the other device is missing, and without "-o degraded" the
filesystem failed to be mounted.
[FIX]
Before we mount the filesystem, also scan them in their passed order
to properly assemble the device list for mount.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case will execute "btrfs subvolume list -u" on the newly
create btrfs.
Since the v0 root item is already deprecated for a long time, newly
created btrfs should be already using the new root item, thus "btrfs
subvolume list -u" should always report the correct uuid.
The test case relies on external program "uuidparse" which should be
provided by util-linux.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The send stream v2 is supported if the file exists and does not contain
"1". The previous fix reversed the condition but this does not work on
kernel with v1 only support.
Signed-off-by: David Sterba <dsterba@suse.com>
We want to notrun if this test fails, not if it succeeds. Additionally
we want -s, as -q will still print an error if it gets ENOENT from the
file we're trying to grep.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To be able to run the test suite on NFS the temporary files need to be
writeable for all, root due to send and owner due to the way it's
created.
Signed-off-by: David Sterba <dsterba@suse.com>
Attempting to dump a bad btrfs superblock returns successful exit status
zero. According to the manual page non-zero should be returned on
failure. Fix this.
$ btrfs inspect-internal dump-super /dev/zero
superblock: bytenr=65536, device=/dev/zero
---------------------------------------------------------
ERROR: bad magic on superblock on /dev/zero at 65536
$ echo $?
0
Signed-off-by: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adapt the existing send/receive tests by passing '-o compress-force' to
the mount commands in a new test. After writing a few files in the
various compression formats, send/receive them with and without
--force-decompress to test both the encoded_write path and the fallback
to decode+write.
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case will have a image file which has dirty log
(btrfs-image supports dumping log tree).
So we can easily check if "btrfstune -S" will reject fs with dirty log.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There's a report that a read-only subvolume with a received_uuid set
emits the warning in command 'btrfs subvolume show', which is obviously
wrong.
The reason is that there are different types of root item flags,
depending on how we read them. The check in cmd_subvol_show uses the
ioctl GET_SUBVOL_INFO and the appropriate flag is raw
BTRFS_ROOT_SUBVOL_RDONLY (0x1), while there's another SUBVOL_GETFLAGS that
maps the flags and the raw value is different (BTRFS_SUBVOL_RDONLY, 0x2).
Due to this the warning was issued. Fix that by using the right flag
constant. The test has been extended to check for all combinations of
read-write and received_uuid.
Issue: #419
Signed-off-by: David Sterba <dsterba@suse.com>
Use the helpers to create temporary files, this save some typing and
we'll have a bit more consistent naming.
Signed-off-by: David Sterba <dsterba@suse.com>
The file names are build from roughly these components:
- btrfs-progs as prefix
- category (mkfs, convert) or what's the type of the file like 'image'
- the substitution template, XXXXXX
Signed-off-by: David Sterba <dsterba@suse.com>
Test case misc/038 uses hard coded backup slot number, this means if we
change how many transactions we commit during mkfs, it will immediately
break the tests.
Such hard coded tests will be a big pain for later btrfs-progs updates.
Update it with runtime backup slot search.
Such search is done by using current filesystem generation as a search
target and grab the slot number.
By this, no matter how many transactions we commit during mkfs, the test
case should be able to handle it.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a slightly more convenient way to identify the subvolumes with bad
combination of flags and received uuid.
Signed-off-by: David Sterba <dsterba@suse.com>
