Patch "btrfs-progs: fix false alert on tree block crossing 64K page
boundary" fixes a false alert (warning) when 'btrfs check' is called
right after mkfs.
As we have an extensive mkfs coverage in test mkfs/001, add the check for
the warning there instead of a separate test.
Signed-off-by: David Sterba <dsterba@suse.com>
Some tests report that decompressing the image failed, which did not
fail the test but could lead to wrong errors in case the image is not
overwritten and leaves some old state. Use --force parameter.
[TEST] fuzz-tests.sh
[TEST/fuzz] 001-simple-check-unmounted
xz: btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw: File exists
failed to decompress image btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw.xz
[TEST/fuzz] 002-simple-image
xz: btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw: File exists
failed to decompress image btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw.xz
Signed-off-by: David Sterba <dsterba@suse.com>
The current mount detection code in btrfs receive is not quite perfect.
For example, suppose /tmp is mounted as a tmpfs. In that case,
btrfs receive /tmp2 will find /tmp as the longest mount that matches a
prefix of /tmp2 and blow up because it is not a btrfs filesystem, even
if /tmp2 is just a directory in / mounted as btrfs.
Fix this by replacing the substring check with a dirname recursion to
only check the directories in the path of the dir, rather than every
substring.
Add a new test for this case.
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new function, check_test_results(), for
misc/fsck/convert/mkfs test cases.
This function is currently to catch warning message for subpage support,
but can be later expanded for other usages.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a test case which ensures that when resize is tried on an image
instead of a directory appropriate warning is produced and the command
fails.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When the test environment is in 'docker', there's some delay before the
device mapper nodes appear in /dev/mapper. Add a delay before the test
continues, usually 1 second was enough but give it more just in case.
Add ad fallback to skip the test if the device node does not show up,
this is a problem in the running environment and the testsuite should
continue.
Signed-off-by: David Sterba <dsterba@suse.com>
Add some randomization to the long device name in case the testsuite
runs multiple times on the same host.
Signed-off-by: David Sterba <dsterba@suse.com>
In some environments the which utility might not be available and the
shell builtin 'type -p' is readily available.
Signed-off-by: David Sterba <dsterba@suse.com>
Testing the statically built binaries is not straightforward, add a
convenient way to do that:
$ make TEST_FLAVOR=static
There should be no difference in the test results.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a simple framework to exercise the json formatter and add testing
target that validates the output.
Run 'make test-json' to execute all available tests, requires 'jq'
utility for validation (https://github.com/stedolan/jq).
Signed-off-by: David Sterba <dsterba@suse.com>
Add a Makefile so tests can be run the same way from the standalone
testsuite as they are from git. The build dependencies are not checked
and the default path is for the system binaries.
Because of the path auto detection, running 'make' from the tests/
directory now works the same way as from the toplevel git directory.
Signed-off-by: David Sterba <dsterba@suse.com>
Pre-created image contains a subvolume and a snapshot so that cleaning
of multiple roots is also tested. The mount option 'inode_cache' will be
removed so we need the crafted image.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will:
- Add a new test image for fsck/044
This new image has a corrupted extent item generation for tree block.
This image can expose a bug in original mode, which can't detect the
problem.
This image also utilize the tree block generation detection code,
which the existing image doesn't.
- Rename the existing image
To reflect the fact that the existing one is only for data extent.
- Remove the test.sh
So that the generic path will test both detection and repair.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Subvolumes that are part of send must not be deleted, we can check that
only by the error code EPERM, otherwise it's also in the system log.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a runtime feature (-R) flag for the free space tree. A filesystem
that is mkfs'd with -R free-space-tree then mounted with no options has
the same contents as one mkfs'd without the option, then mounted with
'-o space_cache=v2'.
The only tricky thing is in exactly how to call the tree creation code.
Using btrfs_create_free_space_tree as is did not quite work, because an
extra reference to the eb (root->commit_root) is leaked, which mkfs
complains about with a warning. I opted to follow how the uuid tree is
created by adding it to the dirty roots list for cleanup by
commit_tree_roots in commit_transaction. As a result,
btrfs_create_free_space_tree no longer exactly matches the version in
the kernel sources.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
The test fails inside travis, check for a multiple-profile detection.
This was verified to work on a proper host so the workaround is in place
let the whole suite pass.
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case will test whether btrfs-convert can handle 64G ext*
fs.
This exercise the cctx->total_bytes calculation where multiplying 4K
(unsigned int) and 16777216 (u32) could lead to bit overflow for
unsigned int and got 0, and screw up later free space calculation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Kernel patch ff51bf02d107 ("btrfs: block-group: fix free-space bitmap
threshold") is needed to exercise both branches of the test, this can be
detected by lack of the objectid and offset parsed from the dump.
Issue: #288
Signed-off-by: David Sterba <dsterba@suse.com>
For workarounds or known missing support add helper to print a
notification about that, when _not_run is not suitable.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a test case to check if the converted fs has device extent beyond
boundary.
The disk layout of source ext4 fs needs some extents to make them
allocated at the very end of the fs. The script is from the original
reporter.
Also, since the existing convert tests always uses 512M as device size,
which is not suitable for this test case, make it to grab the existing
device size to co-operate with this test case.
Reported-by: Jiachen YANG <farseerfc@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a test case to ensure we can create a 350M fs with 128M rootdir.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image used in test case 035 has one problem:
- Bad total_bytes of device item in chunk tree
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
devid 1 total_bytes 72564736 bytes_used 92274688
The total_bytes is even smaller than bytes_used.
While the dev_item in super block contains the correct value:
dev_item.total_bytes 134217728
dev_item.bytes_used 92274688
This patch will update the image, using the correct value in super block
to replace the wrong one in chunk tree.
So that later btrfs-image restore without device/chunk fixup can still
pass fsck-tests.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image used in 012 has several problems:
- Bad dev_item total_bytes in superblock
dev_item.total_bytes 20918272
dev_item.bytes_used 67108864
The fs has 64M chunks allocated, and the original fs is also 64M.
Thus the total_bytes is completely wrong.
- Bad total_bytes in chunk tree
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 3897 itemsize 98
devid 1 total_bytes 20918272 bytes_used 67108864
The same problem as super block.
This problem is covered by btrfs-image, as it will recalculate the
device size.
However if btrfs-image is doing exactly same data replay without any
modification, such bad image would cause test failure, as we're unable
to fix such bad total_bytes.
This patch will update the image to fix such problem.
Also since we're here, there is no need to use custom .tar.gz image, as
we have extra_image() to handle the uncompression.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Basic test of mkfs-time enabled quotas and filesystem filled with files.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image has a csum item overlap with next csum item, make sure btrfs
check can detect it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance the clean-tests.sh script to detach all loop devices created by
tests, lookup is based on the path prefix. Now 'make test-clean' is
supposed to prepare the environment for next test round.
Signed-off-by: David Sterba <dsterba@suse.com>
Use simpler output format for easier parsing and place each block group
type on a separate line.
Example output:
WARNING: Multiple block group profiles detected, see 'man btrfs(5)'.
WARNING: Data: single, raid1
Signed-off-by: David Sterba <dsterba@suse.com>
Several commands now warn about multiple profiles as it is considered a
potentially problematic filesystem state that might need user attention.
- device delete
- device add
- balance pause
- balance cancel
- filesystem usage
WARNING: Multiple profiles detected. See 'man btrfs(5)'.
WARNING: data -> [raid1, single]
Signed-off-by: David Sterba <dsterba@suse.com>
[PROBLEM]
We want to inject $INSTRUMENT (mostly valgrind) before btrfs command but
after root_helper.
Currently we won't inject $INSTRUMENT at all if we are using
root_helper.
This means the coverage is not good enough.
[FIX]
This patch introduce a new function, expand_command(), to handle all
parameter/argument injection, including existing 'btrfs check' inject.
This function will:
- Detect where to inject $INSTRUMENT
If we have root_helper and the command is target command
(btrfs/mkfs.btrfs/btrfs-convert), then we inject $INSTRUMENT after
root_helper.
If we don't have root_helper, and the command is target command,
we inject $INSTRUMENT before the command.
Or we don't inject $INSTRUMENT (it's not the target command).
- Use existing spec facility to inject extra arguments
- Use an array to restore to result
To avoid bash interpret the IFS inside path/commands.
Now we can make sure no matter if we use root_helper, $INSTRUMENT is
always injected corrected.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since run_check_stdout() can insert INSTRUMENT for all btrfs related
programs, which could easily pollute the stdout, any caller of
run_check_stdout() should do proper filter.
The following callers are affected:
- misc/004
Filter the output of "btrfs ins min-dev-size"
- misc/009
- misc/013
- misc/024
They are all calling "btrfs ins rootid", so introduce get_subvolid()
function to grab the subvolid properly.
- misc/031
Loose the filter for "btrfs qgroup show". No need for "tail -n 1".
So we still have the same coverage, but now these tests won't cause
false alert if we insert INSTRUMENT for all btrfs commands.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As the TEST_TOP is often only a base path, the slash gets doubled. This
is not a problem as the slashes are in the middle of the path and cannot
be confused with the initial "//" that gets interpreted as a network
path. Update all the test drivers.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Commit 31f477ee ("btrfs-progs: mkfs-tests: skip test if truncate fails
with EFBIG") tried to detect a failure in truncate command by checking
the $? expecting it to be an errno, when it actually returns 0 or 1.
To fix this test just check if the command failed (returned 1) and look
for the output, skipping the test if the OS cannot create a 6E file.
Fixes: #241
Tested-by: Erhard Furtner <erhard_f@mailbox.org>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
With INSTRUMENT=valgrind set, some fsck tests will fail, e.g. fsck/013:
====== RUN CHECK mount -t btrfs -o loop /home/adam/btrfs/btrfs-progs/tests//test.img /home/adam/btrfs/btrfs-progs/tests//mnt
==114106==
==114106== Warning: Can't execute setuid/setgid/setcap executable: /usr/bin/mount
==114106== Possible workaround: remove --trace-children=yes, if in effect
==114106==
valgrind: /usr/bin/mount: Permission denied
failed: mount -t btrfs -o loop /home/adam/btrfs/btrfs-progs/tests//test.img /home/adam/btrfs/btrfs-progs/tests//mnt
test failed for case 013-extent-tree-rebuild
[CAUSE]
Just as stated by valgrind itself, it can't handle program with
setuid/setgid/setcap.
Thankfully in our case it's mount and we don't really care about it at
all.
[FIX]
Although we could use complex skip pattern to skip mount in valgrind, we
don't really want to run valgrind on mount or sudo command anyway.
So here we do extra check if we're running mount command. And if that's
the case, just skip $INSTRUMENT command.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Cleanups done separately:
* use the default test image, loop devices not needed for the test
* trim TEST_MNT from all paths
* send output is created inside the test filesystem
Signed-off-by: David Sterba <dsterba@suse.com>
This test case is the reproducer for the previous fix.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Skip the test 013-subvolume-delete-by-id if the first valid attempt to
use the ioctl fails with 'Inappropriate ioctl for device'.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a variant of mayfail helper that will duplicate the output to
results log and also provides it to the caller for processing. Can be
used for catching unsupported functionality or other special cases.
Signed-off-by: David Sterba <dsterba@suse.com>
Move the check of dmsetup to check_dm_target_support, and adapt the only
two places checking if dmsetup is present in the system. Now we skip the
tests if dmsetup isn't available, instead of marking the test as failed.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This way we ensure the linear target is available and skip the test.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If dm-thin or dm-linear are not supported, let's skip the test
altogether instead of throwing an error.
Issue: #192
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function will be used later to test if dm-thin is supported.
Suggested-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
One reload_btrfs is lost, add it.
Fixes: 0de2e22ad2 ("btrfs-progs: tests: Add tests for changing fsid feature")
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The truncate command can fail in some platforms like PPC32 because it
can't create files up to 6EiB in size. Skip the test if this was the
problem why truncate failed.
Issue: #192
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance the prerequisite check for internal tools, up to now missing
btrfs-find-root and btrfs-select-super. The correct path in the checks
is INTERNAL_BIN.
The testsuite is now self-contained.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These two binaries are not usually shipped with the package, but they
are used by the testsuite, they need to be included in the exported tar.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As seem in issue #192, this test can fail from time to time. The
issue happens when a mount is issued before the new device is processed
by systemd-udevd, as we can see by the og bellow:
[ 2346.028809] BTRFS: device fsid 593e23af-a7e6-4360-b16a-229f415de697 devid 1 transid 6 /dev/loop10 scanned by systemd-udevd (3418)
[ 2346.265401] BTRFS info (device loop10): found metadata UUID change in progress flag, clearing
[ 2346.272474] BTRFS info (device loop10): disk space caching is enabled
[ 2346.277472] BTRFS info (device loop10): has skinny extents
[ 2346.281840] BTRFS info (device loop10): flagging fs with big metadata feature
[ 2346.308428] BTRFS error (device loop10): devid 2 uuid cde07de6-db7e-4b34-909e-d3db6e7c0b06 is missing
[ 2346.315363] BTRFS error (device loop10): failed to read the system array: -2
[ 2346.329887] BTRFS error (device loop10): open_ctree failed
failed: mount /dev/loop10 /home/marcos/git/suse/btrfs-progs/tests//mnt
test failed for case 034-metadata-uuid
make: *** [Makefile:401: test-misc] Error 1
[ 2346.666865] BTRFS: device fsid 593e23af-a7e6-4360-b16a-229f415de697 devid 2 transid 5 /dev/loop11 scanned by systemd-udevd (3422)
[ 2346.853233] BTRFS: device fsid 1c2debeb-e829-4d6b-84df-aa7c5d246fd5 devid 1 transid 7 /dev/loop6 scanned by systemd-udevd (3418)
A few moments after the test failed systemd-udevd processed the new
device (registered the new device under btrfs). This can be tested by
executing a mount after the test failed, resulting in a successful
mount:
$ mount /dev/loop10 /mnt
[ 2398.955254] BTRFS info (device loop10): found metadata UUID change in progress flag, clearing
[ 2398.959416] BTRFS info (device loop10): disk space caching is enabled
[ 2398.962483] BTRFS info (device loop10): has skinny extents
[ 2398.965070] BTRFS info (device loop10): flagging fs with big metadata feature
[ 2399.012617] BTRFS info (device loop10): enabling ssd optimizations
[ 2399.022375] BTRFS info (device loop10): checking UUID tree
This problem can be avoided is we execute "udevadm settle" before the
mount is executed.
Issue: #192
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Don't hardcode the supported checksums and use the list provided by the
kernel module. The status check of support is moved to the top of the
script so it's more visible.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
[ move checks ]
Signed-off-by: David Sterba <dsterba@suse.com>
There are two utilities that are required for the testsuite but not
installed on systems. The path in tests must be INTERNAL_BIN.
Fix this issue for below tests:
* fsck-tests/013-extent-tree-rebuild
* fsck-tests/037-freespacetree-repair
* misc-tests/038-backup-root-corruption
Issue: #198
Signed-off-by: Long An <lan@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This adds cooked images to exercise the case when a filesystem with
metadata uuid incompat flag is switched back to having fsid/metadata
uuid being equal.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new image has a bad data extent item generation:
item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16230 itemsize 53
refs 1 gen 16384 flags DATA
extent data backref root FS_TREE objectid 257 offset 0 count 1
Just like what older `btrfs check --init-extent-tree` could cause.
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Ran misc-tests/038 in /root/btrfs-progs:
make test-misc TEST=038\*
[TEST] misc-tests.sh
[TEST/misc] 038-backup-root-corruption
./test.sh: line 33: [: bytenr=65536,: integer expression expected
Backup slot 2 is not in use
test failed for case 038-backup-root-corruption
make: *** [Makefile:401: test-misc] Error 1
It's caused by the wrong line filtered by
$(dump_super | grep root | head -n1 | awk '{print $2}').
The command $(dump-super | grep root) outputs:
superblock: bytenr=65536, device=/root/btrfs-progs/tests/test.img
root 30605312
chunk_root_generation 5
root_level 0
chunk_root 22036480
chunk_root_level 0
log_root 0
log_root_transid 0
log_root_level 0
root_dir 6
backup_roots[4]:
Here
"superblock: bytenr=65536, device=/root/btrfs-progs/tests/test.img" is
selected but not the line "root 30605312".
Restricting the awk rule can fix it.
Fixes: 78a3831d46 ("btrfs-progs: tests: Test backup root retention logic")
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since commit e388bf38 ("btrfs-progs: check: warn users about the
possible dangers of --repair") `btrfs check --repair` will wait 10
seconds before really repair the fs.
This hugely slow down the fsck tests. Add --force for check_image()
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Documentation folder path is wrong on exported testsutie. Fix this by
replace TOP with INTERNAL_BIN.
Signed-off-by: An Long <lan@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
It seems to be a typo, since $dev1 was not set, and we are dealing with
TEST_DEV and not with loop devices. As this is now equivalent to the
common mount/umount helpers, use them.
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The value of checksum was not set making mkfs fail.
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add separate tests for basic coverage of new checksum algorithms. It
comes in two parts to do a full mkfs test and also a condition mount
test.
Signed-off-by: David Sterba <dsterba@suse.com>
The manual page of btrfsck clearly states 'btrfs check --repair' is a
dangerous operation.
Although this warning is in place users do not read the manual page
and/or are used to the behaviour of fsck utilities which repair the
filesystem, and thus potentially cause harm.
Similar to 'btrfs balance' without any filters, add a warning and a
countdown, so users can bail out before eventual corrupting the
filesystem more than it already is.
To override the timeout, let --force skip it and continue.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
All the run_* helpers have unused variable cmd, probably a leftover from
debugging the option injection magic.
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for TEST_ARGS_CONVERT to allow injection of eg. checksum
command for the all tests. Use like
$ make TEST_ARGS_CONVERT='--csum=xxhash' TEST_ENABLE_OVERRIDE=true test-convert
This affects all btrfs-convert commands that are run by run_check and
other helpers, IOW this affects all tests, not just convert specific ones.
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for TEST_ARGS_MKFS to allow injection of eg. checksum
command for the all tests. Use like
$ make TEST_ARGS_MKFS='--csum=xxhash' TEST_ENABLE_OVERRIDE=true test-mkfs
This affects all mkfs.btrfs commands that are run by run_check and other
helpers, IOW this affects all tests, not just mkfs specific ones.
Signed-off-by: David Sterba <dsterba@suse.com>
This tests ensures that the kernel correctly persists backup roots in
case the filesystem has been mounted from a backup root.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
[ cleanup to use common helpers ]
Signed-off-by: David Sterba <dsterba@suse.com>
This test uses tool dmsetup so add the global prereq.
Issue: #192
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image contains one inode item with invalid generation. The image
can be crafted by "btrfs-corrupt-block -i 257 -f generation". It should
emulate the bad inode generation caused by older kernel around 2014.
The image is repairable for both original and lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new test image for imode repair in subvolume trees.
The new test cases including the following cases:
- Regular file with bad imode
It still has the valid INODE_REF and parent dir has correct DIR_INDEX
and DIR_ITEM.
In this case, no matter if the file is empty or not, it should be
repaired using the info from DIR_INDEX of parent dir.
- Non-empty regular file with bad imode, and without INODE_REF
The file should be mostly an orphan, so no INODE_REF for imode lookup.
But it has EXTENT_DATA which should be enough for imode repair.
The repair also involves moving the orphan to lost+found dir.
- Non-empty dir with bad imode, and without INODE_REF
Pretty much the same case, but now a directory.
The repair also involves moving the orphan to lost+found dir.
Also rename the existing test case 039-bad-free-space-cache-inode-mode
to 039-bad-inode-mode, since now we can fix all bad imode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The SHA256 is going to be used in the future, so this makes it a second
user and we also have the appropriate directory now.
Signed-off-by: David Sterba <dsterba@suse.com>
If we generate testsuite tarball by 'make testsuite', there is no
clean-test.sh in it. But some tests need cleanup if a testcase failed.
Signed-off-by: An Long <lan@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Discovered with cppcheck. Fix signed/unsigned int mismatches, sizeof and
long formats.
Pull-request: #197
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As misc-tests/021 image dump is restored on the same original loop
device, this overlaps with the stale data and makes the test pass
falsely.
Fix this by using a new device for restore.
And also, the btrfs-image dump and restore doesn't restore the file
data, so any read on the files should be avoided. So instead of file
data use file stat data for the checksum.
Reported-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Btrfs check used to report false alerts on half dropped orphan inodes.
Add test cases to prevent such problem from happening again.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The 'max' keyword checks were missing. We care about correct parsing,
not necessarily the exact result of the filesystem resize.
Signed-off-by: David Sterba <dsterba@suse.com>
Test case for mkfs to handle if total device size overflows 16EiB. The
device images are created inside temporary btrfs so this works on any
underlying filesystem (unlike NFS or tmpfs).
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ split test from original patch and update to work on any filesystem ]
Signed-off-by: David Sterba <dsterba@suse.com>
The image contains a fs tree whose generation is over 100 larger than
super block generation.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Move the full-balance warning to before the fork, so that the user can
see and react to it.
Notes on test:
- Don't use grep -q, as it causes a SIGPIPE during the countdown, and
the balance thus doesn't start.
- The "balance cancel" is superfluous as the last command, but it
provides some idempotence and allows adding more tests below it.
Issue: #168
Signed-off-by: Vladimir Panteleev <git@vladimir.panteleev.md>
Signed-off-by: David Sterba <dsterba@suse.com>
grep's exit code was never checked (and -o errexit is not in effect),
thus the test was ineffectual and regressed.
Add the missing exit code check, and update the error messages to
make the test pass again.
Signed-off-by: Vladimir Panteleev <git@vladimir.panteleev.md>
Signed-off-by: David Sterba <dsterba@suse.com>
Instead of checking the kernel version, explicitly check for the
presence of metadata_uuid file in sysfs. This allows the test to be run
on older kernels that might have this feature backported.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When running misc-test/034, we got unexpected log output:
[TEST/misc] 033-filename-length-limit
[TEST/misc] 034-metadata-uuid
Checking btrfstune logic
Checking dump-super output
Checking output after fsid change
Checking for incompat textual representation
Checking setting fsid back to original
Testing btrfs-image restore
This is caused by commit 2570cff076 ("btrfs-progs: test: cleanup misc-tests/034")
which uses _log facility which also populates stdout.
Just change _log() to echo "$*" >> "$RESULTS" to fix it.
Unlike the initial commit, there is no other user of _log, so it
shouldn't affect other tests.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
'make clean' followed by 'make test-clean' will fail due to the sanity
check for 'btrfs' binary. We don't need that for cleaning the test, so
turn the error into a warning.
Signed-off-by: David Sterba <dsterba@suse.com>
The shell keyword function is not necessary and not used in many tests,
remove it from the few places that use it right now.
Signed-off-by: David Sterba <dsterba@suse.com>
Build several standalone tools into one binary and switch the function
by name (symlink or hardlink).
* btrfs
* mkfs.btrfs
* btrfs-image
* btrfs-convert
* btrfstune
The static target is also supported. The name of resulting boxed
binaries is btrfs.box and btrfs.box.static . All the binaries can be
built at the same time without prior configuration.
text data bss dec hex filename
822454 27000 19724 869178 d433a btrfs
927314 28816 20812 976942 ee82e btrfs.box
2067745 58004 44736 2170485 211e75 btrfs.static
2627198 61724 83800 2772722 2a4ef2 btrfs.box.static
File sizes:
857496 btrfs
968536 btrfs.box
2141400 btrfs.static
2704472 btrfs.box.static
Standalone utilities:
512504 btrfs-convert
495960 btrfs-image
471224 btrfstune
491864 mkfs.btrfs
1747720 btrfs-convert.static
1411416 btrfs-image.static
1304256 btrfstune.static
1361696 mkfs.btrfs.static
So the shared 900K binary saves ~2M, or ~5.7M for static build.
Signed-off-by: David Sterba <dsterba@suse.cz>
The documentation lacks clarity about depth to which recursive
'fi du' goes, and was pointed out by a user.
Add test that creates another mount inside a filesystem and verifies
that 'fi du' does not go there.
Issue: #185
Signed-off-by: David Sterba <dsterba@suse.com>
The documentation lacks clarity about depth to which recursive
defragmentation go, and was pointed out by a user.
The problem here is that the subvolume behaves the same as mount point
regarding path traversal. The nftw stops on mount boundary (FTW_MOUNT).
Add test that verifies this behaviour. Defrag has to be updated to allow
descending to subvolumes (and not mountpoints).
Issue: #185
Signed-off-by: David Sterba <dsterba@suse.com>
The travis-ci fails at test misc-tests/021-image-multi-devices because
the 'btrfs check' is not run with root permissions, unlike all the other
commands. The check is read-only by default, so that should be safe.
Signed-off-by: David Sterba <dsterba@suse.com>
The transfer lines from dd bloat the logs and other lines may not fit.
Disable xfer in all dd commands but still allow errors to be caught.
Signed-off-by: David Sterba <dsterba@suse.com>
Since the commmit 8dd3e5dc2d
("btrfs-progs: tests: fix misc-tests/029 to run on NFS") added the
compatibility of NFS, it called run_mayfail() in the last of the test.
However, run_mayfail() always return the original code. If the test
case is not running on NFS, the last `run_mayfail rmdir "$SUBVOL_MNT"`
will fail with return value 1 then the test fails:
================================================================
====== RUN MAYFAIL rmdir btrfs-progs/tests/misc-tests/029-send-p-different-mountpoints/subvol_mnt
rmdir: failed to remove 'btrfs-progs/tests/misc-tests/029-send-p-different-mountpoints/subvol_mnt': No such file or director
failed (ignored, ret=1): rmdir btrfs-progs/tests/misc-tests/029-send-p-different-mountpoints/subvol_mnt
test failed for case 029-send-p-different-mountpoints
=================================================================
Every instrument in this script handles its error well, so do exit 0
manually in the last.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=202645
Fixes: 8dd3e5dc2d ("btrfs-progs: tests: fix misc-tests/029 to run on NFS")
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Cleanups:
- add quotation
- use _log for messages
- unify spelling of metadata_uuid as feature and incompat bit
- add -- separators
- minor whitespace adjustments
Signed-off-by: David Sterba <dsterba@suse.com>
The directories created under default path needs a fallback in case it's
on NFS and root cannot create the directory. Update the test to create
only one additional mount point and use the TEST_MNT as the toplevel
one.
The error message we're looking for is now a bit weaker due to the less
specific path.
Signed-off-by: David Sterba <dsterba@suse.com>
Lots of test opencode the mkfs phase with no special needs, add a helper
that forcibly creates fileystem on TEST_DEV. Any options can be added,
except devices.
Signed-off-by: David Sterba <dsterba@suse.com>
This ensures that 'btrfs filesystem show' can correctly identify a
filesystem on a newly created local file.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test misc-tests/035-receive-common-mount-point-prefix does another
mount inside TEST_MNT but current 'make test-clean' will not properly
undo the nested mount and this will break subsequent tests. The
recursive unmount can handle that.
Signed-off-by: David Sterba <dsterba@suse.com>
The temporary files are not accessible if the testsuite is hosted on
NFS, pre-create them and allow writes.
Signed-off-by: David Sterba <dsterba@suse.com>
The image has one free space cache inode with invalid mode (0).
item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
generation 30 transid 30 size 65536 nbytes 1507328
block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
atime 0.0 (1970-01-01 08:00:00)
ctime 1553491158.189771625 (2019-03-25 13:19:18)
mtime 0.0 (1970-01-01 08:00:00)
otime 0.0 (1970-01-01 08:00:00)
Both lowmem and original mode should be able to detect and fix it.
The extracted test image is pretty big (1G extracted), as kernel won't
cache small chunks.
Even with SSD, such test may still take some seconds just extracting the
image.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Lowmem can repair after commit
'btrfs-progs: lowmem: move nbytes check before isize check',
so add the beacon file.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
The missing extent will lead to the existence of the gap between adjacent
extents. The fsck should can detect the gap correctly and repair by punch
a hole.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Add a test for a scenario that used to fail due to find_mount_root()
incorrectly determining the mount point for the receive path due to the
fact that a different mount point with a path that is a prefix of the
receive path exists.
This is fixed by a recent patch titled:
"Btrfs-progs: fix mount point detection due to partial prefix match"
Reported-by: David Disseldorp <ddiss@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
[ adjustments to make the test work when hosted on NFS ]
Signed-off-by: David Sterba <dsterba@suse.com>
The runtime of the test is over 4 minutes when hosted on NFS, the
fallocate phase takes the most time. As fallocate can't take multiple
arguments, we can't save the overhead by creating the files at once.
Create the files in batches in the background, this improves the runtime
to 2 minutes.
Signed-off-by: David Sterba <dsterba@suse.com>
Now two locations can detect such problem, either by device item
used/total bytes check, or by early dev extents check against device
boundary.
The image is hand-crafted image which uses DATA SINGLE chunk to feed
btrfs check. As expected, as long as block group item, chunk item,
device used bytes match, older versions check can't detect such problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a bunch of tests exercising the new btrfstune functionality. In
particular check that various restrictions are implemented correctly,
test that btrfs-image works as expected and also test the output of
btrfs inspect-internal dump-super is correct.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Test case misc/021 is testing if we could mount a single disk btrfs
image recovered from multi disk fs.
The problem is, current kernel has extra check for block group, chunk
and dev extent. This means no image can pass btrfs check for chunk
tree and the filesystem will not mount.
So do extra btrfs check before mount, this will also help us to locate
the problem in btrfs-image easier.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For reloc tree, despite of its short lifespan, it's still the backref,
where reloc tree root backref points back to itself, makes it special.
So it's more approriate to put them into 020-extent-ref-cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Total of three conditions are tested. One for short name, one with
name length 255, the last one with more than 255.
This case should pass after commit
'btrfs-progs: change filename limit to 255 when creating subvolume'.
Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Simple test case which preps a filesystem, then corrupts the FST and
finally repairs it. Tests both extent based and bitmap based FSTs.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a test which ensures the kernel returns the correct error value
when missing device removal is requested. This test verifies that kernel
refactoring didn't break the return value.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The commit d99615284a ("btrfs-progs:
fsck-tests: Add test image to check if btrfs check reports uninitialized
rescan as error") added test 035, should have been 036.
Signed-off-by: David Sterba <dsterba@suse.com>
For trees have been balanced, leaves are with flag BTRFS_HEADER_FLAG_RELOC and
extent data backrefs are shared. Like:
=====================
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3932 itemsize 63
refs 129 gen 7 flags DATA
shared data backref parent 35897344 count 41
shared data backref parent 35426304 count 37
shared data backref parent 35422208 count 51
=====================
Then make the leaf which owns the extent data cowed. The shared data
backref was to transferred to keyed data ref, but remaining backrefs are
still shared. Like:
=====================
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3887 itemsize 108
refs 129 gen 7 flags DATA
extent data backref root 5 objectid 258 offset 0 count 40
extent data backref root 5 objectid 257 offset 0 count 1
shared data backref parent 35426304 count 37
shared data backref parent 35422208 count 51
=====================
However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.
Add the test case to check it.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This fuzzed image will not only cause kernel BUG_ON(), but also btrfs
check BUG_ON() for original mode.
Checking filesystem on /home/adam/btrfs/crafted_images/runtime/0.img
UUID: 3381d111-94a3-4ac7-8f39-611bbbdab7e6
checking extents
check/main.c:3677: check_owner_ref: BUG_ON `rec->is_root` triggered, value 1
btrfs(+0x572c2)[0x562d65da72c2]
btrfs(+0x6098d)[0x562d65db098d]
btrfs(+0x60bb6)[0x562d65db0bb6]
btrfs(+0x6179b)[0x562d65db179b]
btrfs(cmd_check+0x1199)[0x562d65db5589]
btrfs(main+0x88)[0x562d65d62768]
/usr/lib/libc.so.6(__libc_start_main+0xeb)[0x7f4fcbb1b06b]
btrfs(_start+0x2a)[0x562d65d6288a]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Test misc/029 only works if the test case is executed as root, while for
sudo usage, it doesn't work as initial mkdir and final cleanup doesn't
use $SUDO_HELPER.
Add "run_check $SUDO_HELPER" for such cases to allow it works under sudo
usage.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
