This new image misses one extent which leads lowmem mode to allocate new
chunks in repair.
Rename original image to no_extent_bad_dev.img.
Because of its bad used bytes, it should let lowmem mode
exclude blocks in repair.
Due to problems of btrfs-image, choose xz as compression tool.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Verify that a filesystem check operation (fsck) does not report the
following scenario as an error:
An extent is shared between two inodes, as a result of clone/reflink
operation, and for one of the inodes, lets call it inode A, the extent is
referenced through a file extent item as a prealloc extent, while for the
other inode, call it inode B, the extent is referenced through a regular
file extent item, that is, it was written to. The goal of this test is to
make sure a filesystem check operation will not report "odd csum items"
errors for the prealloc extent at inode A, because this scenario is valid
since the extent was written through inode B and therefore it is expected
to have checksum items in the filesystem's checksum btree for that shared
extent.
Such scenario can be created with the following steps for example:
mkfs.btrfs -f /dev/sdb
mount /dev/sdb /mnt
touch /mnt/foo
xfs_io -c "falloc 0 256K" /mnt/foo
sync
xfs_io -c "pwrite -S 0xab 0 256K" /mnt/foo
touch /mnt/bar
xfs_io -c "reflink /mnt/foo 0 0 256K" /mnt/bar
xfs_io -c "fsync" /mnt/bar
<power fail>
mount /dev/sdb /mnt
umount /mnt
This scenario is fixed by the following patch for the filesystem checker:
"Btrfs-progs: check, fix false error reports for shared prealloc extents"
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The commit cebf3b3722 ("btrfs-progs: introduce TEST_TOP and
INTERNAL_BIN for tests") did not convert all test paths. This would
break the exported testsutie.
Signed-off-by: David Sterba <dsterba@suse.com>
Regression test for false alerts in lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update test ]
Signed-off-by: David Sterba <dsterba@suse.com>
Add the testcase for false alert of data extent backref lost with the
extent offset.
The image can be reproduced by the following commands:
------
dev=~/test.img
mnt=/mnt/btrfs
umount $mnt &> /dev/null
fallocate -l 128M $dev
mkfs.btrfs $dev
mount $dev $mnt
for i in `seq 1 10`; do
xfs_io -f -c "pwrite 0 2K" $mnt/file$i
done
xfs_io -f -c "falloc 0 64K" $mnt/file11
for i in `seq 1 32`; do
xfs_io -f -c "reflink $mnt/file11 0 $(($i * 64))K 64K" $mnt/file11
done
xfs_io -f -c "reflink $mnt/file11 32K $((33 * 64))K 32K" $mnt/file11
btrfs subvolume snapshot $mnt $mnt/snap1
umount $mnt
btrfs-image -c9 $dev extent_data_ref.img
------
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use TEST_TOP as base for tests to reference any files, this will be used
for git and external testsuite.
INTERNAL_BIN is needed for referencing binaries that could reside in
different paths in git vs external testsuite.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ add quotes around sourced files, update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
There are 2 fsck tests with the same number 027:
tree-reloc-tree
bad-extent-inline-ref-type
And we also have a hole in 015, so just rename tree-reloc-tree to 015,
to get rid of the duplicated test number and fill in the hole.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Transform the test to the common helpers and don't manage the loop
devices here. The test category changes from check to misc.
Signed-off-by: David Sterba <dsterba@suse.com>
This functionality regressed some time ago and it was never caught. Seems no
one complained of that, but to be sure add a regression test to prevent future
regressions.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Reloc tree is a special tree with very short life span. It acts as a
special snapshot for any tree, with related nodes/leaves or EXTENT_DATA
modified to point to new position.
Considering the short life span and its special purpose, it should be
quite reasonable to keep them as both corner case for fsck and
educational dump for anyone interested in relocation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For snapshot shared tree blocks with source subvolume, the keyed backref
counter only counts the exclusive owned references.
In the following case, 258 is a snapshot of 257, which inherits all the
reference to this data extent.
------
item 4 key (12582912 EXTENT_ITEM 524288) itemoff 3741 itemsize 140
refs 179 gen 9 flags DATA
extent data backref root 257 objectid 258 offset 0 count 49
extent data backref root 257 objectid 257 offset 0 count 1
extent data backref root 256 objectid 258 offset 0 count 128
extent data backref root 256 objectid 257 offset 0 count 1
------
However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.
Add the test case to check it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image is dumped by modifying kernel to sleep long enough before
merging relocation trees, so we can just copy the whole image to other
place before kernel begins to merge reloc trees.
And the base image is created by the following script to bump metadata
size:
------
dev=~/test.img
mnt=/mnt/btrfs
umount $mnt &> /dev/null
fallocate -l 128M $dev
mkfs.btrfs -f -n 4k -m single -d single $dev
mount $dev $mnt -o nospace_cache,max_inline=2048
btrfs subvolume create $mnt/src
for i in $(seq -w 0 128); do
xfs_io -f -c "pwrite 0 2k" $mnt/src/file_$i > /dev/null
done
for i in $(seq -w 0 64); do
btrfs subvolume snapshot $mnt/src/ $mnt/snapshot_$i
touch $mnt/snapshot_$i/new
done
sync
------
The image triggers several corner cases that the old lowmem mode didn't
consider.
Like metadata backref with FULL_BACKREF flag and only SHARED_BLOCK_REF
backrefs for metadata. And several tree reloc trees with shared
leaves/nodes to confuse old lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new test image, which has an extent item with no inlined
extent data ref, but all keyed extent data ref.
Only in this case we can trigger fase data extent backref lost bug in
lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a image which can reproduce the extent item referencer count
mismatch false alert for lowmem mode.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a image that the inlined extent coexist with the regular extent.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image has 2 problems mixed:
1) Too small super total_bytes
This super total_bytes is manually modified to create such problem.
2) Unaligned dev item total_bytes
This is created by v4.12 kernel, with 128M + 2K device added, and
original device removed.
Then we can create such image with unaligned dev item total_bytes.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This case is for avoiding crash in lowmem check mode.
Field type of extent_inline_ref in an extent is corrupted.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since lowmem mode can repair certain corruptions (mostly in fs tree),
insert a beacon into each fsck test cases to allow some of them be
tested in lowmem mode.
With this patch, fsck option override will check the beacon file
".lowmem_repairable" in the same directory of the test image, and if the
beacon exists, then it will also run lowmem mode repair to repair the
image.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In this test case, all name in dir_item, dir_index, inode_ref
are corrupted to another one.
btrfs check should report errors about the corrupted dir_item but
btrfs can't repair the case now.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Without it, mount (at least from util-linux 2.20.1) tries (and fails) to
mount some filesystems as NTFS.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case which we have NO_HOLES incompat flag while still have
hole file extent.
This can be created by enabling NO_HOLES feature on an existing
filesystem, which lowmem mode would cause false alert for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new image, which contains external SHARED_DATA_REF items to
trigger a lowmem mode false alert.
The image only contains external SHARED_DATA_REF and no inlined data
backref.
Before the image, we only have inlined shared data ref, which is not
enough to trigger lowmem mode false alert.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Drop the dependency on xfs_io as it's not a standard tool, though it
provides convenience. We use a simple write here so dd can manage.
Signed-off-by: David Sterba <dsterba@suse.com>
If the tests are started from non-root user, the fallocate and xfs_io
fail. Use the root helper as a workaround, we'd should fix the perms
instead.
Signed-off-by: David Sterba <dsterba@suse.com>
Lowmem mode exposed several false alerts, all related to file extents
check.
1) Partly written prealloc extent
Cause lowmem mode to report missing csum or prealloc extent should
not have csum
2) Compressed inline extent
Cause lowmem mode to find mismatch on inline len and item len.
While no error message is output but exit silently.
Reported-by: Chris Murphy <chris@colorremedies.com>
Reported-by: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
fsck-tests/013-extent-tree-rebuild uses "--init-extent-tree", which
implies "--repair".
But the test script doesn't specify "--repair" for lowmem mode test to
detect it.
Add it so lowmem mode test can be happy with it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a minimal image which can reproduce the block group used space
false alert for lowmem mode fsck.
Reported-by: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case fsck-tests/015-check-bad-memory-access can't be repair by
btrfs check, and it's a fortunate bug makes original mode to forget the
error code from extent tree, making original mode pass it.
So fuzz-tests is more suitable for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In my test environment, following error was occurred because the size
of /lib/modules/`uname -r`/* is larger than 1GB.
# make test-fsck
[TEST] fsck-tests.sh
[TEST/fsck] 013-extent-tree-rebuild
failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
test failed for case 013-extent-tree-rebuild
Makefile:272: recipe for target 'test-fsck' failed
make: *** [test-fsck] Error 1
#
In this test case, 'generate_dataset small' is enough for making the
test files, so I will use 'generate_dataset' instead of 'cp'.
For this, move 'generate_dataset()' from 'common.convert' to 'common'.
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case to check the basic function of --clear-space-cache.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For image with tree reloc tree, if its height is over level 2, the root
node's backref will point to itself. It's valid for kernel, but quota
verify code can't handle it and cause a infinite call, overflowing the
stack.
Add minimal image to reproduce the bug, as regression test.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor cleanups in test.sh ]
Signed-off-by: David Sterba <dsterba@suse.com>
Fixed by commit 7c646c538e btrfs-progs: qgroup: Fix regression leads
to corrupted qgroup status.
Add minimal test image for that fix.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor cleanups in test.sh ]
Signed-off-by: David Sterba <dsterba@suse.com>
In order to confirm that btrfsck supports to check a variety of
refs, add the
following cases:
* keyed_block_ref
* keyed_data_ref
* shared_block_ref
* shared_data_ref
* no_inline_ref (a extent item without inline ref)
* no_skinny_ref
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This commit extends the leaf corruption test to try to repair a file
linked from multiple directory. It stresses a case that some links to a
file is broken but others kept valid.
Signed-off-by: Naohiro Aota <naota@elisp.net>
Signed-off-by: David Sterba <dsterba@suse.com>
Catch a buggy condition fixed by "btrfs-progs: fsck: Fix a false alert
where extent record has wrong metadata flag"
Signed-off-by: David Sterba <dsterba@suse.com>
To test if fsck can check detec "leaf crossing stripes".
This function was introduced from patch titled:
btrfs-progs: fsck: Check if a metadata tree block crossing stripe boundary
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[renamed and other minor changes]
Signed-off-by: David Sterba <dsterba@suse.com>
mount command in old system can not add "-o loop" option automatically
for loop device, and make following test failed:
# ./fsck-tests.sh
...
[TEST] 013-extent-tree-rebuild
failed: mount /data/btrfsprogs/tests/test.img /data/btrfsprogs/tests/mnt
test failed for case 013-extent-tree-rebuild
Considering that $TEST_DEV can be block or loop device, we need determine
our mount option in a condition for both case.
This patch create a wrapper function for above request, to solve current
problem in 013-extent-tree-rebuild, and support similar request in future.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
