Create a filesystem on a file backed loop block device, then shrink the
file (and its loop block device), then make sure btrfs check can detect
such shrunk device.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
While testing some changes to how we reclaim block groups I started
hitting failures with my TEST_DEV. This occurred because I had a bug
and failed to properly remove a block groups free space tree entries.
However this wasn't caught in testing when it happened because
btrfs check only checks that the free space cache for the existing block
groups is valid, it doesn't check for free space entries that don't have
a corresponding block group.
Fix this by checking for free space entries that don't have a
corresponding block group. Additionally add a test image to validate
this fix.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Previously 'btrfs check --check-data-csum' will report tons of false
alerts for RAID56.
Add a test case to make sure with the new RAID56 rebuild ability, there
should be no false alerts.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Previously we had a bug that btrfs check would report false warning for
a sprouted filesystem.
So this patch will add a new test case to make sure neither seed nor
and sprouted filesystem will cause such false warning.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new image has incorrect super num devices (have 8 expect 1).
The image has to be raw format, as btrfs-image restore will reset super
num devices to correct value.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When adding the GC support I noticed we were failing fsck when we had a
directory that hadn't been cleaned up yet because of rm -rf. However
this isn't limited to extent-tree-v2, we'll actually fail in the same
way if we were unable to do the evict portion of the deletion and left
the orphan items around for everybody.
This is a valid file system, it'll be cleaned up properly at mount time,
so fsck shouldn't fail in this case.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image has a key in extent tree, (30457856 METADATA_ITEM 256), which
has invalid level (256 > BTRFS_MAX_LEVEL).
Make sure check can at least detect such problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This new test script will create a fs with the following situations:
- Preallocated extents (no data csum)
- Nodatasum inodes (no data csum)
- Partially written preallocated extents (no data csum for part of the
extent)
- Regular data extents (with data csum)
- Regular data extents then hole punched (with data csum)
- Preallocated data, then written, then hole punched (with data csum)
- Compressed extents (with data csum)
And make sure after --init-csum-tree (with or without
--init-extent-tree) the result fs can still pass check.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test looks for a pattern FREE_SPACE that also matches
FREE_SPACE_TREE when it's enabled. Use word regexp to match the whole
word only.
Signed-off-by: David Sterba <dsterba@suse.com>
This is used to validate the detection and correction code in both fsck
modes for an invalid bytes_used value in the super block.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This image has a broken used field of a block group item to validate
fsck does the correct thing.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Make sure btrfs check can detect such problem. Right now we have no way
to fix it yet.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test image is manually crafted with 1MiB offset in the device item
of devid 1.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Pre-created image contains a subvolume and a snapshot so that cleaning
of multiple roots is also tested. The mount option 'inode_cache' will be
removed so we need the crafted image.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will:
- Add a new test image for fsck/044
This new image has a corrupted extent item generation for tree block.
This image can expose a bug in original mode, which can't detect the
problem.
This image also utilize the tree block generation detection code,
which the existing image doesn't.
- Rename the existing image
To reflect the fact that the existing one is only for data extent.
- Remove the test.sh
So that the generic path will test both detection and repair.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Kernel patch ff51bf02d107 ("btrfs: block-group: fix free-space bitmap
threshold") is needed to exercise both branches of the test, this can be
detected by lack of the objectid and offset parsed from the dump.
Issue: #288
Signed-off-by: David Sterba <dsterba@suse.com>
The image used in test case 035 has one problem:
- Bad total_bytes of device item in chunk tree
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
devid 1 total_bytes 72564736 bytes_used 92274688
The total_bytes is even smaller than bytes_used.
While the dev_item in super block contains the correct value:
dev_item.total_bytes 134217728
dev_item.bytes_used 92274688
This patch will update the image, using the correct value in super block
to replace the wrong one in chunk tree.
So that later btrfs-image restore without device/chunk fixup can still
pass fsck-tests.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image used in 012 has several problems:
- Bad dev_item total_bytes in superblock
dev_item.total_bytes 20918272
dev_item.bytes_used 67108864
The fs has 64M chunks allocated, and the original fs is also 64M.
Thus the total_bytes is completely wrong.
- Bad total_bytes in chunk tree
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 3897 itemsize 98
devid 1 total_bytes 20918272 bytes_used 67108864
The same problem as super block.
This problem is covered by btrfs-image, as it will recalculate the
device size.
However if btrfs-image is doing exactly same data replay without any
modification, such bad image would cause test failure, as we're unable
to fix such bad total_bytes.
This patch will update the image to fix such problem.
Also since we're here, there is no need to use custom .tar.gz image, as
we have extra_image() to handle the uncompression.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image has a csum item overlap with next csum item, make sure btrfs
check can detect it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are two utilities that are required for the testsuite but not
installed on systems. The path in tests must be INTERNAL_BIN.
Fix this issue for below tests:
* fsck-tests/013-extent-tree-rebuild
* fsck-tests/037-freespacetree-repair
* misc-tests/038-backup-root-corruption
Issue: #198
Signed-off-by: Long An <lan@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new image has a bad data extent item generation:
item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16230 itemsize 53
refs 1 gen 16384 flags DATA
extent data backref root FS_TREE objectid 257 offset 0 count 1
Just like what older `btrfs check --init-extent-tree` could cause.
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The manual page of btrfsck clearly states 'btrfs check --repair' is a
dangerous operation.
Although this warning is in place users do not read the manual page
and/or are used to the behaviour of fsck utilities which repair the
filesystem, and thus potentially cause harm.
Similar to 'btrfs balance' without any filters, add a warning and a
countdown, so users can bail out before eventual corrupting the
filesystem more than it already is.
To override the timeout, let --force skip it and continue.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
The image contains one inode item with invalid generation. The image
can be crafted by "btrfs-corrupt-block -i 257 -f generation". It should
emulate the bad inode generation caused by older kernel around 2014.
The image is repairable for both original and lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new test image for imode repair in subvolume trees.
The new test cases including the following cases:
- Regular file with bad imode
It still has the valid INODE_REF and parent dir has correct DIR_INDEX
and DIR_ITEM.
In this case, no matter if the file is empty or not, it should be
repaired using the info from DIR_INDEX of parent dir.
- Non-empty regular file with bad imode, and without INODE_REF
The file should be mostly an orphan, so no INODE_REF for imode lookup.
But it has EXTENT_DATA which should be enough for imode repair.
The repair also involves moving the orphan to lost+found dir.
- Non-empty dir with bad imode, and without INODE_REF
Pretty much the same case, but now a directory.
The repair also involves moving the orphan to lost+found dir.
Also rename the existing test case 039-bad-free-space-cache-inode-mode
to 039-bad-inode-mode, since now we can fix all bad imode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Btrfs check used to report false alerts on half dropped orphan inodes.
Add test cases to prevent such problem from happening again.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image contains a fs tree whose generation is over 100 larger than
super block generation.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image has one free space cache inode with invalid mode (0).
item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
generation 30 transid 30 size 65536 nbytes 1507328
block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
atime 0.0 (1970-01-01 08:00:00)
ctime 1553491158.189771625 (2019-03-25 13:19:18)
mtime 0.0 (1970-01-01 08:00:00)
otime 0.0 (1970-01-01 08:00:00)
Both lowmem and original mode should be able to detect and fix it.
The extracted test image is pretty big (1G extracted), as kernel won't
cache small chunks.
Even with SSD, such test may still take some seconds just extracting the
image.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Lowmem can repair after commit
'btrfs-progs: lowmem: move nbytes check before isize check',
so add the beacon file.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
The missing extent will lead to the existence of the gap between adjacent
extents. The fsck should can detect the gap correctly and repair by punch
a hole.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
The runtime of the test is over 4 minutes when hosted on NFS, the
fallocate phase takes the most time. As fallocate can't take multiple
arguments, we can't save the overhead by creating the files at once.
Create the files in batches in the background, this improves the runtime
to 2 minutes.
Signed-off-by: David Sterba <dsterba@suse.com>
Now two locations can detect such problem, either by device item
used/total bytes check, or by early dev extents check against device
boundary.
The image is hand-crafted image which uses DATA SINGLE chunk to feed
btrfs check. As expected, as long as block group item, chunk item,
device used bytes match, older versions check can't detect such problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For reloc tree, despite of its short lifespan, it's still the backref,
where reloc tree root backref points back to itself, makes it special.
So it's more approriate to put them into 020-extent-ref-cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Simple test case which preps a filesystem, then corrupts the FST and
finally repairs it. Tests both extent based and bitmap based FSTs.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The commit d99615284a ("btrfs-progs:
fsck-tests: Add test image to check if btrfs check reports uninitialized
rescan as error") added test 035, should have been 036.
Signed-off-by: David Sterba <dsterba@suse.com>
For trees have been balanced, leaves are with flag BTRFS_HEADER_FLAG_RELOC and
extent data backrefs are shared. Like:
=====================
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3932 itemsize 63
refs 129 gen 7 flags DATA
shared data backref parent 35897344 count 41
shared data backref parent 35426304 count 37
shared data backref parent 35422208 count 51
=====================
Then make the leaf which owns the extent data cowed. The shared data
backref was to transferred to keyed data ref, but remaining backrefs are
still shared. Like:
=====================
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3887 itemsize 108
refs 129 gen 7 flags DATA
extent data backref root 5 objectid 258 offset 0 count 40
extent data backref root 5 objectid 257 offset 0 count 1
shared data backref parent 35426304 count 37
shared data backref parent 35422208 count 51
=====================
However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.
Add the test case to check it.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba
Qu Wenruo
Josef Bacik
Adam Borowski
Nikolay Borisov
Long An
Johannes Thumshirn
Su Yue
Lu Fengqi
Josh Soref