convert-tests now test both 4096 and 16384 nodesizes.
Signed-off-by: Sebastian Thorarensen <sebth@naju.se>
Signed-off-by: David Sterba <dsterba@suse.cz>
Although fsck-test/012 uses sudo, it uses 'sudo -n', which won't prompt
user to input password and will return 1 if no valid credential is
found.
And this makes test result quite annoying since it fails to mount and
still continue, which will always fail.
This patch will check 'sudo -v -n' and 'sudo -n true' to determine
whether sudo works fine in different version/settings, since in some
setting/version, 'sudo -v -n' will fail even the user is set NOPASSWD.
Also, remove the 'have_root_helper' variant, since there is a
possibility that sudo credential will timeout during the test and
'have_root_helper' won't help to detect such problem.
New '_sudo' command will do credential check if needed to avoid such
problem.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Previously we used to just set FULL_BACKREF if we couldn't lookup an extent info
for an extent. Now we just bail out if we can't lookup the extent info, which
is less than good since fsck is supposed to fix these very problems. So instead
figure out the flag we are supposed to use and pass that along instead. This
patch also provides a test image to test this functionality. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Sudo may not be configured to run without user prompt, in that case the
tests would be stuck.
Reported-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Most of the checks run fine without root, but some of them may need to
do a mount test or access the data. Add the support to selectively run
commands under root, hardcoded to sudo for now.
Signed-off-by: David Sterba <dsterba@suse.cz>
Move leaf-corruption type no extent data case to its dir, and add
verification script in test.sh
The verification script is based on manual btrfs-debug-tree check.
The image can also be reused to other leaf-corruption type, like
corrupted leaf contains regular file extent data case.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Now 006-bad-root-items has two cases, one for default case, and one for
skinny metadata case.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Move these obvious btrfs-image to its corresponding dir to use the new
infrastructure.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Change the old btrfsck test infrastructure (btrfs-image dump or xz raw
dump) to the new test infrastructure.
1) Test case layout
The new infrastructure is dir based, each dir is one test type, and can
contain multiple images/scripts for different corner cases.
So layout will be the following:
btrfs-progs
|-tests
|-fsck-tests
|-001-SOME-CORRUPT-TYPE
|-IMAGE-FOR-CASE1
|-IMAGE-FOR-CASE2
2) Test case image types
Only 2 types for test case images.
a) btrfs-image dump
This one is the simplest case, one only needs to add the image to
corresponding dir.
b) custom script
This one is for all the resting cases which can't fit btrfs-image,
like csum error or script can generate the image (this reduces the
size obviously and good for review)
The old binary dump also belongs to this type, so need to add script
to extract them.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This adds two new test images
1) 008-bad-offset-snapshots. This has a corrupt item with multiple snapshots
pointing to it, to make sure the bad block repair stuff doesn't loop and
actually repairs stuff. It also requires the dir index repair stuff to pass our
built in tests which is why it's not tied to the same commit.
2) 009-bad-dir-index-name.img. This has a corrupt name in a dir index to make
sure our dir index repair stuff is working properly.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
These test the recreating of missing dir item/dir index pairs, fixing the no
rootdir inode item and no inode item for normal files. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
This change adds code to detect and fix the issue introduced in the kernel
release 3.17, where creation of read-only snapshots lead to a corrupted
filesystem if they were created at a moment when the source subvolume/snapshot
had orphan items. The issue was that the on-disk root items became incorrect,
referring to the pre orphan cleanup root node instead of the post orphan
cleanup root node.
A test filesystem can be generated with the test case recently submitted for
xfstests/fstests, which is essencially the following (bash script):
workout()
{
ops=$1
procs=$2
num_snapshots=$3
_scratch_mkfs >> $seqres.full 2>&1
_scratch_mount
snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
run_check $FSSTRESS_PROG -p $procs \
-x "$snapshot_cmd" -X $num_snapshots -d $SCRATCH_MNT -n $ops
}
ops=10000
procs=4
snapshots=500
workout $ops $procs $snapshots
Example of btrfsck's (btrfs check) behaviour against such filesystem:
$ btrfsck /dev/loop0
root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
Found 7 roots with an outdated root item.
Please run a filesystem check with the option --repair to fix them.
$ echo $?
1
$ btrfsck --repair /dev/loop0
enabling repair mode
fixing root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
fixing root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
fixing root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
fixing root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
fixing root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
fixing root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
fixing root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
Fixed 7 roots.
Checking filesystem on /dev/loop0
UUID: 2186e9b9-c977-4a35-9c7b-69c6609d4620
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 618537000 bytes used err is 0
total csum bytes: 130824
total tree bytes: 601620480
total fs tree bytes: 580288512
total extent tree bytes: 18464768
btree space waste bytes: 136939144
file data blocks allocated: 34150318080
referenced 27815415808
Btrfs v3.17-rc3-2-gbbe1dd8
$ echo $?
0
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We can deal with corrupt items by deleting them in a few cases. Fsck can easily
recover from a missing extent item or a dir index item. So if we notice a item
is completely bogus and it is of a key that we know we can repair then just
delete it and carry on. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
A user had a corrupted fs where the items had been shifted improperly. This
patch adds the ability to fix this sort of problem within fsck. We will simply
shift the item over to the proper offset and update the offsets to make sure
they are correct. I tested this with a hand crafted fs that was broken in the
same way as the user, and I've included the file as a new test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If we have an inode backref entry then we know enough to add back a missing dir
index. When messing with the inode backrefs we need to do all of that first
before we process the inode recs themselves as we may clear errors on the inode
recs as we fix the directory indexes. This adds the framework for fixing
backref errors and fixes missing dir index issues. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
It is highly obnoxious to have to go put in a testdev when all you really want
is to run the quick image tests. Make this part optional so if we don't have a
testdev specified we just don't run that particular test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We need test to verify extent tree rebuilding work, this test
create a strange filesystem with some snapshots, destroy
extent root node, and run fsck with "--init-extent-tree".
Since this tests need btrfs internal tool(btrfs-corrupt-block),so
i add this test into btrfs-progs.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
On my system, this brings the FS conversion test suite's runtime from over
ten seconds down to under two.
Thanks to Julien Muchembled for the suggestion.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
These use the system's mke2fs, and don't require loop devices
or root privileges.
They don't pick up anything with the default flags right now,
but they do pick up some sanitizer issues when the tools are
compiled with any of -fsanitize={address,memory,thread}.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This is a verification test for the transid recow functionality of btrfsck.
I've also adjusted the test script to spit out which image it's testing so I can
be sure the image was getting tested. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
We need to start adding some sanity tests to btrfs-progs to make sure we aren't
breaking things with our patches. The most important of these tools is btrfsck.
This patch gets things started by adding a basic btrfsck test that makes sure we
can fix a corruption problem we know we can fix. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
