[BUG]
There are some reports on fsck/001 test segfault failure with lowmem mode.
While I failed to reproduce it, valgrind still catches it with the
following output:
Delete backref in extent [12845056 1048576]
ERROR: file extent [257, 0] has unaligned disk bytenr: 755944791, should be aligned to 4096
ERROR: file extent[257 0] root 5 owner 5 backref lost
Deleted root 5 item[257, 108, 0]
==29080== Conditional jump or move depends on uninitialised value(s)
==29080== at 0x1A81D7: btrfs_release_path (ctree.c:97)
==29080== by 0x192C33: repair_extent_data_item (mode-lowmem.c:3330)
==29080== by 0x1962FF: check_leaf_items (mode-lowmem.c:4696)
==29080== by 0x196ABF: walk_down_tree (mode-lowmem.c:4858)
==29080== by 0x197762: check_btrfs_root (mode-lowmem.c:5157)
==29080== by 0x198335: check_chunks_and_extents_lowmem (mode-lowmem.c:5450)
==29080== by 0x166414: do_check_chunks_and_extents (main.c:8829)
==29080== by 0x169CF7: cmd_check (main.c:10313)
==29080== by 0x11CDC6: cmd_execute (commands.h:125)
==29080== by 0x11D712: main (btrfs.c:386)
==29080==
[CAUSE]
In repair_extent_data_item() if we find unaligned file extent, we just
delete it and kick in hole punch procedure.
The problem is, file extent deletion is done before initializing @path.
And when the deletion is done without problem, we will goto out tag,
which will release @path, containing uninitialized values, and
triggering segfault.
[FIX]
Don't try to abort trans nor free path if we're going through file
extent deletion routine.
Fixes: 0617bde3bc ("btrfs-progs: lowmem: delete unaligned bytes extent data under repair")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Cleanups done separately:
* use the default test image, loop devices not needed for the test
* trim TEST_MNT from all paths
* send output is created inside the test filesystem
Signed-off-by: David Sterba <dsterba@suse.com>
This test case is the reproducer for the previous fix.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When we process a clone request, we look up the source subvolume by
UUID, even if the source is the subvolume that we're currently
receiving. Usually, this is fine. However, if for some reason we
previously received the same subvolume, then this will use paths
relative to the previously received subvolume instead of the current
one. This is incorrect, since the send stream may use temporary names
for the clone source. This can be reproduced as follows:
btrfs subvolume create subvol
dd if=/dev/urandom of=subvol/foo bs=1M count=1
cp --reflink subvol/foo subvol/bar
mkdir subvol/dir
mv subvol/foo subvol/dir/
btrfs property set subvol ro true
btrfs send -f send.data subvol
mkdir first second
btrfs receive -f send.data first
btrfs receive -f send.data second
The second receive results in this error:
ERROR: cannot open first/subvol/o259-7-0/foo: No such file or directory
Fix it by always cloning from the current subvolume if its UUID matches.
This has the nice side effect of avoiding unnecessary UUID tree lookups
in that case.
Fixes: f1c24cd80d ("Btrfs-progs: add btrfs send/receive commands")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The checks for a subvolume being modified after it was received have
been commented out since they were added back in commit f1c24cd80d
("Btrfs-progs: add btrfs send/receive commands"). Let's just get rid of
the noise.
If they were ever in place, it would have never been possible
to do an incremental send and running dedupe against the parent
snapshot.
That particular use case used to cause send, the kernel side, to fail
(initially with a BUG_ON() and later with -EIO returned to user
space), see commit b4f9a1a87a48 ("Btrfs: fix incremental send failure
after deduplication").
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
[ add Filipe's note ]
Signed-off-by: David Sterba <dsterba@suse.com>
Since 1d5b2ad9 ("btrfs-progs: qgroup-verify: Don't treat qgroup
difference as error if the fs hasn't initialized a rescan") a new
message is being printed when the qgroups is incosistent and the rescan
hasn't being executed, so remove the later message send to stderr.
While in this function, simplify the check for a not executed rescan
since !counts.rescan_running and counts.rescan_running == 0 means the
same thing.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Running the lowmem mode for check needs some setup and is not usually
tested, so add a new target that sets up the variables.
Signed-off-by: David Sterba <dsterba@suse.com>
Skip the test 013-subvolume-delete-by-id if the first valid attempt to
use the ioctl fails with 'Inappropriate ioctl for device'.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a variant of mayfail helper that will duplicate the output to
results log and also provides it to the caller for processing. Can be
used for catching unsupported functionality or other special cases.
Signed-off-by: David Sterba <dsterba@suse.com>
xxhash's state and results are always in little, but in progs after the
hash was calculated it was copied to the final buffer via memcpy,
meaning it'd be parsed as a big endian number on big endian machines.
This is incompatible with the kernel implementation of xxhash which
results in erroneous "checksum didn't match" errors on mount.
Fix it by using put_unaligned_le64 which always ensures the resulting
checksum will be copied in little endian format as the kernel expects
it.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=206835
Fixes: f070ece2e9 ("btrfs-progs: add xxhash64 to mkfs")
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If we don't find holes in our hole rb tree we'll just assume there's a
gap from 0 to the length of the file and print that out. But this
simply isn't correct, we could have a gap between the last extent and
the isize, or 0 and the start of the first extent. Fix the error
message to tell us exactly where the hole is.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lowmem check had the opposite problem of normal check, it caught gaps
that started at 0, but would still fail with my fixes in place. This is
because lowmem check doesn't take into account the isize of the inode.
Address this by making sure we do not complain about gaps that are after
isize. This makes lowmem pass with my fixes applied, and still fail
without my fixes.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When writing my test for the i_size patches, I noticed that I was not
actually failing without my patches as I should have been. This is
because we only check if the inode record extent end is < isize, we
don't check if the inode record extent start is > 0. Add this check to
make sure we're catching holes that start at the beginning of the file.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On aarch64 with pagesize 64k, btrfs-convert of ext4 is successful,
but it won't mount because we don't yet support subpage blocksize, ie.
when page size and sectorsize don't match.
BTRFS error (device vda): sectorsize 4096 not supported yet, only support 65536
So in this case during convert provide a warning but let the conversion
proceed.
Example:
WARNING: Blocksize 4096 is not equal to the pagesize 65536,
converted filesystem won't mount on this system.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The old code of copy_one_extent() is a mess:
- The main loop is implemented using goto
- @mirror_num is reset to 1 for each loop
- @mirror num check against @num_copies is wrong for decompression error
This patch will fix this mess by:
- Use read_extent_data()
read_extent_data() has all the good wrapping of btrfs_map_block()
and length check.
This removes a lot of complexity.
- Add extra file extent offset check
To prevent underflow for memory allocation
- Do proper mirror_num check for decompression error
Issue: #221
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
That's a simple tool to calculate direntry hash that's used as part of
key in the b-trees, based on crc32c. There's also btrfs-crc that does
the same and has some additional features, so we can remove hasher.c.
Signed-off-by: David Sterba <dsterba@suse.com>
Move the check of dmsetup to check_dm_target_support, and adapt the only
two places checking if dmsetup is present in the system. Now we skip the
tests if dmsetup isn't available, instead of marking the test as failed.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This way we ensure the linear target is available and skip the test.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If dm-thin or dm-linear are not supported, let's skip the test
altogether instead of throwing an error.
Issue: #192
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function will be used later to test if dm-thin is supported.
Suggested-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Previously, no filenames/xattrs would be printed with --nofilename, but
to keep the format of dump, print a placeholder instead of all names.
This is:
* directory entries (files, directories, subvolumes)
* default subvolume
* extended attributes (name, value)
* hardlink names if stored inside another item
Note that lengths are not hidden because they can be calculated from the
item size anyway.
Signed-off-by: David Sterba <dsterba@suse.com>
In the mail list, it's pretty common that a developer is asking dump tree
output from the reporter, it's better to protect those kind reporters by
hiding the filename if the reporter wants.
This option will skip @name/@data output for the following items:
- DIR_INDEX
- DIR_ITEM
- INODE_REF
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
According to the documentation, btrfs qgroup remove takes the same
options as qgroup assign, i.e., --rescan and --no-rescan. However,
currently no options are accepted. Activate option handling also for
qgroup remove, so that automatic rescan can be disabled by the user.
Signed-off-by: Michael Lass <bevan@bi-co.net>
Signed-off-by: David Sterba <dsterba@suse.com>
One reload_btrfs is lost, add it.
Fixes: 0de2e22ad2 ("btrfs-progs: tests: Add tests for changing fsid feature")
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
LOGICAL_INO v1 ignored the reserved fields, so they could be filled
with random stack garbage and have no effect. LOGICAL_INO_V2 requires
all unused reserved bits to be set to zero, and returns EINVAL if they
are not, to guard against future kernel versions which may interpret
non-zero bit values.
Sometimes when 'btrfs ins log' runs, the stack garbage is zeros, so the
-o (ignore offsets) option for logical-resolve works. Sometimes the
stack garbage is something else, and 'btrfs ins log -o' fails with
invalid argument. This depends mostly on compiler version and build
environment details, so a binary typically either always works or never
works.
Fix by initializing logical-resolve's argument structure with a C99
compound literal zero.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
Signed-off-by: David Sterba <dsterba@suse.com>
The library exports should have the symbols versioned to allow future
extensions. The earlier versions are skipped and current version is the
first one, no change for existing applications as the same set of
functions is exported.
Signed-off-by: David Sterba <dsterba@suse.com>
This ioctl will be responsible for deleting a subvolume using its id.
This can be used when a system has a file system mounted from a
subvolume, rather than the root file system, like below:
/
@subvol1/
@subvol2/
@subvol_default/
If only @subvol_default is mounted, we have no path to reach @subvol1
(id 256) and @subvol2 (id 257), thus no way to delete them. Current
subvolume delete ioctl takes a file handle point as argument, and if
@subvol_default is mounted, we can't reach @subvol1 and @subvol2 from
the same mount point.
$ mount -o subvol=subvol_default /mnt
$ btrfs subvolume delete -i 257 /mnt
This will delete @subvol2 although it's path is hidden.
Fixes: #152
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new ioctl and helpers to allow extended arguments to be passed to
subvolume deletion ioctl. The parent_fs argument should be a mount
point.
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new ioctl to allow extended arguments to be passed to subvolume
deletion ioctl. This will be used for deletion specified by subvolume
id.
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
All callers of write_and_map_eb(), except btrfs-corrupt-block, have
handled error, but inside write_and_map_eb() itself, the only error
handling is BUG_ON().
This patch will kill all the BUG_ON()s inside write_and_map_eb(), and
enhance the the caller in btrfs-corrupt-block() to handle the error.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The truncate command can fail in some platforms like PPC32 because it
can't create files up to 6EiB in size. Skip the test if this was the
problem why truncate failed.
Issue: #192
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance the prerequisite check for internal tools, up to now missing
btrfs-find-root and btrfs-select-super. The correct path in the checks
is INTERNAL_BIN.
The testsuite is now self-contained.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These two binaries are not usually shipped with the package, but they
are used by the testsuite, they need to be included in the exported tar.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As seem in issue #192, this test can fail from time to time. The
issue happens when a mount is issued before the new device is processed
by systemd-udevd, as we can see by the og bellow:
[ 2346.028809] BTRFS: device fsid 593e23af-a7e6-4360-b16a-229f415de697 devid 1 transid 6 /dev/loop10 scanned by systemd-udevd (3418)
[ 2346.265401] BTRFS info (device loop10): found metadata UUID change in progress flag, clearing
[ 2346.272474] BTRFS info (device loop10): disk space caching is enabled
[ 2346.277472] BTRFS info (device loop10): has skinny extents
[ 2346.281840] BTRFS info (device loop10): flagging fs with big metadata feature
[ 2346.308428] BTRFS error (device loop10): devid 2 uuid cde07de6-db7e-4b34-909e-d3db6e7c0b06 is missing
[ 2346.315363] BTRFS error (device loop10): failed to read the system array: -2
[ 2346.329887] BTRFS error (device loop10): open_ctree failed
failed: mount /dev/loop10 /home/marcos/git/suse/btrfs-progs/tests//mnt
test failed for case 034-metadata-uuid
make: *** [Makefile:401: test-misc] Error 1
[ 2346.666865] BTRFS: device fsid 593e23af-a7e6-4360-b16a-229f415de697 devid 2 transid 5 /dev/loop11 scanned by systemd-udevd (3422)
[ 2346.853233] BTRFS: device fsid 1c2debeb-e829-4d6b-84df-aa7c5d246fd5 devid 1 transid 7 /dev/loop6 scanned by systemd-udevd (3418)
A few moments after the test failed systemd-udevd processed the new
device (registered the new device under btrfs). This can be tested by
executing a mount after the test failed, resulting in a successful
mount:
$ mount /dev/loop10 /mnt
[ 2398.955254] BTRFS info (device loop10): found metadata UUID change in progress flag, clearing
[ 2398.959416] BTRFS info (device loop10): disk space caching is enabled
[ 2398.962483] BTRFS info (device loop10): has skinny extents
[ 2398.965070] BTRFS info (device loop10): flagging fs with big metadata feature
[ 2399.012617] BTRFS info (device loop10): enabling ssd optimizations
[ 2399.022375] BTRFS info (device loop10): checking UUID tree
This problem can be avoided is we execute "udevadm settle" before the
mount is executed.
Issue: #192
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Don't hardcode the supported checksums and use the list provided by the
kernel module. The status check of support is moved to the top of the
script so it's more visible.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
[ move checks ]
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_fs_info::block_group_cache and the bit BLOCK_GROUP_DIRY are not
used anymore, so is the block_group_state_bits(). Remove them.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
