The functionality of pretty unit printing was duplicated by
df_pretty_sizes, merge it with pretty_size and enhance the interface
with more suffix mode. Raw, binary or decimal.
Signed-off-by: David Sterba <dsterba@suse.cz>
In a filesystem corrupted by a faulty memory module, btrfsck would get
very confused attempting to access backrefs that weren't data backrefs
as if they were. Besides invoking undefined behavior for accessing
potentially-uninitialized data past the end of objects, or with
dynamic types unrelated with the static types held in the
corresponding memory, it used offsets and lengths from such fields
that did not correspond to anything in the filesystem proper.
Moving the test for full backrefs and checking that they're data
backrefs earlier avoided the crash I was running into, but that was
not enough to make the filesystem complete a successful repair.
Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
If we attempt to repair a filesystem with metadata blocks that need
recowing, we'll get into an infinite loop repeatedly recowing the
first entry in the list, without ever removing it from the list.
Oops. Fixed.
Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
If checksum root is corrupted, fsck will get segmentation. This
is because if we fail to load checksum root, root's node is NULL which
cause NULL pointer deferences later.
To fix this problem, we just did something like extent tree rebuilding.
Allocate a new one and clear uptodate flag. We will do sanity check
before fsck going on.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The reason that we allow partial opening is that sometimes,
we may have some corrupted trees.(for example extent tree), for
fsck repair case, the broken tree may be rebuilt later.
So if users only want to do check but not repair anything, this
patch will make fsck return failure as soon as possible and
tell users that some critial roots have been corrupted.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If btrfs tree root is corrupted, fsck will hit the following segmentation.
enabling repair mode
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
read block failed check_tree_block
Couldn't read tree root
Checking filesystem on /dev/sda9
UUID: 0e1a754d-04a5-4256-ae79-0f769751803e
Critical roots corrupted, unable to fsck the FS
Segmentation fault (core dumped)
In btrfs_setup_all_roots(), we could tolerate some trees(extent tree, csum tree)
corrupted, and we have did careful check inside that function, it will
return NULL if critial roots corrupt(for example tree root).
The problem is that we check @OPEN_CTREE_PARTIAL flag again after
calling btrfs_setup_all_roots() which will successfully return
@fs_info though critial roots corrupted.
Fix this problem by removing @OPEN_CTREE_PARTIAL flag check outsize
btrfs_setup_all_roots().
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
As mentioned in the kernel patch
btrfs: ioctl BTRFS_IOC_FS_INFO and
BTRFS_IOC_DEV_INFO miss-matched with slots
The count as returned by BTRFS_IOC_FS_INFO is the number of slots that
btrfs-progs would allocate for the BTRFS_IOC_DEV_INFO ioctl. Since
BTRFS_IOC_DEV_INFO would loop across the seed devices, So its better
ioctl BTRFS_IOC_FS_INFO returns the total_devices instead of num_devices.
The above mentioned patch just does that. That is, it returns
total_devices instead of num_devices.
Which means we need to probe for the replacing device separately.
This patch will probe for the replacing device separately.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
*Note*
this handles the problem under umounted state, the similar problem
under mounted state is already fixed by Anand.
Steps to reproduce:
# mkfs.btrfs -f /dev/sda1
# btrfstune -S 1 /dev/sda1
# mount /dev/sda1 /mnt
# btrfs dev add /dev/sda2 /mnt
# umount /mnt <== (umounted)
# btrfs fi show /dev/sda2
result:
Label: none uuid: XXXXXXXXXXXXXXXXXX
Total devices 2 FS bytes used 368.00KiB
devid 2 size 9.31GiB used 1.25GiB path /dev/sda2
*** Some devices missing
Btrfs v3.16-67-g69f54ea-dirty
It is because @btrfs_scan_lblkid() won't establish mappinig
between the seed and sprout devices. So seeding devices are missing.
We could use @open_ctree_* to detect all seed/sprout mappings
for each fs scanned after @btrfs_scan_lblkid().
sth worthes mention:
o If there are multi-level of seeds, all devices in them will be shown
in the ascending order of @devid
o If device replace is execed on a sprout fs with a device in a seed fs,
the replaced device still exist in the seed fs together with
the replacing device in the sprout fs, so we only keep the latest device
with the newest generation
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Error out cmd_start_replace() if canonicalize_path() dstdev failed, add
the missing "goto leave_with_error;"
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This commit adds the support for a make variable named
"DISABLE_BACKTRACE" which allows to disable the support for backtrace()
usage on ASSERT(), BUG() and BUG_ON() calls.
This is useful because some alternative C libraries like uClibc have
optional support for backtrace() which is rarely built when debugging
isn't taking place.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: David Sterba <dsterba@suse.cz>
Steps to reproduce:
# mkfs.btrfs -f /dev/sda[1-2]
# btrfstune -S 1 /dev/sda1
# mount /dev/sda /mnt
# btrfs dev add /dev/sda3 /mnt
# umount /mnt
# mkfs.ext4 /dev/sda1 // kill seed dev
# mkfs.ext4 /dev/sda2 // kill seed dev
# btrfs-debug-tree /dev/sda3 <== BUG_ON
Output msg:
volumes.c:1824: btrfs_read_chunk_tree: Assertion `ret` failed.
btrfs-debug-tree[0x41cb36]
btrfs-debug-tree(btrfs_read_chunk_tree+0x3ca)
btrfs-debug-tree(btrfs_setup_chunk_tree_and_device_map
btrfs-debug-tree[0x40f695]
btrfs-debug-tree(open_ctree_fs_info+0x86)
btrfs-debug-tree(main+0x12d)
/lib64/libc.so.6(__libc_start_main+0xf5)
btrfs-debug-tree[0x4062e9]
This BUG_ON complains about a failed @read_one_dev() call when
@open_seed_devices() failed to find the seed @fs_devices object
for a dev_item in chunk tree.
In this case, just insert a "shadow" @fs_devices with the fsid in
dev_item shall make no harm since no other tools will try to
make use of the stuff that the "shadow" @fs_devices possesses
after its creation.
After apply this commit, btrfs-debug-tree will report unable
to open the device.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When btrfs-progs walk down the tree, it does not check whether the child
node/leaf is valid.
In fact, there is some corrupted image whose csum is all valid but
parent node points to a invalid leaf.
In my case, the parent node in fs tree point to a invalid leaf(gen 11),
whose generation(15) and first key(EXTENT_TREE ROOT_ITEM 0) is
completely invalid, and will cause BUG_ON in process_inode_item().
Unfortunately, we are unable to fix when it happens.
So we can only output meaningful error message and avoid the insane
node/leaf, which is still much better than the original BUG_ON().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
It is highly obnoxious to have to go put in a testdev when all you really want
is to run the quick image tests. Make this part optional so if we don't have a
testdev specified we just don't run that particular test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If scrub is not cancelled nor finished, the recorded status will prevent
scrub to start again though it's not running. There's a force option to
run it anyway, but this is just a bandaid and the true status of scrub
should be detected automatically. The force option should not be
necessary anymore.
The test introduced in 9681f82853 checks only the status file,
not kernel status of scrub.
Signed-off-by: David Sterba <dsterba@suse.cz>
We should kill free_some_buffers() to stop reclaiming extent buffers or
we will hit a problem described below.
As of commit 53ee1bccf9, we are not
counting a reference for tree->lru anymore. However free_some_buffers()
is still left and is reclaiming extent buffers whose @refs == 1. This
cause extent buffers to be reclaimed unintentionally. Thus the following
steps could happen:
1. A buffer at address A is reclaimed by free_some_buffers()
(address A is also free()ed)
2. Some code call alloc_extent_buffer()
3. Address A is assigned to newly allocated buffer
4. You see a buffer pointed by A suddenly changed its content
This problem is also pointed out here and it has a reproducer:
https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg36703.html
This commit drop free_some_buffers() and related variables, and also it
modify extent_io_tree_cleanup() to catch non-free'ed buffers properly.
Signed-off-by: Naohiro Aota <naota@elisp.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
We have --init-csum-tree, which just empties the csum tree. I'm not sure why we
would ever need this, but we definitely need to be able to rebuild the csum tree
in some cases. This patch adds the ability to completely rebuild the crc tree
by reading all of the data and adding csum entries for them. This patch doesn't
pay attention to NODATASUM inodes, it'll happily add csums for everything.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
With the changes as in the previous patch, now scan_for_btrfs()
is an unused function. So delete it.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The libblkid scan method which was introduced later, will also
scan devices under /proc/partitions. So we don't have to do
the explicit scan of the same.
Remove the scan method BTRFS_SCAN_PROC.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
super-recover collects btrfs devices infomation using existed
functions scan_one_devices().
Problem is fs_devices is freed twice in close_ctree() and
free_recover_superblock() for super correction path.
Fix this problem by checking whether fs_devices memory
have been freed before we free it.
Cc: Eric Sandeen <sandeen@redhat.com>
Cc: Chris Murphy <lists@colorremedies.com>
Acked-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
(I am unable to reproduce the issue, tried to go back with progs versions
but still the same. So as of now this code remains untested, suggest to
wait till we have a reproducible test case).
Here is a test case which says it all..
mkfs.xfs -f $DEV
mkfs.btrfs -f $DEV
mount $DEV $MNT
mount: /dev/vdiskc: more filesystems detected. This should not happen,
use -t <type> to explicitly specify the filesystem type or
use wipefs(8) to clean up the device.
mount: you must specify the filesystem type
with this patch btrfs_prepare_device() also wipes old FS if any,
btrfs_prepare_device() is called after we have verified that
user has provided -f option.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The value of sector for space cache was hardcoded to 4k, and used to
calculate bitmap sizes. In kernel, the BITS_PER_BITMAP is derived from
PAGE_CACHE_SIZE which is not available for userspace, that can also deal
with filesystem of varying sectorsize.
Signed-off-by: David Sterba <dsterba@suse.cz>
When we have non-inlined extent references, we were failing to find the
corresponding extent item for an existing csum item in the csum tree.
Reproducer:
mkfs.btrfs -f /dev/sdd
mount /dev/sdd /mnt
xfs_io -f -c "falloc 780366 135302" /mnt/foo
xfs_io -c "falloc 327680 151552" /mnt/foo
xfs_io -c "pwrite -S 0xff -b 131072 0 131072" /mnt/foo
sync
for i in `seq 1 40`; do btrfs subvolume snapshot /mnt /mnt/snap$i ; done
umount /mnt
btrfs check /dev/sdd
The check command exited with status 1 and the following output:
Checking filesystem on /dev/sdd
UUID: 2416ab5f-9d71-457e-bb13-a27d4f6b399a
checking extents
checking free space cache
checking fs roots
checking csums
There are no extents for csum range 12980224-12984320
Csum exists for 12980224-12984320 but there is no extent record
found 1388544 bytes used err is 1
total csum bytes: 132
total tree bytes: 704512
total fs tree bytes: 573440
total extent tree bytes: 16384
btree space waste bytes: 564479
file data blocks allocated: 19341312
referenced 14606336
Btrfs v3.14.1-94-g80597e7
After this change it no longer erroneously reports a missing extent for the
csum item and exits with a status of 0.
Also added missing btrfs_prev_leaf() return value checks, as we were ignoring
errors and non-existence of left siblings completely.
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When encountering system crash or balance enospc errors,
there maybe still some reloc roots left.
The way we store reloc root is different from fs root:
reloc root's root key(BTRFS_RELOC_TREE_OBJECTID, ROOT_ITEM, objectid)
fs root's root key(objectid, ROOT_ITEM, -1)
reloc data's root key(BTRFS_DATA_RELOC_TREE_OBJECTID, ROOT_ITEM, 0)
So this patch use right key to search corresponding root node, and
avoid using normal fs root cache for reloc roots.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If btrfsck fail to repair, we hit something like following:
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
read block failed check_tree_block
found 98304 bytes used err is 1
total csum bytes: 0
total tree bytes: 0
total fs tree bytes: 0
total extent tree bytes: 0
btree space waste bytes: 0
file data blocks allocated: 0
referenced 0
Btrfs v3.14.2-rc2-63-g3944f15
btrfs: transaction.h:38: btrfs_start_transaction: Assertion `!(root->commit_root)' failed.
Aborted (core dumped)
This is because under repair mode, we will start a transaction, and if we error out,
we don't finish this transaction. So in close_ctree(), it will try
to start and commit transaction which causes the above segmentation.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Now btrfsck would hit assertation failure for some searching tree failure.
It is true that filesystem may get some metadata block corrupted,
and btrfsck could not deal with these corruptings. But, Users really
don't want a BUG_ON() here, Instead, just return errors to caller.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Repair mode will commit transaction which will make us
fail to load log tree anymore.
Give a warning to common users, if they really want to
coninue, we will clear out log tree.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This can not only give some speedups but also avoid forever loop
with a really broken filesystem.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
After the previous 2 patches, nothing uses
whole-dev-tree scanning, so remove the code which
implemented that functionality.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If we didn't find what we are looking for in /proc/partitions,
we're not going to find it by scanning every node under /dev, either.
But that's just what btrfs_scan_for_fsid() does.
Remove that fallback; at that point btrfs_scan_for_fsid() just calls
scan_for_btrfs(), so remove the wrapper & call it directly.
Side note: so, these paths always use /proc/partitions, not libblkid.
Userspace-intiated scans default to libblkid. I presume this is
part of the design, and intentional? Anyway, not changing it now!
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We can scan for btrfs devices in a few ways. By default
libblkid is used for "device scan" and "filesystem show";
with the -m option only mounted filesystems are scanned,
and with -d we physically read every system device.
But there's no reason for the complexity of a descent through
/dev; /proc/partitions has every device known to the kernel, so
just use that when -d is specified.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Enhance the 'subvolume' subcommand to wait until a given list of
subvolumes or all currently scheduled for deletion are cleaned
completely from the filesystem.
Signed-off-by: David Sterba <dsterba@suse.cz>
recover_prepare() in chunk-recover.c alloc memory which only contains
sizeof(struct btrfs_super_block). This will cause glibc malloc error
after superblock csum is calculated.
Use BTRFS_SUPER_INFO_SIZE to fix the bug.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
There's an off by one error in btrfs_check_leaf, we should be going to nritems -
1, not nritems - 2, we were missing problems with items in the very last slot.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
[BUG]
Some fsfuzzed btrfs image will cause btrfsck segfault.
[REPRODUCER]
Run btrfsck on a csum tree block corrupted image.
[REASON]
check_csums() function call btrfs_search_slot() on csum_tree but doesn't
check whether the csum_tree contains a valid extent_buffer, which causes
the segfault.
[FIX]
Check the csum_root->node before any search.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
A user reported a WARN_ON() when trying to run btrfsck --repair on his fs with
bad key ordering. This was because the root that was broken wasn't part of the
transaction yet. We do this open coded thing in a few other places in fsck, so
just make it a helper function and make sure all the places that need to call it
do call it. With this patch he was able to run repair without it dying.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We use the read extent buffer infrastructure to read the super block when we are
creating a btrfs-image. This works out fine most of the time except when the fs
has been balanced, then it fails to map the super block. So we could fix
btrfs-image to read in the super in a special way, but thats more code. So
instead just check in the eb reading code if we are reading the super and then
don't bother mapping the block, just read the actual offset. This fixed some
poor guy who was trying to btrfs-image his fs that had been balanced. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Asserting is no fun, we may be able to recover from this error in certain cases
(like btrfs-image and btrfsck). Just do what the kernel does and spit out an
error and return that there is only 1 copy. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Currently these macros just tie to assert(), which gives us line number and such
but no backtrace so no actual context. This patch adds support for spitting out
a backtrace so we can see how we got to the given assert. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
[backtrace_symbols_fd]
Signed-off-by: Naohiro Aota <naota@elisp.net>
[minor fixups]
Signed-off-by: David Sterba <dsterba@suse.cz>
Add human readable incompat flags output for btrfs-show-super,
now no longer needs to calculate the hex flags by hand.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The new option -f will force to do dangerous changes.
e.g. clear the seeding flag.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
[more text from 3db4c0a3d3 changelog]
Signed-off-by: David Sterba <dsterba@suse.cz>
Bug-Debian: http://bugs.debian.org/539433
Bug-Debian: http://bugs.debian.org/583768
Authors:
Luca Bruno <lucab@debian.org>
Alexander Kurtz <kurtz.alex@googlemail.com>
Daniel Baumann <daniel.baumann@progress-technologies.net>
Signed-off-by: Dimitri John Ledkov <xnox@debian.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
There are many trivial typos in Documentation/*.txt.
All of these use "exist status" to mean "exit status"
by mistake. I guess someone first made this mistake
and it has spread by copy-and-paste :-D
Signed-off-by: Naohiro Aota <naota@elisp.net>
Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
man 8 btrfs-property refers to `setattr(8)` which does not actually exist.
It should refer to `chattr (1)` instead.
Signed-off-by: Naohiro Aota <naota@elisp.net>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Before this patch, you could see the following after exec restore
# :too few arguments
The tool name "btrfs restore" is missing.
The @set_argv0() function is introduced by:
commit a184abc70f
btrfs-progs: move the check_argc_* functions into utils.c
...
Also add a new function "set_argv0" to set the correct tool name:
*btrfs-image*: too few arguments
But @set_argv0() only applies to the independent tools with
the name pattern btrfs-***.
Since restore is now is subcommand under "btrfs",
there is no need to use @set_argv0() before check_argc_* to
repair the prompt tool name before "too few arguments".
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
