Introduce new function raid5_gen_result() to calculate parity or data
stripe.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If this flag is passed to open_ctree(), we'll clear the
FREE_SPACE_TREE_VALID compat_ro bit. The kernel will then reconstruct
the free space tree the next time the filesystem is mounted.
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This allows us to put raid5 codes into that file other than creating a
new raid5.c.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The filesystem existence on a device is manifested by the signature,
during the mkfs process we write it first and then create other
structures. Such filesystem is not valid and should not be registered
during device scan nor listed among devices from blkid.
This patch will introduce two staged creation. In the first phase, the
signature is wrong, but recognized as a partially created filesystem (by
open or scan helpers). Once we successfully create and write everything,
we fixup the signature. At this point automated scanning should find
a valid filesystem on all devices.
We can also rely on the partially created filesystem to do better error
handling during creation. We can just bail out and do not need to clean
up.
The partial signature is '!BHRfS_M', can be shown by
btrfs inspect-internal dump-super -F image
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As we're passing a set of flags, the enum type is not appropriate.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new btrfsck option, '--chunk-root', to specify chunk root bytenr.
And allow open_ctree_fs_info() function accept chunk_root_bytenr to
override the bytenr in superblock. This will be mainly used when chunk
tree corruption.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since open_ctree_fs_info() now may return a fs_info even without any
roots, modify functions like read_tree_block() to operate with such
fs_info.
This provides the basis for btrfs-find-root to operate on chunk tree
with corrupted fs.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ coding style adjustments, unified declarations ]
Signed-off-by: David Sterba <dsterba@suse.com>
Current open_ctree_fs_info() won't return anything if chunk tree root is
corrupted.
This makes some function, like btrfs-find-root, unable to find any older
chunk tree root, even it is possible to use system_chunk_array in super
block.
And at least two users in mail list has reported such heavily chunk
corruption.
Although we have 'btrfs rescue chunk-recovery' but it's too time
consuming and sometimes not able to cope with a specific filesystem
corruption.
This patch adds a new open ctree flag,
OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR, allowing fs_info to be returned from
open_ctree_fs_info() even there is no valid tree root in it.
Also adds a new close_ctree() variant, close_ctree_fs_info() to handle
possible fs_info without any root.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ adjusted error messages ]
Signed-off-by: David Sterba <dsterba@suse.com>
Add includes that let the header files compile or add explicit include
of kerncompat if the uXX types are used.
Signed-off-by: David Sterba <dsterba@suse.cz>
Export write_tree_block() function and allow it write extent without
transaction.
This provides the basis for later uuid change function.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Now open_ctree will exit if it found the superblock is marked
CHANGING_FSID, except given IGNORE_FSID open ctree flags.
Kernel will do the same thing later.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[removed the chunk tree flag, reworded the error message]
Signed-off-by: David Sterba <dsterba@suse.cz>
Add new flag CHUNK_ONLY and internal used only flag __RETURN_CHUNK.
CHUNK_ONLY will imply __RETURN_CHUNK, SUPPRESS_ERROR and PARTIAL, which
will allow the fs to be opened with only chunk tree OK.
This will improve the usability for btrfs-find-root.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Add new open ctree flag OPEN_CTREE_SUPPRESS_CHECK_BLOCK_ERRORS to
suppress tree block csum error output.
Provides the basis for new btrfs-find-root and other enhancement on
btrfs offline tools output.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[renamed vars and funcs, added comments]
Signed-off-by: David Sterba <dsterba@suse.cz>
When we go to fixup the dev items after a restore we scan all existing devices.
If you happen to be a btrfs developer you could possibly open up some random
device that you didn't just restore onto, which gives you weird errors and makes
you super cranky and waste a day trying to figure out what is failing. This
will make it so that we use the fd we've already opened for opening our ctree.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Change the immediate number in btrfs_open_ctree_flags to bit shift.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
As of now commands mentioned below (with in [..]) are calling call register-device
ioctl BTRFS_IOC_SCAN_DEV for all the devices in the system.
Some issues with it:
BTRFS_IOC_SCAN_DEV: ioctl is a write operation, we don't want command like
btrfs-debug-tree threads to do that..
eg:
----
$ cat /proc/fs/btrfs/devlist | egrep fsid | wc -l
0
$ btrfs-debug-tree /dev/sde (num_device > 1)
$ cat /proc/fs/btrfs/devlist | egrep fsid | wc -l
5
----
btrfs_scan_fs_devices() ends up calling this ioctl only when num_device > 1.
That's inconsistency with in feature/bug.
We don't have to register _all_ the btrfs devices (again) in the system
without user consent.
Why its inconsistent:
function btrfs_scan_fs_devices() calls btrfs_scan_lblkid only when
num_devices is > 1, which in turn calls BTRFS_IOC_SCAN_DEV ioctl, if
conditions are met.
But main issue is we have too many consumers of btrfs_scan_fs_devices()
the names below with in [] is the cli leading to this function.
open_ctree_broken() [btrfs-find-root]
recover_prepare() [btrfs rescue super-recover]
__open_ctree_fd
(updates always except when flag OPEN_CTREE_RECOVER_SUPER is set and
flag OPEN_CTREE_RECOVER_SUPER is set only by 'btrfs rescue super-
recover' but still this thread sneaks through the open_ctree function
to call register-device-ioctl as show below).
open_ctree_fs_info
[btrfs-debug-tree]
[btrfs-image -r]
[btrfs check]
open_fs
[btrfs restore]
open_ctree
[calc-size]
[btrfs-corrupt-block]
[btrfs-image] (create)
[btrfs-map-logical]
[btrfs-select-super]
[btrfstune]
[btrfs-zero-log]
[tester]
[mkfs]
[quick-test.c]
[btrfs label set unmounted]
[btrfs get label unmounted]
[btrfs rescue super-recover]
open_ctree_fd
[btrfs-convert]
Fix:
In an effort to make register-device consistent, all calls to
btrfs_scan_fs_devices() will have 5th parameter set to 0. that means
we don't need 5th parameter at all. And with this function not calling
the register ioctl at all, finally we will have following two cli to call
the ioctl BTRFS_IOC_SCAN_DEV.
btrfs dev scan and
mkfs.btrfs
Threads needing to update kernel about a device would have to use
btrfs_register_one_device() separately.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David sent a quick patch that removed a BUG_ON(). I took a peek and
found that the function was already leaking an eb ref and only returned
0. So this fixes the leak and makes the function void and fixes up the
callers.
Accidentally-motivated-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Zach Brown <zab@zabbo.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs superblock checksum check is somewhat too restricted for
super-recover, since current btrfs-progs will only read the 1st
superblock and if you need super-recover the 1st superblock is
possibly already damaged.
The fix is introducing super_recover parameter for
btrfs_read_dev_super() and callers to allow scan backup superblocks if
needed.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
this patch will make btrfsck operations to open disk in exclusive mode,
so that mount will fail when btrfsck is running
Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
So I needed to add a flag to not try to read block groups when doing
--init-extent-tree since we could hang there, but that meant adding a whole
other 0/1 type flag to open_ctree_fs_info. So instead I've converted it all
over to using a flags setting and added the flag that I needed. This has been
tested with xfstests and make test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
In some cases the tree root is so hosed we can't get anything useful out of it.
So add the -b option to btrfsck to make us look for the most recent backup tree
root to use for repair. Then we can hopefully get ourselves into a working
state. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
mkfs -r wasn't creating chunks properly, making it very difficult to
allocate space for anything except tiny filesystems.
This changes it around to use more of the generic infrastructure, and
to do actual logical->physical block number translation.
It also allocates space to the files in smaller extents (max 1MB), which
keeps the allocator from trying to allocate an extent bigger than a
single chunk.
It doesn't quite support multi-device mkfs -r yet, but is much closer.
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Until now if one of device's first superblock is corrupt,btrfs will
fail to mount. Luckily, btrfs have at least two superblocks for
every disk.
In theory, if silent corrupting happens when we are writting superblocks
into disk, we must hold at least one good superblock.
One side effect is that user must gurantee that the disk must be
a btrfs disk. Otherwise, this tool may destroy other fs.(This is also
reason why btrfs only use first superblock in every disk to mount)
This little program will try to correct bad superblocks from
good superblocks with max generation.
There will be five kinds of return values:
0: all supers are valid, no need to recover
1: usage or syntax error
2: recover all bad superblocks successfully
3: fail to recover bad superblocks
4: abort to recover bad superblocks
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
If some fatal superblocks are damaged, running ioctl will return failure,
in this case, we should avoid run ioctl.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
In files copied from the kernel, mark many functions as static,
and remove any resulting dead code.
Some functions are left unmarked if they aren't static in the
kernel tree.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
For most time, In open_ctree_*(), we use the first superblock
(BTRFS_SUPER_INFO_OFFSET). However, for btrfs-convert, we don't,
we should pass the correct sb_bytenr to btrfs_scan_fs_devices() rather
than always use BTRFS_SUPER_INFO_OFFSET.This patch fix the following
regression:
mkfs.ext2 <dev>
btrfs-convert <dev>
warning, device 1 is missing
Check tree block failed, want=2670592, have=0
read block failed check_tree_block
Couldn't read chunk root
Segmentation fault (core dumped)
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
This adds a 'btrfs-image -m' option, which let us restore an image that
is built from a btrfs of multiple disks onto several disks altogether.
This aims to address the following case,
$ mkfs.btrfs -m raid0 sda sdb
$ btrfs-image sda image.file
$ btrfs-image -r image.file sdc
---------
so we can only restore metadata onto sdc, and another thing is we can
only mount sdc with degraded mode as we don't provide informations of
another disk. And, it's built as RAID0 and we have only one disk,
so after mount sdc we'll get into readonly mode.
This is just annoying for people(like me) who're trying to restore image
but turn to find they cannot make it work.
So this'll make your life easier, just tap
$ btrfs-image -m image.file sdc sdd
---------
then you get everything about metadata done, the same offset with that of
the originals(of course, you need offer enough disk size, at least the disk
size of the original disks).
Besides, this also works with raid5 and raid6 metadata image.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Add chunk-recover program to check or rebuild chunk tree when the system
chunk array or chunk tree is broken.
Due to the importance of the system chunk array and chunk tree, if one of
them is broken, the whole btrfs will be broken even other data are OK.
But we have some hint(fsid, checksum...) to salvage the old metadata.
So this function will first scan the whole file system and collect the
needed data(chunk/block group/dev extent), and check for the references
between them. If the references are OK, the chunk tree can be rebuilt and
luckily the file system will be mountable.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
In fact, the code of many rb-tree insert/search/delete functions is similar,
so we can abstract them, and implement common functions for rb-tree, and then
simplify them.
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
All we need for restore to work is the chunk root, the tree root and the fs root
we want to restore from. So to do this we need to make a few adjustments
1) Make open_ctree_fs_info fail completely if it can't read the chunk tree.
There is no sense in continuing if we can't read the chunk tree since we won't
be able to translate logical to physical blocks.
2) Use open_ctree_fs_info in restore, and if we didn't load a tree root or
fs root go ahead and try to set those up manually ourselves.
This is related to work I did last year on restore, but it uses the
open_ctree_fs_info instead of my open coded open_ctree. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
__setup_root() was present in find-root.c as well
as disk-io.c. No need for the cut and paste, just
use the one in disk-io.c
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Zach Brown <zab@redhat.com>
David Woodhouse originally contributed this code, and Chris Mason
changed it around to reflect the current design goals for raid56.
The original code expected all metadata and data writes to be full
stripes. This meant metadata block size == stripe size, and had a few
other restrictions.
This version allows metadata blocks smaller than the stripe size. It
implements both raid5 and raid6, although it does not have code to
rebuild from parity if one of the drives is missing or incorrect.
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
fsck needs to be able to open a damaged FS, which means open_ctree needs
to be able to return a damaged FS.
This adds a new open_ctree_fs_info which can be used to open any and all
roots that are valid. btrfs-debug-tree is changed to use it.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
Btrfs stores multiple copies of the superblock, and for common power-failure
crashes where barriers were not in use, one of the super copies is often
valid while the first copy is not.
This adds a btrfs-select-super -s N /dev/xxx command, which can
overwrite all the super blocks with a copy that you have already
determined is valid with btrfsck -s
Signed-off-by: Chris Mason <chris.mason@oracle.com>
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
The new back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer by
searching the tree. The shortcoming of the new back ref is that it only works
for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these fuzzy back
references would be O(number_of_snapshots) and quite slow. The solution used
here is to use the fuzzy back references in the common case where a given tree
block is only referenced by one root, and use the full back references when
multiple roots have a reference
This patch updates the ext3 to btrfs converter for the new
disk format. This mainly involves changing the convert's
data relocation and free space management code. This patch
also ports some functions from kernel module to btrfs-progs.
Thank you,
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
This patch updates btrfs-progs for superblock duplication.
Note: I didn't make this patch as complete as the one for
kernel since updating the converter requires changing the
code again. Thank you,
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Qu Wenruo
Omar Sandoval
David Sterba
Domagoj Tršan
Nicholas D Steeves
Lu Fengqi
David Sterba
Josef Bacik
Anand Jain
Zach Brown
Josef Bacik
Chris Mason
Wang Shilong
Eric Sandeen
Liu Bo
Miao Xie
David Woodhouse
Chris Mason
Josef Bacik
Yan Zheng