Before this patch, you could see the following after exec restore
# :too few arguments
The tool name "btrfs restore" is missing.
The @set_argv0() function is introduced by:
commit a184abc70f
btrfs-progs: move the check_argc_* functions into utils.c
...
Also add a new function "set_argv0" to set the correct tool name:
*btrfs-image*: too few arguments
But @set_argv0() only applies to the independent tools with
the name pattern btrfs-***.
Since restore is now is subcommand under "btrfs",
there is no need to use @set_argv0() before check_argc_* to
repair the prompt tool name before "too few arguments".
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When entering the next level node, the @next_leaf in restore forgets to
start at the first slot. Just reset it to the first one.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Steps to reproduce:
# mkfs.btrfs -f <dev>
# mount -o compress-force=lzo <dev> <mnt>
# for ((i=0;i<4000;i++)); do
echo -n 'A' >> <mnt>/inline_data
done
# umount <mnt>
# valgrind --tool=memcheck --leak-check=full \
btrfs restore <dev> <dest_dir>
output:
==32118== Invalid read of size 1
==32118== at 0x4A0A4E4: memcpy@@GLIBC_2.14
==32118== by 0x43DC91: read_extent_buffer
==32118== by 0x421401: search_dir (cmds-restore.c:240)
==32118== by 0x422CBB: cmd_restore (cmds-restore.c:1317)
==32118== by 0x404709: main (btrfs.c:248)
==32118== Address 0x4c4f4ac is not stack'd, malloc'd or...
It is because when deal with inline extent, the read_extent_buffer
is now reading a len of @ram_bytes which is the len of the uncompressed
data. But actually here we want the len of the inline item.
So in the compressed situation, use the len of the inline item.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
A memory problem reported by valgrind as follows:
=== Syscall param pwrite64(buf) points to uninitialised byte(s)
When running:
# valgrind --leak-check=yes btrfs restore /dev/sda9 /mnt/backup
Because the output buf size is alloced with malloc, but the length of
output data is shorter than the sizeof(buf), so valgrind report
uninitialised byte(s).
We could use calloc to repalce malloc and clear this WARNING away.
Reported-by: Marc Dietrich <marvin24@gmx.de>
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The restore tool should only print info of the restoring process
in verbose mode with -v option specified.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The three flags of @btrfs_path:
btrfs_path {
unsigned int keep_locks:1;
unsigned int skip_locking:1;
unsigned int leave_spinning:1;
}
have little meaning, because the userspace @btrfs_search_slot()
is free of locking and no other routines will decide their behavior
on these. So just remove them.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If you are using btrfs restore to try to recover a very large or
fragmented file, you may encounter _lots_ of prompts requiring
you to press 'y' to continue because we are looping a lot.
Add the option to press 'a', to supress these prompts for the rest
of the file.
Signed-off-by: Justin Maggard <jmaggard10@gmail.com>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Fix following build warnings on 32bit platform:
...
utils.c:1708:3: warning: left shift count >= width of
type [enabled by default]
if (x << i & (1UL << 63))
^
qgroup-verify.c:393:9: warning: cast to pointer from integer
of different size [-Wint-to-pointer-cast]
return (struct tree_block *)unode->aux;
^
qgroup-verify.c:407:38: warning: cast from pointer to integer
of different size [-Wpointer-to-int-cast]
if (ulist_add(tree_blocks, bytenr, (unsigned long long)block, 0) >= 0)
^
cmds-restore.c:120:4: warning: format %lu expects argument of type
long unsigned int, but argument 3 has type size_t [-Wformat=]
fprintf(stderr, "bad compress length %lu\n", in_len);
...
BTW, this patch also switches other castings with new helpers.
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Since this patch:
btrfs-progs: move the check_argc_* functions into utils.c
All tools including the independent tools(e.g. btrfs-image, btrfs-convert)
can share the convenience of the check_argc_* functions, so this patch
adopt the argc check functions globally.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Add some missing options, also improve some confusing
expressions.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
These two options are used for same purpose, but they are exclusive with
each other. Make it clear to common users.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Previously if restore could not read users specified fs root, it would
output following message:
Error reading root
With this patch, it will output message like:
Fail to read root 1000: No such file or directory
Signed-off-byr Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Steps to reproduce:
# mkfs.btrfs -f /dev/sda9
# mount /dev/sda9 /mnt
# dd if=/dev/zero of=/mnt/data bs=1M count=1
# btrfs restore -r /dev/sda9 -r 2 -o /tmp
If users don't input a valid fs/file root objectid, btrfs restore still
continue and don't restore anything, this is unfriendly, we could
check it firstly.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When things go wrong for lzo-compressed btrfs, feeding lzo1x_decompress_safe()
with corrupt data during restore can lead to crashes. Reduce the risk by adding
a check on the input length.
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
The following kernel commit changed the definition of the inline function
btrfs_file_extent_inline_len():
commit 514ac8ad8793a097c0c9d89202c642479d6dfa34
Author: Chris Mason <clm@fb.com>
Date: Fri Jan 3 21:07:00 2014 -0800
Btrfs: don't use ram_bytes for uncompressed inline items
If we truncate an uncompressed inline item, ram_bytes isn't updated to reflect
the new size. The fixe uses the size directly from the item header when
reading uncompressed inlines, and also fixes truncate to update the
size as it goes.
Not having this new definition implies that the restore tool might misbehave when
restoring files with an inline extent that got truncated on a kernel older than
release 3.14.
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We need to read a number of bytes corresponding to the disk size of the file extent
item, and not to the number of bytes in the num_bytes field. Normally disk_size is
smaller than num_bytes (when using compression), except for files created with lzo
compression in a kernel older then the one which introduced the following change:
commit 59516f6017c589e7316418fda6128ba8f829a77f
Author: Stefan Agner <stefan@agner.ch>
Date: Mon Jul 1 20:33:39 2013 +0200
Btrfs: return -1 when lzo compression makes data bigger
With this fix the lzo code behaves like the zlib code by returning an
error
code when compression does not help reduce the size of the file.
This is currently not a bug since the compressed size is checked again
in
the calling method compress_file_range.
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The `btrfs` and `mkfs.btrfs` binaries are not linked against libattr
so the correct header to include is <sys/xattr.h>.
This fixes the build when attr header files are not installed.
Signed-off-by: Christophe Vu-Brugier <cvubrugier@yahoo.fr>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
The code was incorrectly adding the file extent items' data offset to the logical
disk address of the extent (bytenr) when the extent is compressed. The offset is
relative to the uncompressed data and not to what we store on disk (compressed).
Also it attempted to copy ram_bytes to destination, which is incorrect when the
data offset field is non-zero, it must use num_bytes instead.
A test case for xfstests follows.
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
switch to arg_strtou64 plus some cleanups to remove unnecessary
codes.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
Sometimes it is useful to see what btrfs restore is going to do
before provisioning enough external storage to restore onto.
Add a dry-run option so we can see what files and paths are found
by restore, without actually restoring any data.
Signed-off-by: Justin Maggard <jmaggard10@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
So I needed to add a flag to not try to read block groups when doing
--init-extent-tree since we could hang there, but that meant adding a whole
other 0/1 type flag to open_ctree_fs_info. So instead I've converted it all
over to using a flags setting and added the flag that I needed. This has been
tested with xfstests and make test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
In some cases the tree root is so hosed we can't get anything useful out of it.
So add the -b option to btrfsck to make us look for the most recent backup tree
root to use for repair. Then we can hopefully get ourselves into a working
state. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Remove unused eb parameter from btrfs_item_nr, unused since introduced
in 7777e63b42
Signed-off-by: Ross Kirk <ross.kirk@gmail.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
When there are files that have parts shared with snapshots, the
restore command was incorrectly restoring them, as it was not
taking into account the offset and number of bytes fields from
the file extent item. Besides leaving the recovered file corrupt,
it was also inneficient as it read and wrote more data than needed
(with each extent copy overwriting portions of the one previously
written).
The following steps show how to reproduce this corruption issue:
$ mkfs.btrfs -f /dev/sdb3
$ mount /dev/sdb3 /mnt/btrfs
$ perl -e '$d = "\x41" . ("\x00" x (1024*1024+349)); open($f,">","/mnt/btrfs/foobar"); print $f $d; close($f);'
$ du -b /mnt/btrfs/foobar
1048926 /mnt/btrfs/foobar
$ md5sum /mnt/btrfs/foobar
f9f778f3a7410c40e4ed104a3a63c3c4 /mnt/btrfs/foobar
$ btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/my_snap
$ perl -e 'open($f, "+<", "/mnt/btrfs/foobar"); seek($f, 4096, 0); print $f "\xff"; close($f);'
$ md5sum /mnt/btrfs/foobar
b983fcefd4622a03a78936484c40272b /mnt/btrfs/foobar
$ umount /mnt/btrfs
$ btrfs restore /dev/sdb3 /tmp/copy
$ du -b /tmp/copy/foobar
1048926 /tmp/copy/foobar
$ md5sum /tmp/copy/foobar
88db338cbc1c44dfabae083f1ce642d5 /tmp/copy/foobar
$ od -t x1 -j 8192 -N 4 /tmp/copy/foobar
0020000 41 00 00 00
0020004
$ mount /dev/sdb3 /mnt/btrfs
$ od -t x1 -j 8192 -N 4 /mnt/btrfs/foobar
0020000 00 00 00 00
0020004
$ md5sum /mnt/btrfs/foobar
b983fcefd4622a03a78936484c40272b /mnt/btrfs/foobar
Tested this change with zlib, lzo compression and file sizes larger
than 1GiB, and found no regression or other corruption issues (so far
at least).
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
These were mostly in option structs but there were a few gross string
pointer arguments given as 0.
Signed-off-by: Zach Brown <zab@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
This fixes all the instances of warnings that symbols declared in blocks
shadow symbols with the same name in surrounding scopes:
cmds-device.c:341:22: warning: symbol 'path' shadows an earlier one
cmds-device.c:285:14: originally declared here
I just renamed or removed the risky shadow symbols instead of pulling
their blocks out into functions.
Signed-off-by: Zach Brown <zab@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Mark many functions as static, and remove any resulting dead code.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Port of commit b3b4aa7 to userspace.
parameter tree root it's not used since commit
5f39d397dfbe140a14edecd4e73c34ce23c4f9ee ("Btrfs: Create extent_buffer
interface for large blocksizes")
This gets userspace a tad closer to kernelspace by removing
this unused parameter that was all over the codebase...
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Update the usage strings of some cmds to keep the them consistent with
the source.
Also some minor changes are done to fit the man page syntax.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
This change adds a new option to the restore command, named -x,
that makes it restore file extented attributes too. This is an
optional behaviour and it's disabled by default.
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Mention that a target path argument is mandatory unless
the -l option is supplied. Also mention about the existence
of the -l option, which was previously not announced.
$ btrfs restore -v /dev/sdb3
usage: btrfs restore [options] <device>
Try to restore files from a damaged filesystem (unmounted)
-s get snapshots
-v verbose
-i ignore errors
-o overwrite
-t tree location
-f <offset> filesystem location
-u <block> super mirror
-d find dir
$ echo $?
129
After specifying a target path, the command works as expected:
$ btrfs restore -v /dev/sdb3 files2/
Restoring files2/file1
Done searching
$ echo $?
0
V2: Updated command synopsis by suggestion of Anand Jain.
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Current way of specifying the path to match is not very comfortable, but
the feature itself is very useful. Let's save the short option -m for a
more user friendly syntax and keep a long option --path-regex with the
current syntax.
CC: Peter Stuge <peter@stuge.se>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
We were unconditionally executing our regular expression, even though we may not
have one, so check to make sure mreg is not null before calling regexec.
Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
The option -m is used to specify the regex string. -c is used to
specify case insensitive matching. -i was already taken.
In order to restore only a single folder somewhere in the btrfs
tree, it is unfortunately neccessary to construct a slightly
nontrivial regex, e.g.:
restore -m '^/(|home(|/username(|/Desktop(|/.*))))$' /dev/sdb2 /output
This is needed in order to match each directory along the way to the
Desktop directory, as well as all contents below the Desktop directory.
Signed-off-by: Peter Stuge <peter@stuge.se>
Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
We have a BUG_ON() in read_fs_root that expects key->offset == (u64)-1. Restore
will just pass in the location it reads out of the inode if it has a subvol
reference and pass that in, which doesn't have offset == (u64)-1, so it causes
restore blow up. This was reported in bugzilla and this patch fixed the
problem. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
See the warnings below:
[CC] btrfs-list.o
btrfs-list.c: In function 'filter_by_parent':
btrfs-list.c:1183:34: warning: cast to pointer from integer of different size
[-Wint-to-pointer-cast]
[CC] cmds-subvolume.o
cmds-subvolume.c: In function 'cmd_subvol_show':
cmds-subvolume.c:917:5: warning: cast from pointer to integer of different size
[-Wpointer-to-int-cast]
[CC] cmds-restore.o
cmds-restore.c: In function 'decompress_lzo':
cmds-restore.c:116:10: warning: passing argument 4 of 'lzo1x_decompress_safe'
from incompatible pointer type [enabled by default]
In file included from cmds-restore.c:31:0:
/usr/include/lzo/lzo1x.h:77:1: note: expected 'lzo_uint *' but argument is of
type 'size_t *'
Reported-by: Russell Coker <russell@coker.com.au>
Signed-off-by: Wang Shilong <wangsl-fnst@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
All we need for restore to work is the chunk root, the tree root and the fs root
we want to restore from. So to do this we need to make a few adjustments
1) Make open_ctree_fs_info fail completely if it can't read the chunk tree.
There is no sense in continuing if we can't read the chunk tree since we won't
be able to translate logical to physical blocks.
2) Use open_ctree_fs_info in restore, and if we didn't load a tree root or
fs root go ahead and try to set those up manually ourselves.
This is related to work I did last year on restore, but it uses the
open_ctree_fs_info instead of my open coded open_ctree. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Since restore has the ability to open really really screwed up file systems, add
a list roots option to it so we can still get the contents of the tree root on a
horribly broken fs. Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
search_dir() recurses down the btrfs tree, and used to take the output
path for every item (i.e. in the running system, output root directory
concatenated with btrfs-local pathname) passed as the only path
parameter. Moving the output root directory to a separate parameter
and passing the btrfs-local pathname for each file and directory
separately allows easy filtering based on the btrfs-local pathname.
Signed-off-by: Peter Stuge <peter@stuge.se>
Signed-off-by: Josef Bacik <josef@redhat.com>
If we hit a bad disk and the read doesn't work, try other mirrors in case we
have other disks with good copies. Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
This will make the restore program fall back on other mirrors if it fails to
decompress an extent for whatever reason. Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
If the normal fs tree is hosed and the user has multiple subvolumes it's handy
to be able to specify just one of the subvolumes to restore. It's also handy if
a user only wants to restore say /home instead of his entire disk. Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
