Introduce 3 new members for btrfs_convert_context:
1) struct cache_tree used
Records accurate byte ranges which are used by old filesystem.
This will be used to create old filesystem image.
2) struct cache_tree data_chunks
Records batched ranges which must be covered by data chunks.
The bytenr range is optimized to meet all the chunk requirement.
3) u64 total_bytenr
Records how large the filesystem is in bytenr.
Yes, we can calculate it easy, but that's for old blocks based
filesystem.
This will make it more friendly for extent based filesystem.
And later cctx->block_counts and may be removed
And 2 for mkfs_config:
1) char *chunk_uuid.
Used as temporary chunk_uuid (unparsed) string for later
make_convert_btrfs()
2) u64 super_bytenr
Records the new temporary super bytenr after make_btrfs().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When creating small Btrfs filesystem instances (i.e. filesystem size <= 1GiB),
mkfs.btrfs fails if both sectorsize and nodesize are specified on the command
line and sectorsize != nodesize, since mixed block groups involves both data
and metadata blocks sharing the same block group. This is an incorrect behavior
when '-M' option isn't specified on the command line.
This commit makes optional the creation of mixed block groups i.e. Mixed block
groups are created only when -M option is specified on the command line.
Since we now allow small filesystem instances with sectorsize != nodesize to
be created, we can end up in the following situation,
[root@localhost ~]# mkfs.btrfs -f -n 65536 /dev/loop0
btrfs-progs v3.19-rc2-405-g976307c
See http://btrfs.wiki.kernel.org for more information.
Performing full device TRIM (512.00MiB) ...
Label: (null)
UUID: 49fab72e-0c8b-466b-a3ca-d1bfe56475f0
Node size: 65536
Sector size: 4096
Filesystem size: 512.00MiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 40.00MiB
System: DUP 12.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 512.00MiB /dev/loop0
[root@localhost ~]# mount /dev/loop0 /mnt/
mount: mount /dev/loop0 on /mnt failed: No space left on device
The ENOSPC occurs during the creation of the UUID tree. This is because of
things like large metadata block size, DUP mode used for metadata and global
reservation consuming space. Also, large nodesize does not make sense on small
filesystems, hence this should not be an issue.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use btrfs_open_dir() in open_path_or_dev_mnt() to make the function
return error when target is neither block device nor btrfs mount point.
Also add "verbose" argument to let function output common error
message instead of putting duplicated lines in caller.
Before patch:
# ./btrfs device stats /mnt/tmp1
ERROR: getting dev info for devstats failed: Inappropriate ioctl for device
# ./btrfs replace start /dev/vdd /dev/vde /mnt/tmp1
ERROR: ioctl(DEV_REPLACE_STATUS) failed on "/mnt/tmp1": Inappropriate ioctl for device
After patch:
# ./btrfs device stats /mnt/tmp1
ERROR: not a btrfs filesystem: /mnt/tmp1
# ./btrfs replace start /dev/vdd /dev/vde /mnt/tmp1
ERROR: not a btrfs filesystem: /mnt/tmp1
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current code use fprintf(stderr, "...") to output warnning and
error information.
The error message have different style, as:
# grep fprintf *.c
fprintf(stderr, "Open ctree failed\n");
fprintf(stderr, "%s: open ctree failed\n", __func__);
fprintf(stderr, "ERROR: cannot open ctree\n");
...
And sometimes, we forgot add tailed '\n', or use printf instead,
as in current code:
printf("warning, device %llu is missing\n",
This patch introduce warning() and error() as common function,
to make:
1: Each warning and error information have same format
2: Easy to search/change all error message
3: Easy to modify function's internal for debug or other requirement,
for example:
print function/linenumber in error()
dumpstack in error()
add some trace for some style of message
add support for -v, -vv, ...
support for locales
custom output functions
support some special device/tty
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[print newline after the message]
Signed-off-by: David Sterba <dsterba@suse.com>
Check nodesize against features, not only sectorsize.
In fact, one of the btrfs-convert and mkfs differs in the nodesize
check.
This patch also provides the basis for later btrfs-convert fix.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Old kernels before 3.9 do not provide ioctl BTRFS_IOC_GET_FSLABEL.
So we need to provide a fail safe logic for btrfs-progs running
on those kernel.
In this patch when get_label_mounted() fails on the old kernel
it will fail back to the old method and uses get_label_unmounted(),
where it will read from the disk directly.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We are using separate code for parsing unit mode in current code,
better to use common function.
This patch introduces a common function to specify units as arguments
and a common help message, to make every tool in btrfs having same unit
argument.
The benefits are:
1: Unify current tool's arguments for unit
2: Make tools in future easy to implement such argument
3: Changes (enhancement) in common function have effect on all
relative tools
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch introduce open_btrfs_dir() to open a dir in btrfs
filesystem.
It can be used for several tools in btrfs-progs.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[renamed from open_btrfs_dir, adjusted error messages]
Signed-off-by: David Sterba <dsterba@suse.com>
Anand reports that the static buffers used for pertty size strings cause
a stack overflow on SPARC. Zach proposed to change the printf format to
wrap the number and the suffix into a macro. This would require to
change all callsites of pretty_size* and is not very convienient to
write.
This patch replaces the per-call-site static buffers with a limited
number for slots that would be used on each invokation of pretty_size
and wrap around. The number of array slots shall be 10 for now, in
current codebase there are no more than 2 calls to pretty_size in a
single argument list.
Reported-by: Anand Jain <Anand.Jain@oracle.com>
CC: Zach Brown <zab@zabbo.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
The error string buffer passed as an argument is of a fixed size, though
we could print up to PATH_MAX + something bytes. Print the error message
directly.
Signed-off-by: David Sterba <dsterba@suse.cz>
The error string buffer passed as an argument is of a fixed size, though
we could print up to PATH_MAX + something bytes. Print the error message
directly.
Signed-off-by: David Sterba <dsterba@suse.cz>
The variable argv0_buf was duplicated and the changes done in utils.c
were not propagated to help.c. So if an unknown commandline token was
found, the error message did not contain the known part:
$ btrfs scrub test
: unknown token 'test'
instead of
$ btrfs scrub test
btrfs scrub: uknown token 'test'
Signed-off-by: David Sterba <dsterba@suse.cz>
Curretnly there are no features set for the converted filesystem and
it's not possible to request it like in mkfs.
Add new option -O|--features, use -O list-all to show all that are
supported for convert.
Note: from now on, convert without any -O option will use the same
defaults as mkfs. The original behaviour was rather confusing.
Convert is now more verbose about the parameters of the btrfs filesytem.
Signed-off-by: David Sterba <dsterba@suse.cz>
Currently BTRFS allows you to make bad choices of data and
metadata levels. For example -d raid1 -m raid0 means you can
only use half your total disk space, but will lose everything
if 1 disk fails. It should give a warning in these cases.
When making a filesystem, check that metadata mode is at least
as redundant as the data mode. For example give warning when:
-d raid1 -m raid0
Signed-off-by: Sam Tygier <samtygier@yahoo.co.uk>
[make the check more visible in mkfs output]
Signed-off-by: David Sterba <dsterba@suse.cz>
After Patch:
remove BTRFS_SCAN_PROC scan method
There isn't any consumer for btrfs_scan_block_devices() so delete it.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
mkfs and convert will not support the same features, -O will print only
the list according to the given mask.
Signed-off-by: David Sterba <dsterba@suse.cz>
Move the constant DEFAULT_MKFS_LEAF_SIZE to utils.h and rename it to
BTRFS_MKFS_DEFAULT_NODE_SIZE for consistency. Move the function
check_leaf_or_node_size to utils.c and rename it to
btrfs_check_node_or_leaf_size.
Signed-off-by: Sebastian Thorarensen <sebth@naju.se>
[added btrfs_ prefix]
Signed-off-by: David Sterba <dsterba@suse.cz>
The commit 1bad43fbe0 ("btrfs-progs: refine btrfs-debug-tree error
prompt when a mount point given")
add judgement on btrfs-debug-tree to restrict only block device to be
executed on, but the command can also be used on regular file, so add
regular file support for the judgement.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The check_arg_type() function does quite generic thing, move it to
utils.c.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Add count_digits() function in utils.h to help calculate filename with
ino suffix.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Enhance the command "btrfs filesystem df" to show space usage information
for a mount point(s). It shows also an estimation of the space available,
on the basis of the current one used.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
[code moved under #if 0 instead of deletion]
Signed-off-by: David Sterba <dsterba@suse.cz>
Make run from a long base path will overflow the argv0 buffer during
tests. Otherwise, this would happen for all the standalone binaries that
use set_argv0.
Original report:
https://bbs.archlinux.org/viewtopic.php?id=189861
Reported-by: WorMzy Tykashi <wormzy.tykashi@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs_scan_lblikd() is called by most the device related command functions.
And btrfs_scan_lblkid() is most expensive function and it becomes more expensive
as number of devices in the system increase. Further some threads call this
function more than once for absolutely no extra benefit and the real waste of
resources. Below list of threads and number of times btrfs_scan_lblkid()
is called in that thread.
btrfs-find-root 1
btrfs rescue super-recover 2
btrfs-debug-tree 1
btrfs-image -r 2
btrfs check 2
btrfs restore 2
calc-size NC
btrfs-corrupt-block NC
btrfs-image NC
btrfs-map-logical 1
btrfs-select-super NC
btrfstune 2
btrfs-zero-log NC
tester NC
quick-test.c NC
btrfs-convert 0
mkfs #number of devices to be mkfs
btrfs label set unmounted 2
btrfs get label unmounted 2
This patch will:
move out calling register_one_device with in btrfs_scan_lblkid()
and so function setting the BTRFS_UPDATE_KERNEL to yes will
call btrfs_register_all_devices() separately.
introduce a global variable scan_done, which is set when scan is
done succssfully per thread. So that following calls to this function
will just return success.
Further if any function needs to force scan after scan_done is set,
then it can be done when there is such a requirement, but as of now there
isn't any such requirement.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This function is to register all devices found after scanning
the system. Before we had this functionality with in the
btrfs_scan_lblkid(), however scanning and registering are two
different distinct operation its better keep them separate.
Also we want to optimize btrfs_scan_lblkid and avoid multiple
system scans unless needed. As of now device scan uses this function.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
cmd_scan_dev() has it own code to register device (calling ioctl
BTRFS_IOC_SCAN_DEV), apparently it could use btrfs_register_one_device().
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This change adds code to detect and fix the issue introduced in the kernel
release 3.17, where creation of read-only snapshots lead to a corrupted
filesystem if they were created at a moment when the source subvolume/snapshot
had orphan items. The issue was that the on-disk root items became incorrect,
referring to the pre orphan cleanup root node instead of the post orphan
cleanup root node.
A test filesystem can be generated with the test case recently submitted for
xfstests/fstests, which is essencially the following (bash script):
workout()
{
ops=$1
procs=$2
num_snapshots=$3
_scratch_mkfs >> $seqres.full 2>&1
_scratch_mount
snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
run_check $FSSTRESS_PROG -p $procs \
-x "$snapshot_cmd" -X $num_snapshots -d $SCRATCH_MNT -n $ops
}
ops=10000
procs=4
snapshots=500
workout $ops $procs $snapshots
Example of btrfsck's (btrfs check) behaviour against such filesystem:
$ btrfsck /dev/loop0
root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
Found 7 roots with an outdated root item.
Please run a filesystem check with the option --repair to fix them.
$ echo $?
1
$ btrfsck --repair /dev/loop0
enabling repair mode
fixing root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
fixing root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
fixing root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
fixing root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
fixing root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
fixing root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
fixing root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
Fixed 7 roots.
Checking filesystem on /dev/loop0
UUID: 2186e9b9-c977-4a35-9c7b-69c6609d4620
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 618537000 bytes used err is 0
total csum bytes: 130824
total tree bytes: 601620480
total fs tree bytes: 580288512
total extent tree bytes: 18464768
btree space waste bytes: 136939144
file data blocks allocated: 34150318080
referenced 27815415808
Btrfs v3.17-rc3-2-gbbe1dd8
$ echo $?
0
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The size unit format is a longstanding annoyance. This patch is based on
the work of Nils and Alexandre and enhances the options. It's possible
to select raw bytes, SI-based or IEC-based compact units (human
frientdly) or a fixed base from kilobytes to terabytes. The default is
compact human readable IEC-based, no change to current version.
CC: Nils Steinger <nst@voidptr.de>
CC: Alexandre Oliva <oliva@gnu.org>
Reviewed-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
The functionality of pretty unit printing was duplicated by
df_pretty_sizes, merge it with pretty_size and enhance the interface
with more suffix mode. Raw, binary or decimal.
Signed-off-by: David Sterba <dsterba@suse.cz>
With the changes as in the previous patch, now scan_for_btrfs()
is an unused function. So delete it.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The libblkid scan method which was introduced later, will also
scan devices under /proc/partitions. So we don't have to do
the explicit scan of the same.
Remove the scan method BTRFS_SCAN_PROC.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
After the previous 2 patches, nothing uses
whole-dev-tree scanning, so remove the code which
implemented that functionality.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
