This is a starting point for a debugfs style python interface using
the search ioctl. For now it can only do one thing, which is to
print out all the extents in a file and calculate the compression ratio.
Over time it will grow more features, especially for the kinds of things
we might run btrfs-debug-tree to find out. Expect the usage and output
to change dramatically over time (don't hard code to it).
Signed-off-by: Chris Mason <clm@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The @fi_args->num_devices in @get_fs_info() does not include seed devices.
We could just correct it by searching the chunk tree and count how
many dev_items there are in total which includes seed devices.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The following BUG_ON:
BUG_ON(ndevs >= fi_args->num_devices)
is not needed, because it always fails with seed devices present.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
There is no need to try to build seed/sprout mapping for those btrfs
without seed devices, so just skip such fs.
We could get the total number of devices from the disk super block, if it
equals the number of items in list @fs_devices->devices, then there shouldn't
be any seed devices.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Before the patch, chunk will be considered bad if the corresponding
block group is missing, even the only uncertain data is the 'used'
member of the block group.
This patch will try to recalculate the 'used' value of the block group
and rebuild it.
So even only chunk item and dev extent item is found, the chunk can be
recovered.
Although if extent tree is damanged and needed extent item can't be
read, the block group's 'used' value will be the block group length, to
prevent any later write/block reserve damaging the block group.
In that case, we will prompt user and recommend them to use
'--init-extent-tree' to rebuild extent tree if possible.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Extract the procedure of searching for a target device for fi show
from the @map_seed_devices() function to make it more clear.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This patch reworks the basic calculations of 'fi usage'. It does not address
all problems but should make the code more prepared to do so.
The original code tries to estimate the free space that could lead to negative
numbers for some raid profiles:
Data, RAID1: total=147.00GiB, used=141.92GiB
System, RAID1: total=32.00MiB, used=36.00KiB
Metadata, RAID1: total=2.00GiB, used=1.17GiB
GlobalReserve, single: total=404.00MiB, used=0.00B
Overall:
Device size: 279.46GiB
Device allocated: 298.06GiB
Device unallocated: 16.00EiB
Used: 286.18GiB
Free (estimated): 8.00EiB (min: 8.00EiB)
Data ratio: 2.00
Metadata ratio: 2.00
Global reserve: 404.00MiB (used: 0.00B)
Eg. "Device size" - "Device allocated" = negative number or a very large
positive, hence the EiB values.
There are logical and raw numbers multiplied by ratios mixed together,
so the new code makes it explicit which kind is being used. The data and
metadata ratios are calculated separately.
Output after this patch will look like:
Overall:
Device size: 558.92GiB
Device allocated: 298.06GiB
Device unallocated: 260.86GiB
Used: 286.18GiB
Free (estimated): 135.51GiB (min: 135.51GiB)
Data ratio: 2.00
Metadata ratio: 2.00
Global reserve: 404.00MiB (used: 0.00B)
Data,RAID1: Size:147.00GiB, Used:141.92GiB
/dev/sdc 147.00GiB
/dev/sdd 147.00GiB
Metadata,RAID1: Size:2.00GiB, Used:1.17GiB
/dev/sdc 2.00GiB
/dev/sdd 2.00GiB
System,RAID1: Size:32.00MiB, Used:36.00KiB
/dev/sdc 32.00MiB
/dev/sdd 32.00MiB
Unallocated:
/dev/sdc 130.43GiB
/dev/sdd 130.43GiB
Changes:
* Device size is now the raw size, same for the following three
* Free is the logical size
* Max/min were reduced to just min
Filesystem Size Used Avail Use% Mounted on
/dev/sdc 280G 144G 141G 51% /mnt/sdc
The difference between Avail and Free is there because userspace tool does a
different guesswork than kernel.
Issues not addressed by this patch:
* RAID56 profiles are not handled
* mixed profiles are not handled
Signed-off-by: David Sterba <dsterba@suse.cz>
Even if run as root:
# su
# btrfs file usage <path> <== path exits outside the mnt point
We get the output:
WARNING: ..., run as root
WARNING: ..., run as root
ERROR:...
It is because in load_chunk_info, the errno of ioctl is not judged
but rather the ret value of ioctl is judged. And the ret value of
ioctl is -1 which happens to match -EPERM exactly.
So the outer warning is printed.
Just judge the errno of ioctl and prevent the ret value of load_chunk_info
to be -1 in other error conditions.
For load_device_info, the problem and fix is the same.
After the fix, the 'run as root' WARNINGs will not show up in this condition.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When we exec the following cmd:
# btrfs file usage -t <path> <-- an invalid path
output:
# ERROR： can't access '-t'
should be:
# ERROR: can't access 'path'
Just replace the static 'argv[1]' with 'argv[i]'.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When run btrfs-file-usage on a btrfs with data profile raid5/6,
the output message for "Free" & "Data to device ratio" seems wrong
as follows:
...
Device size: 100.00GiB
Device allocated: 2.04GiB
Device unallocated: 97.96GiB
Used: 1.12MiB
Free (Estimated): 197.89GiB <== Free > Device size
Data to device ratio: 198 % <== > 100%
Global reserve: 0.00B
...
It is because the function get_raid56_used() is not iterating the
chunk_info array correctly, it is just repeating adding the first
chunk_info statistics.
Just add a ptr to iterate over the array.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Current error messages are like following:
Error: unable to create FS with metadata profile 32 (have 2 devices)
Error: unable to create FS with metadata profile 256 (have 2 devices)
Obviously it is hard for users to interpret "profile XX" to proper
meaning, such as "raidN". So use recongizable string instead of
internal numerical value. In case of "DUP", use an explicit message.
Plus this patch fix a bug that message mistake metadata profile
for data profile.
After applying this patch, messages will be like:
Error: DUP is not allowed when FS have multiple devices
Error: unable to create FS with metadata profile RAID6 (have 2
devices but 3 devices are required)
Signed-off-by: Hidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The term has not seen an agreement and we don't want to change it once
it's in non-development branches or even released.
Discussion under the patch:
http://thread.gmane.org/gmane.comp.file-systems.btrfs/34627
Signed-off-by: David Sterba <dsterba@suse.cz>
It looks confusing among the chunks, it is not in fact a chunk type.
Sample:
Overall:
Device size: 35.00GiB
Device allocated: 8.07GiB
Device unallocated: 26.93GiB
Used: 1.12MiB
Free (Estimated): 17.57GiB (Max: 30.98GiB, min: 17.52GiB)
Data to device ratio: 50 %
Global reserve: 16.00MiB (used: 0.00B)
...
Signed-off-by: David Sterba <dsterba@suse.cz>
The main point of this is to load the device and chunk infos at one
place and pass down to the printers. The EPERM is handled separately, in
case kernel does not give us all the information about chunks or
devices, but we want to warn and print at least something.
For non-root users, 'filesystem usage' prints only the overall stats and
warns about RAID5/6.
The sole cleanup changes affect mostly the modified code and the related
functions, should be reasonably small.
Signed-off-by: David Sterba <dsterba@suse.cz>
The 'fi usage' lacks an overall report, this used to be in the enhanced
df command. Add it back.
Sample:
Overall:
Device size: 35.00GiB
Device allocated: 8.07GiB
Device unallocated: 26.93GiB
Used: 1.12MiB
Free (Estimated): 17.57GiB (Max: 30.98GiB, min: 17.52GiB)
Data to device ratio: 50 %
...
Signed-off-by: David Sterba <dsterba@suse.cz>
The device may not be fully occupied by the filesystem, the value of
Unallocated should not be calculated against the device size but the
size provided by DEV_INFO.
Signed-off-by: David Sterba <dsterba@suse.cz>
The entire device size may not be available to the filesystem, eg. if
it's modified via resize. Print this information if it can be obtained
from the DEV_INFO ioctl.
Print the device ID on the same line as the device name and move size to
the next line.
Sample:
/dev/sda7, ID: 3
Device size: 10.00GiB
FS occupied: 5.00GiB
Data,RAID10: 512.00MiB
Metadata,RAID10: 512.00MiB
System,RAID10: 4.00MiB
Unallocated: 9.00GiB
Signed-off-by: David Sterba <dsterba@suse.cz>
The TREE_SEARCH ioctl is root-only, FS_INFO will be available for
non-root users with an updated kernel, let the user know.
Signed-off-by: David Sterba <dsterba@suse.cz>
Move the command definitions where they belong, keep common 'usage'
functions in cmds-fi-disk_usage.c and add exports.
Rename structures containing 'disk' to 'device'.
Fix whitespace in the modified code.
Signed-off-by: David Sterba <dsterba@suse.cz>
Add back the original output of the 'btrfs fi df' command for backward
compatibility. The rich output is moved from 'disk_usage' to 'usage'.
Agreed in http://www.spinics.net/lists/linux-btrfs/msg31698.html
Signed-off-by: David Sterba <dsterba@suse.cz>
Lets not assign *info_ptr to 0 before calling free on it and lose
track of already allocated memory if realloc fails in
add_info_to_list. Lets call free first.
Signed-off-by: Rakesh Pandit <rakesh@tuxera.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The usage() calls exit() internally, so remove the return after it.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This patch adds some functions to manage the printing of the data in
tabular format.
The function
struct string_table *table_create(int columns, int rows)
creates an (empty) table.
The functions
char *table_printf(struct string_table *tab, int column,
int row, char *fmt, ...)
char *table_vprintf(struct string_table *tab, int column,
int row, char *fmt, va_list ap)
populate the table with text. To align the text to the left, the text
shall be prefixed with '<', otherwise the text shall be prefixed by a
'>'. If the first character is a '=', the the text is replace by a
sequence of '=' to fill the column width.
The function
void table_free(struct string_table *)
frees all the data associated to the table.
The function
void table_dump(struct string_table *tab)
prints the table on stdout.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.cz>
Enhance the command "btrfs filesystem df" to show space usage information
for a mount point(s). It shows also an estimation of the space available,
on the basis of the current one used.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
[code moved under #if 0 instead of deletion]
Signed-off-by: David Sterba <dsterba@suse.cz>
Before this patch, when btrfsck found an error in root dir, it will only
output the following message "root %llu root dir %llu error" without any
detailed error.
Just add print_inode_error() to print out the whole error.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
resolve_one_root() returns the objectid of a tree rather than the logical
address of the root node. Hence using root_bytenr is misleading. Fix this.
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
For now,
# btrfs fi show /mnt/btrfs
gives info correctly, while
# btrfs fi show /mnt/btrfs/
gives nothing.
This implies that the @realpath() function should be applied to
unify the behavior.
Made a more clear comment right above the call as well.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If you exec:
# btrfs sub show <dir> <== non-subvolume dir
The cmd print error messages as expected, but returns 0.
By convetion, it should return non-zero and we should explicitly
set it before it goto out.
With other pieces adopted:
1) removed a unnecessary return value set -EINVAL
2) fixed another code branch which may return 0 upon error.
3) with 2) applied, the ret = 0 follows can be removed
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When converting a sparse ext* filesystem, btrfs-convert adds checksum extents
for empty extents, whose disk_bytenr = 0, and this can end up with some weird
problems, one of them is the failure of reading free space cache inode on
mounting converted btrfs.
The fix is simple, just to skip making checksum on empty extents.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
A new dev replace result was introduced by kernel commit
Btrfs: return failure if btrfs_dev_replace_finishing() failed
Make the userspace know about the new result too.
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We use the attr version provided by system in other places already,
now we can remove dependency on the separate attr library.
Signed-off-by: David Sterba <dsterba@suse.cz>
gcc 4.9.0 gives a warning: array subscript is above array bounds
Checking for "greater or equal" instead of just "equal" fixes this.
The warning is a false positive, appears with -ftree-vrp, but we'd
rather fix it to avoid noise during build.
http://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg34338.html
Signed-off-by: Christian Hesse <mail@eworm.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Make run from a long base path will overflow the argv0 buffer during
tests. Otherwise, this would happen for all the standalone binaries that
use set_argv0.
Original report:
https://bbs.archlinux.org/viewtopic.php?id=189861
Reported-by: WorMzy Tykashi <wormzy.tykashi@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When using lvm volumes to check fstests: btrfs/006, it fails like:
Label: 'TestLabel.006' uuid: <UUID>
Total devices <EXACTNUM> FS bytes used <SIZE>
devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+ devid <DEVID> size <SIZE> used <SIZE> path /dev/dm-4
+ devid <DEVID> size <SIZE> used <SIZE> path /dev/dm-5
+ devid <DEVID> size <SIZE> used <SIZE> path /dev/dm-6
The /dev/dm-* points to lvm volumes, use @canonicalize_path() to convert them
and we will make it through. Of course we should do the same thing for dev stat.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When using btrfs check with -s option, if using '-s 2' on a small
device which doesn't have the third superblock, "No valid Btrfs found"
will be output, but it is not appropriate.
So check sb_bytenr against device size before scanning a device and
output proper error message.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
These test the recreating of missing dir item/dir index pairs, fixing the no
rootdir inode item and no inode item for normal files. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
If we just don't have the root dirid stuff go ahead and re-create it, since it
is easily recreated. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If we have all the other items but no inode item we can recreate it for the most
part, with the exception of the permissions and ownership. Add this ability to
btrfsck. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If we have everything except the dir item and dir index we can easily replace
them, so add this ability to btrfsck. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If there are errors when opening the fs because of PARTIAL we could think that
the zero-log didn't actually work. Add a printf so we know that it was
successfull. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
