When searching the extent tree for the target extent item, we can miss it
if the extent item is the first item in a leaf and if there is a previous
leaf in the extent tree.
For example, if we call btrfs-map-logical like this:
$ btrfs-map-logical -l 5382144 /dev/sdc
And we have the following extent tree layout:
leaf 5386240 items 26 free space 2505 generation 7 owner EXTENT_TREE
leaf 5386240 flags 0x1(WRITTEN) backref revision 1
(...)
item 25 key (5373952 METADATA_ITEM 0) itemoff 3155 itemsize 33
refs 1 gen 7 flags TREE_BLOCK
tree block skinny level 0
(176 0x5) tree block backref root FS_TREE
leaf 5480448 items 56 free space 276 generation 7 owner EXTENT_TREE
leaf 5480448 flags 0x1(WRITTEN) backref revision 1
(...)
item 0 key (5382144 METADATA_ITEM 0) itemoff 3962 itemsize 33
refs 1 gen 7 flags TREE_BLOCK
tree block skinny level 0
(176 0x7) tree block backref root CSUM_TREE
(...)
Then the following happens:
1) We enter map_one_extent() with search_forward == 0 and
*logical_ret == 5382144;
2) We search for the key (5382144 0 0) which leaves us with a path
pointing to leaf 5386240 at slot 26 - one slot beyond the last item;
3) We then call:
btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0])
Which is not valid since there's no item at that slot, but since the
area of the leaf where an item at that slot should be is zeroed out,
we end up getting a key of (0 0 0);
4) We then enter the "if" statement bellow, since key.type is 0, and call
btrfs_previous_extent_item(), which leaves at slot 25 of leaf 5386240,
point to the extent item of the extent 5373952.
The requested extent, 5382144, is the first item of the next leaf
(5480448), but we totally miss it;
5) We return to the caller, the main() function, with 'cur_logical'
pointing to the metadata extent at 5373952, and not to the requested
one at 5382144.
In the last while loop of main() we have 'cur_logical' == 5373952,
which makes the loop have no iterations and therefore the local
variable 'found' remains with a value of 0, and then the program fails
like this:
$ btrfs-map-logical -l 5382144 /dev/sdc
ERROR: no extent found at range [5382144,5386240)
Fix this by never accessing beyond the last slot of a leaf. If we ever end
up at a slot beyond the last item in a leaf, just call btrfs_next_leaf()
and process the first item in the returned path.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function and it's related functions only exist for the utilities
that populate existing file systems, and do not exist in the upstream
kernel. Move this function and the related function into it's own
common source file and out of the kernel-shared sources, and then update
all of the users to include the new location of this code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The radix-tree is not used in userspace code. In kernel it's for
tracking unpersisted and in-memory structures and has been replaced by
the xarray.
Signed-off-by: David Sterba <dsterba@suse.com>
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers
Signed-off-by: David Sterba <dsterba@suse.com>
The function read_extent_from_disk() is only a wrapper to read tree
block.
And read_extent_data() is just a while loop to eliminate short read
caused by stripe boundary.
In fact, a lot of call sites of read_extent_data() are either reading
metadata (thus no possible short read) or doing extra loop by
themselves.
This patch will replace those two functions with read_data_from_disk(),
making it the only entrance for data/metadata read.
And update read_data_from_disk() to return the read bytes, so caller can
do a simple while loop.
For the few callers of read_extent_data(), open-code a small while loop
for them.
This will allow later RAID56 read repair using P/Q much easier.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When we switch to multiple global trees we'll need to access the
appropriate extent root depending on the block group or possibly root.
To handle this, use a helper in most places and then the actual root in
places where it is required. We will whittle down the direct accessors
with future patches, but this does the bulk of the preparatory work.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The error message about the unsatisfied argument count is scrolled away
by the full usage string dump. This is not considered a good usability
practice.
This commit switches all direct usage -> return patterns, where the
argument check has no other constraint, eg. dependency on an option.
Signed-off-by: David Sterba <dsterba@suse.com>
Similar to the changes where strerror(errno) was converted, continue
with the remaining cases where the argument was stored in another
variable.
The savings in object size are about 4500 bytes:
$ size btrfs.old btrfs.new
text data bss dec hex filename
805055 24248 19748 849051 cf49b btrfs.old
804527 24248 19748 848523 cf28b btrfs.new
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_next_extent_item() looks for BTRFS_EXTENT_ITEM_KEY and
BTRFS_METADATA_KEY, which are the types we're looking for.
Signed-off-by: James Harvey <jamespharvey20@gmail.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Just to keep the 1st paramter the same as kernel.
We can also save a few lines since the parameter is shorter now.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Large numbers like (1024 * 1024 * 1024) may cost reader/reviewer to
waste one second to convert to 1G.
Introduce kernel include/linux/sizes.h to replace any intermediate
number larger than 4096 (not including 4096) to SZ_*.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
I have a valid btrfs image which contains,
...
item 10 key (1103101952 BLOCK_GROUP_ITEM 1288372224) itemoff 15947 itemsize 24
block group used 655360 chunk_objectid 256 flags DATA|RAID5
item 11 key (1103364096 EXTENT_ITEM 131072) itemoff 15894 itemsize 53
extent refs 1 gen 11 flags DATA
extent data backref root 5 objectid 258 offset 0 count 1
item 12 key (1103888384 EXTENT_ITEM 262144) itemoff 15841 itemsize 53
extent refs 1 gen 15 flags DATA
extent data backref root 1 objectid 256 offset 0 count 1
item 13 key (1104281600 EXTENT_ITEM 262144) itemoff 15788 itemsize 53
extent refs 1 gen 15 flags DATA
extent data backref root 1 objectid 257 offset 0 count 1
...
The extent [1103364096, 131072) has length 131072, but if we run
"btrfs-map-logical -l 1103364096 -b $((65536 * 3)) /dev/sda"
it will return mapping info 's of non-existing extents.
It's because it assumes that extents's are contiguous on logical address,
when it's not true, after one loop (cur_logical += cur_len) and mapping
the next extent, we can get an extent that is out of our search range and
we end up with a negative @real_len and printing all mapping infos till
the disk end.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nodesize is used in kernel, the values are always equal. We have to keep
leafsize in headers, similarly the tree setting functions still take and
set leafsize, but it's effectively a no-op.
Signed-off-by: David Sterba <dsterba@suse.com>
This patch add all missing btrfs_close_all_devices() to standalone
tools in btrfs progs, to avoid memory leak.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In btrfs-map-logical main(), strdup() allocates memory to output_file,
but that memory is not freed.
So add missing free() calls before return.
Signed-off-by: Byongho Lee <bhlee.kernel@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
The original map-logical has the following problems:
1) Assert if we pass any tree root bytenr.
The problem is easy to trigger, here the number 29622272 is the bytenr of tree root:
# btrfs-map-logical -l 29622272 /dev/sda6
mirror 1 logical 29622272 physical 38010880 device /dev/sda6
mirror 2 logical 29622272 physical 1111752704 device /dev/sda6
extent_io.c:582: free_extent_buffer: Assertion `eb->refs < 0` failed.
btrfs-map-logical[0x41c464]
btrfs-map-logical(free_extent_buffer+0xc0)[0x41cf10]
btrfs-map-logical(btrfs_release_all_roots+0x59)[0x40e649]
btrfs-map-logical(close_ctree+0x1aa)[0x40f51a]
btrfs-map-logical(main+0x387)[0x4077c7]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f80a5562790]
btrfs-map-logical(_start+0x29)[0x4078f9]
The problem is that, btrfs-map-logical always use sectorsize as default
block size to call alloc_extent_buffer.
And when it failes to find the block with the same size, it will free
the extent buffer in a incorrect method(Free and create a new one with
refs == 1).
2) Will return map result for non-exist extent.
# btrfs-map-logical -l 1 -b 123456 /dev/sda6
mirror 1 logical 1 physical 1 device /dev/sda6
mirror 1 logical 4097 physical 4097 device /dev/sda6
mirror 1 logical 8193 physical 8193 device /dev/sda6
...
Normally, before bytenr 12582912, there should be no extent as that's
the mkfs time temp metadata/system chunk.
But map-logical will still map them out.
Not to mention the 1 offset among all results.
[FIX]
This patch will rework the whole map logical by the following methods:
1) Always do things inside a extent
Even under the following case, map logical will only return covered
range in existing extents.
|<------ range given ------->|
|<-Extent A->| |<-Extent B->| |<---Extent C->|
Result:
|<-->| |<---------->| |<-->|
So with this patch, we will search extent tree to ensure all operation
are inside a extent before we do some stupid things.
2) No direct call on alloc_extent_buffer function.
That low-level function shouldn't be called at such high level.
It's only designed for low-level tree operation.
So in this patch we will only use safe high level functions avoid such
problem.
[RESULT]
With this patch, no assert will be triggered and better handle on
non-exist extents.
# btrfs-map-logical -l 29622272 /dev/sda6
mirror 1 logical 29622272 physical 38010880 device /dev/sda6
mirror 2 logical 29622272 physical 1111752704 device /dev/sda6
# btrfs-map-logical -l 1 -b 123456 /dev/sda6
No extent found at range [1,123457)
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This function will write extent content info desired file.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The new function will print the mapping info of given range
[logical, logical+len).
Note, caller must ensure the ranges are completely inside an extent.
Or btrfs_map_block can return -ENOENT.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Introduce the function to get accurate extent length based on extent
tree search.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We're not using it anywhere. The best practice is to add enums with
values > 255 for the long options, option index counting is error prone.
Signed-off-by: David Sterba <dsterba@suse.cz>
- use standard PACKAGE_{NAME,VERSION,STRING,URL,...} autoconf macros
rather than homemade BTRFS_BUILD_VERSION
- don't #include version.h, now the file is necessary for library API only
Note that "btrfs version" returns "btrfs-progs <version>" instead of
the original confusing "btrfs <version>".
Signed-off-by: Karel Zak <kzak@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
glibc 2.10+ (5+ years old) enables all the desired features:
_XOPEN_SOURCE 700, __XOPEN2K8, POSIX_C_SOURCE, DEFAULT_SOURCE; with a
single _GNU_SOURCE define in the makefile alone. For portability to
other libc implementations (e.g. dietlibc) _XOPEN_SOURCE=700 is also
defined.
This also resolves Debian bug report filed by Michael Tautschnig -
"Inconsistent use of _XOPEN_SOURCE results in conflicting
declarations". Whilst I was not able to reproduce the results, the
reported fact is that _XOPEN_SOURCE set to 500 in one set of files
(e.g. cmds-filesystem.c) generates/defines different struct stat from
other files (cmds-replace.c).
This patch thus cleans up all feature defines, and sets them at a
consistent level.
Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=747969
Signed-off-by: Dimitri John Ledkov <dimitri.j.ledkov@intel.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Filipe Manana
David Sterba
Josef Bacik
Qu Wenruo
james harvey
Qu Wenruo
Liu Bo
Zhao Lei
Byongho Lee
David Sterba
Karel Zak
Dimitri John Ledkov