[BUG]
There is a report about failed btrfs-convert, which shows the following
error:
Create btrfs metadata
corrupt leaf: root=5 block=5001931145216 slot=1 ino=89911763, invalid previous key objectid, have 89911762 expect 89911763
leaf 5001931145216 items 336 free space 7 generation 90 owner FS_TREE
leaf 5001931145216 flags 0x1(WRITTEN) backref revision 1
fs uuid 8b69f018-37c3-4b30-b859-42ccfcbe2449
chunk uuid 448ce78c-ea41-49f6-99dc-46ad80b93da9
item 0 key (89911762 INODE_REF 3858733) itemoff 16222 itemsize 61
index 171 namelen 51 name: [FILENAME1]
item 1 key (89911763 INODE_REF 3858733) itemoff 16161 itemsize 61
index 103 namelen 51 name: [FILENAME2]
[CAUSE]
When iterating a directory, btrfs-convert would insert the DIR_ITEMs,
along with the INODE_REF of that inode.
This leads to above stray INODE_REFs, and trigger the tree-checker.
This can only happen for large fs, as for most cases we have all these
modified tree blocks cached, thus tree-checker won't be triggered.
But when the tree block cache is not hit, and we have to read from disk,
then such behavior can lead to above tree-checker error.
[FIX]
Insert a dummy INODE_ITEM for the INODE_REF first, the inode items would
be updated when iterating the child inode of the directory.
Issue: #731
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function and it's related functions only exist for the utilities
that populate existing file systems, and do not exist in the upstream
kernel. Move this function and the related function into it's own
common source file and out of the kernel-shared sources, and then update
all of the users to include the new location of this code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This simply zero's out the path, and this is used everywhere we use a
stack path. Drop this usage and simply init the path's to empty instead
of using a function to do the memset.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are cases where the BUG_ON should be replaced by error
handling as it's validating the data from the source filesystem or
possibility to convert. The unconverted cases are asserts and will be
replaced later.
Signed-off-by: David Sterba <dsterba@suse.com>
We currently open code a similar operation in create_image_file_range().
By exposing intersect_with_reserved() outside of source-fs.c and
slightly changing its semantics to return the entire range instead of
just the end address, we can reuse it in create_image_file_range().
Author: Thomas Hebb <tommyhebb@gmail.com>
Pull-request: #494
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
intersect_with_reserved() currently succeeds if (bytenr + num_bytes) is
greater than or equal to the first address in the range, assuming that
bytenr is also not past the end of the range.
This is wrong. (bytenr + num bytes) is one byte past the last address in
the range we're checking, meaning that our range only overlaps the
reserved range if it's strictly greater than the reserved range's start
address.
For example, imagine a range at 0x3000 with length 0x1000 that we're
checking against a reserved range that starts at 0x4000. The addresses
in our range are 0x3000-0x3fff: it doesn't overlap. But the current
check, (0x3000 + 0x1000 >= 0x4000), will erroneously pass.
Fix the issue by changing >= to >.
Issue: #297
Issue: #349
Author: Thomas Hebb <tommyhebb@gmail.com>
Pull-request: #494
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers
Signed-off-by: David Sterba <dsterba@suse.com>
Now if an ENOSPC error happened, the free space report would help user
to determine if it's a real ENOSPC or a bug in convert.
The reported free space is the calculated free space, which doesn't
include super block space, nor merged data chunks.
The free space is always smaller than the reported available space of
the original fs, as we need extra padding space for used space to avoid
too fragmented data chunks.
The output would be:
$ ./btrfs-convert /dev/sda
create btrfs filesystem:
blocksize: 4096
nodesize: 16384
features: extref, skinny-metadata (default)
checksum: crc32c
free space report:
total: 10737418240
free: 0 (0.00%)
ERROR: unable to create initial ctree: No space left on device
WARNING: an error occurred during conversion, the original filesystem is not modified
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ put total, free to separate lines ]
Signed-off-by: David Sterba <dsterba@suse.com>
This would sync the code between kernel and btrfs-progs, and save at
least 1 byte for each btrfs_block_group_cache.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Create directory for all sources that can be used by anything that's not
rellated to a relevant kernel part, all common functions, helpers,
utilities that do not fit any other specific category.
The traditional location would be probably lib/ with all things that are
statically linked to the main binaries, but we have libbtrfs and
libbtrfsutil so this would be confusing.
Signed-off-by: David Sterba <dsterba@suse.com>
When pread64() returns value smaller than expected, it normally means
EIO, so just return -EIO to replace the intermediate number. So when IO
fails, we should be able to get more meaningful error number of than
EPERM.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Commit 522ef705e3 (btrfs-progs: convert: Introduce function to calculate
the available space) changed how we handle migrating file data so that
we never have btrfs space associated with the reserved ranges. This
works pretty well and when we iterate over the file blocks, the
associations are redirected to the migrated locations.
This commit missed the case in block_iterate_proc where we just check
for intersection with a superblock location before looking up a block
group. intersect_with_sb checks to see if the range intersects with
a stripe containing a superblock but, in fact, we've reserved the
full 0-1MB range at the start of the disk. So a file block located
at e.g. 160kB will fall in the reserved region but won't be excepted
in block_iterate_block. We ultimately hit a BUG_ON when we fail
to look up the block group for that location.
This is reproducible using convert-tests/003-ext4-basic.
The fix is to have intersect_with_sb and block_iterate_proc understand
the full size of the reserved ranges. Since we use the range to
determine the boundary for the block iterator, let's just return the
boundary. 0 isn't a valid boundary and means that we proceed normally
with block group lookup.
Cc: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since btrfs_reserved_ranges array is just used to store btrfs reserved
ranges, no one will nor should modify them at run time, make them static
and const will be better.
This also eliminates the use of immediate number 3.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ definition stays in source-fs.c ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new strucutre, simple_range, to present one contingous
range.
Also, use such structure to define btrfs_reserved_ranges(), which
convert and rollback will use.
Suggested-by: David Sterba <dsterba@suse.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ split hunks to new file structure ]
Signed-off-by: David Sterba <dsterba@suse.com>
The context is zeroed in convert_open_fs after and overwrites the rbtree
initialization, which accidentally is the same (NULL).
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo
David Sterba
Josef Bacik
Thomas Hebb
Jeff Mahoney
Qu Wenruo