Commit Graph

3743 Commits

Author SHA1 Message Date
Qu Wenruo
4ab47d1e90 btrfs-progs: Allow __btrfs_map_block_v2 to remove unrelated stripes
For READ, caller normally hopes to get what they request, other than
full stripe map.

In this case, we should remove unrelated stripe map, just like the
following case:
               32K               96K
               |<-request range->|
         0              64k           128K
RAID0:   |    Data 1    |   Data 2    |
              disk1         disk2
Before this patch, we return the full stripe:
Stripe 0: Logical 0, Physical X, Len 64K, Dev disk1
Stripe 1: Logical 64k, Physical Y, Len 64K, Dev disk2

After this patch, we limit the stripe result to the request range:
Stripe 0: Logical 32K, Physical X+32K, Len 32K, Dev disk1
Stripe 1: Logical 64k, Physical Y, Len 32K, Dev disk2

And if it's a RAID5/6 stripe, we just handle it like RAID0, ignoring
parities.

This should make caller easier to use.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
2017-09-12 14:33:59 +02:00
Qu Wenruo
066745d697 btrfs-progs: Introduce new btrfs_map_block function which returns more unified result.
Introduce a new function, __btrfs_map_block_v2().

Unlike old btrfs_map_block(), which needs different parameter to handle
different RAID profile, this new function uses unified btrfs_map_block
structure to handle all RAID profile in a more meaningful method:

Return physical address along with logical address for each stripe.

For RAID1/Single/DUP (none-stripped):
result would be like:
Map block: Logical 128M, Len 10M, Type RAID1, Stripe len 0, Nr_stripes 2
Stripe 0: Logical 128M, Physical X, Len: 10M Dev dev1
Stripe 1: Logical 128M, Physical Y, Len: 10M Dev dev2

Result will be as long as possible, since it's not stripped at all.

For RAID0/10 (stripped without parity):
Result will be aligned to full stripe size:
Map block: Logical 64K, Len 128K, Type RAID10, Stripe len 64K, Nr_stripes 4
Stripe 0: Logical 64K, Physical X, Len 64K Dev dev1
Stripe 1: Logical 64K, Physical Y, Len 64K Dev dev2
Stripe 2: Logical 128K, Physical Z, Len 64K Dev dev3
Stripe 3: Logical 128K, Physical W, Len 64K Dev dev4

For RAID5/6 (stripped with parity and dev-rotation):
Result will be aligned to full stripe size:
Map block: Logical 64K, Len 128K, Type RAID6, Stripe len 64K, Nr_stripes 4
Stripe 0: Logical 64K, Physical X, Len 64K Dev dev1
Stripe 1: Logical 128K, Physical Y, Len 64K Dev dev2
Stripe 2: Logical RAID5_P, Physical Z, Len 64K Dev dev3
Stripe 3: Logical RAID6_Q, Physical W, Len 64K Dev dev4

The new unified layout should be very flex and can even handle things
like N-way RAID1 (which old mirror_num basic one can't handle well).

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
2017-09-12 14:33:59 +02:00
David Sterba
cb1be701ce
Btrfs progs v4.13
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:20:05 +02:00
David Sterba
3d341f5baa btrfs-progs: update CHANGES for v4.13
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro
a351dd8478 btrfs-progs: test: add new test for inspect-internal rootid
This new test checks inspect-internal rootid
 - handle path to subvolume/directory/file as an argument
 - get different id for each subvolume
 - get the expected id for each file/directory (i.e. the same as
	 containing subvolume)

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
6fb88e2859 btrfs-progs: tests: check for kernel support for reiserfs
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
7299e0d294 btrfs-progs: tests: enhance post-rollback fsck tests
The post-rollback helper still assumes just extN, we need an extra
argument that'll get passed to fsck. Change all callsites at once so the
tests do not fail temporarily.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Zhang Yu
e96921bcaa Btrfs-progs: print-tree: check num_stripes in print_chunk
[TEST/fuzz] case: 004-simple-dump-tree

Since the wrong key(DATA_RELOC_TREE CHUNK_ITEM 0) in root tree,
error calling print_chunk(), resulting in num_stripes == 0.

ERROR:
     [TEST/fuzz]   004-simple-dump-tree
ctree.h:317: btrfs_chunk_item_size: BUG_ON `num_stripes == 0`
        triggered, value 1

failed (ignored, ret=134): /myproject/btrfs-progs/btrfs
inspect-internal dump-tree
/myproject/btrfs-progs/tests/fuzz-tests/images/
bko-155201-wrong-chunk-item-in-root-tree.raw.restored

test failed for case 004-simple-dump-tree
Makefile:288: recipe for target 'test-fuzz' failed
make: *** [test-fuzz] Error 1

So, check on num_stripes in print_chunk

Signed-off-by: Zhang Yu <zhangyu-fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro
cb39164f9d btrfs-progs: test: fix name generation not to contain trailing spaces
First patch causes test-convert fails.  This is because
generate_dataset() creates a name containing trailing spaces for
"slow_symlink" type, and cause getfacl error in convert_test_perm().
(This is not noticed since original run_check_stdout() throws away the
error.)

Fix this by use space for delimiter for cut.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro
02e9bb9f23 btrfs-progs: test: fix run_check_stdout() call _fail()
run_check_stdout() uses "... | tee ... || _fail".  However, since tee
won't fail, _fail() is not called even if first command fails.

Fix this by checking PIPESTATUS in the end.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Qu Wenruo
dbe96ecd3f btrfs-progs: tests: Add test case for mkfs --rootdir parameter
Add test case which checks if -r|--rootdir mkfs option can handle
symlink/char/block/fifo files.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Qu Wenruo
081e4e9bb8 btrfs-progs: mkfs: Fix wrong file type for dir items and indexes when specifying root directory
[Bug]
If using mkfs.btrfs with "-r" parameter and specified directory has
fifo/socket/char/block special file, then created filesystem can't pass
fsck:

------
checking fs roots
	unresolved ref dir 241158 index 3 namelen 9 name S.dirmngr filetype 0 errors 80, filetype mismatch
ERROR: errors found in fs roots
------

[Reason]
Btrfs dir items/indexes records inode type, while "-r" only handles
directories, regular files and symlink, it makes such special files type
to be regular file and caused the problem.

[Fix]
Add missing types for add_directory_items(), so that result of
"mkfs.btrfs -r" can pass mkfs.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Gu Jinxiang
944e14da14 btrfs-progs: mkfs: Move the tree root creation to own function
make_btrfs is too long to understand, make creatation of root tree
in a function.

Some of the tree roots are now created in a loop, where the code is just
copypasted. We now make use of the reference_root_table to translate
block index to root objectid.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ updated changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
df4a04484a btrfs-progs: tests: missing device and slack space report
Verify that a missing device will not result in reporting a negative
value interpreted as 16EiB.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Patrik Lundquist
8c6b05d726 btrfs-progs: device usage: don't calculate slack on missing device
Print      Device slack:              0.00B
instead of Device slack:           16.00EiB

Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro
88ef0b8397 btrfs-progs: inspect rootid: Allow a file to be specified
Since cmd_inspect_rootid() calls btrfs_open_dir(), it rejects a file to
be specified. But as the document says, a file should be supported.

This patch introduces btrfs_open_file_or_dir(), which is a counterpart
of btrfs_open_dir(), to safely check and open btrfs file or directory.
The original btrfs_open_dir() content is moved to btrfs_open() and shared
by both function.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
f47587d83d btrfs-progs: tests: convert misc/011-delete-missing-device to loopdevs
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
83fe48c54b btrfs-progs: tests: convert misc/006-image-on-missing-device to loopdevs
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
528a5bf6ad btrfs-progs: tests: move loopdev helpers out of the testcase to common
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
3a6895b823 btrfs-progs: tests: cleanup loop device helpers
Make the loop device helpers a bit more generic before moving them to
the common helpers.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
36db1080c3 btrfs-progs: print-tree: factor out extent_csum dump
Factor out code to own helper and tweak the format so it matches the
rest.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Josef Bacik
5fbc00cc73 btrfs-progs: print the csum length in debug-tree
While looking at a log of a corrupted fs I needed to verify we were
missing csums for a given range.  Make this easier by printing out the
range of bytes a csum item covers.

Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
872837ebbf btrfs-progs: tests: add testcase for 'fi du' and empty subvol
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Goffredo Baroncelli
b5665d66e8 btrfs-progs: fi du: don't call lookup_path_rootid for BTRFS_EMPTY_SUBVOL_DIR_OBJECTID
When ino is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID, the item is not referred to
any file-tree. So lookup_path_rootid() doesn't return any meaningful
value.

As was reported, this can be triggered by

$ btrfs sub create test1
$ btrfs sub create test1/test2
$ btrfs sub snap test1 test1.snap
$ btrfs fi du -s test1
  Total   Exclusive  Set shared  Filename
  0.00B       0.00B       0.00B  test1
$ btrfs fi du -s test1.snap
  Total   Exclusive  Set shared  Filename
ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Goffredo Baroncelli
02d04d8b23 btrfs-progs: reset the ret value when ignoring an error from du_add_file
In du_walk_dir(), when du_add_file() returns an error it is usually
ignored. However if the error is returned querying the last item, the
error is returned to the caller.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
a36d92cb8b btrfs-progs: tests: add test for check --force
Basic test of the --force functionality, on an empty filesystem.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
162fdf9538 btrfs-progs: check: add option to skip mount checks
Sometimes it's needed to do a check on a mounted filesystem. This should
work fine on a quiescent filesystem or a read-only mount. Changes on the
block device done by kernel might confuse the userspace checker and it
might crash when it reads some stale data.

Repair without mount checks is not supported right now.

Signed-off-by: David Sterba <dsterba@suse.cz>
2017-09-08 16:15:05 +02:00
David Sterba
8609c8bad6 btrfs-progs: print-tree: factor out temporary_item dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
a4b65f00d5 btrfs-progs: print-tree: factor out persistent_item dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
61a578751d btrfs-progs: print-tree: factor out qgroup_limit dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
90631b721e btrfs-progs: print-tree: factor out qgroup_info dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
346d2e16dd btrfs-progs: print-tree: factor out qgroup_status dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
59714a77c3 btrfs-progs: print-tree: factor out dev_extent dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
c3b767a208 btrfs-progs: print-tree: factor out free_space_info dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
b3122697f6 btrfs-progs: print-tree: factor out shared_data_ref dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
c23c1271d3 btrfs-progs: print-tree: factor out extent_data_ref dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
e853a1abcd btrfs-progs: print-tree: factor out block_group_item dump
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
077a7024cd btrfs-progs: print-tree: don't print the item type for all items
In some cases it's clear from the context which item is being printed,
so we can remove them. If the item has no data, some description is
still desired (eg. orphan or various backrefs).

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
1224f7343e btrfs-progs: print-tree: rename item callbacks to match the key name
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
b3d343fc9c btrfs-progs: check: move fs roots check mode switch to a helper
Simplify main a bit.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
902535a951 btrfs-progs: check: cleanup use of fs_info in check_fs_roots
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
12fe1a57f6 btrfs-progs: check: pass fs_info to check_fs_roots
The root pointer is not used anyway, will be cleaned up next.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
3ca3f20071 btrfs-progs: check: move chunk and extent check mode switch to a helper
Simplify main a bit.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
b3e08c27e1 btrfs-progs: check: pass fs_info to check_chunks_and_extents_v2
The root pointer is set to fs_root as was originally.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
7e9a270a12 btrfs-progs: check: use local fs_info in check_chunks_and_extents
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
7349edaf28 btrfs-progs: check: pass fs_info to check_chunks_and_extents
The root pointer is set to fs_root as was originally.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
04f480bc01 btrfs-progs: check: verify critical root pointers before use
The pointers to critical roots must be valid before we start using them,
eg. as the space clearing code.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
f03b293fdf btrfs-progs: check: move more of space cache clearing to a helper
Move the code out of main.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
af3369ae31 btrfs-progs: check: remove old mixed backref code
A code added in 2009 (95d3f20b51) for a very short-lived change in
the format is no concern to us nowadays.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
4851de27f1 btrfs-progs: tests: add more sanitizer message patterns to log scanner
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00