Commit Graph

3276 Commits

Author SHA1 Message Date
Qu Wenruo
45f782abf3 btrfs-progs: fsck: Fix patch allocation check and leak in check_fs_first_inode
Allocated 'path' in check_fs_first_inode() is not checked and for
btrfs_search_slot() error, it will leak 'path'.

Fix it.

Resolves-Coverity-CID: 1374098
Resolves-Coverity-CID: 1374099
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:34 +01:00
Qu Wenruo
8b125cdd7e btrfs-progs: check: Enhance leaf traversal function to handle missing inode item
The leaf traversal function in lowmem mode will skip to the first inode
item of leaf and begin to check the inode.

That's designed to avoid checking overlapping part of a leaf.

But that will cause problem in fsck/010 test case, as in that case inode
item of the first inode(256) is missing.
So above traversal will check from inode 257 and found nothing wrong.

The fix is done in 2 part:
1) Manually check the first inode
   To avoid case like fsck/010

2) Check inode if ino changes from the first ino of a leaf
   To avoid missing inode_item problem.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:34 +01:00
Wang Xiaoguang
5e2dc77047 btrfs-progs: check: skip shared node or leaf check for low_memory mode
The basic idea is simple. Assume a middle tree node A is shared and
its referencing fs/file tree root ids are 5, 258 and 260, then we
only check node A in the tree who has the smallest root id. That means
in this case, when checking root tree(5), we check inode A, for root
tree 258 and 260, we can just skip it.

Notice even with this patch, we still may visit a shared node or leaf
multiple times. This happens when a inode metadata occupies multiple
leaves.

                 leaf_A     leaf_B
When checking inode item in leaf_A, assume inode[512] have file extents
in leaf_B, and leaf_B is shared. In the case, for inode[512], we must
visit leaf_B to have inode item checked. After finishing inode[512] check,
here we walk down tree root to leaf_B to check whether node or leaf
is shared, if some node or leaf is shared, we can just skip it and below
nodes or leaf's check.

I also fill a disk partition with linux source codes and create 3
snapshots in it. Before this patch, it averagely took 46s to finish one
btrfsck execution, with this patch, it averagely took 15s.

Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
d1e8227ceb btrfs-progs: check: fix the return value bug of cmd_check()
The function cmd_check() is called by the main function of btrfs.c, its
return value will be returned by exit(). Resulting in the loss of
significant bits in some cases, for example this value is greater than
0377. If use a bool value "err" to store all of the return value, this
will solve the problem.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
21ca38e7df btrfs-progs: check: introduce low_memory mode fs_tree check
Introduce a new function check_fs_roots_v2() for check fs_tree in
low_memory mode. It call check_fs_root_v2() to check fs_root, and call
check_root_ref() to check root_ref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
2f94517bc4 btrfs-progs: check: introduce function to check root ref
Introduce a new function check_root_ref() to check
root_ref/root_backref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
262425b650 btrfs-progs: check: introduce function to check fs root
Introduce a new function check_fs_root_v2() to check fs root,
and call check_inode_item to check the items in the tree.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
b1d1f627ce btrfs-progs: check: introduce function to check inode item
Introduce a new function check_inode_item() to check INODE_ITEM and
related ITEMs that have the same inode id.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
d88da10ddd btrfs-progs: check: introduce function to check file extent
Introduce a new function check_file_extent() to check file extent,
such as datasum, hole, size.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
52bfe9ef78 btrfs-progs: check: introduce function to check dir_item
Introduce a new function check_dir_item() to check DIR_ITEM/DIR_INDEX,
and call find_inode_ref() to find the related INODE_REF/INODE_EXTREF.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
cf4ada1f13 btrfs-progs: check: introduce function to find inode_ref
Introduce a new function find_inode_ref() to find
INODE_REF/INODE_EXTREF for the given key, and check it with the
specified DIR_ITEM/DIR_INDEX match.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
ec7b85da52 btrfs-progs: check: introduce function to check inode_extref
Introduce a new function check_inode_extref() to check INODE_EXTREF, and
call find_dir_item() to find the related DIR_ITEM/DIR_INDEX.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
6c9569fac2 btrfs-progs: check: introduce function to check inode_ref
Introduce a new function check_inode_ref() to check INODE_REF,
and call find_dir_item() to find the related DIR_ITEM/DIR_INDEX.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Lu Fengqi
7c7deecda9 btrfs-progs: check: introduce function to find dir_item
Introduce a new function find_dir_item() to find DIR_ITEM for the given
key, and check it with the specified INODE_REF/INODE_EXTREF match.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Qu Wenruo
c00278a9e8 btrfs-progs: volumes: Use new raid5_gen_result to calculate raid5 parity
Use thew raid5_gen_result() function to calculate raid5 parity.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
Qu Wenruo
ef67f448c7 btrfs-progs: raid56: Introduce new function to calculate raid5 parity or data stripe
Introduce new function raid5_gen_result() to calculate parity or data
stripe.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:33 +01:00
David Sterba
144a19145e
Btrfs progs v4.8.5
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 13:06:08 +01:00
David Sterba
52a9f80f17 btrfs-progs: update CHANGES for v4.8.5
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 13:04:34 +01:00
David Sterba
917f2c36e1 btrfs-progs: test: fix static build of library-test
The static variant of the test was broken, but not really used anyway.
Use the right compilation and linking flags.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 12:42:28 +01:00
David Sterba
18b1364e68 btrfs-progs: tests: add correct rpath to library-test
Our library-test dynamically linked binary was not using the built
libbtrfs. To fix that, use -rpath.

Reported-by: Mike Gilbert <floppymaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 12:42:27 +01:00
David Sterba
9d133083aa btrfs-progs: tests: end of stream conditions
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 12:42:24 +01:00
David Sterba
71f220034d btrfs-progs: receive: properly detect end of stream conditions
Read buffer helper mistakenly reported end of data as an error. Next, we
have to check if the first stream exists as an empty file is not a valid
stream.

Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 12:42:22 +01:00
David Sterba
d960f81d8e btrfs-progs: docs: fix typo in btrfs-man5
Reported via IRC.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-30 12:42:09 +01:00
David Sterba
3ae37eaad1
Btrfs progs v4.8.4
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-25 09:58:48 +01:00
David Sterba
6aea3ddac9 btrfs-progs: update CHANGES for v4.8.4
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-25 09:56:36 +01:00
David Sterba
e11315343f btrfs-progs: build: link against local libraries
If there's a system provided libbtrfs.0, it will be used for the
library-test build and could miss linker errors, as reported.

Reported-by: Mike Gilbert <floppymaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
15f2582e54 btrfs-progs: test: extend library-test coverage
Add test for linkage error of lookup_path_rootid. The built binary has
to be run to catch also run-time errors.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
e5362f5feb btrfs-progs: move lookup_path_rootid to library utils
The refactoring in commit 1c85c3de5a has
broken use of libbtrfs that does not exhibit during build but at the run
time.

Fixes: 1c85c3de5a
Bugzilla: https://bugs.gentoo.org/show_bug.cgi?id=600078
Reported-by: Mike Gilbert <floppymaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
f8278f94ff btrfs-progs: print-tree: fix format string on 32bit
We're passing a size_t, use the right format specifier.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
90969c8897 btrfs-progs: move bitops.h to kernel-lib
Forgotten when the rest got moved.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
78939e5fe5 btrfs-progs: deprecate standalone btrfs-show-super
The functionality is now provided by 'inspect dump-super'.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
74a95ec84c btrfs-progs: deprecate standalone btrfs-calc-size
The functionality is now provided by 'inspect tree-stats'.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:36 +01:00
David Sterba
e5b7a27bf7 btrfs-progs: move btrfs-image sources to own directory
There are no pending patches to btrfs-image, this looks like a good time
to move it.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-24 13:06:29 +01:00
David Sterba
50e742a6b0 btrfs-progs: tests: add support for additional command arguments
Add convenient support for extending command arguments, now implemented
for 'btrfs check' to cover the low-memory mode. If defined, arguments
are inserted to any 'btrfs check' command in tests. Exceptions could be
defined in common.local.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:06 +01:00
David Sterba
55db7bf059 btrfs-progs: tests: more variable quoting in common scripts
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:06 +01:00
David Sterba
c1d714f944 btrfs-progs: tests: drop redundant test for rwx number validity
The list of rwx permissions is now hardcoded but used to begenerated and
the invalid numbers filtered out. Not necessary anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:06 +01:00
Tsutomu Itoh
b0d8975582 btrfs-progs: test: fix how to make test files in fsck-tests 013
In my test environment, following error was occurred because the size
of /lib/modules/`uname -r`/* is larger than 1GB.

# make test-fsck
    [TEST]   fsck-tests.sh
    [TEST/fsck]   013-extent-tree-rebuild
failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
test failed for case 013-extent-tree-rebuild
Makefile:272: recipe for target 'test-fsck' failed
make: *** [test-fsck] Error 1
#

In this test case, 'generate_dataset small' is enough for making the
test files, so I will use 'generate_dataset' instead of 'cp'.

For this, move 'generate_dataset()' from 'common.convert' to 'common'.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:06 +01:00
David Sterba
d8f93ce802 btrfs-progs: Update README and other docs
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
776cd164b7 btrfs-progs: check: fix missing newlines
Also, the other progress messages go to stderr, so "checking extents"
probably should, as well.

Fixes: c7a1f66a20 ("btrfs-progs: check: switch some messages to common helpers")
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Tsutomu Itoh
fc4a828061 btrfs-progs: qgroup: fix error in ASSERT condition expression
Option -f, -F and --sort don't work because a conditional expression
of ASSERT is wrong.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
David Sterba
d4ce61f7a2 btrfs-progs: tests: add fallback to current directory for check_all_images
Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Tsutomu Itoh
3a8234ee73 btrfs-progs: test: fix error of test target of Makefile
Add test-cli to test target of Makefile.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
a064f24e2e btrfs-progs: document space_cache=v2 more thoroughly
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
87895ef4da btrfs-progs: implement btrfs check --clear-space-cache v2
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
[ adjusted error messages ]
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
9814411021 btrfs-progs: add btrfs_clear_free_space_tree() from the kernel
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
441d8aea8f btrfs-progs: add OPEN_CTREE_INVALIDATE_FST flag
If this flag is passed to open_ctree(), we'll clear the
FREE_SPACE_TREE_VALID compat_ro bit. The kernel will then reconstruct
the free space tree the next time the filesystem is mounted.

Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
9e02fbfcd3 btrfs-progs: format FREE_SPACE_TREE{,_VALID} nicely in dump-super
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Omar Sandoval
41eaae6d74 btrfs-progs: add the FREE_SPACE_TREE_VALID compat_ro bit definition
This is just the definition; we don't support it yet.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
Tsutomu Itoh
f9a13d867b btrfs-progs: send: fix failure of fstests btrfs/038
The following patch was imperfect, so xfstests btrfs/038 was failed.

  6d4fb3d  btrfs-progs: send: fix handling of multiple snapshots (-p option)

[before]
| # ./check btrfs/038
| FSTYP         -- btrfs
| PLATFORM      -- Linux/x86_64 luna 4.9.0-rc5
| MKFS_OPTIONS  -- /dev/sdb3
| MOUNT_OPTIONS -- /dev/sdb3 /test6
|
| btrfs/038 1s ... [failed, exit status 1] - output mismatch (see /For_RT/xfstests2/results//btrfs/038.out.bad)
|     --- tests/btrfs/038.out     2015-08-04 16:09:38.000000000 +0900
|     +++ /For_RT/xfstests2/results//btrfs/038.out.bad    2016-11-15 13:39:48.589435290 +0900
|     @@ -7,3 +7,5 @@
|      XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|      wrote 10000/10000 bytes at offset 300000
|      XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|     +failed: '/usr/local/bin/btrfs send -p /test6/mysnap1 -c /test6/clones_snap /test6/mysnap2 -f /tmp/tmp.aLpvAC7lsx/2.snap'
|     +(see /For_RT/xfstests2/results//btrfs/038.full for details)
|     ...
|     (Run 'diff -u tests/btrfs/038.out /For_RT/xfstests2/results//btrfs/038.out.bad'  to see the entire diff)
| Ran: btrfs/038
| Failures: btrfs/038

[after]
| # ./check btrfs/038
| FSTYP         -- btrfs
| PLATFORM      -- Linux/x86_64 luna 4.9.0-rc5
| MKFS_OPTIONS  -- /dev/sdb3
| MOUNT_OPTIONS -- /dev/sdb3 /test6
|
| btrfs/038 1s ... 1s
| Ran: btrfs/038
| Passed all 1 tests

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
[ Verified that this fixes both btrfs/038 and btrfs/117 ]
Tested-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:05 +01:00
David Sterba
a2d91697c1 btrfs-progs: tests: add more tests to the CI build
Run more tests in the travis build environment. The mkfs tests are still
left out due to missing capabilities of losetup.

Other minor travis adjustments.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-11-23 11:07:03 +01:00