Commit Graph

4353 Commits

Author SHA1 Message Date
Nikolay Borisov
43f14d06fd btrfs-progs: Merge alloc_reserved_tree_block2 and alloc_reserved_tree_block
Now that delayed refs have been wired let's merge the two function. In
the process also remove one BUG_ON since alloc_reserved_tree_block's
callers can handle errors. No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:42 +02:00
Nikolay Borisov
8bbb72cfc5 btrfs-progs: Remove __free_extent2, now unused
Now that delayed refs have been all wired up clean up the __free_extent2
adapter function since it's no longer needed. No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Nikolay Borisov
6de2debdb0 btrfs-progs: Remove old delayed refs infrastructure
Given that the new delayed refs infrastructure is implemented and wired
up, there is no point in keeping the old code. So just remove it.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Nikolay Borisov
909357e867 btrfs-progs: Wire up delayed refs
This commit enables the delayed refs infrastructures. This entails doing
the following:

1. Replacing existing calls of btrfs_extent_post_op (which is the
   equivalent of delayed refs) with the proper btrfs_run_delayed_refs.
   As well as eliminating open-coded calls to finish_current_insert and
   del_pending_extents which execute the delayed ops.

2. Wiring up the addition of delayed refs when freeing extents
   (btrfs_free_extent) and when adding new extents (alloc_tree_block).

3. Adding calls to btrfs_run_delayed refs in the transaction commit
   path alongside comments why every call is needed, since it's not
   always obvious (those call sites were derived empirically by running
   and debugging existing tests)

4. Correctly flagging the transaction in which we are reinitialising
   the extent tree.

5. Moving btrfs_write_dirty_block_groups to
   btrfs_write_dirty_block_groups since blockgroups should be written to
   disk after the last delayed refs have been run.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Nikolay Borisov
d8a5e756be btrfs-progs: Make btrfs_write_dirty_block_groups take only trans argument
The root argument is used only to get a reference to the fs_info, this
can be achieved with the transaction handle being passed so use that.
This is in preparation for moving this function in the main transaction
commit routine. No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Nikolay Borisov
c6039704c5 btrfs-progs: Add delayed refs infrastructure
This commit pulls those portions of the kernel implementation of
delayed refs which are necessary to have them working in user-space.
I've done the following modifications:

1. Replaced all kmem_cache_alloc calls to kmalloc.

2. Removed all locking-related code, since we are single threaded in
   userspace.

3. Removed code which deals with data refs - delayed refs in user space
   are going to be used only for cowonly trees.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Nikolay Borisov
3c8faad5cf btrfs-progs: add alloc_reserved_tree_block2 function
This is a simple adapter function to convert the delayed-refs structures
to the current arguments of alloc_reserved_tree_block.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Nikolay Borisov
98197fbcc5 btrfs-progs: add __free_extent2 function
This is a simple adapter to convert the arguments delayed ref arguments
to the existing arguments of __free_extent.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Qu Wenruo
51e1faee06 btrfs-progs: qgroup: don't return 1 if qgroup is marked inconsistent during relationship assignment
BTRFS_IOC_QGROUP_ASSIGN ioctl could return >0 if qgroup is marked
inconsistent after successful relationship assignment/removal.

We leak the return value as the final return value of btrfs command.
But according to the man page, return value other than 0 means failure.

Fix this by resetting the return value to 0 for --no-rescan case.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo
5d24e630b0 btrfs-progs: rescue-super: don't double free fs_devices
[BUG]
During fuzz/007 we hit the following error:

  ====== RUN MAYFAIL btrfs rescue super-recover -y -v tests/fuzz-tests/images/bko-200409.raw.restored.scratch
  ERROR: tree_root block unaligned: 33554431
  ERROR: superblock checksum matches but it has invalid members
  ERROR: tree_root block unaligned: 33554431
  ERROR: superblock checksum matches but it has invalid members
  ERROR: tree_root block unaligned: 33554431
  ERROR: superblock checksum matches but it has invalid members
  ERROR: failed to add chunk map start=12582912 len=8454144: -17 (File exists)
  Couldn't read chunk tree
  failed (ignored, ret=139): btrfs rescue super-recover -y -v tests/fuzz-tests/images/bko-200409.raw.restored.scratch
  mayfail: returned code 139 (SEGFAULT), not ignored
  test failed for case 007-simple-super-recover

[CAUSE]
In __open_ctree_fd(), if we have valid @open_ctree_flags and
btrfs_scan_fs_devices() succeeds without problems, no matter what
happens we will call btrfs_close_devices(), thus free all related
devices.

In super-recover, before we call open_ctree(), we have called
btrfs_scan_fs_devices() already, so btrfs_scan_fs_devices() should not
fail in open_ctree(), fs_devices will always be freed in open_ctree() or
close_ctree().

[FIX]
So in super-recover.c, we should not call btrfs_close_devices(), or we
will find fs_devices->list freed, and trigger segfault when exiting.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo
40073c3ca9 btrfs-progs: exit gracefully when device extent allocation fails
Another BUG_ON() during fuzz/003:

  ====== RUN MAYFAIL btrfs check --repair tests/fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  bad block 29409280
  ERROR: errors found in extent allocation tree or chunk allocation
  WARNING: minor unaligned/mismatch device size detected
  WARNING: recommended to use 'btrfs rescue fix-device-size' to fix it
  [3/7] checking free space cache
  [4/7] checking fs roots
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  root 18446744073709551608 missing its root dir, recreating
  Unable to find block group for 0
  Unable to find block group for 0
  Unable to find block group for 0
  volumes.c:564: btrfs_alloc_dev_extent: BUG_ON `ret` triggered, value -28
  failed (ignored, ret=134): btrfs check --repair tests/fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored
  mayfail: returned code 134 (SIGABRT), not ignored
  test failed for case 003-multi-check-unmounted

However the culprit function btrfs_alloc_dev_extent() has proper error
handling label err:, just using that label would solve the problem easily.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo
b02b426a78 btrfs-progs: fix infinite loop when bad key order repair fails
An infinite loop can be triggered during fuzz/003:

  ====== RUN MAYFAIL btrfs check --repair tests/fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19

[CAUSE]
In try_to_fix_bad_block() it's possible that btrfs_find_all_roots()
finds no root referring to that tree block, thus we can't do any repair.

However in that case, we still return 0 since the last caller assigning
@ret is btrfs_find_all_roots(), and the ulist while loop doesn't get run
at all.

And since try_to_fix_bad_block() returns 0, check_block() in
check/main.c will return -EAGAIN to re-check the tree block.

This leads to the infinite loop.

[FIX]
Change the default return value from 0 to -EIO in
try_to_fix_bad_block(), so if there is no tree referring to the bad tree
block, it won't cause infinite loop anymore.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo
6dbc43b867 btrfs-progs: only warn if there are leaked extent buffers after transaction abort
Another BUG_ON() during fuzz/003:

  ====== RUN MAYFAIL btrfs check --init-csum-tree tests/fuzz-tests/images/bko-161821.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  parent transid verify failed on 4198400 wanted 14 found 1114126
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  owner ref check failed [4198400 4096]
  repair deleting extent record: key [4198400,169,0]
  adding new tree backref on start 4198400 len 4096 parent 0 root 5
  Repaired extent references for 4198400
  ref mismatch on [4222976 4096] extent item 1, found 0
  backref 4222976 root 7 not referenced back 0x5617f8ecf780
  incorrect global backref count on 4222976 found 1 wanted 0
  backpointer mismatch on [4222976 4096]
  owner ref check failed [4222976 4096]
  repair deleting extent record: key [4222976,169,0]
  Repaired extent references for 4222976
  [3/7] checking free space cache
  [4/7] checking fs roots
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  Wrong generation of child node/leaf, wanted: 1114126, have: 14
  root 5 missing its root dir, recreating
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  ERROR: child eb corrupted: parent bytenr=4222976 item=0 parent level=1 child level=2
  ERROR: errors found in fs roots
  extent buffer leak: start 4222976 len 4096
  extent_io.c:611: free_extent_buffer_internal: BUG_ON `eb->flags & EXTENT_DIRTY` triggered, value 1
  failed (ignored, ret=134): btrfs check --init-csum-tree tests/fuzz-tests/images/bko-161821.raw.restored
  mayfail: returned code 134 (SIGABRT), not ignored
  test failed for case 003-multi-check-unmounted

Since we're shifting to use btrfs_abort_transaction() in btrfs-progs,
it will be more and more common to see dirty leaked eb.  Instead of
BUG_ON(), we only need to report it as a warning.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo
0adf626374 btrfs-progs: exit gracefully when root dir item repair fails
Another BUG_ON() during fuzz/003:

  ====== RUN MAYFAIL btrfs check --init-csum-tree tests/fuzz-tests/images/bko-161821.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  parent transid verify failed on 4198400 wanted 14 found 1114126
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  owner ref check failed [4198400 4096]
  repair deleting extent record: key [4198400,169,0]
  adding new tree backref on start 4198400 len 4096 parent 0 root 5
  Repaired extent references for 4198400
  ref mismatch on [4222976 4096] extent item 1, found 0
  backref 4222976 root 7 not referenced back 0x55e9cc694780
  incorrect global backref count on 4222976 found 1 wanted 0
  backpointer mismatch on [4222976 4096]
  owner ref check failed [4222976 4096]
  repair deleting extent record: key [4222976,169,0]
  Repaired extent references for 4222976
  [3/7] checking free space cache
  [4/7] checking fs roots
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  Wrong generation of child node/leaf, wanted: 1114126, have: 14
  root 5 missing its root dir, recreating
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  ERROR: child eb corrupted: parent bytenr=4222976 item=0 parent level=1 child level=2
  check/main.c:2738: check_inode_recs: BUG_ON `ret` triggered, value -5
  failed (ignored, ret=134): btrfs check --init-csum-tree tests/fuzz-tests/images/bko-161821.raw.restored
  mayfail: returned code 134 (SIGABRT), not ignored
  test failed for case 003-multi-check-unmounted

Just abort current transaction and exit gracefully in this case, the
caller handles errors.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo
f2cd3f394b btrfs-progs: exit gracefully if we hit ENOSPC when allocating tree block
When running test fuzz/003, we could hit the following BUG_ON:

  ====== RUN MAYFAIL btrfs check --init-csum-tree tests//fuzz-tests/images/bko-155621-bad-block-group-offset.raw.restored
  Unable to find block group for 0
  Unable to find block group for 0
  Unable to find block group for 0
  extent-tree.c:2657: alloc_tree_block: BUG_ON `ret` triggered, value -28
  failed (ignored, ret=134): btrfs check --init-csum-tree tests/fuzz-tests/images/bko-155621-bad-block-group-offset.raw.restored
  mayfail: returned code 134 (SIGABRT), not ignored
  test failed for case 003-multi-check-unmounted

Just remove that BUG_ON() and allow us to exit gracefully, the caller
handles the errors.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
David Sterba
739313d223 btrfs-progs: clean up .gitignore
- group entries that belong together
- add / prefix for files that are at fixed location
- remove obsolte build targets
- remove automake support scripts
- add missing targets (.static)

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
David Sterba
51074d2f78 btrfs-progs: docs: update clean target file masks
There's a regular manual page that matches the file glob mask *.8 so we
have to be more careful and remove only the known intermediate files.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
David Sterba
678d6c9c0f btrfs-progs: build: remove gzip dependency
The manual pages are not compressed anymore and we can remove gzip from
build dependencies and build steps.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
Mike Gilbert
14357b3e34 btrfs-progs: docs: install uncompressed manual pages
Build systems do not typically compress man pages when installing them.
This is generally left to distro packaging mechanisms, which may end up
recompressing them using a different compressor.

Author: Mike Gilbert <floppym@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
David Sterba
c0bf61709b btrfs-progs: docs: use manual page link instead of symlink
In order to install uncompressed manual pages we can't use the symlink
for the deprecated btrfsck page. Replace it by source command provided
by the manual page format.

Old: man8/btrfsck.8.gz (symlink)
New: man8/btrfsck.8 (file)

Reported-by: Mike Gilbert <floppym@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
Qu Wenruo
d980546c71 btrfs-progs: print-tree: skip deprecated blockptr / nodesize output
When printing tree nodes, we output slots like:

  key (EXTENT_TREE ROOT_ITEM 0) block 73625600 (17975) gen 16

The number in the parentheses is blockptr / nodesize.

However this number doesn't really do anything useful.  And in fact for
unaligned metadata block group (block group start bytenr is not aligned
to 16K), the number doesn't even make sense as it's rounded down.

In fact kernel doesn't ever output such divided result in its
print-tree.c

Remove it so later reader won't wonder what the number means.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
Su Yue
ea18e30d6e btrfs-progs: dump-tree: print invalid argument and strerror
Before this patch:

  $ ls nothingness
  ls: cannot access 'nothingness': No such file or directory
  $ btrfs inspect-internal dump-tree nothingness
  ERROR: not a block device or regular file: nothingness

The confusing error message makes users think that nonexistent file
exiss but is of a wrong type.

This patch lets check_arg_type return -errno if realpath failed.  And
print strerror if check_arg_type failed and the returned code is
negative. Like:

  $ btrfs inspect-internal dump-tree nothingness
  ERROR: invalid argument: nothingness: No such file or directory

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
Nikolay Borisov
5950bcd121 btrfs-progs: tests: add test for missing device delete error value
Add a test which ensures the kernel returns the correct error value
when missing device removal is requested. This test verifies that kernel
refactoring didn't break the return value.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:39 +02:00
Su Yue
3894560deb btrfs-progs: docs: update document about option -R of btrfs-scrub
The option '-R' of btrfs-scrub was documented by mistake as
'print raw statistics per-device instead of a summary'.

Here change it to 'raw print mode, print full data instead of
summary' which it works actually.

Fixes: 162257574a ("btrfs-progs: docs: update btrfs-scrub")
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:38 +02:00
David Sterba
b3b0c40465 btrfs-progs: tests: renumber last fsck test to 036-rescan-not-kicked-in
The commit d99615284a ("btrfs-progs:
fsck-tests: Add test image to check if btrfs check reports uninitialized
rescan as error") added test 035, should have been 036.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:38 +02:00
Qu Wenruo
89fcbbc041 btrfs-progs: completion: let dump-tree/dump-super/inode-resolve accept any file
For dump-tree/dump-super the completion uses default filedir -d, which
is far from convenient.  Use filedir for
dump-tree/dump-super/inode-resolve just like rootid.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:38 +02:00
Qu Wenruo
26300de037 btrfs-progs: completion: use _filedir to replace _btrfs_devs
For developers it's pretty common to call "btrfs check" on a raw image
dump other than real block device.  It's also possible to end users to
do some tests on loop devices.

So current _btrfs_devs() is really making things worse. Use _filedir()
to replace _btrfs_devs() so it can complete any filenames, no matter if
it's just a file or a real block device.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:38 +02:00
Qu Wenruo
9f45658fd2 btrfs-progs: transaction: do proper error handling in transaction commit
There are cases that btrfs_commit_transaction() itself can fail, mostly
due to ENOSPC when allocating space.

Don't panic out in this case.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-09-13 20:11:12 +02:00
David Sterba
a8985f549c btrfs-progs: btrfstune: allow to continue uuid change
When the 'btrfsune -u' command is interrupted, the final filesystem fsid
is not written to the superblock and it cannot be mounted. Too bad that
'btrfstune' cannot continue to finish the UUID change as it should.

This patch fixes that and passes the relaxed flags for superblock and
only warns when it detects the fsid mismatch. As this is something that
should be noted in case it would be needed for further debugging, it's
not just silent.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-07 11:18:54 +02:00
David Sterba
7faaca0d9f
Btrfs progs v4.17.1
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 16:14:22 +02:00
David Sterba
80d1be5df7 btrfs-progs: update CHANGES for v4.17.1
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 16:00:51 +02:00
Misono Tomohiro
5e2b3eedfc btrfs-progs: inspect logical-resolve: Set correct error value
Reset ret value to zero after snprintf(), which returns the number
of written chars. Otherwise non-zero value returns after command
success with -P option. Also set return value from __ino_to_path_fd() to
reflect the final status for default behavior.

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:36 +02:00
Qu Wenruo
d99615284a btrfs-progs: fsck-tests: Add test image to check if btrfs check reports uninitialized rescan as error
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:32 +02:00
Qu Wenruo
1d5b2ad921 btrfs-progs: qgroup-verify: Don't treat qgroup difference as error if the fs hasn't initialized a rescan
During fstests/btrfs/166, it's possible to hit a certain case where
qgroup is just enabled but rescan hasn't kicked in.

Since at qgroup enable time, we set the flag INCONSISTENT, and let later
rescan clear that flag. If power loss occurs before the rescan starts,
it's possible we get a qgroup status item with ON|INCONSISTENT but
without the RESCAN flag.

And in that case, it will definitely cause difference in qgroup
accounting as all numbers in the qgroup tree are 0.

Fix this false alert by also checking rescan progress from
btrfs_status_item.  And if we find rescan progress is still 0,
INCONSISTENT flag set and no RESCAN flag set, we won't treat it as an
error.

Reported-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:30 +02:00
David Sterba
4dca0ccf3d btrfs-progs: check: reorder options to sections
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:24 +02:00
Qu Wenruo
21d1109168 btrfs-progs: check/original: Don't overwrite return value when we failed to repair
In check_inode_recs(), for repair mode we always reset @ret to 0.  It
makes no sense and later we check @ret to determine if the repair is
successful.

Fix it by removing the offending overwrite.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:22 +02:00
Qu Wenruo
159f177cf9 btrfs-progs: check/original: Avoid infinite loop when failed to repair inode
Exposed by fuzz-tests/003-multi-check-unmounted/ on fuzzed image
bko-161811.raw.xz.

It's caused by the fact when check_fs_roots() finds tree root is
modified, it re-search tree root by goto again: label.
However again: label. will also reset root objectid to 0.
If we failed to repair one fs root but still modified tree root, we will
go into such infinite loop.

Fix it by recording which root we should skip for repair mode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:20 +02:00
Qu Wenruo
5f594c3ce3 btrfs-progs: check: Remove the ability to rebuild root overwriting existing tree blocks
We have function btrfs_fsck_reinit_root() to reinit csum or extent tree.
However this function allows us to let it overwrite existing tree blocks
using @overwrite parameter.

Such behavior is pretty dangerous while no caller is using this feature
explicitly.

So just remove @overwrite parameter and allow btrfs_fsck_reinit_root()
to error out when it fails to allocate tree block.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:04:58 +02:00
Omar Sandoval
a4770b0a01 btrfs-progs: build: add --disable-shared and --disable-static
The build system mentioned in the previous commit builds libraries in
both PIC and non-PIC mode. Shared libraries don't work in PIC mode, so
it expects a --disable-shared configure option, which most open source
libraries using autoconf have. Let's add it, too.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:41 +02:00
Omar Sandoval
996a07c602 btrfs-progs: build: add --disable-programs
We have a build system internally which only needs to build and install
the libraries out of a repository, not any binaries. There's no easy way
to do this in btrfs-progs currently. Add --disable-programs to
./configure to support this.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:38 +02:00
james harvey
7c5a355b74 btrfs-progs: btrfs_close_devices: only fsync if device->writeable is set
Prevent unnecessary error from failing fsync(), if opened read only.

Performed 'grep "writeable = " *.h *.c' to make sure there were no odd
situations where fsync() might still be desired here.  They're all straight-
forward.  The only situation where writeable will be 0 is if btrfs_open_devices
is given flags without O_RDWR.  There is no situation where a writeable volume
temporarily becomes unwriteable, or anything like that.  Given that it's being
opened O_RDWR, there's no reason to attempt fsync().

utils.c

   int btrfs_add_to_fsid() {
      ...
      device->writeable = 1;

volumes.c

   int btrfs_close_devices() {
      ...
      while (!list_empty(&fs_devices->devices)) {
         ...
         // just after the fsync() being patched
267:     device->writeable = 0;
   ...
   int btrfs_open_devices() {
      ...
      list_for_each_entry(device, &fs_devices->devices, dev_list) {
         ...
         if (flags & O_RDWR)
332:        device->writeable = 1

kernel btrfs_close_devices() does not have a corresponding fsync() that I see.

Signed-off-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:33 +02:00
james harvey
ef00e4264c btrfs-progs: map-logical: Use btrfs_next_extent_item
btrfs_next_extent_item() looks for BTRFS_EXTENT_ITEM_KEY and
BTRFS_METADATA_KEY, which are the types we're looking for.

Signed-off-by: James Harvey <jamespharvey20@gmail.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:27 +02:00
Stéphane Lesimple
078e9a1cc9 btrfs-progs: check: enhanced progress indicator
We reuse the task_position enum and task_ctx struct of the original progress
indicator, adding more values and fields for our needs.

Then add hooks in all steps of the check to properly record progress.

Here's how the output looks like on a 22 Tb 5-disk RAID1 FS:

Opening filesystem to check...
Checking filesystem on /dev/mapper/luks-ST10000VN0004-XXXXXXXX
UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[1/7] checking extents           (0:20:21 elapsed, 950958 items checked)
[2/7] checking root items        (0:01:29 elapsed, 15121 items checked)
[3/7] checking free space cache  (0:00:11 elapsed, 4928 items checked)
[4/7] checking fs roots          (0:51:31 elapsed, 600892 items checked)
[5/7] checking csums             (0:14:35 elapsed, 754522 items checked)
[6/7] checking root refs         (0:00:00 elapsed, 232 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 5286458060800 bytes used, no error found

Signed-off-by: Stéphane Lesimple <stephane_btrfs@lesimple.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:23 +02:00
Stéphane Lesimple
48a2818159 btrfs-progs: fix nanosecs in task_period_start
This is a single-line fix on the preexisting task_period_start function.

Signed-off-by: Stéphane Lesimple <stephane_btrfs@lesimple.fr>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:21 +02:00
Su Yue
e6203e6ff9 btrfs-progs: fsck-tests: add test case with keyed data backref with reloc tree blocks
For trees have been balanced, leaves are with flag BTRFS_HEADER_FLAG_RELOC and
extent data backrefs are shared. Like:
=====================
       item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3932 itemsize 63
                refs 129 gen 7 flags DATA
                shared data backref parent 35897344 count 41
                shared data backref parent 35426304 count 37
                shared data backref parent 35422208 count 51
=====================

Then make the leaf which owns the extent data cowed. The shared data
backref was to transferred to keyed data ref, but remaining backrefs are
still shared. Like:
=====================
        item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3887 itemsize 108
                refs 129 gen 7 flags DATA
                extent data backref root 5 objectid 258 offset 0 count 40
                extent data backref root 5 objectid 257 offset 0 count 1
                shared data backref parent 35426304 count 37
                shared data backref parent 35422208 count 51
=====================

However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.

Add the test case to check it.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:16 +02:00
Su Yue
d53d42fa21 btrfs-progs: lowmem: fix false alerts of referencer count mismatch for blocks relocated
Btrfs lowmem check reports such false alerts:
=========
ERROR: extent[1419709677568, 1703936] referencer count mismatch (root: 2192, owner: 327635, offset: 0) wanted: 9, have: 13
=========

While in extent tree, the extent has:
=========
        item 98 key (1419709677568 EXTENT_ITEM 1703936) itemoff 10745 itemsize 92
                refs 35 gen 16921 flags DATA
                extent data backref root 2192 objectid 327635 offset 0 count 9
                shared data backref parent 1232784752640 count 4
                shared data backref parent 1232784736256 count 13
                shared data backref parent 1232784719872 count 9
=========

The extent data 327635 starts in this leaf without BTRFS_HEADER_FLAG_RELOC:
=========
leaf 1471521390592 items 146 free space 3671 generation 18581 owner 2192
leaf 1471521390592 flags 0x1(WRITTEN) backref revision 1
...
 item 137 key (327635 EXTENT_DATA 0) itemoff 7745 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 0 nr 131072 ram 1703936
                extent compression 0 (none)
        item 138 key (327635 EXTENT_DATA 131072) itemoff 7692 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 131072 nr 131072 ram 1703936
                extent compression 0 (none)
...(Exactly 9 items here)
=========

The next leaf is flaged as relocated, so extent data backrefs are
shared not keyed:
=========
leaf 1232784752640 items 159 free space 2503 generation 16924 owner 2192
leaf 1232784752640 flags 0x3(WRITTEN|RELOC) backref revision 1
fs uuid 0f43f49d-6e63-4b1b-bc8c-c54da409872d
chunk uuid e558e6f7-4f08-4292-91fb-9a775fdd530b
        item 0 key (327635 EXTENT_DATA 1179648) itemoff 16230 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 1179648 nr 131072 ram 1703936
                extent compression 0 (none)
        item 1 key (327635 EXTENT_DATA 1310720) itemoff 16177 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 1310720 nr 131072 ram 1703936
                extent compression 0 (none)
        item 2 key (327635 EXTENT_DATA 1441792) itemoff 16124 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 1441792 nr 131072 ram 1703936
                extent compression 0 (none)
        item 3 key (327635 EXTENT_DATA 1572864) itemoff 16071 itemsize 53
=======

Lowmem mode starts to count extent data backrefs in first leaf, those
backrefs are keyed as wanted.
It calls btrfs_next_item to skip to next leaf.
However, the next leaf was relocated, extent data items in this leaf
should have shared backrefs shouldn't be counted.

So let lowmem mode do not count data backrefs if leaf is flaged with
BTRFS_HEADER_FLAG_RELOC.

Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:14 +02:00
Qu Wenruo
d6f5504239 btrfs-progs: test/fuzz: Add image for BUG_ON() when opening the fs by btrfs check
Link: https://bugzilla.kernel.org/show_bug.cgi?id=199839
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:11 +02:00
Qu Wenruo
3fcef50906 btrfs-progs: tests/fuzz: Add fuzzed test image for btrfs check BUG_ON
This fuzzed image will not only cause kernel BUG_ON(), but also btrfs
check BUG_ON() for original mode.

Checking filesystem on /home/adam/btrfs/crafted_images/runtime/0.img
UUID: 3381d111-94a3-4ac7-8f39-611bbbdab7e6
checking extents
check/main.c:3677: check_owner_ref: BUG_ON `rec->is_root` triggered, value 1
btrfs(+0x572c2)[0x562d65da72c2]
btrfs(+0x6098d)[0x562d65db098d]
btrfs(+0x60bb6)[0x562d65db0bb6]
btrfs(+0x6179b)[0x562d65db179b]
btrfs(cmd_check+0x1199)[0x562d65db5589]
btrfs(main+0x88)[0x562d65d62768]
/usr/lib/libc.so.6(__libc_start_main+0xeb)[0x7f4fcbb1b06b]
btrfs(_start+0x2a)[0x562d65d6288a]

Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:09 +02:00
Qu Wenruo
c010437455 btrfs-progs: check: orig: Don't panic when unexpected root item is referring to one extent
With crafted image, expected root item can refer to certain extent, and
original mode uses BUG_ON() to handle such case.

Fix it by gracefully return error.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:07 +02:00
Qu Wenruo
beb3ede39a btrfs-progs: tests/fuzz: Add image for bko-200409
Reported-by: Xu Wen <wen.xu@gatech.edu>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=200409
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:02 +02:00