Commit Graph

261 Commits

Author SHA1 Message Date
Mark Harmstone
945cfa1ae3 btrfs-progs: check: add check for too many csum entries
My tool ntfs2btrfs has been creating btrfs volumes in a way that the
kernel doesn't like, but which isn't picked up by btrfs check - see
maharmstone/ntfs2btrfs#23 for the details, including a backtrace. This
patch adds a check for when a csum item contains too many entries -
effectively it ensures that there's always at least sizeof(struct
btrfs_item) bytes free in the tree, otherwise btrfs_del_csums can throw
an error.

max_entries is the value of the __MAX_CSUM_ITEMS macro in
fs/btrfs/file-item.c.

Pull-request: #401
Signed-off-by: Mark Harmstone <mark@harmstone.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-11-18 10:17:21 +01:00
Qu Wenruo
c4ff87c3d1 btrfs-progs: cache csum_size and csum_type in btrfs_fs_info
Just like kernel commit 22b6331d9617 ("btrfs: store precalculated
csum_size in fs_info"), we can cache csum_size and csum_type in
btrfs_fs_info.

Furthermore, there is already a 32 bits hole in btrfs_fs_info, and we
can fit csum_type and csum_size into the hole without increase the size
of btrfs_fs_info.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-11-05 12:50:03 +01:00
Qu Wenruo
5bee5c99bf btrfs-progs: fix printf formats on 32bit x86
When compiling btrfs-progs on 32bit x86 using GCC 11.1.0, there are
several warnings:

  In file included from ./common/utils.h:30,
                   from check/main.c:36:
  check/main.c: In function 'run_next_block':
  ./common/messages.h:42:31: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'u32' {aka 'unsigned int'} [-Wformat=]
     42 |                 __btrfs_error((fmt), ##__VA_ARGS__);                    \
        |                               ^~~~~
  check/main.c:6496:33: note: in expansion of macro 'error'
   6496 |                                 error(
        |                                 ^~~~~

  In file included from ./common/utils.h:30,
                   from kernel-shared/volumes.c:32:
  kernel-shared/volumes.c: In function 'btrfs_check_chunk_valid':
  ./common/messages.h:42:31: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'u32' {aka 'unsigned int'} [-Wformat=]
     42 |                 __btrfs_error((fmt), ##__VA_ARGS__);                    \
        |                               ^~~~~
  kernel-shared/volumes.c:2052:17: note: in expansion of macro 'error'
   2052 |                 error("invalid chunk item size, have %u expect [%zu, %lu)",
        |                 ^~~~~

  image/main.c: In function 'search_for_chunk_blocks':
  ./common/messages.h:42:31: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Wformat=]
     42 |                 __btrfs_error((fmt), ##__VA_ARGS__);                    \
        |                               ^~~~~
  image/main.c:2122:33: note: in expansion of macro 'error'
   2122 |                                 error(
        |                                 ^~~~~

There are two types of problems:

- __BTRFS_LEAF_DATA_SIZE()
  This macro has no type definition, making it behaves differently on
  different arches.

  Fix this by following kernel to use inline function to make its return
  value fixed to u32.

- size_t related output
  For x86_64 %lu is OK but not for x86.

  Fix this by using %zu.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-11-05 12:50:03 +01:00
Qu Wenruo
8f81113021 btrfs-progs: check: fix a lowmem mode crash where fatal error is not properly handled
[BUG]
When a special image (diverted from fsck/012) has its unused slots (slot
number >= nritems) with garbage, lowmem mode btrfs check can crash:

  (gdb) run check --mode=lowmem ~/downloads/good.img.restored
  Starting program: /home/adam/btrfs/btrfs-progs/btrfs check --mode=lowmem ~/downloads/good.img.restored
  ...
  ERROR: root 5 INODE[5044031582654955520] nlink(257228800) not equal to inode_refs(0)
  ERROR: root 5 INODE[5044031582654955520] nbytes 474624 not equal to extent_size 0

  Program received signal SIGSEGV, Segmentation fault.
  0x0000555555639b11 in btrfs_inode_size (eb=0x5555558a7540, s=0x642e6cd1) at ./kernel-shared/ctree.h:1703
  1703	BTRFS_SETGET_FUNCS(inode_size, struct btrfs_inode_item, size, 64);
  (gdb) bt
  #0  0x0000555555639b11 in btrfs_inode_size (eb=0x5555558a7540, s=0x642e6cd1) at ./kernel-shared/ctree.h:1703
  #1  0x0000555555641544 in check_inode_item (root=0x5555556c2290, path=0x7fffffffd960) at check/mode-lowmem.c:2628

[CAUSE]
At check_inode_item() we have path->slot[0] at 29, while the tree block
only has 26 items.

This happens because two reasons:

- btrfs_next_item() never reverts its slots
  Even if we failed to read next leaf.

- check_inode_item() doesn't inform the caller that a fatal error
  happened
  In check_inode_item(), if btrfs_next_item() failed, it goes to out
  label, which doesn't really set @err properly.

This means, when check_inode_item() fails at btrfs_next_item(), it will
increase path->slots[0], while it's already beyond current tree block
nritems.

When the slot increases furthermore, and if the unused item slots have
some garbage, we will get invalid btrfs_item_ptr() result, and causing
above segfault.

[FIX]
Fix the problems by two ways:

- Make btrfs_next_item() to revert its path->slots[0] on failure

- Properly detect fatal error from check_inode_item()

By this, we will no longer crash on the crafted image.

Reported-by: Wang Yugui <wangyugui@e16-tech.com>
Issue: #412
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-11-04 20:56:42 +01:00
David Sterba
00610f5853 btrfs-progs: libbtrfs: remove unneeded BTRFS_FLAT_INCLUDES protections
Remove the switch for local and system-wide headers in headers that are
not part of libbtrfs anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba
38356d456b btrfs-progs: libbtrfs: drop radix-tree.h from exported headers
The header is only included from ctree.h but not actually used, we can
drop it from the exported files.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
Nikolay Borisov
7c58b09548 btrfs-progs: remove root argument from btrfs_fixup_low_keys
It's not used, so just remove it.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
Qu Wenruo
9d2460fb3f btrfs-progs: check: fix indentation of --clear-ino-cache option
Help string for "--clear-ino-cache" option is not following the
indentation of other help strings:

      repair options:
           --init-csum-tree            create a new CRC tree
           --init-extent-tree          create a new extent tree
           --clear-space-cache v1|v2   clear space cache for v1 or v2
           --clear-ino-cache 	    clear ino cache leftover items

The problem is caused by the usage of tab instead of space.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:00 +02:00
Qu Wenruo
60651ad9da btrfs-progs: introduce OPEN_CTREE_ALLOW_TRANSID_MISMATCH flag
[BUG]
There is a report that, btrfstune can even work while the fs has transid
mismatch problems.

  $ btrfstune -f -u /dev/sdb1
  Current fsid: b2b5ae8d-4c49-45f0-b42e-46fe7dcfcb07
  New fsid: b2b5ae8d-4c49-45f0-b42e-46fe7dcfcb07
  Set superblock flag CHANGING_FSID
  Change fsid in extents
  parent transid verify failed on 792854528 wanted 20103 found 20091
  parent transid verify failed on 792854528 wanted 20103 found 20091
  parent transid verify failed on 792854528 wanted 20103 found 20091
  Ignoring transid failure
  parent transid verify failed on 792870912 wanted 20103 found 20091
  parent transid verify failed on 792870912 wanted 20103 found 20091
  parent transid verify failed on 792870912 wanted 20103 found 20091
  Ignoring transid failure
  parent transid verify failed on 792887296 wanted 20103 found 20091
  parent transid verify failed on 792887296 wanted 20103 found 20091
  parent transid verify failed on 792887296 wanted 20103 found 20091
  Ignoring transid failure
  ERROR: child eb corrupted: parent bytenr=38010880 item=69 parent level=1 child level=1
  ERROR: failed to change UUID of metadata: -5
  ERROR: btrfstune failed

This leaves a corrupted fs even more corrupted, and due to the extra
CHANGING_FSID flag, btrfs check will not even try to run on it:

  Opening filesystem to check...
  ERROR: Filesystem UUID change in progress
  ERROR: cannot open file system

[CAUSE]
Unlike kernel, btrfs-progs has a less strict check on transid mismatch.

In read_tree_block() we will fall back to use the tree block even its
transid mismatch if we can't find any better copy.

However not all commands in btrfs-progs needs this feature, only
btrfs-check (which may fix the problem) and btrfs-restore (it just tries
to ignore any problems) really utilize this feature.

[FIX]
Introduce a new open ctree flag, OPEN_CTREE_ALLOW_TRANSID_MISMATCH, to
be explicit about whether we really want to ignore transid error.

Currently only btrfs-check and btrfs-restore will utilize this new flag.

Also add btrfs-image to allow opening such fs with transid error.

Link: https://www.reddit.com/r/btrfs/comments/pivpqk/failure_during_btrfstune_u/
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-20 12:17:29 +02:00
David Sterba
400b66689e btrfs-progs: move btrfsck.h to check/
This is part of the checker and unfortunately also the public header, so
we can only copy it to the right directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 19:49:33 +02:00
Qu Wenruo
1f8dfe681f btrfs-progs: use btrfs_key for btrfs_check_node() and btrfs_check_leaf()
In kernel space we hardly use btrfs_disk_key, unless for very lowlevel
code.

There is no need to intentionally use btrfs_disk_key in btrfs-progs
either.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 13:58:44 +02:00
Qu Wenruo
325dba6432 btrfs-progs: check: output proper csum values for --check-data-csum
[BUG]
When running "btrfs check --check-data-csum" on fs with corrupted data,
the error message almost makes no sense:

  $ btrfs check --check-data-csum /dev/test/test
  Opening filesystem to check...
  Checking filesystem on /dev/test/test
  UUID: c31afe0a-55bc-4e7d-aba0-9dfa9ddf8090
  [1/7] checking root items
  [2/7] checking extents
  [3/7] checking free space cache
  [4/7] checking fs roots
  [5/7] checking csums against data
  mirror 1 bytenr 13631488 csum 19 expected csum 152 <<<
  ERROR: errors found in csum tree
  [6/7] checking root refs
  [7/7] checking quota groups skipped (not enabled on this FS)
  found 147456 bytes used, error(s) found
  total csum bytes: 16
  total tree bytes: 131072
  total fs tree bytes: 32768
  total extent tree bytes: 16384
  btree space waste bytes: 124799
  file data blocks allocated: 16384
   referenced 16384

[CAUSE]
We're just outputting the first byte and in decimal, which is completely
different from what we did in kernel space, nor what we did for metadata
csum mismatch.

[FIX]
Use btrfs_format_csum() for btrfs-check to output csum.

Now the result looks much better:

  [5/7] checking csums against data
  mirror 1 bytenr 13631488 csum 0x13fec125 expected csum 0x98757625

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-26 14:27:40 +02:00
Qu Wenruo
773afad3e6 btrfs-progs: slightly enhance btrfs_format_csum()
- Change it void
  The old one always return csum_size.

- Use snprintf()

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-26 14:27:01 +02:00
Josef Bacik
e64af00bd1 btrfs-progs: check: detect and fix problems with super_bytes_used
We do not detect problems with our bytes_used counter in the super
block.  Thankfully the same method to fix block groups is used to re-set
the value in the super block, so simply add some extra code to validate
the bytes_used field and then piggy back on the repair code for block
groups.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:39:48 +02:00
Josef Bacik
04520da77b btrfs-progs: check: do not infinite loop on corrupt keys with lowmem mode
By enabling the lowmem checks properly I uncovered the case where test
fsck/007 will infinite loop at the detection stage.  This is because
when checking the inode item we will just btrfs_next_item(), and because
we ignore check tree block failures at read time we don't get an -EIO
from btrfs_next_leaf.  Generally what check usually does is validate the
leaves/nodes as we hit them, but in this case we're not doing that.  Fix
this by checking the leaf if we move to the next one and if it fails
bail.  This allows us to pass the fsck/007 test with lowmem.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:39:48 +02:00
Josef Bacik
4a1863d638 btrfs-progs: check: do not double add unaligned extent records
The repair cycle in the main check will drop all of our cache and loop
through again to make sure everything is still good to go.
Unfortunately we record our unaligned extent records on a per-root list
so they can be retrieved when we're checking the fs roots.  This isn't
straightforward to clean up, so instead simply check our current list of
unaligned extent records when we are adding a new one to make sure we're
not duplicating our efforts.  This makes us able to pass fsck/001 with
my super bytes_used fix applied.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Josef Bacik
a4190da45e btrfs-progs: check btrfs_super_used in lowmem check
We can already fix this problem with the block accounting code, we just
need to keep track of how much we should have used on the file system,
and then check it against the bytes_super.  The repair just piggy backs
on the block group used repair.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Josef Bacik
c3521f8a57 btrfs-progs: propagate extent item errors in lowmem mode
Test 044 was failing with lowmem because it was not bubbling up the
error to the user.  This is because we try to allow repair the
opportunity to clear the error, however if repair isn't set we simply do
not add the temporary error to the main error return variable.  Fix this
by adding the tmp_err to err before moving on to the next item.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Josef Bacik
477775946d btrfs-progs: propagate fs root errors in lowmem mode
We have a check that will return an error only if ret < 0, but we return
the lowmem specific errors which are all > 0.  Fix this by simply
checking if (ret).  This allows test 010 to pass with lowmem properly.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Josef Bacik
a832d36b59 btrfs-progs: check: detect and fix invalid used for block groups
The lowmem mode validates the used field of the block group item, but
the normal mode does not.  Fix this by keeping a running tally of what
we think the used value for the block group should be, and then if it
mismatches report an error and fix the problem if we have repair set.
We have to keep track of pending extents because we process leaves as we
see them, so it could be much later in the process that we find the
block group item to associate the extents with.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Qu Wenruo
9df3619a9d btrfs-progs: require full nodesize alignement for subpage support
For the incoming extra page size support for subpage (sectorsize <
PAGE_SIZE) cases, the support for metadata will be a critical point.

Currently for subpage support, we require 64K page size, so that no
matter whatever the nodesize is, it will be contained inside one page.
And we will reject any tree block which crosses page boundary.

But for other page size, especially 16K page size, we must support
nodesize differently.

For nodesize < PAGE_SIZE, we will have the same requirement (tree blocks
can't cross page boundary).
While for nodesize >= PAGE_SIZE, we will require the tree blocks to be
page aligned.

To support such feature, we will make btrfs-check to reports more
subpage related warnings for metadata.

This patch will report any tree block which is not nodesize aligned as a
warning.

Existing mkfs/convert has already make sure all new tree blocks are
nodesize aligned, this is just for older converted filesystems.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:45:58 +02:00
Qu Wenruo
478e98cd01 btrfs-progs: check/original: detect directory inode with nlinks >= 2
Linux VFS doesn't allow directory to have hard links, thus for btrfs
on-disk directory inode items, their nlinks should never go beyond 1.

Lowmem mode already has the check and will report it without problem.
Only original mode needs this update.

Reported-by: Pepperpoint <pepperpoint@mb.ardentcoding.com>
Link: https://lore.kernel.org/linux-btrfs/162648632340.7.1932907459648384384.10178178@mb.ardentcoding.com/
Reviewed-by: Su Yue <l@damenly.su>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-30 15:50:48 +02:00
Qu Wenruo
07ecf878c1 btrfs-progs: check: batch v1 space cache inodes when clearing
Currently v1 space cache clearing will delete one cache inode just in
one transaction, and then start a new transaction to delete the next
inode.

This is far from efficient and can make the already slow v1 space cache
deleting even slower, as large fs has tons of cache inodes to delete.

This patch will speed up the process by batching up to 16 inode deletion
into one transaction.

A quick benchmark of deleting 702 v1 space cache inodes would look like
this:

Unpatched:		4.898s
Patched:		0.087s

Which is obviously a big win.

Reported-by: Joshua <joshua@mailmag.net>
Link: https://lore.kernel.org/linux-btrfs/0b4cf70fc883e28c97d893a3b2f81b11@mailmag.net/
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-22 16:26:05 +02:00
Qu Wenruo
0ebd1f47c5 btrfs-progs: check: add the ability to reset btrfs_dev_item::bytes_used
There is a report from the mailing list that one user got its filesystem
with device item bytes_used mismatch.

This problem leaves the device item with some ghost bytes_used, meaning
even if we delete all device extents of that device, the bytes_used
still won't be 0.

This itself is not a big deal, but when the user used up all its
unallocated space, write time tree-checker can be triggered and make the
fs RO, as the new device::bytes_used can be larger than
device::total_bytes.

Thus we need to fix the problem in btrfs-check to avoid above write-time
tree check warning.

This patch will add the ability to reset a device's bytes_used to both
original mode and lowmem mode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
Su Yue
4c282411eb btrfs-progs: check: continue to check space cache if sb cache_generation is 0
User reported that test fsck-tests/037-freespacetree-repair fails:

 # TEST=037\* ./fsck-tests.sh
    [TEST/fsck]   037-freespacetree-repair
 btrfs check should have detected corruption
 test failed for case 037-freespacetree-repair

The test tries to corrupt FST, call btrfs check readonly then repair FST
using btrfs check. Above case failed at the second readonly check step.
Test log said "cache and super generation don't match, space cache will
be invalidated" which is printed by validate_free_space_cache().
If cache_generation of the superblock is not -1ULL,
validate_free_space_cache() requires that cache_generation must equal
to the superblock's generation. Otherwise, it skips the check of space
cache(v1, v2) like the above case where the sb cache_generation is 0.

Since kernel commit 948462294577 ("btrfs: keep sb cache_generation
consistent with space_cache"), sb cache_generation will be set to be 0
once space cache v1 is disabled (nospace_cache/space_cache=v2). But
progs check was forgotten to be added the 0 case support.

Fix it by adding the condition if sb cache_generation is 0 in
validate_free_space_cache() as the 0 case is valid now since the
kernel commit mentioned above.

Issue: #338
Signed-off-by: Su Yue <l@damenly.su>
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-12 20:44:12 +02:00
David Sterba
c19ac510a7 btrfs-progs: move repair.[ch] to common/
Move the file to common as it's used by several parts, while still
keeping the name 'repair' although the only thing it does is adding a
corrupted extent.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba
cfbcfaa4e4 btrfs-progs: mkfs: move btrfs_make_root_dir from utils.c
The helper is used in several tools but logically belongs to mkfs, so
put it to the common section.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba
7fa07e2abb btrfs-progs: split open/close helpers from utils.c
There's a group of functions that are related to opening filesystem in
various modes, this can be moved to a separate file.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
Johannes Thumshirn
94b60b67a9 btrfs-progs: pass in fs_info to btrfs_csum_data
For passing authentication keys to the checksumming functions we need a
container for the key.

Pass in a btrfs_fs_info to btrfs_csum_data() so we can use the fs_info
as a container for the authentication key.

Note this is not always possible for all callers of btrfs_csum_data() so
we're just passing in NULL for now

Functions calling btrfs_csum_data() with a NULL fs_info argument are
currently not supported in the context of an authenticated file system.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
David Sterba
6bb6d1215d btrfs-progs: factor open_ctree parameters to a structure
Extending open_ctree with more parameters would be difficult, we'll need
to add more so factor out the parameters to a structure for easier
extension.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
Jaak Ristioja
2f9736d051 btrfs-progs: check: fix typos in code comment and 1 typo in warning
Pull-request: #356
Signed-off-by: Jaak Ristioja <jaak@ristioja.ee>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:22 +01:00
Qu Wenruo
33d84be030 btrfs-progs: fix false alert on tree block crossing 64K page boundary
[BUG]
When btrfs-check is executed on even newly created fs, it can report
tree blocks crossing 64K page boundary like this:

  Opening filesystem to check...
  Checking filesystem on /dev/test/test
  UUID: 80d734c8-dcbc-411b-9623-a10bd9e7767f
  [1/7] checking root items
  [2/7] checking extents
  WARNING: tree block [30523392, 30539776) crosses 64K page boudnary, may cause problem for 64K page system
  [3/7] checking free space cache
  [4/7] checking fs roots
  [5/7] checking only csums items (without verifying data)
  [6/7] checking root refs
  [7/7] checking quota groups skipped (not enabled on this FS)
  found 131072 bytes used, no error found
  total csum bytes: 0
  total tree bytes: 131072
  total fs tree bytes: 32768
  total extent tree bytes: 16384
  btree space waste bytes: 125199
  file data blocks allocated: 0
   referenced 0

[CAUSE]
Tree block [30523392, 30539776) is at the last 16K slot of page.
As 30523392 % 65536 = 49152, and 30539776 % 65536 = 0.

The cross boundary check is using exclusive end, which causes false
alerts.

[FIX]
Use inclusive end to do the cross 64K boundary check.

Reported-by: Wang Yugui <wangyugui@e16-tech.com>
Issue: #352
Issue: #354
Fixes: fc38ae7f48 ("btrfs-progs: check: detect and warn about tree blocks crossing 64K page boundary")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-15 15:40:48 +01:00
Qu Wenruo
fc38ae7f48 btrfs-progs: check: detect and warn about tree blocks crossing 64K page boundary
For the incoming subpage support, there is a new requirement for tree
blocks.  Tree blocks should not cross 64K page boundary.

For current btrfs-progs and kernel, there shouldn't be any causes to
create such tree blocks.  But still, we want to detect such tree blocks
in the wild before subpage support fully lands in upstream.

This patch will add such check for both lowmem and original mode.
Currently it's just a warning, since there aren't many users using 64K
page size yet.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 15:24:42 +01:00
Qu Wenruo
501dc44c7f btrfs-progs: check: only warn if clearing v1 cache and v2 found
There are cases where v1 free space cache is still left while user has
already enabled v2 cache.  In that case, we still want to force v1 space
cache cleanup in btrfs-check.

This patch will only warn and not exit if v2 is detected while the user
asked to clear v1.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-18 17:49:23 +01:00
Josef Bacik
3f467adb34 btrfs-progs: check: properly exclude leaves for lowmem mode
The lowmem mode excludes all referenced blocks from the allocator in
order to avoid accidentally overwriting blocks while fixing the file
system.  However for leaves it wouldn't exclude anything, it would just
pin them down, which gets cleaned up on transaction commit.  We're safe
for the first modification, but subsequent modifications could blow up
in our face.  Fix this by properly excluding leaves as well as all of
the nodes.

Reviewed-by: Su Yue <l@damenly.su>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:10 +01:00
Nikolay Borisov
04f1977832 btrfs-progs: check: add option to remove ino cache
Inode cache feature is going to be removed in kernel 5.11. After this
kernel version items left on disk by this feature will take some extra
space. Testing showed that the size is actually negligible but for
completeness' sake give ability to users to remove such left-overs.

This is achieved by iterating every fs root and removing respective
items as well as relevant csum extents since the ino cache used the csum
tree for csums.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:53 +01:00
Josef Bacik
bc06a75a04 btrfs-progs: only print the parent or ref root for ref mismatches
While debugging some corruption, I got confused because it appeared as
if we had an invalid parent set on a extent reference, because of this
message:

  tree backref 67014213632 parent 5 root 5 not found in extent tree

But it turns out that parent and the root are a union, and we were just
printing it out regardless of the type of backref it was.  Fix the error
message to be consistent with the other mismatch messages, simply print
parent or root, depending on the ref type.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:53 +01:00
Qu Wenruo
05f25a1460 btrfs-progs: check/original: add ability to repair extent item generation
This is pretty much the same as for lowmem mode, it will try to reset
the extent item generation using either the tree block generation or
current transid.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:30 +01:00
Qu Wenruo
b858e3b5c2 btrfs-progs: check/original: don't reset extent generation for check_block
In check_block(), we unconditionally reset extent_record::generation.

This is in fact correct, but this makes original mode fail to detect bad
extent item generation.

So change to behavior to set the generation if and only if the tree
block generation is higher.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:30 +01:00
Qu Wenruo
289e87d9f8 btrfs-progs: check/lowmem: add ability to repair extent item generation
There is an internal report about bad extent item generation triggering
tree-checker.

This patch will add the repair ability to btrfs check --mode=lowmem
mode, by resetting the generation field of extent item.

Currently the correct generation for tree block is fetched from its
header, while for data extent it uses transid as fallback.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:30 +01:00
Nikolay Borisov
c08983291e btrfs-progs: check: fix error reporting on root inode
If btrfs check detects an error on the root inode of a subvolume it
prints:

    Opening filesystem to check...
    Checking filesystem on /dev/vdc
    UUID: 4ac7a216-bf97-4c5f-9899-0f203c20d8af
    [1/7] checking root items
    [2/7] checking extents
    [3/7] checking free space cache
    [4/7] checking fs roots
    root 5 root dir 256 error
    ERROR: errors found in fs roots
    found 196608 bytes used, error(s) found
    total csum bytes: 0
    total tree bytes: 131072
    total fs tree bytes: 32768
    total extent tree bytes: 16384
    btree space waste bytes: 124376
    file data blocks allocated: 65536
     referenced 65536

This is not very helpful since there is no specific information about
the exact error. This is due to the fact that check_root_dir doesn't
set inode_record::errors accordingly. This patch rectifies this and now
the output would look like:

	[1/7] checking root items
	[2/7] checking extents
	[3/7] checking free space cache
	[4/7] checking fs roots
	root 5 inode 256 errors 2000, link count wrong
	ERROR: errors found in fs roots
	found 196608 bytes used, error(s) found
	total csum bytes: 0
	total tree bytes: 131072
	total fs tree bytes: 32768
	total extent tree bytes: 16384
	btree space waste bytes: 124376
	file data blocks allocated: 65536
	 referenced 65536

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:16:36 +02:00
Qu Wenruo
0eafacf668 btrfs-progs: check/original: add inode transid detect and repair support
The inode transid detect and repair is reusing the existing inode
geneartion code.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Qu Wenruo
ce2dcb3615 btrfs-progs: check/lowmem: add inode transid detect and repair support
There are quite some reports on kernel rejecting invalid inode
generation, but it turns out to be that, kernel is just rejecting inode
transid. It's a bug in kernel error message.

To solve the problem and make the fs mountable again, add the detect and
repair support for lowmem mode.

The implementation is pretty much the same, just re-use the existing
inode generation detect and repair code.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Marcos Paulo de Souza
c655b5e4b1 btrfs-progs: make btrfs_lookup_dir_index in parity with kernel code
This function exists in kernel side but using the _item suffix, and
objectid argument is placed before the name argument. Change the
function to reflect the kernel version.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:49 +02:00
David Sterba
0144bcb713 btrfs-progs: move volumes.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba
6069bc52a9 btrfs-progs: move transaction.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba
abb670f883 btrfs-progs: move ctree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba
772f0da6df btrfs-progs: move disk-io.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba
cf529f36ad btrfs-progs: move print-tree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba
da90f38ad9 btrfs-progs: move free-space-tree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:04 +02:00