Commit Graph

4983 Commits

Author SHA1 Message Date
Alexandru Ungureanu
378a1ab8ff btrfs-progs: docs: improved asciidoc syntax to fix rendering issues
This example in device replace doesn't render properly in browsers,
think it's just misuse of syntax. Unfortunately, the github preview
renderer doesn't seem to catch that. Otherwise, manual page and html
rendering is ok.

The syntax uses Example delimited block:

.Heading
====
contents
====

Pull-request: #245
Author: Alexandru Ungureanu <khakcarot@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:52 +02:00
Qu Wenruo
e445c1c381 btrfs-progs: tests: introduce expand_command() to inject aruguments more accurately
[PROBLEM]
We want to inject $INSTRUMENT (mostly valgrind) before btrfs command but
after root_helper.

Currently we won't inject $INSTRUMENT at all if we are using
root_helper.
This means the coverage is not good enough.

[FIX]
This patch introduce a new function, expand_command(), to handle all
parameter/argument injection, including existing 'btrfs check' inject.

This function will:
- Detect where to inject $INSTRUMENT
  If we have root_helper and the command is target command
  (btrfs/mkfs.btrfs/btrfs-convert), then we inject $INSTRUMENT after
  root_helper.
  If we don't have root_helper, and the command is target command,
  we inject $INSTRUMENT before the command.
  Or we don't inject $INSTRUMENT (it's not the target command).

- Use existing spec facility to inject extra arguments

- Use an array to restore to result
  To avoid bash interpret the IFS inside path/commands.

Now we can make sure no matter if we use root_helper, $INSTRUMENT is
always injected corrected.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:48 +02:00
Qu Wenruo
8c759d5a06 btrfs-progs: tests: filter output for run_check_stdout
Since run_check_stdout() can insert INSTRUMENT for all btrfs related
programs, which could easily pollute the stdout, any caller of
run_check_stdout() should do proper filter.

The following callers are affected:
- misc/004
  Filter the output of "btrfs ins min-dev-size"

- misc/009
- misc/013
- misc/024
  They are all calling "btrfs ins rootid", so introduce get_subvolid()
  function to grab the subvolid properly.

- misc/031
  Loose the filter for "btrfs qgroup show". No need for "tail -n 1".

So we still have the same coverage, but now these tests won't cause
false alert if we insert INSTRUMENT for all btrfs commands.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:42 +02:00
David Sterba
82464a03e7 btrfs-progs: add more hash implementation providers
For environments that require certified implementations of cryptographic
primitives allow to select a library providing them. The requirements
are SHA256 and BLAKE2 (with the 2b variant and 256 bit digest).

For now there are two: libgrcrypt and libsodium (openssl does not
provide the BLAKE2b-256). Accellerated versions are typically provided
and automatically selected.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:40 +02:00
Qu Wenruo
b29b3e6b73 btrfs-progs: fix bad kernel header non-flat include case
If we're compiling using system header, we should include
<btrfs/crc32c.h> other than "btrfs/crc32.h".

Fixes: 2efe160bc7 ("btrfs-progs: move name hashing functions to ctree.h and delete hash.h")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:36 +02:00
David Sterba
8c85b34420 btrfs-progs: move backref.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Qu Wenruo
4aa0dcba8c btrfs-progs: do proper error handling in add_cache_extent()
If we have memory allocation failure in add_cache_extent(), it will
simply exit with one error message.

That's definitely not proper, especially when all but one call sites
have handled the error.

This patch will return -ENOMEM for add_cache_extent(), and fix the only
call site which doesn't handle error from it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Qu Wenruo
91a925a01a btrfs-progs: remove extent_buffer::tree member
This member can be fetched from eb::fs_info, and no caller really
depends on that member to determine if an eb is dummy. We have eb::flags
to determine that.

Kernel doesn't have such member either.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
David Sterba
b8143d0cda btrfs-progs: remove obsolete fs_info::fs_mutex
The code using it has been removed in 4e34bdb868 ("btrfs-progs: Remove
old commented code") and at that time it wasn't in use since
95d3f20b51 ("Mixed back reference  (FORWARD ROLLING FORMAT CHANGE)").

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Anand Jain
c360199c6c btrfs-progs: tests: drop trailing slash from TEST_TOP path
As the TEST_TOP is often only a base path, the slash gets doubled. This
is not a problem as the slashes are in the middle of the path and cannot
be confused with the initial "//" that gets interpreted as a network
path. Update all the test drivers.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Marcos Paulo de Souza
2f8aac5375 btrfs-progs: tests: mkfs/018, fix check for truncate command failure
Commit 31f477ee ("btrfs-progs: mkfs-tests: skip test if truncate fails
with EFBIG") tried to detect a failure in truncate command by checking
the $? expecting it to be an errno, when it actually returns 0 or 1.

To fix this test just check if the command failed (returned 1) and look
for the output, skipping the test if the OS cannot create a 6E file.

Fixes: #241
Tested-by: Erhard Furtner <erhard_f@mailbox.org>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Goffredo Baroncelli
24dcafc491 btrfs-progs: docs: add section about multiple profiles
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Goffredo Baroncelli
843b3c7bdc btrfs-progs: add further checks for multiple profiles
Add the warning to 'device usage' and 'filesystem df'.

Signed-off-by: Goffredo Baroncelli <kreijack@inwid.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Goffredo Baroncelli
6b535a04ee btrfs-progs: fi usage: add check for multiple profiles
A new line in the "Overall" section is added to inform that 'Multiple
profiles' are present.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:35 +02:00
Goffredo Baroncelli
138460757a btrfs-progs: add mixed profiles check to some btrfs sub-commands.
Add a check in some btrfs subcommands to detect if a filesystem
has mixed profiles for data/metadata/system. In this case
a warning is showed.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:30 +02:00
Goffredo Baroncelli
b499dbe8e1 btrfs-progs: add code for checking mixed profile function
Add code to show a warning if a mixed profiles filesystem is detected.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:24 +02:00
David Sterba
471b4cf7e3
Btrfs progs v5.6
Signed-off-by: David Sterba <dsterba@suse.com>
2020-04-05 21:43:49 +02:00
David Sterba
a7a50df517 btrfs-progs: update CHANGES for 5.6
Signed-off-by: David Sterba <dsterba@suse.com>
2020-04-05 21:41:50 +02:00
Goffredo Baroncelli
8fe2497861 btrfs-progs: Add BTRFS_EXTENDED_PROFILE_MASK mask
Add BTRFS_EXTENDED_PROFILE_MASK to consider also the
BTRFS_AVAIL_ALLOC_BIT_SINGLE bit.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-04-01 20:25:11 +02:00
Goffredo Baroncelli
9fd37f951b btrfs-progs: complete the implementation RAID1C34 definitions
- complete the function btrfs_err_str adding some missing cases
- sync the enum btrfs_err_code (in libbtrfsutil/btrfs.h) with the
  rest of the codes (user space and kernel space).
- add missing fields to btrfs_raid_array[] for raid1c[34]

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-04-01 20:20:59 +02:00
Qu Wenruo
6384f9df92 btrfs-progs: check: sanitize the return value for qgroup error
[BUG]
btrfs check can return strange return value for shell:
 [Inferior 1 (process 48641) exited with code 0213]
					      ^^^^

[CAUSE]
It's caused by the incorrect handling of qgroup error.

qgroup_report_ret can be -117 (-EUCLEAN), using that value with exit()
can cause overflow, causing return value not properly recognized.

[FIX]
Fix it by sanitize the return value to 0 or 1.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-04-01 19:56:42 +02:00
David Sterba
766c2fbd1f btrfs-progs: README: list third-party source repositories
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 20:29:45 +02:00
Adam Borowski
3d379b1341 btrfs-progs: lots of typo fixes (codespell)
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:38 +02:00
Adam Borowski
ac405aa597 btrfs-progs: check: typo in an error message: "boudnary"
Reviewed-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Marcos Paulo de Souza
58552e0a06 btrfs-progs: restore: avoid SYMLINK messages by default
Some scripts can still rely on this message, so make it available with
-vv, so -v stays sane.

Fixes: #127
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
b70eb104df btrfs-progs: extent-tree: Fix wrong post order rb tree cleanup for block groups
[BUG]
Valgrind reports memory leak for fsck/012, even after the image is
repair, the memory leak can still be reproduced.
  ==107060== HEAP SUMMARY:
  ==107060==     in use at exit: 176 bytes in 1 blocks
  ==107060==   total heap usage: 10,647 allocs, 10,646 frees, 3,000,654 bytes allocated
  ==107060==
  ==107060== 176 bytes in 1 blocks are definitely lost in loss record 1 of 1
  ==107060==    at 0x483BB65: calloc (vg_replace_malloc.c:762)
  ==107060==    by 0x1BD953: read_one_block_group (extent-tree.c:2661)
  ==107060==    by 0x1BDBD8: btrfs_read_block_groups (extent-tree.c:2719)
  ==107060==    by 0x1B3A2C: btrfs_setup_all_roots (disk-io.c:1024)
  ==107060==    by 0x1B44CA: __open_ctree_fd (disk-io.c:1299)
  ==107060==    by 0x1B46C6: open_ctree_fs_info (disk-io.c:1345)
  ==107060==    by 0x16952E: cmd_check (main.c:10154)
  ==107060==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==107060==    by 0x11D712: main (btrfs.c:386)
  ==107060==
  ==107060== LEAK SUMMARY:
  ==107060==    definitely lost: 176 bytes in 1 blocks
  ==107060==    indirectly lost: 0 bytes in 0 blocks
  ==107060==      possibly lost: 0 bytes in 0 blocks
  ==107060==    still reachable: 0 bytes in 0 blocks
  ==107060==         suppressed: 0 bytes in 0 blocks

[CAUSE]
In btrfs_free_block_groups(), we use
rbtree_postorder_for_each_entry_safe() to iterate all block group cache.

However since we're already doing post order iteration, we shouldn't
call rb_erase() during that iteration, as it would re-balance the tree,
and break the post order iteration.

This wrong rb_erase() call leads to above memory leak.

[FIX]
Kill that wrong rb_erase() call.

Fixes: b1bd3cd93f ("btrfs-progs: reform block groups caches structure")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
09e374eac9 btrfs-progs: check/original: Fix uninitialized extent buffer contents
[BUG]
Valgrind reports the following error for fsck/012:
  adding new tree backref on start 4206592 len 4096 parent 0 root 5
  ==100735== Syscall param pwrite64(buf) points to uninitialised byte(s)
  ==100735==    at 0x49F303A: pwrite (in /usr/lib/libpthread-2.31.so)
  ==100735==    by 0x1A5C85: write_extent_to_disk (extent_io.c:815)
  ==100735==    by 0x1B2507: write_and_map_eb (disk-io.c:512)
  ==100735==    by 0x1B26A7: write_tree_block (disk-io.c:545)
  ==100735==    by 0x1D4822: __commit_transaction (transaction.c:148)
  ==100735==    by 0x1D4AA2: btrfs_commit_transaction (transaction.c:213)
  ==100735==    by 0x16360D: fixup_extent_refs (main.c:7662)
  ==100735==    by 0x16449F: check_extent_refs (main.c:8033)
  ==100735==    by 0x166199: check_chunks_and_extents (main.c:8786)
  ==100735==    by 0x166441: do_check_chunks_and_extents (main.c:8842)
  ==100735==    by 0x169D13: cmd_check (main.c:10324)
  ==100735==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==100735==  Address 0x4e8aeb0 is 128 bytes inside a block of size 4,224 alloc'd
  ==100735==    at 0x483BB65: calloc (vg_replace_malloc.c:762)
  ==100735==    by 0x1A54C5: __alloc_extent_buffer (extent_io.c:609)
  ==100735==    by 0x1A5AD1: alloc_extent_buffer (extent_io.c:752)
  ==100735==    by 0x1B1A0A: btrfs_find_create_tree_block (disk-io.c:222)
  ==100735==    by 0x1BD4A2: btrfs_alloc_free_block (extent-tree.c:2538)
  ==100735==    by 0x1A8CE3: __btrfs_cow_block (ctree.c:322)
  ==100735==    by 0x1A91C6: btrfs_cow_block (ctree.c:415)
  ==100735==    by 0x1AB16C: btrfs_search_slot (ctree.c:1185)
  ==100735==    by 0x160BBC: delete_extent_records (main.c:6652)
  ==100735==    by 0x16343F: fixup_extent_refs (main.c:7629)
  ==100735==    by 0x16449F: check_extent_refs (main.c:8033)
  ==100735==    by 0x166199: check_chunks_and_extents (main.c:8786)
  ==100735==

[CAUSE]
For new extent buffer allocated, we don't initialize its content.

This is not a major concern, at all.
For the above report, the reported range is inside the unused part of
the extent buffer, thus won't cause anything.

Regular btrfs_cow_block() will cover all the used ranges of one extent
buffer.

[FIX]
But still, since kernel initialize the extent buffer with 0, it won't
hurt to do extra initialized to make valgrind happy.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
3972c27db6 btrfs-progs: check/original: Fix uninitialized return value from btrfs_write_dirty_block_groups()
[BUG]
Valgrind reports the following error for fsck/007, which is only
repairable for original mode:
  ==97599== Conditional jump or move depends on uninitialised value(s)
  ==97599==    at 0x1D4A42: btrfs_commit_transaction (transaction.c:207)
  ==97599==    by 0x16475C: check_extent_refs (main.c:8097)
  ==97599==    by 0x166199: check_chunks_and_extents (main.c:8786)
  ==97599==    by 0x166441: do_check_chunks_and_extents (main.c:8842)
  ==97599==    by 0x169D13: cmd_check (main.c:10324)
  ==97599==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==97599==    by 0x11D712: main (btrfs.c:386)
  ==97599==

[CAUSE]
If btrfs_write_dirty_block_groups() get called with no block group
dirtied (no dirty extents created), the return value of it is
uninitialized, as the stack @ret is not initialized at all.

[FIX]
Initialize @ret to 0 for btrfs_write_dirty_block_groups() as if there is
no dirty block groups, we do nothing and shouldn't fail.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
3c2dbee299 btrfs-progs: check/original: Fix uninitialized memory for newly allocated data_backref
[BUG]
Valgrind reports the following error for fsck/002 (which only supports
original mode):
  ==97088== Conditional jump or move depends on uninitialised value(s)
  ==97088==    at 0x15BFF6: add_data_backref (main.c:4884)
  ==97088==    by 0x16025C: run_next_block (main.c:6452)
  ==97088==    by 0x165539: deal_root_from_list (main.c:8471)
  ==97088==    by 0x166040: check_chunks_and_extents (main.c:8753)
  ==97088==    by 0x166441: do_check_chunks_and_extents (main.c:8842)
  ==97088==    by 0x169D13: cmd_check (main.c:10324)
  ==97088==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==97088==    by 0x11D712: main (btrfs.c:386)

[CAUSE]
In alloc_data_backref(), only ref->node is set to 0.
While ref->disk_bytenr is not initialized at all.

And then in add_data_backref(), if @back is a newly allocated data
backref, we use the garbage from back->disk_bytenr to determine if we
should reset them.

[FIX]
Fix it by initialize the whole data_backref structure in
alloc_data_backref().

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
0a472ef887 btrfs-progs: check/original: Fix uninitialized stack memory access for deal_root_from_list()
[BUG]
With valgrind, fsck/002 test with original mode would report the
following valgrind error:
  ==90600== Conditional jump or move depends on uninitialised value(s)
  ==90600==    at 0x15C280: pick_next_pending (main.c:4949)
  ==90600==    by 0x15F3CF: run_next_block (main.c:6175)
  ==90600==    by 0x1655CC: deal_root_from_list (main.c:8486)
  ==90600==    by 0x1660C7: check_chunks_and_extents (main.c:8762)
  ==90600==    by 0x166439: do_check_chunks_and_extents (main.c:8842)
  ==90600==    by 0x169D0B: cmd_check (main.c:10324)
  ==90600==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==90600==    by 0x11D712: main (btrfs.c:386)

[CAUSE]
The problem happens like this:
deal_root_from_list(@list is empty)
|- stack @last is not initialized
|- while(!list_empty(list)) {} is skipped
|- run_next_block(&last);
   |- pick_next_pending(*last);
      |- node_start = last;

Since the stack @last is not initialized in deal_root_from_list(), the
final node_start = last assignment would just fetch the garbage from
stack.

[FIX]
Fix the problem by initializing @last to 0, as that's exactly what the
first while loop did.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
e711c843df btrfs-progs: tests/common: Don't call INSTRUMENT on mount command
[BUG]
With INSTRUMENT=valgrind set, some fsck tests will fail, e.g. fsck/013:
  ====== RUN CHECK mount -t btrfs -o loop /home/adam/btrfs/btrfs-progs/tests//test.img /home/adam/btrfs/btrfs-progs/tests//mnt
  ==114106==
  ==114106== Warning: Can't execute setuid/setgid/setcap executable: /usr/bin/mount
  ==114106== Possible workaround: remove --trace-children=yes, if in effect
  ==114106==
  valgrind: /usr/bin/mount: Permission denied
  failed: mount -t btrfs -o loop /home/adam/btrfs/btrfs-progs/tests//test.img /home/adam/btrfs/btrfs-progs/tests//mnt
  test failed for case 013-extent-tree-rebuild

[CAUSE]
Just as stated by valgrind itself, it can't handle program with
setuid/setgid/setcap.

Thankfully in our case it's mount and we don't really care about it at
all.

[FIX]
Although we could use complex skip pattern to skip mount in valgrind, we
don't really want to run valgrind on mount or sudo command anyway.

So here we do extra check if we're running mount command. And if that's
the case, just skip $INSTRUMENT command.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:37 +02:00
Qu Wenruo
3a2d040070 btrfs-progs: check/lowmem: Fix access on uninitialized memory
[BUG]
There are some reports on fsck/001 test segfault failure with lowmem mode.

While I failed to reproduce it, valgrind still catches it with the
following output:

  Delete backref in extent [12845056 1048576]
  ERROR: file extent [257, 0] has unaligned disk bytenr: 755944791, should be aligned to 4096
  ERROR: file extent[257 0] root 5 owner 5 backref lost
  Deleted root 5 item[257, 108, 0]
  ==29080== Conditional jump or move depends on uninitialised value(s)
  ==29080==    at 0x1A81D7: btrfs_release_path (ctree.c:97)
  ==29080==    by 0x192C33: repair_extent_data_item (mode-lowmem.c:3330)
  ==29080==    by 0x1962FF: check_leaf_items (mode-lowmem.c:4696)
  ==29080==    by 0x196ABF: walk_down_tree (mode-lowmem.c:4858)
  ==29080==    by 0x197762: check_btrfs_root (mode-lowmem.c:5157)
  ==29080==    by 0x198335: check_chunks_and_extents_lowmem (mode-lowmem.c:5450)
  ==29080==    by 0x166414: do_check_chunks_and_extents (main.c:8829)
  ==29080==    by 0x169CF7: cmd_check (main.c:10313)
  ==29080==    by 0x11CDC6: cmd_execute (commands.h:125)
  ==29080==    by 0x11D712: main (btrfs.c:386)
  ==29080==

[CAUSE]
In repair_extent_data_item() if we find unaligned file extent, we just
delete it and kick in hole punch procedure.

The problem is, file extent deletion is done before initializing @path.
And when the deletion is done without problem, we will goto out tag,
which will release @path, containing uninitialized values, and
triggering segfault.

[FIX]
Don't try to abort trans nor free path if we're going through file
extent deletion routine.

Fixes: 0617bde3bc ("btrfs-progs: lowmem: delete unaligned bytes extent data under repair")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
David Sterba
1311f1b83a btrfs-progs: misc-tests/039: cleanup test
Cleanups done separately:

* use the default test image, loop devices not needed for the test
* trim TEST_MNT from all paths
* send output is created inside the test filesystem

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Omar Sandoval
25e3e5cb12 btrfs-progs: tests: add test for receiving clone from duplicate subvolume
This test case is the reproducer for the previous fix.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Omar Sandoval
018306809b btrfs-progs: receive: don't lookup clone root for received subvolume
When we process a clone request, we look up the source subvolume by
UUID, even if the source is the subvolume that we're currently
receiving. Usually, this is fine. However, if for some reason we
previously received the same subvolume, then this will use paths
relative to the previously received subvolume instead of the current
one. This is incorrect, since the send stream may use temporary names
for the clone source. This can be reproduced as follows:

  btrfs subvolume create subvol
  dd if=/dev/urandom of=subvol/foo bs=1M count=1
  cp --reflink subvol/foo subvol/bar
  mkdir subvol/dir
  mv subvol/foo subvol/dir/
  btrfs property set subvol ro true
  btrfs send -f send.data subvol
  mkdir first second
  btrfs receive -f send.data first
  btrfs receive -f send.data second

The second receive results in this error:

  ERROR: cannot open first/subvol/o259-7-0/foo: No such file or directory

Fix it by always cloning from the current subvolume if its UUID matches.
This has the nice side effect of avoiding unnecessary UUID tree lookups
in that case.

Fixes: f1c24cd80d ("Btrfs-progs: add btrfs send/receive commands")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Omar Sandoval
d451fe2018 btrfs-progs: receive: remove commented out transid checks
The checks for a subvolume being modified after it was received have
been commented out since they were added back in commit f1c24cd80d
("Btrfs-progs: add btrfs send/receive commands"). Let's just get rid of
the noise.

If they were ever in place, it would have never been possible
to do an incremental send and running dedupe against the parent
snapshot.
That particular use case used to cause send, the kernel side, to fail
(initially with a BUG_ON() and later with -EIO returned to user
space), see commit b4f9a1a87a48 ("Btrfs: fix incremental send failure
after deduplication").

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
[ add Filipe's note ]
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Marcos Paulo de Souza
59af046fd2 btrfs-progs: qgroup-verify: Remove duplicated message in report_qgroups
Since 1d5b2ad9 ("btrfs-progs: qgroup-verify: Don't treat qgroup
difference as error if the fs hasn't initialized a rescan") a new
message is being printed when the qgroups is incosistent and the rescan
hasn't being executed, so remove the later message send to stderr.

While in this function, simplify the check for a not executed rescan
since !counts.rescan_running and counts.rescan_running == 0 means the
same thing.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
David Sterba
7b2712bf92 btrfs-progs: build: add target for check lowmem mode
Running the lowmem mode for check needs some setup and is not usually
tested, so add a new target that sets up the variables.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
David Sterba
f541f59e9a btrfs-progs: tests: check if subvolume delete by id is supported
Skip the test 013-subvolume-delete-by-id if the first valid attempt to
use the ioctl fails with 'Inappropriate ioctl for device'.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
David Sterba
a3b12468c0 btrfs-progs: tests: add run_mayfail_stdout helper
Add a variant of mayfail helper that will duplicate the output to
results log and also provides it to the caller for processing. Can be
used for catching unsupported functionality or other special cases.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:36 +02:00
Nikolay Borisov
5b13164cf9 btrfs-progs: fix xxhash on big endian machines
xxhash's state and results are always in little, but in progs after the
hash was calculated it was copied to the final buffer via memcpy,
meaning it'd be parsed as a big endian number on big endian machines.
This is incompatible with the kernel implementation of xxhash which
results in erroneous "checksum didn't match" errors on mount.

Fix it by using put_unaligned_le64 which always ensures the resulting
checksum will be copied in little endian format as the kernel expects
it.

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=206835
Fixes: f070ece2e9 ("btrfs-progs: add xxhash64 to mkfs")
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Josef Bacik
1173a4ba26 btrfs-progs: fix hole error output in fsck
If we don't find holes in our hole rb tree we'll just assume there's a
gap from 0 to the length of the file and print that out.  But this
simply isn't correct, we could have a gap between the last extent and
the isize, or 0 and the start of the first extent.  Fix the error
message to tell us exactly where the hole is.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Josef Bacik
f2832d534b btrfs-progs: fix lowmem check's handling of holes
Lowmem check had the opposite problem of normal check, it caught gaps
that started at 0, but would still fail with my fixes in place.  This is
because lowmem check doesn't take into account the isize of the inode.
Address this by making sure we do not complain about gaps that are after
isize.  This makes lowmem pass with my fixes applied, and still fail
without my fixes.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Josef Bacik
9123fc6142 btrfs-progs: fix check to catch gaps at the start of the file
When writing my test for the i_size patches, I noticed that I was not
actually failing without my patches as I should have been.  This is
because we only check if the inode record extent end is < isize, we
don't check if the inode record extent start is > 0.  Add this check to
make sure we're catching holes that start at the beginning of the file.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Anand Jain
fae40e9d73 btrfs-progs: convert, warn if converting a fs which won't mount
On aarch64 with pagesize 64k, btrfs-convert of ext4 is successful,
but it won't mount because we don't yet support subpage blocksize, ie.
when page size and sectorsize don't match.

 BTRFS error (device vda): sectorsize 4096 not supported yet, only support 65536

So in this case during convert provide a warning but let the conversion
proceed.

Example:

WARNING: Blocksize 4096 is not equal to the pagesize 65536,
         converted filesystem won't mount on this system.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
Qu Wenruo
41ff76b833 btrfs-progs: restore: Do proper mirror iteration in copy_one_extent()
The old code of copy_one_extent() is a mess:
- The main loop is implemented using goto
- @mirror_num is reset to 1 for each loop
- @mirror num check against @num_copies is wrong for decompression error

This patch will fix this mess by:
- Use read_extent_data()
  read_extent_data() has all the good wrapping of btrfs_map_block()
  and length check.
  This removes a lot of complexity.

- Add extra file extent offset check
  To prevent underflow for memory allocation

- Do proper mirror_num check for decompression error

Issue: #221
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
David Sterba
2efe160bc7 btrfs-progs: move name hashing functions to ctree.h and delete hash.h
The two functions are trivial and kernel has them in ctree.h, so do the
same and remove hash.h.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
David Sterba
35ba1f5fb6 btrfs-progs: move common-defs to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
David Sterba
58bcd4260f btrfs-progs: move free-space-tree.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00
David Sterba
019489a143 btrfs-progs: move delayed-ref.[ch] to kernel-shared/
The files are very close to kernel versions, for that keep them in the
shared directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:34 +02:00