Commit Graph

357 Commits

Author SHA1 Message Date
Anand Jain
32c2e57c65 btrfs-progs: read fsid from the sysfs in device_is_seed
The kernel commit a26d60dedf9a ("btrfs: sysfs: add devinfo/fsid to
retrieve actual fsid from the device") introduced a sysfs interface
to access the device's fsid from the userspace. This is a more
reliable method to obtain the fsid compared to reading the
superblock, and it even works if the device is not present.
Additionally, this sysfs interface can be read by non-root users.

Therefore, it is recommended to utilize this new sysfs interface to
retrieve the fsid.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
Anand Jain
bdd0ca8a9a btrfs-progs: prepare helper device_is_seed
load_device_info() checks if the device is a seed device by reading
superblock::fsid and comparing it with the mount fsid, and it fails
to work if the device is missing (a RAID1 seed fs). Move this part
of the code into a new helper function device_is_seed() in
preparation to make device_is_seed() work with the new sysfs
devinfo/<devid>/fsid interface.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
b08b429ee2 btrfs-progs: fi mkswapfile: add option --uuid
Add option --uuid with same semantics that is provided by command
'mkswap'. By default a random UUID is generated, to not set any use
'btrfs filesystem mkswapfile -U clear swapfile'.

Issue: #581
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
5c663502bf btrfs-progs: build: use plain fcntl.h instead of sys/fcntl.h
The CI build on musl warns:

    [CC]     cmds/balance.o
In file included from cmds/inspect.c:19:
/usr/include/sys/fcntl.h:1:2: warning: #warning redirecting incorrect #include <sys/fcntl.h> to <fcntl.h> [-Wcpp]
    1 | #warning redirecting incorrect #include <sys/fcntl.h> to <fcntl.h>
      |  ^~~~~~~

On glibc the header directly includes fcntl.h.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
Qu Wenruo
f914949b1a btrfs-progs: fix set but not used variables
[WARNING]
Clang 15.0.7 warns about several unused variables:

  kernel-shared/zoned.c:829:6: warning: variable 'num_sequential' set but not used [-Wunused-but-set-variable]
          u32 num_sequential = 0, num_conventional = 0;
              ^
  cmds/scrub.c:1174:6: warning: variable 'n_skip' set but not used [-Wunused-but-set-variable]
          int n_skip = 0;
              ^
  mkfs/main.c:493:6: warning: variable 'total_block_count' set but not used [-Wunused-but-set-variable]
          u64 total_block_count = 0;
              ^
  image/main.c:2246:6: warning: variable 'bytenr' set but not used [-Wunused-but-set-variable]
          u64 bytenr = 0;
              ^

[CAUSE]
Most of them are just straightforward set but not used variables.

The only exception is total_block_count, which has commented out code
relying on it.

[FIX]
Just remove those variables, and for @total_block_count, also remove the
comments.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-18 17:44:03 +01:00
Qu Wenruo
3a1d4aa089 btrfs-progs: fix fallthrough cases with proper attributes
[FALSE ALERT]
Unlike gcc, clang doesn't really understand the comments, thus it's
reportings tons of fall through related errors:

  cmds/reflink.c:124:3: warning: unannotated fall-through between switch labels [-Wimplicit-fallthrough]
                  case 'r':
                  ^
  cmds/reflink.c:124:3: note: insert '__attribute__((fallthrough));' to silence this warning
                  case 'r':
                  ^
                  __attribute__((fallthrough));
  cmds/reflink.c:124:3: note: insert 'break;' to avoid fall-through
                  case 'r':
                  ^
                  break;

[CAUSE]
Although gcc is fine with /* fallthrough */ comments, clang is not.

[FIX]
So just introduce a fallthrough macro to handle the situation properly,
and use that macro instead.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-18 17:44:02 +01:00
Filipe Manana
e3209f8792 btrfs-progs: receive: fix a corruption when decompressing zstd extents
Before decompressing, we zero out the content of the entire output buffer,
so that we don't get any garbage after the last byte of data. We do this
for all compression algorithms. However zstd, at least with libzstd 1.5.2
on Debian (version 1.5.2+dfsg-1), the decompression routine can end up
touching the content of the output buffer beyond the last valid byte of
decompressed data, resulting in a corruption.

Example reproducer:

   $ cat test.sh
   #!/bin/bash

   DEV=/dev/sdj
   MNT=/mnt/sdj

   rm -f /tmp/send.stream

   umount $DEV &> /dev/null
   mkfs.btrfs -f $DEV &> /dev/null || echo "MKFS failed!"
   mount -o compress=zstd $DEV $MNT

   # File foo is not sector size aligned, 127K.
   xfs_io -f -c "pwrite -S 0xab 0 3" \
             -c "pwrite -S 0xcd 3 130042" \
             -c "pwrite -S 0xef 130045 3" $MNT/foo

   # Now do an fallocate that increases the size of foo from 127K to 128K.
   xfs_io -c "falloc 0 128K " $MNT/foo

   btrfs subvolume snapshot -r $MNT $MNT/snap

   btrfs send --compressed-data -f /tmp/send.stream $MNT/snap

   echo -e "\nFile data in the original filesystem:\n"
   od -A d -t x1 $MNT/snap/foo

   umount $MNT
   mkfs.btrfs -f $DEV &> /dev/null || echo "MKFS failed!"
   mount $DEV $MNT

   btrfs receive --force-decompress -f /tmp/send.stream $MNT

   echo -e "\nFile data in the new filesystem:\n"
   od -A d -t x1 $MNT/snap/foo

   umount $MNT

Running the reproducer gives:

   $ ./test.sh
   (...)
   File data in the original filesystem:

   0000000 ab ab ab cd cd cd cd cd cd cd cd cd cd cd cd cd
   0000016 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
   *
   0130032 cd cd cd cd cd cd cd cd cd cd cd cd cd ef ef ef
   0130048 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   *
   0131072
   At subvol snap

   File data in the new filesystem:

   0000000 ab ab ab cd cd cd cd cd cd cd cd cd cd cd cd cd
   0000016 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
   *
   0130032 cd cd cd cd cd cd cd cd cd cd cd cd cd ef ef ef
   0130048 cd cd cd cd 00 00 00 00 00 00 00 00 00 00 00 00
   0130064 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   *
   0131072

The are 4 bytes with a value of 0xcd instead of 0x00, at file offset
127K (130048).

Fix this by explicitly zeroing out the part of the output buffer that was
not used after decompressing with zstd.

The decompression of compressed extents, sent when using the send v2
stream, happens in the following cases:

1) By explicitly passing --force-decompress to the receive command, as in
   the reproducer above;

2) Calling the BTRFS_IOC_ENCODED_WRITE ioctl failed with -ENOTTY, meaning
   the kernel on the receiving side is old and does not implement that
   ioctl;

3) Calling the BTRFS_IOC_ENCODED_WRITE ioctl failed with -ENOSPC;

4) Calling the BTRFS_IOC_ENCODED_WRITE ioctl failed with -EINVAL.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-18 17:44:02 +01:00
David Sterba
7305c69cf9 btrfs-progs: subvol sync: print count and progress
By default print how many subvolumes are considered for checks, either
found or specified on the command line. Once a subvolume is removed from
the list print the progress from the total count.

  $ btrfs subvolume sync /path
  Waiting for 130 subvolumes
  Subvolume id 256 is gone (1/130)
  Subvolume id 257 is gone (2/130)
  ...
  Subvolume id 384 is gone (129/130)
  Subvolume id 385 is gone (130/130)

Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-25 19:55:47 +01:00
David Sterba
54b90cb6e5 btrfs-progs: fi mkswapfile: fix page count in header
Per user report on https://old.reddit.com/r/btrfs/comments/107fnw1/btrfs_filesystem_mkswapfile_results_in_an/
the swapfile header does not contain the correct number of pages that
matches the file size and the activated swapfile is only 1GiB:

  # btrfs filesystem mkswapfile -s 10g swapfile
  # swapon swapfile
  # cat /proc/swaps
  Filename          Type    Size       Used    Priority
  /swap/swapfile    file    1048572    0       -2

A workaround is to run 'mkswap swapfile' before activation. Proper fix
is to calculate the number of (fixed size) 4K pages available for the
swap.

Issue: #568
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-25 16:19:36 +01:00
David Sterba
6fd4241e87 btrfs-progs: inspect: add build conditionals around list-chunk
In default build there's a warning (reported by CI) that the
experimental list-chunk command and related functions are not used, so
add the condition there as well.

  In file included from cmds/inspect.c:45:
  ./cmds/commands.h:67:26: warning: 'cmd_struct_inspect_list_chunks' defined but not used [-Wunused-const-variable=]
   #define __CMD_NAME(name) cmd_struct_ ##name
			    ^~~~~~~~~~~
  ./cmds/commands.h:73:26: note: in expansion of macro '__CMD_NAME'
    const struct cmd_struct __CMD_NAME(name) =   \
			    ^~~~~~~~~~
  ./cmds/commands.h:88:2: note: in expansion of macro 'DEFINE_COMMAND'
    DEFINE_COMMAND(name, token, cmd_ ##name,   \
    ^~~~~~~~~~~~~~
  cmds/inspect.c:1115:8: note: in expansion of macro 'DEFINE_SIMPLE_COMMAND'
   static DEFINE_SIMPLE_COMMAND(inspect_list_chunks, "list-chunks");
	  ^~~~~~~~~~~~~~~~~~~~~

Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-05 14:59:12 +01:00
David Sterba
e78fe2d92e Revert "btrfs-progs: rename qgroup items to match the kernel naming scheme"
This reverts commit 03451430de7cd2fad18b0f01745545758bc975a5.
(It's not 1:1, there are some additional trivial fixups in cmds/qgroup.c)

This breaks a lot of 3rd party tools that depend on it as Neal reports:

* btrfs-assistant
* buildah
* cri-o
* podman
* skopeo
* containerd
* moby/docker
* snapper
* source-to-image

Link: https://lore.kernel.org/linux-btrfs/CAEg-Je8L7jieKdoWoZBuBZ6RdXwvwrx04AB0fOZF1fr5Pb-o1g@mail.gmail.com/
Reported-by: Neal Gompa <ngompa@fedoraproject.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-03 13:10:54 +01:00
Qu Wenruo
b7e3acd615 btrfs-progs: subvol list: fix the wrong timestamp and UUID check for root items
[BUG]
Since commit d729048be6ef ("btrfs-progs: stop using
btrfs_root_item_v0"), "btrfs subvolume list -u" not longer correctly
reports UUID nor timestamp, while older (btrfs-progs v6.0.2) still works
correctly:

 v6.0.2:
 # btrfs subv list -u  /mnt/btrfs/
 ID 256 gen 12 top level 5 uuid ed4af580-d512-2644-b392-2a71aaeeb99e path subv1
 ID 257 gen 13 top level 5 uuid a22ccba7-0a0a-a94f-af4b-5116ab58bb61 path subv2

 v6.1:
 # ./btrfs subv list -u /mnt/btrfs/
 ID 256 gen 12 top level 5 uuid -                                    path subv1
 ID 257 gen 13 top level 5 uuid -                                    path subv2

[CAUSE]
Commit d729048be6ef ("btrfs-progs: stop using btrfs_root_item_v0")
removed old btrfs_root_item_v0, but incorrectly changed the check for
v0 root item.

Now we will treat v0 root items as latest root items, causing possible
out-of-bound access, while treating current root items as older v0 root
items, ignoring the UUID nor timestamp.

[FIX]
Fix the bug by using correct checks, and add extra comments on the
branches.

Issue: #562
Fixes: d729048be6ef ("btrfs-progs: stop using btrfs_root_item_v0")
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-01-02 13:57:29 +01:00
David Sterba
5684d2f3d9 btrfs-progs: device stats: fix json formatter type for devid
Devid is u64 but the formatter is just for u32.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-16 00:49:07 +01:00
David Sterba
62d818e365 btrfs-progs: use escaped json format for paths
Use the new escaped formatter for paths in command 'device stats' and
'qgroup show'.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-16 00:43:25 +01:00
David Sterba
6ecaa3ee4f btrfs-progs: qgroup show: add json output
Support json for 'qgroup show' with values printed by "btrfs qgroup
-pcre", the accounted size and the limits.  It's implemented as a
separate call and not sharing the printing routines so any visible
changes need to by synchronized.

Formatter updates: don't print key name if .out_json is NULL.

Example output:

 # btrfs --format json qgroup show /mnt/path
{
  "__header": {
    "version": "1"
  },
  "qgroup-show": [
    {
      "qgroupid": "0/5",
      "referenced": "8831393792",
      "max_referenced": "none",
      "exclusive": "8224075776",
      "max_exclusive": "none",
      "path": "",
      "parents": [
      ],
      "children": [
      ]
    },
    {
      "qgroupid": "0/361",
      "referenced": "611459072",
      "max_referenced": "none",
      "exclusive": "65536",
      "max_exclusive": "none",
      "path": "subv1",
      "parents": [
        "1/1"
      ],
      "children": [
      ]
    },
    {
      "qgroupid": "0/362",
      "referenced": "611459072",
      "max_referenced": "none",
      "exclusive": "65536",
      "max_exclusive": "none",
      "path": "snap1-r",
      "parents": [
      ],
      "children": [
      ]
    },
    {
      "qgroupid": "1/1",
      "referenced": "611459072",
      "max_referenced": "none",
      "exclusive": "65536",
      "max_exclusive": "none",
      "path": "",
      "parents": [
      ],
      "children": [
        "0/361"
      ]
    }
  ]
}

Issue: #555
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-14 23:42:03 +01:00
David Sterba
900d56f1b9 btrfs-progs: add new group reflink and command
Add initial reflink group with example command 'clone' to test the
interface. Work in progress, experimental build needed.

Issue: #396
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-07 01:43:30 +01:00
David Sterba
427bf75a1e btrfs-progs: fi mkswapfile: update help text
The help text of mkswapfile was a copy&paste leftover, fix it.

Issue: #554
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 23:19:13 +01:00
David Sterba
0b3df991db btrfs-progs: fi df: implement json output
Print JSON output with example output:

  $ btrfs --format json filesystem df /path
  {
    "__header": {
      "version": "1"
    },
    "filesystem-df": [
      {
	"bg-type": "Data",
	"bg-profile": "single",
	"total": "841813590016",
	"used": "494085484544"
      },
      {
	"bg-type": "System",
	"bg-profile": "single",
	"total": "33554432",
	"used": "114688"
      },
      {
	"bg-type": "Metadata",
	"bg-profile": "single",
	"total": "24696061952",
	"used": "5743165440"
      },
      {
	"bg-type": "GlobalReserve",
	"bg-profile": "single",
	"total": "536870912",
	"used": "0"
      }
    ]
  }

Optionally there's "zone_unusable" key. The rowspec cannot print the
same output as now so the original implementation is preserved.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 21:03:31 +01:00
David Sterba
b16b35e074 btrfs-progs: inspect: new command map-swapfile
Verify if a given file is suitable for a swapfile and print the physical
offset (ie. the ultimate on-device physical offset), and the resume
offset value (physical / page size).

This can be the kernel parameter or written to /sys/power/resume_offset
before hibernation.  Option -r or --resume-offset prints just the value.

Copied and simplified from Omar Sandoval's tool to print extents:

https://github.com/osandov/osandov-linux/blob/master/scripts/btrfs_map_physical.c

Issue: #544
Issue: #533
Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-06 21:03:31 +01:00
David Sterba
99979a53bb btrfs-progs: inspect: new subcommand to list chunks
New command 'btrfs inspect-internal list-chunks' will list layout of
chunks as stored on the devices. This corresponds to the physical
layout, sorted by the physical offset. The block group usage can be
shown as well, but the search is too slow so it's off by default.

If the physical offset sorting is selected, the empty space between
chunks is also shown.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-12-05 20:28:35 +01:00
Josef Bacik
5324ca5ae0 btrfs-progs: don't use btrfs_header_csum helper
This is a useless helper, the csum is always the first thing in the
header, simply read from offset 0.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-30 19:14:29 +01:00
Josef Bacik
788a71c16a btrfs-progs: sync compression.h from the kernel
This patch copies in compression.h from the kernel.  This is relatively
straightforward, we just have to drop the compression types definition
from ctree.h, and update the image to use BTRFS_NR_COMPRESS_TYPES
instead of BTRFS_COMPRESS_LAST, and add a few things to kerncompat.h to
make everything build smoothly.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-30 19:14:29 +01:00
Josef Bacik
d729048be6 btrfs-progs: stop using btrfs_root_item_v0
This isn't defined in the kernel, we simply check if the root item size
is less than btrfs_root_item, so adjust the user of btrfs_root_item_v0
to make a similar check.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik
d8de39615a btrfs-progs: ioctl: move dev-replace NO_RESULT definition into replace.c
BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_RESULT is defined to make sure we
differentiate internal errors from actual error codes that come back
from the device replace ioctl.  Take this out of ioctl.c and move it
into replace.c.

Though it's in public header ioctl.h, there are no users for that so it
can be moved.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik
53f8b8fd01 btrfs-progs: make btrfs_qgroup_level helper match the kernel
We return __u16 in the kernel, as this is actually the size of
btrfs_qgroup_level.  Adjust the existing helpers and update all the
callers to deal with the new size appropriately.  This will make syncing
the kernel code cleaner.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Josef Bacik
03451430de btrfs-progs: rename qgroup items to match the kernel naming scheme
We're going to sync the kernel source into btrfs-progs, and in the
kernel we have all these qgroup fields named with short names instead of
the full name, so rename

referenced -> rfer
compressed -> cmpr
exclusive -> excl

to match the kernel and update all the users, this will make the sync
cleaner.

ioctl.h is a public header but there are no users of the
btrfs_qgroup_limit structure.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-28 18:57:43 +01:00
Filipe Manana
6f4a51886b btrfs-progs: receive: fix silent data loss after fall back from encoded write
When attempting an encoded write, if it fails for some specific reason
like -EINVAL (when an offset is not sector size aligned) or -ENOSPC, we
then fallback into decompressing the data and writing it using regular
buffered IO. This logic however is not correct, one of the reasons is
that it assumes the encoded offset is smaller than the unencoded file
length and that they can be compared, but one is an offset and the other
is a length, not an end offset, so they can't be compared to get correct
results. This bad logic will often result in not copying all data, or even
no data at all, resulting in a silent data loss. This is easily seen in
with the following reproducer:

   $ cat test.sh
   #!/bin/bash

   DEV=/dev/sdj
   MNT=/mnt/sdj

   umount $DEV &> /dev/null
   mkfs.btrfs -f $DEV > /dev/null
   mount -o compress $DEV $MNT

   # File foo has a size of 33K, not aligned to the sector size.
   xfs_io -f -c "pwrite -S 0xab 0 33K" $MNT/foo

   xfs_io -f -c "pwrite -S 0xcd 0 64K" $MNT/bar

   # Now clone the first 32K of file bar into foo at offset 0.
   xfs_io -c "reflink $MNT/bar 0 0 32K" $MNT/foo

   # Snapshot the default subvolume and create a full send stream (v2).
   btrfs subvolume snapshot -r $MNT $MNT/snap

   btrfs send --compressed-data -f /tmp/test.send $MNT/snap

   echo -e "\nFile bar in the original filesystem:"
   od -A d -t x1 $MNT/snap/bar

   umount $MNT
   mkfs.btrfs -f $DEV > /dev/null
   mount $DEV $MNT

   echo -e "\nReceiving stream in a new filesystem..."
   btrfs receive -f /tmp/test.send $MNT

   echo -e "\nFile bar in the new filesystem:"
   od -A d -t x1 $MNT/snap/bar

   umount $MNT

Running the test without this patch:

   $ ./test.sh
   (...)
   File bar in the original filesystem:
   0000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
   *
   0065536

   Receiving stream in a new filesystem...
   At subvol snap

   File bar in the new filesystem:
   0000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd
   *
   0033792

We end up with file bar having less data, and a smaller size, than in the
original filesystem.

This happens because when processing file bar, send issues the following
commands:

   clone bar - source=foo source offset=0 offset=0 length=32768
   write bar - offset=32768 length=1024
   encoded_write bar - offset=33792, len=4096, unencoded_offset=33792, unencoded_file_len=31744, unencoded_len=65536, compression=1, encryption=0

The first 32K are cloned from file foo, as that file ranged is shared
between the files.

Then there's a regular write operation for the file range [32K, 33K),
since file foo has different data from bar for that file range.

Finally for the remainder of file bar, the send side issues an encoded
write since the extent is compressed in the source filesystem, for the
file offset 33792 (33K), remaining 31K of data. The receiver will try the
encoded write, but that fails with -EINVAL since the offset 33K is not
sector size aligned, so it will fallback to decompressing the data and
writing it using regular buffered writes. However that results in doing
no writes at decompress_and_write() because 'pos' is initialized to the
value of 33K (unencoded_offset) and unencoded_file_len is 31K, so the
while loop has no iterations.

Another case where we can fallback to decompression plus regular buffered
writes is when the destination filesystem has a sector size larger then
the sector size of the source filesystem (for example when the source
filesystem is on x86_64 with a 4K sector size and the destination
filesystem is PowerPC with a 64K sector size). In that scenario encoded
write attempts will fail with -EINVAL due to offsets not being aligned
with the sector size of the destination filesystem, and the receive will
attempt the fallback of decompressing the buffer and writing the
decompressed using regular buffered IO.

Fix this by tracking the number of written bytes instead, and increment
it, and the unencoded offset, after each write.

Fixes: d20e759fc9 ("btrfs-progs: receive: encoded_write fallback to explicit decode and write")
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:29:12 +01:00
Filipe Manana
77e0684029 btrfs-progs: receive: add debug messages when processing fallocate
Unlike for commands from the v1 stream, we have no debug messages logged
when processing fallocate commands, which makes it harder to debug issues.

So add log messages, when the log verbosity level is >= 3, for fallocate
commands, mentioning the value of all fields.

Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:29:12 +01:00
Filipe Manana
4c847b5133 btrfs-progs: receive: add debug messages when processing encoded writes
Unlike for commands from the v1 stream, we have no debug messages logged
when processing encoded write commands, which makes it harder to debug
issues.

So add log messages, when the log verbosity level is >= 3, for encoded
write commands, mentioning the value of all fields and also log when we
fallback from an encoded write to the decompress and write approach.

The log messages look like this:

  encoded_write f3 - offset=33792, len=4096, unencoded_offset=33792, unencoded_file_len=31744, unencoded_len=65536, compression=1, encryption=0
  encoded_write f3 - falling back to decompress and write due to errno 22 ("Invalid argument")

Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:29:12 +01:00
David Sterba
9b9216668d btrfs-progs: defrag: fix verbosity with default level
Defrag should not print the filenames by default, this got accidentally
changed in v6.0. Do a workaround that restores the original behaviour,
ie. no filenames and print them with -v, either as global or local
option. Proper fix is not to initialize with BTRFS_BCONF_UNSET and only
adjust the levels by -v/-q options.

Issue: #540
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:29:12 +01:00
Filipe Manana
dd7c458cb3 btrfs-progs: receive: work around failure of fileattr commands
Currently fileattr commands, introduced in the send stream v2, always
fail, since we have commented the FS_IOC_SETFLAGS ioctl() call and set
'ret' to -EOPNOTSUPP, which is then overwritten to -errno, which may
have a random value since it was not initialized before. This results
in a failure like this:

   ERROR: fileattr: set file attributes on p0/f1 failed: Invalid argument

The error reason may be something else, since errno is undefined at
this point.

Unfortunately we don't have a way yet to apply attributes, since the
attributes value we get from the kernel is what we store in flags field
of the inode item. This means that for example we can not just call
FS_IOC_SETFLAGS with the values we got, since they need to be converted
from BTRFS_INODE_* flags to FS_* flags

Besides that we'll have to reorder how we apply certain attributes like
FS_NOCOW_FL for example, which must happen always on an empty file and
right now we run write commands before attempting to change attributes,
as that's the order the kernel sends the operations.

So for now comment all the code, so that anyone using the v2 stream will
not have a receive failure but will get a behaviour like the v1 stream:
file attributes are ignored. This will have to be fixed later, but right
now people can't use a send stream v2 for the purpose of getting better
performance by avoid decompressing extents at the source and compression
of the data at the destination.

Link: https://lore.kernel.org/linux-btrfs/6cb11fa5-c60d-e65b-0295-301a694e66ad@inbox.ru/
Fixes: 8356c423e6 ("btrfs-progs: receive: implement FILEATTR command")
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:29:12 +01:00
David Sterba
4286eb552e Revert "btrfs-progs: resize: return error value from check_resize_args()"
This reverts commit 55438f3930.

The patch breaks resize cancel.

Reproducer:

    #!/bin/bash

    fallocate -l 7g /var/tmp/7g1 && fallocate -l 7g /var/tmp/7g2

    thing1=$(sudo losetup --show -f /var/tmp/7g1)
    thing2=$(sudo losetup --show -f /var/tmp/7g2)

    echo Make the fs
    mkfs.btrfs -L test539 $thing1
    mkdir test539
    mount -L test539 test539

    echo Get rid of devid:1 by adding a new device and removing the original
    btrfs dev add $thing2 test539
    btrfs dev del $thing1 test539
    echo Creating wiggleroom
    fallocate -l 3g test539/3g1 && fallocate -l 3g test539/3g2
    rm test539/3g1

    echo Start a resize operation and wait 3s to run a cancel
    echo Under 6.0 cancel, under 6.0.1 no cancel and runs out of space
    btrfs fi re 2:-4g test539 &
    sleep 3s && btrfs fi re cancel test539
    wait

    echo Cleanup
    umount test539

    losetup -d $thing1 && losetup -d $thing2

    rm /var/tmp/7g{1,2}
    rmdir test539

Issue: #539
Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-24 17:26:24 +01:00
David Sterba
90548b8295 btrfs-progs: filesystem: new subcommand mkswapfile
Add a command to create a new swapfile. The same can be achieved by
seandalone tools but they're just wrappers around the syscalls. The swap
format is simple enough to be created directly without mkswap command so
the swapfile can be created in one go.

The file must not exist before, this is to avoid problems with file
attributes or any other effects of existing extents. This also means the
command can't be used on block devices.

Default size is 2G, minimum size is 40KiB.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-11-08 11:30:21 +01:00
David Sterba
8bc13b39aa btrfs-progs: subvol: fix help text reference to subvolume
The commonly used reference to subvolume id is the one without a dash.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-26 10:15:40 +02:00
David Sterba
c24f14c6a4 btrfs-progs: qgroups: update help texts
Where missing, add the second line with more verbose description, fix
typos or formatting.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-26 10:15:40 +02:00
David Sterba
1414ecbb6f btrfs-progs: replace strerror(errno) with %m in printf formats
We're using the %m format where possible.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-26 09:46:22 +02:00
David Sterba
44efde7fa0 btrfs-progs: unify naming of qgroup subvolid helpers
Kernel function name is btrfs_qgroup_subvolid so rename it in progs. The
libbtrfs can't API be changed without versioning so at least add the new
helper.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-26 09:36:44 +02:00
David Sterba
701ab151c2 btrfs-progs: qgroup: new command to delete stale qgroups
A stale qgroup is level 0 and without a corresponding subvolume. There's
no convenient command for removing them and kernel does not remove them
automatically. Add a command so users don't have to parse and script the
output and/or delete them manually.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-26 09:21:35 +02:00
David Sterba
69b0d7756d btrfs-progs: qgroup show: adjust column widths and names
Use more human readable column description and adjust the width. Use a
single "-" for an empty value as is done elsewhere too.

Sample output:

Qgroupid    Referenced    Exclusive   Path
--------    ----------    ---------   ----
0/5           16.00KiB     16.00KiB   <toplevel>
0/256         16.00KiB     16.00KiB   subv1
0/257         16.00KiB     16.00KiB   <stale>
0/258         16.00KiB     16.00KiB   dir1/subv3
0/259         16.00KiB     16.00KiB   snap1
1/1           16.00KiB     16.00KiB   <0 member qgroups>

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 21:44:28 +02:00
David Sterba
f486f0f01e btrfs-progs: qgroup show: print pretty names of columns
There are two column name definitions, one for sorting and one for more
human readable format but it was not used for some reason.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 21:36:51 +02:00
David Sterba
274a53a4d2 btrfs-progs: qgroup show: adjust printed path format
Convert fputs and printf to message helpers that respect the verbosity
levels.

- print <stale> instead of <missing> for qgroups without a corresponding
  subvolume after it was deleted
- print <toplevel> for toplevel
- for higher level qgroups print the number of member groups, 0 if empty
  and not a special string
- drop the <FS_ROOT>
- print paths relative to toplevel path, like subvolume list does by
  default

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 21:34:28 +02:00
David Sterba
dac73d6e2c btrfs-progs: qgroup show: print subvolume path by default
Previous patch optionally printed the path but it would be better to
print it by default, so drop the option and verbosity. This is a
separate change as the original change was from an old pull request and
it was ported without significant changes first.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 21:12:29 +02:00
Jeff Mahoney
8d1c854094 btrfs-progs: qgroup: add path to show output
The 'btrfs qgroup show' command currently only prints qgroup IDs,
forcing the user to resolve which subvolume each corresponds to.

Adds subvolume path resolution to 'qgroup show' so that when
the -P option is used, the last column contains the pathname of
the root of the subvolume it describes.  In the case of nested
qgroups, it will show the number of member qgroups or the paths
of the members if the -v option is used.

Path can also be used as a sort parameter.

Sample output:

qgroupid         rfer         excl       path
--------         ----         ----       ----
0/5          16.00KiB     16.00KiB   <FS_ROOT>
0/256        16.00KiB     16.00KiB   <FS_ROOT>/subv1
0/257        16.00KiB     16.00KiB   <missing>
0/258        16.00KiB     16.00KiB   <FS_ROOT>/subv3
0/259        16.00KiB     16.00KiB   <FS_ROOT>/snap1

Pull-request: #139
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 21:12:24 +02:00
Jeff Mahoney
f80a6b40b9 btrfs-progs: quota: add -W option to rescan to wait without starting rescan
Adds a new options -W and --wait-norescan to wait for a rescan without
starting a new operation.  This is useful for things like fstests where
we want do to do a "btrfs quota enable" and not continue until the
subsequent rescan has finished.

In addition to documenting the new option in the man page, clean up the
rescan entry to document the -w option a bit better.

Pull-request: #139
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 21:12:24 +02:00
David Sterba
f6a212b8e6 btrfs-progs: unify naming of subvolume command definitions
Use full 'subvolume' in all cmd defintions to unify that with other
commands.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 20:45:00 +02:00
David Sterba
eb439bb833 btrfs-progs: quota rescan: add long options for status and wait
Add more descriptive long options to 'btrfs quota rescan'.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-25 11:54:58 +02:00
David Sterba
447f976472 btrfs-progs: subvol delete: update EPERM error message
The message could be confusing in case there's no send in progress and
the real reason is lack of permissions when deleting a subvolume.
Mention the permissions as first reason. Also update documentation.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-24 15:14:47 +02:00
Sidong Yang
55438f3930 btrfs-progs: resize: return error value from check_resize_args()
check_resize_args() function checks user argument amount but does not
return the correct value in case it's not valid.

Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-18 15:57:59 +02:00
Wang Yugui
f0c2bb0057 btrfs-progs: send: sync splice buf size with kernel when proto 2
When 'btrfs send --proto 2', the max buffer in kernel is changed from
BTRFS_SEND_BUF_SIZE_V1(SZ_64K) to (SZ_16K + BTRFS_MAX_COMPRESSED).

The performance is improved when we use the same buffer size in
btrfs-progs:

without this patch:  57.96s
with this patch:     48.44s

Bigger buffer size 512K was tested too, but it did not improve protocol
2 over 1 significantly.

Signed-off-by: Wang Yugui <wangyugui@e16-tech.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-18 15:43:38 +02:00
David Sterba
b2729396d2 btrfs-progs: cmds: use bool for status variables
Using bool for on/off variables is cleaner, convert all in cmds/ .

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:12 +02:00