Commit Graph

176 Commits

Author SHA1 Message Date
David Sterba 96aedd1121 btrfs-progs: fi usage: swap order of Used and zoned information
The Used and Free should be together, while all the device information
is in the first section.

Example:

  Overall:
      Device size:                 128.00GiB
      Device allocated:             24.00GiB
      Device unallocated:          104.00GiB
      Device missing:                  0.00B
      Device zone unusable:          5.13MiB
      Device zone size:            256.00MiB
      Used:                        213.33MiB
      Free (estimated):            111.79GiB      (min: 111.79GiB)
      Free (statfs, df):           111.79GiB
      Data ratio:                       1.00
      Metadata ratio:                   1.00
      Global reserve:               25.58MiB      (used: 16.00KiB)
      Multiple profiles:                  no

Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-02 17:27:53 +02:00
David Sterba a410805acc btrfs-progs: fi usage: print zone size in the overview
Read device size and print it in the overall overview in zoned mode. The
total unusable size is there so the zone size is complementing it. It's
read from the first device assuming that kernel mandates that all
devices have the same zone size.

Example:

  Overall:
      Device size:                 128.00GiB
      Device allocated:             24.00GiB
      Device unallocated:          104.00GiB
      Device missing:                  0.00B
      Used:                        213.33MiB
      Device zone unusable:          5.13MiB
      Device zone size:            256.00MiB
      Free (estimated):            111.79GiB      (min: 111.79GiB)
      Free (statfs, df):           111.79GiB
      Data ratio:                       1.00
      Metadata ratio:                   1.00
      Global reserve:               25.58MiB      (used: 16.00KiB)
      Multiple profiles:                  no

Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-02 17:27:53 +02:00
Sidong Yang 4693e82261 btrfs-progs: device usage: print number of stripes in relevant profiles
Print number of stripes for striped profiles in device usage commands.
It helps to see profiles easily. The output is like below.

/dev/vdc, ID: 1
   Device size:             1.00GiB
   Device slack:              0.00B
   Data,RAID0/2:          912.62MiB
   Data,RAID0/3:          912.62MiB
   Metadata,RAID1:        102.38MiB
   System,RAID1:            8.00MiB
   Unallocated:             1.00MiB

Multiple lines can appear in case a balance conversion process was
interrupted or if there's been a new device added and new data written
to the full stripe.

Issue: #372
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-23 13:47:35 +02:00
David Sterba d51075f05b btrfs-progs: restore: group help options
Group the options in the help text by the purpose for clarity.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
David Sterba 9a83a0b5c0 btrfs-progs: restore: remove loop check during directory scan
There's another loop protection during scan of directory items. This can
fire under invalid conditions, ie. when there's no real endless loop.
The layout of b-tree items could trigger that and has been observed in
practice. This prevents automated restoration as it requires user
attention.

The number of loops is 1024, unjustified and without explanation. Errors
during traversing the leaves are checked so most errors would be caught.
A real loop in the directory items would require some crafting and would
not happen on a normal filesystem.

Issue: #59
Issue: #164
Issue: #237
Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
David Sterba 4258b7161f btrfs-progs: restore: remove loop check during file copy
There's some kind of looping protection during copying file extents,
mostly likely to avoid endless loops on severely damaged filesystems.
This has been bothering users and makes restoring hard to automate as
it requires user attention to press 'y' or 'a'. This has not been well
documented either.

The number of loops is 1024 which looks arbitrary and hard to justify.
This eg. means that a file with many fragments hits the interactive
question more than once.

There are other checks when iterating the leaves that would catch
corruptions or other errors, so the looping would happen in some rare
and rather artificial case when some kind of loop exists inside the
extent items. This is not easily possible if possible at all as the
items do not directly reference other.

In case there's some genuine error found that would require a looping
protection, we'll add it or extend the checks to identify the loop.

Issue: #59
Issue: #164
Issue: #237
Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
David Sterba afe055f438 btrfs-progs: restore: convert to error message helpers
- convert to error() helpers, remove trailing \n
- switch messages printing %d as errno to %m
- rewording

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
David Sterba b1f374dd1d btrfs-progs: switch %Lu to %llu format
The %Lu format is not standard and we use %llu everywhere else, so
switch the remaining cases.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
David Sterba 9f6c055e38 btrfs-progs: dump-tree: add options to dump checksums
Add new options to dumps checksums in node headers and in the checksum
items:

  $ btrfs inspect dump-tree --csum-headers image
  root tree
  leaf 471515136 items 19 free space 12186 generation 15 owner ROOT_TREE
  leaf 471515136 flags 0x1(WRITTEN) backref revision 1 csum 0x756b2d54
  fs uuid df0348df-5773-47dd-81e9-a18221461239

For nodes/leaves it's appended on the 2nd line of the header.

Checksum items are stored in leaves as EXTENT_CSUM key type, with offset
value as the logical offset starting. As the array would be hard to
parse or match, each offset value is printed with the checksum. For
crc32c it's 4 values on a line, for xxhash it's 2 and for the long
256bit checksums it's one checksum per line.

  $ btrfs inspect dump-tree --csum-items image
  leaf 5423104 items 1 free space 30 generation 6 owner CSUM_TREE
  leaf 5423104 flags 0x1(WRITTEN) backref revision 1
  fs uuid bd7c981e-16ff-4081-a734-3ef5d50cafc1
  chunk uuid 13f4c76c-7845-4984-88ed-f01b52e05cf8
	  item 0 key (EXTENT_CSUM EXTENT_CSUM 22020096) itemoff 55 itemsize 16228
		  range start 22020096 end 38637568 length 16617472
		  [22020096] 0x8941f998 [22024192] 0x8941f998 [22028288] 0x8941f998 [22032384] 0x8941f998
		  [22036480] 0x8941f998 [22040576] 0x8941f998 [22044672] 0x8941f998 [22048768] 0x8941f998
		  ...

  $ btrfs inspect dump-tree --csum-items image
  leaf 5718016 items 1 free space 7746 generation 6 owner CSUM_TREE
  leaf 5718016 flags 0x1(WRITTEN) backref revision 1
  fs uuid f453a5b4-8b4a-4fbf-90a2-2925e4fe2335
  chunk uuid eb1da63b-248b-44c2-82da-71b2564bf50e
	  item 0 key (EXTENT_CSUM EXTENT_CSUM 52387840) itemoff 7771 itemsize 8512
		  range start 52387840 end 53477376 length 1089536
		  [52387840] 0x686ede9288c391e7e05026e56f2f91bfd879987a040ea98445dabc76f55b8e5f
		  [52391936] 0x686ede9288c391e7e05026e56f2f91bfd879987a040ea98445dabc76f55b8e5f
		  ...

The options are not on by default, the header checksum is not important
for the structures. Data checksums can be quite big so that would make
the dump long and without any actual data to match against.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
David Sterba 72d710637c btrfs-progs: print-tree: convert mode to bitmask
Replace follow and traverse by one parameter that takes bits to affect
the behaviour. This allows to extend btrfs_print_tree output with more
modes from one place.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-09 20:31:49 +02:00
David Sterba f123c28412 btrfs-progs: fi resize: add support for cancel
Recognize special resize amount 'cancel' for resize operation.  This
will request kernel to stop running any resize operation (most likely
shrinking resize). This needs support in kernel, otherwise this will
fail due to another exclusive operation running (though could be the
same one).

The command returns after kernel finishes any work that got interrupted,
but this should not take long in kernels 5.10+ that allow interruptible
relocation. The waiting inside kernel is interruptible so this command
(and the waiting stage) can be interrupted.

The resize operation could relocate block groups but the nominal
filesystem size will be restored when resize won't finish. It's
recommended to review the filesystem state.

Note: in kernels 5.10+ sending a fatal signal (TERM, KILL, Ctrl-C) to
the process running the resize will cancel it too.

Example:

  $ btrfs fi resize -10G /mnt
  ...
  $ btrfs fi resize cancel /mnt

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-20 22:43:29 +02:00
David Sterba 71d68ab8b2 btrfs-progs: device remove: add support for cancel
Recognize special name 'cancel' for device deletion, that will request
kernel to stop running device deletion. This needs support in kernel,
otherwise this will fail due to another exclusive operation running
(though could be the same one).

The command returns after kernel finishes any work that got interrupted,
but this should not take long in kernels 5.10+ that allow interruptible
relocation. The waiting inside kernel is interruptible so this command
(and the waiting stage) can be interrupted.

The device size is restored when deletion does not finish but it's
recommended to review the filesystem state.

Note: in kernels 5.10+ sending a fatal signal (TERM, KILL, Ctrl-C) to
the process running the device deletion will cancel it too.

Example:

    $ btrfs device delete /dev/sdx /mnt
    ...
    $ btrfs device delete cancel /mnt

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-20 22:14:18 +02:00
Anand Jain efe4844e01 btrfs-progs: fix inspect-internal --help incomplete sentence
btrfs inspect-internal --help shows incomplete sentence. As shown
below:

  btrfs inspect-internal --help
  <snip>
      btrfs inspect-internal min-dev-size [options] <path>
          Get the minimum size the device can be shrunk to. The
      btrfs inspect-internal dump-tree [options] <device> [<device> ..]
  <snip>

The short help string can be multi-line but must be in one string.  This
patch fixes it.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-17 14:26:32 +02:00
David Sterba 780355d225 btrfs-progs: fi usage: print zone unusable in the overview
Print the total zone_unusable size in the summary for 'fi usage' for a
filesystem in zoned mode. It's a sum of all the zone_unusable values
from 'fi df'. Per-device stats are not implemented and would need more
complicated calculations from raw data, kernel does not export that (but
it could).

As of 5.12, the zone_unusable is stored only in memory so we'd have to
map raw block device zones to the block groups and the live extents in
the associated block groups to get the exact numbers.

Example:

  # btrfs fi usage /mnt
  Overall:
      Device size:                   2.00GiB
      Device allocated:            768.00MiB
      Device unallocated:            1.25GiB
      Device missing:                  0.00B
      Device zone unusable:        320.00KiB
      Used:                        128.00KiB
      Free (estimated):              1.50GiB      (min: 1.50GiB)
      Free (statfs, df):             1.50GiB
      Data ratio:                       1.00
      Metadata ratio:                   1.00
      Global reserve:                3.25MiB      (used: 32.00KiB)
      Multiple profiles:                  no

  Data,single: Size:256.00MiB, Used:0.00B (0.00%)
     /dev/nullb0   256.00MiB

  Metadata,single: Size:256.00MiB, Used:112.00KiB (0.04%)
     /dev/nullb0   256.00MiB

  System,single: Size:256.00MiB, Used:16.00KiB (0.01%)
     /dev/nullb0   256.00MiB

  Unallocated:
     /dev/nullb0     1.25GiB

  # btrfs fi df
  Data, single: total=256.00MiB, used=0.00B, zone_unusable=0.00B
  System, single: total=256.00MiB, used=16.00KiB, zone_unusable=160.00KiB
  Metadata, single: total=256.00MiB, used=112.00KiB, zone_unusable=160.00KiB
  GlobalReserve, single: total=3.25MiB, used=32.00KiB

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba a2495d1e6e btrfs-progs: export get_zone_unusable and move to utils.c
Getting the per bg type zone unusable space will be used in other size
reports like 'fi us', so export it to the device utils.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba e295a8ad4c btrfs-progs: fi df: report zone_unusable on zoned filesystem
In the zoned mode there are parts of chunks that become unusable once
they get COWed and the zone must be reclaimed and reset to make the
space usable again. Provide a way to show the total size per block group
type in fi df:

  $ btrfs fi df .
  Data, single: total=1.00GiB, used=257.51MiB, zone_unusable=238.43MiB
  System, single: total=256.00MiB, used=16.00KiB, zone_unusable=224.00KiB
  Metadata, single: total=256.00MiB, used=816.00KiB, zone_unusable=8.61MiB
  GlobalReserve, single: total=3.25MiB, used=0.00B

This will not be shown on non-zoned filesystems.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-08 00:58:50 +02:00
David Sterba d591cd7c08 btrfs-progs: split unit related helpers from utils.c
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba 51f15d393a btrfs-progs: build: remove incomplete android support
There is a support to build on android but it's incomplete and there's
little interest to fix it.

To reinstate we'll need:

* fix remaining issues from
  lore.kernel.org/linux-btrfs/20170802185111.187922-1-filipbystricky@google.com

* find CI host with Android support to verify build, either local eg. in
  docker or in a hosted environment

* switch the make-based build to 'soong' (source.android.com/setup/build)

Issue: #357
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba 7fa07e2abb btrfs-progs: split open/close helpers from utils.c
There's a group of functions that are related to opening filesystem in
various modes, this can be moved to a separate file.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba b19a603d62 btrfs-progs: remove unnecessary linux/*.h includes
Decrease dependency on system headers, remove where they're not needed
or became stale after code moved. The path-utils.h encapsulate path
operations so include linux/limits.h here, that's where PATH_MAX is
defined.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba 51c0ece9f6 btrfs-progs: add prefix to get_partition_size
This is a public helper for devices, add the prefix to make it clear.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:46 +02:00
Naohiro Aota 86f9eb3906 btrfs-progs: zoned: introduce zoned support for device replace
This patch checks if the target file system is flagged as ZONED. If it is,
the device to be added is flagged PREP_DEVICE_ZONED.  Also add checks to
prevent mixing non-zoned devices and zoned devices.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:46 +02:00
Naohiro Aota 568f9ed26f btrfs-progs: device add: support adding zoned device
Check if the target file system is flagged as ZONED. If it is, the
device to be added is flagged PREP_DEVICE_ZONED.  Also add checks to
prevent mixing non-zoned devices and zoned devices.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:46 +02:00
Naohiro Aota 8ef9313cf2 btrfs-progs: zoned: implement log-structured superblock
Superblock (and its copies) is the only data structure in btrfs which has a
fixed location on a device. Since we cannot overwrite in a sequential write
required zone, we cannot place superblock in the zone.  One easy solution
is limiting superblock and copies to be placed only in conventional zones.

However, this method has two downsides: one is reduced number of superblock
copies. The location of the second copy of superblock is 256GB, which is in
a sequential write required zone on typical devices in the market today.
So, the number of superblock and copies is limited to be two.  Second
downside is that we cannot support devices which have no conventional zones
at all.

To solve these two problems, we employ superblock log writing. It uses two
adjacent zones as a circular buffer to write updated superblocks.  Once the
first zone is filled up, start writing into the second one.  Then, when
both zones are filled up and before starting to write to the first zone
again, reset the first zone.

We can determine the position of the latest superblock by reading write
pointer information from a device. One corner case is when both zones are
full. For this situation, we read out the last superblock of each zone, and
compare them to determine which zone is older.

The following zones are reserved as the circular buffer on ZONED btrfs.

- primary superblock: offset   0B (and the following zone)
- first copy:         offset 512G (and the following zone)
- Second copy:        offset   4T (4096G, and the following zone)

If these reserved zones are conventional, superblock is written fixed at
the start of the zone without logging.

Currently, superblock reading/writing is done by pread/pwrite. This
commit replace the call sites with sbread/sbwrite to wrap the functions.
For zoned btrfs, btrfs_sb_io which is called from sbread/sbwrite
reverses the IO position back to a mirror number, maps the mirror number
into the superblock logging position, and do the IO.

Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:45 +02:00
Naohiro Aota acdd22ab68 btrfs-progs: provide fs_info from btrfs_device
Likewise in the kernel code, provide fs_info access from struct
btrfs_device. This will help to unify the code between the kernel and
the userland.

Since fs_info can be NULL at the time of btrfs_add_to_fsid(), let's use
btrfs_open_devices() to set fs_info to the devices.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:45 +02:00
Su Yue 31934b1161 btrfs-progs: fi resize: fix false 0.00B size output
Resize to nums without sign prefix makes false output:

  $ btrfs fi resize 1:150g /srv/extra
  Resize device id 1 (/dev/sdb1) from 298.09GiB to 0.00B

The resize operation would take effect though.

Fix it by handling the case if mod is 0 in check_resize_args().

Issue: #307
Reported-by: Chris Murphy <lists@colorremedies.com>
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Su Yue <l@damenly.su>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:44 +02:00
Johannes Thumshirn 94b60b67a9 btrfs-progs: pass in fs_info to btrfs_csum_data
For passing authentication keys to the checksumming functions we need a
container for the key.

Pass in a btrfs_fs_info to btrfs_csum_data() so we can use the fs_info
as a container for the authentication key.

Note this is not always possible for all callers of btrfs_csum_data() so
we're just passing in NULL for now

Functions calling btrfs_csum_data() with a NULL fs_info argument are
currently not supported in the context of an authenticated file system.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
David Sterba 6bb6d1215d btrfs-progs: factor open_ctree parameters to a structure
Extending open_ctree with more parameters would be difficult, we'll need
to add more so factor out the parameters to a structure for easier
extension.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
Sidong Yang 78aa1d95dd btrfs-progs: fi resize: make output more readable
Make output of 'btrfs filesystem resize' command more readable and
describe the changes in more detail.

Before:
Resize '/mnt' of '1:-1G'

After:
Resize device id 1 (/dev/vdb) from 4.00GiB to 3.00GiB

Issue: #307
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:22 +01:00
David Sterba 3a22f0b25b btrfs-progs: balance: fix raid56 warning for other profiles
The warning is printed for profiles where it's not intended (like raid0
or raid1c4). Check the correct variable for the target profiles.

Issue: #355
Fixes: 1ed5db8db4 ("btrfs-progs: balance convert: add a warning and countdown for RAID56 conversion")
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:22 +01:00
David Sterba 2910ca8009 btrfs-progs: balance: use --force to override timeout for raid56 conversions
Enhance --force to also skip the timeout, similar to what --full-balance
does. As this is only to warn about RAID56 that won't be necessary in
the future, don't add a separate option. The warning is still printed.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-04 21:14:41 +01:00
Josef Bacik 1ed5db8db4 btrfs-progs: balance convert: add a warning and countdown for RAID56 conversion
Similar to the mkfs warning, add a warning to btrfs balance convert
options, with a countdown to allow the user to have time to cancel the
operation.

Issue: #265
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-04 21:14:41 +01:00
Anand Jain ddf878187e btrfs-progs: fix return code for failed replace start
When replace starts with no-background and fails for the reason that
a BTRFS_FS_EXCL_OP is in progress, we still return the value 0 and also
leak the target device open, because in cmd_replace_start() we missed
the goto leave_with_error for this error.

So the test case btrfs/064 in its seqres.full output reports...

  Replacing /dev/sdf with /dev/sdc
  ERROR: /dev/sdc is mounted

instead of...

  Replacing /dev/sdc with /dev/sdf
  ERROR: ioctl(DEV_REPLACE_START) '/mnt/scratch': add/delete/balance/replace/resize operation in progress

for the failed replace attempts in the test case

Fix it by jumping to the error label which also fixes the leaked open
device.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 19:58:01 +01:00
Daniel Xu c11874ae81 btrfs-progs: rescue: add create-control-device subcommand
Add a new subcommand 'btrfs rescue create-control-device' that creates
/dev/btrfs-control. This is helpful on systems that may not have `mknod`
installed and the device node is missing for some reason.

Issue: #223
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
[ update docs ]
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 16:19:37 +01:00
Nikolay Borisov 3e653e801c btrfs-progs: remove duplicate checks from cmd_filesystem_resize
btrfs_open_dir already has a check whether the passed path is a
directory and if so it returns a specific error code (-3) when such an
error occurs. Use this instead of open-coding the directory check. To
avoid regression in cli/003 test also move directory checks before fs
type in btrfs_open.

Output before this check:

  ERROR: resize works on mounted filesystems and accepts only
  directories as argument. Passing file containing a btrfs image
  would resize the underlying filesystem instead of the image.

After:

  ERROR: not a directory: /root/btrfs-progs/tests/test.img
  ERROR: resize works on mounted filesystems and accepts only
  directories as argument. Passing file containing a btrfs image
  would resize the underlying filesystem instead of the image.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 15:24:42 +01:00
Filipe Manana 81d8ea9346 btrfs-progs: receive: remove workaround for setting capabilities
We had a few bugs on the kernel side of send/receive where capabilities
ended up being lost after receiving a send stream. They all stem from the
fact that the kernel used to send all xattrs before issuing the chown
command, and the later clears any existing capabilities in a file or
directory.

Initially a workaround was added to btrfs-progs' receive command, in commit
123a2a0850 ("btrfs-progs: receive: restore capabilities after chown"),
and that fixed some instances of the problem. More recently, other instances
of the problem were found, a proper fix for the kernel was made, which fixes
the root problem by making send always emit the setxattr command for setting
capabilities after issuing a chown command. This was done in kernel commit
89efda52e6b693 ("btrfs: send: emit file capabilities after chown"), which
landed in kernel 5.8.

However, the workaround on the receive command now causes us to incorrectly
set a capability on a file that should not have it, because it assumes all
setxattr commands for a file always comes before a chown.

Example reproducer:

  $ cat send-caps.sh
  #!/bin/bash

  DEV1=/dev/sdh
  DEV2=/dev/sdi

  MNT1=/mnt/sdh
  MNT2=/mnt/sdi

  mkfs.btrfs -f $DEV1 > /dev/null
  mkfs.btrfs -f $DEV2 > /dev/null

  mount $DEV1 $MNT1
  mount $DEV2 $MNT2

  touch $MNT1/foo
  touch $MNT1/bar
  setcap cap_net_raw=p $MNT1/foo

  btrfs subvolume snapshot -r $MNT1 $MNT1/snap1

  btrfs send $MNT1/snap1 | btrfs receive $MNT2

  echo
  echo "capabilities on destination filesystem:"
  echo
  getcap $MNT2/snap1/foo
  getcap $MNT2/snap1/bar

  umount $MNT1
  umount $MNT2

When running the test script, we can see that both files foo and bar get
the capability set, when only file foo should have it:

  $ ./send-caps.sh
  Create a readonly snapshot of '/mnt/sdh' in '/mnt/sdh/snap1'
  At subvol /mnt/sdh/snap1
  At subvol snap1

  capabilities on destination filesystem:

  /mnt/sdi/snap1/foo cap_net_raw=p
  /mnt/sdi/snap1/bar cap_net_raw=p

Since the kernel fix was backported to all currently supported stable
releases (5.10.x, 5.4.x, 4.19.x, 4.14.x, 4.9.x and 4.4.x), remove the
workaround from receive. Having such a workaround relying on the order
of commands in a send stream is always troublesome and doomed to break
one day.

A test case for fstests will come soon.

Issue: #85
Issue: #202
Issue: #292
Reported-by: Richard Brown <rbrown@suse.de>
Reviewed-by: Su Yue <l@damenly.su>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 15:18:39 +01:00
David Sterba 922eaa7b54 btrfs-progs: build: fix linking with static libmount
The libmount dependency has been added in commit 61ecaff036
("btrfs-progs: build: add libmount dependency"), and static build got
broken. There are functions that do basically the same thing and also
share the name, which in turn fails at link time.

  ld: /../lib64/libmount.a(libcommon_la-canonicalize.o): in function `canonicalize_dm_name':
  util-linux-2.34/lib/canonicalize.c:58: multiple definition of `canonicalize_dm_name';
	  common/path-utils.static.o:btrfs-progs/common/path-utils.c:286: first defined here

In case the collision can be resolved by renaming, it's done
(canonicalize_path and parse_size). There are 2 symbols from selinux
that are substituted by a weak aliases during the static build.

There's one new warning due to use of getgrnam_r in libmount that
depends on dynamic linking and may not work properly with static build.
We're not using the related functions directly or indirectly, so it
should be safe to ignore the warnings.

  ld: ../lib64/libmount.a(la-utils.o): in function `mnt_get_gid':
  util-linux-2.34/libmount/src/utils.c:625: warning: Using 'getgrnam_r' in statically linked applications
  +requires at runtime the shared libraries from the glibc version used for linking

Issue: #333
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-25 23:31:56 +01:00
David Sterba 3c49295e8a btrfs-progs: subvol set-default: change id to 5 if specified as 0
The id 0 of the default subvolume is an internal alias for the toplevel
fs tree, kernel does that conversion. Until 2116398b1d ("btrfs-progs:
use libbtrfsutil for set-default") there was no manual conversion and
the value was passed to kernel as-is. With the switch to the
libbtrfsutil API this got broken (4.19).

  $ btrfs subvol set-default 0 /path

In this case the default subvolume would be containing subvolume of
/path instead of the toplevel one.

Fix it by manually switching the 0 to 5 in case user specifies that to
avoid the difference in the API, that we can't change.

Issue: #327
Reported-by: Chris Murphy
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-18 17:49:23 +01:00
Marcos Paulo de Souza 6b8fed9e79 btrfs-progs: inspect: use find_mount_fsroot in logical-resolve
By using find_mount_fsroot we ensure that we return a valid path to the
final user, by ensuring that even if we return a bind mount, the
pathname of btrfs used was the same from the original mount.

This for a case when bind mounts and normal mount -o subvol=/path are
mixed.

Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-18 17:49:22 +01:00
Adam Borowski c615287cc0 btrfs-progs: a bunch of typo fixes
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:10 +01:00
Omar Sandoval 8b7d1fc918 btrfs-progs: send: fix crash on unknown option
The long options array for send is missing the zero terminator, so
unknown options result in a crash:

  # btrfs send --foo
  Segmentation fault (core dumped)

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:09 +01:00
Sidong Yang 103d365a8f btrfs-progs: device stats: add json output format
Add support for json formatting. Switch hard coded printing code to
formatted print with output formatter. Json output would be useful for
other programs that parse output of the command.

The plain text format is not changed for backward compatibility but this
requires to do another switch by the output type.

Example text format:

device:                 /dev/vdb
devid			1
write_io_errs:          0
read_io_errs:           0
flush_io_errs:          0
corruption_errs:        0
generation_errs:        0

Example json format:

{
  "__header": {
    "version": "1"
  },
  "device-stats": [
    {
      "device": "/dev/vdb",
      "devid": "1",
      "write_io_errs": "0",
      "read_io_errs": "0",
      "flush_io_errs": "0",
      "corruption_errs": "0",
      "generation_errs": "0"
    }
  ]
}

Issue: #291
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:53 +01:00
David Sterba 5f7191135f btrfs-progs: fix short/long unit size options
The two variants of unit options are not suitable for all commands, the
short options could interfere with existing options or limit future
extensions.

In 'filesystem du' the short options are not documented neither in help
text, nor in documentation so fix the code

In 'scrub status' it's the same but the documentation needs to be fixed
as well.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:52 +01:00
David Sterba 022168c32d btrfs-progs: subvol show: fix required arguments in help texts
The help text and documentation of the --rootid and --uuid parameters
is wrong as it does not say there's a required parameter. Add it and
enhance the docs to clarify what the options do.

Issue: #317
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:52 +01:00
Su Yue b431ee33e8 btrfs-progs: subvol show: reset subvol_path to NULL after free
User reported that 'btrfs subvolume show -u -- /mnt' causes double free.

Pointer subvol_path was freed in iterations but still keeps the old
value.  In the last iteration, error BTRFS_UTIL_ERROR_STOP_ITERATION
returned, then the double free of subvol_path happens in the out goto
label.

Set subvol_path to NULL after each free() in the loop to fix the issue.

Issue: #317
Signed-off-by: Su Yue <l@damenly.su>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:52 +01:00
David Sterba e198c6674a btrfs-progs: add enqueue parameter for exclusive ops
The exclusive ops will not start if there's one already running. Now
that we have the sysfs export (since kernel 5.10) to check if there's
one already running, use it to allow enqueueing of the operations as a
convenience.

Supported enqueuing:

  btrfs balance start --enqueue
  btrfs filesystem resize --enqueue
  btrfs device add --enqueue
  btrfs device delete --enqueue
  btrfs replace start --enqueue

This patch implements the functionality based on Goldwyn's patch
https://lore.kernel.org/linux-btrfs/?q=20200825150338.32610-4-rgoldwyn%40suse.de
but on top of previous preparatory patches.

Note that 'filesystem resize' options could confuse getopt as the
negative size change looks like a series of short options and there's no
way to make getopt ignore the short options, so there's a custom option
parser.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-16 17:08:12 +01:00
Sidong Yang 0c77d2cd03 btrfs-progs: fi usage: add avail info from statfs()
Add available space information from statfs(). This can be different from
'Free (estimated)' in some cases. This patch provide more information
about filesystem usage like below.

Overall:
    Device size:                   5.00GiB
    Device allocated:              1.02GiB
    Device unallocated:            3.98GiB
    Device missing:                  0.00B
    Used:                         88.00KiB
    Free (estimated):              4.48GiB      (min: 2.49GiB)
    Free (statfs, df)              4.48GiB
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              832.00KiB      (used: 0.00B)
    Multiple profiles:                  no

Issue: #306
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:30 +01:00
Goldwyn Rodrigues 80df942fe7 btrfs-progs: check for exclusive operation before issuing another
If the exclusive operation is available in sysfs file, check if there's
one already running. The check is done for:

- device add, remove, replace
- balance
- filesystem resize

All commands will validate arguments and check before the ioctl or
before any potentially irreversible operations (like clearing device
before replacing).

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:29 +01:00
David Sterba d60d48fce5 btrfs-progs: scrub status: add unit mode options
Add long options for size units, affecting total and currently scrubbed
bytes. The rate depends on the device speed and could be
disproportionate to the size so it is not affected, except the --raw
option that is in bytes without unit suffix.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 22:44:36 +01:00
David Sterba d9e9198485 btrfs-progs: scrub status: print percents of scrubbed bytes
Add ratio of the bytes scrubbed to total in the status output, like:

  Total to scrub:   2.54TiB
  Bytes scrubbed:   1.59TiB  (62.58%)

Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 20:28:31 +01:00