Commit Graph

6086 Commits

Author SHA1 Message Date
Thomas Hebb 2f43f2dc7b btrfs-progs: convert: move simple_range into common.h
This is currently defined in source-fs.h, but main.c uses it far more
than source-fs.c does. Put it in common.h instead, since it's a useful
standalone type.

Author: Thomas Hebb <tommyhebb@gmail.com>
Pull-request: #494
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
David Sterba 163aa3cc62 btrfs-progs: tests: add mkfs/025 to test parallel zone reset
Create a few emulated zoned devices and run mkfs, the zone reset is
expected to be run in parallel. It's using memory-backed devices so it's
too fast to measure the differences and we can't expect availability of
slow zoned devices so this test is very simplistic.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
David Sterba 6a968e0f55 btrfs-progs: tags: add nullb for testing zoned devices
I've written a simple shell wrapper for null_blk configuration
(https://github.com/kdave/nullb). Make a local copy of version 0.1 to
avoid external dependency for our tests.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
Li Zhang bb2eed3aa5 btrfs-progs: mkfs: run device preparation in parallel
When devices are formatted as btrfs, btrfs_prepare_device is called
sequentially for each device, which takes too much time.

Put each btrfs_prepare_device into a thread, wait for the first thread
to complete to mkfs.btrfs, and wait for other threads to complete before
adding other devices to the file system.

During the preparation it's either trim/discard or zone reset.

This was tested with TCMU emulation with two zoned devices.  Each device
is 2000G (about 19.53 TiB), the region size is 4MB, Use the following
parameters for targetcli:

  create name=zbc0 size=20000G cfgstring=model-HM/zsize-4/conv-100@~/zbc0.raw

Call difftime to calculate the running time of the function
btrfs_prepare_device.  Calculate the time from thread creation to
completion of all threads after patching:

  $ lsscsi -p
  [10:0:1:0]   (0x14)  LIO-ORG  TCMU ZBC device  0002  /dev/sdb   -          none
  [11:0:1:0]   (0x14)  LIO-ORG  TCMU ZBC device  0002  /dev/sdc   -          none

  $ sudo mkfs.btrfs -d single -m single -O zoned /dev/sdc /dev/sdb -f
  ....
  time for prepare devices:4.000000.
  ....

  $ sudo mkfs.btrfs -d single -m single -O zoned /dev/sdc /dev/sdb -f
  ...
  time for prepare devices:2.000000.
  ...

Issue: #496
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Li Zhang <zhanglikernel@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
David Sterba 9cb5b0d058 btrfs-progs: tests: in mkfs/002 use grep -E instead of egrep
The egrep command is deprecated (per manual page of grep) for a long
time and will probably be removed, the replacement is 'grep -E'.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
Boris Burkov 980ba4e842 btrfs-progs: receive: add support for fs-verity
Process an enable_verity cmd by running the enable verity ioctl on the
file. Since enabling verity denies write access to the file, it is
important that we don't have any open write file descriptors.

This also revs the send stream format to version 3 with no format
changes besides the new commands and attributes. This version is not
finalized and commands may change, also this needs to be synchronized
with any kernel changes.

Note: the build is conditional on the header linux/fsverity.h

Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
Qu Wenruo 0bfd7004d3 btrfs-progs: hide block group tree behind experimental feature
The block group tree doesn't yet have full bi-directional conversion
support from btrfstune, and it seems we may want one or two release
cycles to rule out some extra bugs before really releasing the progs
support.

This patch will hide the block group tree feature behind experimental
flag for the following tools:

- btrfstune
  "-b" option to convert to bg tree.

- mkfs.btrfs
  hide "block-group-tree" feature from both -O (the new default position
  for all features) and -R (the old, soon to be deprecated one).

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:08 +02:00
David Sterba 958e2fc442 btrfs-progs: docs: note about read-only mount for send
Clarify that read-only mount is not sufficient.

Issue: #493
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
Guillaume Legrand f68139dbb5 btrfs-progs: README: update links to manual pages
The online manual pages of the btrfs utilities seem to have been moved to
`readthedocs.io`; update references in the README accordingly.

Author: Guillaume Legrand
Pull-request: #500
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
Torstein Eide 3f8817de7f btrfs-progs: docs: add a section about troubleshooting swapfile
swapon fails with an unclear error message, add some hints were to look
for more information.

Author: Torstein Eide
Pull-request: #491
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
Solt Budavári fe7aabafdf btrfs-progs: docs: fix typo in subvolume intro
Author: solt87
Pull-request: #513
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
AtticFinder65536 f70ca45961 btrfs-progs: docs: update reflink cross-mount constraint
Mention the version support for the cross-mount support, since 5.18.

Author: AtticFinder65536
Pull-request: #480
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba feef6aaaf6 btrfs-progs: kernel-lib: remove radix-tree
The radix-tree is not used in userspace code. In kernel it's for
tracking unpersisted and in-memory structures and has been replaced by
the xarray.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 7b7492d075 btrfs-progs: remove random-test.c
The random-test exercises the b-tree operations but hasn't been in use
for a long time and we won't probably resurrect it. Also it's the only
user of the radix_tree structures, that are otherwise used in the kernel
code, it needs the kerne-lib radix-tree implementation. Let's remove it
as it's basically dead code.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 94a04f8c9d btrfs-progs: btrfstune: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 3f65090ba9 btrfs-progs: select-super: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba b959a9a16b btrfs-progs: map-logical: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 972156289e btrfs-progs: corrupt-block: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba b610fc1612 btrfs-progs: convert: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba b19ab3ba9f btrfs-progs: mkfs: use message helpers for error messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 70a2f5520c libbtrfsutil: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes. Update the includes of implementation files only.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 4f23f0e711 btrfs-progs: use warning helper for multiple profile messages
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 0e38e1c4f2 btrfs-progs: use error helper for messages in non-kernel code
Lots of code still uses fprintf(stderr, "...") that should be the
error() helper. The kernel-shared code is left out of the conversion for
now.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba b5aac254c7 btrfs-progs: common: update include lists, part 1
The tool IWYU (include what you use) suggests to remove and add some
includes. This is only partial to avoid accidental build breakage, the
includes are entangled and will have to be cleaned in the future again.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:08:07 +02:00
David Sterba 40c4ba74ec btrfs-progs: cmds: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:07:59 +02:00
David Sterba dda894259f btrfs-progs: libbtrfs: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba d04a1fa10e btrfs-progs: factor filesystem helpers out of utils.c
Group helpers that retrieve information from the filesystem out of
utils.c.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba eac3cb577c btrfs-progs: move parse_qgroupid_or_path to parse-utils
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba 6edd4b2121 btrfs-progs: factor string helpers out of utils.c
Utils is the catch-all file, we can now separate some string utility
functions.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba 466e025c74 btrfs-progs: ci: fix image updater script
There's a typo in the way how parameter is passed and is a syntax error
for docker.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba 8d8f8c7f73 btrfs-progs: prepare merging compat feature lists
The features are split to -O and -R but it does not make much sense from
user POV, there are different levels of compatibility but it does not
need to be selected that way. Merge the tables into one but hide it
behind experimental build until the conversion is complete.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba 20496884a9 btrfs-progs: tests: detect send stream version in misc/053
The encoded stream is supported since protocol version 2, detect
support.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba 7d06a7e561 btrfs-progs: tests: use _mktemp_local for temporary files
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:13 +02:00
David Sterba caa57c1689 btrfs-progs: tests: use _mktemp for creating files
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 500f826f31 btrfs-progs: tests: add helper for creating local temporary file
Some tests don't use the /tmp temporary files and store it locally in
the test directory. To support NFS this needs to be created by a few
commands. To avoid accidental breakage add a convenience helper.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba bd94a00de0 btrfs-progs: check: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 0f03da53cc btrfs-progs: mkfs: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba ec55de0ac5 btrfs-progs: convert: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 6d9b3835a6 btrfs-progs: image: update include lists
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 288f8cb78f btrfs-progs: update include list in standalone tools
The tool IWYU (include what you use) suggests to remove and add some
includes.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 384bc17f63 btrfs-progs: reorder includes in standalone tools
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba b121ac4404 btrfs-progs: fix path to internal libbtrfsutil includes
All files include the <btrfsutil.h> which could be confused with the
system-wide installation. Drop the -I path from build and use full path
for any libbtrfsutil headers.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 3df6617b9b btrfs-progs: cmds: reorder includes
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 7ab0c46a81 btrfs-progs: fi usage: fix unallocated and print total and slack
The size reported as Unallocated in the table was different that the one
in the listing, calculated differently. The values should reflect the
unallocated area available for the filesystem - not necessarily the
total size of the device. If there's such slack space it's reported
separately.

The values in the table mean:

- Unallocated: block device size - slack - allocated
- Total:       block device size - slack
- Slack:       block device size - filesystem

The new columns make the table wider but the values are deemed to be
important by users and for filesystems with normal profiles it fits
under reasonable line width. During balance or with multiple profiles it
can get wider but this should not be a serious problem.

Example output:

Overall:
    Device size:                  13.00GiB
    Device allocated:            536.00MiB
    Device unallocated:           12.48GiB
    Device missing:                  0.00B
    Device slack:                  1.00GiB
    Used:                          2.31MiB
    Free (estimated):             12.48GiB      (min: 6.24GiB)
    Free (statfs, df):            12.48GiB
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:                3.50MiB      (used: 0.00B)
    Multiple profiles:                  no

              Data    Metadata  System
Id Path       single  DUP       DUP      Unallocated Total    Slack
-- ---------- ------- --------- -------- ----------- -------- -------
 1 /dev/loop0 8.00MiB 512.00MiB 16.00MiB     2.48GiB  3.00GiB 1.00GiB
 2 /dev/loop1       -         -        -    10.00GiB 10.00GiB       -
-- ---------- ------- --------- -------- ----------- -------- -------
   Total      8.00MiB 256.00MiB  8.00MiB    12.48GiB 13.00GiB 1.00GiB
   Used       2.00MiB 144.00KiB 16.00KiB

Issue: #508
Pull-request: #509 (partial fix)
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 1c3ab87c92 btrfs-progs: tests: check that receive --dump escapes target paths
Verify that receive --dump escapes paths for rename, symlink and
hardlink when it's the "dest=" value.

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 2f2473248f btrfs-progs: receive: escape target paths for rename, symlink and hardlink
The stream dump escapes the path on which the operation is done but
there are a some that use another path that's the target. A file with
eg. a newline then does not format properly on one line as expected.

Extend the printing helpers to skip printing the newline and then print
the escaped path manually.

Issue: #510
Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 272bff8978 btrfs-progs: common: reorder includes
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba 8802a81eb8 btrfs-progs: check: reorder includes
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:12 +02:00
David Sterba fe7a78f5df btrfs-progs: image: reorder includes
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00
David Sterba be6043ecd2 btrfs-progs: convert: reorder includes
The preferred order:
- system headers
- standard headers
- libraries
- kernel library
- kernel shared
- common headers
- other tools
- own headers

Signed-off-by: David Sterba <dsterba@suse.com>
2022-10-11 09:06:11 +02:00