Commit Graph

6917 Commits

Author SHA1 Message Date
David Sterba
f9ebee6838 btrfs-progs: ci: install RTD sphinx theme for devel workflow
With the recent updates to documentation build the theme must be now
installed as a package. Disable building documentation in all workflows
that do functional tests.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-06 16:03:37 +01:00
David Sterba
c1ce48ed3f btrfs-progs: docs: document device add and remove ioctls
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 20:17:31 +01:00
David Sterba
216c9f0ffb btrfs-progs: docs: move doc conventions to developer docs
The DocConventions are now fairly complete and can be moved to the
proper section.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 19:33:28 +01:00
Qu Wenruo
eae4109054 btrfs-progs: check: remove inode cache clearing functionality
Since we're already directing the end user to use "btrfs rescue
clear-ino-cache" command, there is not much need to support it in
btrfs-check.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 17:57:04 +01:00
Qu Wenruo
171dd56995 btrfs-progs: docs: update the man page for btrfs check lowmem mode
Lowmem mode has improved quite a lot since its introduction, for
read-only check it's definitely fine.

For repair mode, both lowmem and original mode are considered dangerous
especially for complex corruptions with unknown cause.

For now lowmem mode is only bad at fixing fundamentally corrupted cases,
like bad shift offsets or transid, which in real world it's not an easy
repair for the original mode either.

This patch would move the --mode option out of the dangerous section and
update the notes for the lowmem mode on its limitation.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 17:37:25 +01:00
David Sterba
5c91264d2d btrfs-progs: subvol delete: print the id of the deleted subvolume
Currently the path of deleted subvolume is printed, we should also print
the numeric id as it's another identifier commonly found and can be used
for a cross reference. In connection with the qgroup deletion it's
making the output clear:

...
Delete subvolume 258 (no-commit): '/mnt/subv1'
Delete qgroup 0/258
...

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 17:08:42 +01:00
David Sterba
8b9684f965 btrfs-progs: docs: add config file for readthedocs.io
There's another config required for building the RTD documentation,
https://docs.readthedocs.io/en/stable/config-file/ .

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 16:36:24 +01:00
David Sterba
cc2e3ab03d btrfs-progs: docs: update and restyle links of source repositories
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 16:19:09 +01:00
David Sterba
71042e8bb0 btrfs-progs: tests: add test for subvolume delete and qgroups
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 12:36:44 +01:00
David Sterba
9da773aa46 btrfs-progs: subvol delete: add options to delete the qgroup
The 0/subvolid qgroups are not automatically deleted when the subvolume
is deleted, for historical reasons. There's a command to clean up all
such stale qgroups (btrfs qgroup clean-stale) but this should be also
possible with the subvolume deletion.

With the options we can switch the default to delete the qgroup by
default eventually, if somebody depends on the not deleting behaviour
the negation option can be used.

Issue: #366
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 12:36:44 +01:00
David Sterba
a396f49cb8 btrfs-progs: build: conditionally detect x86_64 compiler flags
Compiling with clang on aarch64 leads to an error when detecting the
SIMD instruction support. Gcc ignores the arch/feature mismatch.
Conditionally detect the -m flags only on x86_64.

Issue: #712
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-05 12:36:44 +01:00
David Sterba
bc70e12877 btrfs-progs: ci: add clang to devel build tests
Add clang as compiler for the basic build checks in the CI.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-04 18:55:47 +01:00
David Sterba
4ac630b5dc btrfs-progs: ci: update packages providing pahole
Centos 8 does not provide the package at all, on musl it's in the
package 'pahole'.

Issue: #710
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-04 18:47:31 +01:00
David Sterba
8ea9a3e2c6 btrfs-progs: print a message when enqueued operation is waiting
The enqueue option should let the user know that the expected operation
hasn't started yet and that it's waiting for another one. Although the
exclusive operations can take long, the two reason should be
distinguished.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 03:22:29 +01:00
David Sterba
c0592c5591 btrfs-progs: tests: warning and timeout when removing multiple devices
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 03:04:58 +01:00
David Sterba
9e55abfd77 btrfs-progs: device delete: add timeout when removing multiple devices
Reported on IRC, that it's unexpected that passing several devices on
command line for 'btrfs device delete' still uses some of the devices
during deletion. The expectation was that they'd be removed at once (and
thus not used for the intermediate chunk relocation).

As it works now, the ioctl removes only one device. As a workaround, add
a timeout (like we have for the full balance and others) when there are
more devices passed on the command line. This can be skipped by the
--force parameter.

Issue: #708
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 02:54:09 +01:00
David Sterba
a0468efe86 btrfs-progs: reset errno before strtoull()
strtoull may return the boundary values, if the callers could expect
that and verify it then the errno must be reset before the call.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 02:13:37 +01:00
David Sterba
e9a66cde91 btrfs-progs: use get_sysfs_proto_supported() in device_get_zone_unusable()
Use the sysfs helper to read u64 values of the zoned stats.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 02:08:37 +01:00
David Sterba
9da7e92af9 btrfs-progs: use get_sysfs_proto_supported() in get_sysfs_proto_supported()
Use the sysfs helper to read u64 value of send stream.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:58:56 +01:00
David Sterba
22495fc8fd btrfs-progs: tests: add coverage for size options of 'fi du'
Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:46:05 +01:00
David Sterba
b512558c02 btrfs-progs: build: add target for all internal APIs
There are several APIs that have tests, add a single build target for
convenience and enable that in the CI.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:39:27 +01:00
David Sterba
960e7b3800 btrfs-progs: tests: test log levels
The log levels have been added in 6.2, add the basic coverage.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:32:40 +01:00
David Sterba
41781638f9 btrfs-progs: README: document static builds
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:22:06 +01:00
David Sterba
0c155146c3 btrfs-progs: README: update links
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:08:08 +01:00
David Sterba
ac8edc1513 btrfs-progs: docs: restyle the landing page
Add an introductory paragraph with most important links, put TOC
listings to columns.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 00:54:27 +01:00
David Sterba
32880fa518 btrfs-progs: crypto: add openssl as crypto provider
https://www.openssl.org/ Is a well known cryptography library and since
freshly released version 3.2 it also supports variable digest size of
blake2b, so we can now add it among the crypto providers.

Configure with --with-crypto=openssl.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-30 22:04:45 +01:00
David Sterba
5221aedc00 btrfs-progs: crypto: add Botan as crypto provider
https://botan.randombit.net/ Botan is a cryptography library with C
bindings and provides what we need (sha256 and blake2b), among many
others. Add it to the list of crypto backends if somebody wants to use
it.

Currently the version 2.19 is the latest one. Botan3 3.2.0 exists but
does not seem to be widely available in distros yet.

Configure with --with-crypto=botan.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-30 22:04:45 +01:00
Qu Wenruo
d79eb4b726 btrfs-progs: tests: add test case for subvolume create multiple arguments
The test case would verify the following behaviors:

- Partial failure
  Should return 1, but the remaining valid destinations would still be
  created.

- All success
  That's as usual.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:21:58 +01:00
Qu Wenruo
5aa959fb34 btrfs-progs: subvolume create: accept multiple arguments
This patch would make "btrfs subvolume create" to accept multiple
arguments, just like "mkdir".

The existing options like "-i <qgroupid>" and "-p" would all be applied
to all subvolume(s).

If one destination failed, the command would return 1, while still retry
the remaining destinations.

Issue: #695
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:15:47 +01:00
Qu Wenruo
943a69ef2d btrfs-progs: subvolume create: handle failure for strdup()
The function strdup() can return NULL if the system is out of memory,
thus we need to hanle the rare but possible -ENOMEM case.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:11:17 +01:00
David Sterba
69a3871fdd btrfs-progs: build: fix reported version of crypto libraries
The option --version actually reports version of pkg-config not the
module/library we're interested in.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 16:20:28 +01:00
David Sterba
afa23d4e27
Btrfs progs v6.6.2
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-15 15:56:12 +01:00
David Sterba
c2b77c03a5 btrfs-progs: update CHANGES for 6.6.2
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-15 15:55:01 +01:00
David Sterba
4d7ea19ca0 btrfs-progs: property set: skip opening char devices completely
Previous fix for char devices and properties opens the path in non
blocking mode but this still triggers the watchdog, as reported. Add a
workaround to properties to completely skip opening the path and just
stat() it.

Issue: #699
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-14 15:48:00 +01:00
Matteo Croce
f5ad6b0444 btrfs-progs: docs: subvolume list -o is not recursive, align documentation with tools behaviour
The `btrfs subvolume list -o` command intentionally doesn't
recurse subvolumes, so make it clear in the documentation.

Pull-reques: #709
Signed-off-by: Matteo Croce <teknoraver@meta.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-14 13:17:35 +01:00
Sam James
ae9ead781d btrfs-progs: fix -Walloc-size warnings reported by gcc 14
GCC 14 introduces a new -Walloc-size included in -Wextra which gives:
```
common/utils.c:983:15: warning: allocation of insufficient size ‘1’ for type ‘struct config_param’ with size ‘32’ [-Walloc-size]
cmds/qgroup.c:1644:13: warning: allocation of insufficient size ‘1’ for type ‘struct btrfs_qgroup_inherit’ with size ‘72’ [-Walloc-size]
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
```

So, just swap the number of members and size arguments to match the prototype, as
we're initialising 1 struct of size `sizeof(struct ...)`. GCC then sees we're not
doing anything wrong.

Pull-request: #707
Signed-off-by: Sam James <sam@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-14 13:14:24 +01:00
Anand Jain
6049b56971 btrfs-progs: tests: use new sysfs interface to check prereq ACL
With the kernel commit 070bb0011ccf ("btrfs: sysfs: show if ACL
support has been compiled in") we can now check if ACL is compiled
without requiring a btrfs device. Retain older method for older
kernels.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-09 15:26:44 +01:00
Boris Burkov
d7c6baf82f btrfs-progs: make OWNER_REF_KEY type value smallest among inline refs
Companion patch to progs for the same change in the kernel. Inline refs
are expected to have non-decreasing type value but owner ref violated
this and got away with it via special parsing. Fix the inconsistency
while it is still experimental.

Link: https://lore.kernel.org/linux-btrfs/20231103134547.GA3548732@perftesting/T/#mca2c0e21ecb7a0da616dd09980b9f008c3c00f63
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-09 15:24:46 +01:00
David Sterba
a361f5bf42
Btrfs progs v6.6.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-05 23:13:28 +01:00
David Sterba
e421a49310 btrfs-progs: update CHANGES for 6.6.1
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-05 23:13:11 +01:00
Sergei Trofimovich
8a687cf954 kernel-shared: uapi: fix BTRFS_IOC_SCAN_DEV defiintion
Without the change `BTRFS_IOC_SCAN_DEV` aliased with `BTRFS_IOC_FORGET_DEV`.
It's a regression introduced in fcd9142b6 "btrfs-progs: docs: formatting,
fixups, updates".

It manifests as a sudden device disappearance when device is scanned:

    machine # [    4.095032] Btrfs loaded, crc32c=crc32c-intel, zoned=no, fsverity=no
    machine # ERROR: device scan failed on '/dev/vdb': No such file or directory
    machine # ERROR: device scan failed on '/dev/vdc': No such file or directory
    (finished: must succeed: mkfs.btrfs -d raid0 /dev/vdb /dev/vdc, in 10.31 seconds)

Issue: #704
Pull-request: #706
Reported-by: Atemu <atemu.main@gmail.com>
Bug: https://github.com/NixOS/nixpkgs/issues/265668
Author: Sergei Trofimovich <slyich@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-05 23:10:28 +01:00
David Sterba
f99258456f
Btrfs progs v6.6
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:23:01 +01:00
David Sterba
661a1207e5 btrfs-progs: update CHANGES for 6.6
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:19:55 +01:00
David Sterba
e39c04ef44 btrfs-progs: tests: fix parsing zone size in nullb
The zone size may not be parsed correctly in 'nullb create' due to
copy&paste error from the 'size'. This is already fixed upstream.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
b1e2de452a btrfs-progs: mkfs: print zone count for each device
In zoned mode print zone count for each device, the zone size must be
the same so it's sufficient to print it in the summary.

  $ mkfs.btrfs -O zoned /dev/nullb[0-3]
  ...
  Zoned device:       yes
    Zone size:        16.00MiB
  ...
  Devices:
     ID        SIZE  ZONES  PATH
      1   512.00MiB     32  /dev/nullb0
      2   256.00MiB     16  /dev/nullb1
      3     1.00GiB     64  /dev/nullb2
      4     2.00GiB    128  /dev/nullb3

Issue: #693
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
f240c6114d btrfs-progs: open path in non-blocking mode when reading fs id
There's a report that reading properties from a sound device the system
is stuck and then gets rebooted by watchdog. Reading from fifo files
gets stuck as well, although this would not trigger the watchdog.

The reason is that open() on fifo files is blocking until the other end
of the pipe is opened. For device nodes it's driver specific, most
device nodes fail right away:

  $ btrfs prop get /dev/tty
  ERROR: object is not a btrfs object: /dev/tty

In case of the sound device the consequences were fatal. We can fix that
by opening the path on non-blocking mode. This is only for reading the
fsid, the fd is closed right after the ioctl so the non-blocking mode
does not affect other operation.

The blocking mode must be used for block devices as e.g. loop devices
may not be finalized when the open() call returns and get_fsid fails.
The known problematic devices are character and fifos.

Issue: #699
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
d185801b45 btrfs-progs: ci: split functional tests for devel workflow
We can run more tests in parallel, the overall CI run time will be
shorter (from 12m to 5m) at the cost of building the same sources.
Also build documentation only once.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
1faaa874a1 btrfs-progs: ci: wait for loop devices before mount
Since a few days the CI started to fail randomly when there were loop
devices used in the tests. The mount fails because some device is
reported to be missing:

  $ losetup --show --find
  /dev/loop3
  ...

  $ mkfs ...
  ERROR: device scan failed on '/dev/loop3': No such file or directory
  ...

  $ mount
  mount: /home/runner/work/btrfs-progs/btrfs-progs/tests/mnt: wrong fs
  type, bad option, bad superblock on /dev/loop3, missing codepage or
  helper program, or other error.

  $ dmesg
  ...
  BTRFS error (device loop0): devid 3 uuid 11d9c345-9527-433e-a024-7102659fa0ee is missing
  BTRFS error (device loop0): failed to read the system array: -2
  BTRFS error (device loop0): open_ctree failed

This was reproducible in the "cli" tests, but also happened on a local
machine.

To fix that wait for all loop devices before mount, the command
'btrfs device ready' should block until that. The convenience helper
does that, for any standalone 'mount' used with loop devices this must
be done manually.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
eed542859f btrfs-progs: ci: split devel tests to groups
The tests are run in one job, we can make it parallel by grouping
related tests together.

- musl test, using docker, does not have to wait for build
- simple tests, library, ioctl, api, ...
- the rest are functional tests

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
Martin Ligabue
8397fee2a3 btrfs-progs: docs: fix typo in btrfs-rescue
"that are do not match -> that do not match"

[ci skip]

Pull-request: #700
Author: Martin Ligabue <martinligabue@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00