Commit Graph

6451 Commits

Author SHA1 Message Date
David Sterba
c75cc187a8 btrfs-progs: scrub: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
808fdcee3b btrfs-progs: restore: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
dc6a4265bf btrfs-progs: rescue: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
7b1e2e3b88 btrfs-progs: replace: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
e9614be78e btrfs-progs: reflink: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
0a09370313 btrfs-progs: receive: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
a958ad85ec btrfs-progs: quota: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
e8569299c4 btrfs-progs: qgroup: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
8c0d870ac7 btrfs-progs: property: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
251d397870 btrfs-progs: inspect-internal: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:24 +01:00
David Sterba
5dd9506409 btrfs-progs: filesystem: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
fd112d04fc btrfs-progs: device: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
4b8d51ded9 btrfs-progs: check: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
38f90c6ad6 btrfs-progs: balance: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
dde2f15010 btrfs-progs: docs: remove mentions of btrfs-debug-tree, btrfs-show-super, btrfs-zero-log
The standalone tools have been deprecated and removed in 2018 and nobody
should miss them.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
5edf4950b4 btrfs-progs: reformat global options and update docs
With the growing list of global option we need to print them somewhere
in the help text and document them.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
67964cbcda btrfs-progs: add global option --log=level
Add an option to set the log level exactly instead of -vv and similar.
The combined option may not be known to all users as reported and one
option for the level is for convenience.

Issue: #570
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
c811ccc6c9 btrfs-progs: convert: convert help text to option formatter
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
a7fa81f296 btrfs-progs: open code print_usage where applicable
After previous change to usage() that now has the return code, there's
no purpose of the print_usage() wrapper so it can be removed.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
Qu Wenruo
e3a24cf11a btrfs-progs: tests: fix cli/017 test case failure
[BUG]
Test case cli/017 fails with the following errors:

    [TEST]   cli-tests.sh
    [TEST/cli]   017-fi-show-missing
  didn't find exact missing device
  test failed for case 017-fi-show-missing

[CAUSE]
After kernel commit cb3e217bdb39 ("btrfs: use btrfs_dev_name() helper to
handle missing devices better"), all dev info ioctl call on missing
device would only return "<missing disk>" for its path.

Thus "btrfs filesystem show" would never report detailed device path for
missing disks.

[FIX]
Instead of relying on the device path, change the check to rely on devid
instead.

Now cli/017 can properly pass.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
Qu Wenruo
f61b90aff9 btrfs-progs: make usage call properly return an exit value
[BUG]
Currently cli/009 test case failed with different exit number:

  ====== RUN CHECK /home/adam/btrfs-progs/btrfstune --help
  usage: btrfstune [options] device
  [...]
  failed: /home/adam/btrfs-progs/btrfstune --help
  test failed for case 009-btrfstune

[CAUSE]
In tune/main.c, we have the following call on usage():

  static void print_usage(int ret)
  {
	usage(&tune_cmd);
	exit(ret);
  }

However usage() itself would always call exit(1):

  void usage(const struct cmd_struct *cmd)
  {
	usage_command_usagestr(cmd->usagestr, NULL, 0, true, true);
	exit(1);
  }

This makes prevents any caller of usage() to modify its exit number.

[FIX]
Add a new argument @error for print_usage(), so we can properly return 0
for -h/--help usage.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
Anand Jain
32c2e57c65 btrfs-progs: read fsid from the sysfs in device_is_seed
The kernel commit a26d60dedf9a ("btrfs: sysfs: add devinfo/fsid to
retrieve actual fsid from the device") introduced a sysfs interface
to access the device's fsid from the userspace. This is a more
reliable method to obtain the fsid compared to reading the
superblock, and it even works if the device is not present.
Additionally, this sysfs interface can be read by non-root users.

Therefore, it is recommended to utilize this new sysfs interface to
retrieve the fsid.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
Anand Jain
bdd0ca8a9a btrfs-progs: prepare helper device_is_seed
load_device_info() checks if the device is a seed device by reading
superblock::fsid and comparing it with the mount fsid, and it fails
to work if the device is missing (a RAID1 seed fs). Move this part
of the code into a new helper function device_is_seed() in
preparation to make device_is_seed() work with the new sysfs
devinfo/<devid>/fsid interface.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
8098e6d5c1 btrfs-progs: tests: fix paths of library-test.c includes
There are now own copies of ioctl.h and kerncompat.h just for libbtrfs
so the library test should use them.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
140234dc0d btrfs-progs: move include from toplevel directory to include/
In order to reduce number of files in the toplevel directory,

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
602096162a btrfs-progs: README: replace or remove wiki links
The site btrfs.wiki.kernel.org will be archived in the near future.
Replace link where there is an existing page on RTD and remove the rest.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
629d013f48 btrfs-progs: README: update links and process
Some links point to wiki where a RTD page already exists, change that.
Reword the development process so it does not sound that it's
mailinglist-only. In the past pull requests were sent and processed so
this may encourage more contributions.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:23 +01:00
David Sterba
c8c27b291a btrfs-progs: README: remove link to coverity
The coverity status has been stale and we don't have any CI or manual
process to update that, so remove it for now. It was unreliable in the
past (#74).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
b08b429ee2 btrfs-progs: fi mkswapfile: add option --uuid
Add option --uuid with same semantics that is provided by command
'mkswap'. By default a random UUID is generated, to not set any use
'btrfs filesystem mkswapfile -U clear swapfile'.

Issue: #581
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
9be33f558c btrfs-progs: tune: update checksum conversion
The checksum conversion is still experimental and still does not convert
all filesystems correctly. Do not use on valuable data.

Previous implementation copied the UUID conversion which was not a good
base for the checksum conversion so it left out basically all trees
except extent and checksum.

This update adds the base for the required safety features:

- let the old csum tree intact until the full conversion is done (ie.
  all data are still verifiable)
- add on-disk status tracking item, this should keep the from/to
  checksum conversion, last generation to catch potential updates of the
  underlying filesystem if conversion is interrupted and the filesystem
  mounted
- convert most of the fundamental trees, the subvolumes, tree log and
  relocation trees are not converted
- trees are converted in-place to avoid potentially running out of space
  but this might be better done by transaction protection with a
  temporary tree

Known issues:

- not all trees are converted
- not all checksums are correctly inserted into the new tree and reading
  the files leads to EIO

Issue: #438
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
0a464d4155 btrfs-progs: ci: enable gcc 13 on tumbleweed
GCC 13 is now available, add it as default compiler to tumbleweed image
to catch new warnings.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
f629b5041b btrfs-progs: ci: update documentation
Write a quick start howto and point to the admin scripts.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
256e64dfa8 btrfs-progs: ci: update and sync scripts
The main script ci/images/test-build should be most up to date and
copied to all docker templates, sync them. Fix descriptions that were
copy&pasted from musl. Add missing test-build script from musl image.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
cfc093bb6a btrfs-progs: ci: fix Centos 8 package urls
A fresh build of Centos 8 fails due to changes in upstream repositoris:

"Error: Failed to download metadata for repo ‘appstream’: Cannot prepare
internal mirrorlist: No URLs in mirrorlist"

There are repositories no longer updated (and missing some packages like
reiserfs-utils) but for our CI purposes it's sufficient so make it work.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
3d9217f7ab btrfs-progs: hash-speedtest: select implementation by features
Now put all the recent changes into action. Add a callback that will
reinitialize the implementation pointers according to the desired
feature. Reference implementations use the NONE CPU flag to distinguish
them from the rest.

Example results:

$ hash-speedtest
CPU flags: 0xff
CPU features: SSE2 SSSE3 SSE41 SSE42 SHA AVX AVX2
Block size:     4096
Iterations:     1000000
Implementation: builtin
Units:          CPU cycles

    NULL-NOP: cycles:     67129026, cycles/i       67
 NULL-MEMCPY: cycles:    231303654, cycles/i      231,    60792.500 MiB/s
  CRC32C-ref: cycles:  23982698042, cycles/i    23982,      586.322 MiB/s
   CRC32C-NI: cycles:   1168017624, cycles/i     1168,    12038.828 MiB/s
      XXHASH: cycles:    838434468, cycles/i      838,    16771.152 MiB/s
  SHA256-ref: cycles:  68296865380, cycles/i    68296,      205.889 MiB/s
   SHA256-NI: cycles:  29748853920, cycles/i    29748,      472.676 MiB/s
  BLAKE2-ref: cycles:  14532177414, cycles/i    14532,      967.617 MiB/s
 BLAKE2-SSE2: cycles:  17762215810, cycles/i    17762,      791.657 MiB/s
BLAKE2-SSE41: cycles:  12370044656, cycles/i    12370,     1136.744 MiB/s
 BLAKE2-AVX2: cycles:   9472823338, cycles/i     9472,     1484.412 MiB/s

Previously:

Block size:     4096
Iterations:     1000000
Implementation: builtin
Units:          CPU cycles

    NULL-NOP: cycles:     67714016, cycles/i       67
 NULL-MEMCPY: cycles:    234140818, cycles/i      234,    60055.762 MiB/s
      CRC32C: cycles:   1187358432, cycles/i     1187,    11842.733 MiB/s
      XXHASH: cycles:   1897530684, cycles/i     1897,     7410.448 MiB/s
      SHA256: cycles:  69855340702, cycles/i    69855,      201.296 MiB/s
      BLAKE2: cycles:  14713130972, cycles/i    14713,      955.716 MiB/s

The CPU is i7-11700 3.60GHz and not the same as previous results
mentioned in changelogs so the results are incomparable. Otherwise, the
updated xxhash implementation is twice as fast, no significant changes
for the rest.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
8a60fde969 btrfs-progs: crypto: use common CPU feature detection for crc32c
The crc32c selection has been already using the pointer-based approach
so drop the cpuid detection and use our common code for that.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
979fe686d2 btrfs-progs: libbtrfs: own copy of crc32c
The crc32c.c is going to be updated with CPU feature detection from
common/ which is not suitable for libbtrfs, so make own copy of the
file.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:11:22 +01:00
David Sterba
bbf703bfd3 btrfs-progs: crypto: call sha256 implementations by pointer
Change how sha256 implementation is selected. Instead of an if-else
check in the block processing function select the best version and assign
the function pointer. This is slightly faster.

At this point the selection is not implemented properly in
hash-speedtest so all results are from the fastest version. This will
be fixed once all algorithms are converted.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 20:10:58 +01:00
David Sterba
d1c366ee42 btrfs-progs: crypto: call blake2 implementations by pointer
Change how blake2 implementation is selected. Instead of an if-else check
inside blake2b_compress each time, select the best one and assign the
function pointer. This is slightly faster.

At this point the selection is not implemented properly in
hash-speedtest so all results are from the fastest version. This will
be fixed once all algorithms are converted.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:53:43 +01:00
David Sterba
24ec095295 btrfs-progs: crypto: add common function for accelerated initialization
Prepare a single location that will detect or set accelerated versions
of hash algorithms. Right now it's the crc32c, blake2 and sha256 do
an if-else switch while crc32c sets a function pointer.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:31 +01:00
David Sterba
7fba4bf4db btrfs-progs: add special CPU flag bit representing no features
Add CPU_FLAG_NONE that's first in the list and represents a CPU without
any acceleration features. Can be used for reference/fallback
implementation.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:31 +01:00
David Sterba
79b3ed9ab8 btrfs-progs: tune: fix typos and mistakes in messages
The conversion have been copy&pasted from one code but not all messages
reflect that and mistakenly say fsid instead of csum, etc. Also rename
functions converting the trees to more descriptive names.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:31 +01:00
David Sterba
85f49ce2cd btrfs-progs: tune: convert printf to message helpers
Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:31 +01:00
David Sterba
7f0035c9d1 btrfs-progs: crypto: update xxhash
Update xxhash implementation from https://github.com/Cyan4973/xxHash.
This has moved a lot of code so the diff is huge, plus the code we don't
need now for btrfs has been removed (XXH3, XXH32).

There's no significant change in performance.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:31 +01:00
David Sterba
5c663502bf btrfs-progs: build: use plain fcntl.h instead of sys/fcntl.h
The CI build on musl warns:

    [CC]     cmds/balance.o
In file included from cmds/inspect.c:19:
/usr/include/sys/fcntl.h:1:2: warning: #warning redirecting incorrect #include <sys/fcntl.h> to <fcntl.h> [-Wcpp]
    1 | #warning redirecting incorrect #include <sys/fcntl.h> to <fcntl.h>
      |  ^~~~~~~

On glibc the header directly includes fcntl.h.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba
18e12e95ab btrfs-progs: build: fix detection of std=gnu11 build
Commit 0d5767e664 ("btrfs-progs: build: use -std=gnu11") was supposed
to set C level to gnu11 to match kernel but it was done only in the C=1
(checker) branch. The std used was still gnu90 that can be verified by
running with V=1.

Set the std option to gnu11 unconditionally and delete the m4 macros
that were detecting support on older gcc (4.5). The reference build on
lowest supported distro (Centos 7) compiles fine.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba
b473169d2d btrfs-progs: ci: drop reiserfs support from Tumbleweed image
The devel package for reiserfs is no longer available in openSUSE
Tumbleweed. Build auto-detects the support for convert.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba
01faa47136 btrfs-progs: build: adjust spacing for ioctl-test output
The messages of ioctl-test related build is not aligned with other
commands:

    [LD]     hash-speedtest
    [LD]     hash-vectest
    [CC]   ioctl-test.o
    [LD]   ioctl-test

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba
d4bc1bf290 btrfs-progs: ci: build test programs
The test programs are not build by default but we want at least the
build coverage, so add them. Also make the build parallel.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00
David Sterba
6048ff5a47 btrfs-progs: crypto: clean up types and includes for crc32c
There are some stale headers that we don't need and the int types are
using the kernel types and pull kerncompat.h. As this is a basic header
that should minimize dependencies use the standard int types.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-02-28 19:49:30 +01:00