Commit Graph

689 Commits

Author SHA1 Message Date
David Sterba
960e7b3800 btrfs-progs: tests: test log levels
The log levels have been added in 6.2, add the basic coverage.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-12-01 01:32:40 +01:00
David Sterba
32880fa518 btrfs-progs: crypto: add openssl as crypto provider
https://www.openssl.org/ Is a well known cryptography library and since
freshly released version 3.2 it also supports variable digest size of
blake2b, so we can now add it among the crypto providers.

Configure with --with-crypto=openssl.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-30 22:04:45 +01:00
David Sterba
5221aedc00 btrfs-progs: crypto: add Botan as crypto provider
https://botan.randombit.net/ Botan is a cryptography library with C
bindings and provides what we need (sha256 and blake2b), among many
others. Add it to the list of crypto backends if somebody wants to use
it.

Currently the version 2.19 is the latest one. Botan3 3.2.0 exists but
does not seem to be widely available in distros yet.

Configure with --with-crypto=botan.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-30 22:04:45 +01:00
Qu Wenruo
d79eb4b726 btrfs-progs: tests: add test case for subvolume create multiple arguments
The test case would verify the following behaviors:

- Partial failure
  Should return 1, but the remaining valid destinations would still be
  created.

- All success
  That's as usual.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-23 20:21:58 +01:00
Anand Jain
6049b56971 btrfs-progs: tests: use new sysfs interface to check prereq ACL
With the kernel commit 070bb0011ccf ("btrfs: sysfs: show if ACL
support has been compiled in") we can now check if ACL is compiled
without requiring a btrfs device. Retain older method for older
kernels.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-09 15:26:44 +01:00
David Sterba
e39c04ef44 btrfs-progs: tests: fix parsing zone size in nullb
The zone size may not be parsed correctly in 'nullb create' due to
copy&paste error from the 'size'. This is already fixed upstream.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
1faaa874a1 btrfs-progs: ci: wait for loop devices before mount
Since a few days the CI started to fail randomly when there were loop
devices used in the tests. The mount fails because some device is
reported to be missing:

  $ losetup --show --find
  /dev/loop3
  ...

  $ mkfs ...
  ERROR: device scan failed on '/dev/loop3': No such file or directory
  ...

  $ mount
  mount: /home/runner/work/btrfs-progs/btrfs-progs/tests/mnt: wrong fs
  type, bad option, bad superblock on /dev/loop3, missing codepage or
  helper program, or other error.

  $ dmesg
  ...
  BTRFS error (device loop0): devid 3 uuid 11d9c345-9527-433e-a024-7102659fa0ee is missing
  BTRFS error (device loop0): failed to read the system array: -2
  BTRFS error (device loop0): open_ctree failed

This was reproducible in the "cli" tests, but also happened on a local
machine.

To fix that wait for all loop devices before mount, the command
'btrfs device ready' should block until that. The convenience helper
does that, for any standalone 'mount' used with loop devices this must
be done manually.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
5fce099396 btrfs-progs: tests: rename cli/012-defrag-recursion, it's about 'fi du'
The test names 011 and 012 are the same but 012 is actually testing
recursion of the 'btrfs fi du' command.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
f7fb93f1ac btrfs-progs: remove old syntax of 'balance /path'
There was a support for a short syntax of 'btrfs balance' that accepted
a path where normally would be the mandatory subcommand. This was a
heuristic and nowadays everybody should be using the
'btrfs balance action' syntax. The warning was in place for a year, it's
time to remove the short syntax completely.

Issue: #517
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
c9d04fae03 btrfs-progs: tests: test --dry-run for subvolume delete
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
2c747dd83f btrfs-progs: tests: drop 8k and 32k sector sizes from convert tests
The most common sector sizes are 4k, 16k and 64k, we don't really need
to test that with convert, this takes a long time for little benefit as
the node size is only for metadata, while the rest remains the same.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
f195461b0e btrfs-progs: tests: add basic test for ntrfs conversion
The ntfs2btrfs tool recently found a bug in 'check', add the conversion
test support to our testsuite. It's optional and depends on kernel
support of the 'ntfs3' driver and the availability of mkntfs and
ntfs2btrfs, which might not be available everywhere.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
4126a425df btrfs-progs: tests: update btree-test
Minor refactoring, check arguments and print parameters. This can be
used to stress btree allocation by changing run_size.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
836e7ca8ff btrfs-progs: tests: fix quick-test compilation and rename it
There's an old test for btree code that could be used in the testsuite.
Still needs some polishing and review what tests make sense.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
54439594a6 btrfs-progs: tests: add back removed functionality
Previously the XFS-specific code was commented out so we don't need the
headers for building fsstress, this changes how the utility behaves
compared to the one in fstests, e.g. randomness or additional open/close
operations. Add enough code from xfsprogs to make it compile and enable
the #if0-ed code.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
9dcb5afa15 btrfs-progs: tests: fix -Wshadow warnings in fssum and fsstress
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
Qu Wenruo
208073c373 btrfs-progs: tests: add test image of out-of-order inline backref items
Issue #622 reported a case where ntfs2btrfs can generate out-of-order
inline backref items, which can lead to kernel transaction abort, but
not detected by btrfs-check.

This patch would add such image, whose extent tree looks like this for
the only data extent:

        item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16172 itemsize 111
                refs 3 gen 7 flags DATA
                (178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
                (178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
                (178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1

While the original good base image has the following backrefs for the
same data extent:

        item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16172 itemsize 111
                refs 3 gen 7 flags DATA
                (178 0xdfb591fbbf5f519) extent data backref root FS_TREE objectid 258 offset 0 count 1
                (178 0xdfb591fa80d95ea) extent data backref root FS_TREE objectid 257 offset 0 count 1
                (178 0xdfb591f49f9f8e7) extent data backref root FS_TREE objectid 259 offset 0 count 1

Notice the sequence (the 2nd number in the round brackets) should be
descending.  (Meanwhile type should be ascending, but it's way harder to
create.)

For now we don't have a way to fix it, but at least we should detect it.

Issue: #622
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-11-03 18:04:37 +01:00
David Sterba
1357983f69 btrfs-progs: tests: add case for subvolume create -p
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
Qu Wenruo
83ae2d58c9 btrfs-progs: tests: make sure rootdir inode gets its attributes copied
The new test case would:

- Prepare two loopback devices
  One is for storing the source directory (on a btrfs).
  This is to ensure we can set xattr for the directory, as filesystems
  like tmpfs (mostly utilized by mktemp) is not supporting xattr.

  The other one is the real target fs where we call
  "mkfs.btrfs --rootdir" on.

- Create the source directory with the following contents:
  * rootdir inode attributs:
    # mode (750)
    # uid (1000)
    # gid (1000)
    # xattr (user.rootdir)
  * one regular file, with attributes:
    # xattr (user.foorbar)

- Execute "mkfs.btrfs --rootdir" and mount the new fs

- Verify the above attributes
  The target fs should have the same attributes, especially for the
  rootdir inode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-21 15:51:07 +02:00
David Sterba
4f2c9e8ade btrfs-progs: tests: do not run test-convert under test target
The convert tests take a long time and are not necessary for quick test
rounds under the convenience target 'make test'. We still run the tests
in CI.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:34:00 +02:00
David Sterba
05d338d4ab btrfs-progs: tests: add basic mkfs tests for zoned + rst
Test all combinations of data x metadata for the supported profiles.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:33:59 +02:00
David Sterba
863fedd1cb btrfs-progs: tests: add basic mkfs test for raid-stripe-tree
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:33:59 +02:00
David Sterba
63e4532f3b btrfs-progs: tests: fix description of mkfs/024-fst-bitmaps
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:33:59 +02:00
David Sterba
00628edb26 btrfs-progs: tests: add basic mkfs test for block-group-tree
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:33:59 +02:00
David Sterba
8c9243160b btrfs-progs: tests: add array API test template
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-17 19:33:58 +02:00
David Sterba
777b1d8c27 btrfs-progs: tests: check that qgroup show does not crash on stale qgroups
Create subvolumes and delete every N-th to generate output where we can
also see how the stale qgroups are placed (at the end).

Issue: #687
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-13 18:13:12 +02:00
Qu Wenruo
4c6f41f2b8 btrfs-progs: enable -Wshadow for default build
With all the known one fixed, we can enable -Wshadow now.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-10 19:23:10 +02:00
David Sterba
f96e5fe3ba btrfs-progs: tests: disable log scanning after tests
This partially reverts commit 2726a83952,
the part where the test runner scripts call the log scanning. There are
still unfixed extent buffer leak reported by fuzz tests. Disable it
temporarily so CI can pass and do other pre-release checks. Scanning
will be enabled after release again.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 18:49:50 +02:00
Anand Jain
86b1e47c80 btrfs-progs: test: misc/034 remove kernel support
The kernel patch, ("btrfs: reject device with CHANGING_FSID_V2 flag"),
removes kernel support for the CHANGING_FSID_V2 flag. So, drop its
related testcase.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:55 +02:00
Anand Jain
9290442c46 btrfs-progs: tests: update misc/034 for btrfstune -m|M ability to fix previous failures
The misc-test/034-metadata_uuid test case, has four sets of disk images
to simulate failed writes during btrfstune -m|M operations. As of now,
this tests kernel only. Update the test case to verify btrfstune -m|M's
functionality to recover from the same scenarios.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-10-03 01:11:55 +02:00
David Sterba
2726a83952 btrfs-progs: tests: scan test results
Some errors may be reported in the logs only, scan the file each time
there are fresh results. The scanning script will return error that is
supposed to be caught by the testsuite environment.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba
7a7df06f98 btrfs-progs: tests: extend scan-results.sh to scan only a given file
Scanning all results is meant for the whole testsuite, we'd like to make
it more fine grained to verify them once a test is run via the test
running wrappers.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:35 +02:00
David Sterba
e0263f2b31 btrfs-progs: tests: fix test case name detection scan-results.sh
The test markers have changed in 4.17 so the result scanner does not
print the test names (but still detects the errors).

Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
David Sterba
634e9a4792 btrfs-progs: tests: fix build -Wmissing-prototypes warnings
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-12 23:18:34 +02:00
Anand Jain
e8e5537d8c btrfs-progs: tests: add prefixes to error messages in misc-test/034
Add appropriate prefix to the error messages to make it easier to track
down which case failed.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-09-01 13:58:46 +02:00
David Sterba
5c7f36b141 btrfs-progs: tests: fix typo in README
The correct helper name is check_global_prereq, this was once
seen accidentally used in a test.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba
2b35741a39 btrfs-progs: fix more typos found by codespell
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
ebd046eb8d btrfs-progs: tests: renumber json test cases so they match the parameter
The test utility takes a numeric parameter from 1 to max tests but this
is off by one to the test case function names. Unify that so it's clear
which test fails.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
ede538f98d btrfs-progs: print null value for empty uuid in json
This is a potentially breaking change to json output. An all zeros uuid
was printed as "-" but we can utilize native json type null for that.
Note the va_copy must be used as va_arg advances the pointer.

{
	"nulluuid": null
}

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
386765be15 btrfs-progs: tests: fix build warnings in json tests (-Wmissing-prototypes)
The test cases should be static, otherwise it leads to a warning like

    [LD]     json-formatter-test
tests/json-formatter-test.c:40:6: warning: no previous prototype for ‘test_simple_empty’ [-Wmissing-prototypes]
   40 | void test_simple_empty()
      |      ^~~~~~~~~~~~~~~~~

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
be5cb7dfe9 btrfs-progs: add bool type to json
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
Anand Jain
57cd29c895 btrfs-progs: tests: return metadata_uuid or fsid as per METADATA_UUID flag
Previous commit "btrfs-progs: dump-super: print actual metadata_uuid
value" changed the value of the super_block::metadata_uuid to be printed
as it is, without tweaking it depending on the METADATA_UUID flag.

Apply similar tweak in the common helper functions used to read the
metadata_uuid so that test-cases still be successful.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-23 19:36:31 +02:00
Qu Wenruo
cb19235991 btrfs-progs: tests: misc/046: fix false alerts on write detection
[BUG]
Test case misc/046-seed-multi-mount would always fail with the following
error:

    [TEST]   misc-tests.sh
    [TEST/misc]   046-seed-multi-mount
unexpected success: writable file despite read-only mount
test failed for case 046-seed-multi-mount

[CAUSE]
Although mounting seed device is indeed read-only, sprouting it with a
new device would always make it read-write by itself.

The behavior is already there for a long time, thus expecting a new
behavior (not changing the read-only flag) is a little weird.

[FIX]
Instead of doing the write check after the sprout, do it before the
sprout.

This looks more correct, and would not rely on the kernel behavior
change (if we determine to go that path).

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-23 19:36:31 +02:00
Qu Wenruo
e6b412b359 btrfs-progs: tests: misc/058: reduce the space requirement and speed up the test
[BUG]
When I was testing misc/058, the fs still has around 7GiB free space,
but during that test case, btrfs kernel module reports write failures
and even git commands failed inside that fs.

And obviously the test case failed.

[CAUSE]
It turns out that, the test case itself would require 6GiB (4 data
disks) + 1.5GiB x 2 (the two replace target), thus it requires 9 GiB
free space.

And obviously my partition is not that large and failed.

[FIX]
In fact, we really don't need that much space at all.

The test verifies that two consecutive replace operations can be started
and enqueued, the sleep of 1 second is not strictly necessary as the
first command should start the replace right away.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-23 19:36:31 +02:00
Qu Wenruo
885fe371c8 btrfs-progs: tests: misc/030: do not require v1 cache for the test case
[PROBLEM]
Since we have migrated to default v2 cache, the test case
misc/030-missing-device-image is no longer executed:

    [TEST/misc]   030-missing-device-image
    [NOTRUN] unable to create v1 space cache

[CAUSE]
The test case itself is trying its best to cover all paths, including
the data extent read path.

Thus the test case is requiring v1 cache, as that's the only way to
cover the data read path.

[FIX]
Just remove the v1 space cache requirement, it's still better to run the
test even it only exercises the metadata read path.

The good news is, after commit 3ff9d35257 ("btrfs-progs: use
read_data_from_disk() to replace read_extent_from_disk() and replace
read_extent_data()"), all data/metadata read paths are unified.
They only difference is the verification part.

Thus even if we didn't fully exercise the data read path, we didn't lose
much coverage anyway.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-23 19:36:31 +02:00
Qu Wenruo
005d2cc047 btrfs-progs: tests: mkfs/005: use udevadm settle to avoid false alerts
[BUG]
During my test runs of mkfs-tests, 005-long-device-name-for-ssd failed
with the following error messages:

  ====== RUN CHECK dmsetup remove btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc
  device-mapper: remove ioctl on btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc  failed: Device or resource busy
  Command failed.
  failed: dmsetup remove btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc
  test failed for case 005-long-device-name-for-ssd

[CAUSE]
There seems to be a race between "btrfs inspect dump-super" and the
dmsetup removal.

[FIX]
Add a "udevadm settle" before removing the dm devices.

Also since we're here, use the same "udevadm settle" instead of the
manual sleep to wait for the new dm device to show up.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-23 19:36:31 +02:00
David Sterba
835526b120 btrfs-progs: tests: add script to check global prerequisities
Check for each test directory if the utilities requested by
check_global_prereq can be found on the system.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-11 19:59:29 +02:00
David Sterba
54c11ec03f btrfs-progs: tests: check if enqueued replace over replace waits
Signed-off-by: David Sterba <dsterba@suse.com>
2023-07-27 14:45:29 +02:00
David Sterba
177a2eeadb btrfs-progs: tests: don't show grep output when checking stream version
The test misc/058 does not properly filter out the output due to -s that
only ignores non-existent files and was there due to previous changes.
We need to use -q.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-07-26 15:00:47 +02:00
Anand Jain
b51117d68e btrfs-progs: tests: check for btrfs ACL support
Fix failures caused by the lack of ACL support in btrfs. For example:

  $ make test
    ::
    [TEST/misc]   057-btrfstune-free-space-tree
    failed: setfacl -m u:root:x /Volumes/ws/btrfs-progs/tests/mnt/acls/acls.1
    test failed for case 057-btrfstune-free-space-tree
    make: *** [Makefile:493: test-misc] Error 1

Similar failures occurred in the test cases convert/001-ext2-basic,
convert/003-ext4-basic, convert/005-delete-all-rollback, and
convert/006-large-hole-extent.

Resolve it by adding a check for ACL support using the
check_kernel_support_acl() helper function. It gracefully handles the case
when ACL support is not compiled by calling _not_run().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-07-26 14:59:10 +02:00