The warning:
crypto/hash-vectest.c:559:5: warning: no previous prototype for ‘test_hash’ [-Wmissing-prototypes]
559 | int test_hash(const struct hash_testspec *sp
Signed-off-by: David Sterba <dsterba@suse.com>
Copy faster implementation of crc32c from linux kernel as of 6.5-rc7
(x86_64, arch/x86/crypto/crc32c-pcl-intel-asm_64.S). This needs
assembler build support, so detect target architecture so
cross-compilation still works.
Add a special CPU flag so the old and new implementations can be
benchmarked and verified separately.
Sample benchmark:
CPU flags: 0x1ff
CPU features: SSE2 SSSE3 SSE41 SSE42 SHA AVX AVX2 CRC32C_PCL
Block size: 4096
Iterations: 1000000
Implementation: builtin
Units: CPU cycles
NULL-NOP: cycles: 77177218, cycles/i 77
NULL-MEMCPY: cycles: 226313072, cycles/i 226, 62133.395 MiB/s
CRC32C-ref: cycles: 24418596066, cycles/i 24418, 575.859 MiB/s
CRC32C-NI: cycles: 1188335920, cycles/i 1188, 11833.073 MiB/s
CRC32C-PCL: cycles: 463193456, cycles/i 463, 30358.037 MiB/s
XXHASH: cycles: 851606646, cycles/i 851, 16511.916 MiB/s
SHA256-ref: cycles: 74476234956, cycles/i 74476, 188.808 MiB/s
SHA256-NI: cycles: 34198637428, cycles/i 34198, 411.177 MiB/s
BLAKE2-ref: cycles: 14761411664, cycles/i 14761, 952.597 MiB/s
BLAKE2-SSE2: cycles: 18101896796, cycles/i 18101, 776.807 MiB/s
BLAKE2-SSE41: cycles: 12599091062, cycles/i 12599, 1116.087 MiB/s
BLAKE2-AVX2: cycles: 9668247506, cycles/i 9668, 1454.418 MiB/s
The new implementation is about 2.5x faster.
Note: there new version does not work on musl because of linkage
problems (relocations in .rodata), so it's still using the old
implementation.
Signed-off-by: David Sterba <dsterba@suse.com>
In some places we want to read a single u64 value from a sysfs path, or
from fsid directory. Add helpers that do that in one go.
Signed-off-by: David Sterba <dsterba@suse.com>
The sysfs could use more convenience helpers so move the current code to
own file before adding more helpers.
Signed-off-by: David Sterba <dsterba@suse.com>
Currently we only have a very brief explanation on the unexpected error
handling (only ASSERT()/WARN_ON()/BUG_ON()), and no further
recommendation on the proper usage of them.
This patch would improve the guideline by:
- Add btrfs_abort_transaction() usage
Which is the recommended way when possible.
- More detailed explanation on the usage of ASSERT()
Which is only a fail-fast option mostly designed for developers, thus
is only recommended to rule out some invalid function usage.
- More detailed explanation on the usage of WARN_ON()
Mostly for call sites which need a call trace strongly, and is not
applicable for a btrfs_abort_transaction() call.
- Completely discourage the usage of BUG_ON()
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use the sorting API. This is 1:1 transformation of previous single key
sorting and needs to be updated so there are multiple accepted instead.
Signed-off-by: David Sterba <dsterba@suse.com>
API for extensible array of pointers for covenience. A simple wrapper
around a (void *) array with length.
Signed-off-by: David Sterba <dsterba@suse.com>
The test utility takes a numeric parameter from 1 to max tests but this
is off by one to the test case function names. Unify that so it's clear
which test fails.
Signed-off-by: David Sterba <dsterba@suse.com>
This is a potentially breaking change to json output. An all zeros uuid
was printed as "-" but we can utilize native json type null for that.
Note the va_copy must be used as va_arg advances the pointer.
{
"nulluuid": null
}
Signed-off-by: David Sterba <dsterba@suse.com>
Make the timestamp format more descriptive what is actually printed. We
may need separate date or time in the future.
Signed-off-by: David Sterba <dsterba@suse.com>
The test cases should be static, otherwise it leads to a warning like
[LD] json-formatter-test
tests/json-formatter-test.c:40:6: warning: no previous prototype for ‘test_simple_empty’ [-Wmissing-prototypes]
40 | void test_simple_empty()
| ^~~~~~~~~~~~~~~~~
Signed-off-by: David Sterba <dsterba@suse.com>
The json spec allows numeric values and it's recommended to use them
instead of the stringified numbers. This is a potentially breaking change
if some tools relied on the string value.
As most formats we now have are '%llu' and it's convenient to just pass
it to vprintf, don't add a special type for ints. Any new int type must
be added to the list.
{
"number": 1234
}
Signed-off-by: David Sterba <dsterba@suse.com>
The 'str' type was added in ecbb6a7fcd ("btrfs-progs: add json
formatter for escaped string") but not documented. It should be used
e.g. for paths or strings from unknown origin.
Signed-off-by: David Sterba <dsterba@suse.com>
For null or boolean values the "..." quoting must not be done, add
support for that. This is detected internally for each printed value.
Signed-off-by: David Sterba <dsterba@suse.com>
Don't abbreviate generation and use qgroup where it's related to the
qgroup itself and not quotas in general.
Signed-off-by: David Sterba <dsterba@suse.com>
Implements JSON-formatted output for the `subvolume list` command using
the `--format json` global option, much like it is implemented for other
commands.
Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Implements JSON-formatted output for the `subvolume get-default` command
using the `--format json` global option, much like it is implemented for
other commands.
Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Implements JSON-formatted output for the `subvolume list` command using
the `--format json` global option, much like it is implemented for other
commands.
Re-uses the `btrfs_list_layout` infrastructure to nicely fit it into the
existing formatting code.
A notable difference to the normal, text-based output is that in the
JSON output, timestamps include the timezone offset as well.
Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
List and export all fields that may be needed for any subvolume related
json output.
Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Prepare for switching the plain and json output. The format is slightly
different so we can't utilize the unified fmt_* helpers and two separate
printer functions make more sense.
Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
The exact same check is repeated here, with the second being dead code.
Keep the second instance, as that informs the user what is happening.
Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
The upcoming "--device" option requires memory to parse devices, which
should be freed before returning from the main() function. As a
preparation for adding the "--device" option to the "btrfstune" command,
provide a consolidated error return exit from the main function with a
"goto" labeled "free_out". The label "free_out" may not make sense
currently, but it will when the "--device" option is added.
There are several return statements within the main function, and
changing all of them in the main "--device" feature patch would deviate
from the actual for the feature changes. Hence, it made sense to create
a preparatory patch.
The return value for any failure remains the same as in the original code.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When the fsid does not match the metadata_uuid, the METADATA_UUID flag
is set in the superblock.
Changing the fsid using the btrfstune -U|-u option is not possible on a
filesystem with the METADATA_UUID flag set. But we are checking the
METADATA_UUID only from the super_copy, and not from the other scanned
device.
To fix this bug, track the metadata_uuid at the fs_devices level instead
of checking it only on the specified device in the argument, and use it.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Similar to the kernel, introduce the btrfs_fs_devices::total_devs
attribute to know the overall count of devices that are expected to be
present per filesystem.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To prepare for reuniting separated devices due to an incomplete fsid
change task, consolidate and monitor the per device's changing_fsid flag
in the struct btrfs_fs_devices.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If btrfstune is executed on a filesystem that contains a missing device,
the command will now fail.
It is ok to fail when any of the options supported by btrfstune are
used, the filesystem devices should be all available.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Maintain the btrfs_fs_devices::missing counter to track the number of
missing devices, similar to what kernel does.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Previous commit "btrfs-progs: dump-super: print actual metadata_uuid
value" changed the value of the super_block::metadata_uuid to be printed
as it is, without tweaking it depending on the METADATA_UUID flag.
Apply similar tweak in the common helper functions used to read the
metadata_uuid so that test-cases still be successful.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The function btrfs_print_superblock() prints all members of the
superblock as they are, except for the superblock::metadata_uuid.
If the METADATA_UUID flag is unset, it prints the fsid instead of
zero as in the superblock::metadata_uuid.
Perhaps this was done because to match with the kernel
btrfs_fs_devices::metadata_uuid value as it also sets fsid if
METADATA_UUID flag is unset.
However, the actual superblock::metadata_uuid is always zero if the
METADATA_UUID flag is unset. Just to mention the kernel does not alter
the superblock::metadata_uuid value any time.
The dump-super printing fsid instead of zero, is confusing because we
generally expect dump_super to print the superblock value in the raw
format without modification.
Fix this by printing the actual metadata_uuid value instead of fsid.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Test case misc/046-seed-multi-mount would always fail with the following
error:
[TEST] misc-tests.sh
[TEST/misc] 046-seed-multi-mount
unexpected success: writable file despite read-only mount
test failed for case 046-seed-multi-mount
[CAUSE]
Although mounting seed device is indeed read-only, sprouting it with a
new device would always make it read-write by itself.
The behavior is already there for a long time, thus expecting a new
behavior (not changing the read-only flag) is a little weird.
[FIX]
Instead of doing the write check after the sprout, do it before the
sprout.
This looks more correct, and would not rely on the kernel behavior
change (if we determine to go that path).
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When I was testing misc/058, the fs still has around 7GiB free space,
but during that test case, btrfs kernel module reports write failures
and even git commands failed inside that fs.
And obviously the test case failed.
[CAUSE]
It turns out that, the test case itself would require 6GiB (4 data
disks) + 1.5GiB x 2 (the two replace target), thus it requires 9 GiB
free space.
And obviously my partition is not that large and failed.
[FIX]
In fact, we really don't need that much space at all.
The test verifies that two consecutive replace operations can be started
and enqueued, the sleep of 1 second is not strictly necessary as the
first command should start the replace right away.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[PROBLEM]
Since we have migrated to default v2 cache, the test case
misc/030-missing-device-image is no longer executed:
[TEST/misc] 030-missing-device-image
[NOTRUN] unable to create v1 space cache
[CAUSE]
The test case itself is trying its best to cover all paths, including
the data extent read path.
Thus the test case is requiring v1 cache, as that's the only way to
cover the data read path.
[FIX]
Just remove the v1 space cache requirement, it's still better to run the
test even it only exercises the metadata read path.
The good news is, after commit 3ff9d35257 ("btrfs-progs: use
read_data_from_disk() to replace read_extent_from_disk() and replace
read_extent_data()"), all data/metadata read paths are unified.
They only difference is the verification part.
Thus even if we didn't fully exercise the data read path, we didn't lose
much coverage anyway.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
During my test runs of mkfs-tests, 005-long-device-name-for-ssd failed
with the following error messages:
====== RUN CHECK dmsetup remove btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc
device-mapper: remove ioctl on btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc failed: Device or resource busy
Command failed.
failed: dmsetup remove btrfs-test-with-very-long-name-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQPGc
test failed for case 005-long-device-name-for-ssd
[CAUSE]
There seems to be a race between "btrfs inspect dump-super" and the
dmsetup removal.
[FIX]
Add a "udevadm settle" before removing the dm devices.
Also since we're here, use the same "udevadm settle" instead of the
manual sleep to wait for the new dm device to show up.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Check for each test directory if the utilities requested by
check_global_prereq can be found on the system.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba
Qu Wenruo
Francesco Yoshi Gobbo
Christoph Heiss
Anand Jain