There are two configure options for compression so group them and update
the description so it mentions all commands that use compression.
Signed-off-by: David Sterba <dsterba@suse.com>
It does not make sense to pass only the compression option when there
are no files being added by --rootdir.
Signed-off-by: David Sterba <dsterba@suse.com>
Report invalid compression specification while parsing the options. Now
an ivalid level won't be silently accepted and capped when processing
the files. Other checks regarding conditional support of LZO and ZSTD
are left in place.
Signed-off-by: David Sterba <dsterba@suse.com>
The compression support is optional, eg. also in 'btrfs-restore', so
print the support in help text.
usage: mkfs.btrfs [options] <dev> [<dev...>]
...
--compress ALGO[:LEVEL] compress files by algorithm and level, ALGO can be 'no' (default), zstd, lzo, zlib
Built-in:
- ZSTD: yes
- LZO: yes
- ZLIB: yes
...
Signed-off-by: David Sterba <dsterba@suse.com>
Improve readability and add space between sections.
usage: mkfs.btrfs [options] <dev> [<dev...>]
Create a BTRFS filesystem on a device or multiple devices
Allocation profiles:
-d|--data PROFILE data profile, raid0, raid1, raid1c3, raid1c4, raid5, raid6, raid10, dup or single
-m|--metadata PROFILE metadata profile, values like for data profile
-M|--mixed mix metadata and data together
Features:
--csum TYPE
--checksum TYPE checksum algorithm to use, crc32c (default), xxhash, sha256, blake2
-n|--nodesize SIZE size of btree nodes
-s|--sectorsize SIZE data block size (may not be mountable by current kernel)
-O|--features LIST comma separated list of filesystem features (use '-O list-all' to list features)
-L|--label LABEL set the filesystem label
-U|--uuid UUID specify the filesystem UUID (must be unique for a filesystem with multiple devices)
--device-uuid UUID Specify the filesystem device UUID (a.k.a sub-uuid) (for single device filesystem
only)
...
Signed-off-by: David Sterba <dsterba@suse.com>
Clang 19 reports:
In file included from common/fsfeatures.c:34:
./common/tree-search.h:11:2: warning: field '' with variable sized type
'union btrfs_tree_search_args::(anonymous at
./common/tree-search.h:11:2)' not at the end of a struct or class is a
GNU extension [-Wgnu-variable-sized-type-not-at-end]
which is correct. This does not still fix the problem with tree search
v2, disabled in d73e698248 ("btrfs-progs: temporarily disable usage
of v2 of search tree ioctl").
Signed-off-by: David Sterba <dsterba@suse.com>
Define attribute for functions that are not used (but are typically a
copy from kernel headers). Reported by clang 19.
Signed-off-by: David Sterba <dsterba@suse.com>
There are 3 rules repeating the pattern for tags files but it gets out
of sync, e.g. it lacks crypto/. To fix that generate the list in one
place from one place.
Signed-off-by: David Sterba <dsterba@suse.com>
Use glob/fnmatch expression to filter hashes, case insensitive:
$ ./hash-speedtest -f 'crc*'
$ ./hash-speedtest -f '*ref'
The NULL-MEMCPY run is always done to warm up caches.
Signed-off-by: David Sterba <dsterba@suse.com>
(Linux kernel commit 84dd048cf89557e1e4badd20c9522f7aaa0275fe).
crc32c-pcl-intel-asm_64.S has a loop with 1 to 127 iterations fully
unrolled and uses a jump table to jump into the correct location. This
optimization is misguided, as it bloats the binary code size and
introduces an indirect call. x86_64 CPUs can predict loops well, so it
is fine to just use a loop instead. Loop bookkeeping instructions can
compete with the crc instructions for the ALUs, but this is easily
mitigated by unrolling the loop by a smaller amount, such as 4 times.
Therefore, re-roll the loop and make related tweaks to the code.
This reduces the binary code size of crc_pclmul() from 4546 bytes to 418
bytes, a 91% reduction. In general it also makes the code faster, with
some large improvements seen when retpoline is enabled.
More detailed performance results are shown below. They are given as
percent improvement in throughput (negative means regressed) for CPU
microarchitecture vs. input length in bytes. E.g. an improvement from
40 GB/s to 50 GB/s would be listed as 25%.
Table 1: Results with retpoline enabled (the default):
| 512 | 833 | 1024 | 2000 | 3173 | 4096 |
---------------------+-------+-------+-------+------ +-------+-------+
Intel Haswell | 35.0% | 20.7% | 17.8% | 9.7% | -0.2% | 4.4% |
Intel Emerald Rapids | 66.8% | 45.2% | 36.3% | 19.3% | 0.0% | 5.4% |
AMD Zen 2 | 29.5% | 17.2% | 13.5% | 8.6% | -0.5% | 2.8% |
Table 2: Results with retpoline disabled:
| 512 | 833 | 1024 | 2000 | 3173 | 4096 |
---------------------+-------+-------+-------+------ +-------+-------+
Intel Haswell | 3.3% | 4.8% | 4.5% | 0.9% | -2.9% | 0.3% |
Intel Emerald Rapids | 7.5% | 6.4% | 5.2% | 2.3% | -0.0% | 0.6% |
AMD Zen 2 | 11.8% | 1.4% | 0.2% | 1.3% | -0.9% | -0.2% |
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: David Sterba <dsterba@suse.com>
(Linux kernel commit eebcadfa21e66a893846ec168fb7c26a46a510cd).
Fix crc32c-pcl-intel-asm_64.S to access 32-bit arguments as 32-bit
values instead of 64-bit, since the upper bits of the corresponding
64-bit registers are not guaranteed to be zero. Also update the type of
the length argument to be unsigned int rather than int, as the assembly
code treats it as unsigned.
Note: there haven't been any reports of this bug actually causing
incorrect behavior. Neither gcc nor clang guarantee zero-extension to
64 bits, but zero-extension is likely to happen in practice because most
instructions that operate on 32-bit registers zero-extend to 64 bits.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: David Sterba <dsterba@suse.com>
(Linux kernel patch 84ebf9dbe652355461b3e2f4693ce7b7402c30ca).
The assembly code in crc32c-pcl-intel-asm_64.S is invoked only for
lengths >= 512, due to the overhead of saving and restoring FPU state.
Therefore, it is unnecessary for this code to be excessively "optimized"
for lengths < 200. Eliminate the excessive unrolling of this part of
the code and use a more straightforward qword-at-a-time loop.
Note: the part of the code in question is not entirely redundant, as it
is still used to process any remainder mod 24, as well as any remaining
data when fewer than 200 bytes remain after least one 3072-byte chunk.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: David Sterba <dsterba@suse.com>
To support LSP in editors add a convenience command to generate the
commands that eg. clangd uses to generate it's database. This requires
the Bear tool to scan build. Compilation can be done by gcc or clang,
only the .json needs the explicit CC=clang. LSP is sensitive to the
CFLAGS.
Signed-off-by: David Sterba <dsterba@suse.com>
Add missing checks reported by autoscan. Reorder the sections, headers,
types and functions. Some of the reported issues are not fixed as
they're not critical.
Signed-off-by: David Sterba <dsterba@suse.com>
Minor update to the command format and indentation, drop the [ ] and
prefix only by two spaces (like kernel).
Signed-off-by: David Sterba <dsterba@suse.com>
The version 3.2 which provides BLAKE2b with output hash length has been
added to Tumbleweed so we can now test it in the CI.
Signed-off-by: David Sterba <dsterba@suse.com>
The library version 2 is being removed from distros, use the newer one.
The minimum requirement of BLAKE2b support is already in the version
3.0.0.
Signed-off-by: David Sterba <dsterba@suse.com>
Tumbleweed has migrated to botan3, and there is no botan2 package for
backward compatibility.
So here we have to disable it until we migrated to botan3.
Signed-off-by: Qu Wenruo <wqu@suse.com>
When simple quotas is enabled, every new data extent gets a special
inline OWNER_REF item that identifies the owning subvolume. This makes
simple quotas backwards incompatible with kernels older than v6.7. Even
if you disable quotas on the filesystem, the OWNER_REF items are
sprinkled throughout the extent tree and older kernels are unable to
parse them.
However, it is relatively easy to simply walk the extent tree and remove
these inline ref items. This gives squota adopters the option to *fully*
disable squotas on their system and un-set the incompat bit. Add this
capability to btrfstune, which requires only a little tricky btrfs item
data shifting.
This functionality was tested with a new unit test, as well as a similar
but more thorough integration test in fstests
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Used in clear_free_space_tree, this is a totally generic operation.
It will also be used for clearing the qgroup tree from btrfstune.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
[ Make remove_all_tree_items() to use btrfs_clear_tree() ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
The documentation lists -q as the flag for enabling simple quotas, but
the actual parsing only handles --enable-simple-quota. Update the
documentation string.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
[ Add man page entry for `--enable-simple-quota` option ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Allow --compress to work with lzo.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
[ Add extra handling when LZO support is not compiled in ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
There were two major problems with add_file_items(): it was
writing all files sector-by-sector, making compression impossible, and
it was assuming that pread would never do a short read.
Fix these problems, and create a new helper add_file_item_extent().
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Just like insert_reserved_file_extent() from the kernel, we can make
btrfs_insert_file_extent() accept an on-stack file extent item
directly.
This makes btrfs_insert_file_extent() more flex, and it can now handle
the converted file extent where it has an non-zero offset.
And this makes it much easier to expand for future compressed file
extent generation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The function btrfs_record_file_extent() has extra handling that's
specific to convert, like allowing the range to be split by block group
boundary and image file extent boundary.
All of these split can only lead to corruption for non-converted fs.
As the only caller out of btrfs-convert is rootdir, which expects the
file extent item insert to respect the reserved data extent, and never
to be split.
Thankfully this is not going to cause huge problem, as
btrfs_record_file_extent() has extra checks if the data extent overlaps
with any existing one, and if it doesn't the handling will be the same
as the kernel.
But to avoid abuse, change btrfs_record_file_extent() by:
- Rename it to btrfs_convert_file_extent()
And add extra comments on that it is specific to btrfs-convert.
- Move it to convert/common.[ch]
- Introduce a helper insert_reserved_file_extent() for rootdir.c
Signed-off-by: Qu Wenruo <wqu@suse.com>
