The exclusive ops will not start if there's one already running. Now
that we have the sysfs export (since kernel 5.10) to check if there's
one already running, use it to allow enqueueing of the operations as a
convenience.
Supported enqueuing:
btrfs balance start --enqueue
btrfs filesystem resize --enqueue
btrfs device add --enqueue
btrfs device delete --enqueue
btrfs replace start --enqueue
This patch implements the functionality based on Goldwyn's patch
https://lore.kernel.org/linux-btrfs/?q=20200825150338.32610-4-rgoldwyn%40suse.de
but on top of previous preparatory patches.
Note that 'filesystem resize' options could confuse getopt as the
negative size change looks like a series of short options and there's no
way to make getopt ignore the short options, so there's a custom option
parser.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add available space information from statfs(). This can be different from
'Free (estimated)' in some cases. This patch provide more information
about filesystem usage like below.
Overall:
Device size: 5.00GiB
Device allocated: 1.02GiB
Device unallocated: 3.98GiB
Device missing: 0.00B
Used: 88.00KiB
Free (estimated): 4.48GiB (min: 2.49GiB)
Free (statfs, df) 4.48GiB
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 832.00KiB (used: 0.00B)
Multiple profiles: no
Issue: #306
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The limitation is there since first commit implementing swapfiles support.
Pull-request: #315
Signed-off-by: Tomasz Torcz <tomek@pipebreaker.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will:
- Add a new test image for fsck/044
This new image has a corrupted extent item generation for tree block.
This image can expose a bug in original mode, which can't detect the
problem.
This image also utilize the tree block generation detection code,
which the existing image doesn't.
- Rename the existing image
To reflect the fact that the existing one is only for data extent.
- Remove the test.sh
So that the generic path will test both detection and repair.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is pretty much the same as for lowmem mode, it will try to reset
the extent item generation using either the tree block generation or
current transid.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In check_block(), we unconditionally reset extent_record::generation.
This is in fact correct, but this makes original mode fail to detect bad
extent item generation.
So change to behavior to set the generation if and only if the tree
block generation is higher.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is an internal report about bad extent item generation triggering
tree-checker.
This patch will add the repair ability to btrfs check --mode=lowmem
mode, by resetting the generation field of extent item.
Currently the correct generation for tree block is fetched from its
header, while for data extent it uses transid as fallback.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are several problems for current sectorsize check:
- No check at all for sectorsize
This means you can even specify "-s 62k".
- No way to specify sectorsize smaller than page size
Fix all these problems by:
- Introduce btrfs_check_sectorsize()
To do:
* power of 2 check for sectorsize
* lower and upper boundary check for sectorsize
* warn about sectorsize mismatch with page size
- Remove the max() between page size and sectorsize
This allows us to override the sectorsize for 64K page systems.
- Make nodesize calculation based on sectorsize
No need to use page size any more.
Users who specify sectorsize manually really know what they are doing,
and we have warned them already.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The output of 'btrfs help' is quite long and requires scrolling. For
someone who has a vague idea of what a subcommand is called but not
quite sure, alphabetical listing can help them find what they're looking
for faster.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
Signed-off-by: David Sterba <dsterba@suse.com>
Add helper that will either check a running operation or wait until it's
done, so that commands can be started and enqueued. If there are more
enqueued, an attempt to avoid racing is done based on the remaining
waiting time of each command.
Signed-off-by: David Sterba <dsterba@suse.com>
If the exclusive operation is available in sysfs file, check if there's
one already running. The check is done for:
- device add, remove, replace
- balance
- filesystem resize
All commands will validate arguments and check before the ioctl or
before any potentially irreversible operations (like clearing device
before replacing).
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since kernel 5.10, the file /sys/fs/btrfs/FSID/exclusive_operation
exports textual id of the running exclusive operation (balance, device
add/remove, ...). Add definitions and parsing functions so they can be
used to check before another operation is started and potentially
blocks.
Signed-off-by: David Sterba <dsterba@suse.com>
Add helpers to open and read sysfs files from the per-fs directory.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The path-util.[ch] is the right place, keep the send-utils.h prototypes
as it's part of libbtrfs headers.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a function get_fsid_fd() to use an open file fd to get the
fsid of the mounted filesystem.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add long options for size units, affecting total and currently scrubbed
bytes. The rate depends on the device speed and could be
disproportionate to the size so it is not affected, except the --raw
option that is in bytes without unit suffix.
Signed-off-by: David Sterba <dsterba@suse.com>
Add ratio of the bytes scrubbed to total in the status output, like:
Total to scrub: 2.54TiB
Bytes scrubbed: 1.59TiB (62.58%)
Signed-off-by: David Sterba <dsterba@suse.com>
Add .in substitutions for main package version, split and compound
without the 'v' prefix. BTRFS_VERSION_PATCHLEVEL could be empty.
Signed-off-by: David Sterba <dsterba@suse.com>
Neal points out that the python bindings of libbtrfsutil version is set
from the library ABI rather than the package itself. As this brings some
confusion to packaging, derive the verion the main package too.
Link: http://lore.kernel.org/linux-btrfs/CAEg-Je-VLz9zZOKEVa+x0V+dpyojtRcjBw7maO73zpmowdOyTQ@mail.gmail.com
Reported-by: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add pkg-config file for libbtrfsutil which can be used to set up
compilation and link flags.
The paths in pc file depend on prefix variable but ignore DESTDIR.
DESTDIR is used for packaging and it should not affect the paths in pc
file.
Issue: #285
Signed-off-by: Sheng Mao <shngmao@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Update the summary after configure finishes with versions of all
projects that are now hosted in this repository. Mark libbtrfs as
deprecated do discourage new use, formal deprecation will happen in the
future.
Signed-off-by: David Sterba <dsterba@suse.com>
Add helpers and all remaining members of the device item embedded in the
superblock. No UUID types.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For parity with output of 'btrfs inspect dump-super', add the
"dev_item." prefix to the devid member. This allows adding more
members without conflicts (like it would be for device total_bytes).
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We need this patch to create a crafted image with bogus devid.
For example:
./btrfs-sb-mod devid =0
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch adds examples section in btrfs-qgroup documentation.
The example make a quota group that has two subvolumes as children.
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently most btrfs commands separate their output with empty lines
which makes them more human readable. The scrub cmd when used with -d
arg to show per device information does not. It makes it harder to find
values for current disk because they are not separated from each other.
This commit adds an empty line after each device summary to make it
match other btrfs cmd outputs.
For some reason this was the only line in scrub status that did not
start from capital letter. Now it is more consistent with the rest.
Pull-request: #256
Author: Rafostar <Rafostar@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add limits:
# mkfs.btrfs loop.img
btrfs-progs v5.7
See http://btrfs.wiki.kernel.org for more information.
ERROR: 'loop.img' is too small to make a usable filesystem
ERROR: minimum size for each btrfs device is 114294784
# mkfs.btrfs --mixed loop.img
btrfs-progs v5.7
See http://btrfs.wiki.kernel.org for more information.
ERROR: 'loop.img' is too small to make a usable filesystem
ERROR: minimum size for each btrfs device is 16777216
Pull-request: #287
Author: Forza-tng <Forza-tng@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Subvolumes that are part of send must not be deleted, we can check that
only by the error code EPERM, otherwise it's also in the system log.
Signed-off-by: David Sterba <dsterba@suse.com>
If subvolume deletion fails with EPERM, the most common reasons are that
it's a default subvolume (addressed by an earlier patch) or that the
subvolume is part of a send operation. This is printed to the system log
and there's no information available for user space, but at least the
warning can hint the user that something could be going on.
Signed-off-by: David Sterba <dsterba@suse.com>
Deleting the default subvolume is not permitted and kernel prints a
message to the system log. This is not immediately clear to the user and
we had requests to improve that.
This patch will read the default subvolume id and reject deletion
without trying to delete it.
Issue: #274
Issue: #255
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207975
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs-convert currently can't handle more fragmented block groups when
converting ext4 because the minimum size of a data chunk is 32MiB.
When converting an ext4 fs with more fragmented block group with the disk
almost full, we can end up hitting a ENOSPC problem [1] since smaller
block groups (10MiB for example) end up being extended to 32MiB, leaving
the free space tree smaller when converting it to btrfs.
This patch adds error messages telling which needed bytes couldn't be
allocated from the free space tree and shows the largest portion available:
create btrfs filesystem:
blocksize: 4096
nodesize: 16384
features: extref, skinny-metadata (default)
checksum: crc32c
free space report:
total: 1073741824
free: 39124992 (3.64%)
ERROR: failed to reserve 33554432 bytes for metadata chunk, largest available: 33488896 bytes
ERROR: unable to create initial ctree: No space left on device
Issue: #251
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a runtime feature (-R) flag for the free space tree. A filesystem
that is mkfs'd with -R free-space-tree then mounted with no options has
the same contents as one mkfs'd without the option, then mounted with
'-o space_cache=v2'.
The only tricky thing is in exactly how to call the tree creation code.
Using btrfs_create_free_space_tree as is did not quite work, because an
extra reference to the eb (root->commit_root) is leaked, which mkfs
complains about with a warning. I opted to follow how the uuid tree is
created by adding it to the dirty roots list for cleanup by
commit_tree_roots in commit_transaction. As a result,
btrfs_create_free_space_tree no longer exactly matches the version in
the kernel sources.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
The builds on travis started to fail for the clang and it was missing
completely, so add it to the list of packages.
Signed-off-by: David Sterba <dsterba@suse.com>
If btrfs check detects an error on the root inode of a subvolume it
prints:
Opening filesystem to check...
Checking filesystem on /dev/vdc
UUID: 4ac7a216-bf97-4c5f-9899-0f203c20d8af
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
root 5 root dir 256 error
ERROR: errors found in fs roots
found 196608 bytes used, error(s) found
total csum bytes: 0
total tree bytes: 131072
total fs tree bytes: 32768
total extent tree bytes: 16384
btree space waste bytes: 124376
file data blocks allocated: 65536
referenced 65536
This is not very helpful since there is no specific information about
the exact error. This is due to the fact that check_root_dir doesn't
set inode_record::errors accordingly. This patch rectifies this and now
the output would look like:
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
root 5 inode 256 errors 2000, link count wrong
ERROR: errors found in fs roots
found 196608 bytes used, error(s) found
total csum bytes: 0
total tree bytes: 131072
total fs tree bytes: 32768
total extent tree bytes: 16384
btree space waste bytes: 124376
file data blocks allocated: 65536
referenced 65536
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The inode transid detect and repair is reusing the existing inode
geneartion code.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are quite some reports on kernel rejecting invalid inode
generation, but it turns out to be that, kernel is just rejecting inode
transid. It's a bug in kernel error message.
To solve the problem and make the fs mountable again, add the detect and
repair support for lowmem mode.
The implementation is pretty much the same, just re-use the existing
inode generation detect and repair code.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function exists in kernel side but using the _item suffix, and
objectid argument is placed before the name argument. Change the
function to reflect the kernel version.
Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The CI build fails with:
../include/misc.h:76:26: error: unknown type name ‘loff_t’; did you mean ‘off_t’?
76 | int valid_offset(int fd, loff_t offset);
| ^~~~~~
| off_t
Add the right define that will bring the symbol.
Signed-off-by: David Sterba <dsterba@suse.com>
There's a new major version of zstd, without any obvious changes that
would affect our build testing coverage.
Signed-off-by: David Sterba <dsterba@suse.com>
