Commit Graph

5192 Commits

Author SHA1 Message Date
Daniel Xu 3cbb2d630b btrfs-progs: sort main help menu entries
The output of 'btrfs help' is quite long and requires scrolling. For
someone who has a vague idea of what a subcommand is called but not
quite sure, alphabetical listing can help them find what they're looking
for faster.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:30 +01:00
David Sterba 3850054385 btrfs-progs: add helper to check or wait for exclusive operation
Add helper that will either check a running operation or wait until it's
done, so that commands can be started and enqueued. If there are more
enqueued, an attempt to avoid racing is done based on the remaining
waiting time of each command.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:29 +01:00
Goldwyn Rodrigues 80df942fe7 btrfs-progs: check for exclusive operation before issuing another
If the exclusive operation is available in sysfs file, check if there's
one already running. The check is done for:

- device add, remove, replace
- balance
- filesystem resize

All commands will validate arguments and check before the ioctl or
before any potentially irreversible operations (like clearing device
before replacing).

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:29 +01:00
David Sterba 639ddf61ca btrfs-progs: add helpers for parsing filesystem exclusive operation
Since kernel 5.10, the file /sys/fs/btrfs/FSID/exclusive_operation
exports textual id of the running exclusive operation (balance, device
add/remove, ...). Add definitions and parsing functions so they can be
used to check before another operation is started and potentially
blocks.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-12-10 17:10:26 +01:00
Goldwyn Rodrigues 34f0594bca btrfs-progs: add sysfs file reading helpers
Add helpers to open and read sysfs files from the per-fs directory.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-10 16:45:03 +01:00
David Sterba c4b84f1c5d btrfs-progs: docs: fix mknod arguments of the control device
The provided does not work, the name argument is in the wrong place.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-06 14:35:10 +01:00
David Sterba fbc12fd9ec btrfs-progs: move path_cat_out helpers to path-utils
The path-util.[ch] is the right place, keep the send-utils.h prototypes
as it's part of libbtrfs headers.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 23:26:43 +01:00
Goldwyn Rodrigues b79713a33e btrfs-progs: add get_fsid_fd() for getting fsid using fd
Add a function get_fsid_fd() to use an open file fd to get the
fsid of the mounted filesystem.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 22:46:44 +01:00
David Sterba cc2dd28136 btrfs-progs: docs: add missing option to scrub status
Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 22:44:55 +01:00
David Sterba d60d48fce5 btrfs-progs: scrub status: add unit mode options
Add long options for size units, affecting total and currently scrubbed
bytes. The rate depends on the device speed and could be
disproportionate to the size so it is not affected, except the --raw
option that is in bytes without unit suffix.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 22:44:36 +01:00
David Sterba d9e9198485 btrfs-progs: scrub status: print percents of scrubbed bytes
Add ratio of the bytes scrubbed to total in the status output, like:

  Total to scrub:   2.54TiB
  Bytes scrubbed:   1.59TiB  (62.58%)

Signed-off-by: David Sterba <dsterba@suse.com>
2020-11-04 20:28:31 +01:00
David Sterba 2e1fa48230
Btrfs progs v5.9
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-23 15:25:01 +02:00
David Sterba e3269a4c2e btrfs-progs: update CHANGES for 5.9
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-23 15:24:17 +02:00
David Sterba f7cc20b1fa libbtrfsutil: set pkg-config Version to follow main package
The library ABI is not the right version, set also the util package to
follow main package.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-22 20:34:30 +02:00
David Sterba 5da315e765 btrfs-progs: build: provide variables for main package version
Add .in substitutions for main package version, split and compound
without the 'v' prefix. BTRFS_VERSION_PATCHLEVEL could be empty.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-22 20:34:30 +02:00
David Sterba a178b6e508 libbtrfsutil: let python-btrfsutil version follow main package version
Neal points out that the python bindings of libbtrfsutil version is set
from the library ABI rather than the package itself. As this brings some
confusion to packaging, derive the verion the main package too.

Link: http://lore.kernel.org/linux-btrfs/CAEg-Je-VLz9zZOKEVa+x0V+dpyojtRcjBw7maO73zpmowdOyTQ@mail.gmail.com
Reported-by: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-22 20:34:01 +02:00
Sheng Mao 4498fe1a2a libbtrfsutil: add pkg-config spec file
Add pkg-config file for libbtrfsutil which can be used to set up
compilation and link flags.

The paths in pc file depend on prefix variable but ignore DESTDIR.
DESTDIR is used for packaging and it should not affect the paths in pc
file.

Issue: #285
Signed-off-by: Sheng Mao <shngmao@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba f43df8a863 btrfs-progs: ci: add sanitizer coverage
Add builds with ASAN and UBSAN for both gcc. Clang + asan fails at
some tests due to wrong asan runtime, clang + ubsan fails at
libbtrfsutil/btrfsutil_internal.h:71:20:

	memcpy(&tmp, &sh->type, sizeof(__u32));

which seems to be too strict.

    [TEST/cli]   008-subvolume-get-set-default
subvolume get-default: default id is not 256, but
libbtrfsutil/btrfsutil_internal.h:71:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00
             ^

libbtrfsutil/btrfsutil_internal.h:64:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00
             ^

libbtrfsutil/btrfsutil_internal.h:78:20: runtime error: member access within
misaligned address 0x7ffcc8017f67 for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment

0x7ffcc8017f67: note: pointer points here

 00 00 00 00 07  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 05  00 00 00 00 00 00 00 90  00 00 00

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba 726026f7da btrfs-progs: build: print versions for all subprojects in summary
Update the summary after configure finishes with versions of all
projects that are now hosted in this repository. Mark libbtrfs as
deprecated do discourage new use, formal deprecation will happen in the
future.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba 35614d8a64 btrfs-progs: sb-mod: add syntax to the help text
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba bd017ce732 btrfs-progs: sb-mod: add remaining dev_item members
Add helpers and all remaining members of the device item embedded in the
superblock. No UUID types.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
David Sterba 8de6816b47 btrfs-progs: sb-mod: add dev_item prefix for sb::device members
For parity with output of 'btrfs inspect dump-super', add the
"dev_item." prefix to the devid member. This allows adding more
members without conflicts (like it would be for device total_bytes).

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
Anand Jain c875145cb3 btrfs-progs: sb-mod: add devid to the modifiable list
We need this patch to create a crafted image with bogus devid.

For example:
./btrfs-sb-mod devid =0

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-20 15:49:07 +02:00
Sidong Yang 4179885cf9 btrfs-progs: docs: add qgroup examples
This patch adds examples section in btrfs-qgroup documentation.
The example make a quota group that has two subvolumes as children.

Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:09 +02:00
Rafostar 70411ea81b btrfs-progs: scrub: make scrub status summary easier to read
Currently most btrfs commands separate their output with empty lines
which makes them more human readable. The scrub cmd when used with -d
arg to show per device information does not. It makes it harder to find
values for current disk because they are not separated from each other.
This commit adds an empty line after each device summary to make it
match other btrfs cmd outputs.

For some reason this was the only line in scrub status that did not
start from capital letter. Now it is more consistent with the rest.

Pull-request: #256
Author: Rafostar <Rafostar@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:09 +02:00
Forza-tng fbe95d714f btrfs-progs: docs: update limits for minimal device size
Add limits:

  # mkfs.btrfs loop.img
  btrfs-progs v5.7
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: 'loop.img' is too small to make a usable filesystem
  ERROR: minimum size for each btrfs device is 114294784

  # mkfs.btrfs --mixed loop.img
  btrfs-progs v5.7
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: 'loop.img' is too small to make a usable filesystem
  ERROR: minimum size for each btrfs device is 16777216

Pull-request: #287
Author: Forza-tng <Forza-tng@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:09 +02:00
David Sterba f1b428ea3b btrfs-progs: tests: add case for send vs subvolume deletion
Subvolumes that are part of send must not be deleted, we can check that
only by the error code EPERM, otherwise it's also in the system log.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:08 +02:00
David Sterba 74f45db76c btrfs-progs: subvolume delete: print message for EPERM, possible send in progress
If subvolume deletion fails with EPERM, the most common reasons are that
it's a default subvolume (addressed by an earlier patch) or that the
subvolume is part of a send operation. This is printed to the system log
and there's no information available for user space, but at least the
warning can hint the user that something could be going on.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:08 +02:00
David Sterba 8b4f9546c3 btrfs-progs: tests: add case for deleting default subvolume
Deleting the default subvolume must not succeed and must print a
message to terminal.

Issue: #274
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:12:08 +02:00
Sidong Yang 87804a3f06 btrfs-progs: subvolume: check deleting default subvolume
Deleting the default subvolume is not permitted and kernel prints a
message to the system log. This is not immediately clear to the user and
we had requests to improve that.

This patch will read the default subvolume id and reject deletion
without trying to delete it.

Issue: #274
Issue: #255
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207975
Signed-off-by: Sidong Yang <realwakka@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-10-02 20:04:08 +02:00
Marcos Paulo de Souza 65ecbc7e0e btrfs-progs: convert: show more info when reserve_space fails
btrfs-convert currently can't handle more fragmented block groups when
converting ext4 because the minimum size of a data chunk is 32MiB.

When converting an ext4 fs with more fragmented block group with the disk
almost full, we can end up hitting a ENOSPC problem [1] since smaller
block groups (10MiB for example) end up being extended to 32MiB, leaving
the free space tree smaller when converting it to btrfs.

This patch adds error messages telling which needed bytes couldn't be
allocated from the free space tree and shows the largest portion available:

  create btrfs filesystem:
	  blocksize: 4096
	  nodesize:  16384
	  features:  extref, skinny-metadata (default)
	  checksum:  crc32c
  free space report:
	  total:     1073741824
	  free:      39124992 (3.64%)
  ERROR: failed to reserve 33554432 bytes for metadata chunk, largest available: 33488896 bytes
  ERROR: unable to create initial ctree: No space left on device

Issue: #251
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-29 22:46:12 +02:00
Boris Burkov 92d92e99b7 btrfs-progs: mkfs: support free space tree as -R option
Add a runtime feature (-R) flag for the free space tree. A filesystem
that is mkfs'd with -R free-space-tree then mounted with no options has
the same contents as one mkfs'd without the option, then mounted with
'-o space_cache=v2'.

The only tricky thing is in exactly how to call the tree creation code.
Using btrfs_create_free_space_tree as is did not quite work, because an
extra reference to the eb (root->commit_root) is leaked, which mkfs
complains about with a warning. I opted to follow how the uuid tree is
created by adding it to the dirty roots list for cleanup by
commit_tree_roots in commit_transaction. As a result,
btrfs_create_free_space_tree no longer exactly matches the version in
the kernel sources.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 22:06:04 +02:00
David Sterba 5011ef94ac btrfs-progs: ci: install clang unconditionally
The builds on travis started to fail for the clang and it was missing
completely, so add it to the list of packages.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 21:59:52 +02:00
gandalf3 07fd775b13 btrfs-progs: docs: fix typo in btrfs restore message
mirros -> mirrors

Pull-request: #284
Author: gandalf3 <gandalf3@users.noreply.github.com>
Reviewed-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 21:59:52 +02:00
David Sterba cfe26cca7a btrfs-progs: ci: enable free-space-tree for all mkfs tests
Run all mkfs tests with -R free-space-tree in case the kernel supports
that (4.5+ required).

Signed-off-by: David Sterba <dsterba@suse.com>
2020-09-08 21:59:52 +02:00
Nikolay Borisov c08983291e btrfs-progs: check: fix error reporting on root inode
If btrfs check detects an error on the root inode of a subvolume it
prints:

    Opening filesystem to check...
    Checking filesystem on /dev/vdc
    UUID: 4ac7a216-bf97-4c5f-9899-0f203c20d8af
    [1/7] checking root items
    [2/7] checking extents
    [3/7] checking free space cache
    [4/7] checking fs roots
    root 5 root dir 256 error
    ERROR: errors found in fs roots
    found 196608 bytes used, error(s) found
    total csum bytes: 0
    total tree bytes: 131072
    total fs tree bytes: 32768
    total extent tree bytes: 16384
    btree space waste bytes: 124376
    file data blocks allocated: 65536
     referenced 65536

This is not very helpful since there is no specific information about
the exact error. This is due to the fact that check_root_dir doesn't
set inode_record::errors accordingly. This patch rectifies this and now
the output would look like:

	[1/7] checking root items
	[2/7] checking extents
	[3/7] checking free space cache
	[4/7] checking fs roots
	root 5 inode 256 errors 2000, link count wrong
	ERROR: errors found in fs roots
	found 196608 bytes used, error(s) found
	total csum bytes: 0
	total tree bytes: 131072
	total fs tree bytes: 32768
	total extent tree bytes: 16384
	btree space waste bytes: 124376
	file data blocks allocated: 65536
	 referenced 65536

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:16:36 +02:00
Qu Wenruo fedc94091f btrfs-progs: tests: add test image to fsck/043 for inode transid repair
The image would contain an inode with invalid transid:

        item 4 key (257 INODE_ITEM 0) itemoff 15881 itemsize 160
                generation 6 transid 134217734 size 131072 nbytes 131072
                block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Qu Wenruo 0eafacf668 btrfs-progs: check/original: add inode transid detect and repair support
The inode transid detect and repair is reusing the existing inode
geneartion code.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Qu Wenruo ce2dcb3615 btrfs-progs: check/lowmem: add inode transid detect and repair support
There are quite some reports on kernel rejecting invalid inode
generation, but it turns out to be that, kernel is just rejecting inode
transid. It's a bug in kernel error message.

To solve the problem and make the fs mountable again, add the detect and
repair support for lowmem mode.

The implementation is pretty much the same, just re-use the existing
inode generation detect and repair code.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:55 +02:00
Marcos Paulo de Souza c655b5e4b1 btrfs-progs: make btrfs_lookup_dir_index in parity with kernel code
This function exists in kernel side but using the _item suffix, and
objectid argument is placed before the name argument. Change the
function to reflect the kernel version.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:09:49 +02:00
David Sterba e05111d11e btrfs-progs: ci: fix libreiserfs build
The CI build fails with:

../include/misc.h:76:26: error: unknown type name ‘loff_t’; did you mean ‘off_t’?
   76 | int valid_offset(int fd, loff_t offset);
      |                          ^~~~~~
      |                          off_t

Add the right define that will bring the symbol.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 09252e76ae btrfs-progs: ci: updated zstd version to 1.4.5
There's a new major version of zstd, without any obvious changes that
would affect our build testing coverage.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 692426265a btrfs-progs: merge find-root.c and btrfs-find-root.c
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 0144bcb713 btrfs-progs: move volumes.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 6069bc52a9 btrfs-progs: move transaction.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:06 +02:00
David Sterba 978f300c21 btrfs-progs: move inode.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba abb670f883 btrfs-progs: move ctree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba c03619b864 btrfs-progs: move file.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba 772f0da6df btrfs-progs: move disk-io.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00
David Sterba cf529f36ad btrfs-progs: move print-tree.c to kernel-shared/
Signed-off-by: David Sterba <dsterba@suse.com>
2020-08-31 17:01:05 +02:00