With previous btrfstune support to convert to block-group-tree, it has
implemented most of the infrastructure for bi-directional conversion.
This patch will implement the remaining conversion support to go back to
extent tree.
The modification includes:
- New convert_to_extent_tree() function in btrfstune.c
It's almost the same as convert_to_bg_tree(), but with small changes:
* No need to set extra features like NO_HOLES/FST.
* Need to delete the block group tree when everything finished.
- Update btrfs_delete_and_free_root() to handle non-global roots
Currently the function can only accepts global roots (extent/csum/free
space trees)
If we pass a non-global root into the function, we will screw up
global_roots_tree and crash.
Since we're going to use btrfs_delete_and_free_root() to free block
group tree which is not a global tree, this is needed.
- New handling for half converted fs in get_last_converted_bg()
There are two cases need to be handled:
* The bg tree is already empty
We need to grab the first bg in extent tree.
Or at conversion function we will fail at grabbing the first bg.
* The bg tree is not empty
Then we need to grab the last bg in extent tree.
- Extra root switching in involved functions. This involves:
* read_converting_block_groups()
* insert_block_group_item()
* update_block_group_item()
We just need to update our target root according to the current
compat_ro and super flags.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The free tier of travis-ci has ended on 05/2021, remove the scripts.
Nowadays zstd has made it to all distros so we don't need to build
it from sources, reiserfs support is fading out so we don't need to
build it either. The integration with coverity scan needs to be done
again. The rest has been moved to new github actions CI.
Signed-off-by: David Sterba <dsterba@suse.com>
The gitlab CI was experimentally used for btrfs-progs testsuite in
nested VM with recent kernel (as the base CI image was too old). This
unfortunately did not work due to time limit for the CI run, the build
and tests haven't finished.
Gitlab supports self-hosted runners so this can be integrated
eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
The block-group-tree used to be under experimental flag in -R but now
that we've deprecated -R it does not make sense to leave
block-group-tree there for compatibility, this has never been exposed to
users.
Signed-off-by: David Sterba <dsterba@suse.com>
The feedback from the community on block group tree is very positive,
the only complain is, end users need to recompile btrfs-progs with
experimental features to enjoy the new feature.
So let's move it out of experimental features and let more people enjoy
faster mount speed.
Also change the option of btrfstune, from `-b` to
`--enable-block-group-tree` to avoid short option.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The option -R|--runtime-features was introduced to support features that
don't result in a full incompat flag change, thus things like
free-space-tree and quota features are put here.
But to end users, such separation of features is not helpful and can be
sometimes confusing.
Thus we're already migrating those runtime features into -O|--features
option under experimental builds.
I believe this is the proper time to move those runtime features into
-O|--features option, and mark the -R|--runtime-features option
deprecated.
For now we still keep the old option as for compatibility purposes.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Build static binaries (published as artifacts):
- btrfs.static - the main tool
- btrfs.box.static - all-in-one tool (functionality selected by file name)
There are a few problems:
- unstable URL, i.e. not one location to get the binaries from, multiple
steps need on web or by using API
- downloaded files are in .zip (compression cannot be disabled with
upload-artifact@v3)
- permissions of the file inside .zip are lost, manual chmod needed
after extraction
- artifacts can be downloaded only for logged users
Issue: #593
Signed-off-by: David Sterba <dsterba@suse.com>
For a release test check that all compatibility build targets work. This
may take long as it's pulling lots of data (no caching yet) so it's not
run on devel.
Signed-off-by: David Sterba <dsterba@suse.com>
Using 'docker run -it' works for interactive sessions but not inside the
CI environment. We want both so make it optional by detecting if stdin
is a terminal.
Signed-off-by: David Sterba <dsterba@suse.com>
There's an unconditional 'shift' in the scripts that will lead to an
error "cannot shift" and this would fail inside CI. We want an implicit
run on current HEAD so shift only when there are more parameters.
Signed-off-by: David Sterba <dsterba@suse.com>
There are some helpers that invoke setup_root_helper internally so it's
not needed to run test.sh without root, but it should be there in case
the test calls SUDO_HELPER so it's always paired.
Signed-off-by: David Sterba <dsterba@suse.com>
We need to call the setup_root_helper before we start messing with the
loop devices.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We missed a couple of $SUDO_HELPER uses in this test that made it
impossible to run without root. Add them in so we can run as a normal
user.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The setup_root_helper needs to be called before messing with the loop
devices, and btrfs check needs to be run with $SUDO_HELPER.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We need to make sure the root helper is setup before calling the loop
helpers, and additionally we need to use $SUDO_HELPER when we run the
final btrfs check. With this patch we can now run this test as a normal
user.
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
- convert "*" lists to "-" for newer releases, this is preferred in pull
requests
- update nested lists to the more compact layout (add more newlines,
un-indent)
- wrap overly long lines
Signed-off-by: David Sterba <dsterba@suse.com>
The commit 6f7151f499 extended the set of recognized valid subcommands
for the old path syntax but wrongly checks for more than 2 parameters.
That way a shortened and valid new syntax is not recognized (here 'can'
is short for 'cancel' and the short form is not in the list):
btrfs-progs-6.1.3
btrfs bal can /
ERROR: balance cancel on '/' failed: Not in progress
btrfs-progs-6.2.2
btrfs bal can /
WARNING: deprecated syntax, please use 'btrfs balance start'
ERROR: cannot access 'can': No such file or directory
Issue: #612
Fixes: 6f7151f499 ("btrfs-progs: balance: fix some cases wrongly parsed as old syntax")
Signed-off-by: David Sterba <dsterba@suse.com>
There's a report that a static build fails when there's a static version
of libudev:
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/libudev.a(path-util.o): in function `path_is_mount_point':
path-util.c:(.text+0xbc0): multiple definition of `path_is_mount_point'; common/path-utils.o:path-utils.c:(.text+0x290): first defined here
There's a helper path_is_mount_point in libudev too but not exported so
dynamic library is fine, unlike static build. The static build of
libudev is not common but we can support that with a simple rename.
Issue: #611
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Since e2fsprog 1.47, even with a newly created empty ext4 filesystem,
btrfs-convert would result an fs that btrfs-check would complain:
# mkfs.ext4 -F test.img
# btrfs-convert test.img
# btrfs-check test.img
Opening filesystem to check...
Checking filesystem on test.img
UUID: e45da158-8967-4e4d-9c9f-66b0d127dbce
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
root 5 inode 266 errors 2000, link count wrong
ERROR: errors found in fs roots
found 26333184 bytes used, error(s) found <<<
total csum bytes: 25540
total tree bytes: 180224
total fs tree bytes: 49152
total extent tree bytes: 16384
btree space waste bytes: 145423
file data blocks allocated: 33947648
referenced 26284032
[CAUSE]
Ext4 has a new compat feature, COMPAT_ORPHAN_FILE, as a better way to
track all the orphan inodes.
This new feature would create a new special inode for this purpose, and
such orphan file inode would not be reachable from any other inode, but
only from super block.
Unfortunately btrfs-convert only skip ext2 known special inodes, not the
newer one.
[FIX]
According to the kernel document, we can locate the orphan file inode
using ext2 super block s_orphan_file_inum, and skip it for
btrfs-convert.
And such skip would only happen if we have the definition of
EXT4_FEATURE_COMPAT_ORPHAN_FILE, to be compatible with older e2fsprogs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use tabular format instead of the defintion list as it's more compact.
Enable syntax hilighting to code examples.
Signed-off-by: David Sterba <dsterba@suse.com>
Add basic build tests and runtime tests devel, triggered by branch push.
For workflow testing there's a stub triggered by push of devel-ci.
Signed-off-by: David Sterba <dsterba@suse.com>
When 'nullb setup' fails to detect something, e.g. the null_blk module,
the whole test fails though the failure is supposed to be caught and
test not run. Use the correct helper that handles potential failures.
Signed-off-by: David Sterba <dsterba@suse.com>
[FALSE ALERT]
There is a false alert when compiling btrfs-progs using gcc 12.2.1:
$ make D=1
kernel-shared/print-tree.c: In function 'print_sys_chunk_array':
kernel-shared/print-tree.c:1797:9: warning: 'buf' may be used uninitialized [-Wmaybe-uninitialized]
1797 | write_extent_buffer(buf, sb, 0, sizeof(*sb));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./kernel-shared/ctree.h:27,
from kernel-shared/print-tree.c:24:
./kernel-shared/extent_io.h:148:6: note: by argument 1 of type 'const struct extent_buffer *' to 'write_extent_buffer' declared here
148 | void write_extent_buffer(const struct extent_buffer *eb, const void *src,
| ^~~~~~~~~~~~~~~~~~~
[CAUSE]
This is a false alert, the uninitialized part of buf will not be
utilized at all during write_extent_buffer().
[FIX]
Instead of allocating such ad-hoc buffer, go a more formal way by
calling alloc_dummy_extent_buffer(), which would properly set all
the members.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Test case misc/034 can fail like this:
====== RUN CHECK mount /dev/loop16 /home/adam/btrfs-progs/tests/mnt
mount: /home/adam/btrfs-progs/tests/mnt: wrong fs type, bad option, bad superblock on /dev/loop16, missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
failed: mount /dev/loop16 /home/adam/btrfs-progs/tests/mnt
And the dmesg looks like this:
loop16: detected capacity change from 0 to 1024000
loop17: detected capacity change from 0 to 1024000
BTRFS: device fsid 593e23af-a7e6-4360-b16a-229f415de697 devid 1 transid 6 /dev/loop16 scanned by mount (79348)
BTRFS info (device loop16): using crc32c (crc32c-intel) checksum algorithm
BTRFS info (device loop16): found metadata UUID change in progress flag, clearing
BTRFS info (device loop16): disk space caching is enabled
BTRFS error (device loop16): devid 2 uuid cde07de6-db7e-4b34-909e-d3db6e7c0b06 is missing
BTRFS error (device loop16): failed to read the system array: -2
BTRFS error (device loop16): open_ctree failed
[CAUSE]
From the dmesg, it shows that although both loopback devices are
properly registered, only one is properly scanned by mount.
Thus the other device is missing, and without "-o degraded" the
filesystem failed to be mounted.
[FIX]
Before we mount the filesystem, also scan them in their passed order
to properly assemble the device list for mount.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For older kernels, the sysfs interface providing the fsid may not be present yet.
Since 32c2e57c65 ("btrfs-progs: read fsid from the sysfs in
device_is_seed"), the fallback to the previous approach to determine
the fsid was not used anymore.
Ensure negative return values of sysfs_open_fsid_file are handled by
falling back to the dev_to_fsid in this case.
Pull-request: #599
Signed-off-by: David Sterba <dsterba@suse.com>
The name of a mounted sub volume can not be found in /proc/self/mounts
but in /proc/self/mountinfo.
Pull-request: #596
Signed-off-by: Alexander Barton <alex@barton.de>
Signed-off-by: David Sterba <dsterba@suse.com>
The wiki has been archived so remove the links from manual page
footers. Also replace the wiki link by RTD site in configure and
libbtrfsutil.
Signed-off-by: David Sterba <dsterba@suse.com>
Deprecate old 'btrfs balance' syntax since new syntax has been
introduced in 2012. We will remove the old syntax completely in a few
releases.
Signed-off-by: Wang Yugui <wangyugui@e16-tech.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Some cases of 'btrfs balance' are wrongly parsed as old syntax.
$ btrfs balance status
ERROR: cannot access 'status': No such file or directory
Currently, only 'start' is successfully excluded in the check of old
syntax. Fix it by adding others in the check of old syntax.
Signed-off-by: Wang Yugui <wangyugui@e16-tech.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In Dry run the following error appeared and aborted execution:
ERROR: failed to access 'XYZ' to restore metadata/xattrs: No such file or directory
Skip the metadata and xattrs handling when dry run is enabled.
Signed-off-by: Holger Jakob <jakob@dsi.uni-stuttgart.de>
Signed-off-by: David Sterba <dsterba@suse.com>
