Userspace utilities to manage btrfs filesystems
Go to file
Qu Wenruo 0befc6dce2 btrfs-progs: mkfs: recow all tree blocks properly
[BUG]
Since btrfs-progs v5.14, mkfs.btrfs no longer cleans up the temporary
SINGLE metadata chunks if "-R free-space-tree" is specified:

 $ mkfs.btrfs  -f -R free-space-tree -m dup -d dup /dev/test/test
 $ btrfs ins dump-tree -t chunk /dev/test/test | grep "type METADATA"
		length 8388608 owner 2 stripe_len 65536 type METADATA
		length 268435456 owner 2 stripe_len 65536 type METADATA|DUP

[CAUSE]
Since commit 4b6cf2a3eb ("btrfs-progs: mkfs: generate free space tree
at make_btrfs() time"), free space tree is created when the temporary
btrfs image is created.

This behavior itself has no problem at all.  The problem happens when
"-m DUP -d DUP" (or other profiles) is specified.

This makes btrfs to create extra chunks, enlarging free space tree so
that it can be as high as level 1.

During mkfs, we rely on recow_roots() to re-COW all tree blocks to the
newly allocated chunks.

But __recow_root() can only handle tree root at level 0, as it forces
root node to be COWed, not bothering the children leaves/nodes.

This makes part of the free space cache tree still live on the old
temporary chunks, leaving later cleanup_temp_chunks() unable to delete
temporary SINGLE chunks.

[FIX]
Rework __recow_root() to do a proper COW of the whole tree.

But above rework is not enough, as if a free space tree block is
allocated during current transaction, but before new chunks added.
Then the reworked __recow_root() can't COW it, as btrfs_search_slot()
won't COW a tree block allocated in current transaction.

So this patch will also commit current transaction before calling
recow_roots(), to force us to re-cow all tree blocks.

This shouldn't be a problem, as at the time of calling, we should have
less than a dozen tree blocks, thus there won't be a performance impact.

Reported-by: FireFish5000 <firefish5000@gmail.com>
Fixes: 4b6cf2a3eb ("btrfs-progs: mkfs: generate free space tree at make_btrfs() time")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-20 18:59:24 +02:00
check btrfs-progs: libbtrfs: remove unneeded BTRFS_FLAT_INCLUDES protections 2021-10-08 20:47:03 +02:00
ci btrfs-progs: ci: add missing docker scripts for musl image 2021-10-06 16:50:56 +02:00
cmds btrfs-progs: use btrfs_bg_type_is_stripey for enumerated lists 2021-10-20 18:59:24 +02:00
common btrfs-progs: introduce helper to get allowed profiles for a given device number 2021-10-20 18:59:24 +02:00
convert btrfs-progs: set eb::fs_info properly everywhere 2021-10-08 20:47:04 +02:00
crypto btrfs-progs: libbtrfs: remove unneeded BTRFS_FLAT_INCLUDES protections 2021-10-08 20:47:03 +02:00
Documentation btrfs-progs: mkfs: don't autoselect DUP on SSD for metadata anymore 2021-10-08 20:47:03 +02:00
image btrfs-progs: introduce OPEN_CTREE_ALLOW_TRANSID_MISMATCH flag 2021-09-20 12:17:29 +02:00
kernel-lib btrfs-progs: kernel-lib: simplify raid56_recov 2021-10-20 18:59:23 +02:00
kernel-shared btrfs-progs: rename data parameter to profile in extent allocation path 2021-10-20 18:59:24 +02:00
libbtrfs btrfs-progs: libbtrfs: copy btrfs_uuid_tree_lookup_any from uuid-tree.c 2021-10-08 20:46:35 +02:00
libbtrfsutil btrfs-progs: remove support for BTRFS_SUBVOL_CREATE_ASYNC 2021-10-06 16:49:21 +02:00
m4 btrfs-progs: a bunch of typo fixes 2021-01-13 22:33:10 +01:00
mkfs btrfs-progs: mkfs: recow all tree blocks properly 2021-10-20 18:59:24 +02:00
tests btrfs-progs: tests: make misc/038 more robust when searching backup slots 2021-10-20 18:59:24 +02:00
.editorconfig btrfs-progs: add basic .editorconfig 2020-08-31 17:01:02 +02:00
.gitignore btrfs-progs: gitignore: update paths in Documentation 2021-10-06 16:50:57 +02:00
64-btrfs-dm.rules
autogen.sh btrfs-progs: build: simplify version tracking 2018-01-31 15:14:01 +01:00
btrfs-completion btrfs-progs: rescue: add create-control-device subcommand 2021-02-19 16:19:37 +01:00
btrfs-corrupt-block.c btrfs-progs: libbtrfs: drop radix-tree.h from exported headers 2021-10-08 20:46:35 +02:00
btrfs-crc.c btrfs-progs: move crc32c implementation to crypto/ 2019-11-18 19:20:02 +01:00
btrfs-debugfs btrfs-progs: port btrfs-debugfs to python3 2020-07-02 22:24:33 +02:00
btrfs-find-root.c btrfs-progs: factor open_ctree parameters to a structure 2021-03-24 22:20:19 +01:00
btrfs-fragments.c btrfs-progs: move ctree.c to kernel-shared/ 2020-08-31 17:01:05 +02:00
btrfs-map-logical.c btrfs-progs: libbtrfs: drop radix-tree.h from exported headers 2021-10-08 20:46:35 +02:00
btrfs-sb-mod.c btrfs-progs: sb-mod: improve help 2021-10-06 16:50:30 +02:00
btrfs-select-super.c btrfs-progs: libbtrfs: drop radix-tree.h from exported headers 2021-10-08 20:46:35 +02:00
btrfs.c btrfs-progs: sort main help menu entries 2020-12-10 17:10:30 +01:00
btrfstune.c btrfs-progs: clean up test_uuid_unique 2021-10-08 20:46:33 +02:00
CHANGES btrfs-progs: update CHANGES for 5.14.2 2021-10-07 22:00:09 +02:00
configure.ac btrfs-progs: build: capitalize variable names in the summary 2021-10-06 16:50:44 +02:00
COPYING
fsck.btrfs
INSTALL btrfs-progs: build: make libudev selectable 2021-10-06 16:50:43 +02:00
ioctl.h btrfs-progs: remove support for BTRFS_SUBVOL_CREATE_ASYNC 2021-10-06 16:49:21 +02:00
kerncompat.h btrfs-progs: kerncompat: add const_ilog2 2021-05-06 16:41:46 +02:00
libbtrfs.sym btrfs-progs: libbtrfs: hide unused symbols, same version 2021-09-08 18:17:30 +02:00
Makefile btrfs-progs: subvol list: merge the rest of btrfs-list.c 2021-10-08 20:47:03 +02:00
Makefile.extrawarn btrfs-progs: build: disable -Waddress-of-packed-member by default 2019-06-14 15:09:53 +02:00
Makefile.inc.in btrfs-progs: build: add optional dependency on libudev 2021-10-06 16:50:32 +02:00
quick-test.c btrfs-progs: move transaction.c to kernel-shared/ 2020-08-31 17:01:06 +02:00
random-test.c btrfs-progs: move transaction.c to kernel-shared/ 2020-08-31 17:01:06 +02:00
README.md btrfs-progs: README: update links 2021-07-08 14:53:16 +02:00
show-blocks btrfs-progs: Remove btrfs-debug-tree command 2018-04-24 13:00:10 +02:00
VERSION Btrfs progs v5.14.2 2021-10-08 15:30:04 +02:00
version.h.in

Btrfs-progs coverity status

Userspace utilities to manage btrfs filesystems. License: GPLv2.

Btrfs is a copy on write (COW) filesystem for Linux aimed at implementing advanced features while focusing on fault tolerance, repair and easy administration.

This repository hosts following utilities:

  • btrfs — the main administration tool (manual page)
  • mkfs.btrfs — utility to create the filesystem (manual page)
  • all-in-one binary in the busybox style with mkfs.btrfs, btrfs-image and other tools built-in (standalone tools)
  • libbtrfsutil (LGPL v2.1) — C and python 3 bindings, see libbtrfsutil/README.md for more

See INSTALL for build instructions and tests/README.md for testing information.

Release cycle

The major version releases are time-based and follow the cycle of the linux kernel releases. The cycle usually takes 2 months. A minor version releases may happen in the meantime if there are bug fixes or minor useful improvements queued.

The release tags are signed with a GPG key ID F2B4 1200 C54E FB30 380C 1756 C565 D5F9 D76D 583B, release tarballs are hosted at kernel.org. See file CHANGES or changelogs on wiki.

Reporting bugs

There are several ways, each has its own specifics and audience that can give feedback or work on a fix. The following list is sorted in the order of preference:

  • github issue tracker
  • to the mailing list linux-btrfs@vger.kernel.org -- (not required to subscribe), beware that the mail might get overlooked in other traffic
  • IRC (irc.freenode.net #btrfs) -- good for discussions eg. if a bug is already known, but reports could miss developers' attention
  • bugzilla.kernel.org -- (requires registration), set the product to Filesystems and component Btrfs, please put 'btrfs-progs' into the subject so it's clear that it's not a kernel bug report

Development

The patch submissions, development or general discussions take place at linux-btrfs@vger.kernel.org mailinglist, subsciption is not required to post.

The GitHub pull requests will not be accepted directly, the preferred way is to send patches to the mailinglist instead. You can link to a branch in any git repository if the mails do not make it to the mailinglist or just for convenience (makes it easier to test).

The development model of btrfs-progs shares a lot with the kernel model. The github way is different in some ways. We, the upstream community, expect that the patches meet some criteria (often lacking in github contributions):

  • one logical change per patch: eg. not mixing bugfixes, cleanups, features etc., sometimes it's not clear and will be usually pointed out during reviews
  • proper subject line: eg. prefix with btrfs-progs: subpart, ... , descriptive yet not too long, see git log --oneline for some inspiration
  • proper changelog: the changelogs are often missing or lacking explanation why the change was made, or how is something broken, what are user-visible effects of the bug or the fix, how does an improvement help or the intended usecase
  • the Signed-off-by line: this documents who authored the change, you can read more about the The Developer's Certificate of Origin (chapter 11)
    • if you are not used to the signed-off style, your contributions won't be rejected just because of it's missing, the Author: tag will be added as a substitute in order to allow contributions without much bothering with formalities

Source code coding style and preferences follow the kernel coding style. You can find the editor settings in .editorconfig and use the EditorConfig plugin to let your editor use that, or update your editor settings manually.

Testing

The testing documentation can be found in tests/ and continuous integration/container images in ci/.

Documentation updates

Documentation fixes or updates do not need much explanation so sticking to the code rules in the previous section is not necessary. GitHub pull requests are OK, patches could be sent to me directly and not required to be also in the mailinglist. Pointing out typos via IRC also works, although might get accidentally lost in the noise.

Third-party sources

Build dependencies are listed in INSTALL. Implementation of checksum/hash functions is provided by copies of the respective sources to avoid adding dependencies that would make deployments in rescure or limited environments harder. The implementations are portable and not optimized for speed nor accelerated. Optionally it's possible to use libgcrypt, libsodium or libkcapi implementations.

References