The inode-map feature is not implemented in userspace and the only
function does the ordinary free inode number lookup. Move it to inode.c
and delte inode-map.c
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new function, btrfs_create_tree(), to create an empty tree.
Currently, there is only one caller to create new tree, namely data
reloc tree in mkfs. However it's copying fs tree to create a new root.
This copy fs tree method is not a good idea if we only need an empty
tree.
So here introduce a new function, btrfs_create_tree() to create new tree.
Which will handle the following things:
1) New tree root leaf
Using generic tree allocation
2) New root item in tree root
3) Modify special tree root pointers in fs_info
Only quota_root is supported yet, but can be expended easily
This patch provides the basis to implement quota support in mkfs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ solved minor conflicts ]
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current kernel only supports qgroup version 1. Make qgroup-verify to
follow this standard.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To sync with the refactored kernel code. Also since we're here, sync
the function parameters with kernel too.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This would sync the code between kernel and btrfs-progs, and save at
least 1 byte for each btrfs_block_group_cache.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We can easily get the level from @eb parameter, thus the level is not
needed.
This is inspired by the work of Marek in U-boot.
Cc: Marek Behun <marek.behun@nic.cz>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If we're compiling using system header, we should include
<btrfs/crc32c.h> other than "btrfs/crc32.h".
Fixes: 2efe160bc7 ("btrfs-progs: move name hashing functions to ctree.h and delete hash.h")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The code using it has been removed in 4e34bdb868 ("btrfs-progs: Remove
old commented code") and at that time it wasn't in use since
95d3f20b51 ("Mixed back reference (FORWARD ROLLING FORMAT CHANGE)").
Signed-off-by: David Sterba <dsterba@suse.com>
Add BTRFS_EXTENDED_PROFILE_MASK to consider also the
BTRFS_AVAIL_ALLOC_BIT_SINGLE bit.
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
Previously, no filenames/xattrs would be printed with --nofilename, but
to keep the format of dump, print a placeholder instead of all names.
This is:
* directory entries (files, directories, subvolumes)
* default subvolume
* extended attributes (name, value)
* hardlink names if stored inside another item
Note that lengths are not hidden because they can be calculated from the
item size anyway.
Signed-off-by: David Sterba <dsterba@suse.com>
In the mail list, it's pretty common that a developer is asking dump tree
output from the reporter, it's better to protect those kind reporters by
hiding the filename if the reporter wants.
This option will skip @name/@data output for the following items:
- DIR_INDEX
- DIR_ITEM
- INODE_REF
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_fs_info::block_group_cache and the bit BLOCK_GROUP_DIRY are not
used anymore, so is the block_group_state_bits(). Remove them.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We are going to touch dirty_bgs in transaction directly, so every call
chain should pass @trans to the leaf functions.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The old style uses extent bit BLOCK_GROUP_DIRTY to mark dirty block
groups in extent cache. To replace it, add btrfs_trans_handle::dirty_bgs
and btrfs_block_group_cache::dirty_list.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To convert from existing extent_cache to plain rb_tree, add
btrfs_block_group_cache::cache_node and
btrfs_fs_info::block_group_cache_tree.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The comment about data checksum on disk_bytes is completely wrong. Sync
it with fixed kernel comment to avoid confusion.
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We access btrfs_block_group_cache::item mostly for @used and @flags.
@flags is already a dedicated member in btrfs_block_group_cache, only
@used doesn't have a dedicated member.
This patch will remove btrfs_block_group_cache::item and add
btrfs_block_group_cache::used.
It's the btrfs-progs equivalent of the following kernel patches:
btrfs: move block_group_item::used to block group
btrfs: move block_group_item::flags to block group
btrfs: remove embedded block_group_cache::item
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch does the following refactor:
- Refactor parameter from @root to @fs_info
- Refactor the large loop body into another function
Now we have a helper function, read_one_block_group(), to handle
block group cache and space info related routine.
- Refactor the return value
Even we have the code handling ret > 0 from find_first_block_group(),
it never works, as when there is no more block group,
find_first_block_group() just return -ENOENT other than 1.
This is super confusing, it's almost a mircle it even works.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The following functions are just using @root to reach fs_info:
- exclude_super_stripes
- free_excluded_extents
- add_excluded_extent
Refactor them to use fs_info directly.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add definition, crypto wrappers and support to mkfs for blake2 for
checksumming. There are 2 aliases either blake2 or blake2b.
Signed-off-by: David Sterba <dsterba@suse.com>
Add the definition to the checksum types and let mkfs accept it.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Adding this table will make extending btrfs-progs with new checksum types
easier.
Also add accessor functions to access the table fields.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add checksum type to the definition structure for a new filesystem, this
will be used in following patches.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
The old parameters, @ref_generation and @owner_objectid, are pretty
confusing when using auto-completion.
Unify the parameters as a quick fix.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
While testing snapshot deletion with dm-log-writes I saw that I was
failing the fsck sometimes when the fs was actually in the correct
state. This is because we only skip blocks on the same level of
root_item->drop_level. If the drop_level < the root level then we could
very well walk into nodes that we wouldn't actually walk into on fs
mount, because the drop_progress is further ahead in the slot of the
root. Instead only process the slots of the nodes that are above the
drop_progress key. With this patch in place we no longer improperly
fail to check fs'es that have a drop_progress set with a drop_level <
root level.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
GCC 9.1.0 will report the following error when compiling btrfs-progs:
In file included from print-tree.c:24:
ctree.h: In function 'btrfs_dev_stats_values':
ctree.h:2408:9: warning: taking address of packed member of 'struct btrfs_dev_stats_item' may result in an unaligned pointer value [-Waddress-of-packed-member]
2408 | return p->values;
| ^
[FIX]
Follow the kernel way of accessing dev stats by using
btrfs_dev_stats_value(eb, ptr, index).
So that we don't need to bother accessing the packed member.
This also unifies the helper function in kernel and btrfs-progs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a bug report of unexpected ENOSPC from btrfs-convert, issue #123.
After some debugging, even when we have enough unallocated space, we
still hit ENOSPC at btrfs_reserve_extent().
[CAUSE]
Btrfs-progs relies on chunk preallocator to make enough space for
data/metadata.
However after the introduction of delayed-ref, it's no longer reliable
to rely on btrfs_space_info::bytes_used and
btrfs_space_info::bytes_pinned to calculate used metadata space.
For a running transaction with a lot of allocated tree blocks,
btrfs_space_info::bytes_used stays its original value, and will only be
updated when running delayed ref.
This makes btrfs-progs chunk preallocator completely useless. And for
btrfs-convert/mkfs.btrfs --rootdir, if we're going to have enough
metadata to fill a metadata block group in one transaction, we will hit
ENOSPC no matter whether we have enough unallocated space.
[FIX]
This patch will introduce btrfs_space_info::bytes_reserved to track how
many space we have reserved but not yet committed to extent tree.
To support this change, this commit also introduces the following
modification:
- More comment on btrfs_space_info::bytes_*
To make code a little easier to read
- Export update_space_info() to preallocate empty data/metadata space
info for mkfs.
For mkfs, we only have a temporary fs image with SYSTEM chunk only.
Export update_space_info() so that we can preallocate empty
data/metadata space info before we start a transaction.
- Proper btrfs_space_info::bytes_reserved update
The timing is the as kernel (except we don't need to update
bytes_reserved for data extents)
* Increase bytes_reserved when call alloc_reserved_tree_block()
* Decrease bytes_reserved when running delayed refs
With the help of head->must_insert_reserved to determine whether we
need to decrease.
Issue: #123
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Switch the function to take a key instead of objectid and offset
separately, gcc 9 complains with enabled -Waddress-of-packed-member:
[CC] ctree.o
ctree.c: In function ‘btrfs_uuid_tree_lookup’:
ctree.c:3133:26: error: taking address of packed member of ‘struct btrfs_key’ may result in an unaligned pointer value [-Werror=address-of-packed-member]
3133 | btrfs_uuid_to_key(uuid, &key.objectid, &key.offset);
Signed-off-by: David Sterba <dsterba@suse.com>
GCC9 9.1.1 with -Waddress-of-packed-member warns about passing an
unaligned pointer from btrfs_dev_stats_values. It is up to the caller to
access the array properly, which does happen in print_dev_stats.
In file included from print-tree.c:24:
ctree.h: In function ‘btrfs_dev_stats_values’:
ctree.h:2408:9: warning: taking address of packed member of ‘struct btrfs_dev_stats_item’ may result in an unaligned pointer value [-Waddress-of-packed-member]
2408 | return p->values;
Drop the helper as print-tree.c is the only user and access the dev stat
values using the unaligned helper.
Signed-off-by: David Sterba <dsterba@suse.com>
BTRFS_COMPAT_EXTENT_TREE_V0 is introduced for a short time in kernel,
and it's over 10 years ago.
Nowadays there should be no user for that feature, and kernel has remove
this support in Jun, 2018. There is no need for btrfs-progs to support
it.
This patch will remove EXTENT_TREE_V0 related code and replace those
BUG_ON() to a more graceful error message.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will refactor btrfs_finish_extent_commit():
- Make it return void
There is no failure pattern for btrfs_finish_extent_commit(), thus it
always return 0. And the caller doesn't care about the return value.
So no need to return int.
- Remove @root and @unpin parameters
@root is only used to extract fs_info, which can be extracted from
transaction handler already.
@unpin is always fs_info->pinned_extents.
All these parameters can be extracted from @trans, no need to pass
them.
The function signature now matches the kernel counterpart.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For test case fsck-tests/001-bad-file-extent-bytenr, we have an
obviously hand crafted image with unaligned file extent:
item 7 key (257 EXTENT_DATA 0) itemoff 3453 itemsize 53
generation 6 type 1 (regular)
extent data disk byte 755944791 nr 1048576
extent data offset 0 nr 1048576 ram 1048576
extent compression 0 (none)
disk bytenr 755944791 is obviously unaligned (not even).
For such obviously corrupted file extent, we should just delete the file
extent.
Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
[Update commit message and comment]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Commit 0ddf63c09f ("btrfs-progs: Record orphan data extent ref to
corresponding root.") introduces the ability to record a file extent
even all other related info is lost (data backref, inode item).
However this patch only records such info without doing any proper
repair, further more, it could even record invalid file extents, and the
report part only happens after all check is done.
Since we will later introduce proper file extent repair functionality,
we could revert that patch.
Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
[Update commit message, solve merge conflicts]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Commit 75b5eabb61 ("btrfs-progs: uuid: Port
btrfs_uuid_tree_add() function") brings code from kernel and introduces
an unsatisfied build dependency for libbtrfs. This fails build for
external library users like snapper.
As a hotfix, shuffle the new UUID functions so the build works. Simply
adding uuid-tree.o to libbtrfs_objects does not work due to other
missing symbols.
[CC] uuid-tree.o
[LD] libbtrfs.so.0.1
[LN] libbtrfs.so
[TEST PREP] library-test
ld: .../libbtrfs.so: undefined reference to `btrfs_alloc_path'
ld: .../libbtrfs.so: undefined reference to `btrfs_extend_item'
ld: .../libbtrfs.so: undefined reference to `btrfs_insert_empty_items'
ld: .../libbtrfs.so: undefined reference to `btrfs_free_path'
ld: .../libbtrfs.so: undefined reference to `read_extent_buffer'
ld: .../libbtrfs.so: undefined reference to `btrfs_mark_buffer_dirty'
ld: .../libbtrfs.so: undefined reference to `write_extent_buffer'
ld: .../libbtrfs.so: undefined reference to `btrfs_search_slot'
LinK: https://bugs.gentoo.org/675974
Link: https://github.com/openSUSE/snapper/issues/473
Fixes: 75b5eabb61 ("btrfs-progs: uuid: Port btrfs_uuid_tree_add() function")
Signed-off-by: David Sterba <dsterba@suse.com>
This function provides the offline functionality to add new uuid tree
entry. Also port fs_info->uuid and its initialization and cleanup code
to support uuid tree.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For DIR_ITEM with mismatch hash, we could just remove the offending dir
item from the tree.
Lowmem mode will handle the rest, either re-create the correct dir_item
or move the orphan inode to lost+found.
This is especially important for old filesystems, since later kernel
introduces stricter tree-checker, which could detect such hash mismatch
and refuse to read the corrupted leaf.
With this repair ability, user could repair with 'btrfs check
--mode=lowmem --repair'.
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1111991
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This member was used only by btrfstune when using the old method to
change fsid. It's only an in-memory value with a very specific
purpose so it makes no sense to pollute a generic structure such as
btrfs_fs_info with it. Just remove it and pass it as a function
argument where pertinent. No functional changes.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This allows us to change the use-visible UUID on filesytems from
userspace if desired, by copying the existing UUID to the new location
for metadata comparisons. If this is done, an incompat flag must be
set to prevent older filesystems from mounting the filesystem, but
the original UUID can be restored, and the incompat flag removed.
This introduces the new -m|-M UUID options similar to current
-u|-U UUID ones with the difference that we don't rewrite the fsid but
just copy the old uuid and set a new one. Additionally running with
[-M old-uuid] clears the incompat flag and retains only fsid on-disk.
Additionally it's not allowed to intermix -m/-u/-U/-M options in a
single invocation of btrfstune, nor is it allowed to change the uuid
while there is a uuid rewrite in-progress. Also changing the uuid of a
seed device is not currently allowed (can change in the future).
Example:
btrfstune -m /dev/loop1
btrfs inspect-internal dump-super /dev/loop1
superblock: bytenr=65536, device=/dev/loop1
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0x4b7ea749 [match]
<ommitted for brevity>
fsid 0efc41d3-4451-49f3-8108-7b8bdbcf5ae8
metadata_uuid 352715e7-62cf-4ae0-92ee-85a574adc318
<ommitted for brevity>
incompat_flags 0x541
( MIXED_BACKREF |
EXTENDED_IREF |
SKINNY_METADATA |
METADATA_UUID )
<omitted for brevity>
dev_item.uuid 0610deee-dfc3-498b-9449-a06533cdec98
dev_item.fsid 352715e7-62cf-4ae0-92ee-85a574adc318 [match]
<ommitted for brevity>
mount /dev/loop1 btrfs-mnt/
btrfs fi show btrfs-mnt/
Label: none uuid: 0efc41d3-4451-49f3-8108-7b8bdbcf5ae8
Total devices 1 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 536.00MiB path /dev/loop1
In this case a new btrfs filesystem was created and the original uuid
was 352715e7-62cf-4ae0-92ee-85a574adc318, then btrfstune was run which
copied that value over to metadata_uuid field and set the current fsid
to 0efc41d3-4451-49f3-8108-7b8bdbcf5ae8. And as far as userspace is
concerned this is the fsid of the fs.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for a new metadata_uuid field. This is just a preparatory
commit which switches all users of the fsid field for metdata comparison
purposes to utilize the new field. This more or less mirrors the
kernel patch, additionally:
* Update 'btrfs inspect-internal dump-super' to account for the new
field. This involes introducing the 'metadata_uuid' line to the
output and updating the logic for comparing the fs uuid to the
dev_item uuid.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba
Qu Wenruo
Goffredo Baroncelli
Adam Borowski
Su Yue
Johannes Thumshirn
Josef Bacik
Su Yanjun
Nikolay Borisov
Josh Soref