This is a mirror of the change I've done in the kernel, but in progs
it's even more simply because clean_tree_block was just a wrapper around
clear_extent_buffer_dirty. Change this to btrfs_clear_buffer_dirty, and
then update all the callers to use this helper instead of
clean_tree_block and clear_extent_buffer_dirty.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In the kernel we just pass in the extent_buffer and get the fields we
need from that to update the extent ref flags. Update this helper to
match the kernel to make syncing ctree.c easier.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This helper in the kernel is named btrfs_set_disk_extent_flags, which is
a more accurate description than btrfs_set_block_flags. Rename to the
in kernel name to make syncing ctree.c easier.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The following are some extent buffer helpers we have in the kernel but
not in btrfs-progs. Sync these in to make syncing ctree.c easier.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is how btrfs_alloc_tree_block is defined in the kernel, so when we
go to sync this code in it'll be easier if we're already setup to accept
this argument. Since we're in progs we don't care about nesting so just
use BTRFS_NORMAL_NESTING everywhere, as we sync in the kernel code it'll
get updated to whatever is appropriate.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We want locking.h to have all the definitions that get used throughout
the codebase, however we don't want to actually use any of the actual
locking. This sync's the bulk of locking.h, and then stubs out all of
the definitions. We need a locking.c for the root lock helpers that
return the extent buffer, but everything else can simply be inlined out.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is in keeping with what the function actually does, and is named
this way in the kernel.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We changed from members in the root for all the different flags to a
bit based flag system. In order to make syncing the kernel code into
btrfs-progs easier go ahead and sync in the bits we use and update all
the users of the old ->track_dirty and ->ref_cows to use the state bits.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These helpers are all take const struct extent_buffer in the kernel, do
the same in btrfs-progs in order to enable us to more easily sync
ctree.c.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is a bit larger than the previous syncs, because we use
extent_io_tree's everywhere. There's a lot of stuff added to
kerncompat.h, and then I went through and cleaned up all the API
changes, which were
- extent_io_tree_init takes an fs_info and an owner now.
- extent_io_tree_cleanup is now extent_io_tree_release.
- set_extent_dirty takes a gfpmask.
- clear_extent_dirty takes a cached_state.
- find_first_extent_bit takes a cached_state.
The diffstat looks insane for this, but keep in mind extent-io-tree.c
and extent-io-tree.h are ~2000 loc just by themselves.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We won't actually use the async code in progs, however we call the
helpers and such all over the normal code, so sync this into btrfs-progs
to make syncing other parts of the kernel easier.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch syncs file-item.h into btrfs-progs. This carries with it an
API change for btrfs_del_csums, which takes a root argument in the
kernel, so all callsites have been updated accordingly.
I didn't sync file-item.c because it carries with it a bunch of bio
related helpers which are difficult to adapt to the kernel.
Additionally there's a few helpers in the local copy of file-item.c that
aren't in the kernel that are required for different tools.
This requires more cleanups in both the kernel and progs in order to
sync file-item.c, so for now just do file-item.h in order to pull things
out of ctree.h.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This syncs accessors.[ch] from the kernel. For the most part
accessors.h will remain the same, there's just some helpers that need to
be adjusted for eb->data instead of eb->pages. Additionally accessors.c
needed to be completely updated to deal with this as well.
This is a set of files where we will likely only sync the header going
forward, and leave the C file in place as it needs to be specific to
btrfs-progs.
This forced a few "unrelated" changes
- Using btrfs_dir_item_ftype() instead of btrfs_dir_item_type(). This
is due to the encryption changes, and was simpler to just do in this
patch.
- Adjusting some of the print tree code to use the actual helpers and
not the btrfs-progs ones.
A local definition of static_assert is used to avoid compilation
failures on older gcc (< 9) where the 2nd parameter is mandatory.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Glibc provides an interface to extend the printf formats but this is not
standardized and does not work on musl. The code brought from kernel
uses %pV for varargs and also has own implementation of printk.
As a workaround for musl expand the pV value to a string and then
simply print it. The details are hidden behind macros:
- DECLARE_PV(vaf)
- PV_ASSIGN(vaf, format, args)
- PV_FMT in printf string
- PV_VAL in arguments
Signed-off-by: David Sterba <dsterba@suse.com>
We use the struct va_format to do nested printk's internally with our
message handling. Add the appropriate user space code to make this work
properly so when we start copying this code into btrfs-progs we get the
proper messages.
Note: this breaks build on musl, printf.h is not available.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These are the printk helpers from the kernel. There were a few
modifications, the hi-lights are
- We do not have fs_info::fs_state, so that needed to be removed.
- We do not have discard.h sync'ed yet, so that dependency was dropped.
- Anything related to struct super_block was commented out.
- The transaction abort had to be modified to fit with the current
btrfs-progs code.
- Added a btrfs_no_printk() helper to common/messages.* so that the
print statements still worked.
- The 32bit limit checkers are not needed so are behind __KERNEL__
Additionally there were kerncompat.h changes that needed to be made to
handle the dependencies properly. Those are easier to spot.
Any function that needed to be modified has a MODIFIED tag in the
comment section with a list of things that were changed.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This pulls in the kernel's uapi/btrfs_tree.h, which now has all of the
on-disk definitions. Include this into ctree.h, and then yank out all
the duplicate code from ctree.h.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We want to keep this file locally as we want to be uptodate with
upstream, so we can build btrfs-progs regardless of which kernel is
currently installed. Sync this with the upstream version and put it in
kernel-shared/uapi to maintain some semblance of where this file comes
from.
There are some changes that need to be synced back to kernel. A local
definition of static_assert is used to avoid compilation problems on gcc
(< 9) due to mandatory 2nd parameter.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Now that this is unused by these helpers and only used by the repair
related code we can remove this argument from the main helpers.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs check wants to be able to record corrupted extents if it finds any
bad blocks. This has been done directly inside of the
btrfs_check_leaf/btrfs_check_node helpers, but these are going to be
sync'ed from the kernel in the future. Add another helper and move the
corrupt block handling into this helper and keep it inside of the check
code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This can be pulled out of the extent buffer that is passed in, drop the
fs_info argument from the function.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Now that the libbtrfs stuff has it's own local copy of ctree.h and
ioctl.h, let's rename these qgroup struct members to match the kernel
names, this way it'll make it easier to sync the kernel code into
btrfs-progs.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These helpers all do variations on the same thing, so add a helper to
just do the printf part, and a macro to handle the special prefix and
postfix, and then make the helpers just use the macro and new helper.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have different helpers for warning_on and error_on(), which take the
condition and then do the printf. However we can just check the
condition in the macro and call the normal warning or error helper, so
clean this usage up and delete the unneeded message helpers.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Everybody who calls print_trace wraps it around this check, move the
check instead to print_trace and remove the check from all the callers.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
While syncing messages.[ch] I had to back out the ASSERT() code in
kerncompat.h, which means we now rely on the kernel code for ASSERT().
In order to maintain some semblance of separation introduce UASSERT()
and use that in all the purely userspace code.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Now that we're properly separated with libbtrfs/kerncompat.h and
include/kerncompat.h, go ahead and add the __init definition back so we
can have it available for the kernel synced files.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
While running make test-convert as a normal user I ran into this problem
where we do sudo find <blah> into a mktemp file that's created as the
normal user. This results in find getting a EPERM while trying to mess
with that temp file. Fix this by using $SUDO_HELPER for all the
tempfile manipulations so that root is the owner of everything, which
allows the convert tests to run as a normal user.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We're keeping a libbtrfs compatible kerncompat.h around to make it
easier to modify the rest of btrfs-progs. Unfortunately we also use
some of kernel-lib in libbtrfs, and those also include kerncompat.h.
Those getting included first means we'll pull include/kerncompat.h
instead of libbtrfs/kerncompat.h, which will mess things up.
Fix this by making sure we include our local copy of kerncompat.h first
before we include any other header in btrfs-progs.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new test case would create a fs without free space tree, then
populate it, convert to free-space-tree feature, and make sure
everything is fine.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
From the very beginning of free-space-tree feature, we allow mount
option "space_cache=v2" to convert the filesystem to the new feature.
But this is not the proper practice for new features (no matter if it's
incompat or compat_ro).
This is already making the clear_cache/space_cache mount option more
complex.
Thus this patch introduces the proper way to enable free-space-tree, and
I hope one day we can deprecate the "space_cache=" mount option.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently check/clear-cache.c still uses a lot of global variables like
gfs_info and g_task_ctx, which are only implemented in check/main.c.
Since we have separated clear-cache code into its own c and header files,
we should not utilize those global variables.
Or it would lead to compiling errors for any tools which only has
check/clear-cache.o linked:
/usr/bin/ld: check/clear-cache.o: in function `clear_free_space_cache':
/home/adam/btrfs/btrfs-progs/check/clear-cache.c:46: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:56: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:67: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:73: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:84: undefined reference to `gfs_info'
/usr/bin/ld: check/clear-cache.o:/home/adam/btrfs/btrfs-progs/check/clear-cache.c:85: more undefined references to `gfs_info' follow
/usr/bin/ld: check/clear-cache.o: in function `check_space_cache':
/home/adam/btrfs/btrfs-progs/check/clear-cache.c:357: undefined reference to `g_task_ctx'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:357: undefined reference to `g_task_ctx'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:358: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:365: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:373: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:374: undefined reference to `gfs_info'
/usr/bin/ld: /home/adam/btrfs/btrfs-progs/check/clear-cache.c:382: undefined reference to `gfs_info'
/usr/bin/ld: check/clear-cache.o:/home/adam/btrfs/btrfs-progs/check/clear-cache.c:383: more undefined references to `gfs_info' follow
collect2: error: ld returned 1 exit status
This provides the basis for later clear cache usage out of check realm.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
I have a setup where null_blk is not a module but is built-in, so to check
if the kernel supports null_blk, use 'modinfo -n'.
Also fix a comment.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
-R option is deprecated since commit 4dbe66ca2f ("btrfs-progs: mkfs: make
-R|--runtime-features option deprecated"), migrate the test case to
follow the change.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When compiling the documents, we have the following warning:
~/btrfs-progs/Documentation/dev/On-disk-format.rst:369: WARNING: Bullet list ends without a blank line; unexpected unindent.
It's caused by a mismatched indent.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add helper scripts to build static binaries manually or clean old runs.
Requires the 'gh' command line tool and write access the git repository.
Signed-off-by: David Sterba <dsterba@suse.com>
Add new CI image target based on 32bit Alpine with musl. It's a copy of
musl and requires docker's buildx extension to properly select the
platform.
Signed-off-by: David Sterba <dsterba@suse.com>
Calculate checksums for static binaries that are published as release
artifacts. Do that when they're built in the workflow so the checksum
appears in the build log and also provide the artifacts of the checksums
so they can be published along the releases.
Signed-off-by: David Sterba <dsterba@suse.com>
Rename the options so they more accurately reflect what the command is
actually doing. The feature is enabled/disabled in the end but it's not
a simple on/off like for others, the conversion takes time.
Signed-off-by: David Sterba <dsterba@suse.com>
There has been no update since 2014 and it's likely broken due to many
new features added to btrfs since then.
Signed-off-by: David Sterba <dsterba@suse.com>
- "Wipe" in storage terms is often understood as some kind of secure
deletion. Use "remove" instead in order to indicate that the space
cache is fully removed (and not just cleared and then e.g.
automatically rebuild).
- The --clear-space-cache option for btrfs check actually clears the
whole space cache, just as documented. Thus move any documentation
about the clear_cache mount option not doing so for v1 to that.
Instead, refer to the mount option.
- Also note that when clear_cache is used with v1, the free space cache
for block groups that are modified gets always cleared, but rebuilt
only if nospace_cache is not used.
Pull-request: #571
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: David Sterba <dsterba@suse.com>
Clarify active vs. inactive swapfile regarding snapshotting and further minor
improvements.
Issue: #608
Pull-request: #618
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently "btrfs filesystem du" auto-completes for directories only,
but it can also be used against files to determine shared vs exclusive
extents.
Signed-off-by: David Disseldorp <ddiss@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
