Fixes so 'python3 -m build' works and package can be uploaded to pypi
(https://pypi.org/project/btrfsutil/).
- setup.py is still used for local build (make)
- for pypi it must be done by 'python3 -m build' that is build in a
temporary directory
- btrfsutilpy.h must be also distributed
- version is set manually (the git VERSION file is not accessible)
- the project page metadata is empty, the README.md should be added
Issue: #310
Signed-off-by: David Sterba <dsterba@suse.com>
Recent changes to the python code reworked path_converter() so that it
does not use fd_converter() anymore. Assuming it may be used in the
future again comment it out.
Signed-off-by: David Sterba <dsterba@suse.com>
Python 3.13, currently in beta, removed the internal
_PyObject_LookupSpecial function. The libbtrfsutil Python bindings use
it in the path_converter() function because it was based on internal
path_converter() function in CPython [1]. This is causing build failures
on Fedora Rawhide [2] and Gentoo [3]. Replace path_converter() with a
version that only uses public functions based on the one in drgn [4].
1: d9efa45d74/Modules/posixmodule.c (L1253)
2: https://bugzilla.redhat.com/show_bug.cgi?id=2245650
3: https://github.com/kdave/btrfs-progs/issues/838
4: 9ad29fd864/libdrgn/python/util.c (L81)
Issue: #838
Reported-by: Neal Gompa <neal@gompa.dev>
Reported-by: Sam James <sam@gentoo.org>
Reviewed-by: Neal Gompa <neal@gompa.dev>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For an unprivileged subvolume iterator, append_to_search_stack() closes
cur_fd. On the first call to btrfs_util_subvolume_iterator_next(),
cur_fd is equal to the fd that was passed to
btrfs_util_create_subvolume_iterator_fd(). We're not supposed to close
that. We didn't notice it because it's more common to use it through
btrfs_util_create_subvolume_iterator(), which opens its own fd that
should be closed, and because the fd number is often reused internally
by the subvolume iterator.
pop_search_stack() already has a check to avoid closing the passed fd;
add the same check to append_to_search_stack(). Also add a regression
test.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The search header is used for extracting data from buffer returned by
the SEARCH_TREE ioctl and needs special access helpers as there are no
guarantees about alignment.
With -fsanitize=alignment this still leads to an error because address
of the members is taken, regardless of the unaligned access method is
used (both temporary memcpy to a structure or the packed struct cast).
Add another hint to compiler that the structure is special and add the
packed attribute. This fixes the sanitizer error.
Signed-off-by: David Sterba <dsterba@suse.com>
There's a lot of places with unsafe access to data that come from a
search buffer, which is packed and the structures there are not
guaranteed to be aligned, also accessing the on-disk format structures.
- search header - this is an in-memory buffer with a series of on-disk
structures, no alignment must be assumed
- anything that's not a byte buffer must be accessed as an unaligned
buffer (the exceptions are name-like buffers)
Signed-off-by: David Sterba <dsterba@suse.com>
We will need generic helpers for unaligned access with LE->CPU
conversion, so add them. Should be use for potentially unaligned read
from tree search buffer.
Signed-off-by: David Sterba <dsterba@suse.com>
Reported by 'gcc -fanalyzer':
libbtrfsutil/subvolume.c:415:20: warning: dereference of NULL ‘subvol’ [CWE-476] [-Wanalyzer-null-dereference]
The analyzer found a path where the NULL pointer passed as argument to
btrfs_util_subvolume_info_fd() could be dereferenced. This is unlikely
unless there's a corruption on the disk as the header->offset would have
to be 0. Pass a valid temporary buffer so this does not happen but
there's no use of it otherwise.
Signed-off-by: David Sterba <dsterba@suse.com>
Bit shifts should be done on unsigned type as a matter of good practice
to avoid any problems with bit overflowing to the sign bit.
Signed-off-by: David Sterba <dsterba@suse.com>
The initial version of libbtrfsutil did not follow a unified naming
scheme that's usually used for libraries like those provide by
util-linux. Add aliases that are "btrfs_util_" + object + action +
suffix.
The library version changes to 1.3 but there's no new functionality,
only the aliases added. New functions can be added in the future without
possible confusion when the same action could apply to different
objects.
Issue: #574
Signed-off-by: David Sterba <dsterba@suse.com>
The wiki has been archived so remove the links from manual page
footers. Also replace the wiki link by RTD site in configure and
libbtrfsutil.
Signed-off-by: David Sterba <dsterba@suse.com>
We return __u16 in the kernel, as this is actually the size of
btrfs_qgroup_level. Adjust the existing helpers and update all the
callers to deal with the new size appropriately. This will make syncing
the kernel code cleaner.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The tool IWYU (include what you use) suggests to remove and add some
includes. Update the includes of implementation files only.
Signed-off-by: David Sterba <dsterba@suse.com>
Add the on disk definitions for the block group tree. This will be part
of the super block so we need to add the appropriate helpers to the
super block, as well as adding it to the backup roots.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are missing incompat bits in the util headers:
- metadata_uuid
- raid1c34
- zoned
This does not change library ABI, no version change needed.
Signed-off-by: David Sterba <dsterba@suse.com>
Kernel has removed support for this feature in 5.7 so let's remove
support from progs as well.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Subvolume iteration has a window between when we get a root ref (with
BTRFS_IOC_TREE_SEARCH or BTRFS_IOC_GET_SUBVOL_ROOTREF) and when we look
up the path of the parent directory (with BTRFS_IOC_INO_LOOKUP{,_USER}).
If the subvolume is moved or deleted and its old parent directory is
deleted during that window, then BTRFS_IOC_INO_LOOKUP{,_USER} will fail
with ENOENT. The iteration will then fail with ENOENT as well.
We originally encountered this bug with an application that called
`btrfs subvolume show` (which iterates subvolumes to find snapshots) in
parallel with other threads creating and deleting subvolumes. It can be
reproduced almost instantly with the included test cases.
Subvolume iteration should be robust against concurrent modifications to
subvolumes. So, if a subvolume's parent directory no longer exists, just
skip the subvolume, as it must have been deleted or moved elsewhere.
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The user transaction ioctls have been removed in kernel 4.17 by commit
7a5a07a81062 ("btrfs: Remove userspace transaction ioctls"), the
definitions are not relevant and can be removed.
The numbers could be reused in the future, eg. when there are no
maintained LTS kernels older than 4.19.
Signed-off-by: David Sterba <dsterba@suse.com>
In file test_filesystem.py the class name should be TestFilesystem, this
looks like a typo and does not affect functionality.
Signed-off-by: David Sterba <dsterba@suse.com>
This relicenses the libbtrfsutil library to LGPLv2.1+ from LGPLv3.
People that have contributed non-trivial changes acknowledged the change
and are listed below.
There's a potential licensing conflict with the 'btrfs' utility that is
GPLv2 and statically links libbtrfsutil, this is not a valid combination
per the compatibility matrix as found in
https://www.gnu.org/licenses/gpl-faq.html#AllCompatibility or
http://gplv3.fsf.org/dd3-faq .
We also have an explicit request to change the license [1] (issue #323)
from LGPLv3 to allow use in environments that don't like GPLv3. Though
the library license is not GPLv3, the full text of the license is in the
repository and the 'lesser' part is an addendum. This was perhaps a bit
confusing, nevertheless this gets clarified as well.
[1] https://lore.kernel.org/linux-btrfs/b927ca28-e280-4d79-184f-b72867dbdaa8@denx.de/
Acked-by: Omar Sandoval <osandov@fb.com>
Acked-by: Misono Tomhiro <misono.tomohiro@jp.fujitsu.com>
Acked-by: Qu Wenruo <wqu@suse.com>
Acked-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Acked-by: Anand Jain <anand.jain@oracle.com>
Acked-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
Link: https://bugs.debian.org/985400
Issue: #323
Signed-off-by: Neal Gompa <ngompa@fedoraproject.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Neal points out that the python bindings of libbtrfsutil version is set
from the library ABI rather than the package itself. As this brings some
confusion to packaging, derive the verion the main package too.
Link: http://lore.kernel.org/linux-btrfs/CAEg-Je-VLz9zZOKEVa+x0V+dpyojtRcjBw7maO73zpmowdOyTQ@mail.gmail.com
Reported-by: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add pkg-config file for libbtrfsutil which can be used to set up
compilation and link flags.
The paths in pc file depend on prefix variable but ignore DESTDIR.
DESTDIR is used for packaging and it should not affect the paths in pc
file.
Issue: #285
Signed-off-by: Sheng Mao <shngmao@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_util_error btrfs_util_delete_subvolume requires elevated
privileges, despite the fact that the documentation does not say that.
The following does not work in all scenarios:
$ btrfs subvolume create ~/mytest
$ btrfs subvolume delete ~/mytest
Document the additional requirements.
Pull-request: #253
Issue: #252
Author: cezarmathe <me@cezarmathe.com>
Signed-off-by: David Sterba <dsterba@suse.com>
- complete the function btrfs_err_str adding some missing cases
- sync the enum btrfs_err_code (in libbtrfsutil/btrfs.h) with the
rest of the codes (user space and kernel space).
- add missing fields to btrfs_raid_array[] for raid1c[34]
Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new ioctl and helpers to allow extended arguments to be passed to
subvolume deletion ioctl. The parent_fs argument should be a mount
point.
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Update the args structure, add the flags constant and the ioctl magic
number.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When compiling btrfs-progs with libbtrfsutil on a python3.8 system, we
got the following warning:
subvolume.c:636:2: warning: initialization of ‘long int’ from ‘void *’ makes integer from pointer without a cast [-Wint-conversion]
636 | NULL, /* tp_print */
| ^~~~
subvolume.c:636:2: note: (near initialization for ‘SubvolumeIterator_type.tp_vectorcall_offset’)
[CAUSE]
C definition of PyTypeObject changed in python 3.8.
Now at the old tp_print, we have tp_vectorcall_offset.
So we got above warning.
[FIX]
C has designated initialization, which can assign values to each named
member, without hard coding to match the offset.
And all the other uninitialized values will be set to 0, so we can save
a lot of unneeded "= 0" or "= NULL" lines.
Just use that awesome feature to avoid any future breakage.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When compiling btrfs-progs with libbtrfsutil on a python3.8 system, we
got the following warning:
qgroup.c:110:2: warning: initialization of ‘long int’ from ‘void *’ makes integer from pointer without a cast [-Wint-conversion]
110 | NULL, /* tp_print */
| ^~~~
qgroup.c:110:2: note: (near initialization for ‘QgroupInherit_type.tp_vectorcall_offset’)
[CAUSE]
C definition of PyTypeObject changed in python 3.8.
Now at the old tp_print, we have tp_vectorcall_offset.
So we got above warning.
[FIX]
C has designated initialization, which can assign values to each named
member, without hard coding to match the offset.
And all the other uninitialized values will be set to 0, so we can save
a lot of unneeded "= 0" or "= NULL" lines.
Just use that awesome feature to avoid any future breakage.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When compiling btrfs-progs with libbtrfsutil on a python3.8 system, we
got the following warning:
error.c:169:2: warning: initialization of ‘long int’ from ‘void *’ makes integer from pointer without a cast [-Wint-conversion]
169 | NULL, /* tp_print */
| ^~~~
error.c:169:2: note: (near initialization for ‘BtrfsUtilError_type.tp_vectorcall_offset’)
[CAUSE]
C definition of PyTypeObject changed in python 3.8.
Now at the old tp_print, we have tp_vectorcall_offset.
So we got above warning.
[FIX]
C has designated initialization, which can assign values to each named
member, without hard coding to match the offset.
Also, uninitialized values will be 0, so we can also save a lot of
unneeded "= 0" or "= NULL" lines.
Just use that awesome feature to avoid any future breakage.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The BTRFS_DEV_REPLACE_ITEM_STATE_x series defines as shown in [1] are
unused in both kernel and btrfs-progs.
[1]
btrfs.h:#define BTRFS_IOCTL_DEV_REPLACE_STATE_FINISHED 2
btrfs.h:#define BTRFS_IOCTL_DEV_REPLACE_STATE_CANCELED 3
btrfs.h:#define BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED 4
Further the BTRFS_DEV_REPLACE_ITEM_STATE_x values are different form its
counterpart BTRFS_IOCTL_DEV_REPLACE_STATE_x series as shown in [2].
[2]
btrfs_tree.h:#define BTRFS_DEV_REPLACE_ITEM_STATE_SUSPENDED 2
btrfs_tree.h:#define BTRFS_DEV_REPLACE_ITEM_STATE_FINISHED 3
btrfs_tree.h:#define BTRFS_DEV_REPLACE_ITEM_STATE_CANCELED 4
So this patch deletes the BTRFS_DEV_REPLACE_ITEM_STATE_x altogether,
they're not used by anything.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test cli-tests/008-subvolume-get-set-default fails when compiled
with 'D=ubsan', the access to search header items does not follow the
type alignment, so use the accessors.
The error:
subvolume get-default: default id is not 256, but
libbtrfsutil/subvolume.c:361:13: runtime error: member access within
misaligned address 0x7ffc147e4b6f for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment
Note that using the accessors does not fix the ubsan warning, as it
warns on taking the address of a member whose _base_ type is unaligned,
ie. it's the 'sh'.
Fixing that would need to play tricks with pointers to do &sh->type
manually, but to avoid triggering ubsan.
Signed-off-by: David Sterba <dsterba@suse.com>
Add helpers that do proper unaligned access of search heade items. This
is done in the non-libbtrfsutil code already, use the same helpers here
too. We can't use the get_unaligned_* helpers that are defined in
kerncompat, so use plain memcpy that will work everywhere.
Signed-off-by: David Sterba <dsterba@suse.com>
The caller owns the fd passed to btrfs_util_subvolume_id_fd(), so we
shouldn't close it on error. Fix it, add a regression test, and bump the
library patch version.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
I apparently didn't test this on a pre-4.18 kernel.
test_subvolume_info_unprivileged() checks for an ENOTTY, but this
doesn't seem to work correctly with subTest().
test_subvolume_iterator_unprivileged() doesn't have a check at all. Add
an explicit check to both before doing the actual test.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfsutil.h and the Python docstrings are thorough, but I've gotten a
couple of requests for a high-level overview of the available interfaces
and example usages. Add them to README.md.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba
Omar Sandoval
Qu Wenruo
Josef Bacik
Nikolay Borisov
er888kh
Neal Gompa
Sheng Mao
cezarmathe
Goffredo Baroncelli
Marcos Paulo de Souza
Zygo Blaxell
Anand Jain
Josh Soref