Commit Graph

7024 Commits

Author SHA1 Message Date
Qu Wenruo
5f87b467a9 btrfs-progs: subvolume: output the prompt line only when the ioctl succeeded
[BUG]
With the latest kernel patch to reject invalid qgroupids in
btrfs_qgroup_inherit structure, "btrfs subvolume create" or "btrfs
subvolume snapshot" can lead to the following output:

 # mkfs.btrfs -O quota -f $dev
 # mount $dev $mnt
 # btrfs subvolume create -i 2/0 $mnt/subv1
 Create subvolume '/mnt/btrfs/subv1'
 ERROR: cannot create subvolume: No such file or directory

The "btrfs subvolume" command output the first line, seemingly to
indicate a successful subvolume creation, then followed by an error
message.

This can be a little confusing on whether if the subvolume is created or
not.

[FIX]
Fix the output by only outputting the regular line if the ioctl
succeeded.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-03-04 22:55:07 +01:00
Qu Wenruo
4da5f22b23 btrfs-progs: qgroups: remove support for num_ref_copies/num_excl_copies
Remove btrfs_qgroup_inherit_add_copy() and the command line interface.

This was designed to add a pair of source/destination qgroups into
btrfs_qgroup_inherit structure, so that rfer/excl numbers would be
copied from the source qgroup into the destination one.

This behavior has been intentionally hidden since 2016, as such copy will
cause qgroup inconsistent immediately and a rescan would reset whatever
numbers copied anyway.

Now we're going to reject the copy behavior from kernel, there is no
need to keep those hidden (and already disabled for "subvolume create")
case.

Remove btrfs_qgroup_inherit_add_copy() call, and cleanup the
undocumented options.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-03-04 22:55:07 +01:00
Martin
aba8adabcc btrfs-progs: docs/conf.py: enable navigation_with_keys on RTD
Feature from https://github.com/sphinx-doc/sphinx/pull/2064 enable
navigation to be able to navigate documentation using the arrow keys.

Pull-request: #754
Author: Martin <spleefer90@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-29 12:46:55 +01:00
David Sterba
3c8200302b btrfs-progs: rename btrfs_open_fd2() to btrfs_open_path()
Use a more descriptive name, the interface is generic so it should use
the generic term for file/directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:59:29 +01:00
David Sterba
7b9a3669e1 btrfs-progs: drop _fd from btrfs_open_file_or_dir_fd()
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:58:27 +01:00
David Sterba
1669446fd9 btrfs-progs: drop _fd from btrfs_open_mnt_fd()
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:57:50 +01:00
David Sterba
590bcf20bf btrfs-progs: drop _fd from btrfs_open_dir_fd()
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:56:24 +01:00
David Sterba
52ea295038 btrfs-progs: use wrapper btrfs_open_dir_fd() in btrfs_open_mnt_fd()
The arguments now match the wrapper, use it.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:51:21 +01:00
David Sterba
221114c36f btrfs-progs: drop verbosity parameter from btrfs_open_fd2()
All calls now pass true, drop the parameter.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:47:46 +01:00
David Sterba
9b5f104558 btrfs-progs: always print error messages from btrfs_open_fd2()
There are some cases that disable verbosity (of errors) and then print
own message. Enable the verbose error messages printed by
btrfs_open_fd2() as they are specific.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:47:39 +01:00
David Sterba
3fbcce4544 btrfs-progs: drop verbosity parameter from btrfs_open_mnt_fd()
All calls now pass true, drop the parameter.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:47:35 +01:00
David Sterba
9f9b5028d5 btrfs-progs: scrub start: use global verbosity options
The code in scrub predates the global verbosity options and sets its own
quiet status. This is still used only for error messages that should be
printed even with -q. Drop that or replace with bconf.verbose status
check.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:30:44 +01:00
David Sterba
58b1c4de03 btrfs-progs: use btrfs_open_file_or_dir_fd for error verbosity
There are many places that pass false as verbosity argument and then
print an error message, or don't print any message in error cases.
Use btrfs_open_file_or_dir_fd() that will be verbose in case of an error
with the same semantics.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:14 +01:00
David Sterba
75522225f7 btrfs-progs: switch open helper functions to return negative errno
It's commonly used elsewhere in the code to return the -errno values if
possible, do that for the open helpers too.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:14 +01:00
Goffredo Baroncelli
67a0cfd914 btrfs-progs: remove unused opening functions
Remove the following unused functions:

- btrfs_open_dir()
- open_file_or_dir()
- btrfs_open_file_or_dir()
- btrfs_open()
- open_path_or_dev_mnt()
- open_file_or_dir3()
- close_file_or_dir()

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:14 +01:00
Goffredo Baroncelli
89867394e5 btrfs-progs: remove open_file_or_dir3 from du_add_file
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return
only the fd.

Replace the last reference to btrfs_open_file_or_dir3() with
btrfs_open_fd2().

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:14 +01:00
Goffredo Baroncelli
6a08424a50 btrfs-progs: replace open_file_or_dir with btrfs_open_fd2
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return
only the fd.

Replace open_file_or_dir() with btrfs_open_fd2() removing any reference
to the unused/useless dirstream variables.  btrfs_open_fd2() is required
to avoid spurious error messages.

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:14 +01:00
Goffredo Baroncelli
cafef41840 btrfs-progs: replace btrfs_open_file_or_dir with btrfs_open_file_or_dir_fd
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return
only the fd.

Replace btrfs_open_file_or_dir() with btrfs_open_file_or_dir_fd()
removing any references to the unused/useless dirstream variables.

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:14 +01:00
Goffredo Baroncelli
50770da81e btrfs-progs: replace open_file_or_dir3 with btrfs_open_fd2
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return
only the fd.

Replace open_file_or_dir3() with btrfs_open_fd2() removing any reference
to the unused/useless dirstream variables.  btrfs_open_fd2() is needed
because sometime the callers need to set the RDONLY/RDWRITE mode, and to
avoid spurious messages.

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 12:19:09 +01:00
Goffredo Baroncelli
900f21b4f1 btrfs-progs: replace open_path_or_dev_mnt with btrfs_open_mnt_fd
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return
only the fd.

Replace open_path_or_dev_mnt() with btrfs_open_mnt_fd() removing
any reference to the unused/useless dirstream variables.

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 11:40:08 +01:00
Goffredo Baroncelli
43dbe63dba btrfs-progs: replace btrfs_open_dir with btrfs_open_dir_fd
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return
only the fd.

Replace the last btrfs_open_dir() call with btrfs_open_dir_fd()
removing any reference to the unused/useless dirstream variables.

Also update the add_seen_fsid() function removing any reference to dir
stream (again this is never used).

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 11:40:08 +01:00
Goffredo Baroncelli
372c79e7ad btrfs-progs: replace btrfs_open_dir with btrfs_open_dir_fd
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

Replace btrfs_open_dir() with btrfs_open_dir_fd() removing
any reference to the unused/useless dirstream variables.
Calling btrfs_open_dir_fd() with only the path is equivalent to
btrfs_open_dir(_, _, 1).

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 11:40:08 +01:00
Goffredo Baroncelli
efbe4ab8e2 btrfs-progs: add path opening helpers without dirstream
For historical reasons the helpers [btrfs_]open_dir... return also
the 'DIR *dirstream' value when a directory is opened.

However this is never used. So avoid calling diropen() and return only
the fd.  This is a preparatory patch.

Signed-off-by: Goffredo Baroncelli <kreijack@libero.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 11:40:08 +01:00
David Sterba
f5db48e053 btrfs-progs: docs: clarify potential problems with convert
Add a note summarizing the problems.

[ci skip]

Issue: #257
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 10:52:08 +01:00
David Sterba
93cdc1c2fc btrfs-progs: docs: clarify inode numbers
Update wording and add an example.

[ci skip]

Issue: #729
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 10:26:09 +01:00
David Sterba
d03594b031 btrfs-progs: fix exclusive op enqueue timeout
There's a report that 'btrfs balance start --enqueue' does not properly
wait when there are multiple instances started. The command does a busy
wait instead of timeouts.

Strace output:

     0.000006 pselect6(5, NULL, NULL, [4], {tv_sec=60, tv_nsec=0}, NULL) = 1 (except [4], left {tv_sec=59, tv_nsec=999999716})
     0.000008 pselect6(5, NULL, NULL, [4], {tv_sec=29, tv_nsec=999999000}, NULL) = 1 (except [4], left {tv_sec=29, tv_nsec=999998786})

After the first select there's almost the entire time left, the second
one starts right after it.

Polling/selecting sysfs files is possible under some conditions:

- the file descriptor must be reopened before each poll/select
- the whole buffer must be read too

With that in place it now works as expected. The remaining timeout logic
is slightly adjusted to wait at most 10 seconds so the pending jobs do
not wait too long if there's still a lot of time left from the first
select.

Issue: #746
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-20 09:56:00 +01:00
David Sterba
06d0ed483d btrfs-progs: docs: update fstests requirements
Add more packages providing some utilities and add new DM targests.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-19 13:03:53 +01:00
David Sterba
fc9753e743 btrfs-progs: docs: add plots of develoment stats
Add visuals of the tabulated stats.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-16 12:29:34 +01:00
David Sterba
f463c0dcc1 btrfs-progs: docs: use manref role for all manual page references
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-16 09:38:16 +01:00
David Sterba
831a23616a btrfs-progs: docs: add role for manual page references
Add a new role to render manual page references and also link to the web
pages on man7.org.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-16 09:37:09 +01:00
David Sterba
60abf7a907
Btrfs progs v6.7.1
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-14 07:39:07 +01:00
David Sterba
0294b7fe6d btrfs-progs: update CHANGES for 6.7.1
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-14 07:38:45 +01:00
David Sterba
570d1c3cae btrfs-progs: docs: fix warnings and links to duplicated labels
Use the new docref and duplabel syntax to fix build warnings and fix the
link targets in the pages.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 18:24:19 +01:00
David Sterba
2700ab22d4 btrfs-progs: docs: add document and label reference extensions
Sphinx/RST does not(?) have native support for cross references to
labels in specific documents (doc, ref but not both at the same time),
also allowing duplicate labels. We have web and manual pages and to
share the same text there are common files included from each, defining
labels. This leads to warnings and clicking the links ends up in the
included document (like ch-volume-management-intro.rst) instead of the
parent.

As a last resort, implement own role that does the doc and ref in one
go. A special directive is used to define a label that is expected
to be duplicate (but otherwise it's an ordinary label) and this gets rid
of the warning too.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 18:18:29 +01:00
David Sterba
d80dc96460 btrfs-progs: docs: status page updates
Subpage promoted to 'OK', no serious problems since 6.0. Otherwise
references, adjustments, wording updates.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:41 +01:00
David Sterba
c0677a6a71 btrfs-progs: mkfs: update descriptions of -O/--features
Minor wording and clarifications for 'mkfs.btrfs -O list-all'.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:39 +01:00
David Sterba
3d1a3c8c5a btrfs-progs: mkfs: print incompat and runtime features on one line
We've deprecated the -R option and runtime feature distinction, there's
only one option -O recommended so let it also print on the same line.
Incompat/compat/runtime status of a feature shall be consulted with the
documentation.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:37 +01:00
David Sterba
e406abb157 btrfs-progs: mkfs: warn if page does not match sectorsize
Be verbose about the potential compatibility problems with the
sectorsize and page size. Also print the page size on the overview.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:33 +01:00
David Sterba
42f98fcc88 btrfs-progs: docs: recalculate checksumming performance
There are new backends and builtin accelerated implementations.
Recalculate the table results, a different host than before, with SHANI
extension.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-13 07:15:31 +01:00
David Sterba
152cd4de97 btrfs-progs: docs: add kernel changelogs
Add 6.6 and 6.7 changes, mention the subpage vs zoned limitation found
recently.

[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-12 15:52:49 +01:00
Max-Julian Pogner
0f165240b5 btrfs-progs: docs: typo in btrfs subvolume create
"desinations" => "destinations"

[ci skip]

Pull-request: #743
Signed-off-by: Max-Julian Pogner <max-julian@pogner.at>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-12 11:59:42 +01:00
David Sterba
bec6bc8eee btrfs-progs: minor source sync with kernel 6.8-rc3
Sync a few more file on the source level with kernel 6.8-rc3, no
functional changes.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 09:30:16 +01:00
LiuYan
8e6f651421 btrfs-progs: README: fix typo in changelog link
Pull-request: #740
Author: LiuYan <lovetide@qq.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 08:30:37 +01:00
Qu Wenruo
e54514aaea btrfs-progs: fix stray fd close in open_ctree_fs_info()
[BUG]
Although commit b2a1be83b8 ("btrfs-progs: mkfs: keep file descriptors
open during whole time") is making sure we're only closing the writeable
fds after the fs is properly created, there is still a missing fd not
following the requirement.

And this explains the issue why sometimes after mkfs.btrfs, lsblk still
doesn't give a valid uuid.

Shown by the strace output (the command is "mkfs.btrfs -f
/dev/test/scratch1"):

  openat(AT_FDCWD, "/dev/test/scratch1", O_RDWR) = 5 <<< Writeable open
  fadvise64(5, 0, 0, POSIX_FADV_DONTNEED) = 0
  sysinfo({uptime=2529, loads=[8704, 6272, 2496], totalram=4104548352, freeram=3376611328, sharedram=9211904, bufferram=43016192, totalswap=3221221376, freeswap=3221221376, procs=190, totalhigh=0, freehigh=0, mem_unit=1}) = 0
  lseek(5, 0, SEEK_END)                   = 10737418240
  lseek(5, 0, SEEK_SET)                   = 0
  ......
  close(5)                                = 0 <<< Closed now
  pwrite64(6, "O\250\22\261\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1163264) = 16384
  pwrite64(6, "\201\316\272\342\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1179648) = 16384
  pwrite64(6, "K}S\t\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1196032) = 16384
  pwrite64(6, "\207j$\265\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 1212416) = 16384
  pwrite64(6, "q\267;\336\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 16384, 5242880) = 16384
  fsync(6) <<< But we're still writing into the disk.

[CAUSE]
After more digging, it turns out we have a very obvious escape in
open_ctree_fs_info():

	open_ctree_fs_info()
	|- fp = open(oca->filename, flags);
	|- info = __open_ctree_fd();
	|- close(fp);

As later we only do IO using the device fd, this close() seems fine.

But the truth is, for mkfs usage, this fs_info is a temporary one, with
a special magic number for the disk.  And since mkfs is doing writeable
operations, this close() would immediately trigger udev scan.

And since at this stage, the fs is not yet fully created, udev can race
with mkfs, and may get the invalid temporary superblock.

[FIX]
Introduce a new btrfs_fs_info member, initial_fd, for
open_ctree_fs_info() to record the fd.

And on close_ctree(), if we find fs_info::initial_fd is a valid fd, then
close it.

By this, we make sure all writeable fds are only closed after we have
written valid super blocks into the disk.

Issue: #734
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 08:30:37 +01:00
Qu Wenruo
ace7d241cc btrfs-progs: tune: fix the missing close() of filesystem fd
[BUG]
In "btrfs tune" subcommand, we're using open_ctree_fd(), which
requires passing a valid fd, and the caller is responsible to properly
handling the lifespan of the fd.

But we just call close_ctree() and btrfs_close_all_devices(), without
closing the fd.

[FIX]
Just manually close the fd.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 08:30:37 +01:00
Qu Wenruo
da1d497a16 btrfs-progs: rescue: properly close the fs for clear-ino-cache
[BUG]
In cmd_rescue_clear_ino_cache(), we opened the fs, but without
closing it using close_ctree().

[CAUSE]

This was introduced in 42404a4e44 ("btrfs-progs: move inode cache
removal to rescue group"), the original code inside btrfs check
had a "goto out_close;" to properly close the fs.

[FIX]
Manually call close_ctree() on the fs_info->tree_root.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 08:30:37 +01:00
David Sterba
fed3806d08 btrfs-progs: remove unused BTRFS_UPDATE_KERNEL
The symbol BTRFS_UPDATE_KERNEL seems to be unused since 2f55fd7019
("btrfs-progs: optimize btrfs_scan_lblkid() for multiple calls"), remove
it.

Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 08:30:37 +01:00
David Sterba
1601cd0d4f btrfs-progs: convert defined constants to enums
Signed-off-by: David Sterba <dsterba@suse.com>
2024-02-08 08:30:37 +01:00
David Sterba
a763179798 btrfs-progs: ci: add script to delete runs of a branch
[ci skip]

Signed-off-by: David Sterba <dsterba@suse.com>
2024-01-31 18:47:29 +01:00
David Sterba
328e3e632c btrfs-progs: ci: save test logs to artifacts on failure
Wherever it makes sense, save the logs as artifacts when something
fails (most likely the main step with real tests).

Signed-off-by: David Sterba <dsterba@suse.com>
2024-01-31 18:35:31 +01:00