There were two major problems with add_file_items(): it was
writing all files sector-by-sector, making compression impossible, and
it was assuming that pread would never do a short read.
Fix these problems, and create a new helper add_file_item_extent().
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Just like insert_reserved_file_extent() from the kernel, we can make
btrfs_insert_file_extent() accept an on-stack file extent item
directly.
This makes btrfs_insert_file_extent() more flex, and it can now handle
the converted file extent where it has an non-zero offset.
And this makes it much easier to expand for future compressed file
extent generation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The function btrfs_record_file_extent() has extra handling that's
specific to convert, like allowing the range to be split by block group
boundary and image file extent boundary.
All of these split can only lead to corruption for non-converted fs.
As the only caller out of btrfs-convert is rootdir, which expects the
file extent item insert to respect the reserved data extent, and never
to be split.
Thankfully this is not going to cause huge problem, as
btrfs_record_file_extent() has extra checks if the data extent overlaps
with any existing one, and if it doesn't the handling will be the same
as the kernel.
But to avoid abuse, change btrfs_record_file_extent() by:
- Rename it to btrfs_convert_file_extent()
And add extra comments on that it is specific to btrfs-convert.
- Move it to convert/common.[ch]
- Introduce a helper insert_reserved_file_extent() for rootdir.c
Signed-off-by: Qu Wenruo <wqu@suse.com>
Making all in Documentation
[SPHINX] man
Documentation/Kernel-by-version.rst:: ERROR: Anonymous hyperlink mismatch: 10 references but 0 targets.
See "backrefs" attribute for IDs.
[PY] libbtrfsutil
The build is complaining about the missing space after (2) and
<https:...>, as shown in the example below:
v6.10-rc3 (2)<https://git.kernel.org/linus/07978330e63456a75a6d5c1c5053de24bdc9d16f>__,
The compiler interprets this as a backref attribute. Add a space to make
it render as text.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Text ending with `_` is treated as an anonymous hyperlink. Use an escape
character `\` to prevent this.
Making all in Documentation
[SPHINX] man
btrfs-progs/Documentation/dev/On-disk-format.rst:32: ERROR: Unknown target name: "btrfs".
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On python3-sphinx 7.2.6. and python3-sphinx_rtd_theme 2.0.0 there are
build errors.
Making all in Documentation
[SPHINX] man
../CHANGES:26: ERROR: Unexpected indentation.
../CHANGES:29: WARNING: Block quote ends without a blank line; unexpected unindent.
../CHANGES:204: ERROR: Unexpected indentation.
../CHANGES:205: WARNING: Block quote ends without a blank line; unexpected unindent.
Fix them by adjusting the indentation in the CHANGES file. The sublist
should be indented two spaces further than the main list.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This involves the following error cases:
- Unable to find the original item
Return -EAGAIN and release the path (which is not done in the original
code)
- Error from split_leaf()
Remove the BUG_ON() and handle the error.
The most common error is ENOSPC.
- Error from kmalloc()
Just handle the error and return -ENOMEM.
Issue: #312
Signed-off-by: Qu Wenruo <wqu@suse.com>
When stdio is line buffered printf will not flush anything (on musl?),
leaving the program hanging without displaying any prompt and weird
dialogs such as the following:
alpine:~# btrfstune -S 0 /dev/mmcblk1p1
WARNING: this is dangerous, clearing the seeding flag may cause the derived device not to be mountable!
y
WARNING: seeding flag is not set on /dev/mmcblk1p1
We are going to clear the seeding flag, are you sure? [y/N]: alpine:~#
Forcing flush makes the prompt display properly.
Signed-off-by: Dominique Martinet <dominique.martinet@atmark-techno.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
* fix repetition
* wording and punctuation in 'Nested subvolumes'
* wording and punctuation in 'system root layouts'
* wording and punctuation in 'Mount options'
* wording in 'Inode numbers'
* wording and punctuation in 'Performance'
- Explain that scrub is device based
- Add extra warning on NOCOW files
Which implies NODATASUM, and can cause unexpected stale data to be
returned.
- Explain the limitation of scrub
As it can only do very basic checksum verification and very basic
mirror based repair.
Signed-off-by: Colin Snover <csnover@users.noreply.github.com>
[ Add an SoB line and commit message, remove the mention of btrfs-check errors,
as there is no evidence/example where btrfs-check failed to choose a good mirror. ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Since balance is copying the old good data/metadata into a new chunk
(which can be on the same failed device), it's not a safe way to handle
failed devices.
Signed-off-by: Colin Snover <csnover@users.noreply.github.com>
[ Add an SoB and simple commit message, remove the unnecessary
explanation, and guide the user to use `btrfs dev replace` ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
- btrfs-check does device assembly automatically
Thus no difference when specifying different devices of the same
filesystem
- btrfs-check automatically chooses good metadata
Thus as long as there is any good mirror for metadata, it will not
report error for that repariable metadata.
Signed-off-by: Colin Snover <csnover@users.noreply.github.com>
[ Add an SoB line, remove the scrub recommendation as btrfs-check is
supposed to choose the good mirror ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
- Fix the format of the transid mismatch reason and type
- Fix a typo in the reason
- Explain more on the recoverable case
That both a regular metadata read and read-write scrub can do the
same trick.
- Add an extra data salvage method using "rescue=all,ro" mount option
Signed-off-by: Colin Snover <csnover@users.noreply.github.com>
[ Add an SoB line ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
- The IRC channel
- The mailling list
- The btrfs-progs repo for user space bugs
Signed-off-by: Colin Snover <csnover@users.noreply.github.com>
[ Add an SoB line ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
The current print-tree can not handle unsupported inode flags, e.g.
created by Synology's out-of-tree btrfs implementation.
The existing one just checks all the supported flags, and if no flag
hits, it will output "none" no matter if there is any unsupported one.
Fix this by implementing sprint_readable_flag(), and use the same
handling of print_readable_flag().
Although for inode flag, adds one extra handling to output "none" if no
flag hit at all.
Signed-off-by: Qu Wenruo <wqu@suse.com>
This includes:
- Remove the "__" prefix
Now the "__" is no longer recommended, and there is no function taking
the "print_readable_flag" in the first place.
- Move the supported flags calculation into print_readable_flag()
Since all callers are doing the same work before calling the function.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Added DEV_ITEM object id to the reserved object id list. It's historical reason
to let both of DEV_ITEM and ROOT_TREE have same object id. Developers should
be aware of it.
Signed-off-by: HAN Yuwei <hrx@bupt.moe>
[ Replace immediate number with key names ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
This "stil" -> "still" typo is causing the latest CI spellchecks to fail.
Fix that so we can get a good CI run.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Fixes false positive in btrfs check that was causing btrfs/192 to fail.
It looks like there's circumstances in which btrfs_log_changed_extents
can also log unmodified extents, but not their csums. If that happens,
check the main csum root as well before showing an error.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Use the correct address and size when looking for the csums for a
compressed extent in the tree log.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Fix a false positive in btrfs check, where we were returning an error
because an explicit hole in the log tree had no associated csum entry.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
The slides for the talk "Kernel maintainership: an oral tradition",
linked to in the documentation, seem to have gone from the Linux
Foundation website. Change to the version on bootlin.com.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
On systems with glibc 2.34 and 2.39, the following warning appears when
building the binary:
[CC] common/help.o
common/help.c: In function ‘usage’:
common/help.c:315:58: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
315 | fprintf(outf, "No short description for '%s'\n", token);
| ^~
common/help.c:312:46: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
312 | fprintf(outf, "No usage for '%s'\n", token);
| ^~
This happens for usage() which passes NULL pointer as token. Normally
this is fine, as fprintf() will output "(null)" for the NULL pointer,
but it's still not ideal.
Fix the warning by changing the token to "" if it's NULL.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Adds tests to mkfs-tests/036-rootdir-subvol for the modifiers to
mkfs.btrfs --subvol: ro, rw, default, and default-ro.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Reworks mkfs.btrfs --subvol so that dir and full_path in struct
rootdir_subvol are stored as arrays rather than pointers.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Change mkfs.btrfs --subvol so that instead of being of the form --subvol
DIR:FLAGS, it's instead --subvol MODIFIER:DIR, with MODIFIER being ro,
rw, default, or ro-default.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
With --chroot, the receive subcommand unconditionally sent a non-error
status message to stderr, e.g.:
$ btrfs --quiet receive --chroot /some/path
Chroot to /some/path
Signed-off-by: Sebastian Hamann <code@ares-macrotechnology.com>
Instead of copying the file during custom build commands, just use a
soft link to re-use the existing README.d from libbtrfsutil.
Issue: #310
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
Currently with python3.12, the python binding will always result the
following warning:
[PY] libbtrfsutil
/usr/lib/python3.12/site-packages/setuptools/_distutils/extension.py:134: UserWarning: Unknown Extension options: 'headers'
warnings.warn(msg)
[CAUSE]
In the setup.py which specifies the files to be included into the package,
we use setuptools::Extension to specify the file lists and include paths.
But there is no handling of Extension::headers member, thus resulting the
above warning.
[FIX]
According to the docs of setuptools, MANIFEST.in is the file controlling
what files should be included.
So instead of the non-supported headers, use MANIFEST.in to include the
needed headers.
Signed-off-by: Qu Wenruo <wqu@suse.com>
This introduces two new cases:
- 3 hardlinks without any subvolume
This should results 3 hard links inside the btrfs.
- 3 hardlinks, but a subvolume will split 2 of them
Then the 2 inside the same subvolume should still report 2 nlinks,
but the lone one inside the new subvolume can only report 1 nlink.
Signed-off-by: Qu Wenruo <wqu@suse.com>
The new hard link detection and creation support is done by maintaining
an rb tree with the following members:
- st_ino, st_dev
This is to record the stat() report from the host fs.
With this two, we can detect if it's really a hard link (st_dev
determines one filesystem/subvolume, and st_ino determines the inode
number inside the fs).
- root
This is btrfs root pointer. This a special requirement for the recent
introduced "--subvol" option.
As we can have the following corner case:
rootdir/
|- foobar_hardlink1
|- foobar_hardlink2
|- subv/ <- To be a subvolume inside btrfs
|- foobar_hardlink3
In above case, on the host fs, `subv/` directory is just a regular
directory, but in the new btrfs it will be a subvolume.
In that case, `foobar_hardlink3` cannot be created as a hard link,
but a new inode.
- st_nlink and found_nlink
Records the original reported number of links, and the nlinks we
created inside btrfs.
This is recorded in case we created all hard links and can remove
the entry early.
- btrfs_ino
This is the inode number inside btrfs.
And since we can handle hard links safely, remove all the related
warnings, and add a new note for `--subvol` option, warning about the
case where we need to split hard links due to subvolume boundary.
Pull-request: #873
Signed-off-by: Qu Wenruo <wqu@suse.com>
Remove functions that after the previous two patches are no longer
referenced.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Co-authored-by: Omar Sandoval <osandov@fb.com>
Call btrfs_util_subvolume_snapshot in cmd_subvolume_snapshot rather than
calling the ioctl directly.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Co-authored-by: Omar Sandoval <osandov@fb.com>
Call btrfs_util_subvolume_create in create_one_subvolume rather than
calling the ioctl directly.
Pull-request: #878
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
Co-authored-by: Omar Sandoval <osandov@fb.com>
