As of bfcf6d04f8ee ("btrfs: handle FS_IOC_READ_VERITY_METADATA ioctl")
the kernel supports FS_IOC_READ_VERITY_METADATA for btrfs (6.14). Add
that to the docs.
Pull-request: #970
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There was a question in the mailing list regarding option --quiet and
why it's not working as 'btrfs subvolume create --quiet ...'.
Historically there were per-command verbosity options but this was
unified under the global options with it's specific syntax. This is not
uncommon and e.g. git also does have this option split.
The documentation and help strings are not making this clear enough, so
print syntax examples at the end of the help of 'btrfs', and do other
adjustments.
Signed-off-by: David Sterba <dsterba@suse.com>
The zlib and zstd compression methods support using compression levels.
Enable defrag to pass them to kernel.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Daniel Vacek <neelx@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Replace the use of IOW as abbreviation of "In other words" with the
original expanded meaning, as users who don't have English as their
first language may not know what it means.
Pull-request: #966
Signed-off-by: David Sterba <dsterba@suse.com>
Add "duration" format in seconds to fmt_print which will convert the
input to one of the following strings:
1. if number of seconds represents more than one day, the output will be
for example: "1 days 01:30:00" (left the plural so parsing back the
string is easier)
2. if less then a day: "23:30:10"
Author: Racz Zoltan <racz.zoli@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In v6.14 kernel release, btrfs will force a direct IO to fall back to
a buffered one if the inode requires a data checksum.
This will cause a small performance drop, to solve the false data
checksum mismatch problem caused by direct IOs.
Although such a change is small to most end users, for those requiring
such a zero-copy direct IO this will be a behavior change, and this
requires a proper documentation update.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
There is a bug report that when deleting a device using sysfs
/sys/block/<dev>/device/delete, the kernel module will still try to read
and write the device.
Normally it's fine as long as all chunks can tolerate that removed
device (e.g. all RAID1).
But the problem is when one is trying to lower the redundancy by
converting to another profile:
# mkfs.btrfs -f -m raid1 -d raid1 /dev/sdd /dev/sde
# mount /dev/sdd /mnt
# echo 1 > /sys/block/sde/device/delete
# btrfs balance start --force -mdup -dsingle /mnt
This will lead to the filesystem mounted RO, with the following error messages:
sd 6:0:0:0: [sde] Synchronizing SCSI cache
ata7.00: Entering standby power mode
btrfs: attempt to access beyond end of device
sde: rw=6145, sector=21696, nr_sectors = 32 limit=0
btrfs: attempt to access beyond end of device
sde: rw=6145, sector=21728, nr_sectors = 32 limit=0
btrfs: attempt to access beyond end of device
sde: rw=6145, sector=21760, nr_sectors = 32 limit=0
BTRFS error (device sdd): bdev /dev/sde errs: wr 1, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device sdd): bdev /dev/sde errs: wr 2, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device sdd): bdev /dev/sde errs: wr 3, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device sdd): bdev /dev/sde errs: wr 3, rd 0, flush 1, corrupt 0, gen 0
btrfs: attempt to access beyond end of device
sde: rw=145409, sector=128, nr_sectors = 8 limit=0
BTRFS warning (device sdd): lost super block write due to IO error on /dev/sde (-5)
BTRFS error (device sdd): bdev /dev/sde errs: wr 4, rd 0, flush 1, corrupt 0, gen 0
btrfs: attempt to access beyond end of device
sde: rw=14337, sector=131072, nr_sectors = 8 limit=0
BTRFS warning (device sdd): lost super block write due to IO error on /dev/sde (-5)
BTRFS error (device sdd): bdev /dev/sde errs: wr 5, rd 0, flush 1, corrupt 0, gen 0
BTRFS error (device sdd): error writing primary super block to device 2
BTRFS info (device sdd): balance: start -dconvert=single -mconvert=dup -sconvert=dup
BTRFS info (device sdd): relocating block group 1372585984 flags data|raid1
BTRFS error (device sdd): bdev /dev/sde errs: wr 5, rd 0, flush 2, corrupt 0, gen 0
BTRFS warning (device sdd): chunk 2446327808 missing 1 devices, max tolerance is 0 for writable mount
BTRFS: error (device sdd) in write_all_supers:4044: errno=-5 IO failure (errors while submitting device barriers.)
BTRFS info (device sdd state E): forced readonly
BTRFS warning (device sdd state E): Skipping commit of aborted transaction.
BTRFS error (device sdd state EA): Transaction aborted (error -5)
BTRFS: error (device sdd state EA) in cleanup_transaction:2017: errno=-5 IO failure
BTRFS info (device sdd state EA): balance: ended with status: -5
[CAUSE]
Btrfs doesn't have any runtime device error handling, it fully rely on
the extra copy provided.
For the sysfs block device removal, normally there is a device shutdown
callback to the running fs, but unfortunately btrfs doesn't support this
callback yet.
Thus even with that device removed, btrfs will still access that
removed device (both read and write, even if they will fail).
Normally for a full RAID1 btrfs, it will still be fine reading/write the
fs as usual. The proper action is to replace the
removed/missing/failing device with a newer one using `btrfs device
replace`.
But when doing the convert, btrfs will allocate new metadata chunks on
to the removed device (which will lose all writes).
And since the new metadata profile is DUP, which can not handle any
missing device of that metadata chunk, finally it triggers the final
protection at transaction commit time, and flips the filesystem to RO,
before causing any real data loss.
[DOC ENHANCEMENT]
Add a warning to the `convert` filter about the dangerous doing convert
to a lower redundancy profile when there is a known failing/removed
device.
And mention the proper way to handle such failing/missing device.
The root fix is to introduce a failing/removed device detection for
btrfs, but that will be a pretty big feature and will take quite some
time before landing it upstream.
Link: https://lore.kernel.org/linux-btrfs/2cb1d81e-12a8-4fb1-b3fc-e7e83d31e059@siddall.name/
Reported-by: Jeff Siddall <news@siddall.name>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Just like all qgroup functions, if a qgroup is marked inconsistent, limit
will not work as expected. In fact with recent kernels, limit and
qgroup number updating will be fully skipped if qgroup is already
inconsistent.
Add one extra note on `btrfs qgroup limit` subcommand for it.
Link: https://bugzilla.suse.com/show_bug.cgi?id=1235765
Reported-by: Vojtech Lacina <vlacina@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
The references used in :doc: need to reference relative path of the
document, otherwise this leads to a warning.
btrfs-progs/Documentation/Send-receive.rst:19: WARNING: unknown document: 'dev-send-stream' [ref.doc]
btrfs-progs/Documentation/dev/CmdLineConventions.rst:60: WARNING: unknown document: 'btrfstune' [ref.doc]
Signed-off-by: David Sterba <dsterba@suse.com>
Add sections for each directory so it's more visible where the sections
start and end. Update formatting, enhance descriptions.
Signed-off-by: David Sterba <dsterba@suse.com>
For parity with 'btrfs', also print the builtin features.
$ btrfstune --version
btrfs-image, part of btrfs-progs v6.12
+EXPERIMENTAL -INJECT -STATIC +LZO +ZSTD +UDEV +FSVERITY +ZONED CRYPTO=builtin
Signed-off-by: David Sterba <dsterba@suse.com>
For parity with 'btrfs', also print the builtin features.
$ btrfs-image --version
btrfs-image, part of btrfs-progs v6.12
+EXPERIMENTAL -INJECT -STATIC +LZO +ZSTD +UDEV +FSVERITY +ZONED CRYPTO=builtin
Signed-off-by: David Sterba <dsterba@suse.com>
For parity with 'btrfs', also print the builtin features, although not
all of them are relevant.
$ btrfs-convert --version
btrfs-convert, part of btrfs-progs v6.12
+EXPERIMENTAL -INJECT -STATIC +LZO +ZSTD +UDEV +FSVERITY +ZONED CRYPTO=builtin
Signed-off-by: David Sterba <dsterba@suse.com>
For parity with 'btrfs', also print the builtin features as the
compression is now available.
$ mkfs.btrfs --version
mkfs.btrfs, part of btrfs-progs v6.12
+EXPERIMENTAL -INJECT -STATIC +LZO +ZSTD +UDEV +FSVERITY +ZONED CRYPTO=builtin
Signed-off-by: David Sterba <dsterba@suse.com>
This has been deprecated since 4.0 and mkfs fails since 6.0 with that
option. No need to keep it around anymore.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new option `--limit <throughput_limit>` to `btrfs scrub start`.
This has some extra behavior differences compared to `btrfs scrub limit`:
- Only set the value for the involved scrub device(s)
If it's a full fs scrub, it will be the same as
`btrfs scrub limit -a -l <value>`.
If it's a single device, it will bt the same as
`btrfs scrub limit -d <devid> -l <value>`.
- Automatically revert to the old limit after scrub is finished
- It only needs one single command line to set the limit
Issue: #943
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
There is a bug report that read-only scrub on a read-write fs still
causes writes into the fs, and that will be caught if there is a
read-only block device among the storage stack.
This will cause a kernel warning on failed transaction commit:
BTRFS info (device dm-3): first mount of filesystem e18f0c40-88de-413f-9d7e-dcc8136ad6dd
BTRFS info (device dm-3): using crc32c (crc32c-intel) checksum algorithm
BTRFS info (device dm-3): using free-space-tree
BTRFS info (device dm-3): scrub: started on devid 1
Trying to write to read-only block-device md127
btrfs_dev_stat_inc_and_print: 362 callbacks suppressed
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 1, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 2, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 3, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 4, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 5, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 6, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 7, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 8, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 9, rd 0, flush 0, corrupt 0, gen 0
BTRFS error (device dm-3): bdev /dev/mapper/data errs: wr 10, rd 0, flush 0, corrupt 0, gen 0
BTRFS: error (device dm-3) in btrfs_commit_transaction:2523: errno=-5 IO failure (Error while writing out transaction)
BTRFS info (device dm-3 state E): forced readonly
BTRFS warning (device dm-3 state E): Skipping commit of aborted transaction.
BTRFS error (device dm-3 state EA): Transaction aborted (error -5)
BTRFS: error (device dm-3 state EA) in cleanup_transaction:2017: errno=-5 IO failure
BTRFS warning (device dm-3 state EA): failed setting block group ro: -5
BTRFS info (device dm-3 state EA): scrub: not finished on devid 1 with status: -5
[CAUSE]
The root cause is inside btrfs_inc_block_group_ro(), where we need to
hold a transaction handle, to prevent the transaction to be committed,
until we hold ro_block_group_mutex.
This will cause an empty transaction by itself, thus even if we can mark
the block group read-only without any extra workload, we still need to
commit the new and empty transaction.
Unfortunately this means RO scrub on RW filesystem will always cause the
fs to be updated.
[FIX]
The best fix is to make btrfs to avoid empty commit transaction, but
even with that done, read-only scrub on rw mount can still cause real
metadata updates (e.g. allocate new chunks and update device error
statistics).
It will be very complex to make read-only scrub to be fully read-only
on a read-write btrfs.
Thankfully read-only scrub on read-write mount with read-only device in
the storage stack is pretty rare, thus a documentation update should be
enough.
Issue: #934
Pull-request: #935
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The new option for experimental featues was added in 6.12, mention it
where the DEBUG option was previously used for the same purpose.
[ci skip]
Signed-off-by: David Sterba <dsterba@suse.com>
The compression support is optional, eg. also in 'btrfs-restore', so
print the support in help text.
usage: mkfs.btrfs [options] <dev> [<dev...>]
...
--compress ALGO[:LEVEL] compress files by algorithm and level, ALGO can be 'no' (default), zstd, lzo, zlib
Built-in:
- ZSTD: yes
- LZO: yes
- ZLIB: yes
...
Signed-off-by: David Sterba <dsterba@suse.com>
Minor update to the command format and indentation, drop the [ ] and
prefix only by two spaces (like kernel).
Signed-off-by: David Sterba <dsterba@suse.com>
When simple quotas is enabled, every new data extent gets a special
inline OWNER_REF item that identifies the owning subvolume. This makes
simple quotas backwards incompatible with kernels older than v6.7. Even
if you disable quotas on the filesystem, the OWNER_REF items are
sprinkled throughout the extent tree and older kernels are unable to
parse them.
However, it is relatively easy to simply walk the extent tree and remove
these inline ref items. This gives squota adopters the option to *fully*
disable squotas on their system and un-set the incompat bit. Add this
capability to btrfstune, which requires only a little tricky btrfs item
data shifting.
This functionality was tested with a new unit test, as well as a similar
but more thorough integration test in fstests
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
The documentation lists -q as the flag for enabling simple quotas, but
the actual parsing only handles --enable-simple-quota. Update the
documentation string.
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
[ Add man page entry for `--enable-simple-quota` option ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Allow --compress to work with lzo.
Signed-off-by: Mark Harmstone <maharmstone@fb.com>
[ Add extra handling when LZO support is not compiled in ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Making all in Documentation
[SPHINX] man
Documentation/Kernel-by-version.rst:: ERROR: Anonymous hyperlink mismatch: 10 references but 0 targets.
See "backrefs" attribute for IDs.
[PY] libbtrfsutil
The build is complaining about the missing space after (2) and
<https:...>, as shown in the example below:
v6.10-rc3 (2)<https://git.kernel.org/linus/07978330e63456a75a6d5c1c5053de24bdc9d16f>__,
The compiler interprets this as a backref attribute. Add a space to make
it render as text.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Text ending with `_` is treated as an anonymous hyperlink. Use an escape
character `\` to prevent this.
Making all in Documentation
[SPHINX] man
btrfs-progs/Documentation/dev/On-disk-format.rst:32: ERROR: Unknown target name: "btrfs".
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On python3-sphinx 7.2.6. and python3-sphinx_rtd_theme 2.0.0 there are
build errors.
Making all in Documentation
[SPHINX] man
../CHANGES:26: ERROR: Unexpected indentation.
../CHANGES:29: WARNING: Block quote ends without a blank line; unexpected unindent.
../CHANGES:204: ERROR: Unexpected indentation.
../CHANGES:205: WARNING: Block quote ends without a blank line; unexpected unindent.
Fix them by adjusting the indentation in the CHANGES file. The sublist
should be indented two spaces further than the main list.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
