From fcd9142b67fe4e73d275b7d854bf58842bbe133a Mon Sep 17 00:00:00 2001 From: David Sterba Date: Mon, 30 Oct 2023 23:15:31 +0100 Subject: [PATCH] btrfs-progs: docs: formatting, fixups, updates - update Status page - new features in 6.7 - more ioctls - CSS fix to wrap long lines in tables [ci skip] Signed-off-by: David Sterba --- Documentation/Feature-by-version.rst | 10 ++ Documentation/Introduction.rst | 76 ++++++------- Documentation/Status.rst | 31 +++++- Documentation/_static/style.css | 8 ++ Documentation/btrfs-ioctl.rst | 153 +++++++++++++++++++-------- Documentation/mkfs.btrfs.rst | 12 +++ kernel-shared/uapi/btrfs.h | 2 +- 7 files changed, 207 insertions(+), 85 deletions(-) diff --git a/Documentation/Feature-by-version.rst b/Documentation/Feature-by-version.rst index f4d35642..38b6dae8 100644 --- a/Documentation/Feature-by-version.rst +++ b/Documentation/Feature-by-version.rst @@ -93,6 +93,16 @@ features see :doc:`Status` page. finalizing the directory, the cache also gains significant speedups (up to 10x). +6.7 - raid-stripe-tree + New tree for logical mapping, allows some RAID modes for zoned mode. + +6.7 - simplified quota accounting + A simplified mode of qgroups accounting + +6.7 - temporary fsid + Mount of cloned devices is now possible, the filesystem will get a new + randomly generated UUID on mount + 5.x --- diff --git a/Documentation/Introduction.rst b/Documentation/Introduction.rst index d4f62419..1b0cc461 100644 --- a/Documentation/Introduction.rst +++ b/Documentation/Introduction.rst @@ -5,43 +5,45 @@ BTRFS is a modern copy on write (COW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration. Its main features and benefits are: -* Snapshots which do not make a full copy of the files -* Built-in volume management, support for software-based RAID 0, RAID 1, RAID 10 and others -* Self-healing - checksums for data and metadata, automatic detection of silent data corruptions +* Snapshots which do not make a full copy of the files +* Built-in volume management, support for software-based RAID 0, RAID 1, RAID 10 and others +* Self-healing - checksums for data and metadata, automatic detection of silent data corruptions Feature overview: -* Extent based file storage -* 2\ :sup:`64` byte == 16 EiB maximum file size (practical limit is 8 EiB due to Linux VFS) -* Space-efficient packing of small files -* Space-efficient indexed directories -* Dynamic inode allocation -* Writable snapshots, read-only snapshots -* Subvolumes (separate internal filesystem roots) -* Checksums on data and metadata (crc32c, xxhash, sha256, blake2b) -* Compression (ZLIB, LZO, ZSTD), heuristics -* Integrated multiple device support - * File Striping - * File Mirroring - * File Striping+Mirroring - * Single and Dual Parity implementations (experimental, not production-ready) -* SSD (flash storage) awareness (TRIM/Discard for reporting free blocks for - reuse) and optimizations (e.g. avoiding unnecessary seek optimizations, - sending writes in clusters, even if they are from unrelated files. This - results in larger write operations and faster write throughput) -* Efficient incremental backup -* Background scrub process for finding and repairing errors of files with redundant copies -* Online filesystem defragmentation -* Offline filesystem check -* In-place conversion of existing ext2/3/4 and reiserfs file systems -* Seed devices. Create a (readonly) filesystem that acts as a template to seed - other Btrfs filesystems. The original filesystem and devices are included as - a readonly starting point for the new filesystem. Using copy on write, all - modifications are stored on different devices; the original is unchanged. -* Subvolume-aware quota support -* Send/receive of subvolume changes - * Efficient incremental filesystem mirroring -* Batch, or out-of-band deduplication (happens after writes, not during) -* Swapfile support -* Tree-checker, post-read and pre-write metadata verification -* Zoned mode support (SMR/ZBC/ZNS friendly allocation) +* Extent based file storage +* 2\ :sup:`64` byte == 16 EiB maximum file size (practical limit is 8 EiB due to Linux VFS) +* Space-efficient packing of small files +* Space-efficient indexed directories +* Dynamic inode allocation +* Writable snapshots, read-only snapshots +* Subvolumes (separate internal filesystem roots) +* Checksums on data and metadata (crc32c, xxhash, sha256, blake2b) +* Compression (ZLIB, LZO, ZSTD), heuristics +* Integrated multiple device support + + * File Striping + * File Mirroring + * File Striping+Mirroring + * Single and Dual Parity implementations (experimental, not production-ready) +* SSD (flash storage) awareness (TRIM/Discard for reporting free blocks for + reuse) and optimizations (e.g. avoiding unnecessary seek optimizations, + sending writes in clusters, even if they are from unrelated files. This + results in larger write operations and faster write throughput) +* Efficient incremental backup +* Background scrub process for finding and repairing errors of files with redundant copies +* Online filesystem defragmentation +* Offline filesystem check +* In-place conversion of existing ext2/3/4 and reiserfs file systems +* Seed devices. Create a (readonly) filesystem that acts as a template to seed + other Btrfs filesystems. The original filesystem and devices are included as + a readonly starting point for the new filesystem. Using copy on write, all + modifications are stored on different devices; the original is unchanged. +* Subvolume-aware quota support +* Send/receive of subvolume changes + + * Efficient incremental filesystem mirroring +* Batch, or out-of-band deduplication (happens after writes, not during) +* Swapfile support +* Tree-checker, post-read and pre-write metadata verification +* Zoned mode support (SMR/ZBC/ZNS friendly allocation) diff --git a/Documentation/Status.rst b/Documentation/Status.rst index b082f4ed..3e56e434 100644 --- a/Documentation/Status.rst +++ b/Documentation/Status.rst @@ -13,7 +13,7 @@ in meeting your performance expectations for your specific workload. Combination of features can vary in performance, the table does not cover all possibilities. -**The table is based on the latest released linux kernel: 6.5** +**The table is based on the latest released linux kernel: 6.6** The columns for each feature reflect the status of the implementation in following ways: @@ -123,6 +123,10 @@ in following ways: - :statusok:`OK` - OK - + * - Temporary UUID + - 6.7 + - 6.7 + - * - :doc:`Seeding` - :statusok:`OK` - OK @@ -131,6 +135,10 @@ in following ways: - :statusmok:`mostly OK` - mostly OK - qgroups with many snapshots slows down balance + * - :doc:`Quotas, simple qgroups` + - 6.7 + - 6.7 + - simplified qgroup accounting, better performance * - :doc:`Swapfile` - :statusok:`OK` - n/a @@ -226,7 +234,11 @@ converted later). - :statusok:`OK` - OK - - * - :ref:`Block group tree`` + * - :ref:`Block group tree` + - :statusok:`OK` + - OK + - + * - :ref:`Raid stripe tree` - :statusok:`OK` - OK - @@ -321,7 +333,7 @@ are unaffected by the zoned device constraints. - Notes * - Boot - :statusincompat:`incompatible` - - The blocks where partition table is stored is used for super block + - The blocks where partition table is stored are used for super block * - Mixed block groups - :statusincompat:`incompatible` - Interleaving data and metadata would lead to out of order write @@ -343,6 +355,12 @@ are unaffected by the zoned device constraints. * - Free space tree - :statusok:`supported` - + * - Block group tree + - :statusok:`supported` + - + * - Raid stripe tree + - :statusok:`supported` + - Allows to use RAID in zoned mode * - Filesystem resize - :statusok:`supported` - @@ -352,9 +370,12 @@ are unaffected by the zoned device constraints. * - Metadata UUID change - :statusok:`supported` - - * - RAID (all) + * - RAID0, RAID1* + - :statusok:`supported` + - requires `raid-stripe-tree` + * - RAID56 - not implemented - - This requires raid-stripe-tree feature which is still work in progress + - will be supported once raid-stripe-tree support is implemented * - discard - not implemented - May not be required at all due to automatic zone reclaim diff --git a/Documentation/_static/style.css b/Documentation/_static/style.css index 27a188e8..ee529d3c 100644 --- a/Documentation/_static/style.css +++ b/Documentation/_static/style.css @@ -2,6 +2,14 @@ max-width: 1200px !important; } +.wy-table-responsive { + white-space: normal !important; +} + +.wy-table-responsive table td, .wy-table-responsive table th { + white-space: normal !important; +} + .green { color: green; } diff --git a/Documentation/btrfs-ioctl.rst b/Documentation/btrfs-ioctl.rst index 123dacba..831312bd 100644 --- a/Documentation/btrfs-ioctl.rst +++ b/Documentation/btrfs-ioctl.rst @@ -240,10 +240,7 @@ LIST OF IOCTLS * - Name - Description - Data - * - BTRFS_IOC_SUBVOL_CREATE - - (obsolete) create a subvolume - - :ref:`struct btrfs_ioctl_vol_args` - * - BTRFS_IOC_SNAP_CREATE + * - :ref:`BTRFS_IOC_SNAP_CREATE` - (obsolete) create a snapshot of a subvolume - :ref:`struct btrfs_ioctl_vol_args` * - BTRFS_IOC_DEFRAG @@ -252,9 +249,9 @@ LIST OF IOCTLS * - BTRFS_IOC_RESIZE - - - * - BTRFS_IOC_SCAN_DEV - - - - + * - :ref:`BTRFS_IOC_SCAN_DEV` + - scan and register a given device path with filesystem module + - :ref:`struct btrfs_ioctl_vol_args` * - :ref:`BTRFS_IOC_SYNC` - Sync the filesystem, possibly process queued up work - NULL @@ -273,12 +270,12 @@ LIST OF IOCTLS * - BTRFS_IOC_CLONE_RANGE - - - * - BTRFS_IOC_SUBVOL_CREATE - - - - - * - BTRFS_IOC_SNAP_DESTROY - - - - + * - :ref:`BTRFS_IOC_SUBVOL_CREATE` + - (obsolete) create a subvolume + - :ref:`struct btrfs_ioctl_vol_args` + * - :ref:`BTRFS_IOC_SNAP_DESTROY` + - (obsolete) delete a subvolume + - :ref:`struct btrfs_ioctl_vol_args` * - BTRFS_IOC_DEFRAG_RANGE - - @@ -291,9 +288,9 @@ LIST OF IOCTLS * - BTRFS_IOC_INO_LOOKUP - - - * - BTRFS_IOC_DEFAULT_SUBVOL - - - - + * - :ref:`BTRFS_IOC_DEFAULT_SUBVOL` + - set the default subvolume id + - uint64_t * - BTRFS_IOC_SPACE_INFO - - @@ -421,29 +418,7 @@ LIST OF IOCTLS DETAILED DESCRIPTION -------------------- -BTRFS_IOC_SUBVOL_CREATE -~~~~~~~~~~~~~~~~~~~~~~~ - -.. note:: - obsoleted by :ref:`BTRFS_IOC_SUBVOL_CREATE_V2` - -*(since: 3.0, obsoleted: 4.0)* Create a subvolume. - -.. list-table:: - :header-rows: 1 - - * - Field - - Description - * - ioctl fd - - file descriptor of the parent directory of the new subvolume - * - ioctl args - - :ref:`struct btrfs_ioctl_vol_args` - * - args.fd - - ignored - * - args.name - - name of the subvolume, although the buffer can be almost 4k, the file - size is limited by Linux VFS to 255 characters and must not contain a slash - ('/') +.. _BTRFS_IOC_SNAP_CREATE: BTRFS_IOC_SNAP_CREATE ~~~~~~~~~~~~~~~~~~~~~ @@ -466,10 +441,34 @@ BTRFS_IOC_SNAP_CREATE - file descriptor of any directory inside the subvolume to snapshot, must be on the same filesystem * - args.name - - name of the subvolume, although the buffer can be almost 4k, the file + - name of the subvolume, although the buffer can be almost 4KiB, the file size is limited by Linux VFS to 255 characters and must not contain a slash ('/') +.. _BTRFS_IOC_SCAN_DEV: + +BTRFS_IOC_SCAN_DEV +~~~~~~~~~~~~~~~~~~ + +Scan and register a given device in the filesystem module, which can be later +used for automatic device and filesystem association at mount time. This +operates on the control device, not files from a mounted filesystem. +Can be safely called repeatedly with same device path. + +.. list-table:: + :header-rows: 1 + + * - Field + - Description + * - ioctl fd + - file descriptor of the control device :file:`/dev/btrfs-control` + * - ioctl args + - :ref:`struct btrfs_ioctl_vol_args` + * - args.fd + - ignored + * - args.name + - full path of the device + .. _BTRFS_IOC_SYNC: BTRFS_IOC_SYNC @@ -489,6 +488,76 @@ subvolume cleaning or queued defragmentation. * - ioctl args - NULL +.. _BTRFS_IOC_SUBVOL_CREATE: + +BTRFS_IOC_SUBVOL_CREATE +~~~~~~~~~~~~~~~~~~~~~~~ + +.. note:: + obsoleted by :ref:`BTRFS_IOC_SUBVOL_CREATE_V2` + +*(since: 3.0, obsoleted: 4.0)* Create a subvolume. + +.. list-table:: + :header-rows: 1 + + * - Field + - Description + * - ioctl fd + - file descriptor of the parent directory of the new subvolume + * - ioctl args + - :ref:`struct btrfs_ioctl_vol_args` + * - args.fd + - ignored + * - args.name + - name of the subvolume, although the buffer can be almost 4KiB, the file + size is limited by Linux VFS to 255 characters and must not contain a slash + ('/') + +.. _BTRFS_IOC_SNAP_DESTROY: + +BTRFS_IOC_SNAP_DESTROY +~~~~~~~~~~~~~~~~~~~~~~ + +.. note:: + obsoleted by :ref:`BTRFS_IOC_SNAP_DESTROY_V2` + +*(since: 2.6.33, obsoleted: 5.7)* Delete a subvolume. + +.. list-table:: + :header-rows: 1 + + * - Field + - Description + * - ioctl fd + - file descriptor of the parent directory of the new subvolume + * - ioctl args + - :ref:`struct btrfs_ioctl_vol_args` + * - args.fd + - ignored + * - args.name + - name of the subvolume, although the buffer can be almost 4KiB, the file + size is limited by Linux VFS to 255 characters and must not contain a slash + ('/') + +.. _BTRFS_IOC_DEFAULT_SUBVOL: + +BTRFS_IOC_DEFAULT_SUBVOL +~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the given subvolume id as the default one when mounting the filesystem +without `subvol=path` or `subvolid=id` options. + +.. list-table:: + :header-rows: 1 + + * - Field + - Description + * - ioctl fd + - file descriptor of the directory inside which to create the new snapshot + * - ioctl args + - numeric value of subvolume to become default (uint64_t) + .. _BTRFS_IOC_SNAP_CREATE_V2: BTRFS_IOC_SNAP_CREATE_V2 @@ -513,7 +582,7 @@ Create a snapshot of a subvolume. * - args.flags - any subset of `BTRFS_SUBVOL_RDONLY` to make the new snapshot read-only, or `BTRFS_SUBVOL_QGROUP_INHERIT` to apply the `qgroup_inherit` field - * - name + * - args.name - the name, under the ioctl fd, for the new subvolume .. _BTRFS_IOC_SUBVOL_CREATE_V2: @@ -547,7 +616,7 @@ BTRFS_IOC_SUBVOL_CREATE_V2 (:ref:`struct btrfs_qgroup_inherit`) and limits (:ref:`struct btrfs_qgroup_limit`) * - name - - name of the subvolume, although the buffer can be almost 4k, the file + - name of the subvolume, although the buffer can be almost 4KiB, the file size is limited by Linux VFS to 255 characters and must not contain a slash ('/') diff --git a/Documentation/mkfs.btrfs.rst b/Documentation/mkfs.btrfs.rst index e4b53b86..7e23b9f6 100644 --- a/Documentation/mkfs.btrfs.rst +++ b/Documentation/mkfs.btrfs.rst @@ -333,6 +333,18 @@ block-group-tree enabled at *mkfs* time is possible, see :doc:`btrfstune`. Online conversion is not possible. +.. _mkfs-feature-raid-stripe-tree: + +raid-stripe-tree + (kernel support since 6.7) + + New tree for logical file extent mapping where the physical mapping + may not match on multiple devices. this is now used in zoned mode to + implement RAID0/RAID1* profiles, but can be used in non-zoned mode as + well. The support for RAID56 is in development and will eventually + fix the problems with the current implementation. This is a backward + incompatible feature and has to be enabled at mkfs time. + squota (kernel support since 6.7) diff --git a/kernel-shared/uapi/btrfs.h b/kernel-shared/uapi/btrfs.h index 34e84b89..fb24c5f7 100644 --- a/kernel-shared/uapi/btrfs.h +++ b/kernel-shared/uapi/btrfs.h @@ -1171,7 +1171,7 @@ enum btrfs_err_code { struct btrfs_ioctl_vol_args) #define BTRFS_IOC_RESIZE _IOW(BTRFS_IOCTL_MAGIC, 3, \ struct btrfs_ioctl_vol_args) -#define BTRFS_IOC_SCAN_DEV _IOW(BTRFS_IOCTL_MAGIC, 4, \ +#define BTRFS_IOC_SCAN_DEV _IOW(BTRFS_IOCTL_MAGIC, 5, \ struct btrfs_ioctl_vol_args) #define BTRFS_IOC_FORGET_DEV _IOW(BTRFS_IOCTL_MAGIC, 5, \ struct btrfs_ioctl_vol_args)