diff --git a/Documentation/Contributors.rst b/Documentation/Contributors.rst index 525e757c..603c554f 100644 --- a/Documentation/Contributors.rst +++ b/Documentation/Contributors.rst @@ -13,13 +13,7 @@ Sorted by amount of contributions: * Oracle The following contributed in the past (sorted alphabetically): - -* Fujitsu -* Fusion-IO -* Intel -* Linux Foundation -* Red Hat -* STRATO AG +Fujitsu, Fusion-IO, Intel, Linux Foundation, Red Hat, STRATO AG. Statistics for 6.x series ------------------------- @@ -37,13 +31,13 @@ Statistics for 6.x series Legend: -- *Files:* fs/btrfs/\*.[ch], fs/btrfs/tests/\*.[ch], include/uapi/linux/btrfs.h, include/uapi/linux/btrfs_tree.h, include/linux/btrfs.h, include/trace/events/btrfs.h -- *Version:* mainline version -- *Contributors:* number of people that sent patches that modified ''Files'', direct btrfs development or originating from other tree-wide changes -- *SLOC:* lines of code, http://dwheeler.com/sloccount/ (generated by version 2.26) -- *Raw lines:* counted by ''wc -l'' over ''Files'' -- *Patches:* number of patches from ''Contributors'', merge commits excluded -- *Diffstat:* lines added and deleted in ''Files'' +- *Files:* fs/btrfs/\*.[ch], fs/btrfs/tests/\*.[ch], include/uapi/linux/btrfs.h, include/uapi/linux/btrfs_tree.h, include/linux/btrfs.h, include/trace/events/btrfs.h +- *Version:* mainline version +- *Contributors:* number of people that sent patches that modified ''Files'', direct btrfs development or originating from other tree-wide changes +- *SLOC:* lines of code, http://dwheeler.com/sloccount/ (generated by version 2.26) +- *Raw lines:* counted by ''wc -l'' over ''Files'' +- *Patches:* number of patches from ''Contributors'', merge commits excluded +- *Diffstat:* lines added and deleted in ''Files'' Statistics for 5.x series @@ -74,6 +68,7 @@ Statistics for 5.x series "5.18", "30", "109159", "155372", "143", "+3489 -1523" "5.19", "21", "109140", "155848", "202", "+4448 -3972" + Statistics for 4.x series ------------------------- @@ -103,6 +98,7 @@ Statistics for 4.x series "4.19", "25", "97547", "132655", "193", "+2058 -3070" "4.20", "22", "97830", "133283", "128", "+1560 -932" + Statistics for 3.x series ------------------------- @@ -131,6 +127,7 @@ Statistics for 3.x series "3.18", "25", "83910", "112906", "149", "+3696 -1631" "3.19", "18", "85420", "115031", "82", "+2802 -677" + Statistics for 2.6.x series --------------------------- diff --git a/Documentation/Kernel-by-version.rst b/Documentation/Kernel-by-version.rst index 0d554af1..11656f96 100644 --- a/Documentation/Kernel-by-version.rst +++ b/Documentation/Kernel-by-version.rst @@ -45,13 +45,13 @@ Summary of kernel changes for each version. Performance improvements: -- reduced amount of reserved metadata for delayed items +- reduced amount of reserved metadata for delayed items - - when inserted items can be batched into one leaf - - when deleting batched directory index items - - when deleting delayed items used for deletion - - overall improved count of files/sec, decreased subvolume lock - contention + - when inserted items can be batched into one leaf + - when deleting batched directory index items + - when deleting delayed items used for deletion + - overall improved count of files/sec, decreased subvolume lock + contention - metadata item access bounds checker micro-optimized, with a few percent of improved runtime for metadata-heavy operations @@ -275,6 +275,9 @@ Fixes: new value for maximum active threads would not be set to the actual work queues (since 6.0) +6.4 (Jun 2022) +^^^^^^^^^^^^^^ + 5.x --- diff --git a/Documentation/Quick-start.rst b/Documentation/Quick-start.rst index 23e1dcd3..4335ae53 100644 --- a/Documentation/Quick-start.rst +++ b/Documentation/Quick-start.rst @@ -1,4 +1,15 @@ Quick start =========== -... +For a really quick start you can simply create and mount the filesystem. Make +sure that the block device you'd like to use is suitable so you don't overwrite existing data. + +.. code-block:: shell + + # mkfs.btrfs /dev/sdx + # mount /dev/sdx /mnt/test + +The default options should be acceptable for most users and sometimes can be +changed later. The example above is for a single device filesystem, creating a +*single* profile for data (no redundant copies of the blocks), and *DUP* +for metadata (each block is duplicated). diff --git a/Documentation/ReleaseChecklist b/Documentation/ReleaseChecklist deleted file mode 100644 index 98f23e05..00000000 --- a/Documentation/ReleaseChecklist +++ /dev/null @@ -1,35 +0,0 @@ -Release checklist -================= - -Last code touches: - -* make the code ready, collect patches queued for the release -* look to mailinglist for any relevant last-minute fixes - -Pre-checks: - -* update package in OBS, (multi arch build checks) -* submit sources to coverity -* run all internal functional tests - * defaults - * D=asan - * D=ubsan -* run all build tests -* run with fstests - -Pre-release: - -* write CHANGES entry - -Release: - -* tag release, sign -* make tar -* build check of unpacked tar -* upload tar to kernel.org -* refresh git branches, push tags - -Post-release: - -* write and send announcement mail to mailinglist -* update title on IRC diff --git a/Documentation/Status.rst b/Documentation/Status.rst index 5a131232..91d9345e 100644 --- a/Documentation/Status.rst +++ b/Documentation/Status.rst @@ -13,7 +13,7 @@ in meeting your performance expectations for your specific workload. Combination of features can vary in performance, the table does not cover all possibilities. -**The table is based on the latest released linux kernel: 6.3** +**The table is based on the latest released linux kernel: 6.4** The columns for each feature reflect the status of the implementation in following ways: @@ -32,7 +32,9 @@ in following ways: .. role:: statusok .. role:: statusmok -.. role:: statusunst +.. role:: statusunstable +.. role:: statusunsupp +.. role:: statusincompat .. list-table:: :header-rows: 1 @@ -126,7 +128,7 @@ in following ways: - mostly OK - reading from mirrors in parallel can be optimized further (see below) * - :ref:`RAID56` - - :statusunst:`unstable` + - :statusunstable:`unstable` - n/a - (see below) * - Mixed block groups @@ -220,11 +222,12 @@ in following ways: * - :doc:`Subpage block size` - :statusmok:`mostly OK` - mostly OK - - + - Also see table below for more detailed compatibility. * - :doc:`Zoned mode` - :statusmok:`mostly OK` - mostly OK - - there are known bugs, use only for testing + - Not yet feature complete but moderately stable, also see table below + for more detailed compatibility. Please open an issue if: @@ -233,6 +236,8 @@ Please open an issue if: worth mentioning separately - you know of a bug that lowers the feature status +.. _status-subpage-block-size: + Subpage block size ------------------ @@ -247,19 +252,27 @@ with subpage or require another feature to work: - Status - Notes * - Inline files - - unsupported - - The max_inline mount option value is ignored + - :statusunsupp:`unsupported` + - The max_inline mount option value is ignored, as if mounted with max_inline=0 * - Free space cache v1 - - unsupported - - Free space tree is mandatory + - :statusunsupp:`unsupported` + - Free space tree is mandatory, v1 has some assumptions about page size * - Compression - - partial support + - :statusok:`partial support` - Only page-aligned ranges can be compressed + * - Sectorsize + - :statusok:`supported` + - The list of supported sector sizes on a given version can be found + in file :file:`/sys/fs/btrfs/features/supported_sectorsizes` Zoned mode ---------- +Features that completely incompatible with zoned mode are listed below. +Compatible features may not be listed and are assumed to work as they +are unaffected by the zoned device constraints. + .. list-table:: :header-rows: 1 @@ -267,35 +280,56 @@ Zoned mode - Status - Notes * - Boot - - incompatible + - :statusincompat:`incompatible` - The blocks where partition table is stored is used for super block * - Mixed block groups - - incompatible + - :statusincompat:`incompatible` - Interleaving data and metadata would lead to out of order write * - NODATACOW - - incompatible + - :statusincompat:`incompatible` - In-place overwrite * - fallocate - - incompatible + - :statusincompat:`incompatible` - Preallocation of blocks would require an out of order write * - Free space cache v1 - - incompatible + - :statusincompat:`incompatible` - Cache data are updated in a NODATACOW-way + * - Swapfile + - :statusincompat:`incompatible` + - Swap blocks are written out of order + * - Offline UUID change + - :statusincompat:`incompatible` + - Metadata blocks are updated in-place * - Free space tree - - supported + - :statusok:`supported` - - * - fstrim - - not implemented - - This would require free space v1 * - single profile - - supported + - :statusok:`supported` - Both data and metadata * - DUP profile - - partial support + - :statusok:`partial support` - Only for metadata + * - Filesystem resize + - :statusok:`supported` + - + * - Balance + - :statusok:`supported` + - + * - Metadata UUID change + - :statusok:`supported` + - * - RAID (all) - not implemented - This requires raid-stripe-tree feature which is still work in progress + * - discard + - not implemented + - May not be required at all due to automatic zone reclaim + * - fsverity + - TBD + - + * - seeding + - TBD + - Details that do not fit the table diff --git a/Documentation/Subpage.rst b/Documentation/Subpage.rst index bd1b41a8..c762b6a3 100644 --- a/Documentation/Subpage.rst +++ b/Documentation/Subpage.rst @@ -9,40 +9,18 @@ to the exactly same size of the block and page. On x86_64 this is typically pages, like 64KiB on 64bit ARM or PowerPC. This means filesystems created with 64KiB sector size cannot be mounted on a system with 4KiB page size. -While with subpage support, systems with 64KiB page size can create (still needs -"-s 4k" option for :command:`mkfs.btrfs`) and mount filesystems with 4KiB sectorsize. +While with subpage support systems with 64KiB page size can create +and mount filesystems with 4KiB sectorsize. This still needs to use option "-s +4k" option for :command:`mkfs.btrfs`. Requirements, limitations ------------------------- The initial subpage support has been added in v5.15, although it's still considered as experimental, most features are already working without problems. +On a 64KiB page system filesystem with 4KiB sectorsize can be mounted and used +as usual as long as the initial mount succeeds. There are cases a mount will be +rejected when verifying compatible features. -End users can mount filesystems with 4KiB sectorsize and do their usual -workload, while should not notice any obvious change, as long as the initial -mount succeeded (there are cases a mount will be rejected though). - -The following features has some limitations for subpage: - -- Supported page sizes: 4KiB, 8KiB, 16KiB, 32KiB, 64KiB - -- Supported filesystem sector sizes on a given host are exported in - :file:`/sys/fs/btrfs/features/supported_sectorsizes` - -- No inline extents - - This is an artificial limitation, to prevent mixed inline and regular extents. - - Thus max_inline mount option will be silently ignored for subpage mounts, - and it always acts as "max_inline=0". - -- Compression writes are limited to page aligned ranges - - Compression write for subpage has been introduced in v5.16, with the - limitation that only page aligned range can be compressed. This limitation - is due to how btrfs handles delayed allocation. - -- No support for v1 space cache - - The old v1 cache has quite some hard coded page size usage, and considering - it already deprecated, we force v2 cache for subpage. +Please refer to status page of :ref:`status-subpage-block-size` for +compatibility. diff --git a/Documentation/Trim.rst b/Documentation/Trim.rst index 533fa8a1..541e30ba 100644 --- a/Documentation/Trim.rst +++ b/Documentation/Trim.rst @@ -38,4 +38,6 @@ relocate the data, however this leads to unexpected performance drop. Running trim periodically could prevent that too. When a filesystem is created by :doc:`mkfs.btrfs` and is capable -of trim, then it's by default performed on all devices. +of trim, then it's by default performed on all devices. Since kernel 6.2 the +*discard=async* mount option is automatically enabled on devices that support +that. diff --git a/Documentation/_static/style.css b/Documentation/_static/style.css index 2f9564c8..27a188e8 100644 --- a/Documentation/_static/style.css +++ b/Documentation/_static/style.css @@ -16,6 +16,14 @@ color: darkorange; } -.statusunst { +.statusunstable { color: red; } + +.statusunsupp { + color: darkorange; +} + +.statusincompat { + color: goldenrod; +}; diff --git a/Documentation/btrfs-ioctl.rst b/Documentation/btrfs-ioctl.rst index f40e350b..df86d8bc 100644 --- a/Documentation/btrfs-ioctl.rst +++ b/Documentation/btrfs-ioctl.rst @@ -25,91 +25,130 @@ DATA STRUCTURES AND DEFINITIONS .. code-block:: c - struct btrfs_ioctl_vol_args { - __s64 fd; - char name[BTRFS_PATH_NAME_MAX + 1]; - }; + struct btrfs_ioctl_vol_args { + __s64 fd; + char name[BTRFS_PATH_NAME_MAX + 1]; + }; .. _struct_btrfs_ioctl_vol_args_v2: .. code-block:: c - struct btrfs_ioctl_vol_args_v2 { - __s64 fd; - __u64 transid; - __u64 flags; - union { - struct { - __u64 size; - struct btrfs_qgroup_inherit __user *qgroup_inherit; - }; - __u64 unused[4]; - }; - union { - char name[BTRFS_SUBVOL_NAME_MAX + 1]; - __u64 devid; - __u64 subvolid; - }; - }; + #define BTRFS_SUBVOL_RDONLY (1ULL << 1) + #define BTRFS_SUBVOL_QGROUP_INHERIT (1ULL << 2) + #define BTRFS_DEVICE_SPEC_BY_ID (1ULL << 3) + #define BTRFS_SUBVOL_SPEC_BY_ID (1ULL << 4) + + struct btrfs_ioctl_vol_args_v2 { + __s64 fd; + __u64 transid; + __u64 flags; + union { + struct { + __u64 size; + struct btrfs_qgroup_inherit __user *qgroup_inherit; + }; + __u64 unused[4]; + }; + union { + char name[BTRFS_SUBVOL_NAME_MAX + 1]; + __u64 devid; + __u64 subvolid; + }; + }; .. _struct_btrfs_ioctl_get_subvol_info_args: .. code-block:: c - struct btrfs_ioctl_get_subvol_info_args { - /* Id of this subvolume */ - __u64 treeid; + struct btrfs_ioctl_get_subvol_info_args { + /* Id of this subvolume */ + __u64 treeid; - /* Name of this subvolume, used to get the real name at mount point */ - char name[BTRFS_VOL_NAME_MAX + 1]; + /* Name of this subvolume, used to get the real name at mount point */ + char name[BTRFS_VOL_NAME_MAX + 1]; - /* - * Id of the subvolume which contains this subvolume. - * Zero for top-level subvolume or a deleted subvolume. - */ - __u64 parent_id; + /* + * Id of the subvolume which contains this subvolume. + * Zero for top-level subvolume or a deleted subvolume. + */ + __u64 parent_id; - /* - * Inode number of the directory which contains this subvolume. - * Zero for top-level subvolume or a deleted subvolume - */ - __u64 dirid; + /* + * Inode number of the directory which contains this subvolume. + * Zero for top-level subvolume or a deleted subvolume + */ + __u64 dirid; - /* Latest transaction id of this subvolume */ - __u64 generation; + /* Latest transaction id of this subvolume */ + __u64 generation; - /* Flags of this subvolume */ - __u64 flags; + /* Flags of this subvolume */ + __u64 flags; - /* UUID of this subvolume */ - __u8 uuid[BTRFS_UUID_SIZE]; + /* UUID of this subvolume */ + __u8 uuid[BTRFS_UUID_SIZE]; - /* - * UUID of the subvolume of which this subvolume is a snapshot. - * All zero for a non-snapshot subvolume. - */ - __u8 parent_uuid[BTRFS_UUID_SIZE]; + /* + * UUID of the subvolume of which this subvolume is a snapshot. + * All zero for a non-snapshot subvolume. + */ + __u8 parent_uuid[BTRFS_UUID_SIZE]; - /* - * UUID of the subvolume from which this subvolume was received. - * All zero for non-received subvolume. - */ - __u8 received_uuid[BTRFS_UUID_SIZE]; + /* + * UUID of the subvolume from which this subvolume was received. + * All zero for non-received subvolume. + */ + __u8 received_uuid[BTRFS_UUID_SIZE]; - /* Transaction id indicating when change/create/send/receive happened */ - __u64 ctransid; - __u64 otransid; - __u64 stransid; - __u64 rtransid; - /* Time corresponding to c/o/s/rtransid */ - struct btrfs_ioctl_timespec ctime; - struct btrfs_ioctl_timespec otime; - struct btrfs_ioctl_timespec stime; - struct btrfs_ioctl_timespec rtime; + /* Transaction id indicating when change/create/send/receive happened */ + __u64 ctransid; + __u64 otransid; + __u64 stransid; + __u64 rtransid; + /* Time corresponding to c/o/s/rtransid */ + struct btrfs_ioctl_timespec ctime; + struct btrfs_ioctl_timespec otime; + struct btrfs_ioctl_timespec stime; + struct btrfs_ioctl_timespec rtime; - /* Must be zero */ - __u64 reserved[8]; - }; + /* Must be zero */ + __u64 reserved[8]; + }; + +.. _struct_btrfs_qgroup_inherit: + +.. code-block:: c + + #define BTRFS_QGROUP_INHERIT_SET_LIMITS (1ULL << 0) + + struct btrfs_qgroup_inherit { + __u64 flags; + __u64 num_qgroups; + __u64 num_ref_copies; + __u64 num_excl_copies; + struct btrfs_qgroup_limit lim; + __u64 qgroups[]; + }; + +.. _struct_btrfs_qgroup_limit: + +.. code-block:: c + + #define BTRFS_QGROUP_LIMIT_MAX_RFER (1ULL << 0) + #define BTRFS_QGROUP_LIMIT_MAX_EXCL (1ULL << 1) + #define BTRFS_QGROUP_LIMIT_RSV_RFER (1ULL << 2) + #define BTRFS_QGROUP_LIMIT_RSV_EXCL (1ULL << 3) + #define BTRFS_QGROUP_LIMIT_RFER_CMPR (1ULL << 4) + #define BTRFS_QGROUP_LIMIT_EXCL_CMPR (1ULL << 5) + + struct btrfs_qgroup_limit { + __u64 flags; + __u64 max_rfer; + __u64 max_excl; + __u64 rsv_rfer; + __u64 rsv_excl; + }; .. list-table:: :header-rows: 1 @@ -150,7 +189,7 @@ The 'BTRFS_IOC_NUMBER' is says which operation should be done on the given arguments. Some ioctls take a specific data structure, some of them share a common one, no argument structure ioctls exist too. -The library 'libbtrfsutil' wraps a few ioctls for convenience. Using raw ioctls +The library *libbtrfsutil* wraps a few ioctls for convenience. Using raw ioctls is not discouraged but may be cumbersome though it does not need additional library dependency. Backward compatibility is guaranteed and incompatible changes usually lead to a new version of the ioctl. Enhancements of existing @@ -174,8 +213,8 @@ LIST OF IOCTLS - (obsolete) create a subvolume - :ref:`struct btrfs_ioctl_vol_args` * - BTRFS_IOC_SNAP_CREATE - - - - + - (obsolete) create a snapshot of a subvolume + - :ref:`struct btrfs_ioctl_vol_args` * - BTRFS_IOC_DEFRAG - - @@ -185,9 +224,9 @@ LIST OF IOCTLS * - BTRFS_IOC_SCAN_DEV - - - * - BTRFS_IOC_SYNC - - - - + * - :ref:`BTRFS_IOC_SYNC` + - Sync the filesystem, possibly process queued up work + - NULL * - BTRFS_IOC_CLONE - - @@ -375,6 +414,50 @@ BTRFS_IOC_SUBVOL_CREATE size is limited by Linux VFS to 255 characters and must not contain a slash ('/') +BTRFS_IOC_SNAP_CREATE +~~~~~~~~~~~~~~~~~~~~~ + +.. note:: + obsoleted by :ref:`BTRFS_IOC_SNAP_CREATE_V2` + +*(since: 3.0, obsoleted: 4.0)* Create a snapshot of a subvolume. + +.. list-table:: + :header-rows: 1 + + * - Field + - Description + * - ioctl fd + - file descriptor of the parent directory of the new subvolume + * - ioctl args + - :ref:`struct btrfs_ioctl_vol_args` + * - args.fd + - file descriptor of any directory inside the subvolume to snapshot, + must be on the same filesystem + * - args.name + - name of the subvolume, although the buffer can be almost 4k, the file + size is limited by Linux VFS to 255 characters and must not contain a slash + ('/') + +.. _BTRFS_IOC_SYNC: + +BTRFS_IOC_SYNC +~~~~~~~~~~~~~~ + +Sync the filesystem data as would ``sync()`` syscall do, additionally +wake up the internal transaction thread that may trigger actions like +subvolume cleaning or queued defragmentation. + +.. list-table:: + :header-rows: 1 + + * - Field + - Description + * - ioctl fd + - file descriptor of any file or directory in the filesystem + * - ioctl args + - NULL + .. _BTRFS_IOC_SNAP_CREATE_V2: BTRFS_IOC_SNAP_CREATE_V2 @@ -392,7 +475,8 @@ Create a snapshot of a subvolume. * - ioctl args - :ref:`struct btrfs_ioctl_vol_args_v2` * - args.fd - - file descriptor of any directory inside the subvolume to snapshot + - file descriptor of any directory inside the subvolume to snapshot, + must be on the filesystem * - args.transid - ignored * - args.flags @@ -406,7 +490,7 @@ Create a snapshot of a subvolume. BTRFS_IOC_SUBVOL_CREATE_V2 ~~~~~~~~~~~~~~~~~~~~~~~~~~ -*(since: 3.6)* Create a subvolume, qgroup inheritance can be specified. +*(since: 3.6)* Create a subvolume, qgroup inheritance and limits can be specified. .. list-table:: :header-rows: 1 @@ -422,17 +506,19 @@ BTRFS_IOC_SUBVOL_CREATE_V2 * - args.transid - ignored * - args.flags - - ignored + - flags to set on the subvolume, ``BTRFS_SUBVOL_RDONLY`` for readonly, + ``BTRFS_SUBVOL_QGROUP_INHERIT`` if the qgroup related fileds should be + processed * - args.size - - ... + - number of entries in ``args.qgroup_inherit`` * - args.qgroup_inherit - - ... + - inherit the given qgroups + (:ref:`struct btrfs_qgroup_inherit`) and + limits (:ref:`struct btrfs_qgroup_limit`) * - name - name of the subvolume, although the buffer can be almost 4k, the file size is limited by Linux VFS to 255 characters and must not contain a slash ('/') - * - devid - - ... .. _BTRFS_IOC_SUBVOL_GETFLAGS: diff --git a/Documentation/dev/ReleaseChecklist.rst b/Documentation/dev/ReleaseChecklist.rst new file mode 100644 index 00000000..7789cc83 --- /dev/null +++ b/Documentation/dev/ReleaseChecklist.rst @@ -0,0 +1,49 @@ +Release checklist +================= + +Last code touches: + +* make the code ready, collect patches queued for the release +* look to mailinglist for any relevant last-minute fixes +* skim patches for typos, inconsistent subjects + +Pre-checks: + +* update package in OBS, (multi arch build checks) +* run all functional tests locally with + + * defaults + * D=asan + * D=ubsan +* run all build tests (``tests/build-tests.sh``) +* run with fstests +* check Github actions for status (https://github.com/kdave/btrfs-progs/actions) + + * branch *devel* + * branch *release-test* -- extensive pre-release build checks + * branch *coverage-test* -- code coverage, for information purposes only + +Pre-release: + +* write CHANGES entry (will be visible on RTD right away) + +Release: + +* tag release, sign +* build check of unpacked tar +* generate documentation +* make tar +* upload tar to kernel.org +* refresh git branches, push tags + +Post-release: + +* write and send announcement mail to the mailinglist +* update title on IRC +* github updates + + * create a new release from the latest tag + * copy text from CHANGES as contents, formatting is the same + * wait for static binaries github action to finish + * run ``ci/actions/update-artifacts`` to copy the built static binaries to the + release (requires github command line tool ``gh``) diff --git a/Documentation/index.rst b/Documentation/index.rst index e80b1121..32c7655c 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -61,6 +61,7 @@ Welcome to BTRFS documentation! dev/dev-send-stream dev/dev-json dev/dev-internal-apis + dev/ReleaseChecklist btrfs-ioctl .. toctree::