From 1202e4207bc182d83f193adb1557359d52687103 Mon Sep 17 00:00:00 2001 From: Zac Dover Date: Thu, 25 May 2023 10:16:51 +1000 Subject: [PATCH] doc/rados: edit filestore-config-ref.rst Edit doc/rados/configuration/filestore-config-ref.rst. https://tracker.ceph.com/issues/58485 Signed-off-by: Zac Dover --- .../configuration/filestore-config-ref.rst | 162 +++++++++--------- doc/rados/configuration/storage-devices.rst | 1 + doc/rados/operations/bluestore-migration.rst | 2 + 3 files changed, 88 insertions(+), 77 deletions(-) diff --git a/doc/rados/configuration/filestore-config-ref.rst b/doc/rados/configuration/filestore-config-ref.rst index c9a3cb285f4..9d65d00a6cf 100644 --- a/doc/rados/configuration/filestore-config-ref.rst +++ b/doc/rados/configuration/filestore-config-ref.rst @@ -1,10 +1,15 @@ ============================ Filestore Config Reference ============================ -.. warning:: Filestore has been deprecated in the Reef release and is no longer supported. -The Filestore back end is no longer the default when creating new OSDs, -though Filestore OSDs are still supported. +.. note:: Since the Luminous release of Ceph, Filestore has not been Ceph's + default storage back end. Since the Luminous release of Ceph, BlueStore has + been Ceph's default storage back end. However, Filestore OSDs are still + supported. See :ref:`OSD Back Ends + `. See :ref:`BlueStore Migration + ` for instructions explaining how to + replace an existing Filestore back end with a BlueStore back end. + ``filestore_debug_omap_check`` @@ -19,26 +24,31 @@ though Filestore OSDs are still supported. Extended Attributes =================== -Extended Attributes (XATTRs) are important for Filestore OSDs. -Some file systems have limits on the number of bytes that can be stored in XATTRs. -Additionally, in some cases, the file system may not be as fast as an alternative -method of storing XATTRs. The following settings may help improve performance -by using a method of storing XATTRs that is extrinsic to the underlying file system. +Extended Attributes (XATTRs) are important for Filestore OSDs. However, Certain +disadvantages can occur when the underlying file system is used for the storage +of XATTRs: some file systems have limits on the number of bytes that can be +stored in XATTRs, and your file system might in some cases therefore run slower +than would an alternative method of storing XATTRs. For this reason, a method +of storing XATTRs extrinsic to the underlying file system might improve +performance. To implement such an extrinsic method, refer to the following +settings. -Ceph XATTRs are stored as ``inline xattr``, using the XATTRs provided -by the underlying file system, if it does not impose a size limit. If -there is a size limit (4KB total on ext4, for instance), some Ceph -XATTRs will be stored in a key/value database when either the +If the underlying file system has no size limit, then Ceph XATTRs are stored as +``inline xattr``, using the XATTRs provided by the file system. But if there is +a size limit (for example, ext4 imposes a limit of 4 KB total), then some Ceph +XATTRs will be stored in a key/value database when the limit is reached. More +precisely, this begins to occur when either the ``filestore_max_inline_xattr_size`` or ``filestore_max_inline_xattrs`` threshold is reached. ``filestore_max_inline_xattr_size`` -:Description: The maximum size of an XATTR stored in the file system (i.e., XFS, - Btrfs, EXT4, etc.) per object. Should not be larger than the - file system can handle. Default value of 0 means to use the value - specific to the underlying file system. +:Description: Defines the maximum size per object of an XATTR that can be + stored in the file system (for example, XFS, Btrfs, ext4). The + specified size should not be larger than the file system can + handle. Using the default value of 0 instructs Filestore to use + the value specific to the file system. :Type: Unsigned 32-bit Integer :Required: No :Default: ``0`` @@ -46,8 +56,9 @@ threshold is reached. ``filestore_max_inline_xattr_size_xfs`` -:Description: The maximum size of an XATTR stored in the XFS file system. - Only used if ``filestore_max_inline_xattr_size`` == 0. +:Description: Defines the maximum size of an XATTR that can be stored in the + XFS file system. This setting is used only if + ``filestore_max_inline_xattr_size`` == 0. :Type: Unsigned 32-bit Integer :Required: No :Default: ``65536`` @@ -55,8 +66,9 @@ threshold is reached. ``filestore_max_inline_xattr_size_btrfs`` -:Description: The maximum size of an XATTR stored in the Btrfs file system. - Only used if ``filestore_max_inline_xattr_size`` == 0. +:Description: Defines the maximum size of an XATTR that can be stored in the + Btrfs file system. This setting is used only if + ``filestore_max_inline_xattr_size`` == 0. :Type: Unsigned 32-bit Integer :Required: No :Default: ``2048`` @@ -64,8 +76,8 @@ threshold is reached. ``filestore_max_inline_xattr_size_other`` -:Description: The maximum size of an XATTR stored in other file systems. - Only used if ``filestore_max_inline_xattr_size`` == 0. +:Description: Defines the maximum size of an XATTR that can be stored in other file systems. + This setting is used only if ``filestore_max_inline_xattr_size`` == 0. :Type: Unsigned 32-bit Integer :Required: No :Default: ``512`` @@ -73,9 +85,8 @@ threshold is reached. ``filestore_max_inline_xattrs`` -:Description: The maximum number of XATTRs stored in the file system per object. - Default value of 0 means to use the value specific to the - underlying file system. +:Description: Defines the maximum number of XATTRs per object that can be stored in the file system. + Using the default value of 0 instructs Filestore to use the value specific to the file system. :Type: 32-bit Integer :Required: No :Default: ``0`` @@ -83,8 +94,8 @@ threshold is reached. ``filestore_max_inline_xattrs_xfs`` -:Description: The maximum number of XATTRs stored in the XFS file system per object. - Only used if ``filestore_max_inline_xattrs`` == 0. +:Description: Defines the maximum number of XATTRs per object that can be stored in the XFS file system. + This setting is used only if ``filestore_max_inline_xattrs`` == 0. :Type: 32-bit Integer :Required: No :Default: ``10`` @@ -92,8 +103,8 @@ threshold is reached. ``filestore_max_inline_xattrs_btrfs`` -:Description: The maximum number of XATTRs stored in the Btrfs file system per object. - Only used if ``filestore_max_inline_xattrs`` == 0. +:Description: Defines the maximum number of XATTRs per object that can be stored in the Btrfs file system. + This setting is used only if ``filestore_max_inline_xattrs`` == 0. :Type: 32-bit Integer :Required: No :Default: ``10`` @@ -101,8 +112,8 @@ threshold is reached. ``filestore_max_inline_xattrs_other`` -:Description: The maximum number of XATTRs stored in other file systems per object. - Only used if ``filestore_max_inline_xattrs`` == 0. +:Description: Defines the maximum number of XATTRs per object that can be stored in other file systems. + This setting is used only if ``filestore_max_inline_xattrs`` == 0. :Type: 32-bit Integer :Required: No :Default: ``2`` @@ -112,18 +123,19 @@ threshold is reached. Synchronization Intervals ========================= -Filestore needs to periodically quiesce writes and synchronize the -file system, which creates a consistent commit point. It can then free journal -entries up to the commit point. Synchronizing more frequently tends to reduce -the time required to perform synchronization, and reduces the amount of data -that needs to remain in the journal. Less frequent synchronization allows the -backing file system to coalesce small writes and metadata updates more -optimally, potentially resulting in more efficient synchronization at the -expense of potentially increasing tail latency. +Filestore must periodically quiesce writes and synchronize the file system. +Each synchronization creates a consistent commit point. When the commit point +is created, Filestore is able to free all journal entries up to that point. +More-frequent synchronization tends to reduce both synchronization time and +the amount of data that needs to remain in the journal. Less-frequent +synchronization allows the backing file system to coalesce small writes and +metadata updates, potentially increasing synchronization +efficiency but also potentially increasing tail latency. + ``filestore_max_sync_interval`` -:Description: The maximum interval in seconds for synchronizing Filestore. +:Description: Defines the maximum interval (in seconds) for synchronizing Filestore. :Type: Double :Required: No :Default: ``5`` @@ -131,7 +143,7 @@ expense of potentially increasing tail latency. ``filestore_min_sync_interval`` -:Description: The minimum interval in seconds for synchronizing Filestore. +:Description: Defines the minimum interval (in seconds) for synchronizing Filestore. :Type: Double :Required: No :Default: ``.01`` @@ -143,14 +155,14 @@ Flusher ======= The Filestore flusher forces data from large writes to be written out using -``sync_file_range`` before the sync in order to (hopefully) reduce the cost of -the eventual sync. In practice, disabling 'filestore_flusher' seems to improve -performance in some cases. +``sync_file_range`` prior to the synchronization. +Ideally, this action reduces the cost of the eventual synchronization. In practice, however, disabling +'filestore_flusher' seems in some cases to improve performance. ``filestore_flusher`` -:Description: Enables the filestore flusher. +:Description: Enables the Filestore flusher. :Type: Boolean :Required: No :Default: ``false`` @@ -159,7 +171,7 @@ performance in some cases. ``filestore_flusher_max_fds`` -:Description: Sets the maximum number of file descriptors for the flusher. +:Description: Defines the maximum number of file descriptors for the flusher. :Type: Integer :Required: No :Default: ``512`` @@ -177,7 +189,7 @@ performance in some cases. ``filestore_fsync_flushes_journal_data`` -:Description: Flush journal data during file system synchronization. +:Description: Flushes journal data during file-system synchronization. :Type: Boolean :Required: No :Default: ``false`` @@ -188,11 +200,11 @@ performance in some cases. Queue ===== -The following settings provide limits on the size of the Filestore queue. +The following settings define limits on the size of the Filestore queue: ``filestore_queue_max_ops`` -:Description: Defines the maximum number of in progress operations the file store accepts before blocking on queuing new operations. +:Description: Defines the maximum number of in-progress operations that Filestore accepts before it blocks the queueing of any new operations. :Type: Integer :Required: No. Minimal impact on performance. :Default: ``50`` @@ -200,23 +212,20 @@ The following settings provide limits on the size of the Filestore queue. ``filestore_queue_max_bytes`` -:Description: The maximum number of bytes for an operation. +:Description: Defines the maximum number of bytes permitted per operation. :Type: Integer :Required: No :Default: ``100 << 20`` - - .. index:: filestore; timeouts Timeouts ======== - ``filestore_op_threads`` -:Description: The number of file system operation threads that execute in parallel. +:Description: Defines the number of file-system operation threads that execute in parallel. :Type: Integer :Required: No :Default: ``2`` @@ -224,7 +233,7 @@ Timeouts ``filestore_op_thread_timeout`` -:Description: The timeout for a file system operation thread (in seconds). +:Description: Defines the timeout (in seconds) for a file-system operation thread. :Type: Integer :Required: No :Default: ``60`` @@ -232,7 +241,7 @@ Timeouts ``filestore_op_thread_suicide_timeout`` -:Description: The timeout for a commit operation before cancelling the commit (in seconds). +:Description: Defines the timeout (in seconds) for a commit operation before the commit is cancelled. :Type: Integer :Required: No :Default: ``180`` @@ -246,17 +255,17 @@ B-Tree Filesystem ``filestore_btrfs_snap`` -:Description: Enable snapshots for a ``btrfs`` filestore. +:Description: Enables snapshots for a ``btrfs`` Filestore. :Type: Boolean -:Required: No. Only used for ``btrfs``. +:Required: No. Used only for ``btrfs``. :Default: ``true`` ``filestore_btrfs_clone_range`` -:Description: Enable cloning ranges for a ``btrfs`` filestore. +:Description: Enables cloning ranges for a ``btrfs`` Filestore. :Type: Boolean -:Required: No. Only used for ``btrfs``. +:Required: No. Used only for ``btrfs``. :Default: ``true`` @@ -268,7 +277,7 @@ Journal ``filestore_journal_parallel`` -:Description: Enables parallel journaling, default for Btrfs. +:Description: Enables parallel journaling, default for ``btrfs``. :Type: Boolean :Required: No :Default: ``false`` @@ -276,7 +285,7 @@ Journal ``filestore_journal_writeahead`` -:Description: Enables writeahead journaling, default for XFS. +:Description: Enables write-ahead journaling, default for XFS. :Type: Boolean :Required: No :Default: ``false`` @@ -284,7 +293,7 @@ Journal ``filestore_journal_trailing`` -:Description: Deprecated, never use. +:Description: Deprecated. **Never use.** :Type: Boolean :Required: No :Default: ``false`` @@ -296,8 +305,8 @@ Misc ``filestore_merge_threshold`` -:Description: Min number of files in a subdir before merging into parent - NOTE: A negative value means to disable subdir merging +:Description: Defines the minimum number of files permitted in a subdirectory before the subdirectory is merged into its parent directory. + NOTE: A negative value means that subdirectory merging is disabled. :Type: Integer :Required: No :Default: ``-10`` @@ -306,8 +315,8 @@ Misc ``filestore_split_multiple`` :Description: ``(filestore_split_multiple * abs(filestore_merge_threshold) + (rand() % filestore_split_rand_factor)) * 16`` - is the maximum number of files in a subdirectory before - splitting into child directories. + is the maximum number of files permitted in a subdirectory + before the subdirectory is split into child directories. :Type: Integer :Required: No @@ -317,10 +326,10 @@ Misc ``filestore_split_rand_factor`` :Description: A random factor added to the split threshold to avoid - too many (expensive) Filestore splits occurring at once. See - ``filestore_split_multiple`` for details. - This can only be changed offline for an existing OSD, - via the ``ceph-objectstore-tool apply-layout-settings`` command. + too many (expensive) Filestore splits occurring at the same time. + For details, see ``filestore_split_multiple``. + To change this setting for an existing OSD, it is necessary to take the OSD + offline before running the ``ceph-objectstore-tool apply-layout-settings`` command. :Type: Unsigned 32-bit Integer :Required: No @@ -329,7 +338,7 @@ Misc ``filestore_update_to`` -:Description: Limits Filestore auto upgrade to specified version. +:Description: Limits automatic upgrades to a specified version of Filestore. Useful in cases in which you want to avoid upgrading to a specific version. :Type: Integer :Required: No :Default: ``1000`` @@ -337,7 +346,7 @@ Misc ``filestore_blackhole`` -:Description: Drop any new transactions on the floor. +:Description: Drops any new transactions on the floor, similar to redirecting to NULL. :Type: Boolean :Required: No :Default: ``false`` @@ -345,7 +354,7 @@ Misc ``filestore_dump_file`` -:Description: File onto which store transaction dumps. +:Description: Defines the file that transaction dumps are stored on. :Type: Boolean :Required: No :Default: ``false`` @@ -353,7 +362,7 @@ Misc ``filestore_kill_at`` -:Description: inject a failure at the n'th opportunity +:Description: Injects a failure at the *n*\th opportunity. :Type: String :Required: No :Default: ``false`` @@ -361,8 +370,7 @@ Misc ``filestore_fail_eio`` -:Description: Fail/Crash on eio. +:Description: Fail/Crash on EIO. :Type: Boolean :Required: No :Default: ``true`` - diff --git a/doc/rados/configuration/storage-devices.rst b/doc/rados/configuration/storage-devices.rst index 3b9d45a3d58..c83e87da773 100644 --- a/doc/rados/configuration/storage-devices.rst +++ b/doc/rados/configuration/storage-devices.rst @@ -25,6 +25,7 @@ There are several Ceph daemons in a storage cluster: additional monitoring and providing interfaces to external monitoring and management systems. +.. _rados_config_storage_devices_osd_backends: OSD Back Ends ============= diff --git a/doc/rados/operations/bluestore-migration.rst b/doc/rados/operations/bluestore-migration.rst index ecd7a4c5e4c..d24782c46f4 100644 --- a/doc/rados/operations/bluestore-migration.rst +++ b/doc/rados/operations/bluestore-migration.rst @@ -1,3 +1,5 @@ +.. _rados_operations_bluestore_migration: + ===================== BlueStore Migration =====================