RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-01-03 09:32:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc/rados: edit filestore-config-ref.rst

Browse Source 
Edit doc/rados/configuration/filestore-config-ref.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
This commit is contained in:
Zac Dover 2023-05-25 10:16:51 +10:00
parent 17f4abe9c9
commit 1202e4207b
3 changed files with 88 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

162
doc/rados/configuration/filestore-config-ref.rst
View File

@ -1,10 +1,15 @@
============================
Filestore Config Reference
============================
.. warning:: Filestore has been deprecated in the Reef release and is no longer supported.
The Filestore back end is no longer the default when creating new OSDs,
though Filestore OSDs are still supported.
.. note:: Since the Luminous release of Ceph, Filestore has not been Ceph's
default storage back end. Since the Luminous release of Ceph, BlueStore has
been Ceph's default storage back end. However, Filestore OSDs are still
supported. See :ref:`OSD Back Ends
<rados_config_storage_devices_osd_backends>`. See :ref:`BlueStore Migration
<rados_operations_bluestore_migration>` for instructions explaining how to
replace an existing Filestore back end with a BlueStore back end.
``filestore_debug_omap_check``
@ -19,26 +24,31 @@ though Filestore OSDs are still supported.
Extended Attributes
===================
Extended Attributes (XATTRs) are important for Filestore OSDs.
Some file systems have limits on the number of bytes that can be stored in XATTRs.
Additionally, in some cases, the file system may not be as fast as an alternative
method of storing XATTRs. The following settings may help improve performance
by using a method of storing XATTRs that is extrinsic to the underlying file system.
Extended Attributes (XATTRs) are important for Filestore OSDs. However, Certain
disadvantages can occur when the underlying file system is used for the storage
of XATTRs: some file systems have limits on the number of bytes that can be
stored in XATTRs, and your file system might in some cases therefore run slower
than would an alternative method of storing XATTRs. For this reason, a method
of storing XATTRs extrinsic to the underlying file system might improve
performance. To implement such an extrinsic method, refer to the following
settings.
Ceph XATTRs are stored as ``inline xattr``, using the XATTRs provided
by the underlying file system, if it does not impose a size limit. If
there is a size limit (4KB total on ext4, for instance), some Ceph
XATTRs will be stored in a key/value database when either the
If the underlying file system has no size limit, then Ceph XATTRs are stored as
``inline xattr``, using the XATTRs provided by the file system. But if there is
a size limit (for example, ext4 imposes a limit of 4 KB total), then some Ceph
XATTRs will be stored in a key/value database when the limit is reached. More
precisely, this begins to occur when either the
``filestore_max_inline_xattr_size`` or ``filestore_max_inline_xattrs``
threshold is reached.
``filestore_max_inline_xattr_size``
:Description: The maximum size of an XATTR stored in the file system (i.e., XFS,
Btrfs, EXT4, etc.) per object. Should not be larger than the
file system can handle. Default value of 0 means to use the value
specific to the underlying file system.
:Description: Defines the maximum size per object of an XATTR that can be
stored in the file system (for example, XFS, Btrfs, ext4). The
specified size should not be larger than the file system can
handle. Using the default value of 0 instructs Filestore to use
the value specific to the file system.
:Type: Unsigned 32-bit Integer
:Required: No
:Default: ``0``
@ -46,8 +56,9 @@ threshold is reached.
``filestore_max_inline_xattr_size_xfs``
:Description: The maximum size of an XATTR stored in the XFS file system.
Only used if ``filestore_max_inline_xattr_size`` == 0.
:Description: Defines the maximum size of an XATTR that can be stored in the
XFS file system. This setting is used only if
``filestore_max_inline_xattr_size`` == 0.
:Type: Unsigned 32-bit Integer
:Required: No
:Default: ``65536``
@ -55,8 +66,9 @@ threshold is reached.
``filestore_max_inline_xattr_size_btrfs``
:Description: The maximum size of an XATTR stored in the Btrfs file system.
Only used if ``filestore_max_inline_xattr_size`` == 0.
:Description: Defines the maximum size of an XATTR that can be stored in the
Btrfs file system. This setting is used only if
``filestore_max_inline_xattr_size`` == 0.
:Type: Unsigned 32-bit Integer
:Required: No
:Default: ``2048``
@ -64,8 +76,8 @@ threshold is reached.
``filestore_max_inline_xattr_size_other``
:Description: The maximum size of an XATTR stored in other file systems.
Only used if ``filestore_max_inline_xattr_size`` == 0.
:Description: Defines the maximum size of an XATTR that can be stored in other file systems.
This setting is used only if ``filestore_max_inline_xattr_size`` == 0.
:Type: Unsigned 32-bit Integer
:Required: No
:Default: ``512``
@ -73,9 +85,8 @@ threshold is reached.
``filestore_max_inline_xattrs``
:Description: The maximum number of XATTRs stored in the file system per object.
Default value of 0 means to use the value specific to the
underlying file system.
:Description: Defines the maximum number of XATTRs per object that can be stored in the file system.
Using the default value of 0 instructs Filestore to use the value specific to the file system.
:Type: 32-bit Integer
:Required: No
:Default: ``0``
@ -83,8 +94,8 @@ threshold is reached.
``filestore_max_inline_xattrs_xfs``
:Description: The maximum number of XATTRs stored in the XFS file system per object.
Only used if ``filestore_max_inline_xattrs`` == 0.
:Description: Defines the maximum number of XATTRs per object that can be stored in the XFS file system.
This setting is used only if ``filestore_max_inline_xattrs`` == 0.
:Type: 32-bit Integer
:Required: No
:Default: ``10``
@ -92,8 +103,8 @@ threshold is reached.
``filestore_max_inline_xattrs_btrfs``
:Description: The maximum number of XATTRs stored in the Btrfs file system per object.
Only used if ``filestore_max_inline_xattrs`` == 0.
:Description: Defines the maximum number of XATTRs per object that can be stored in the Btrfs file system.
This setting is used only if ``filestore_max_inline_xattrs`` == 0.
:Type: 32-bit Integer
:Required: No
:Default: ``10``
@ -101,8 +112,8 @@ threshold is reached.
``filestore_max_inline_xattrs_other``
:Description: The maximum number of XATTRs stored in other file systems per object.
Only used if ``filestore_max_inline_xattrs`` == 0.
:Description: Defines the maximum number of XATTRs per object that can be stored in other file systems.
This setting is used only if ``filestore_max_inline_xattrs`` == 0.
:Type: 32-bit Integer
:Required: No
:Default: ``2``
@ -112,18 +123,19 @@ threshold is reached.
Synchronization Intervals
=========================
Filestore needs to periodically quiesce writes and synchronize the
file system, which creates a consistent commit point. It can then free journal
entries up to the commit point. Synchronizing more frequently tends to reduce
the time required to perform synchronization, and reduces the amount of data
that needs to remain in the journal. Less frequent synchronization allows the
backing file system to coalesce small writes and metadata updates more
optimally, potentially resulting in more efficient synchronization at the
expense of potentially increasing tail latency.
Filestore must periodically quiesce writes and synchronize the file system.
Each synchronization creates a consistent commit point. When the commit point
is created, Filestore is able to free all journal entries up to that point.
More-frequent synchronization tends to reduce both synchronization time and
the amount of data that needs to remain in the journal. Less-frequent
synchronization allows the backing file system to coalesce small writes and
metadata updates, potentially increasing synchronization
efficiency but also potentially increasing tail latency.
``filestore_max_sync_interval``
:Description: The maximum interval in seconds for synchronizing Filestore.
:Description: Defines the maximum interval (in seconds) for synchronizing Filestore.
:Type: Double
:Required: No
:Default: ``5``
@ -131,7 +143,7 @@ expense of potentially increasing tail latency.
``filestore_min_sync_interval``
:Description: The minimum interval in seconds for synchronizing Filestore.
:Description: Defines the minimum interval (in seconds) for synchronizing Filestore.
:Type: Double
:Required: No
:Default: ``.01``
@ -143,14 +155,14 @@ Flusher
=======
The Filestore flusher forces data from large writes to be written out using
``sync_file_range`` before the sync in order to (hopefully) reduce the cost of
the eventual sync. In practice, disabling 'filestore_flusher' seems to improve
performance in some cases.
``sync_file_range`` prior to the synchronization.
Ideally, this action reduces the cost of the eventual synchronization. In practice, however, disabling
'filestore_flusher' seems in some cases to improve performance.
``filestore_flusher``
:Description: Enables the filestore flusher.
:Description: Enables the Filestore flusher.
:Type: Boolean
:Required: No
:Default: ``false``
@ -159,7 +171,7 @@ performance in some cases.
``filestore_flusher_max_fds``
:Description: Sets the maximum number of file descriptors for the flusher.
:Description: Defines the maximum number of file descriptors for the flusher.
:Type: Integer
:Required: No
:Default: ``512``
@ -177,7 +189,7 @@ performance in some cases.
``filestore_fsync_flushes_journal_data``
:Description: Flush journal data during file system synchronization.
:Description: Flushes journal data during file-system synchronization.
:Type: Boolean
:Required: No
:Default: ``false``
@ -188,11 +200,11 @@ performance in some cases.
Queue
=====
The following settings provide limits on the size of the Filestore queue.
The following settings define limits on the size of the Filestore queue:
``filestore_queue_max_ops``
:Description: Defines the maximum number of in progress operations the file store accepts before blocking on queuing new operations.
:Description: Defines the maximum number of in-progress operations that Filestore accepts before it blocks the queueing of any new operations.
:Type: Integer
:Required: No. Minimal impact on performance.
:Default: ``50``
@ -200,23 +212,20 @@ The following settings provide limits on the size of the Filestore queue.
``filestore_queue_max_bytes``
:Description: The maximum number of bytes for an operation.
:Description: Defines the maximum number of bytes permitted per operation.
:Type: Integer
:Required: No
:Default: ``100 << 20``
.. index:: filestore; timeouts
Timeouts
========
``filestore_op_threads``
:Description: The number of file system operation threads that execute in parallel.
:Description: Defines the number of file-system operation threads that execute in parallel.
:Type: Integer
:Required: No
:Default: ``2``
@ -224,7 +233,7 @@ Timeouts
``filestore_op_thread_timeout``
:Description: The timeout for a file system operation thread (in seconds).
:Description: Defines the timeout (in seconds) for a file-system operation thread.
:Type: Integer
:Required: No
:Default: ``60``
@ -232,7 +241,7 @@ Timeouts
``filestore_op_thread_suicide_timeout``
:Description: The timeout for a commit operation before cancelling the commit (in seconds).
:Description: Defines the timeout (in seconds) for a commit operation before the commit is cancelled.
:Type: Integer
:Required: No
:Default: ``180``
@ -246,17 +255,17 @@ B-Tree Filesystem
``filestore_btrfs_snap``
:Description: Enable snapshots for a ``btrfs`` filestore.
:Description: Enables snapshots for a ``btrfs`` Filestore.
:Type: Boolean
:Required: No. Only used for ``btrfs``.
:Required: No. Used only for ``btrfs``.
:Default: ``true``
``filestore_btrfs_clone_range``
:Description: Enable cloning ranges for a ``btrfs`` filestore.
:Description: Enables cloning ranges for a ``btrfs`` Filestore.
:Type: Boolean
:Required: No. Only used for ``btrfs``.
:Required: No. Used only for ``btrfs``.
:Default: ``true``
@ -268,7 +277,7 @@ Journal
``filestore_journal_parallel``
:Description: Enables parallel journaling, default for Btrfs.
:Description: Enables parallel journaling, default for ``btrfs``.
:Type: Boolean
:Required: No
:Default: ``false``
@ -276,7 +285,7 @@ Journal
``filestore_journal_writeahead``
:Description: Enables writeahead journaling, default for XFS.
:Description: Enables write-ahead journaling, default for XFS.
:Type: Boolean
:Required: No
:Default: ``false``
@ -284,7 +293,7 @@ Journal
``filestore_journal_trailing``
:Description: Deprecated, never use.
:Description: Deprecated. **Never use.**
:Type: Boolean
:Required: No
:Default: ``false``
@ -296,8 +305,8 @@ Misc
``filestore_merge_threshold``
:Description: Min number of files in a subdir before merging into parent
NOTE: A negative value means to disable subdir merging
:Description: Defines the minimum number of files permitted in a subdirectory before the subdirectory is merged into its parent directory.
NOTE: A negative value means that subdirectory merging is disabled.
:Type: Integer
:Required: No
:Default: ``-10``
@ -306,8 +315,8 @@ Misc
``filestore_split_multiple``
:Description: ``(filestore_split_multiple * abs(filestore_merge_threshold) + (rand() % filestore_split_rand_factor)) * 16``
is the maximum number of files in a subdirectory before
splitting into child directories.
is the maximum number of files permitted in a subdirectory
before the subdirectory is split into child directories.
:Type: Integer
:Required: No
@ -317,10 +326,10 @@ Misc
``filestore_split_rand_factor``
:Description: A random factor added to the split threshold to avoid
too many (expensive) Filestore splits occurring at once. See
``filestore_split_multiple`` for details.
This can only be changed offline for an existing OSD,
via the ``ceph-objectstore-tool apply-layout-settings`` command.
too many (expensive) Filestore splits occurring at the same time.
For details, see ``filestore_split_multiple``.
To change this setting for an existing OSD, it is necessary to take the OSD
offline before running the ``ceph-objectstore-tool apply-layout-settings`` command.
:Type: Unsigned 32-bit Integer
:Required: No
@ -329,7 +338,7 @@ Misc
``filestore_update_to``
:Description: Limits Filestore auto upgrade to specified version.
:Description: Limits automatic upgrades to a specified version of Filestore. Useful in cases in which you want to avoid upgrading to a specific version.
:Type: Integer
:Required: No
:Default: ``1000``
@ -337,7 +346,7 @@ Misc
``filestore_blackhole``
:Description: Drop any new transactions on the floor.
:Description: Drops any new transactions on the floor, similar to redirecting to NULL.
:Type: Boolean
:Required: No
:Default: ``false``
@ -345,7 +354,7 @@ Misc
``filestore_dump_file``
:Description: File onto which store transaction dumps.
:Description: Defines the file that transaction dumps are stored on.
:Type: Boolean
:Required: No
:Default: ``false``
@ -353,7 +362,7 @@ Misc
``filestore_kill_at``
:Description: inject a failure at the n'th opportunity
:Description: Injects a failure at the *n*\th opportunity.
:Type: String
:Required: No
:Default: ``false``
@ -361,8 +370,7 @@ Misc
``filestore_fail_eio``
:Description: Fail/Crash on eio.
:Description: Fail/Crash on EIO.
:Type: Boolean
:Required: No
:Default: ``true``

1
doc/rados/configuration/storage-devices.rst
View File

@ -25,6 +25,7 @@ There are several Ceph daemons in a storage cluster:
additional monitoring and providing interfaces to external
monitoring and management systems.
.. _rados_config_storage_devices_osd_backends:
OSD Back Ends
=============

2
doc/rados/operations/bluestore-migration.rst
View File

@ -1,3 +1,5 @@
.. _rados_operations_bluestore_migration:
=====================
BlueStore Migration
=====================