Merge pull request #38520 from lixiaoy1/wip-doc-pwl

doc: add RBD persistent write-back cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd/rbd-operations.rst

@@ -9,6 +9,7 @@
    Exclusive Locking <rbd-exclusive-locks>
    Mirroring <rbd-mirroring>
    Live-Migration <rbd-live-migration>
-   Persistent Cache <rbd-persistent-cache>
+   Persistent Read-only Cache <rbd-persistent-read-only-cache>
+   Persistent Write-back Cache <rbd-persistent-write-back-cache>
    Config Settings (librbd) <rbd-config-ref/>
    RBD Replay <rbd-replay>

doc/rbd/rbd-persistent-read-only-cache.rst

@@ -1,8 +1,8 @@
-=======================
- RBD Persistent Cache
-=======================
+===============================
+ RBD Persistent Read-only Cache
+===============================
 
-.. index:: Ceph Block Device; Persistent Cache
+.. index:: Ceph Block Device; Persistent Read-only Cache
 
 Shared, Read-only Parent Image Cache
 ====================================

doc/rbd/rbd-persistent-write-back-cache.rst

@@ -0,0 +1,103 @@
+================================
+ RBD Persistent Write-back Cache
+================================
+
+.. index:: Ceph Block Device; Persistent Write-back Cache
+
+Persistent Write-back Cache
+===========================
+
+The persistent write-back cache provides a persistent, fault-tolerant write-back
+cache for librbd-based RBD clients.
+
+This cache uses a log-ordered write-back design which maintains checkpoints
+internally so that writes that get flushed back to the cluster are always
+crash consistent. Even if the client cache is lost entirely, the disk image is
+still consistent but the data will appear to be stale.
+
+Currently this cache uses PMEM devices as cache devices, and later SSDs will
+be supported.
+
+Usage
+=====
+
+The persistent write-back cache manages the cache data in a persistent device.
+It looks for and creates cache files in a configured directory, and then caches
+data in the file.
+
+The persistent write-back cache can't be enabled without the exclusive-lock
+feature. It tries to enable the write-back cache only when the exclusive lock
+is acquired.
+
+The cache provides two different persistence modes. In persistent-on-write mode,
+the writes are completed only when they are persisted to the cache device and
+will be readable after a crash. In persistent-on-flush mode, the writes are
+completed as soon as it no longer needs the caller's data buffer to complete
+the writes, but does not guarantee that writes will be readable after a crash.
+The data is persisted to the cache device when a flush request is received.
+
+Initially it defaults to the persistent-on-write mode and it switches to
+persistent-on-flush mode after the first flush request is received.
+
+Enable Persistent Write-back Cache
+========================================
+
+To enable the persistent write-back cache, the following Ceph settings
+need to be enabled.::
+
+        rbd rwl enabled = true
+        rbd plugins = pwl_cache
+
+Here are some cache configuration settings:
+
+- ``rbd_rwl_path`` A file folder to cache data. This folder must have DAX enabled
+  (see `DAX`_) to avoid performance degradation.
+
+- ``rbd_rwl_size`` The cache size per image.
+
+- ``rbd_rwl_log_periodic_stats`` This is a debug option. It is used to emit
+  periodic perf stats to the debug log.
+
+The above configurations can be set per-host, per-pool, per-image etc. Eg, to
+set per-host, add the overrides to the appropriate `section`_ in the host's
+``ceph.conf`` file. To set per-pool, per-image, etc, please refer to the
+``rbd config`` `commands`_.
+
+Cache Status
+------------
+
+The persistent write-back cache is enabled when the exclusive lock is acquired,
+and it is closed when the exclusive lock is released. To check the transient
+cache status, users may use the command ``rbd status``.  ::
+
+        rbd status {pool-name}/{image-name}
+
+The status of the cache is shown, including present, clean, cache size and the
+position.
+
+For example::
+
+        $ rbd status rbd/foo
+        Watchers: none
+        image cache state:
+        clean: false  size: 1 GiB  host: sceph9  path: /tmp
+
+
+Discard Cache
+-------------
+
+To discard a cache file with ``rbd``, specify the ``image-cache invalidate``
+option, the pool name and the image name.  ::
+
+        rbd image-cache invalidate {pool-name}/{image-name}
+
+The command removes the cache metadata of the corresponding image, disable
+the cache feature and deletes the local cache file if it exists.
+
+For example::
+
+        $ rbd image-cache invalidate rbd/foo
+
+.. _section: ../../rados/configuration/ceph-conf/#configuration-sections
+.. _commands: ../../man/8/rbd#commands
+.. _DAX: https://www.kernel.org/doc/Documentation/filesystems/dax.txt