mirror of https://github.com/ceph/ceph
201 lines
6.7 KiB
ReStructuredText
201 lines
6.7 KiB
ReStructuredText
===============================
|
|
RBD Persistent Read-only Cache
|
|
===============================
|
|
|
|
.. index:: Ceph Block Device; Persistent Read-only Cache
|
|
|
|
Shared, Read-only Parent Image Cache
|
|
====================================
|
|
|
|
`Cloned RBD images`_ usually modify only a small fraction of the parent
|
|
image. For example, in a VDI use-case, VMs are cloned from the same
|
|
base image and initially differ only by hostname and IP address. During
|
|
booting, all of these VMs read portions of the same parent
|
|
image data. If we have a local cache of the parent
|
|
image, this speeds up reads on the caching host. We also achieve
|
|
reduction of client-to-cluster network traffic.
|
|
RBD cache must be explicitly enabled in
|
|
``ceph.conf``. The ``ceph-immutable-object-cache`` daemon is responsible for
|
|
caching the parent content on the local disk, and future reads on that data
|
|
will be serviced from the local cache.
|
|
|
|
.. note:: RBD shared read-only parent image cache requires the Ceph Nautilus release or later.
|
|
|
|
.. ditaa::
|
|
|
|
+--------------------------------------------------------+
|
|
| QEMU |
|
|
+--------------------------------------------------------+
|
|
| librbd (cloned images) |
|
|
+-------------------+-+----------------------------------+
|
|
| librados | | ceph--immutable--object--cache |
|
|
+-------------------+ +----------------------------------+
|
|
| OSDs/Mons | | local cached parent image |
|
|
+-------------------+ +----------------------------------+
|
|
|
|
|
|
Enable RBD Shared Read-only Parent Image Cache
|
|
----------------------------------------------
|
|
|
|
To enable RBD shared read-only parent image cache, the following Ceph settings
|
|
need to added in the ``[client]`` `section`_ of your ``ceph.conf`` file::
|
|
|
|
rbd parent cache enabled = true
|
|
rbd plugins = parent_cache
|
|
|
|
Immutable Object Cache Daemon
|
|
=============================
|
|
|
|
Introduction and Generic Settings
|
|
---------------------------------
|
|
|
|
The ``ceph-immutable-object-cache`` daemon is responsible for caching parent
|
|
image content within its local caching directory. For better performance it's
|
|
recommended to use SSDs as the underlying storage.
|
|
|
|
The key components of the daemon are:
|
|
|
|
#. **Domain socket based IPC:** The daemon will listen on a local domain
|
|
socket on start up and wait for connections from librbd clients.
|
|
|
|
#. **LRU based promotion/demotion policy:** The daemon will maintain
|
|
in-memory statistics of cache-hits on each cache file. It will demote the
|
|
cold cache if capacity reaches to the configured threshold.
|
|
|
|
#. **File-based caching store:** The daemon will maintain a simple file
|
|
based cache store. On promotion the RADOS objects will be fetched from
|
|
RADOS cluster and stored in the local caching directory.
|
|
|
|
On opening each cloned rbd image, ``librbd`` will try to connect to the
|
|
cache daemon through its Unix domain socket. Once successfully connected,
|
|
``librbd`` will coordinate with the daemon on the subsequent reads.
|
|
If there's a read that's not cached, the daemon will promote the RADOS object
|
|
to local caching directory, so the next read on that object will be serviced
|
|
from cache. The daemon also maintains simple LRU statistics so that under
|
|
capacity pressure it will evict cold cache files as needed.
|
|
|
|
Here are some important cache configuration settings:
|
|
|
|
``immutable_object_cache_sock``
|
|
|
|
:Description: The path to the domain socket used for communication between
|
|
librbd clients and the ceph-immutable-object-cache daemon.
|
|
:Type: String
|
|
:Required: No
|
|
:Default: ``/var/run/ceph/immutable_object_cache_sock``
|
|
|
|
|
|
``immutable_object_cache_path``
|
|
|
|
:Description: The immutable object cache data directory.
|
|
:Type: String
|
|
:Required: No
|
|
:Default: ``/tmp/ceph_immutable_object_cache``
|
|
|
|
|
|
``immutable_object_cache_max_size``
|
|
|
|
:Description: The max size for immutable cache.
|
|
:Type: Size
|
|
:Required: No
|
|
:Default: ``1G``
|
|
|
|
|
|
``immutable_object_cache_watermark``
|
|
|
|
:Description: The high-water mark for the cache. The value is between (0, 1).
|
|
If the cache size reaches this threshold the daemon will start
|
|
to delete cold cache based on LRU statistics.
|
|
:Type: Float
|
|
:Required: No
|
|
:Default: ``0.9``
|
|
|
|
The ``ceph-immutable-object-cache`` daemon is available within the optional
|
|
``ceph-immutable-object-cache`` distribution package.
|
|
|
|
.. important:: ``ceph-immutable-object-cache`` daemon requires the ability to
|
|
connect RADOS clusters.
|
|
|
|
Running the Immutable Object Cache Daemon
|
|
-----------------------------------------
|
|
|
|
``ceph-immutable-object-cache`` daemon should use a unique Ceph user ID.
|
|
To `create a Ceph user`_, with ``ceph`` specify the ``auth get-or-create``
|
|
command, user name, monitor caps, and OSD caps::
|
|
|
|
ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'allow r' osd 'profile rbd-read-only'
|
|
|
|
The ``ceph-immutable-object-cache`` daemon can be managed by ``systemd`` by specifying the user
|
|
ID as the daemon instance::
|
|
|
|
systemctl enable ceph-immutable-object-cache@ceph-immutable-object-cache.{unique id}
|
|
|
|
The ``ceph-immutable-object-cache`` can also be run in foreground by ``ceph-immutable-object-cache`` command::
|
|
|
|
ceph-immutable-object-cache -f --log-file={log_path}
|
|
|
|
QOS Settings
|
|
------------
|
|
|
|
The immutable object cache supports throttling, controlled by the following settings:
|
|
|
|
``immutable_object_cache_qos_schedule_tick_min``
|
|
|
|
:Description: Minimum schedule tick for immutable object cache.
|
|
:Type: Milliseconds
|
|
:Required: No
|
|
:Default: ``50``
|
|
|
|
|
|
``immutable_object_cache_qos_iops_limit``
|
|
|
|
:Description: The desired immutable object cache IO operations limit per second.
|
|
:Type: Unsigned Integer
|
|
:Required: No
|
|
:Default: ``0``
|
|
|
|
|
|
``immutable_object_cache_qos_iops_burst``
|
|
|
|
:Description: The desired burst limit of immutable object cache IO operations.
|
|
:Type: Unsigned Integer
|
|
:Required: No
|
|
:Default: ``0``
|
|
|
|
|
|
``immutable_object_cache_qos_iops_burst_seconds``
|
|
|
|
:Description: The desired burst duration in seconds of immutable object cache IO operations.
|
|
:Type: Seconds
|
|
:Required: No
|
|
:Default: ``1``
|
|
|
|
|
|
``immutable_object_cache_qos_bps_limit``
|
|
|
|
:Description: The desired immutable object cache IO bytes limit per second.
|
|
:Type: Unsigned Integer
|
|
:Required: No
|
|
:Default: ``0``
|
|
|
|
|
|
``immutable_object_cache_qos_bps_burst``
|
|
|
|
:Description: The desired burst limit of immutable object cache IO bytes.
|
|
:Type: Unsigned Integer
|
|
:Required: No
|
|
:Default: ``0``
|
|
|
|
|
|
``immutable_object_cache_qos_bps_burst_seconds``
|
|
|
|
:Description: The desired burst duration in seconds of immutable object cache IO bytes.
|
|
:Type: Seconds
|
|
:Required: No
|
|
:Default: ``1``
|
|
|
|
.. _Cloned RBD Images: ../rbd-snapshot/#layering
|
|
.. _section: ../../rados/configuration/ceph-conf/#configuration-sections
|
|
.. _create a Ceph user: ../../rados/operations/user-management#add-a-user
|
|
|