mirror of
https://github.com/ceph/ceph
synced 2025-01-17 16:34:22 +00:00
doc/radosgw/qat-accel: Update and Add QATlib information
Signed-off-by: Feng, Hualong <hualong.feng@intel.com>
This commit is contained in:
Feng, Hualong
Zac Dover
parent
e20e5b727b
commit
98723d51b9
@ -33,13 +33,20 @@ QAT Environment Setup
|
||||
encryption and compression services. And QAT driver in kernel space have to
|
||||
be loaded to drive the hardware.
|
||||
|
||||
The driver package can be downloaded from `Intel Quickassist Technology`_.
|
||||
The out-of-tree QAT driver package can be downloaded from `Intel Quickassist
|
||||
Technology`_.
|
||||
|
||||
2. The implementation for QAT based encryption is directly base on QAT API which
|
||||
is included the driver package. But QAT support for compression depends on
|
||||
QATzip project, which is a user space library which builds on top of the QAT
|
||||
API. Currently, QATzip speeds up gzip compression and decompression at the
|
||||
time of writing.
|
||||
The QATlib can be downloaded from `qatlib`_, which is used for the in-tree QAT
|
||||
driver.
|
||||
|
||||
.. note::
|
||||
The out-of-tree QAT driver is gradually turning to intree driver+QATlib.
|
||||
|
||||
2. The implementation of QAT-based encryption is directly based on the QAT API,
|
||||
which is included the driver package. However, QAT support for compression
|
||||
depends on the QATzip project, which is a userspace library that builds on
|
||||
top of the QAT API. At the time of writing (July 2024), QATzip speeds up
|
||||
gzip compression and decompression.
|
||||
|
||||
See `QATzip`_.
|
||||
|
||||
@ -48,36 +55,40 @@ Implementation
|
||||
1. QAT based Encryption for RGW
|
||||
|
||||
`OpenSSL support for RGW encryption`_ has been merged into Ceph, and Intel also
|
||||
provides one `QAT Engine`_ for OpenSSL. So, theoretically speaking, QAT based
|
||||
encryption in Ceph can be directly supported through OpenSSl+QAT Engine.
|
||||
provides one `QAT Engine`_ for OpenSSL. Theoretically, QAT-based encryption in
|
||||
Ceph can be directly supported through OpenSSl+QAT Engine.
|
||||
|
||||
But the QAT Engine for OpenSSL currently supports chained operations only, and
|
||||
so Ceph will not be able to utilize QAT hardware feature for crypto operations
|
||||
based on OpenSSL crypto plugin. As a result, one QAT plugin based on native
|
||||
QAT API is added into crypto framework.
|
||||
However, the QAT Engine for OpenSSL currently supports only chained operations,
|
||||
which means that Ceph will not be able to utilize QAT hardware features for
|
||||
crypto operations based on the OpenSSL crypto plugin. As a result, one QAT plugin
|
||||
based on native QAT API is added into the crypto framework.
|
||||
|
||||
2. QAT Support for Compression
|
||||
|
||||
As mentioned above, QAT support for compression is based on QATzip library in
|
||||
user space, which is designed to take full advantage of the performance provided
|
||||
by QuickAssist Technology. Unlike QAT based encryption, QAT based compression
|
||||
is supported through a tool class for QAT acceleration rather than a compressor
|
||||
plugin. The common tool class can transparently accelerate the existing compression
|
||||
types, but only zlib compressor can be supported at the time of writing. So
|
||||
user is allowed to use it to speed up zlib compressor as long as the QAT
|
||||
hardware is available and QAT is capable to handle it.
|
||||
As mentioned above, QAT support for compression is based on the QATzip library
|
||||
in user space, which is designed to take full advantage of the performance that
|
||||
QuickAssist Technology provides. Unlike QAT-based encryption, QAT-based
|
||||
compression is supported through a tool class for QAT acceleration rather than
|
||||
a compressor plugin. This common tool class can transparently accelerate the
|
||||
existing compression types, but only zlib compressor is supported at the
|
||||
time of writing. This means that the user can use this tool class to speed up
|
||||
zlib compressor if the QAT hardware is available and if QAT is capable of
|
||||
handling it.
|
||||
|
||||
Configuration
|
||||
=============
|
||||
#. Prerequisites
|
||||
|
||||
Make sure the QAT driver with version v1.7.L.4.14.0 or higher has been installed.
|
||||
Remember to set an environment variable "ICP_ROOT" for your QAT driver package
|
||||
root directory.
|
||||
**For out-of-tree QAT**
|
||||
|
||||
To enable the QAT based encryption and compression, user needs to modify the QAT
|
||||
configuration files. For example, for Intel QuickAssist Adapter 8970 product, revise
|
||||
c6xx_dev0/1/2.conf in the directory ``/etc/`` and keep them the same, e.g.:
|
||||
Make sure the out-of-tree QAT driver with version v1.7.L.4.14.0 or higher
|
||||
has been installed. Remember to set an environment variable ``ICP_ROOT``
|
||||
for your QAT driver package root directory.
|
||||
|
||||
To enable the QAT based encryption and compression, the user must modify the
|
||||
QAT configuration files. For example, for the Intel QuickAssist Adapter 8970
|
||||
product, revise ``c6xx_dev0/1/2.conf`` in the directory ``/etc/`` and keep them
|
||||
the same. For example:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
@ -101,51 +112,121 @@ Configuration
|
||||
# List of core affinities
|
||||
Dc0CoreAffinity = 0
|
||||
|
||||
#. QAT based Encryption for RGW
|
||||
**For in-tree QAT**
|
||||
|
||||
The CMake option ``WITH_QAT=ON`` must be configured. If you build Ceph from
|
||||
There are some prerequisites for using QATlib. Make sure that your system
|
||||
meets the `QATlib System Requirements`_ .
|
||||
|
||||
* To properly use the QATlib library, the Intel VT-d and SR-IOV parameters
|
||||
must be enabled in the platform BIOS.
|
||||
* Some qatlib features require a recent kernel driver or firmware version.
|
||||
See `QATlib Kernel Driver Releases`_.
|
||||
* The supported platform contains a 4xxx Intel Communications device or
|
||||
newer.
|
||||
* The ``intel_iommu`` parameter must be enabled. Verify that this setting is
|
||||
enabled by running the following commands:
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
cat /proc/cmdline | grep intel_iommu=on
|
||||
sudo sh -c 'echo "@qat - memlock 204800" >> /etc/security/limits.conf'
|
||||
sudo su -l $USER
|
||||
|
||||
For configuration and Tuning see `QATlib Configuration and Tuning`_.
|
||||
|
||||
#. QAT-based Encryption for RGW
|
||||
|
||||
The CMake option ``WITH_QATDRV=ON`` must be set. If you build Ceph from
|
||||
source code (see: :ref:`build-ceph`), navigate to your cloned Ceph repository
|
||||
and execute the following:
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
cd ceph
|
||||
./do_cmake.sh -DWITH_QAT=ON
|
||||
./do_cmake.sh -DWITH_QATDRV=ON
|
||||
cd build
|
||||
ininja
|
||||
|
||||
.. note::
|
||||
The section name of the QAT configuration files must be ``CEPH`` since
|
||||
the section name is set as "CEPH" in Ceph crypto source code.
|
||||
.. note:: The section name of the QAT configuration files must be ``CEPH``,
|
||||
because the section name is set as ``CEPH`` in the Ceph crypto source code.
|
||||
|
||||
Then, edit the Ceph configuration file to make use of QAT based crypto plugin::
|
||||
Edit the Ceph configuration file (usually ``ceph.conf``) to make use of the
|
||||
QAT-based crypto plugin::
|
||||
|
||||
plugin crypto accelerator = crypto_qat
|
||||
|
||||
#. QAT Support for Compression
|
||||
|
||||
Before starting, make sure both QAT driver and `QATzip`_ have been installed. Besides
|
||||
"ICP_ROOT", remember to set the environment variable "QZ_ROOT" for the root directory
|
||||
of your QATzip source tree.
|
||||
**For out-of-tree QAT**
|
||||
|
||||
The following CMake options have to be configured to trigger QAT based compression
|
||||
when building Ceph:
|
||||
For the out-of-tree QAT driver package, before starting ensure that both QAT
|
||||
driver and `QATzip`_ have been installed. Besides ``ICP_ROOT``, remember to
|
||||
set the environment variable ``QZ_ROOT`` to the root directory of your QATzip
|
||||
source tree.
|
||||
|
||||
The following CMake options have to be configured to trigger QAT-based
|
||||
compression when building Ceph:
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
./do_cmake.sh -DWITH_QAT=ON -DWITH_QATZIP=ON
|
||||
./do_cmake.sh -DWITH_QATDRV=ON -DWITH_QATZIP=ON -DWITH_SYSTEM_QATZIP=ON -DWITH_QATLIB=OFF
|
||||
|
||||
Then, set an environment variable to clarify the section name of User Process Instance
|
||||
Section in QAT configuration files, e.g.:
|
||||
Set an environment variable to clarify the section name of User Process
|
||||
Instance Section in the QAT configuration files. For example:
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
export QAT_SECTION_NAME=CEPH
|
||||
|
||||
Next, edit the Ceph configuration file to enable QAT support for compression::
|
||||
**For in-tree QAT**
|
||||
|
||||
For in-tree QAT, make sure that your system meets the `QATlib System
|
||||
Requirements`_. QATlib can be installed from pre-built packages or from
|
||||
source code. See `QATlib Installation`_ . After QATlib is installed, you
|
||||
can run ``cpa_sample_code`` to check if the QAT environment is OK.
|
||||
|
||||
If you are using QATlib source code, Ceph cmake build has the compilation
|
||||
options of qatlib and qatzip enabled by default. Our normal compilation
|
||||
already includes QAT-compressor-related code.
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
./do_cmake.sh
|
||||
|
||||
If you are using pre-built packages installed on the system, the following
|
||||
CMake options have to be configured when building Ceph:
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
./do_cmake.sh -DWITH_SYSTEM_QATLIB=ON -DWITH_SYSTEM_QATZIP=ON
|
||||
|
||||
|
||||
**For both out-of-tree QAT and in-tree QAT**
|
||||
|
||||
Edit the Ceph configuration file (usually ``ceph.conf``) to enable QAT
|
||||
support for *zlib* compression::
|
||||
|
||||
qat compressor enabled=true
|
||||
|
||||
Set the RGW compression method:
|
||||
|
||||
.. prompt:: bash $
|
||||
|
||||
# for storage class(STANDARD)
|
||||
radosgw-admin zone placement modify --rgw-zone=default --placement-id=default-placement --compression=zlib
|
||||
# or create a new storage class(COLD) and define data pool(default.rgw.cold.data)
|
||||
radosgw-admin zonegroup placement add --rgw-zonegroup default --placement-id default-placement --storage-class COLD
|
||||
radosgw-admin zone placement add --rgw-zone default --placement-id default-placement --storage-class COLD --compression zlib --data-pool default.rgw.cold.data
|
||||
|
||||
CONFIG REFERENCE
|
||||
================
|
||||
The following QAT-related settings can be added to the Ceph configuration file
|
||||
(usually `ceph.conf`) under the ``[client.rgw.{instance-name}]`` section.
|
||||
|
||||
.. confval:: qat_compressor_session_max_number
|
||||
.. confval:: qat_compressor_busy_polling
|
||||
|
||||
|
||||
|
||||
.. _QAT Support for Compression: https://github.com/ceph/ceph/pull/19714
|
||||
.. _QAT based Encryption for RGW: https://github.com/ceph/ceph/pull/19386
|
||||
@ -153,3 +234,9 @@ Configuration
|
||||
.. _QATzip: https://github.com/intel/QATzip
|
||||
.. _OpenSSL support for RGW encryption: https://github.com/ceph/ceph/pull/15168
|
||||
.. _QAT Engine: https://github.com/intel/QAT_Engine
|
||||
.. _qatlib: https://github.com/intel/qatlib
|
||||
.. _QATlib User's Guide: https://intel.github.io/quickassist/qatlib/index.html
|
||||
.. _QATlib System Requirements: https://intel.github.io/quickassist/qatlib/requirements.html
|
||||
.. _QATlib Installation: https://intel.github.io/quickassist/qatlib/install.html
|
||||
.. _QATlib Configuration and Tuning: https://intel.github.io/quickassist/qatlib/configuration.html
|
||||
.. _QATlib Kernel Driver Releases: https://intel.github.io/quickassist/RN/In-Tree/in_tree_firmware_RN.html#qat-kernel-driver-releases-features
|
||||
|
||||
Loading…
Reference in New Issue
Block a user