mirror of
https://github.com/ceph/ceph
synced 2024-12-18 01:16:55 +00:00
doc: Added librados introduction doc. Still wip.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
This commit is contained in:
John Wilkins
parent
33e78a4ad9
commit
7d5674cf78
560
doc/rados/api/librados-intro.rst
Normal file
560
doc/rados/api/librados-intro.rst
Normal file
@ -0,0 +1,560 @@
|
||||
==========================
|
||||
Introduction to librados
|
||||
==========================
|
||||
|
||||
The :term:`Ceph Storage Cluster` provides the basic storage service that allows
|
||||
:term:`Ceph` to uniquely deliver **object, block, and file storage** in one
|
||||
unified system. However, you are not limited to using the RESTful, block, or
|
||||
POSIX interfaces. Based upon :abbr:`RADOS (Reliable Autonomic Distributed Object
|
||||
Store)`, the ``librados`` API, enables you to create your own interface to the
|
||||
Ceph Storage Cluster. In fact, :term:`Ceph Object Storage`, :term:`Ceph Block
|
||||
Device` and :term:`Ceph Filesystem` all use ``librados``, or the same general
|
||||
functionality of ``librados`` to access the Ceph Storage Cluster.
|
||||
|
||||
The ``librados`` API enables you to interact with the two types of daemons in
|
||||
the Ceph Storage Cluster:
|
||||
|
||||
- The :term:`Ceph Monitor`, which maintains a master copy of the cluster map.
|
||||
- The :term:`Ceph OSD Daemon` (OSD), which stores data as objects on a storage node.
|
||||
|
||||
.. ditaa::
|
||||
+---------------------------------+
|
||||
| Ceph Storage Cluster Protocol |
|
||||
| (librados) |
|
||||
+---------------------------------+
|
||||
+---------------+ +---------------+
|
||||
| OSDs | | Monitors |
|
||||
+---------------+ +---------------+
|
||||
|
||||
This guide provides a high-level introduction to using ``librados``.
|
||||
Refer to :doc:`../../architecture` for additional details of the Ceph
|
||||
Storage Cluster. To use the API, you need a running Ceph Storage Cluster.
|
||||
See `Installation (Quick)`_ for details.
|
||||
|
||||
|
||||
Step 1: Getting librados
|
||||
========================
|
||||
|
||||
The ``librados`` API is written in C, with additional bindings for C++, Python
|
||||
and Java. Your client app needs to import ``librados``, which means it must be
|
||||
installed on your client host first.
|
||||
|
||||
|
||||
Getting ``librados`` for C/C++ and Python
|
||||
-----------------------------------------
|
||||
|
||||
To install ``librados`` for C/C++ and Python, execute the following for
|
||||
Debian/Ubuntu distributions::
|
||||
|
||||
sudo apt-get install librados-dev
|
||||
|
||||
For CentOS/RHEL distributions, execute the following::
|
||||
|
||||
sudo yum install ceph-devel
|
||||
|
||||
Once you've installed ``librados`` for developers, you can find the required
|
||||
headers for C/C++ under ``/usr/include/rados``. ::
|
||||
|
||||
ls /usr/include/rados
|
||||
|
||||
For Python, you can find the required library under ``/usr/share/pyshared``. ::
|
||||
|
||||
ls /usr/share/pyshared
|
||||
|
||||
|
||||
Getting ``librados`` for Java
|
||||
-----------------------------
|
||||
|
||||
To install ``librados`` for Java, you need to execute the following procedure:
|
||||
|
||||
#. Install ``jna.jar``. For Debian/Ubuntu, execute::
|
||||
|
||||
sudo apt-get install libjna-java
|
||||
|
||||
For CentOS/RHEL, execute::
|
||||
|
||||
sudo yum install jna
|
||||
|
||||
The JAR files are located in ``/usr/share/java``.
|
||||
|
||||
#. Clone the ``rados-java`` repository::
|
||||
|
||||
git clone --recursive https://github.com/ceph/rados-java.git
|
||||
|
||||
#. Build the ``rados-java`` repository::
|
||||
|
||||
cd rados-java
|
||||
ant
|
||||
|
||||
The JAR file is located under ``rados-java/target``.
|
||||
|
||||
#. Copy the JAR for RADOS to a common location (e.g., ``/usr/share/java``) and
|
||||
ensure that it and the JNA JAR are in your JVM's classpath. For example::
|
||||
|
||||
sudo cp target/rados-0.1.3.jar /usr/share/java/rados-0.1.3.jar
|
||||
sudo ln -s /usr/share/java/jna-3.2.7.jar /usr/lib/jvm/default-java/jre/lib/ext/jna-3.2.7.jar
|
||||
sudo ln -s /usr/share/java/rados-0.1.3.jar /usr/lib/jvm/default-java/jre/lib/ext/rados-0.1.3.jar
|
||||
|
||||
To build the documentation, execute the following::
|
||||
|
||||
ant docs
|
||||
|
||||
|
||||
|
||||
Step 2: Configuring a Cluster Handle
|
||||
====================================
|
||||
|
||||
A :term:`Ceph Client`, via ``librados``, interacts directly with OSDs to store
|
||||
and retrieve data. To interact with OSDs in a manner that's substantially
|
||||
transparent to the client app, the client app must invoke ``librados`` to
|
||||
retrieve the cluster map. Ceph Clients retrieve a :term:`Cluster Map` from a
|
||||
Ceph Monitor, and write objects to pools. Ceph's CRUSH algorithm determines how
|
||||
Ceph will place data. ``librados`` will do this for your client app, so your
|
||||
client app doesn't need to learn about the topology of the cluster.
|
||||
|
||||
.. ditaa::
|
||||
+--------+ Retrieves +---------------+
|
||||
| Client |------------>| Cluster Map |
|
||||
+--------+ +---------------+
|
||||
|
|
||||
v Writes
|
||||
/-----\
|
||||
| obj |
|
||||
\-----/
|
||||
| To
|
||||
v
|
||||
+--------+ +---------------+
|
||||
| Pool |---------->| CRUSH Ruleset |
|
||||
+--------+ Selects +---------------+
|
||||
|
||||
|
||||
The Ceph Storage Cluster handle encapsulates the client configuration, including:
|
||||
|
||||
- The `user ID`_
|
||||
- The authentication key
|
||||
- The monitor ID and address
|
||||
- Logging levels
|
||||
- Debugging levels
|
||||
|
||||
Create a Ceph client source file and import RADOS and any other relevant
|
||||
libraries for your app. Then, create a cluster handle that your app will use to
|
||||
connect to the storage cluster. To connect to the cluster, the handle must have
|
||||
a monitor address, a username and an authentication key (cephx is enabled by
|
||||
default).
|
||||
|
||||
.. tip:: Talking to different Ceph Storage Clusters – or to the same cluster
|
||||
with different users – requires different cluster handles.
|
||||
|
||||
RADOS provides a number of ways for you to set the minimum required values. For
|
||||
the monitor and encryption key settings, an easy way to handle them is to ensure
|
||||
that your Ceph configuration file contains a ``keyring`` path to a keyring file
|
||||
and at least one monitor address (e.g,. ``mon host``). Once you create the
|
||||
handle, you can read a Ceph configuration file to configure the handle. You can
|
||||
also pass arguments to your app and parse them with the function for parsing
|
||||
command line arguments (e.g., ``rados_conf_parse_argv()``), or parse Ceph
|
||||
environment variables (e.g., ``rados_conf_parse_env()``). Some wrappers may not
|
||||
implement convenience methods, so you may need to implement these capabilities.
|
||||
The following diagram provides a high-level flow for the initial connection.
|
||||
|
||||
|
||||
.. ditaa:: +---------+ +---------+
|
||||
| Client | | Monitor |
|
||||
+---------+ +---------+
|
||||
| |
|
||||
|-----+ create |
|
||||
| | cluster |
|
||||
|<----+ handle |
|
||||
| |
|
||||
|-----+ read |
|
||||
| | config |
|
||||
|<----+ file |
|
||||
| |
|
||||
| connect |
|
||||
|-------------->|
|
||||
| |
|
||||
|<--------------|
|
||||
| connected |
|
||||
| |
|
||||
|
||||
|
||||
Once connected, your app can invoke methods that require a cluster handle, but
|
||||
don't require an I/O context. For example, once you have a cluster handle and
|
||||
a connection, you can:
|
||||
|
||||
- Get cluster statistics
|
||||
- Use Pool Operation (exists, create, list, delete)
|
||||
- Get and set the configuration
|
||||
|
||||
The main difference in the various ``librados`` bindings is between C and the
|
||||
object-oriented binds for C++, Java and Python. The object-oriented bindings use
|
||||
objects to represent cluster handles, IO Contexts, iterators, exceptions, etc.
|
||||
|
||||
|
||||
C Example
|
||||
---------
|
||||
|
||||
For C, creating a simple cluster handle using the ``admin`` user, configuring
|
||||
it and connecting to the cluster might look something like this:
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <rados/librados.h>
|
||||
|
||||
main (const char argv**)
|
||||
{
|
||||
/* Declare the cluster handle. */
|
||||
rados_t cluster;
|
||||
int err;
|
||||
|
||||
/* Initialize the cluster handle with the "admin" user */
|
||||
err = rados_create(&cluster, "admin");
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Couldn't create the cluster handle! %s\n", argv[0], strerror(-err));
|
||||
exit(EXIT_FAILURE);
|
||||
} else {
|
||||
printf("\nCreated a cluster handle.\n");
|
||||
}
|
||||
|
||||
/* Read a Ceph configuration file to configure the cluster handle. */
|
||||
err = rados_conf_read_file(cluster, "/etc/ceph/ceph.conf");
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: cannot read config file: %s\n", argv[0], strerror(-err));
|
||||
exit(EXIT_FAILURE);
|
||||
} else {
|
||||
printf("\nRead the config file.\n");
|
||||
}
|
||||
|
||||
/* Read command line arguments */
|
||||
err = rados_conf_parse_argv(cluster, argc, argv);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: cannot parse command line arguments: %s\n", argv[0], strerror(-err));
|
||||
exit(EXIT_FAILURE);
|
||||
} else {
|
||||
printf("\nRead the command line arguments.\n");
|
||||
}
|
||||
|
||||
/* Connect to the cluster */
|
||||
err = rados_connect(cluster);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: cannot connect to cluster: %s\n", argv[0], strerror(-err));
|
||||
exit(EXIT_FAILURE);
|
||||
} else {
|
||||
printf("\nConnected to the cluster.\n");
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
Compile your client and be sure to include the ``rados`` library
|
||||
using ``-lrados``. For example::
|
||||
|
||||
gcc ceph-client.c -lrados -o ceph-client
|
||||
|
||||
|
||||
C++ Example
|
||||
-----------
|
||||
|
||||
For C++, a simple cluster handle using the ``admin`` user requires you to
|
||||
initialize a ``Rados`` cluster handle object:
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
#include <iostream>
|
||||
#include <string>
|
||||
#include <rados/librados.hpp>
|
||||
|
||||
int main(int argc, const char **argv)
|
||||
{
|
||||
int ret = 0;
|
||||
|
||||
/* Declare the cluster handle. */
|
||||
librados::Rados cluster;
|
||||
|
||||
/* Initialize the cluster handle with the "admin" user */
|
||||
{
|
||||
ret = cluster.init("admin");
|
||||
if (ret < 0) {
|
||||
std::cerr << "Couldn't initialize the cluster handle! error " << ret << std::endl;
|
||||
ret = EXIT_FAILURE;
|
||||
return 1;
|
||||
} else {
|
||||
std::cout << "Created a cluster handle." << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
/* Read a Ceph configuration file to configure the cluster handle. */
|
||||
{
|
||||
ret = cluster.conf_read_file("/etc/ceph/ceph.conf");
|
||||
if (ret < 0) {
|
||||
std::cerr << "Couldn't read the Ceph configuration file! error " << ret << std::endl;
|
||||
ret = EXIT_FAILURE;
|
||||
return 1;
|
||||
} else {
|
||||
std::cout << "Read the Ceph configuration file." << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
/* Read command line arguments */
|
||||
{
|
||||
ret = cluster.conf_parse_argv(argc, argv);
|
||||
if (ret < 0) {
|
||||
std::cerr << "Couldn't parse command line options! error " << ret << std::endl;
|
||||
ret = EXIT_FAILURE;
|
||||
return 1;
|
||||
} else {
|
||||
std::cout << "Parsed command line options." << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
/* Connect to the cluster */
|
||||
{
|
||||
ret = cluster.connect();
|
||||
if (ret < 0) {
|
||||
std::cerr << "Couldn't connect to cluster! error " << ret << std::endl;
|
||||
ret = EXIT_FAILURE;
|
||||
return 1;
|
||||
} else {
|
||||
std::cout << "Connected to the cluster." << std::endl;
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
|
||||
Compile the source; then, link the ``rados`` library in using ``-lrados``.
|
||||
For example::
|
||||
|
||||
g++ -g -c ceph-client.cc -o ceph-client.o
|
||||
g++ -g ceph-client.o -lrados -o ceph-client
|
||||
|
||||
|
||||
|
||||
Python Example
|
||||
--------------
|
||||
|
||||
Python uses the ``admin`` user and the ``ceph`` cluster name by default. The
|
||||
wrapper converts C-based errors into exceptions.
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import rados
|
||||
|
||||
try:
|
||||
cluster = rados.Rados()
|
||||
print "Created cluster handle."
|
||||
|
||||
cluster.conf_read_file("/etc/ceph/ceph.conf")
|
||||
print "Read Ceph configuration file."
|
||||
|
||||
cluster.connect()
|
||||
|
||||
except TypeError:
|
||||
print "Encountered an error."
|
||||
finally:
|
||||
print "Connected to the cluster."
|
||||
|
||||
|
||||
Java Example
|
||||
------------
|
||||
|
||||
Java requires you to specify the user ID, and uses the ``ceph`` cluster name by
|
||||
default . The wrapper converts C-based errors into exceptions.
|
||||
|
||||
.. code-block:: java
|
||||
|
||||
import com.ceph.rados.Rados;
|
||||
import com.ceph.rados.RadosException;
|
||||
|
||||
import java.io.File;
|
||||
|
||||
public class CephClient {
|
||||
public static void main (String args[]){
|
||||
|
||||
try {
|
||||
Rados cluster = new Rados("admin");
|
||||
System.out.println("Created a handle.");
|
||||
|
||||
File f = new File("/etc/ceph/ceph.conf");
|
||||
cluster.confReadFile(f);
|
||||
System.out.println("Read the configuration file.");
|
||||
|
||||
cluster.connect();
|
||||
System.out.println("Connected to the cluster.");
|
||||
|
||||
} catch (RadosException e) {
|
||||
System.out.println(e.getMessage() + ": " + e.getReturnValue());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
Step 3: Creating an I/O Context
|
||||
===============================
|
||||
|
||||
Once your app has a cluster handle and a connection to a Ceph Storage Cluster,
|
||||
you may create an I/O Context and begin reading and writing data. An I/O Context
|
||||
binds the connection to a specific pool. The user ID must have appropriate
|
||||
`CAPS`_ permissions to access the specified pool. For example, a user with read
|
||||
access but not write access will only be able to read data.
|
||||
|
||||
- Write/read data and extended attributes
|
||||
- List and iterate over objects and extended attributes
|
||||
- Shapshot pools, list snapshots, etc.
|
||||
|
||||
|
||||
.. ditaa:: +---------+ +---------+ +---------+
|
||||
| Client | | Monitor | | OSD |
|
||||
+---------+ +---------+ +---------+
|
||||
| | |
|
||||
|-----+ create | |
|
||||
| | I/O | |
|
||||
|<----+ context | |
|
||||
| | |
|
||||
| write data | |
|
||||
|---------------+-------------->|
|
||||
| | |
|
||||
| write ack | |
|
||||
|<--------------+---------------|
|
||||
| | |
|
||||
| write xattr | |
|
||||
|---------------+-------------->|
|
||||
| | |
|
||||
| xattr ack | |
|
||||
|<--------------+---------------|
|
||||
| | |
|
||||
| read data | |
|
||||
|---------------+-------------->|
|
||||
| | |
|
||||
| read ack | |
|
||||
|<--------------+---------------|
|
||||
|
||||
|
||||
RADOS enables you to interact both synchronously and asynchronously. Once your
|
||||
app has an I/O Context, read/write operations only require you to know the
|
||||
object/xattr name. The CRUSH algorithm encapsulated in ``librados`` uses the
|
||||
cluster map to identify the appropriate OSD. The OSDs handle the replication,
|
||||
as described in `Smart Daemons Enable Hyperscale`_. The mapping of objects to
|
||||
placement groups is also performed by the library as described in
|
||||
`Calculating PG IDs`_.
|
||||
|
||||
The following examples use the default ``data`` pool. However, you may also
|
||||
use the API to list pools, ensure they exist, or create and delete pools.
|
||||
|
||||
.. important:: Use caution when deleting pools with this API. If you delete
|
||||
a pool, the pool and ALL DATA in the pool will be lost.
|
||||
|
||||
|
||||
C Example
|
||||
---------
|
||||
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <rados/librados.h>
|
||||
|
||||
main (const char argv**)
|
||||
{
|
||||
/* Continued from previous C example, where cluster handle and
|
||||
connection are established. First declare an I/O Context. */
|
||||
|
||||
rados_ioctx_t io;
|
||||
char *poolname = "data";
|
||||
|
||||
err = rados_ioctx_create(cluster, poolname, &io);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: cannot open rados pool %s: %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_shutdown(cluster);
|
||||
exit(EXIT_FAILURE);
|
||||
} else {
|
||||
printf("\nCreated I/O context.\n");
|
||||
}
|
||||
|
||||
err = rados_write_full(io, "hw", "Hello World!", 12);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Cannot write object. %s %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_ioctx_destroy(io);
|
||||
rados_shutdown(cluster);
|
||||
exit(1);
|
||||
} else {
|
||||
printf("\nWrote \"Hello World\" to object \"hw\".\n");
|
||||
}
|
||||
|
||||
char xattr[5] = "en_US";
|
||||
err = rados_setxattr(io, "hw", "lang", xattr, 5);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Cannot write xattr. %s %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_ioctx_destroy(io);
|
||||
rados_shutdown(cluster);
|
||||
exit(1);
|
||||
} else {
|
||||
printf("\nWrote \"en_US\" to xattr \"lang\" for object \"hw\".\n");
|
||||
}
|
||||
|
||||
|
||||
char read_res[100];
|
||||
err = rados_read(io, "hw", read_res, 12, 0);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Cannot read object. %s %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_ioctx_destroy(io);
|
||||
rados_shutdown(cluster);
|
||||
exit(1);
|
||||
} else {
|
||||
printf("\nRead object \"hw\". The contents are:\n %s \n", read_res);
|
||||
}
|
||||
|
||||
char xattr_res[100];
|
||||
err = rados_getxattr(io, "hw", "lang", xattr_res, 5);
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Cannot read xattr. %s %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_ioctx_destroy(io);
|
||||
rados_shutdown(cluster);
|
||||
exit(1);
|
||||
} else {
|
||||
printf("\nRead xattr \"lang\" for object \"hw\". The contents are:\n %s \n", xattr_res);
|
||||
}
|
||||
|
||||
err = rados_rmxattr(io, "hw", "lang");
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Cannot remove xattr. %s %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_ioctx_destroy(io);
|
||||
rados_shutdown(cluster);
|
||||
exit(1);
|
||||
} else {
|
||||
printf("\nRemoved xattr \"lang\" for object \"hw\".\n");
|
||||
}
|
||||
|
||||
err = rados_remove(io, "hw");
|
||||
if (err < 0) {
|
||||
fprintf(stderr, "%s: Cannot remove object. %s %s\n", argv[0], poolname, strerror(-err));
|
||||
rados_ioctx_destroy(io);
|
||||
rados_shutdown(cluster);
|
||||
exit(1);
|
||||
} else {
|
||||
printf("\nRemoved object \"hw\".\n");
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Step 4: Closing Sessions
|
||||
========================
|
||||
|
||||
|
||||
|
||||
|
||||
.. _user ID: ../../operations/authentication#cephx-commandline-options
|
||||
.. _CAPS: ../../operations/auth-intro#ceph-authorization-caps
|
||||
.. _Installation (Quick): ../../../start
|
||||
.. _Smart Daemons Enable Hyperscale: ../../../architecture#smart-daemons-enable-hyperscale
|
||||
.. _Calculating PG IDs: ../../../architecture#calculating-pg-ids
|
||||
Loading…
Reference in New Issue
Block a user