mirror of https://github.com/ceph/ceph
416 lines
12 KiB
ReStructuredText
416 lines
12 KiB
ReStructuredText
===================
|
|
Librados (Python)
|
|
===================
|
|
|
|
The ``rados`` module is a thin Python wrapper for ``librados``.
|
|
|
|
Installation
|
|
============
|
|
|
|
To install Python libraries for Ceph, see `Getting librados for Python`_.
|
|
|
|
|
|
Getting Started
|
|
===============
|
|
|
|
You can create your own Ceph client using Python. The following tutorial will
|
|
show you how to import the Ceph Python module, connect to a Ceph cluster, and
|
|
perform object operations as a ``client.admin`` user.
|
|
|
|
.. note:: To use the Ceph Python bindings, you must have access to a
|
|
running Ceph cluster. To set one up quickly, see `Getting Started`_.
|
|
|
|
First, create a Python source file for your Ceph client. ::
|
|
:linenos:
|
|
|
|
sudo vim client.py
|
|
|
|
|
|
Import the Module
|
|
-----------------
|
|
|
|
To use the ``rados`` module, import it into your source file.
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
|
|
import rados
|
|
|
|
|
|
Configure a Cluster Handle
|
|
--------------------------
|
|
|
|
Before connecting to the Ceph Storage Cluster, create a cluster handle. By
|
|
default, the cluster handle assumes a cluster named ``ceph`` (i.e., the default
|
|
for deployment tools, and our Getting Started guides too), and a
|
|
``client.admin`` user name. You may change these defaults to suit your needs.
|
|
|
|
To connect to the Ceph Storage Cluster, your application needs to know where to
|
|
find the Ceph Monitor. Provide this information to your application by
|
|
specifying the path to your Ceph configuration file, which contains the location
|
|
of the initial Ceph monitors.
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
|
|
import rados, sys
|
|
|
|
#Create Handle Examples.
|
|
cluster = rados.Rados(conffile='ceph.conf')
|
|
cluster = rados.Rados(conffile=sys.argv[1])
|
|
cluster = rados.Rados(conffile = 'ceph.conf', conf = dict (keyring = '/path/to/keyring'))
|
|
|
|
Ensure that the ``conffile`` argument provides the path and file name of your
|
|
Ceph configuration file. You may use the ``sys`` module to avoid hard-coding the
|
|
Ceph configuration path and file name.
|
|
|
|
Your Python client also requires a client keyring. For this example, we use the
|
|
``client.admin`` key by default. If you would like to specify the keyring when
|
|
creating the cluster handle, you may use the ``conf`` argument. Alternatively,
|
|
you may specify the keyring path in your Ceph configuration file. For example,
|
|
you may add something like the following line to you Ceph configuration file::
|
|
|
|
keyring = /path/to/ceph.client.admin.keyring
|
|
|
|
For additional details on modifying your configuration via Python, see `Configuration`_.
|
|
|
|
|
|
Connect to the Cluster
|
|
----------------------
|
|
|
|
Once you have a cluster handle configured, you may connect to the cluster.
|
|
With a connection to the cluster, you may execute methods that return
|
|
information about the cluster.
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
:emphasize-lines: 7
|
|
|
|
import rados, sys
|
|
|
|
cluster = rados.Rados(conffile='ceph.conf')
|
|
print "\nlibrados version: " + str(cluster.version())
|
|
print "Will attempt to connect to: " + str(cluster.conf_get('mon initial members'))
|
|
|
|
cluster.connect()
|
|
print "\nCluster ID: " + cluster.get_fsid()
|
|
|
|
print "\n\nCluster Statistics"
|
|
print "=================="
|
|
cluster_stats = cluster.get_cluster_stats()
|
|
|
|
for key, value in cluster_stats.iteritems():
|
|
print key, value
|
|
|
|
|
|
By default, Ceph authentication is ``on``. Your application will need to know
|
|
the location of the keyring. The ``python-ceph`` module doesn't have the default
|
|
location, so you need to specify the keyring path. The easiest way to specify
|
|
the keyring is to add it to the Ceph configuration file. The following Ceph
|
|
configuration file example uses the ``client.admin`` keyring you generated with
|
|
``ceph-deploy``.
|
|
|
|
.. code-block:: ini
|
|
:linenos:
|
|
|
|
[global]
|
|
# ... elided configuration
|
|
keyring=/path/to/keyring/ceph.client.admin.keyring
|
|
|
|
|
|
Manage Pools
|
|
------------
|
|
|
|
When connected to the cluster, the ``Rados`` API allows you to manage pools. You
|
|
can list pools, check for the existence of a pool, create a pool and delete a
|
|
pool.
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
:emphasize-lines: 6, 13, 18, 25
|
|
|
|
print "\n\nPool Operations"
|
|
print "==============="
|
|
|
|
print "\nAvailable Pools"
|
|
print "----------------"
|
|
pools = cluster.list_pools()
|
|
|
|
for pool in pools:
|
|
print pool
|
|
|
|
print "\nCreate 'test' Pool"
|
|
print "------------------"
|
|
cluster.create_pool('test')
|
|
|
|
print "\nPool named 'test' exists: " + str(cluster.pool_exists('test'))
|
|
print "\nVerify 'test' Pool Exists"
|
|
print "-------------------------"
|
|
pools = cluster.list_pools()
|
|
|
|
for pool in pools:
|
|
print pool
|
|
|
|
print "\nDelete 'test' Pool"
|
|
print "------------------"
|
|
cluster.delete_pool('test')
|
|
print "\nPool named 'test' exists: " + str(cluster.pool_exists('test'))
|
|
|
|
|
|
|
|
Input/Output Context
|
|
--------------------
|
|
|
|
Reading from and writing to the Ceph Storage Cluster requires an input/output
|
|
context (ioctx). You can create an ioctx with the ``open_ioctx()`` or
|
|
``open_ioctx2()`` method of the ``Rados`` class. The ``ioctx_name`` parameter
|
|
is the name of the pool and ``pool_id`` is the ID of the pool you wish to use.
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
|
|
ioctx = cluster.open_ioctx('data')
|
|
|
|
|
|
or
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
|
|
ioctx = cluster.open_ioctx2(pool_id)
|
|
|
|
|
|
Once you have an I/O context, you can read/write objects, extended attributes,
|
|
and perform a number of other operations. After you complete operations, ensure
|
|
that you close the connection. For example:
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
|
|
print "\nClosing the connection."
|
|
ioctx.close()
|
|
|
|
|
|
Writing, Reading and Removing Objects
|
|
-------------------------------------
|
|
|
|
Once you create an I/O context, you can write objects to the cluster. If you
|
|
write to an object that doesn't exist, Ceph creates it. If you write to an
|
|
object that exists, Ceph overwrites it (except when you specify a range, and
|
|
then it only overwrites the range). You may read objects (and object ranges)
|
|
from the cluster. You may also remove objects from the cluster. For example:
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
:emphasize-lines: 2, 5, 8
|
|
|
|
print "\nWriting object 'hw' with contents 'Hello World!' to pool 'data'."
|
|
ioctx.write_full("hw", "Hello World!")
|
|
|
|
print "\n\nContents of object 'hw'\n------------------------\n"
|
|
print ioctx.read("hw")
|
|
|
|
print "\nRemoving object 'hw'"
|
|
ioctx.remove_object("hw")
|
|
|
|
|
|
Writing and Reading XATTRS
|
|
--------------------------
|
|
|
|
Once you create an object, you can write extended attributes (XATTRs) to
|
|
the object and read XATTRs from the object. For example:
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
:emphasize-lines: 2, 5
|
|
|
|
print "\n\nWriting XATTR 'lang' with value 'en_US' to object 'hw'"
|
|
ioctx.set_xattr("hw", "lang", "en_US")
|
|
|
|
print "\n\nGetting XATTR 'lang' from object 'hw'\n"
|
|
print ioctx.get_xattr("hw", "lang")
|
|
|
|
|
|
Listing Objects
|
|
---------------
|
|
|
|
If you want to examine the list of objects in a pool, you may
|
|
retrieve the list of objects and iterate over them with the object iterator.
|
|
For example:
|
|
|
|
.. code-block:: python
|
|
:linenos:
|
|
:emphasize-lines: 1, 6, 7
|
|
|
|
object_iterator = ioctx.list_objects()
|
|
|
|
while True :
|
|
|
|
try :
|
|
rados_object = object_iterator.next()
|
|
print "Object contents = " + rados_object.read()
|
|
|
|
except StopIteration :
|
|
break
|
|
|
|
The ``Object`` class provides a file-like interface to an object, allowing
|
|
you to read and write content and extended attributes. Object operations using
|
|
the I/O context provide additional functionality and asynchronous capabilities.
|
|
|
|
|
|
Cluster Handle API
|
|
==================
|
|
|
|
The ``Rados`` class provides an interface into the Ceph Storage Daemon.
|
|
|
|
|
|
Configuration
|
|
-------------
|
|
|
|
The ``Rados`` class provides methods for getting and setting configuration
|
|
values, reading the Ceph configuration file, and parsing arguments. You
|
|
do not need to be connected to the Ceph Storage Cluster to invoke the following
|
|
methods. See `Storage Cluster Configuration`_ for details on settings.
|
|
|
|
.. currentmodule:: rados
|
|
.. automethod:: Rados.conf_get(option)
|
|
.. automethod:: Rados.conf_set(option, val)
|
|
.. automethod:: Rados.conf_read_file(path=None)
|
|
.. automethod:: Rados.conf_parse_argv(args)
|
|
.. automethod:: Rados.version()
|
|
|
|
|
|
Connection Management
|
|
---------------------
|
|
|
|
Once you configure your cluster handle, you may connect to the cluster, check
|
|
the cluster ``fsid``, retrieve cluster statistics, and disconnect (shutdown)
|
|
from the cluster. You may also assert that the cluster handle is in a particular
|
|
state (e.g., "configuring", "connecting", etc.).
|
|
|
|
.. automethod:: Rados.connect(timeout=0)
|
|
.. automethod:: Rados.shutdown()
|
|
.. automethod:: Rados.get_fsid()
|
|
.. automethod:: Rados.get_cluster_stats()
|
|
|
|
.. documented manually because it raises warnings because of *args usage in the
|
|
.. signature
|
|
|
|
.. py:class:: Rados
|
|
|
|
.. py:method:: require_state(*args)
|
|
|
|
Checks if the Rados object is in a special state
|
|
|
|
:param args: Any number of states to check as separate arguments
|
|
:raises: :class:`RadosStateError`
|
|
|
|
|
|
Pool Operations
|
|
---------------
|
|
|
|
To use pool operation methods, you must connect to the Ceph Storage Cluster
|
|
first. You may list the available pools, create a pool, check to see if a pool
|
|
exists, and delete a pool.
|
|
|
|
.. automethod:: Rados.list_pools()
|
|
.. automethod:: Rados.create_pool(pool_name, auid=None, crush_rule=None)
|
|
.. automethod:: Rados.pool_exists()
|
|
.. automethod:: Rados.delete_pool(pool_name)
|
|
|
|
|
|
|
|
Input/Output Context API
|
|
========================
|
|
|
|
To write data to and read data from the Ceph Object Store, you must create
|
|
an Input/Output context (ioctx). The `Rados` class provides `open_ioctx()`
|
|
and `open_ioctx2()` methods. The remaining ``ioctx`` operations involve
|
|
invoking methods of the `Ioctx` and other classes.
|
|
|
|
.. automethod:: Rados.open_ioctx(ioctx_name)
|
|
.. automethod:: Ioctx.require_ioctx_open()
|
|
.. automethod:: Ioctx.get_stats()
|
|
.. automethod:: Ioctx.change_auid(auid)
|
|
.. automethod:: Ioctx.get_last_version()
|
|
.. automethod:: Ioctx.close()
|
|
|
|
|
|
.. Pool Snapshots
|
|
.. --------------
|
|
|
|
.. The Ceph Storage Cluster allows you to make a snapshot of a pool's state.
|
|
.. Whereas, basic pool operations only require a connection to the cluster,
|
|
.. snapshots require an I/O context.
|
|
|
|
.. Ioctx.create_snap(self, snap_name)
|
|
.. Ioctx.list_snaps(self)
|
|
.. SnapIterator.next(self)
|
|
.. Snap.get_timestamp(self)
|
|
.. Ioctx.lookup_snap(self, snap_name)
|
|
.. Ioctx.remove_snap(self, snap_name)
|
|
|
|
.. not published. This doesn't seem ready yet.
|
|
|
|
Object Operations
|
|
-----------------
|
|
|
|
The Ceph Storage Cluster stores data as objects. You can read and write objects
|
|
synchronously or asynchronously. You can read and write from offsets. An object
|
|
has a name (or key) and data.
|
|
|
|
|
|
.. automethod:: Ioctx.aio_write(object_name, to_write, offset=0, oncomplete=None, onsafe=None)
|
|
.. automethod:: Ioctx.aio_write_full(object_name, to_write, oncomplete=None, onsafe=None)
|
|
.. automethod:: Ioctx.aio_append(object_name, to_append, oncomplete=None, onsafe=None)
|
|
.. automethod:: Ioctx.write(key, data, offset=0)
|
|
.. automethod:: Ioctx.write_full(key, data)
|
|
.. automethod:: Ioctx.aio_flush()
|
|
.. automethod:: Ioctx.set_locator_key(loc_key)
|
|
.. automethod:: Ioctx.aio_read(object_name, length, offset, oncomplete)
|
|
.. automethod:: Ioctx.read(key, length=8192, offset=0)
|
|
.. automethod:: Ioctx.stat(key)
|
|
.. automethod:: Ioctx.trunc(key, size)
|
|
.. automethod:: Ioctx.remove_object(key)
|
|
|
|
|
|
Object Extended Attributes
|
|
--------------------------
|
|
|
|
You may set extended attributes (XATTRs) on an object. You can retrieve a list
|
|
of objects or XATTRs and iterate over them.
|
|
|
|
.. automethod:: Ioctx.set_xattr(key, xattr_name, xattr_value)
|
|
.. automethod:: Ioctx.get_xattrs(oid)
|
|
.. automethod:: XattrIterator.__next__()
|
|
.. automethod:: Ioctx.get_xattr(key, xattr_name)
|
|
.. automethod:: Ioctx.rm_xattr(key, xattr_name)
|
|
|
|
|
|
|
|
Object Interface
|
|
================
|
|
|
|
From an I/O context, you can retrieve a list of objects from a pool and iterate
|
|
over them. The object interface provide makes each object look like a file, and
|
|
you may perform synchronous operations on the objects. For asynchronous
|
|
operations, you should use the I/O context methods.
|
|
|
|
.. automethod:: Ioctx.list_objects()
|
|
.. automethod:: ObjectIterator.__next__()
|
|
.. automethod:: Object.read(length = 1024*1024)
|
|
.. automethod:: Object.write(string_to_write)
|
|
.. automethod:: Object.get_xattrs()
|
|
.. automethod:: Object.get_xattr(xattr_name)
|
|
.. automethod:: Object.set_xattr(xattr_name, xattr_value)
|
|
.. automethod:: Object.rm_xattr(xattr_name)
|
|
.. automethod:: Object.stat()
|
|
.. automethod:: Object.remove()
|
|
|
|
|
|
|
|
|
|
.. _Getting Started: ../../../start
|
|
.. _Storage Cluster Configuration: ../../configuration
|
|
.. _Getting librados for Python: ../librados-intro#getting-librados-for-python
|