doc: Removed old ops directory structure and redundant files.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
This commit is contained in:
John Wilkins 2012-09-27 15:02:05 -07:00
parent ff96789e46
commit 2fc29fe9ef
6 changed files with 0 additions and 588 deletions

View File

@ -1,13 +0,0 @@
============
Operations
============
.. ===========================
.. This Document is deprecated
.. See ceph/doc/cluster-ops
.. ===========================
.. toctree::
manage/index
radosgw

View File

@ -1,199 +0,0 @@
.. _adjusting-crush:
=========================
Adjusting the CRUSH map
=========================
There are a few ways to adjust the crush map:
* online, by issuing commands to the monitor
* offline, by extracting the current map to a file, modifying it, and then reinjecting a new map
For offline changes, some can be made directly with ``crushtool``, and
others require you to decompile the file to text form, manually edit
it, and then recompile.
Adding a new device (OSD) to the map
====================================
.. _adjusting-crush-set:
Adding new devices or moving existing devices to new positions in the
CRUSH hierarchy can be done via the monitor. The general form is::
$ ceph osd crush set <id> <weight> [<loc> [<lo2> ...]]
where
* ``id`` is the numeric device id (the OSD id)
* ``name`` is an alphanumeric name. By convention Ceph uses
``osd.$id``.
* ``weight`` is a floating point weight value controlling how much
data the device will be allocated. A decent convention is to make
this the number of TB the device will store.
* ``loc`` is a list of ``what=where`` pairs indicating where in the
CRUSH hierarchy the device will be stored. By default, the
hierarchy (the ``what``s) includes ``pool`` (the ``default`` pool
is normally the root of the hierarchy), ``rack``, and ``host``.
At least one of these location specifiers has to refer to an
existing point in the hierarchy, and only the lowest (most
specific) match counts. Beneath that point, any intervening
branches will be created as needed. Specifying the complete
location is always sufficient, and also safe in that existing
branches (and devices) won't be moved around.
For example, if the OSD id is ``123``, we want a weight of ``1.0`` and
the device is on host ``hostfoo`` and rack ``rackbar``::
$ ceph osd crush set 123 1.0 root=default rack=rackbar host=hostfoo
will add it to the hierarchy, or move it from its previous position.
(Note: for clusters created prior to v0.52, replace ``root=default``
with ``pool=default``.) The rack ``rackbar`` and host ``hostfoo``
will be added as needed, as long as the pool ``default`` exists (as it
does in the default Ceph CRUSH map generated during cluster creation).
Note that if I later add another device in the same host but specify a
different pool or rack::
$ ceph osd crush set 124 1.0 root=nondefault rack=weirdrack host=hostfoo
the device will still be placed in host ``hostfoo`` at its current
location (rack ``rackbar`` and pool ``default``).
Moving a bucket to a different position in the hierarchy
========================================================
To move an existing bucket to a different position in the hierarchy,
identify the bucket to move by name and specify the new location in
the same fashion as with ``osd crush set ...``::
$ ceph osd crush move <bucket name> [<loc> [<loc2> ...]]
where
* ``name`` is the name of the bucket to move.
* ``loc`` is a list of ``what=where`` pairs indicating where the bucket should
be moved.
Adjusting the CRUSH weight
==========================
You can adjust the CRUSH weight for a device with::
$ ceph osd crush reweight osd.123 2.0
Removing a device
=================
You can remove a device from the crush map with::
$ ceph osd crush remove osd.123
Tunables
========
There are several magic numbers that were used in the original CRUSH
implementation that have proven to be poor choices. To support
the transition away from them, newer versions of CRUSH (starting with
the v0.48 argonaut series) allow the values to be adjusted or tuned.
Clusters running recent Ceph releases support using the tunable values
in the CRUSH maps. However, older clients and daemons will not correctly interact
with clusters using the "tuned" CRUSH maps. To detect this situation,
there is now a feature bit ``CRUSH_TUNABLES`` (value 0x40000) to
reflect support for tunables.
If the OSDMap currently used by the ``ceph-mon`` or ``ceph-osd``
daemon has non-legacy values, it will require the ``CRUSH_TUNABLES``
feature bit from clients and daemons who connect to it. This means
that old clients will not be able to connect.
At some future point in time, newly created clusters will have
improved default values for the tunables. This is a matter of waiting
until the support has been present in the Linux kernel clients long
enough to make this a painless transition for most users.
Impact of legacy values
~~~~~~~~~~~~~~~~~~~~~~~
The legacy values result in several misbehaviors:
* For hiearchies with a small number of devices in the leaf buckets,
some PGs map to fewer than the desired number of replicas. This
commonly happens for hiearchies with "host" nodes with a small
number (1-3) of OSDs nested beneath each one.
* For large clusters, some small percentages of PGs map to less than
the desired number of OSDs. This is more prevalent when there are
several layers of the hierarchy (e.g., row, rack, host, osd).
* When some OSDs are marked out, the data tends to get redistributed
to nearby OSDs instead of across the entire hierarchy.
Which client versions support tunables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* argonaut series, v0.48.1 or later
* v0.49 or later
* Linux kernel version v3.5 or later (for the file system and RBD kernel clients)
A few important points
~~~~~~~~~~~~~~~~~~~~~~
* Adjusting these values will result in the shift of some PGs between
storage nodes. If the Ceph cluster is already storing a lot of
data, be prepared for some fraction of the data to move.
* The ``ceph-osd`` and ``ceph-mon`` daemons will start requiring the
``CRUSH_TUNABLES`` feature of new connections as soon as they get
the updated map. However, already-connected clients are
effectively grandfathered in, and will misbehave if they do not
support the new feature.
* If the CRUSH tunables are set to non-legacy values and then later
changed back to the defult values, ``ceph-osd`` daemons will not be
required to support the feature. However, the OSD peering process
requires examining and understanding old maps. Therefore, you
should not run old (pre-v0.48) versions of the ``ceph-osd`` daemon
if the cluster has previosly used non-legacy CRUSH values, even if
the latest version of the map has been switched back to using the
legacy defaults.
Tuning CRUSH
~~~~~~~~~~~~
If you can ensure that all clients are running recent code, you can
adjust the tunables by extracting the CRUSH map, modifying the values,
and reinjecting it into the cluster.
* Extract the latest CRUSH map::
ceph osd getcrushmap -o /tmp/crush
* Adjust tunables. These values appear to offer the best behavior
for both large and small clusters we tested with. You will need to
additionally specify the ``--enable-unsafe-tunables`` argument to
``crushtool`` for this to work. Please use this option with
extreme care.::
crushtool -i /tmp/crush --set-choose-local-tries 0 --set-choose-local-fallback-tries 0 --set-choose-total-tries 50 -o /tmp/crush.new
* Reinject modified map::
ceph osd setcrushmap -i /tmp/crush.new
Legacy values
~~~~~~~~~~~~~
For reference, the legacy values for the CRUSH tunables can be set
with::
crushtool -i /tmp/crush --set-choose-local-tries 2 --set-choose-local-fallback-tries 5 --set-choose-total-tries 19 -o /tmp/crush.legacy
Again, the special ``--enable-unsafe-tunables`` option is required.
Further, as noted above, be careful running old versions of the
``ceph-osd`` daemon after reverting to legacy values as the feature
bit is not perfectly enforced.

View File

@ -1,36 +0,0 @@
.. _failures-osd:
==========================
Recovering from failures
==========================
The current health of the Ceph cluster, as known by the monitors, can
be checked with the ``ceph health`` command. If all is well, you get::
$ ceph health
HEALTH_OK
If there are problems, you will see something like::
$ ceph health
HEALTH_WARN short summary of problem(s)
or::
$ ceph health
HEALTH_ERROR short summary of very serious problem(s)
To get more detail::
$ ceph health detail
HEALTH_WARN short description of problem
one problem
another problem
yet another problem
...
.. toctree::
radosgw

View File

@ -1,104 +0,0 @@
====================================
Recovering from ``radosgw`` failure
====================================
HTTP Request Errors
===================
Examining the access and error logs for the web server itself is
probably the first step in identifying what is going on. If there is
a 500 error, that usually indicates a problem communicating with the
``radosgw`` daemon. Ensure the daemon is running, its socket path is
configured, and that the web server is looking for it in the proper
location.
Crashed ``radosgw`` process
===========================
If the ``radosgw`` process dies, you will normally see a 500 error
from the web server (apache, nginx, etc.). In that situation, simply
restarting radosgw will restore service.
To diagnose the cause of the crash, check the log in ``/var/log/ceph``
and/or the core file (if one was generated).
Blocked ``radosgw`` Requests
============================
If some (or all) radosgw requests appear to be blocked, you can get
some insight into the internal state of the ``radosgw`` daemon via
its admin socket. By default, there will be a socket configured to
reside in ``/var/run/ceph``, and the daemon can be queried with::
$ ceph --admin-daemon /var/run/ceph/client.rgw help
help list available commands
objecter_requests show in-progress osd requests
perfcounters_dump dump perfcounters value
perfcounters_schema dump perfcounters schema
version get protocol version
Of particular interest::
$ ceph --admin-daemon /var/run/ceph/client.rgw objecter_requests
...
will dump information about current in-progress requests with the
RADOS cluster. This allows one to identify if any requests are blocked
by a non-responsive ceph-osd. For example, one might see::
{ "ops": [
{ "tid": 1858,
"pg": "2.d2041a48",
"osd": 1,
"last_sent": "2012-03-08 14:56:37.949872",
"attempts": 1,
"object_id": "fatty_25647_object1857",
"object_locator": "@2",
"snapid": "head",
"snap_context": "0=[]",
"mtime": "2012-03-08 14:56:37.949813",
"osd_ops": [
"write 0~4096"]},
{ "tid": 1873,
"pg": "2.695e9f8e",
"osd": 1,
"last_sent": "2012-03-08 14:56:37.970615",
"attempts": 1,
"object_id": "fatty_25647_object1872",
"object_locator": "@2",
"snapid": "head",
"snap_context": "0=[]",
"mtime": "2012-03-08 14:56:37.970555",
"osd_ops": [
"write 0~4096"]}],
"linger_ops": [],
"pool_ops": [],
"pool_stat_ops": [],
"statfs_ops": []}
In this dump, two requests are in progress. The ``last_sent`` field is
the time the RADOS request was sent. If this is a while ago, it suggests
that the OSD is not responding. For example, for request 1858, you could
check the OSD status with::
$ ceph pg map 2.d2041a48
osdmap e9 pg 2.d2041a48 (2.0) -> up [1,0] acting [1,0]
This tells us to look at ``osd.1``, the primary copy for this PG::
$ ceph --admin-daemon /var/run/ceph/osd.1.asok
{ "num_ops": 651,
"ops": [
{ "description": "osd_op(client.4124.0:1858 fatty_25647_object1857 [write 0~4096] 2.d2041a48)",
"received_at": "1331247573.344650",
"age": "25.606449",
"flag_point": "waiting for sub ops",
"client_info": { "client": "client.4124",
"tid": 1858}},
...
The ``flag_point`` field indicates that the OSD is currently waiting
for replicas to respond, in this case ``osd.0``.

View File

@ -1,8 +0,0 @@
=========================
Managing a Ceph cluster
=========================
.. toctree::
failures/index
crush

View File

@ -1,228 +0,0 @@
.. index:: RADOS Gateway, radosgw
=========================================
Radosgw installation and administration
=========================================
RADOS Gateway (radosgw or rgw) provides a RESTful API to the object
store. It interfaces with a web server via FastCGI, and with RADOS via
libradospp.
Configuring Ceph for RADOS Gateway
----------------------------------
In order for a host to act as a RADOS gateway, you must add a
``[client.radosgw.<name>]`` section to your Ceph configuration file
(typically ``/etc/ceph/ceph.conf``):
::
[client.radosgw.gateway]
host = gateway
rgw socket path = /tmp/radosgw.sock
``host`` is the name of the host running radosgw. ``keyring`` points
to the keyring file for Cephx authentication. ``rgw socket path`` is
the location of the UNIX socket which radosgw binds to.
If your Ceph cluster has Cephx authentication enabled (highly
recommended) you also need to add the following option to tell radosgw
where it finds its authentication key:
::
[client.radosgw.gateway]
keyring = /etc/ceph/keyring.radosgw.gateway
Creating authentication credentials
-----------------------------------
To allow radosgw to sucessfully authenticate with the Ceph cluster,
use the ``ceph-authtool`` command to create a key and set its
capabilities:
::
ceph-authtool -C -n client.radosgw.gateway \
--gen-key /etc/ceph/keyring.radosgw.gateway
ceph-authtool -n client.radosgw.gateway \
--cap mon 'allow r' --cap osd 'allow rwx' --cap mds 'allow' \
/etc/ceph/keyring.radosgw.gateway
Finally, add this key to the authentication entries:
::
ceph auth add client.radosgw.gateway \
--in-file=/etc/ceph/keyring.radosgw.gateway
Configuring the web server for radosgw
--------------------------------------
The radosgw FastCGI wrapper
---------------------------
A wrapper script, customarily named ``radosgw.cgi`` needs to go into
your preferred location -- typically your web server root directory.
::
#!/bin/sh
exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
The ``-c`` option may be omitted if your Ceph configuration file
resides in its default location ((``/etc/ceph/ceph.conf``)). The
``-n`` option identifies the ``client`` section in the configuration
file that radosgw should parse -- if omitted, this would default to
``client.admin``.
Configuring Apache for radosgw
------------------------------
The recommended way of deploying radosgw is with Apache and
``mod_fastcgi``. Ensure that both ``mod_fastcgi`` and ``mod_rewrite``
are enabled in your Apache configuration. Set the
``FastCGIExternalServer`` option to point to the radosgw FastCGI
wrapper.
::
<IfModule mod_fastcgi.c>
FastCgiExternalServer /var/www/radosgw.fcgi -socket /tmp/radosgw.sock
</IfModule>
Then, create a virtual host configuration as follows:
::
<VirtualHost *:80>
ServerName radosgw.example.com
ServerAlias rgw.example.com
ServerAdmin webmaster@example.com
DocumentRoot /var/www
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^/(.*) /radosgw.fcgi?%{QUERY_STRING} [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
</IfModule>
<IfModule mod_fastcgi.c>
<Directory /var/www>
Options +ExecCGI
AllowOverride All
SetHandler fastcgi-script
Order allow,deny
Allow from all
AuthBasicAuthoritative Off
</Directory>
</IfModule>
AllowEncodedSlashes On
ServerSignature Off
</VirtualHost>
Starting the daemons
--------------------
For the gateway to become operational, start both the radosgw daemon
and your web server:
::
service radosgw start
service apache start
Creating users
--------------
In order to be able to use the RESTful API, create a user with the
``radosgw-admin`` utility:
::
$ radosgw-admin user create --uid=johndoe --display-name="John Doe" --email=john@example.com
{ "user_id": "johndoe",
"rados_uid": 0,
"display_name": "John Doe",
"email": "john@example.com",
"suspended": 0,
"subusers": [],
"keys": [
{ "user": "johndoe",
"access_key": "QFAMEDSJP5DEKJO0DDXY",
"secret_key": "iaSFLDVvDdQt6lkNzHyW4fPLZugBAI1g17LO0+87"}],
"swift_keys": []}
Note that creating a user also creates an ``access_key`` and
``secret_key`` entry for use with any S3 API-compatible client.
Enabling Swift access
---------------------
Allowing access to the object store with Swift (OpenStack Object
Storage) compatible clients requires an additional step, the creation
of a subuser and a Swift access key.
::
# radosgw-admin subuser create --uid=johndoe --subuser=johndoe:swift --access=full
{ "user_id": "johndoe",
"rados_uid": 0,
"display_name": "John Doe",
"email": "john@example.com",
"suspended": 0,
"subusers": [
{ "id": "johndoe:swift",
"permissions": "full-control"}],
"keys": [
{ "user": "johndoe",
"access_key": "QFAMEDSJP5DEKJO0DDXY",
"secret_key": "iaSFLDVvDdQt6lkNzHyW4fPLZugBAI1g17LO0+87"}],
"swift_keys": []}
# radosgw-admin key create --subuser=johndoe:swift --key-type=swift
{ "user_id": "johndoe",
"rados_uid": 0,
"display_name": "John Doe",
"email": "john@example.com",
"suspended": 0,
"subusers": [
{ "id": "johndoe:swift",
"permissions": "full-control"}],
"keys": [
{ "user": "johndoe",
"access_key": "QFAMEDSJP5DEKJO0DDXY",
"secret_key": "iaSFLDVvDdQt6lkNzHyW4fPLZugBAI1g17LO0+87"}],
"swift_keys": [
{ "user": "johndoe:swift",
"secret_key": "E9T2rUZNu2gxUjcwUBO8n\/Ev4KX6\/GprEuH4qhu1"}]}
With this configuration, you are able to use any Swift client to
connect to and use radosgw. As an example, you might use the ``swift``
command-line client utility that ships with the OpenStack Object
Storage packages.
::
$ swift -V 1.0 -A http://radosgw.example.com/auth \
-U johndoe:swift -K E9T2rUZNu2gxUjcwUBO8n\/Ev4KX6\/GprEuH4qhu1 \
post test
$ swift -V 1.0 -A http://radosgw.example.com/auth \
-U johndoe:swift -K E9T2rUZNu2gxUjcwUBO8n\/Ev4KX6\/GprEuH4qhu1 \
upload test myfile
Note that the radosgw ``user:subuser`` tuple maps to the
``tenant:user`` tuple expected by Swift.
Note also that the radosgw Swift authentication service only supports
built-in Swift authentication (``-V 1.0``) at this point. There is
currently no way to make radosgw authenticate users via OpenStack
Identity Service (Keystone).