RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-02-18 16:37:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc: Added many Q/A sections to the FAQ.

Browse Source 
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
This commit is contained in:
John Wilkins 2013-02-13 12:06:25 -08:00
parent 670a14827a
commit 617bde9550
1 changed files with 310 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

321
doc/faq.rst
View File

@ -2,27 +2,304 @@
Frequently Asked Questions
============================
These questions have been frequently asked on the ceph-devel mailing
list, the IRC channel, and on the `Ceph.com`_ blog.
These questions have been frequently asked on the ceph-users and ceph-devel
mailing lists, the IRC channel, and on the `Ceph.com`_ blog.
.. _Ceph.com: http://ceph.com
Is Ceph Production-Quality?
===========================
Ceph's object store is production ready. Large-scale storage systems (i.e.,
petabytes of data) use Ceph's block devices and Ceph's RESTful object store
supporting APIs compatible with Amazon's S3 and OpenStack's Swift. `Inktank`_
provides commercial support for the Ceph object store, block devices, and
RESTful interfaces.
petabytes of data) use Ceph's RESTful object store, which provides APIs
compatible with Amazon's S3 and OpenStack's Swift. Many deployments also use
the Ceph block device, including deployments of OpenStack and CloudStack.
`Inktank`_ provides commercial support for the Ceph object store, RESTful
interfaces and block devices.
The CephFS POSIX-compliant filesystem is functionally-complete and has
been evaluated by a large community of users, but is still undergoing
methodical QA testing. Once Ceph's filesystem passes QA muster, `Inktank`_
will provide commercial support for CephFS in production systems.
The CephFS POSIX-compliant filesystem is functionally complete and has been
evaluated by a large community of users. There are production systems using
CephFS with a single metadata server. The Ceph community is actively testing
clusters with multiple metadata servers for quality assurance. Once Ceph's
filesystem running multiple metadata servres passes QA muster, `Inktank`_ will
provide commercial support for CephFS large scale production systems.
.. _Inktank: http://inktank.com
What Kind of Hardware Does Ceph Require?
========================================
Ceph runs on commodity hardware. A typical configuration involves a
rack mountable server with a baseboard management controller, multiple
processors, multiple drives, and multiple NICs. There are no requirements for
proprietary hardware. For details, see `Ceph Hardware Recommendations`_.
What Kind of OS Does Ceph Require?
==================================
Ceph runs on Linux. Most Ceph users run a Debian/Ubuntu distribution, which you
can install from `APT packages`_. Ceph builds `RPM packages`_ for Federa/RHEL
too. You can also download Ceph source `tarballs`_ and build Ceph for your
distribution. See `Installation`_ for details.
How Many OSDs Can I Run per Host?
=================================
Theoretically, a host can run as many OSDs as the hardware can support. Many
vendors market storage hosts that have large numbers of drives (e.g., 36 drives)
capable of supporting many OSDs. We don't recommend a huge number of OSDs per
host though. Ceph was designed to distribute the load across what we call
"failure domains." See `CRUSH Maps`_ for details.
At the petabyte scale, hardware failure is an expectation, not a freak
occurrence. Failure domains include datacenters, rooms, rows, racks, and network
switches. In a single host, power supplies, motherboards, NICs, and drives are
all potential points of failure.
If you place a large percentage of your OSDs on a single host and that host
fails, a large percentage of your OSDs will fail too. Having too large a
percentage of a cluster's OSDs on a single host can cause disruptive data
migration and long recovery times during host failures. We encourage
diversifying the risk across failure domains, and that includes making
reasonable tradeoffs regarding the number of OSDs per host.
Can I Use the Same Drive for Multiple OSDs?
===========================================
Yes. **Please don't do this!** Except for initial evaluations of Ceph, we do not
recommend running multiple OSDs on the same drive. In fact, we recommend
**exactly** the opposite. Only run one OSD per drive. For better performance,
run journals on a separate drive from the OSD drive, and consider using SSDs for
journals. Run operating systems on a separate drive from any drive storing data
for Ceph.
Storage drives are a performance bottleneck. Total throughput is an important
consideration. Sequential reads and writes are important considerations too.
When you run multiple OSDs per drive, you split up the total throughput between
competing OSDs, which can slow performance considerably.
Why Do You Recommend One Drive Per OSD?
=======================================
Ceph OSD performance is one of the most common requests for assistance, and
running an OS, a journal and an OSD on the same disk is a frequently the
impediment to high performance. Total throughput and simultaneous reads and
writes are a major bottleneck. If you journal data, run an OS, or run multiple
OSDs on the same drive, you will very likely see performance degrade
significantly--especially under high loads.
Running multiple OSDs on a single drive is fine for evaluation purposes. We
even encourage that in our `5-minute quick start`_. However, just because it
works does NOT mean that it will provide acceptable performance in an
operational cluster.
What Filesystem Do You Recommend for OSD Drives?
================================================
Currently, we recommend using XFS in deployment. We think ``btrfs`` will become
the optimal filesystem. However, we still encounter enough issues that we do
not recommend it for production systems yet. See `Filesystem Recommendations`_
for details.
How Does Ceph Ensure Data Integrity Across Replicas?
====================================================
Ceph periodically scrubs placement groups to ensure that they contain the same
information. Low-level or deep scrubbing reads the object data in each replica
of the placement group to ensure that the data is identical across replicas.
How Many NICs Per Host?
=======================
You can use one :abbr:`NIC (Network Interface Card)` per machine. We recommend a
minimum of two NICs: one for a public (front-side) network and one for a cluster
(back-side) network. When you write an object from the client to the primary
OSD, that single write only accounts for the bandwidth consumed during one leg
of the transaction. If you store multiple copies (usually 2-3 copies in a
typical cluster), the primary OSD makes a write request to your secondary and
tertiary OSDs. So your back-end network traffic can dwarf your front-end network
traffic on writes very easily.
What Kind of Network Throughput Do I Need?
==========================================
Network throughput requirements depend on your load. We recommend starting with
a minimum of 1GB Ethernet. 10GB Ethernet is more expensive, but often comes with
some additional advantages, including virtual LANs (VLANs). VLANs can
dramatically reduce the cabling requirements when you run front-side, back-side
and other special purpose networks.
The number of object copies (replicas) you create is an important factor,
because replication becomes a larger network load than the initial write itself
when making multiple copies (e.g., triplicate). Network traffic between Ceph and
a cloud-based system such as OpenStack or CloudStack may also become a factor.
Some deployments even run a separate NIC for management APIs.
Finally load spikes are a factor too. Certain times of the day, week or month
you may see load spikes. You must plan your network capacity to meet those load
spikes in order for Ceph to perform well. This means that excess capacity may
remain idle or unused during low load times.
Can Ceph Support Multiple Data Centers?
=======================================
Yes, but with significant limitations that ensure data safety. When a client
writes data to Ceph the primary OSD will not acknowledge the write to the client
until the secondary OSDs have written the replicas synchronously. See `How Ceph
Scales`_ for details. Additionally, OSD and monitor heartbeats and peering
processes do not tolerate additional latency that may occur when deploying
hardware in different geographic locations. See `Monitor/OSD Interaction`_ for
details.
If your data centers have dedicated bandwidth and low latency, you can
distribute your cluster across data centers. If you use a WAN over the Internet,
you may experience significant peering, heartbeat acknowledgement and write
latency, which makes performance suffer considerably. Dedicated connections are
expensive, so people tend to avoid them. The Ceph community is exploring
asynchronous writes to make distributing a cluster across data centers more
feasible. While it is on the Ceph development roadmap, work has not begun on
asynchronous write capability yet.
How Does Ceph Authenticate Users?
=================================
Ceph provides an authentication framework called ``cephx`` that operates in a
manner similar to Kerberos. The principal difference is that Ceph's
authentication system is distributed too, so that it doesn't constitute a single
point of failure. For details, see `Ceph Authentication & Authorization`_.
Does Ceph Authentication Provide Multi-tenancy?
===============================================
Ceph provides authentication at the `pool`_ level, which may be sufficient
for multi-tenancy in limited cases. Ceph plans on developing authentication
namespaces within pools in future releases, so that Ceph is well-suited for
multi-tenancy within pools.
Can Ceph use other Multi-tenancy Modules?
=========================================
The Bobtail release of Ceph integrates RADOS Gateway with OpenStack's Keystone.
See `Keystone Integration`_ for details.
.. _Keystone Integration: ../radosgw/config#integrating-with-openstack-keystone
Does Ceph Enforce Quotas?
=========================
Currently, Ceph doesn't provide enforced storage quotas. The Ceph community has
discussed enforcing user quotas within CephFS.
Does Ceph Track Per User Usage?
===============================
The CephFS filesystem provides user-based usage tracking on a subtree basis.
RADOS Gateway also provides detailed per-user usage tracking. RBD and the
underlying object store do not track per user statistics. The underlying object
store provides storage capacity utilization statistics.
Does Ceph Provide Billing?
==========================
Ceph does not provide billing functionality at this time. Improvements to
pool-based namespaces and pool-based usage tracking may make it feasible to use
Ceph usage statistics with usage tracking and billing systems in the future.
Can Ceph Export a Filesystem via NFS or Samba/CIFS?
===================================================
Ceph doesn't export CephFS via NFS or Samba. However, you can use a gateway to
serve a CephFS filesystem to NFS or Samba clients.
Can I Access Ceph via a Hypervisor?
===================================
Currently, the `QEMU`_ hypervisor can interact with the Ceph `block device`_.
The :abbr:`KVM (Kernel Virtual Machine)` `module`_ and the `librbd` library
allow you to use QEMU with Ceph. Most Ceph deployments use the `librbd` library.
Cloud solutions like `OpenStack`_ and `CloudStack`_ interact `libvirt`_ and QEMU
to as a means of integrating with Ceph.
Ceph integrates cloud solutions via ``libvirt`` and QEMU, but the Ceph community
is also talking about supporting the Xen hypervisor. Ceph and Citrix engineers
have built a prototype, but they have not released a stable means of integrating
Xen with Ceph for general use yet. Similarly, there is interest in support for
VMWare, but there is no deep-level integration between VMWare and Ceph as yet.
Can Block, CephFS, and Gateway Clients Share Data?
==================================================
For the most part, no. You cannot write data to Ceph using RBD and access the
same data via CephFS, for example. You cannot write data with RADOS gateway and
read it with RBD. However, you can write data with the RADOS Gateway
S3-compatible API and read the same data using the RADOS Gateway
Swift-comptatible API.
RBD, CephFS and the RADOS Gateway each have their own namespace. The way they
store data differs significantly enough that it isn't possible to use the
clients interchangeably. However, you can use all three types of clients, and
clients you develop yourself via ``librados`` simultaneously on the same
cluster.
Which Ceph Clients Support Striping?
====================================
Ceph clients--RBD, CephFS and RADOS Gateway--providing striping capability. For
details on striping, see `Striping`_.
What Programming Languages can Interact with the Object Store?
==============================================================
Ceph's ``librados`` is written in the C programming language. There are
interfaces for other languages, including:
- C++
- Java
- PHP
- Python
- Ruby
Can I Develop a Client With Another Language?
=============================================
Ceph does not have many native bindings for ``librados`` at this time. If you'd
like to fork Ceph and build a wrapper to the C or C++ versions of ``librados``,
please check out the `Ceph repository`_. You can also use other languages that
can use the ``librados`` native bindings (e.g., you can access the C/C++ bindings
from within Perl).
Do Ceph Clients Run on Windows?
===============================
No. There are no immediate plans to support Windows clients at this time. However,
you may be able to emulate a Linux environment on a Windows host. For example,
Cygwin may make it feasible to use ``librados`` in an emulated environment.
How can I add a question to this list?
======================================
@ -32,9 +309,31 @@ main git repository:
`https://github.com/ceph/ceph/blob/master/doc/faq.rst`_
.. _https://github.com/ceph/ceph/blob/master/doc/faq.rst: https://github.com/ceph/ceph/blob/master/doc/faq.rst
We use Sphinx to manage our documentation, and this page is generated
from reStructuredText source. See the section on Building Ceph
Documentation for the build procedure.
.. _Ceph Hardware Recommendations: ../install/hardware-recommendations
.. _APT packages: ../install/debian
.. _RPM packages: ../install/rpm
.. _tarballs: ../install/get-tarballs
.. _Installation: ../install
.. _CRUSH Maps: ../rados/operations/crush-map
.. _5-minute quick start: ../start/quick-start
.. _How Ceph Scales: ../architecture#how-ceph-scales
.. _Monitor/OSD Interaction: ../rados/configuration/mon-osd-interaction
.. _Ceph Authentication & Authorization: ../rados/operations/auth-intro
.. _Ceph repository: https://github.com/ceph/ceph
.. _QEMU: ../rbd/qemu-rbd
.. _block device: ../rbd
.. _module: ../rbd/rbd-ko
.. _libvirt: ../rbd/libvirt
.. _OpenStack: ../rbd/rbd-openstack
.. _CloudStack: ../rbd/rbd-cloudstack
.. _pool: ../rados/operations/pools
.. _Striping: ../architecture##how-ceph-clients-stripe-data
.. _https://github.com/ceph/ceph/blob/master/doc/faq.rst: https://github.com/ceph/ceph/blob/master/doc/faq.rst
.. _Filesystem Recommendations: ../rados/configuration/filesystem-recommendations