From 8ff4edda73abb920c91e1226a330e3659def1fbe Mon Sep 17 00:00:00 2001 From: Loic Dachary Date: Tue, 3 Jun 2014 13:05:19 +0200 Subject: [PATCH 1/2] documentation: update osd pool create erasure The properties are replaced with erasure code profiles. Remove the reference to properties and the documentation of each erasure-code related property. Signed-off-by: Loic Dachary --- doc/rados/operations/pools.rst | 70 ++++------------------------------ 1 file changed, 7 insertions(+), 63 deletions(-) diff --git a/doc/rados/operations/pools.rst b/doc/rados/operations/pools.rst index 61d0269fb3c..da3d5dc0a15 100644 --- a/doc/rados/operations/pools.rst +++ b/doc/rados/operations/pools.rst @@ -10,7 +10,7 @@ pools for storing data. A pool provides you with: A typical configuration stores an object and one additional copy (i.e., ``size = 2``), but you can determine the number of copies/replicas. For erasure coded pools, it is the number of coding chunks - (i.e. ``erasure-code-m=2``) + (i.e. ``m=2`` in the **erasure code profile**) - **Placement Groups**: You can set the number of placement groups for the pool. A typical configuration uses approximately 100 placement groups per OSD to @@ -63,12 +63,7 @@ To create a pool, execute:: ceph osd pool create {pool-name} {pg-num} [{pgp-num}] [replicated] ceph osd pool create {pool-name} {pg-num} {pgp-num} erasure \ - [{crush_ruleset=ruleset}] \ - [{erasure-code-directory=directory}] \ - [{erasure-code-plugin=plugin}] \ - [{erasure-code-k=data-chunks}] \ - [{erasure-code-m=coding-chunks}] \ - [{key=value} ...] + {erasure-code-profile} Where: @@ -112,61 +107,11 @@ Where: :Required: No. :Default: replicated -``{crush_ruleset=ruleset}`` +``{erasure-code-profile=profile}`` -:Description: For **erasure** pools only. Set the name of the CRUSH - **ruleset**. It must be an existing ruleset matching - the requirements of the underlying erasure code plugin. - -:Type: String -:Required: No. - -``{erasure-code-directory=directory}`` - -:Description: For **erasure** pools only. Set the **directory** name - from which the erasure code plugin is loaded. - -:Type: String -:Required: No. -:Default: /usr/lib/ceph/erasure-code - -``{erasure-code-plugin=plugin}`` - -:Description: For **erasure** pools only. Use the erasure code **plugin** - to compute coding chunks and recover missing chunks. - -:Type: String -:Required: No. -:Default: jerasure - -``{erasure-code-k=data-chunks}`` - -:Description: For **erasure** pools using the **jerasure** plugin - only. Each object is split in **data-chunks** parts, - each stored on a different OSD. - -:Type: Integer -:Required: No. -:Default: 4 - -``{erasure-code-m=coding-chunks}`` - -:Description: For **erasure** pools using the **jerasure** plugin - only. Compute **coding chunks** for each object and - store them on different OSDs. The number of coding - chunks is also the number of OSDs that can be down - without losing data. - -:Type: Integer -:Required: No. -:Default: 2 - -``{key=value}`` - -:Description: For **erasure** pools, the semantic of the remaining - key/value pairs is defined by the erasure code plugin. - For **replicated** pools, the key/value pairs are - ignored. +:Description: For **erasure** pools only. Use the erasure code + **profile**. It must be an existing profile as + defined by **osd erasure-code-profile set**. :Type: String :Required: No. @@ -178,11 +123,10 @@ you have many pools with many placement groups (e.g., 50 pools with 100 placement groups each). The point of diminishing returns depends upon the power of the OSD host. -See `Placement Groups`_ for details on calculating an appropriate number of +See `Placement Groups`_ for details on calculating an appropriate number of placement groups for your pool. .. _Placement Groups: ../placement-groups - Set Pool Quotas From 5569d40bb4a42d6614316fd70a5520c269b55469 Mon Sep 17 00:00:00 2001 From: Loic Dachary Date: Tue, 3 Jun 2014 13:47:51 +0200 Subject: [PATCH 2/2] documentation: add osd erasure-code-profile {set,rm,get,ls} And a link from rados/operations. http://tracker.ceph.com/issues/8514 Fixes: #8514 Reported-by: Kenneth Waegeman Signed-off-by: Loic Dachary --- doc/rados/operations/erasure-code-profile.rst | 113 ++++++++++++++++++ doc/rados/operations/index.rst | 1 + 2 files changed, 114 insertions(+) create mode 100644 doc/rados/operations/erasure-code-profile.rst diff --git a/doc/rados/operations/erasure-code-profile.rst b/doc/rados/operations/erasure-code-profile.rst new file mode 100644 index 00000000000..b8630346f0b --- /dev/null +++ b/doc/rados/operations/erasure-code-profile.rst @@ -0,0 +1,113 @@ +===================== +Erasure code profiles +===================== + +Erasure code is defined by a **profile** and is used when creating an +erasure coded pool and the associated crush ruleset. + +The **default** erasure code profile (which is created when the Ceph +cluster is initialized) provides the same level of redundancy as two +copies but requires 25% less disk space. It is described as a profile +with **k=2** and **m=1**, meaning the information is spread over three +OSD (k+m == 3) and one of them can be lost. + +To improve redundancy without increasing raw storage requirements, a +new profile can be created. For instance, a profile with **k=10** and +**m=4** can sustain the loss of four (**m=4**) OSDs by distributing an +object on fourteen (k+m=14) OSDs. The object is first divided in +**10** chunks (if the object is 10MB, each chunk is 1MB) and **4** +coding chunks are computed, for recovery (each coding chunk has the +same size as the data chunk, i.e. 1MB). The raw space overhead is only +40% and the object will not be lost even if four OSDs break at the +same time. + +osd erasure-code-profile set +============================ + +To create a new erasure code profile:: + + ceph osd erasure-code-profile set {name} \ + [{k=data-chunks}] \ + [{m=coding-chunks}] \ + [{directory=directory}] \ + [{plugin=plugin}] \ + [{key=value} ...] \ + [--force] + +Where: + +``{k=data-chunks}`` + +:Description: Each object is split in **data-chunks** parts, + each stored on a different OSD. + +:Type: Integer +:Required: No. +:Default: 2 + +``{m=coding-chunks}`` + +:Description: Compute **coding chunks** for each object and store them + on different OSDs. The number of coding chunks is also + the number of OSDs that can be down without losing data. + +:Type: Integer +:Required: No. +:Default: 1 + +``{directory=directory}`` + +:Description: Set the **directory** name from which the erasure code + plugin is loaded. + +:Type: String +:Required: No. +:Default: /usr/lib/ceph/erasure-code + +``{plugin=plugin}`` + +:Description: Use the erasure code **plugin** to compute coding chunks + and recover missing chunks. + +:Type: String +:Required: No. +:Default: jerasure + +``{key=value}`` + +:Description: The semantic of the remaining key/value pairs is defined + by the erasure code plugin. + +:Type: String +:Required: No. + +``--force`` + +:Description: Override an existing profile by the same name. + +:Type: String +:Required: No. + +osd erasure-code-profile rm +============================ + +To remove an erasure code profile:: + + ceph osd erasure-code-profile rm {name} + +If the profile is referenced by a pool, the deletion will fail. + +osd erasure-code-profile get +============================ + +To display an erasure code profile:: + + ceph osd erasure-code-profile get {name} + +osd erasure-code-profile ls +=========================== + +To list the names of all erasure code profiles:: + + ceph osd erasure-code-profile ls + diff --git a/doc/rados/operations/index.rst b/doc/rados/operations/index.rst index 508e88f420a..96bbdda4d15 100644 --- a/doc/rados/operations/index.rst +++ b/doc/rados/operations/index.rst @@ -33,6 +33,7 @@ CRUSH algorithm. data-placement pools + erasure-code-profile cache-tiering placement-groups crush-map