Commit Graph

190 Commits

Author SHA1 Message Date
Samuel Just
4a25951633 osd_types: introduce spg_t, pg_shard_t
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Kun Huang
e0bae95240 Fix typos in erasure code documents
Signed-off-by: Kun Huang <academicgareth@gmail.com>
2013-12-31 06:17:52 -05:00
Loic Dachary
b8bdbd92d0 doc: update erasure code development doc
With a link to the tracker issue implementing the new indep mode.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 15:19:19 +01:00
Loic Dachary
00ee9a5289 doc: improve ceph-mon usage, fix ceph tell examples
The ceph-mon command usage is updated to document all of the ceph-mon
specific options.

The ceph tell usage examples for log and debug are using a deprecated syntax.

Signed-off-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-11-26 09:44:45 -08:00
John Wilkins
8a36503879 doc: Moved CPU Profiler docs to dev.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-10-22 13:22:17 -07:00
Loic Dachary
238a303cff ErasureCode: update PGBackend description
Based on a dialog with Sam ( as published at http://dachary.org/?p=2320 ).

* Remove PGBackend-h.rst because PGBackend.h is now in master.

* Fix typos caught by ispell

* Update recovery links to point to PGBackend recover methods

* Workaround formating warning
  developer_notes.rst:3: WARNING: Duplicate explicit target name:
  "erasurecodepluginexample" which should be legitimate.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-10-02 15:58:01 +02:00
Loic Dachary
ff4887324a ErasureCode: doc updates
* Update to the current state of the ghobject implementaiton and the fact
  that they encode the shard_t Although the pool also contains the shard
  id, it is less relevant to understand the implementation.

* Update with the erasure code plugin infrastructure and the example
  plugin now in master.

* Move jerasure to a separate page to be expanded and link it from the
  toc

* Kill the partial read and writes notes as it will probably not be
  implemented in the near future. Kill some of the notes because they
  are no longer relevant.

* Add a definition for "chunk rank"

* Reword, update schemas, fix typos.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-10-02 15:02:53 +02:00
Samuel Just
62a1eb6f69 osd/: add PGBackend interfaces and stubs
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-09-26 11:24:25 -07:00
Wido den Hollander
991139bc0d doc: When bootstrapping mon set the correct caps for client.admin
Otherwise client.admin will be privilege-less and the admin can't
access the cluster.
2013-09-23 13:52:51 +02:00
John Wilkins
1e54d835cc doc: Syntax fixes to suppress gitbuilder warnings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-09-09 15:27:12 -07:00
John Wilkins
68659a6563 doc: Syntax fixes to suppress gitbuilder warnings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-09-09 15:26:47 -07:00
John Wilkins
1bb722550f doc: Syntax fixes to suppress gitbuilder warnings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-09-09 15:23:24 -07:00
John Wilkins
a6a714064e doc: Syntax fixes to suppress gitbuilder warnings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-09-09 15:22:59 -07:00
John Wilkins
ff9beb2cf5 doc: Syntax fixes to stop gitbuilder warnings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-09-09 11:19:59 -07:00
John Wilkins
9c0970168b doc: Syntax fixes to stop gitbuilder warnings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-09-09 11:19:21 -07:00
Sage Weil
d33853d380 Merge pull request #555 from dachary/master
minor documentation fixes
2013-09-05 08:51:21 -07:00
Greg Farnum
f566102c3c doc: remove 'Unexpected indentation' from versions doc.
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-09-04 10:10:03 -07:00
Loic Dachary
3c9f8498c8 doc: erasure code ghobject is made of gen_t + shard_t
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-09-04 12:33:15 +02:00
Dan Mick
dcbdeaf6d1 doc: Fix repo URL for Ceph cloning (dev/generatedocs)
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-09-03 15:56:53 -07:00
Greg Farnum
087800ee49 osd: provide better version bounds for cls_current_version and ENOENT replies
Following the changes to when we set or increase the user_version, we
want to continue to return the best lower bound we can on the version
of any newly-created object. For ENOENT replies that means returning
info.last_user_version instead of the (potentially-zero) ctx->user_at_version.

Similarly, for cls_current_version we want to return the last version on
the PG rather than the last update to the object in order to provide
sensible version ordering across object deletes and creates.

Update the versions doc so it continues to be precise.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-09-03 10:54:23 -07:00
Gregory Farnum
b30a1b2889 Merge pull request #554 from ceph/wip-tier-interface
Specify a user and pg_pool_t interface for tiering/caching specifications

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-08-30 14:13:25 -07:00
Sage Weil
cb2abad901 Merge pull request #539 from dachary/master
doc : erasure code developer notes updates
2013-08-28 10:29:17 -07:00
Sage Weil
61b40f481b doc/dev/cache-pool: document cache pool management interface
Signed-off-by: Sage Weil <sage@inktank.com>
2013-08-28 09:34:03 -07:00
Greg Farnum
9101433a88 Merge remote-tracking branch 'origin/master' into wip-6029
Conflicts:
	src/librados/AioCompletionImpl.h
2013-08-27 17:26:36 -07:00
Greg Farnum
6c432f1932 doc: update to describe new OSD version support as it actually exists
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-08-27 17:24:51 -07:00
Greg Farnum
295a84b9d9 doc: include plan for new user_version support
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-08-27 17:00:44 -07:00
Greg Farnum
b5ea74cec4 docs: document how the current OSD PG/object versions work
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-08-27 15:08:28 -07:00
Loic Dachary
14c31ddf10 doc : erasure code developer notes updates
* unify conventions to match those used by jerasure ( data chunk = K,
  coding chunk = M, use coding instead of parity, use erasures instead
  of erased )

* make lines 80 characters long

* modify the descriptions to take into account that the chunk rank
  will encoded in the pool name and not on a per object basis

* remove the doxygen link to ErasureCodeInterface because it fails
  doc: asphyxiate does not support class
  http://tracker.ceph.com/issues/6115

* only systematic codes are considered at this point ( all jerasure
  techniques are systematic). Although the API could be extended to
  include non systematic codes, it is probably a case of over
  engineering at this point.

* add link to
  http://tracker.ceph.com/issues/6113
  add ceph osd pool create [name] [key=value]

* update the plugin system description to match the proposed
  implementation http://tracker.ceph.com/issues/5877

http://tracker.ceph.com/issues/4929 refs #4929

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-27 14:13:56 +02:00
Loic Dachary
157f2227f4 doc: fix erasure code formatting warnings and errors
http://tracker.ceph.com/issues/4929 refs #4929

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-22 17:45:39 +02:00
Loic Dachary
bebba3c858 doc: fix erasure code formatting warnings and errors
http://tracker.ceph.com/issues/4929 refs #4929

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-21 18:09:03 +02:00
Loic Dachary
3a83129283 erasure code : plugin, interface and glossary documentation updates
* replace the erasure code plugin abstract interface with a doxygen link
  that will be populated when the header shows in master
* update the plugin documentation to reflect the current draft implementation
* fix broken link to PGBackend-h
* add a glossary to define chunk, stripe, shard and strip with a drawing

http://tracker.ceph.com/issues/4929 refs #4929

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-20 17:34:52 +02:00
Samuel Just
0c5f842e49 Merge branch 'wip-erasure-coded-doc' 2013-08-19 11:02:45 -07:00
Sage Weil
70790ab6b4 doc/dev/filestore-filesystem-compatibliity: remove outdated xattr notes
Signed-off-by: Sage Weil <sage@inktank.com>
2013-08-17 12:34:11 -07:00
Sage Weil
d7f1d166ea Merge pull request #494 from kri5/wip-s3-compliance-doc
doc: complete S3 features status from existing doc page
2013-08-17 11:00:59 -07:00
Sage Weil
893331be9e Merge pull request #490 from dachary/master
ant is missing from the list of packages to install
2013-08-15 10:52:17 -07:00
Christophe Courtaut
c566c03406 doc: complete S3 features status from existing doc page
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-08-12 10:58:32 +02:00
Loic Dachary
8bf3971b7e rearrange the documentation to be inserted and maintained in master
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-09 23:26:49 +02:00
Christophe Courtaut
5082fec18f doc: Add a page about rgw S3 API compliance
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-08-09 10:26:06 +02:00
Loic Dachary
fd0626108a ant is missing from the list of packages to install
to build the documentation.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-09 10:17:26 +02:00
Loic Dachary
980a07380d document the write / read path for erasure encoding
Explains how objects are stored and used in erasure coded pools. It is
the result of discussions that occured on the ceph-devel mailing list
around june 2013. The rationale behind each change can be found in the
archive of the mailing list. For instance, the coding of the chunk
number with the object or the decision to decode using any K chunks
instead of trying to fetch the data chunks when possible because it
would allow simple concatenation when systematic codes are used.

http://tracker.ceph.com/issues/4929 refs #4929

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-09 09:42:31 +02:00
Loic Dachary
0304dfbc5f Add links to OSD internals related tracker items.
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:36:28 +02:00
Loic Dachary
0d1040d798 Add links to scrub related tracker items.
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:32:17 +02:00
Loic Dachary
349e7900a9 Add links to peering related tracker items.
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:28:33 +02:00
Loic Dachary
423614777d Add links to recovery related tracker items.
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:27:31 +02:00
Loic Dachary
128628824b Add links to backfill related tracker items.
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:09:55 +02:00
Loic Dachary
160fa710ec fix typo
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:09:42 +02:00
Loic Dachary
ac6f3b72a0 M+K means M data + K parity and implies M chunks necessary to recover
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-05 15:08:49 +02:00
Samuel Just
826478cfde dev/osd_internals,src/osd: add erasure_coding.rst and PGBackend.h
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-08-02 16:05:50 -07:00
Loic Dachary
6256d681a4 Add the definition of PG temp to the glossary
it is not easy to figure out what a PG temp is just by reading the
code although it is easy to understand with an example.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-08-02 00:47:13 +02:00
Sage Weil
a46f60afde doc/dev/repo-lab-access: notes
Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-28 16:17:50 -07:00
Sage Weil
b8fad05912 doc/dev: add corpus.rst
Structure of the corpus dir, how to add to it.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-07-12 10:36:47 -07:00
Christophe Courtaut
eec903a6ec doc: Fix env variables in vstart.sh documentation
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-07-08 22:19:21 +02:00
Christophe Courtaut
a793e203fd doc: Add a page to document vstart.sh script
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2013-07-04 14:08:41 +02:00
Dan Mick
8c23d8cb46 dev/rbd-diff: make formats into bullet lists (solves linebreak probs)
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-06-04 17:24:16 -07:00
John Wilkins
72a6767eb3 doc: Added title. Suppresses no title warning messages.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-06-04 15:33:42 -07:00
John Wilkins
145c480807 doc: Text of diagram for osd_throttles.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-31 11:14:29 -07:00
John Wilkins
8d70c091d9 doc: Omitted text diagram, and used literal include to text file.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-31 11:14:04 -07:00
Samuel Just
1c35556b56 doc/dev/osd_internals: add wbthrottle.rst
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-05-28 10:41:57 -07:00
Samuel Just
17a6e7faa7 doc/dev/osd_internals: add some info about throttles
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-05-21 12:51:20 -07:00
Loic Dachary
2a4425af0e reflect recent changes in the pg deletion logic
No need to wait on DeletingStateRef for flush d3dd99b725
Fix typos

http://tracker.ceph.com/issues/5058 refs #5058

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-05-14 22:46:37 +02:00
Loic Dachary
1c53991e4c fix typos and add hyperlink to peering
s/;/:/
s/up_acting_affected/acting_up_affected/
Add relative link to ../../peering

http://tracker.ceph.com/issues/5058 refs #5058

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-05-14 22:46:37 +02:00
Loic Dachary
b7d4012c06 typo s/come/some/
http://tracker.ceph.com/issues/5058 refs #5058

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-05-14 22:46:36 +02:00
Loic Dachary
dbddffef06 update op added to a waiting queue or discarded
The decision to discard an op happens either in OSD or in PG.
The operation queue goes to a single OpWQ object if waiting_map does not impose a delay op_queue.
The decision to add an op to a waiting queue regardless of its type is updated.
The decision to add a CEPH_MSG_OSD_OP to a waiting queue is described in full.

http://tracker.ceph.com/issues/5058 refs #5058

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-05-14 22:44:58 +02:00
David Zafman
ea0e0c7e34 Merge branch 'wip-4273'
Reviewed-by: Sam Just <sam.just@inktank.com>
2013-05-10 16:45:34 -07:00
Samuel Just
b353da6f68 Merge branch 'wip_pg_res'
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-09 22:24:31 -07:00
Samuel Just
0ef9b1e049 osd_internals/pg_removal.rst: update for pg resurrection
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-05-09 17:28:15 -07:00
John Wilkins
f1b13a1796 doc: Added the non-implemented bit for the gateway to the dev/radosgw TOC.
fixes: #4978

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-09 12:22:21 -07:00
John Wilkins
fe164e4438 doc: Republishing the admin operations API for the gateway.
fixes: #4978

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-09 12:08:28 -07:00
David Zafman
00e903160b osd: prioritize recovery for degraded pgs
Three Reservation priorities from RECOVERY, BACKFILL_HIGH, BACKFILL_LOW

fixes: #4273

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-05-09 12:00:29 -07:00
David Zafman
09163a3bbc Add priority option to AsyncReserver class
Add priority to request_reservation()
Change to map of lists by prioriry
Add priority to queue_pointers mappped type

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-05-09 12:00:29 -07:00
caleb miles
da271f7f78 doc: Document admin api web interface.
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2013-05-09 11:15:01 -07:00
John Wilkins
452fb529fa doc: Fixed typo.
fixes: #4422

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-07 16:11:29 -07:00
John Wilkins
4e99dca970 doc: Fixed usage typo.
fixes: #4923

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-07 07:57:16 -07:00
Sage Weil
5b0c68b928 doc/dev/rbd-diff: specify that metadata records come before data
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
7bbaa71a56 doc/dev/rbd-diff: update incremental file format
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
e7167433ae rbd: implement 'export-diff' and 'import-diff' commands
Export a diff of an image from a previous snapshot to a file (or stdout).

Import a diff and apply it to an image, and then create the ending
snapshot.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Samuel Just
78db663ec7 osd_internals/snaps.rst: update description to reflect SnapMapper
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-03-13 19:45:08 -07:00
Samuel Just
ba449ce031 osd_internals/snaps.rst: add a description of snaps and trimming
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-03-13 18:48:24 -07:00
David Zafman
8ce26eaf02 Document scrubbing behavior
Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-03-11 12:44:47 -07:00
Gary Lowell
e694ea58c2 release-process.rst: Fix typos
Signed-off-by: Gary Lowell  <gary.lowell@inktank.com>
2013-03-05 22:08:15 -08:00
Sage Weil
75ce88fb69 doc: fix rst for watch/notify internals
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-26 17:45:33 -08:00
Samuel Just
9a399afd71 doc: add some internal docs for watch/notify
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-02-20 13:29:20 -08:00
Gary Lowell
1867b81883 docs: fix typo in release-process doc
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
2012-12-28 14:05:56 -08:00
Gary Lowell
774a54cbdd docs: update release process documentation.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
2012-12-27 15:39:46 -08:00
Gary Lowell
98e7b59807 docs: remove extra release-process2 file.
This file mostly duplicated the existing release documentation.  Differences
have been merged into the primary file.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
2012-12-27 11:14:19 -08:00
Gary Lowell
cedea1391c docs: Merge changes from release-process2 document. 2012-12-26 12:54:27 -08:00
John Wilkins
0a2a0c075c doc: config-cluser move to new IA.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-11-14 14:57:51 -08:00
Mike Ryan
cdd73bbab0 doc/: document recovery reservation process
Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-11-07 17:21:32 -08:00
John Wilkins
06c62c5217 doc: Added syntax fixes to Peter's session authentication doc.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-10-30 11:20:51 -07:00
Peter Reiher
a67afa4b09 Merge branch 'wip-msgauth4'
Conflicts:
	src/common/config_opts.h
	Added a couple of options related to session authentication, accepted new values for option from master
2012-10-26 09:25:15 -07:00
Sage Weil
69a96bb513 doc: clean up doc build directions
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-19 10:40:25 -07:00
Sage Weil
fcf6273d04 Merge remote-tracking branch 'gh/wip-msgauth4' 2012-10-15 20:57:19 -07:00
Tommi Virtanen
3ba7ab2ba5 doc: Whitespace cleanup.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-04 14:54:17 -07:00
Tommi Virtanen
790b508903 doc: rst syntax fixes.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-04 14:54:17 -07:00
Sage Weil
74ea0cb541 doc: update release-process.rst
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-03 08:44:52 -07:00
Gary Lowell
2519bd3932 doc: Added description of ceph release process. 2012-10-02 23:41:35 -07:00
Peter Reiher
35488061a7 New documentation about Cephx protocol and session authentication.
Signed-off-by: Peter Reiher <reiher@inktank.com>
2012-10-01 13:12:14 -07:00
Sage Weil
a351f7a1f4 Merge remote-tracking branch 'gh/wip_backfill_full2'
Conflicts:
	src/include/ceph_features.h
2012-09-27 13:21:23 -07:00
Josh Durgin
fa3a7472f9 doc: fix typo in dev layering notes
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-09-26 17:41:03 -07:00
Samuel Just
3ebf52dfb5 doc/: added documentation for backfill_reservation
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-09-25 11:43:51 -07:00