When updating submodules, always checkout even if the HEAD is the
desired commit hash (update --force) to avoid the following:
* a directory gmock exists in hammer
* a submodule gmock replaces the directory gmock in master
* checkout master + submodule update : gmock/.git is created
* checkout hammer : the gmock directory still contains the .git from
master because it did not exist at the time and checkout won't
remove untracked directories
* checkout master + submodule update : git rev-parse HEAD is
at the desired commit although the content of the gmock directory
is from hammer
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Document the high level view of the development workflows which are
otherwise implicit and non trivial to figure out.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
When using :doc:`/dev/sepia` the page title is inlined. The modified
title is easier to include as part of a sensible sentence.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Encapsulate the compilation steps (install dependencies, autogen.sh,
configure, make check) in the run-make-check.sh script. Update the
developer documentation to point to this script instead of multiple
steps.
It is intended as a tool to help new developer make sure their patch is
sane, it focuses on efficiency (runs make check in parallel if possible)
and coverage (enables docker based tests if possible).
http://tracker.ceph.com/issues/10265Fixes: #10265
Signed-off-by: Loic Dachary <ldachary@redhat.com>
vstart.sh now creates the users for the default configuration
for the s3-tests, available on https://github.com/ceph/s3-tests.
Also updated the documentation to show the correct RadosGW port.
Signed-off-by: Luis Pabón <lpabon@redhat.com>
The prototype of the init functions of erasure coded plugins is changed
from
int __erasure_code_init(char *plugin_name)
to
int __erasure_code_init(char *plugin_name, char *directory)
The jerasure plugin will find optimized variants in this directory and
load them. The load() and preload() functions of
ErasureCodePluginRegistry only use a directory instead of a more generic
parameters map. The parameters map was only used for the directory entry
anyway.
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: OSDMonitor: 'osd pool' - if we can set it, we must be able to get it
Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Sage Weil <sage@redhat.com>
If unspecified it is ruleset-root=default and will translate into
take default
when a ruleset is created for an erasure-code pool.
Signed-off-by: Loic Dachary <loic@dachary.org>
* Warn the reader that the implementation is ahead and may differ
* Update the links to the Firefly branch
* Remove links to issues used during development to avoid confusion
Signed-off-by: Loic Dachary <loic@dachary.org>
* The parameters come from the erasure code profile
* Add a note about the upstream submodules gf-complete / jerasure
Signed-off-by: Loic Dachary <loic@dachary.org>
Update the introduction to explain erasure code profiles. Remove
obsolete explanations about partial writes etc. Remove links to tickets
used during development. Update permalinks to be closer to
Firefly (v0.78).
Signed-off-by: Loic Dachary <loic@dachary.org>
The "just push the new tag" bit is already done in the list of commands
above. Remove this piece, since it's duplicated by the "git push"
command above.
The ceph-deploy and backports-sections were empty. Remove them.
New script to sign rpms in the repo and create the indexes, and
some description of additional packages that might need to be added.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
Reviewed-by: Ken Dreyer <ken.dreyer@inktank.com>
The ceph-mon command usage is updated to document all of the ceph-mon
specific options.
The ceph tell usage examples for log and debug are using a deprecated syntax.
Signed-off-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Based on a dialog with Sam ( as published at http://dachary.org/?p=2320 ).
* Remove PGBackend-h.rst because PGBackend.h is now in master.
* Fix typos caught by ispell
* Update recovery links to point to PGBackend recover methods
* Workaround formating warning
developer_notes.rst:3: WARNING: Duplicate explicit target name:
"erasurecodepluginexample" which should be legitimate.
Signed-off-by: Loic Dachary <loic@dachary.org>
* Update to the current state of the ghobject implementaiton and the fact
that they encode the shard_t Although the pool also contains the shard
id, it is less relevant to understand the implementation.
* Update with the erasure code plugin infrastructure and the example
plugin now in master.
* Move jerasure to a separate page to be expanded and link it from the
toc
* Kill the partial read and writes notes as it will probably not be
implemented in the near future. Kill some of the notes because they
are no longer relevant.
* Add a definition for "chunk rank"
* Reword, update schemas, fix typos.
Signed-off-by: Loic Dachary <loic@dachary.org>
Following the changes to when we set or increase the user_version, we
want to continue to return the best lower bound we can on the version
of any newly-created object. For ENOENT replies that means returning
info.last_user_version instead of the (potentially-zero) ctx->user_at_version.
Similarly, for cls_current_version we want to return the last version on
the PG rather than the last update to the object in order to provide
sensible version ordering across object deletes and creates.
Update the versions doc so it continues to be precise.
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Specify a user and pg_pool_t interface for tiering/caching specifications
Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
* unify conventions to match those used by jerasure ( data chunk = K,
coding chunk = M, use coding instead of parity, use erasures instead
of erased )
* make lines 80 characters long
* modify the descriptions to take into account that the chunk rank
will encoded in the pool name and not on a per object basis
* remove the doxygen link to ErasureCodeInterface because it fails
doc: asphyxiate does not support class
http://tracker.ceph.com/issues/6115
* only systematic codes are considered at this point ( all jerasure
techniques are systematic). Although the API could be extended to
include non systematic codes, it is probably a case of over
engineering at this point.
* add link to
http://tracker.ceph.com/issues/6113
add ceph osd pool create [name] [key=value]
* update the plugin system description to match the proposed
implementation http://tracker.ceph.com/issues/5877http://tracker.ceph.com/issues/4929 refs #4929
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
* replace the erasure code plugin abstract interface with a doxygen link
that will be populated when the header shows in master
* update the plugin documentation to reflect the current draft implementation
* fix broken link to PGBackend-h
* add a glossary to define chunk, stripe, shard and strip with a drawing
http://tracker.ceph.com/issues/4929 refs #4929
Signed-off-by: Loic Dachary <loic@dachary.org>
Explains how objects are stored and used in erasure coded pools. It is
the result of discussions that occured on the ceph-devel mailing list
around june 2013. The rationale behind each change can be found in the
archive of the mailing list. For instance, the coding of the chunk
number with the object or the decision to decode using any K chunks
instead of trying to fetch the data chunks when possible because it
would allow simple concatenation when systematic codes are used.
http://tracker.ceph.com/issues/4929 refs #4929
Signed-off-by: Loic Dachary <loic@dachary.org>
it is not easy to figure out what a PG temp is just by reading the
code although it is easy to understand with an example.
Signed-off-by: Loic Dachary <loic@dachary.org>
The decision to discard an op happens either in OSD or in PG.
The operation queue goes to a single OpWQ object if waiting_map does not impose a delay op_queue.
The decision to add an op to a waiting queue regardless of its type is updated.
The decision to add a CEPH_MSG_OSD_OP to a waiting queue is described in full.
http://tracker.ceph.com/issues/5058 refs #5058
Signed-off-by: Loic Dachary <loic@dachary.org>
Add priority to request_reservation()
Change to map of lists by prioriry
Add priority to queue_pointers mappped type
Signed-off-by: David Zafman <david.zafman@inktank.com>
Export a diff of an image from a previous snapshot to a file (or stdout).
Import a diff and apply it to an image, and then create the ending
snapshot.
Signed-off-by: Sage Weil <sage@inktank.com>
This file mostly duplicated the existing release documentation. Differences
have been merged into the primary file.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
I'm still not sure about the names for the command line
operations, but they can be changed later if better ones
come up.
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
This isn't quite complete, and should be integrated with the info in the
ops section on mon cluster expansion, but the raw content is here at least.
Signed-off-by: Sage Weil <sage@inktank.com>
If toctree is inside a section, the subtree is inside the section too.
We don't want all of dev/* to be under "Mailing list".
I have not found a decent workaround for this. The toplevel toctree
avoids this purely by the fact that it is the topmost toctree. Right
now that means you should 1) avoid having more than a few paragraphs of
text before the toctree for that subtree (put most of the content after
the toctree; clumsy if the toctree is long), or 2) put the toptree
immediately after the document title, make it :hidden:, and let the
reader use links in the text or the ToC in the sidebar to navigate.
See start/index for an example of this.
Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
- fix discussion of last epoch started
- define terms for current and past intervals
- describe role of pg info
- remove mention of the backlog
- fix discussion of up_thru
- etc.
Signed-off-by: Sage Weil <sage@newdream.net>
This adds a document that I wrote about how Ceph client file data
is striped across Ceph objects to the repository. It's a text
document. Someone with better document preparation skills than I
should use the content below as a basis for something prettier if
that's appropriate.
[Made a few edits... -sage]
Signed-off-by: Alex Elder <elder@dreamhost.com>
Signed-off-by: Sage Weil <sage@newdream.net>
It was not an index, and seems to contain recommendations
for system configuration. I have renamed it to confusing.txt
and will merge it in a future commit.
Signed-off-by: Mark Kampe <mark.kampe@dreamhost.com>
which was a copy of PlanningImplementation.txt
(and not html at all).
restored previous index.rst, which was overwritten with a copy
of PlanninImplementation.txt, but removed all of the recursively
included content from the document.
I will cherry-pick merge the new contents in a subsequent commit.
Signed-off-by: Mark Kampe <mark.kampe@dreamhost.com>
Most of the doc is user-oriented, let's isolate internals more.
Split into multiple files. Clean up RST.
Use Sphinx's graphviz plugin for graphs.
Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
