Commit Graph

25154 Commits

Author SHA1 Message Date
Sage Weil
3694968a05 librbd: implement image.snap_exists()
This is a much more convenient way to tell if a snapshot already exists.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
c5bd978a1d librados: move snap_set_diff to librados/
This is most closely related to the librados list_snaps API; move it there.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
6af769a16f librados: cleanly define SNAP_HEAD, SNAP_DIR constants
We were using the internal CEPH_NOSNAP and CEPH_SNAPDIR constants, and
defining a clone_info_t::HEAD (with a different value).  The docs were
referrring to the internal constant names.

Instead, define librados constants (C and C++) with the same values as the
internal types.

Note that this changes the clone_info_t::HEAD value from -1 to -2 so that
it now matches the internal type.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
10dc0ad09f librados: document list_snaps
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
be8927f598 librbd: drop unused elapsed calc for diff_iterate
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
f0c9a200ec librbd: diff_iterate fromsnapname after the end snap is also invalid
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
a69532e864 librbd: document diff_iterate in header
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
d0baadb9d3 librbd: uint64_t len for diff_iterate
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
7bbaa71a56 doc/dev/rbd-diff: update incremental file format
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
44e295a3cc qa: rbd/diff_continuous.sh: use non-standard striping
Exercise the striping arithmetic by using non-standard striping that
varies between the parent and child.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
39689fea49 librbd: fix diff_iterate arithmetic for non-standard striping
This code is confusing because we are moving back and forth between
image offsets, "buffer" offsets (image offsets relative to off), and
object offsets.  Fix the math.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
f2b0880a89 qa: rbd/diff_continuous.sh: base test off a clone
Get a bit of coverage on clones by starting with a clone.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
fc3f4fda76 rbd: implement simple 'diff' command
Report extents allocated/changed, and whether they contain data or zeros.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
4d02e17f36 librbd: handle diff from clone
If we have a parent image, and the reference is from snap 0 (beginning of
time) we need to look at the diff on the parent from the beginning of time
and report that when we get an ENOENT.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
186ddda58c rbd: send import debug noise to dout, not stdout
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:41 -07:00
Sage Weil
58c2dedded qa: add rbd/diff_continuous.sh stress test
Stress test that does io on an image while we are mirroring a diff from
earlier snaps to a second copy.  At the end, verify that all snaps have
matching content.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
e7167433ae rbd: implement 'export-diff' and 'import-diff' commands
Export a diff of an image from a previous snapshot to a file (or stdout).

Import a diff and apply it to an image, and then create the ending
snapshot.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
cf7d13a7e9 rbd: add --io-pattern <seq|rand> option to bench-write
Write to random offsets instead of sequentially.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
0296c7cdae librbd: implement diff_iterate
Implement a diff_iterate() method that will iterate over an image and
report which extents vary between two snapshots (or a snapshot and the
head).  The callback gets an extent and a flag indicating whether it is
full of data or is known to be zero in the ending snapshot.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
6db5109127 librados: expose snapset seq via list_snaps
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
ebed000014 osdc/Objecter: prval optional for listsnaps
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
4ae977435c osd: fix error codes for list-snaps
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
941cfc26a0 osd: fix clone snap list for list-snaps
We need to return the list of snaps that each clone is defined for, not
the list of snaps we know may or may not exist globally over a similar
interval.  This requires looking at the clone's obc, unfortunately.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
ceee218fd5 osd: wait for all clones on SNAPDIR requests
Wait for all clones to be present.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
556d33442f osd: direct reads on SNAPDIR to either head or snapdir
The list_snaps operation needs to look at the SnapSet, and is logically
querying all revisions of the object.  Make requests to SNAPDIR be
read-only, and grab the head or snapdir obc transparently (whichever one
exists).  This allows us to list snaps when, say, the head does not
exist, but there are in fact snaps.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
b64bb5f860 osd: do not include snaps with head on list_snaps()
If there is a sequence of snaps 1, 2, 3, 4, 5, and we have a clone
2 with [1,2], and the head reflects content at snap times [3,4,5], then
the snap_list should return

 clone 2 snaps [1,2]
 head snaps
 seq 2

because it never saw a write after snap 2, and therefor has the same
content currently as it did in snaps 3,4,5.  If the SnapSet on the
object lists snaps 3,4,5, and the head exists, it actually means the
object was deleted between 2 and 3, and was recreated after 5:

 clone 2 snaps [1,2]
 head snaps []
 seq 5

The key to telling the two situations apart is the seq number on the
SnapSet (now included in the list_snaps reply) that tells us when the
last update was.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
01b74209fb osd: clean up some whitespace
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
0074228911 osd: include SnapSet seq in the list snaps response
It is important to know the latest seq that the object has seen in order
to tell if a response like

 clone 2 snaps=[1,2]
 clone head snaps=[]

was untouched before a hypothetical snap 3, or deleted prior to snap 3,
and then recreated+modified after.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
fa5206ce9b osd: make LIST_WATCHERS and LIST_SNAPS print nicely for OSDOp
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
849a45c67f strings: add 'list-watchers' to MOSDOp strings
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-31 23:32:40 -07:00
Sage Weil
ce8793ce3b Merge pull request #175 from dachary/wip-4594
fix null character in object name triggering segfault

Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-30 18:22:01 -07:00
Loic Dachary
c344ff170d fix null character in object name triggering segfault
Parsing \n in  lfn_parse_object_name is implemented with

  out->append('\0');

which segfaults when using libstdc++ and g++ version 4.6.3 on Debian
GNU/Linux. It is replaced with

  (*out) += '\0';

to avoid the bugous implicit conversion. There is no append(charT)
method in C++98 or C++11, which means it relies on an implicit
conversion that is bugous. It would be better to rely on the
basic_string& operator+=(charT c); method as defined in ISO 14882-1998
(page 385) thru ISO 14882-2012 (page 640)

A set of tests is added to generate and parse object names. They need
access to the private function lfn_parse_object_name because there is
no convenient protected method to exercise it. The tests contain a
LFNIndex derived class, TestWrapLFNIndex which is made a friend of
LFNIndex to gain access to the private methods.

http://tracker.ceph.com/issues/4594 refs #4594

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-03-30 14:28:34 +01:00
Sage Weil
2b8eb31b85 Merge branch 'wip-4490' 2013-03-29 18:02:15 -07:00
Sage Weil
e611937f3e mon: OSDMonitor: add 'osd pool set-quota' command
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-03-29 17:59:35 -07:00
John Wilkins
95328089b8 doc: Added entries for Pool, PG, & CRUSH. Moved heartbeat link.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-03-29 17:38:48 -07:00
John Wilkins
bcc5c65305 doc: Added heartbeat configuration settings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-03-29 17:38:02 -07:00
John Wilkins
6157d68369 doc: Moved PG info to separate page. Moved heartbeat to mon-osd doc.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-03-29 17:36:23 -07:00
John Wilkins
ca77aabbf1 doc: Rewrote monitor configuration section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-03-29 17:34:45 -07:00
John Wilkins
ea3c833d0f doc: Moved to separate section for parallelism.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-03-29 17:32:47 -07:00
John Wilkins
ba73b8301a doc: Cleanup.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-03-29 17:32:00 -07:00
Sage Weil
e9b3f2e6e9 ceph-disk list: say 'unknown cluster $UUID' when cluster is unknown
This makes it clearer that an old osd is in fact old.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-29 17:30:28 -07:00
Greg Farnum
9e7ddf677f config_opts: fix rgw_port comments to be plaintext
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-03-29 17:05:41 -07:00
Samuel Just
3da3129e07 ReplicatedPG: check for full if delta_stats.num_bytes > 0
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-03-29 16:47:29 -07:00
Joao Eduardo Luis
9b09073259 mon: Monitor: check if 'pss' arg is !NULL on parse_pos_long()
We already do it all throughout the function, but this one place didn't.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-03-29 16:47:29 -07:00
Joao Eduardo Luis
e2a936d2ae common: util: add 'unit_to_bytesize()' function
Converts from a numerical value that may or may not contain an unit
modifier ('1024', '1K', '2M', ..., '1E') and returns the parsed size
in bytes.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-03-29 16:47:28 -07:00
Joao Eduardo Luis
23c2fa7fc2 osd: osd_types: add pool quota related fields 2013-03-29 16:03:21 -07:00
Sage Weil
562e1716bd ceph-disk: handle missing journal_uuid field gracefully
Only lower if we know it's not None.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-29 13:59:04 -07:00
Josh Durgin
b504e444fc Merge remote branch 'origin/next' 2013-03-29 12:58:01 -07:00
Josh Durgin
95c4a81be1 Merge pull request #170 from ceph/wip-rbd-aio-flush
Reviewed-by: Sage Weil <sage.weil@inktank.com>
2013-03-29 13:20:32 -07:00
Josh Durgin
4c4d5591bd librados: move snapc creation to caller for aio_operate
The common case already has a snapshot context, so avoid duplicating
it (copying a potentially large vector) in IoCtxImpl::aio_operate().

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-03-29 12:47:17 -07:00