Commit Graph

20199 Commits

Author SHA1 Message Date
Samuel Just
2552a7f430 OSD,PG: _share_map_outgoing must not require osd_lock
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:15:00 -07:00
Samuel Just
35949c541c ReplicatedPG: explicitely block on not active for certain ops
Ops and some subops need to wait for active to ensure correct ordering
with respect to peering operations.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:15:00 -07:00
Samuel Just
e8d09d025e PG,OSD: prevent pg from completing peering until deletion is complete
hobject_t must now be globally unique in the filestore.  Thus, if we
start creating objects in a pg before the removal collections for the
previous incarnation are fully removed, we might end up a second
instance of the same hobject violating the filestore rules.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:15:00 -07:00
Samuel Just
b200710b96 OSD,PG: clean up pg removal
PG opsequencers will be used for removing a pg.  If the pg is recreated
before the removal is complete, we need the new pg incarnation to be
able to inherit the osr of its predecessor.

Previously, we queued the pg for removal and only rendered it unusable
after the contents were fully removed.  Now, we syncronously remove it
from the map and queue a transaction renaming the collections.  We then
asyncronously clean up those collections.  If the pg is recreated, it
will inherit the same osr until the cleanup is complete ensuring correct
op ordering with respect to the collection rename.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:15:00 -07:00
Samuel Just
9a4a9b9af9 PG: flush ops by the end of peering without osr.flush
Rather than explicitely flushing the filestore, send a noop through the
filestore at the beginning of peering and, at the end, wait for it to
finish by adding an extra state.

Also, delay ops until flushed is true.  Until we have finished flushing,
we cannot safetly read objects.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
33b985dcfe OSD,PG: added helper methods for creating and dispatching RecoveryCtxs
This is simpler than having to update all of the RecoveryCtx users
whenever we change the types in RecoveryCtx.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
b80b50053c OSD,PG: Move pg accesible methods, objects to OSDService
In order to clarify data structure locking, PGs will now access
OSDService rather the the OSD directly.  Over time, more structures will
be moved to the OSDService.  osd_lock can no longer be held while pg
locks are held.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
502c033511 PG, OSD: info_map shouldn't contain the MOSDPGInfo*
Rather, we will just pass the same type as the noties.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
f68f52ced6 OSD: queue_want_up_thru in process_peering_event
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
492e6351e5 OSD: do not drop osd_lock in handle_osd_map
PGs have their map updates done in a different thread.  Thus, we no
longer need to grab the pg locks.  activate_map no longer requires
the map_lock in order to allow us to queue events for the pgs.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
14381b38d7 OSD: get map read lock during queue_want_up_thru
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
10ec5bc5ee OSD: push_waiters is no longer used
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
90e3727e2c OSD: do not lock osd during dequeue_op
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
3d32712e41 OSD: don't assume pending pg removals have flushed
_create_lock_pg might encounter a preexisting pg collection simply
because the removal transaction had not yet completed.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
91355fb733 ReplicatedPG: change ReplicatedPG debug output to match PG
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
b6bf573699 ReplicatedPG: do not eval_repop if aborted
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
f6fc1b3031 OSD: remove superfluous pg get/put around enqueue_op
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:59 -07:00
Samuel Just
b0cb96bff6 PG,OSD: fix op checking in pg, take_waiters during ActMap
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
b5078bb2cd PG,OSD: add OSD::queue_for_op, use in PG::queue_op
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
64b71c697e OSD: check for deleting in process_peering_event
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
fff0874c95 PG: use osd->requeue_ops for ops, pg->queue_for_peering to requeue pg
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
e10e81c046 PG: compound messages must carry epoch_sent for each part
Query and Notify messages include logical messages from multiple
pgs.  Each logical message (pg_query_t and pg_notify_t) now
contains an epoch_sent.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
b081bd4c92 PG: CephPeeringEvents can now be descriptively printed
The CephPeeringEvt constructor is now templated to allow
storing a description string for debugging.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
65abfc07df OSD: initialize pgs in get_or_create_pg via handle_create
Previously, pgs were initialized via Info/Log/etc.  Since the event
which triggered the pg creation may now be queued, map update events may
occur before the event is processed.  Thus, get_or_create_pg now handles
the initialization prior to queuing the event.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
c7e34951e8 PG: include info and query by value in peering events
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
12e22b3d44 OSD,PG: handle pg map advance in process_peering_event
The pg map will now be advanced in process_peering_event (in advance_pg)
to allow handle_osd_map to not grab pg locks in-line.  handle_osd_map
queues NullEvts to ensure that each pg is updated in a timely fashion.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
bbd8af0287 osd/: Make pg osdmap be independent of osd, other pg maps
This will allow handle_osd_map to not stop other work queues.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
ddef446dc1 OSD,PG: Move Op,SubOp queueing into PG
PG now handles delaying/discarding messages since pg map epoch may not
be the same as the OSD map.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:58 -07:00
Samuel Just
33bcbb33c9 PG: process peering events in a queue
Peering events are now queued via queue_peering_event in the
peering_queue.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Samuel Just
36d1381a0b PG: use intrusive_ptr in CephPeeringEvt
Properly disposing of the event_base member of CephPeeringEvt
requires use of intrusive_ptr.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Samuel Just
6922f9cfa9 osd/: move history update from handle_pg_query into pg
Previously, replica history was updated in OSD::handle_pg_query.
Updating the history is now handled in the pg state machine.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Samuel Just
d01582301a OSD,PG: push message checking to pg
old_peering_evt now checks CephPeeringEvts generically in
PG::handle_peering_event().

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Samuel Just
7c414c5dab OSD: Remove handle_pg_missing, MOSDPGMissing no longer used
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-07-05 10:14:57 -07:00
Samuel Just
3691e3ca4b PG: Move handle_* methods to PG
PG now calls handle_event in RecoveryState.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Samuel Just
1ffd190e14 PG: CephPeeringEvt
CephPeeringEvt is now the supertype for all peering state machine
events.  This will allow us to generalize checking for stale peering
events and delaying events for future maps.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Samuel Just
18fec69aae OSD,PG::scrub() move pg->put() into queue process
This clarifies ownership of the pg reference.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-05 10:14:57 -07:00
Sage Weil
7fa85790fb osd: add missing formatter close_section() to scrub status
Also add braces to make the open/close matchups easier to see.  Broken
by f366173927.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-07-04 13:59:04 -07:00
Sage Weil
c0b01cda10 Merge branch 'stable'
Conflicts:
	src/test/cli/radosgw-admin/help.t
2012-07-04 09:30:21 -07:00
Wido den Hollander
f67fe4e368 librados: Bump the version to 0.48
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-07-04 09:21:01 -07:00
Samuel Just
bcfcf8efd5 librados: add assert_version as an operation on an ObjectOperation
Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-04 07:32:23 -07:00
Samuel Just
39eaa23076 ReplicatedPG: do not set reply version to last_update
The version should be oi.user_version as set above.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-07-04 07:29:12 -07:00
Sage Weil
e6e36c0a72 rgw: initialize fields of RGWObjEnt
This fixes various valgrind warnings triggered by the s3test
test_object_create_unreadable.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-07-04 07:27:32 -07:00
Sage Weil
f6cdd85223 Merge remote-tracking branch 'gh/wip-crush' 2012-07-03 16:49:29 -07:00
Yehuda Sadeh
35b9ec881a rgw-admin: use correct modifier with strptime
Bug #2658: used %I (12h) instead of %H (24h)

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-07-03 16:24:28 -07:00
Yehuda Sadeh
da251fe885 rgw: send both swift x-storage-token and x-auth-token
older clients need x-storage-token, newer x-auth-token

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-07-03 16:24:20 -07:00
Yehuda Sadeh
4c19ecb9a3 rgw: radosgw-admin date params now also accept time
The date format now is "YYYY-MM-DD[ hh:mm:ss]". Got rid of
the --time param for the old ops log stuff.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>

Conflicts:

	src/test/cli/radosgw-admin/help.t
2012-07-03 16:23:42 -07:00
Yehuda Sadeh
6958aeb898 rgw-admin: fix usage help
s/show/trim

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-07-03 16:10:22 -07:00
Tommi Virtanen
ad97415ef7 ceph-disk-prepare: Partition and format OSD data disks automatically.
Uses gdisk, as it seems to be the only tool that can automate GPT uuid
changes. Needs to run as root.

Adds Recommends: gdisk to ceph.deb.

Closes: #2547
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-07-03 15:25:52 -07:00
John Wilkins
a1696fe0f3 doc: removed /srv/osd.$id.journal from ceph.conf example.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-07-03 14:20:34 -07:00
caleb miles
8f64647805 CrushTester.cc: remove BOOST dependencies.
remove calls to BOOST libraries for computing Chi-squared statistics and
producing discrete random variables with a given probability distribution.

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-07-03 14:16:51 -07:00