1
0
mirror of https://github.com/ceph/ceph synced 2025-01-04 02:02:36 +00:00
Commit Graph

31602 Commits

Author SHA1 Message Date
Samuel Just
45b6c59e5b osd/ECTransaction: add type encapsulating EC transaction
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:16 -08:00
Samuel Just
33c3830158 osd/ECUtil: add ec encode/decode util helpers
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:16 -08:00
Samuel Just
7731a9490c ObjectStore: add bufferlist/string getattr
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:16 -08:00
Samuel Just
258af5ebe6 buffer: allow caller to initialize bufferhash state
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:16 -08:00
Samuel Just
ae827cfe4d osd_types: add swap, claim_append to ObjectModDesc
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:16 -08:00
Samuel Just
7772014a4a OSD,PG: pass through EC messages
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:16 -08:00
Samuel Just
6cd64a507d messages,osd: add EC messages and associated types
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
e8820ac42a src/osd/: add ECBackend stubs
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
0661d8025a PGBackend: make getattrs virtual
This way, the backend can choose to filter private attrs.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
ca323ff627 ReplicatedPG: fix size checking for ECBackend pools
Allow the backend to adjust the logical object size before
checking against the ondisk size.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
510585b795 PGBackend/ReplicatedBackend: factor out logical to ondisk size mapping
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
26745ca899 PGBackend/ReplicatedBackend: move the backend agnostic code into PGBackend for scrub
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
1835c29f6d ReplicatedBackend: factor out be_deep-scrub
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
f0b69b384b osd/: replace bufferptr with bufferlist for attrs in recovery
Now we won't have to keep converting back and forth!

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:15 -08:00
Samuel Just
0b180e6a62 PG::discover_all_missing: request a FULLLOG
Otherwise, we cannot correct for divergent objects in the missing
set.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
84e2f39c55 PG,ReplicatedPG: Generalize missing_loc for ECBackend
Prior to EC pools, unfound => missing.  Now, unfound (unreadable,
really) is dependent on the PGBackend requirements for reconstituting
an object.  This also means recovering an object missing on a replica
but not the primary requires tracking the missing_loc set.

Thus, rather than maintaining missing_loc only for objects missing
on the primary, the MissingLoc structure will track all missing
objects actingbackfill-wide until each object is recovered.

For simplicity, since we don't really know what objects need recovery
until activation (and since we can't do anything with that information
prior to activation anyway), we defer populating the missing_loc
information until activation.

We need peers to rollback divergent log entries before we attempt to
read the relevant objects.  The simplest way to accomplish this seems to
be the simply choose to always activate peers if search_for_missing
turns up missing objects.

Due to EC pools, missing is necessary, but not sufficient for readability.
Thus, we instead check is_unreadable for cases where we need to read the object
and reserve is_missing for cases where we need the object context.

wait_for_missing_object becomes waiting_for_unreadable_object in order to avoid
having another layer of waiting_for_* maps.  These ops may be requeued
either when the primary is recovered or when the object is no longer degraded,
depending on when the object becomes readable.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
8b33d60fc4 PG: allow PGBackend to set criteria for PG up-ness
ECBackend needs to be able to require that a readable
set of the most recent interval to write be available
in order to ensure that it rolls back the log far
enough.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
facec7654c PGBackend: add some additional helpers.
ECBackend's primary specific logic mostly won't treat the
primary shard specially, so it'll be handy to have primary
agnostic helpers for get_shard_info and get_shard_missing.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
74b8a0c05c PG,PGBackend: expose acting and backfill_targets
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
5a8c0bb952 os/: allow objects with NO_GEN but non-NO_SHARD shard
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
2d3205ff1a PG: skip CRUSH_ITEM_NONE in acting set in various places
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
5db3b2dc2c osd/: extend pg_interval_t to include primary
Otherwise, we cannot correctly determine up_from/up_thru for
old intervals.  Also, we need this information to determine
when a new interval starts due to a new primary without a
change in the acting set.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:14 -08:00
Samuel Just
9d8be1b082 Objecter: handle primary other than acting[0]
EC pool particularly might have CRUSH_ITEM_NONE as acting[0] which
is not a valid osd.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:13 -08:00
Samuel Just
904704ac7f OSDMap: handle CRUSH_ITEM_NONE in acting sets
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:13 -08:00
Samuel Just
109b3c9e7f OSDMap: fill in get_primary_shard
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:13 -08:00
Samuel Just
285cf7728a PG: introduce calc_ec_acting
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:13 -08:00
Samuel Just
41e194531b PG: refactor calc_acting to handle pg_shard_t changes
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:13 -08:00
Samuel Just
035d6cc2a3 osd/: refer to pg by spg_t and replica by pg_shard_t
We may have multiple pg shards on the same osd for an
ec pool.  Thus, replicas must be referred to by <osd, shard>
and pgs by <pgid, shard>.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:10 -08:00
Samuel Just
8b46dd1dc0 messages/: include shard information in various pg messages
We can no longer use the messenger source information to determine
the origin of the message since an osd might have more than one
shard of a particular pg.  Thus, we need to include a pg_shard_t
from field to indicate origin.  Similarly, pg_t is no longer
sufficient to specify the destination pg, we instead use spg_t.
In the event that we get a message from an old peer, we default
from to pg_shard_t(get_source().num(), ghobject_t::no_shard())
and spg_t to spg_t(pgid, ghobject_t::no_shard()).  This suffices
because non-NO_SHARD shards can only appear once ec pools have
been enabled -- and doing that bans unenlightened osds.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
fa980644bd ReplicatedBackend: excise OSDService*
This should eventually make it easier to mock out a PGBackend::Listener.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
31b7937bae PGBackend/ReplicatedBackend: move temp handling into PGBackend
Temp handling is also the same in ReplicatedBackend as in ECBackend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
761f0b1e18 PGBackend: use whoami_shard().shard in rollback*, objects*
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
5d01f88caf ReplicatedBackend/PGBackend: move objects_* methods into PGBackend
These also are unchanged in ECBackend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
8ca875b6f4 PGBackend/ReplicatedBackend: move rollback methods into PGBackend
These will end up essentially unchanged in ECBackend as it turns out.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
4a25951633 osd_types: introduce spg_t, pg_shard_t
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
44b43df47f ObjectStore: use ghobject_t for setattrs
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
f2826fa2cf src/osd: pass in version to recover to recover_object
Otherwise we need to do some annoying contortions to
get it out of the missing sets.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
1c93e03f04 osd/: Add a shard parameter to SnapMapper to handle multiple pg shards
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
f678aefde3 TestPGLog: remove test with DELETE op with prior_version = eversion_t()
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
09d611d25c PGLog::merge_old_entry: simplify the oe.version > ne.version case
If ne.version < oe.version, the correct answer is to rollback oe.version
if possible regardless of what the entries are.

Also, rearrange to deal with the fact that we cannot roll back a missing
object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:10:18 -08:00
Samuel Just
d116e55f4a PG: just ignore rollbacks on objects we have already failed to rollback
The relevant changes to missing and/or the filestore will have
already been made.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:25:34 -08:00
Samuel Just
0e7b10ac9b PG::remove_snap_mapped_object: use pg_whoami.shard
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:25:33 -08:00
Samuel Just
fd9da00edc ReplicatedPG: take snapset_obc write lock where appropriate
Otherwise, we might read it for backfill before it's fully created
on a peer.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:25:33 -08:00
Samuel Just
da4652c92d ReplicatedPG: release backfill state on_global_recover not peer recover
There may now be multiple backfill peers.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
d687ed96fd ReplicatedPG: take clone write lock as well in make_writeable
Otherwise, we might start backfilling the clone before the op is
complete.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
643b6a50ae ReplicatedPG,osd_types: seperate require_rollback from ec_pool
It's handy to allow a pool to answer false to ec_pool() and
true to require_rollback() in order to allow a replicated
pool to test the rollback mechanisms without allowing
non-NO_SHARD shards.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
f020ce90f7 ReplicatedPG:: remove_snap_mapped_object already removes the object
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
17e6ecc841 ReplicatedPG: find_object_context debugging: output oi
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
b3b2bab403 ReplicatedPG: handle xattr cache on rollback and delete_head
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
d8b6d4c125 ReplicatedPG: release_op_ctx_locks in all-applied, not all-committed
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00