Commit Graph

31574 Commits

Author SHA1 Message Date
Samuel Just
8b46dd1dc0 messages/: include shard information in various pg messages
We can no longer use the messenger source information to determine
the origin of the message since an osd might have more than one
shard of a particular pg.  Thus, we need to include a pg_shard_t
from field to indicate origin.  Similarly, pg_t is no longer
sufficient to specify the destination pg, we instead use spg_t.
In the event that we get a message from an old peer, we default
from to pg_shard_t(get_source().num(), ghobject_t::no_shard())
and spg_t to spg_t(pgid, ghobject_t::no_shard()).  This suffices
because non-NO_SHARD shards can only appear once ec pools have
been enabled -- and doing that bans unenlightened osds.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
fa980644bd ReplicatedBackend: excise OSDService*
This should eventually make it easier to mock out a PGBackend::Listener.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
31b7937bae PGBackend/ReplicatedBackend: move temp handling into PGBackend
Temp handling is also the same in ReplicatedBackend as in ECBackend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
761f0b1e18 PGBackend: use whoami_shard().shard in rollback*, objects*
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
5d01f88caf ReplicatedBackend/PGBackend: move objects_* methods into PGBackend
These also are unchanged in ECBackend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:06 -08:00
Samuel Just
8ca875b6f4 PGBackend/ReplicatedBackend: move rollback methods into PGBackend
These will end up essentially unchanged in ECBackend as it turns out.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
4a25951633 osd_types: introduce spg_t, pg_shard_t
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
44b43df47f ObjectStore: use ghobject_t for setattrs
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
f2826fa2cf src/osd: pass in version to recover to recover_object
Otherwise we need to do some annoying contortions to
get it out of the missing sets.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
1c93e03f04 osd/: Add a shard parameter to SnapMapper to handle multiple pg shards
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
f678aefde3 TestPGLog: remove test with DELETE op with prior_version = eversion_t()
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Samuel Just
09d611d25c PGLog::merge_old_entry: simplify the oe.version > ne.version case
If ne.version < oe.version, the correct answer is to rollback oe.version
if possible regardless of what the entries are.

Also, rearrange to deal with the fact that we cannot roll back a missing
object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:10:18 -08:00
Samuel Just
d116e55f4a PG: just ignore rollbacks on objects we have already failed to rollback
The relevant changes to missing and/or the filestore will have
already been made.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:25:34 -08:00
Samuel Just
0e7b10ac9b PG::remove_snap_mapped_object: use pg_whoami.shard
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:25:33 -08:00
Samuel Just
fd9da00edc ReplicatedPG: take snapset_obc write lock where appropriate
Otherwise, we might read it for backfill before it's fully created
on a peer.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:25:33 -08:00
Samuel Just
da4652c92d ReplicatedPG: release backfill state on_global_recover not peer recover
There may now be multiple backfill peers.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
d687ed96fd ReplicatedPG: take clone write lock as well in make_writeable
Otherwise, we might start backfilling the clone before the op is
complete.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
643b6a50ae ReplicatedPG,osd_types: seperate require_rollback from ec_pool
It's handy to allow a pool to answer false to ec_pool() and
true to require_rollback() in order to allow a replicated
pool to test the rollback mechanisms without allowing
non-NO_SHARD shards.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
f020ce90f7 ReplicatedPG:: remove_snap_mapped_object already removes the object
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
17e6ecc841 ReplicatedPG: find_object_context debugging: output oi
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
b3b2bab403 ReplicatedPG: handle xattr cache on rollback and delete_head
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
d8b6d4c125 ReplicatedPG: release_op_ctx_locks in all-applied, not all-committed
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
13a41e14b6 FileStore: permit NULL onreadable callback
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
3650da69b5 ErasureCodeJerasure: fill in decoded if all shards are included
This causes it to match minimum_to_decode.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:56 -08:00
Samuel Just
44f5d4de2a ErasureCodeJerasure: fix alignement issues in encode/decode
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:55 -08:00
Samuel Just
06ec9bd42b ObjectStore: fix OP_COLL_ADD dump output
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:55 -08:00
Samuel Just
541a7e2725 TestRados: send aligned appends on erasure pools
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:55 -08:00
Samuel Just
e0b0508586 ReplicatedPG: reject unaligned writes on ec pools
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:55 -08:00
Samuel Just
211fc4e17c librados: add calls to expose required write alignment
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:55 -08:00
Samuel Just
4c1338f457 SimpleMessenger: init_local_connection whenever my_inst changes
This is necessary to correctly handle messages to self.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:24:55 -08:00
Sage Weil
093128a8b1 Merge pull request #1139 from ceph/wip-agent
osd: basic cache tier agent

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-02-17 14:14:21 -08:00
Sage Weil
bd8fcd2543 osd: improve whiteout debug prints
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 13:07:47 -08:00
Sage Weil
63f5a799aa osd/ReplicatedPG: make agent skip blocked obcs
Among other things, this can interfere with an in-flight copy-from and
corrupt the pg stats.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 13:07:39 -08:00
Sage Weil
79976466ca ReplicatedBackend: print char ack_type as int
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 13:07:39 -08:00
Sage Weil
62532fb025 Merge pull request #1254 from ceph/wip-barrier
libcephfs: disable barriers for now
2014-02-17 11:09:09 -08:00
Sage Weil
a4ca4a7347 Merge remote-tracking branch 'gh/wip-7212-sage-b'
We should carefully backport some or all of this.

Backport: emperor, dumpling
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-02-17 11:07:55 -08:00
Sage Weil
a71d829e46 client: disable barrier support
The boost interval_set class is not available on centos6/rhel6.  Until that
dependency is sorted out, fix the build.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 10:27:23 -08:00
Sage Weil
747002c6be client: fix barrier interval
(start, end) not (start, length)

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 10:23:37 -08:00
Sage Weil
d7457f7341 client/barrier: drop unused active_commit_interval
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 10:16:10 -08:00
Sage Weil
ce643e007a client: don't populate Client::barriers from read accesses
If the entry doesn't exist, no need to create it.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-17 10:15:58 -08:00
Sage Weil
a1dbc9ceec qa/workunits/suites/pjd: use test suite with acl tweak
Test 45 in xacl/00.t fails on ext4 and cephfs.  Just disable it.
Everything else passes.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-16 22:25:49 -08:00
Sage Weil
2a19a1bebe osd/ReplicatedPG: allow is_degraded_object() to be called outside of backfil
The agent does this.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-16 20:53:13 -08:00
Sage Weil
75675bc879 Merge pull request #1251 from ceph/wip-7371
ReplicatedPG: return no data if read size is trimmed to zero

Backport: emperor, dumpling
Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-16 08:26:33 -08:00
Yan, Zheng
1dae27c505 ReplicatedPG: return no data if read size is trimmed to zero
OSD should return no data if the read size is trimmed to zero by the
truncate_seq/truncate_size check. We can't rely on ObjectStore::read()
to do that because it reads the entire object when the 'len' parameter
is zero.

Fixes: #7371
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2014-02-16 22:36:09 +08:00
Sage Weil
dcb6d02c52 Merge pull request #1223 from ceph/wip-7395
Improve OSD subscription handling

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-15 22:36:23 -08:00
Sage Weil
8547f43eba Merge pull request #1234 from dachary/wip-format
mon: remove format argument from osd crush dump

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-15 22:21:57 -08:00
Sage Weil
774125c7a8 osd: set client incarnation for Objecter instance
Each ceph-osd process's Objecter instance has a sequence
of tid's that start at 1.  To ensure these are unique
across all time, set the client incarnation to the
OSDMap epoch in which we booted.

Note that the MDS does something similar (except the
incarnation is actually the restart count for the MDS
rank, since the MDSMap tracks that explicitly).

Backport: emperor
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-15 22:09:40 -08:00
Sage Weil
0dd1e07194 osd: schedule agent from a priority queue
We need to focus agent attention on those PGs that most need it.  For
starters, full PGs need immediate attention so that we can unblock IO.
More generally, fuller ones will give us the best payoff in terms of
evicted data vs effort expended finding candidate objects.

Restructure the agent queue with priorities.  Quantize evict_effort so that
PGs do not jump between priorities too frequently.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-15 22:09:40 -08:00
Sage Weil
a8129829ce osd/ReplicatedPG: simplify agent_choose_mode
Use a temp variable.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-15 22:09:40 -08:00
Sage Weil
905df2e729 osd/ReplicatedPG: block requests to cache PGs when they are full
If we are full and get a write request to a new object, put the op on a
wait list.  Wake up when the agent frees up some space.

Note that we do not block writes to existing objects.  That would be a
more aggressive strategy, but it is difficult to know up front whether we
will increase the size of the object or not, so we just leave it be.  I
suspect this strategy is "good enough".

Also note that we do not yet prioritize agent attention to PGs that most
need eviction (e.g., those that are full).

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-15 22:09:40 -08:00