Commit Graph

30554 Commits

Author SHA1 Message Date
Loic Dachary
938f22cae2 common: format Throttle test to 80 columns
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-15 14:30:38 +01:00
Loic Dachary
ba5572397c common: fix perf_counters unittests for trailing newline in m_pretty
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-15 13:24:14 +01:00
Loic Dachary
c744aec660 Merge pull request #929 from kazhang/add-pkg-config
add apt-get install pkg-config for ubuntu server

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-15 03:26:21 -08:00
Sage Weil
0dc59af993 osd/ReplicatedPG: fix promote: set oi.size
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
697151eaec osd/osd_types: fix operator<< on copy-get operation
This was missed in 15c8267e34.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
f50389d741 ceph_test_rados_api_tier: test undirty on non-existent object
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
f86d6e7794 osd/ReplicatedPG: debug: improve maybe_handle_cache() handling
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
81279e3bb6 osd/ReplicatedPG: rename invalidate_forward
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
87547bde9a ceph_test_rados: debug: include exists|dne in update_object_version
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
d1e63b3cfe ceph_test_rados: test is_dirty, undirty
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:57 -08:00
Sage Weil
14f76cc264 ceph_test_rados: fix CopyFromOp locking
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
41be4feb35 librados: seek during object iteration
Add ability to reset iterator to a specific hash position.  For now, we
just truncate this to the current PG.  In the future, this may be more
precise.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
330a13059f osdc/Objecter: remove honor_cache_redirects global flag
We can do this on a per-op basic with CEPH_OSD_FLAG_IGNORE_OVERLAY.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
42d6af1c30 osd/ReplicatedPG: use IGNORE_OVERLAY flag for copy-from
No need to use the Objecter-wide setting now.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
067536c2c6 osdc/Objecter: add CEPH_OSD_FLAG_IGNORE_OVERLAY flag
If the flag is set, send the op to the pool specified and ignore the
overlay.  Note that this obsoletes the global Objecter flag.

It also makes these EINVAL correctly:

  rados -p base cache-flush
  rados -p base cache-evict

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
3d9c49974e osd: rename IGNORE_OVERLAY -> IGNORE_CACHE
This is about skipping cache logic, not the tier pool overlay property.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
ea088fae6a osd/osd_types: operator<< for ObjectContext::RWState
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
c0e4ed3489 osd/ReplicatedPG: more verbose heading for process_copy_chunk
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:56 -08:00
Sage Weil
90eb1ec1e0 osd/ReplicatedPG: set ctx->obc in simple_repop_create
Strangely nobody hss needed this yet, but we will shortly.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:55 -08:00
Sage Weil
ca86656e74 osd/ReplicatedPG: use finish_ctx for finish_promote
Use the common code here to avoid duplicating this logic.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:55 -08:00
Sage Weil
66263bb6ff osd/ReplicatedPG: use get_next_version() in finish_promote
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:55 -08:00
Sage Weil
56ad14ec1f osd/ReplicatedPG: split off finish_ctx from execute_ctx
The second part of execute_ctx() is doing some somewhat generic work to
make the prepared updates in the ctx apply, updating the obc's cached
values.  Factor it out.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:55 -08:00
Sage Weil
3ef731068c osd/ReplicatedPG: add SKIPRWLOCKS flag
Flush puts us in an conundrum:

 - the flush eventually writes, behaving like a write
 - writes take the write lock at the start
 - to flush, we send copy-from to the base pool, which does a copy-get on
   our object
 - the copy-get is a read, that blocks on the write.

This flag will allow an op to skip the initial locking step.  It will need
to take it later, of course.

Signed-off-by: Sage Weil <sage@inktank.com>

Conflicts:

	src/osd/ReplicatedPG.cc
2013-12-13 16:35:55 -08:00
Sage Weil
5e547f8772 osd/ReplicatedPG: be consistent about ctx->obs vs ctx->obc->obs
Just for consistency (ctx->obs =- &ctx->obc->obs).

Signed-off-by: Sage Weil <sage@inktank.com>

Conflicts:

	src/osd/ReplicatedPG.cc
2013-12-13 16:35:55 -08:00
Sage Weil
36bbcf8e55 osd/ReplicatedPG: drop unnecessary temp vars in execute_ctx()
Both of these are pulled out of ctx->obs, which is not updated until the
very end; use that instead!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:55 -08:00
Sage Weil
10c9be3401 osd/ReplicatedPG: allow osds to issue writes to osds
We asserted that the client was not an OSD years ago when we separated out
the client and cluster networks.  Now, we are about to allow an OSD to
trigger a copy_from on another pool (for cache flush) and the assert can
go away.  We've long since verified that the messages are going out on
the correct interfaces.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:55 -08:00
Sage Weil
20d149e198 osd/ReplcatedPG: maybe_handle_cache style
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Sage Weil
0b81ff68c0 osd/ReplicatedPG: skip promote for DELETE
If an op starts with DELETE there is no need to promote the old content
from the base tier.  Note that this only works if the FAILOK flag is
set.  Otherwise, we need to know whether the object existed or not to
return either 0 or -ENOENT.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Sage Weil
4c014eddbe osd/ReplicatedPG: implement cache_evict
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Sage Weil
8b9b7136ba librados: add an aio_operate that takes a write and flags
Until now you could only pass flags to read operations.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Greg Farnum
85282319ee osd/osd_types: introduce helper for osd op flags -> string conversion
Signed-off-by: Sage Weil <sage@inktank.com>

Conflicts:

	src/osd/osd_types.h
2013-12-13 16:35:54 -08:00
Sage Weil
181cb8e83c librados, osd: add IGNORE_OVERLAY flag
Add a flag that will make the OSD bypass the cache overlay logic.  This is
needed in order to handle operations like CACHE_EVICT and CACHE_FLUSH.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Sage Weil
387e224aa2 librados: add cache_flush(), cache_try_flus(), cache_evict() methods
Not yet implemented by the OSD.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Sage Weil
78df1c37df osd/ReplicatedPG: set object_info and snapset xattrs on promote
For the normal write path, prepare_transaction() handles this for us.  In
this case, we need to do it explicitly.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:54 -08:00
Sage Weil
dd079e2a5f osd/ReplicatedPG: handle is_whiteout in do_osd_ops()
Most of the time we handle whiteouts by returning ENOENT before we even
get this far. However, for a mixed read/write transaction (e.g., a guard)
or certain ops (like create exclusive) we need to deal with the
exists == true and whiteout flag set case explicitly.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
fd8f7d295a osd/ReplicatedPG: clear whiteout when writing into cache tier
If we have a whiteout object and then write over it, clear the whiteout
flag.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
fabc6ba161 osd/ReplicatedPG: set whiteout in cache pool on delete
If we delete an object in the cache pool, set the whiteout flag instead of
removing the on-disk object.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
2aea631c4c ceph_test_rados_api_tier: verify delete creates whiteouts
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
e0a49698ec osd/ReplicatedPG: ENOENT when deleting a whiteout
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
0b085b174a osd/ReplicatedPG: create whiteout on promote ENOENT
If we try to fetch an object from the base tier and it is not present, we
can create a whiteout object.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
0b7b16d7fa ceph_test_rados_api_tier: add simple promote-on-read test
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
be29f47ac0 ceph_test_rados_api_tier: rename tests
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:53 -08:00
Sage Weil
66f2e7489d osd/ReplicatedPG: use simple_repop_{create,submit} for finish_promote
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Sage Weil
654d8c3334 osd/ReplicatedPG: UNDIRTY is not a user_modify
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Sage Weil
4a29b22e2b osd/ReplicatedPG: move r<0 handling into finish_promote()
Let logic in header, and will let us handle ENOENT with a whiteout.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Greg Farnum
920c0bff5b workunits: break down cache pool tests to be more precise; expand some
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Greg Farnum
0caa02c5af workunits: check errors propagate on cache pools in caching_redirects.sh
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Greg Farnum
5fa08fb8de ReplicatedPG: promote: handle failed promotes
If we get an error back, reply to the client directly and remove
the op which triggered promotion from our blocked op queue.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Greg Farnum
d15aedbd25 ReplicatedPG: promote: add the OpRequest to the Callback
This way we can do stuff to it, and we're about to.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-13 16:35:52 -08:00
Greg Farnum
b371dd8b71 ReplicatedPG: promote: first draft pass at doing object promotion
This is not yet at all complete -- among other things, it will
retry forever on any object which doesn't exist in the underlying
pool. But it demonstrates the approach reasonably clearly.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>y
2013-12-13 16:35:52 -08:00