Commit Graph

31728 Commits

Author SHA1 Message Date
Samuel Just
6c20728658 Merge remote-tracking branch 'upstream/next' 2014-02-20 17:04:14 -08:00
Samuel Just
2bcddce8b2 Merge remote-tracking branch 'upstream/wip-copyget' into next
Fixes: #7471
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-20 17:02:30 -08:00
Samuel Just
c6e6ced1de PG::build_might_have_unfound: skip CRUSH_ITEM_NONE
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-20 16:00:52 -08:00
Samuel Just
1f30d1ab9d ECBackend: deal with temp collection details in handle_sub_write
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-20 15:01:14 -08:00
Samuel Just
c703a89d0c ReplicatedPG::on_global_recover: requeue degraded, then unreadable
We need to requeue in reverse order since we are requeueing at the
front.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-20 13:22:51 -08:00
Yehuda Sadeh
caf2edfc43 rgw: minor cleanup
Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-02-20 09:19:13 -08:00
Mouad Benchchaoui
5fb90a2f6d Fix generate keystone token from credentials
Request to v2.0/tokens must be a POST request and the request
should specify JSON in the content-type header and fix request
body JSON generation to have the right JSON format.

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
Signed-off-by: Mouad Benchchaoui <m.benchchaoui@x-ion.de>
2014-02-20 08:59:48 -08:00
Ken Dreyer
c5c7f6c8e8 Merge branch 'next'
Conflicts:
	src/test/admin_socket.cc

Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
2014-02-20 03:06:00 +00:00
Samuel Just
ebd4397234 RadosModel: copyfrom should result in a dirty object
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 18:41:19 -08:00
Samuel Just
68184d4574 PG: fix scrub feature check
acting[0] isn't necessarily the primary.  Start at
0 and explicitely skip the primary.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 18:22:36 -08:00
Samuel Just
67d1f364e4 ReplicatedPG: assert no omap for ec pool in _write_copy_chunk and skip
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 17:31:37 -08:00
Samuel Just
4441792710 ECBackend: handle short reads
async reads may have a length past the end of the object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 17:31:37 -08:00
Samuel Just
715cbd3105 ReplicatedPG: ensure that ec appends are aligned
When not writing out the last chunk, _write_copy_chunk
will trim the chunk to a stripe boundary for ec pools
and adjust cursor.data_offset to compensate.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 17:31:37 -08:00
Samuel Just
78d9c0072b ReplicatedPG::fill_in_copy_get, use getattr_maybe_cache
Also, add user_only flag to getattr_maybe_cache and cleanup
the other user.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 17:31:36 -08:00
Samuel Just
c337f63806 ReplicatedPG: fix stat uses to use the correct shard
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-19 17:31:36 -08:00
Samuel Just
aa110af991 OSDMap::_apply_primary_affinity: skip CRUSH_ITEM_NONE in non-default check
Fixes: #7482
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
2014-02-19 17:30:32 -08:00
Sage Weil
b3bb9efea3 doc/release-notes: v0.77
Also pending notes for v0.78

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-19 16:57:25 -08:00
Sage Weil
fa96de9eeb doc/release-notes: v0.67.7
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-19 16:20:35 -08:00
Ken Dreyer
1bca9c5c41 v0.77 2014-02-19 22:54:44 +00:00
Samuel Just
40bdcb8850 osd/,mon/: add (up|acting)_primary to pg_stat_t
We use pg_stat_t information to determine pg create targeting.

Fixes: #7481
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-19 14:14:18 -08:00
Josh Durgin
72f260e07c Merge pull request #1256 from ceph/wip-librados-ops-and-rvals
librados c api for object operations and a few bug fixes

Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Sage Weil <sage.weil@inktank.com>
2014-02-19 11:18:57 -08:00
Samuel Just
314116e250 PG: insert into stray set if !us_up() && !is_acting()
This is simpler than checking actingbackfill
since it may not yet be filled in.

Fixes: #7470
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-18 22:20:40 -08:00
Sage Weil
a33874d036 Merge pull request #1257 from ceph/wip-agent
osd: a few agent fixes

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-02-18 21:10:04 -08:00
Sage Weil
a5c5d924be osd/ReplicatedPG: EOPNOTSUPP on hit_set_get on non-replicated pools
We need to use the async read stuff for this to work.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-18 21:05:34 -08:00
Sage Weil
a40cd5075c osd/ReplicatedPG: load older HitSets into memory
If our evict_mode is non-idle, load older HitSets into memory in the agent
work thread.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-18 21:05:34 -08:00
Sage Weil
0af73755fa osd/ReplicatedPG: strengthen agent_work assertions
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-18 21:05:34 -08:00
Samuel Just
1d907c60d7 OSD: consider up/acting primary for new intervals
Both acting_up_affected and start_peering_interval need
to consider primary changes as well as acting/up changes.

Fixes: #7469
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-18 19:46:33 -08:00
Josh Durgin
1ff0eefeb1 Merge pull request #1259 from ceph/wip-6677
rgw: add compatibility for MultipartUpload

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 15:38:34 -08:00
Josh Durgin
a2c195d85b Merge pull request #1258 from ceph/wip-rgw-gc
radosgw-admin: gc list --include-all

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 15:37:58 -08:00
Yehuda Sadeh
b0dcc798e8 radosgw-admin: gc list --include-all
A new option to the radosgw-admin gc list command that dumps *all* gc
entries, and not just the expired ones. This is useful for debugging.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-02-18 15:43:44 -08:00
Josh Durgin
82cf979546 Merge pull request #1261 from ceph/wip-7463
ceph_rest_api.py: don't fail if no up OSDs found on startup

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 13:41:35 -08:00
Mike Lundy
609f4c5671 Throw a Timeout exception on timeout.
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 13:25:49 -08:00
Josh Durgin
c5d3bdf7fa Merge pull request #1214 from ceph/wip-rgw-manifest-2
Wip rgw manifest 2
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 13:07:53 -08:00
Yehuda Sadeh
197544137e dencoder: fix for new rgw manifest code
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-02-18 13:11:35 -08:00
Yehuda Sadeh
b3ce18891f cls/rgw: fix debug output
print the correct error value

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-02-18 13:11:35 -08:00
Yehuda Sadeh
3fb6e25846 test/rgw: manifest unitest
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-02-18 13:11:35 -08:00
Yehuda Sadeh
da64bf70be rgw: scalable manifest object
Define a new manifest structure. The idea is that the manifest defines a
set of rules for structuring the object parts. There are a few terms to
note:
 - head: the head part of the object, which is the part that contains
   the first chunk of data. An object might not have a head (as in the
   case of multipart-part objects.
 - stripe: data portion of a single rgw object that resides on a single
   rados object.
 - part: a collection of stripes that make a contiguous part of an
   object. A regular object will only have one part (although might have
   many stripes), a multipart object might have many parts. Each part
   has a fixed stripe size, although the last stripe of a part might
   be smaller than that. Consecutive parts may be merged if their stripe
   value is the same.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-02-18 13:11:34 -08:00
Dan Mick
db88e7f385 ceph_rest_api.py: don't fail if no up OSDs found on startup
allow find_up_osd() to return None, and have caller check for it

Fixes: #7463
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2014-02-18 13:07:32 -08:00
Josh Durgin
7a019b386c test/librados: remove unused lines and fix return code for cmp guards
Now that return values are actually set by the osd and client, fix up
the tests that were checking the old behavior.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
bfbfcd6ead ReplicatedPG: fix successful write condition
ctx->modify no longer implies that the operation is a write,
if it ever did. These days op->may_write() is checked reliably
on the OSD, so just use that instead.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
18f08cd9c0 ReplicatedPG: clear osd op reply output for writes
Since the vector of OSDOps used by the reply is the same
as the one processed by do_osd_ops() now, any output data
needs to be cleared for writes. To be compatible with current behavior,
allow writes that aren't applying anything or have failed to return data
still.

Add a new parameter to the MOSDOpReply constructor to determine
whether the output data should be cleared. Clear it for successful
writes, and remove a redundant result < 0 -> result > 0 check in the
process.  This was caught by ceph_test_cls_hello and its
writes_dont_return_data method.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
1d661ca972 ReplicatedPG: set individual osd op rvals
This takes care of things that can fail before transaction is
executed, like omap comparison. Getting individual rvals from a
transaction requires more refactoring, so I'm leaving it for a later
cleanup.

Fixes: #6483
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
7743698907 ReplicatedPG: don't copy the ops vector into OpContext
There's no need to copy it, and if we refer to the same vector from
the original MOSDOp, we can modify rvals for individual ops, since the
MOSDOp ops vector is copied to the MOSDOpReply when it is constructed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
e550ad7915 ReplicatedPG: avoid undefined behavior in xattr comparison
Reading past the end of a pointer returned by string.data() in c++98
is undefined. While we're fixing this, also allow comparison of xattrs
containing null bytes.

Fixes: #7250
Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
4b229f71ca qa: add librados c object operations tests to librados test script
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
7a760337e2 packaging: add test programs for librados c object operations
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
bac7bb6480 librados: bump version number
This covers the many recent changes leading up to the firefly release,
including the c object operations api.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
995fea9a44 librados: add omap object operations to the c api
Create an iterator type similar to XattrIter for returning results.
It just wraps the std::map to preserve sorting.

When getting omap keys or values, use an extra callback to set up the
iterator, and default optional parameters to the empty string.

Since it's hard to test read or write omap ops in isolation, combine
them all in the c read ops test case.

One difference between this and the c++ api is that omap_cmp only
allows one assertion per sub-op, for simplicity of the
interface. Multiple omap_cmp operations can still be added to the same
op to get the same effect.

Fixes: #7194
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
242a6c8523 librados: add read xattr functions for c object operations
Only implement string xattr comparison by since integer xattr
comparison assumes ceph-encoded integers in little-endian 64 bit
format, which should not be exposed to librados users.

Add an extra callback for getxattr so that we can initialize the
iterator.

Fixes: #7193
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00
Josh Durgin
f68b706e5e Objecter: add a way to chain callbacks
The librados C api needs to do extra things like converting c++ data
structures or setting lengths, but some objecter operations already
have out_handlers that librados shouldn't override.

Add a method that chains contexts together by using a new context that
calls both the original and the newly added one.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-18 12:34:33 -08:00