Commit Graph

20894 Commits

Author SHA1 Message Date
Yehuda Sadeh
93218aeab7 rgw_rados: adjustments for refcount objclass
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:01:25 -07:00
Yehuda Sadeh
cfe4830516 rgw: set atomic context for copy operation src and dest
This is required so that we handle both src and dest atomically. We
also set the prefetch flag on the src object, so that we read the
first chunk along with its attrs.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 12:56:02 -07:00
Yehuda Sadeh
3cd17fb9dd rgw: copy of objects using refcount
For objects with manifest that have a tail, we
copy only the head, and the manifest, and increasing
the reference count on the tail objects.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 12:52:26 -07:00
Yehuda Sadeh
20c785278c rgw: use refcount put instead of obj delete where needed
Tie in new refcount objclass methods.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 12:51:35 -07:00
Yehuda Sadeh
5868b00c78 cls_refcount: a new class for objects refcounting
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-05 14:57:08 -07:00
Yehuda Sadeh
f85863e9c5 objclass: add remove operation
Added a missing remove object operation

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-30 13:37:03 -07:00
Yehuda Sadeh
4d9802c414 rgw: remove unused block of code
We were reading bucket info, but that wasn't necessary.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-29 14:20:38 -07:00
caleb miles
e3834f023b rgw_admin.cc: Prevent clobbering the index when linking a bucket.
Prevent the 'bucket link' command from overwriting the index of an
existing bucket. Corrects bug 2935:

http://tracker.newdream.net/issues/2935

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-08-29 12:45:13 -07:00
Yehuda Sadeh
82c62bd977 rgw: clear usage map before reading usage
Fixes: #3057
Since we read usage in chunks we need to clear the
usage map before reading the next chunk, otherwise
we're going to aggregate the old data as well.

Backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 16:47:41 -07:00
Sage Weil
0052a78726 Merge branch 'next' 2012-08-28 15:15:08 -07:00
Sage Weil
5f36b8d784 osd: fix waiting_for_disk assertion
If requeue is false, we won't have cleared out waiting_for_ondisk; adjust
assert placement as appropriate.  Also, make sur we handle the requeue
and !op case properly (although I'm not sure offhand if/when it would
come up).

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-08-28 15:14:41 -07:00
Mike Ryan
745a3c9ba0 rados_bench: wait for completion callbacks before returning
If we don't wait for the callback, the finisher may cleanup the callback
context before the callback is actually invoked, causing a
use-after-free error.

This fixes #3048.

Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-08-28 14:21:13 -07:00
Yehuda Sadeh
dfb7f80409 Makefile.am: add missing .h
Was missing rgw_html_errors.h

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 14:13:53 -07:00
Yehuda Sadeh
0ab6232503 Merge remote-tracking branch 'origin/wip-multi-delete'
Conflicts:
	src/Makefile.am

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 13:36:35 -07:00
Mike Ryan
2d79a62a11 rados_bench: wait for completion callbacks before returning
If we don't wait for the callback, the finisher may cleanup the callback
context before the callback is actually invoked, causing a
use-after-free error.

This fixes #3048.

Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-08-28 13:28:52 -07:00
John Wilkins
a7867c27cb doc: Completed and reviewed RGW config reference.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 13:25:44 -07:00
John Wilkins
8d7912cba0 doc: added admonishment. Updated header syntax, copy semantics and x-ref.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 13:24:29 -07:00
Yehuda Sadeh
5507665a58 rgw: fix usage log read
The usage log read got broken in a recent cleanup work.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 12:51:55 -07:00
John Wilkins
3eea9104af doc: Added internal references. Clarified language in disk prepare.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 11:41:59 -07:00
John Wilkins
a466dfff2d doc: added sudo for hdparm command.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 11:02:13 -07:00
John Wilkins
080e5722b3 doc: added internal hyperlink targets.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 10:55:04 -07:00
John Wilkins
dd011aba90 doc: Clean up quick start to ensure nobody uses "localhost".
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 10:01:20 -07:00
John Wilkins
9755466619 doc: Cleaned up syntax errors, and converted table to list.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 09:24:22 -07:00
Sage Weil
5630e81232 Merge branch 'next' 2012-08-27 17:44:53 -07:00
Sage Weil
15995ea1c4 Merge branch 'wip-objecter' into next
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-08-27 17:26:13 -07:00
Sage Weil
2a3b7961c0 objecter: fix skipped map handling
If we skip a map, we want to translate NO_ACTION to NEED_RESEND, but leave
POOL_DNE alone.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:57 -07:00
Sage Weil
8d1efd1b82 objecter: send queued requests when we get first osdmap
If we get our first osdmap and already have requests queued, send them.

Fixes: #3050
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:57 -07:00
Sage Weil
e59b9daefc objecter: fix is_latest_map() retry on mon session restart
If the mon session drops, we get an EAGAIN callback, which we already
correctly ignored.  (Clean this up and comment so it's clearer what is
going on.)

Fix ms_handle_connect() to resubmit those requests.

Noticed while fixing #3049.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:57 -07:00
Sage Weil
7d40cba241 monclient: pass EAGAIN to is_latest_map() callers
If our map get_version check needs to be retried, tell the
is_latest_map() callers instead of giving returning 0 ("no").

Fixes: #3049
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:56 -07:00
Sage Weil
0adc2289d6 monclient: document get_version(), and fix return value
Return -EAGAIN instead of -1, since that's more meaningful, and
document it.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:54 -07:00
caleb miles
0a1f4a97da Implement multi-object delete.
An implimentation of multi-object delete described in
the latest Amazon S3 API provied at

http://docs.amazonwebservices.com/AmazonS3/latest/API

This commit is in response to tracker issue 2797

http://tracker.newdream.net/issues/2797

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-08-27 17:08:44 -07:00
Sage Weil
17ceec0d10 osd: requeue dup ops inline with in-progress ops
We should requeue the dups along with the originals.  This avoids
situations where, after requeue, the dups are reordered with respect to
each other.  For example:

 - client sends A, B, C
 - osd receives A
 - connection drops
 - client sends A', B', C'
 - osd puts A' in waiting_for_ondisk, starts B' and C'
 - on_change() requeues everything

Final queue order (before this patch) is
    A, B', C', A'

After this patch, the resulting queue order is
    A, A', B', C'

Or somewhat more generally, it might be:

    A, A', B, B', B'', C', C'', D'', ....

Fixes (another source of): #2947
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-08-27 16:47:36 -07:00
Sage Weil
c7054933e0 Merge remote-tracking branch 'gh/wip-mon-intparsing'
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-08-27 15:10:35 -07:00
Sage Weil
d5cacaca50 osd: include notif pointer in notify debug output
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 14:57:39 -07:00
Sage Weil
0a2ec988f1 config: add 'fatal signal handlers' option
This will let us disable the sighandlers for SEGV, etc.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 14:57:35 -07:00
Sage Weil
7a631f9476 cls_rgw_client: fix #include path
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 14:41:09 -07:00
Yehuda Sadeh
fa74e0476c Merge remote-tracking branch 'origin/master' into wip-gc2 2012-08-27 12:43:27 -07:00
Yehuda Sadeh
6f68ff5cca cls_rgw: add cls_rgw unitest, test gc api
Test various cls_rgw gc related functionality.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-27 12:42:43 -07:00
Yehuda Sadeh
a30f7140f0 rgw-admin: get rid of lazy remove option, other fixes
was mishandling parsing of binary flag arguments.
also, fix argument parsing and update radosgw-admin
cli test reference.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-27 12:41:27 -07:00
Yehuda Sadeh
721a6bef9e rgw: implement garbage collector
Add a garbage collector thread that is responsible for clean
up of clutter. When removing an object, store info about the
leftovers in a special gc map (via rgw objclass). A new
radosgw-admin commands to list objects in gc, and to run the
gc process manually. Also, gc processors can run in parallel,
however, each will handle a single gc shard (synchronized
using lock objclass).

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-27 12:41:19 -07:00
Sage Weil
bd534bf328 mon: make parse_pos_long() error message more helpful
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 08:36:41 -07:00
Sage Weil
c7d11cd7b8 osd: turn off lockdep during shutdown signal handler
We don't shut down all threads, and the surviving ones fight with
exit()'s teardown.  Kludge until we have a clean shutdown process.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-26 08:42:06 -07:00
Sage Weil
0e091d81a1 v0.51
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.11 (GNU/Linux)
 
 iQIcBAABAgAGBQJQOVjIAAoJEH6/3V0X7TFtdFIQAKyU+6kldJE2YZO5GOO7jPb2
 vGAhsYpvuS/Vx87yrSa7Xavz/C/frKz5m+5SsmxbZl+ditRLCGAD/BlQIuj0UWAW
 MxURFK9hjwJK23fuuuXUXEbMmABmRP8XlzG9IGl5yRM07+IUl8aMGy7+i4yGzGFX
 QVMHC1qMD70SAQ+q2/JVXlVxkVPzqzf9iT+xuFk28V8A0ZLlSAfTuSHD9YLJiWaV
 SjR/vVLpajaTR3ytkSxrG1fwuqENf9OThLXxHuyplZvTUIuAxbxBlWSMJmuLQ3JF
 JNX/N0/z9Omw+ipJAvM/nS6TbT0X2KhMYjObINOVUiDkwC9jBznCl8A1b/hy7wJX
 haTUat6OW3taGysP3AkOddwkyDHLJxz/UoUtPbEgT/mDOB8CwWEdpgkL8wsvNUgK
 n2yEJNpjhQ2QG2LC/n0x67jVlt0B4IRMbijFAoySyklfnJjU8J+Uyjl4bentDvM7
 cQrIIBobQMbc9urcSWzxMd6+fCvxEvtXY027LVP7K3hS3thS2tPRT3WT6vAZ7vih
 foOyc2a9SQxwDWa3bf7d5yoL7nLB9KfRXIbHu31EKgM5pw8Lgy1vRtaqEKOh9Lup
 l8pk5/2ABmy2pYaeLGyTnZN+8BsR5ZYyqJ2nUL/VbSmReto1BIRrI4zhEAAlNWWN
 nKrNwX4xOZjDX9ghsMUv
 =t5SY
 -----END PGP SIGNATURE-----

Merge tag 'v0.51'

v0.51
2012-08-26 08:18:45 -07:00
Sage Weil
c03ca95d23 v0.51 2012-08-25 15:58:39 -07:00
Sage Weil
aa91cf81af mon: require --id
Fixes: #2997
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-25 15:29:56 -07:00
Sage Weil
5fd2f10266 mon: fix int parsing in monmon
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-24 16:05:07 -07:00
Sage Weil
31c8ccb849 mon: check for int parsing errors in mdsmon
Fixes: #3014
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-24 16:03:02 -07:00
Sage Weil
304c08efbe mon: check for int parsing errors in osdmon
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-24 16:02:02 -07:00
Sage Weil
3996076722 interval_set: predeclare const_iterator
This makes the coverity build happier.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-08-24 14:55:41 -07:00
Sage Weil
ef4ab901b3 Makefile: update coverity rules
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-08-24 14:55:40 -07:00