Commit Graph

20902 Commits

Author SHA1 Message Date
Sage Weil
1b27a7cd4d client: renew/submit subs on startup
This fixes a delay in getting the initial MDSMap during startup (we wait
for the MonClient tick()).

Reported-by: Noah Watkins <jayhawk@cs.ucsc.edu>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-30 14:03:11 -07:00
John Wilkins
a23351efda :doc: Added constraints and revised a description per joshd.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 11:48:47 -07:00
Tommi Virtanen
f2c4510848 upstart, ceph-create-keys: Make client.admin key generation automatic.
This should help simplify Chef etc deployments. Now (when using the
Upstart jobs), when a ceph-mon is started, ceph-create-admin-key is
triggered. If /etc/ceph/$cluster.client.admin.keyring already exists,
it does nothing; otherwise, it waits for ceph-mon to reach quorum, and
then does a "ceph auth get-or-create" to create the key, and writes it
atomically to disk.

The equivalent code can be removed from the Chef cookbook once this is
in.
2012-08-30 13:41:20 -04:00
Tommi Virtanen
0de719567f upstart: Make instance jobs export their cluster and id variables.
This allows other jobs listening to Upstart "started ceph-mon" events
to see what instance started.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-08-30 10:25:07 -04:00
Tommi Virtanen
3b0e2a2d98 config: Add a per-name default keyring to front of keyring search path.
This lets us have e.g. /etc/ceph/ceph.client.admin.keyring that is
owned by root:admin and mode u=rw,g=r,o= without making every non-root
run of the command line tools complain and fail.

This is what the Chef cookbook has been doing for a while already.
2012-08-30 10:25:07 -04:00
John Wilkins
8b4a9ade32 doc: Modified index page with table. Pools and Auth will go to Ops.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:38:01 -07:00
John Wilkins
b961a660be doc: Some improvements. This still needs review.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:37:02 -07:00
John Wilkins
c41fa39460 doc: Added additional note to mount data disk to data path.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:35:31 -07:00
John Wilkins
b80a1b73ed doc: Added a document for general config settings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:33:58 -07:00
John Wilkins
2407b0cdef doc: Added a doc for filestore settings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:29:14 -07:00
John Wilkins
17dee291e8 doc: Added a document for RBD cache settings.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:28:29 -07:00
John Wilkins
69e10167f4 doc: Added substantial detail to OSD setting descriptions.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:26:38 -07:00
John Wilkins
c0c9fe5400 doc: minor edit to use # for comments.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:25:17 -07:00
John Wilkins
98eaaccd13 doc: Restructured ceph config. Added network and OSD config details.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-30 00:24:26 -07:00
Yehuda Sadeh
4d9802c414 rgw: remove unused block of code
We were reading bucket info, but that wasn't necessary.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-29 14:20:38 -07:00
caleb miles
e3834f023b rgw_admin.cc: Prevent clobbering the index when linking a bucket.
Prevent the 'bucket link' command from overwriting the index of an
existing bucket. Corrects bug 2935:

http://tracker.newdream.net/issues/2935

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-08-29 12:45:13 -07:00
Yehuda Sadeh
82c62bd977 rgw: clear usage map before reading usage
Fixes: #3057
Since we read usage in chunks we need to clear the
usage map before reading the next chunk, otherwise
we're going to aggregate the old data as well.

Backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 16:47:41 -07:00
Sage Weil
0052a78726 Merge branch 'next' 2012-08-28 15:15:08 -07:00
Sage Weil
5f36b8d784 osd: fix waiting_for_disk assertion
If requeue is false, we won't have cleared out waiting_for_ondisk; adjust
assert placement as appropriate.  Also, make sur we handle the requeue
and !op case properly (although I'm not sure offhand if/when it would
come up).

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-08-28 15:14:41 -07:00
Mike Ryan
745a3c9ba0 rados_bench: wait for completion callbacks before returning
If we don't wait for the callback, the finisher may cleanup the callback
context before the callback is actually invoked, causing a
use-after-free error.

This fixes #3048.

Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-08-28 14:21:13 -07:00
Yehuda Sadeh
dfb7f80409 Makefile.am: add missing .h
Was missing rgw_html_errors.h

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 14:13:53 -07:00
Yehuda Sadeh
0ab6232503 Merge remote-tracking branch 'origin/wip-multi-delete'
Conflicts:
	src/Makefile.am

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 13:36:35 -07:00
Mike Ryan
2d79a62a11 rados_bench: wait for completion callbacks before returning
If we don't wait for the callback, the finisher may cleanup the callback
context before the callback is actually invoked, causing a
use-after-free error.

This fixes #3048.

Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-08-28 13:28:52 -07:00
John Wilkins
a7867c27cb doc: Completed and reviewed RGW config reference.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 13:25:44 -07:00
John Wilkins
8d7912cba0 doc: added admonishment. Updated header syntax, copy semantics and x-ref.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 13:24:29 -07:00
Yehuda Sadeh
5507665a58 rgw: fix usage log read
The usage log read got broken in a recent cleanup work.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-28 12:51:55 -07:00
John Wilkins
3eea9104af doc: Added internal references. Clarified language in disk prepare.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 11:41:59 -07:00
John Wilkins
a466dfff2d doc: added sudo for hdparm command.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 11:02:13 -07:00
John Wilkins
080e5722b3 doc: added internal hyperlink targets.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 10:55:04 -07:00
John Wilkins
dd011aba90 doc: Clean up quick start to ensure nobody uses "localhost".
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 10:01:20 -07:00
John Wilkins
9755466619 doc: Cleaned up syntax errors, and converted table to list.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-08-28 09:24:22 -07:00
Sage Weil
5630e81232 Merge branch 'next' 2012-08-27 17:44:53 -07:00
Sage Weil
15995ea1c4 Merge branch 'wip-objecter' into next
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-08-27 17:26:13 -07:00
Sage Weil
2a3b7961c0 objecter: fix skipped map handling
If we skip a map, we want to translate NO_ACTION to NEED_RESEND, but leave
POOL_DNE alone.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:57 -07:00
Sage Weil
8d1efd1b82 objecter: send queued requests when we get first osdmap
If we get our first osdmap and already have requests queued, send them.

Fixes: #3050
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:57 -07:00
Sage Weil
e59b9daefc objecter: fix is_latest_map() retry on mon session restart
If the mon session drops, we get an EAGAIN callback, which we already
correctly ignored.  (Clean this up and comment so it's clearer what is
going on.)

Fix ms_handle_connect() to resubmit those requests.

Noticed while fixing #3049.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:57 -07:00
Sage Weil
7d40cba241 monclient: pass EAGAIN to is_latest_map() callers
If our map get_version check needs to be retried, tell the
is_latest_map() callers instead of giving returning 0 ("no").

Fixes: #3049
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:56 -07:00
Sage Weil
0adc2289d6 monclient: document get_version(), and fix return value
Return -EAGAIN instead of -1, since that's more meaningful, and
document it.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 17:25:54 -07:00
caleb miles
0a1f4a97da Implement multi-object delete.
An implimentation of multi-object delete described in
the latest Amazon S3 API provied at

http://docs.amazonwebservices.com/AmazonS3/latest/API

This commit is in response to tracker issue 2797

http://tracker.newdream.net/issues/2797

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-08-27 17:08:44 -07:00
Sage Weil
17ceec0d10 osd: requeue dup ops inline with in-progress ops
We should requeue the dups along with the originals.  This avoids
situations where, after requeue, the dups are reordered with respect to
each other.  For example:

 - client sends A, B, C
 - osd receives A
 - connection drops
 - client sends A', B', C'
 - osd puts A' in waiting_for_ondisk, starts B' and C'
 - on_change() requeues everything

Final queue order (before this patch) is
    A, B', C', A'

After this patch, the resulting queue order is
    A, A', B', C'

Or somewhat more generally, it might be:

    A, A', B, B', B'', C', C'', D'', ....

Fixes (another source of): #2947
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-08-27 16:47:36 -07:00
Sage Weil
c7054933e0 Merge remote-tracking branch 'gh/wip-mon-intparsing'
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2012-08-27 15:10:35 -07:00
Sage Weil
d5cacaca50 osd: include notif pointer in notify debug output
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 14:57:39 -07:00
Sage Weil
0a2ec988f1 config: add 'fatal signal handlers' option
This will let us disable the sighandlers for SEGV, etc.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 14:57:35 -07:00
Sage Weil
7a631f9476 cls_rgw_client: fix #include path
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 14:41:09 -07:00
Yehuda Sadeh
fa74e0476c Merge remote-tracking branch 'origin/master' into wip-gc2 2012-08-27 12:43:27 -07:00
Yehuda Sadeh
6f68ff5cca cls_rgw: add cls_rgw unitest, test gc api
Test various cls_rgw gc related functionality.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-27 12:42:43 -07:00
Yehuda Sadeh
a30f7140f0 rgw-admin: get rid of lazy remove option, other fixes
was mishandling parsing of binary flag arguments.
also, fix argument parsing and update radosgw-admin
cli test reference.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-27 12:41:27 -07:00
Yehuda Sadeh
721a6bef9e rgw: implement garbage collector
Add a garbage collector thread that is responsible for clean
up of clutter. When removing an object, store info about the
leftovers in a special gc map (via rgw objclass). A new
radosgw-admin commands to list objects in gc, and to run the
gc process manually. Also, gc processors can run in parallel,
however, each will handle a single gc shard (synchronized
using lock objclass).

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-08-27 12:41:19 -07:00
Sage Weil
bd534bf328 mon: make parse_pos_long() error message more helpful
Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-27 08:36:41 -07:00
Sage Weil
c7d11cd7b8 osd: turn off lockdep during shutdown signal handler
We don't shut down all threads, and the surviving ones fight with
exit()'s teardown.  Kludge until we have a clean shutdown process.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-08-26 08:42:06 -07:00