Commit Graph

21862 Commits

Author SHA1 Message Date
Yehuda Sadeh
2b6a27d3c1 rgw: add a flusher to handle output data, simplify client api
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:12:24 -07:00
Yehuda Sadeh
cecd7afa4f Formatter: fix XML formatter
XML formatter wasn't clearing stream string buffer
when completing the section.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:17 -07:00
Yehuda Sadeh
02f7870548 rgw: add rest-usage handler
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:17 -07:00
Yehuda Sadeh
fca6d3b992 rgw: REST resources init uses configurables
Also, move it to main().

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:17 -07:00
Yehuda Sadeh
cea28f8cfb rgw: refactor protocol stacking
New RESTMgr* classes to handle a specific uri entry
point. Actions are handled by RGWHandler*.

Changing init order, newly instantiated RGWHandler
for every op.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:12 -07:00
Yehuda Sadeh
85f137d2a1 rgw: tie callbacks in different handlers directly to REST
Don't translate RESTful operations into a more meaningful
callback name. The handlers themselves should do that
translation. This way we can later register different
handlers with different meanings for the operations.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:48 -07:00
Yehuda Sadeh
b71b8b09e1 rgw: sed 's/_REST/_ObjStore/g' *
It's all REST, we want to add new non objstore handling,
so let's differentiate by the actual type.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:48 -07:00
Yehuda Sadeh
e153bceef2 rgw: start moving usage management functions
Move usage management functionality to its own place.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:48 -07:00
Yehuda Sadeh
a9c9f96ba5 rgw: hide fcgi details from client code
client code (rgw_rest* et al) does not need to see the
fcgi library details.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:43 -07:00
John Wilkins
c038c3f653 doc: Added admonishments to use separate host for clients.
fixes: 3264

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-10-08 10:48:27 -07:00
Sam Lang
fa66eaa162 mds: Reset session stale timer at end of open
The mds gets an open session request and immediately sets the session
stale timer.  If logging the session and sending the reply takes longer
than the timer, the client sees a stale session message before getting
the open session reply.  This fix resets the session stale timer on
completion of the mds logging operation to reduce the likelihood of
the session getting marked stale before the client can receive the
open session reply and start sending renewcaps messages.

We reset the session timer for renewcaps and force_open_sessions right
before returning the SESSION_OPEN reply to the client.  This avoids stale
sessions in the cases where the mds is highly loaded and the session_open
reply back to the client is delayed.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-10-08 10:58:07 -05:00
Josh Durgin
c8721b956c Merge branch 'wip-osd-caps'
Conflicts:
	src/osd/OSDCap.cc
	src/test/osd/osdcap.cc

Reviewed-by: Sage Weil <sage.weil@inktank.com>
2012-10-05 16:21:12 -07:00
Tommi Virtanen
48fc340ce8 upstart: OSD journal can be a symlink; if it's dangling, don't start.
This lets a $osd_data/journal symlink point to
/dev/disk/by-partuuid/UUID and the osd will not attempt to start until
that disk is available.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Sage Weil
8d718caf00 osd: Make --get-journal-fsid not really start the osd.
This way, it won't need -i ID and it won't access the osd_data_dir.
That makes it useful for locating the right osd to use with an
external journal partition.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
435a5c1435 osd: Make --get-journal-fsid not attempt aio or direct_io.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
88fc8b7885 ceph-disk-prepare: Use the OSD uuid as the partition GUID.
This will make locating the right data partition for a given journal
partition a lot easier.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
d6b065ce91 debian/control, ceph-disk-prepare: Depend on xfsprogs, use xfs by default.
Ext4 as a default is a bad choice, as we don't perform enough QA with
it. To use XFS as the default for ceph-disk-prepare, we need to depend
on xfsprogs.

btrfs-tools is already recommended, so no change there. If you set
osd_fs_type=btrfs, and don't have the package installed, you'll just
get an error message.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
2e9afc21d0 ceph-disk-{prepare,activate}: Default mkfs arguments and mount options.
The values for the settings were copied from teuthology task "ceph".

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
b40b2d6cc9 ceph-disk-prepare: Avoid triggering activate before prepare is done.
Earlier testing never saw this, but now a mount of a disk triggers a
udev blockdev-added event, causing ceph-disk-activate to run even
before ceph-disk-prepare has had a chance to write the files and
unmount the disk.

Avoid this by using a temporary partition type uuid ("ceph 2 be"), and
only setting it to the permanent ("ceph osd"). The hotplug event won't
match the type uuid, and thus won't trigger ceph-disk-activate.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
79320ff48d ceph-disk-activate: Add a comment about user_xattr being default now.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
ab84949da2 ceph-disk-activate: Use mount options from ceph.conf
Always uses default cluster name ("ceph") for now, see
http://tracker.newdream.net/issues/3253

Closes: #2548
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
8eeebe865c ceph-disk-activate: Refactor to extract detect_fstype call.
This allows us to use the fstype for a config lookup.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
d94e42179e ceph-disk-activate: Unmount on errors (if it did the mount).
This cleans up the error handling to not leave disks mounted
in /var/lib/ceph/tmp/mnt.* when something fails, e.g. when
the ceph command line tool can't talk to mons.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:35 -07:00
Tommi Virtanen
a146f053e5 ceph-disk-prepare: Allow setting mkfs arguments and mount options in ceph.conf
Tested with meaningless but easy-to-verify values:

  [global]
  osd_fs_type = xfs
  osd_fs_mkfs_arguments_xfs = -i size=512
  osd_fs_mount_options_xfs = noikeep

ceph-disk-activate does not respect the mount options yet.

Closes: #2549
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:34 -07:00
Tommi Virtanen
0b934e19d9 ceph-disk-prepare: Allow specifying fs type to use.
Either use ceph.conf variable osd_fs_type or command line option
--fs-type=

Default is still ext4, as currently nothing guarantees xfsprogs
or btrfs-tools are installed.

Currently both btrfs and xfs seems to trigger a disk hotplug event at
mount time, thus triggering a useless and unwanted ceph-disk-activate
run. This will be worked around in a later commit.

Currently mkfs and mount options cannot be configured.

Bug: #2549
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 15:41:34 -07:00
Josh Durgin
18d1580d1c doc: clarify osd journal as a block device
Someone on IRC was confused by this and thought it had to be a file.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-05 14:48:25 -07:00
Dan Mick
075c321a7a rbd: ls -l format tweaks:
Add PROT and LOCK columns, for protection status and presence of any
locks of type "excl" or "shr" (lock list for the gory details)

Shrink FORMAT to FMT

Remove TYPE column; one can infer type from presence of @ in name (snap)
or presence of parent (clone)

Dump prettybyte_t in favor of new si_t for compactness

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-05 14:57:08 -07:00
Dan Mick
8a484be5f5 rbd: Make lock list adapt 'number' of msg (1 lock, 2 locks)
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-05 14:56:56 -07:00
Dan Mick
19147c00a1 rbd: set_conf_param() rewhack:
1) comment set_conf_param and the loop that uses it
2) put back error checking for "called with full param list" in macro
3) make all the loop calls consistent
4) add a third arg placeholder to handle lock remove

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-05 14:56:41 -07:00
Dan Mick
88ea027160 Add si_t type for large-number "just the suffix, no space"
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-05 14:56:08 -07:00
Sam Lang
082746d6d0 client: Fix assert when stale arrives before open
The client sends an open session request to the mds, which may not
get a chance to reply before the mds timer times out the session and
sends the client a stale message.  This fix avoids the assertion by
checking that a pending open session request is in progress.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-10-05 16:12:32 -05:00
Tommi Virtanen
0405a443b5 ceph-disk-activate, upstart: Use "initctl emit" to start OSDs.
This avoids an error if the daemon was running already, and is
already being done with the other services.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 11:10:22 -07:00
Tommi Virtanen
1f3a50a521 ceph-create-keys: Refactor to share wait_for_quorum call.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 11:09:06 -07:00
Tommi Virtanen
4302f12a60 ceph-create-keys: Create a bootstrap-osd key too.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-05 11:09:06 -07:00
Sam Lang
a53a4a21a0 client: Don't keep sending SESSION_REQUEST_OPEN
This fixes a bug where the client keeps resending session open requests
to the target mds because the waiting_for_session.count check was
incorrectly using the passed in mds number instead of the target.

Also output the mds the message is coming from for debugging.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-10-05 10:54:54 -05:00
Dan Mick
2d6953a44c Revert "rbd: gracefully handle extra arguments"
Insufficient understanding of fragile algorithm.  This needs more
thought and I don't want the parsing broken as it is now.

This reverts commit 0d48879320.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2012-10-04 21:14:18 -07:00
Josh Durgin
20496b8d2b OSD: separate class caps from normal read/write
This properly accounts for multi-op requests. Use MOSDOp->rmw_flags for
internal caps requirements, leaving MOSDOp->flags for client specified
options. Use accessors so the flags don't need to be known by the callers.

Also separate capability checks (need_*_cap) from the nature of the MOSDOp
(may_{read,write}). This preserves the semantics of may_{read,write},
which are used in several places outside of capability checks.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 18:36:07 -07:00
Dan Mick
0d48879320 rbd: gracefully handle extra arguments
Instead of looping across all args, with increments inside the loop,
which can run off the end of the vector, demand that the final
argument parsing have exactly the right number of args, or complain
about the extras and die.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 18:08:28 -07:00
Sage Weil
aed3612f87 MOSDBoot: fix compatibility with ~argonaut
I revved this message and forgot to set the compat version correctly,
preventing post-change (e.g., bobtail) OSDs from talking to pre-change
(e.g., argonaut) monitors.  This was in b64641c.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-04 17:01:33 -07:00
Josh Durgin
f1ad419282 qa: report success if race is not detected
This test still verifies that the race is handled correctly if it
occurs, but will no longer clutter test results with spurious failures
when the race is not reproduced.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 16:50:46 -07:00
Dan Mick
37ffb278c8 qa scripts: Replace cut -f 1 with awk '{print $1;}'
This is to handle TextTable output, which doesn't use tabs

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 17:04:09 -07:00
Dan Mick
bed55369a9 rbd: Use TextTable for showmapped and lock list
Also update ocf/rbd.in to handle new output form (spaces)

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2012-10-04 16:20:13 -07:00
Sage Weil
c109329244 librbd: simplify math
Bending over backwards hasn't made coverity happy.  We'll just ignore it
there.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c517fde269)
2012-10-04 15:45:50 -07:00
Dan Mick
61b2e4d6ac rbd: use TextTable to implement ls -l
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 15:20:29 -07:00
Dan Mick
fb88683cd1 Add TextTable class for table output
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 15:20:29 -07:00
Dan Mick
9889d819d7 librbd: fix error handling in get_parent_info
Return ENOENT if no parent.
Return error if pool reverse lookup fails.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-04 15:20:29 -07:00
Tommi Virtanen
3ba7ab2ba5 doc: Whitespace cleanup.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-04 14:54:17 -07:00
Tommi Virtanen
790b508903 doc: rst syntax fixes.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-04 14:54:17 -07:00
Tommi Virtanen
36f1698486 rbd.py: Fix docstring syntax of list_lockers().
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-04 14:54:17 -07:00
Tommi Virtanen
2411ba696d rbd.py: Docstring syntax.
It's meth for referring to Python class methods:
http://sphinx.pocoo.org/domains.html#python-roles

The links to format() and features() are currently
dead because those methods don't have docstrings.
They'll start working once docstrings are added.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-04 14:54:17 -07:00