Commit Graph

21063 Commits

Author SHA1 Message Date
Yehuda Sadeh
cecd7afa4f Formatter: fix XML formatter
XML formatter wasn't clearing stream string buffer
when completing the section.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:17 -07:00
Yehuda Sadeh
02f7870548 rgw: add rest-usage handler
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:17 -07:00
Yehuda Sadeh
fca6d3b992 rgw: REST resources init uses configurables
Also, move it to main().

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:17 -07:00
Yehuda Sadeh
cea28f8cfb rgw: refactor protocol stacking
New RESTMgr* classes to handle a specific uri entry
point. Actions are handled by RGWHandler*.

Changing init order, newly instantiated RGWHandler
for every op.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 11:06:12 -07:00
Yehuda Sadeh
85f137d2a1 rgw: tie callbacks in different handlers directly to REST
Don't translate RESTful operations into a more meaningful
callback name. The handlers themselves should do that
translation. This way we can later register different
handlers with different meanings for the operations.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:48 -07:00
Yehuda Sadeh
b71b8b09e1 rgw: sed 's/_REST/_ObjStore/g' *
It's all REST, we want to add new non objstore handling,
so let's differentiate by the actual type.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:48 -07:00
Yehuda Sadeh
e153bceef2 rgw: start moving usage management functions
Move usage management functionality to its own place.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:48 -07:00
Yehuda Sadeh
a9c9f96ba5 rgw: hide fcgi details from client code
client code (rgw_rest* et al) does not need to see the
fcgi library details.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-08 10:57:43 -07:00
Yehuda Sadeh
66b1eb2cff rgw: better protocols separation
This change will make it easier to add new protocol
handlers. We now determine which protocol handler to
use by querying the list of existing protocol handlers
whether the query belongs to that protocol. If so,
then just run the specific protocol's handler code.
Separated some mixed s3 and swift code into the specific
handlers.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-26 09:50:21 -07:00
Yehuda Sadeh
35cd5f3c94 rgw: clean up ops initialization, completion
ops init() was a reminiscent of an old design. Since we're
allocating new op for every request we can just init everything
in the constructor.

We now add a new complete() stage in request processing. By
default complete() will call send_response().

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-26 09:42:35 -07:00
Yehuda Sadeh
63c0909399 Merge branch 'wip-swift-manifest'
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-10 12:42:55 -07:00
Yehuda Sadeh
f8c365e690 rgw: add missing ret code check
Also, for some reason we used cerr instead of dout so fix
that too.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-10 12:41:33 -07:00
Yehuda Sadeh
a4bc7953a7 Merge branch 'wip-3060'
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-10 11:28:09 -07:00
Greg Farnum
d77205d8f9 objecter: remove the now-unused "registering" flag on LingerOps
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-09-07 21:03:39 -07:00
Yehuda Sadeh
c4c4b20fea rgw: encode total usage for backwards compatibility
Adding the encoding of the total usage allows us to
make the format change backwards compatible.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-07 16:44:28 -07:00
Yehuda Sadeh
06290f6dff rgw: configurable swift auth entry point
Fixes: #1776
Swift auth entry point is not hard coded anymore.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-07 16:22:48 -07:00
Sage Weil
03136d057f osd: fill in user log entry last after snapdir tran
Reorder the snapdir logic and ctx->at_version adjustments prior to filling
in the object_info_t and user_versions and all that stuff.  Adjust
at_version after appending the log entry (so that it points to the next
position/version we will write at.. culminating in the actual user
event).

The user log entry contains the request id, which will be used
by replay ops to put themselves in the correct place in the
waiting_for_commit/ack maps.  Thus, the repop needs to be tagged
with the same version as the log entry with the request id.
Thus, the request id bearing log entry should be the last in
the log entry vector.

This should fix #3072, wherein a replay which should wait on
the repop tagged as version '36 will instead wait on '35.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-09-07 15:34:51 -07:00
John Wilkins
e2129ab472 :doc: Fixed broken hyperlinks.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-06 21:00:29 -07:00
John Wilkins
9bbe73cc1d :doc: Addresses Documentation #3096. Also added new information.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-06 20:31:46 -07:00
Yehuda Sadeh
793bebb82e rgw: rgw-admin filter usage by categories
rgw can now filter usage output by specific op
categories.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-06 16:46:26 -07:00
Tommi Virtanen
9d878bfb4d gitignore: Remove "nbproject", no idea what that even is.
Looks like this line was accidentally introduced in commit
af4d8db55f.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-09-06 16:12:35 -07:00
Yehuda Sadeh
ddaac56c84 rgw: usage now account categories
Instead of just keeping a flat usage info per bucket, we
now maintain a list of categories for which requests
usage is aggregated in. Ops are put in categories based
on their names.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-06 13:17:09 -07:00
Yehuda Sadeh
a5b9ba7773 rgw: fix usage
Fixes: #3085
usage was showing 'bucket info' command that never
existed.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-06 10:15:54 -07:00
John Wilkins
3fed44acd1 :doc: Minor tweak to heading text.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:33:45 -07:00
John Wilkins
53d982926b Merge branch 'master' of github.com:ceph/ceph 2012-09-05 17:26:39 -07:00
John Wilkins
6957ae78e1 :doc: Modified the index page to point to the new cluster-ops section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:25:34 -07:00
John Wilkins
4ff60c7a0e :doc: Made minor changes to restructuredText headers.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:24:54 -07:00
John Wilkins
8fa81ab54c :doc: Added comment redirecting editors to new page location.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:23:03 -07:00
John Wilkins
c22ca7230f :doc: Added index/toctree page for cluster ops.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:22:32 -07:00
John Wilkins
b8dfe8227d :doc: Added new auth settings to reference doc.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:22:02 -07:00
John Wilkins
199dc5d0dc :doc: Refactored and moved control.rst page.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:21:31 -07:00
John Wilkins
d90fea6cad :doc: Consolidated file system recommendations.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:21:04 -07:00
John Wilkins
ab020da912 :doc: Minor syntax update.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:20:30 -07:00
John Wilkins
64b9e9f505 :doc: restructuredText syntax corrections.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:19:51 -07:00
John Wilkins
c94034a29e :doc: Added index page. This will be refactored again soon.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:17:57 -07:00
John Wilkins
32398f347d :doc: Refactored and moved to ceph/docs/cluster-ops/pools.rst
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:17:20 -07:00
John Wilkins
c53a110406 :doc: Removed. New section is in ceph/doc/cluster-ops/authentication.rst
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:16:29 -07:00
John Wilkins
f1c869cbe3 :doc: Fixed heading syntax.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-05 17:15:27 -07:00
Samuel Just
78c844e578 PG: clear want_acting in choose_acting if want == acting
Otherwise, a pg_temp from a previous peering sequence
(but not a different peering_interval) might leak through
into Active and incorrectly trip the
Active::react(AdvMap&) asserts regarding want_acting.
Those asserts assume that want_acting is either empty or is
a results of recovery completion.  In the latter case, the
want_acting set much consist only of elements of up and
acting.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-09-05 16:38:47 -07:00
Sage Weil
d31140cec9 Merge remote-tracking branch 'gh/wip_deep_scrub_final' 2012-09-05 14:23:49 -07:00
Sage Weil
df89c96323 osd: initialize pg_log_entry_t::invalid_pool in default ctor
Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-05 14:19:19 -07:00
Mike Ryan
3fd5914cf3 doc: explain what scrub, deep-scrub, and repair actually do
Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 14:19:19 -07:00
Mike Ryan
9013efd3a3 osd: deep scrub, read file contents from disk and compare digest
Deep scrub reads the contents of every file from the store and computes
a crc32 digest. The primary compares the digest of all replicas and will
mark the PG inconsistent if any don't match.

OSDs that do not support deep scrub simply perform an ordinary chunky
scrub. Any subset of OSDs that do support deep scrub will have their
digests compared.

Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 14:19:19 -07:00
Mike Ryan
a0b04bcecf buffer: class for efficiently calculating CRC32 of >= 1 bufferlist
Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 14:19:19 -07:00
Mike Ryan
a08a28d019 pg: store scrubber state in its own object
Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 14:19:18 -07:00
Mike Ryan
f5046798f4 osd: chunky scrub, scrub PGs a chunk of objects at a time
Chunky scrub is a more efficient scrub. It blocks writes on a subset of
objects and scrubs those, allowing writes through to the rest of the PG.

The scrub takes longer to complete than a classic scrub, but improves
overall write throughput.

This feature is backward-compatible with classic scrub. If the primary
detects that any replica does not have the chunky scrub feature, it
falls back to the less efficient classic scrub.

Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 14:19:13 -07:00
Sage Weil
79b3054347 crush: change default type from 'pool' to 'root'
The 'pool=default' in the default crush maps is confusing wrt rados pools.
'root' makes more sense given that we are talking about hierarchies/trees.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-05 13:20:48 -07:00
Samuel Just
a0c87b8675 Merge remote-tracking branch 'upstream/wip-kvstore' 2012-09-05 10:50:50 -07:00
Mike Ryan
a83fb799ea FileStore: get objects whose names fall within a range
Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 10:13:33 -07:00
Mike Ryan
8bad708e75 pg: change _scrub() to take out parameters as pointers
Signed-off-by: Mike Ryan <mike.ryan@inktank.com>
2012-09-05 10:13:33 -07:00