Commit Graph

21321 Commits

Author SHA1 Message Date
Yehuda Sadeh
0f82a3931a rgw: fix suggested object size
We can't just stat the head object, we need to go through
get_obj_state() to get obj attributes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:57:05 -07:00
Yehuda Sadeh
6a054ea9d8 rgw: set exists flag when suggesting existing bucket entry
We weren't setting the 'exists' flag on the bucket entry,
so we ended up not updating the index correctly.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:57:05 -07:00
Yehuda Sadeh
b12c1477a3 cls_rgw: change scoping of suggested changes vars
Fixes: #3127
Bad variable scoping made it so that specific variables
weren't initialized between suggested changes iterations.
This specifically affected a case where in a specific
change we had an updated followed by a remove, and the
remove was on a non-existent key (e.g., was already
removed earlier). We ended up re-substracting the
object stats, as the entry wasn't reset between
the iterations (and we didn't read it because the
key didn't exist).

backport:argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:57:05 -07:00
Yehuda Sadeh
c11e05f532 test/rgw: add a test for index suggest
test the bucket index suggest api

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:57:02 -07:00
John Wilkins
5b7e60d0d8 Merge branch 'master' of https://github.com/ceph/ceph 2012-09-18 12:55:35 -07:00
Yehuda Sadeh
12175386d9 cls_rgw: configurable tag timeout
we'll be using that for bucket index unitest

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:55:33 -07:00
Yehuda Sadeh
64b9a898b6 test/rgw: add bucket index unitest
Tests objects creation, removal, prepare, complete

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:50:07 -07:00
Gary Lowell
519ed2471c cpeh.spec.in: Don't package boto_tool. 2012-09-18 12:43:36 -07:00
Yehuda Sadeh
6ff23bb9e3 cls_rgw: client api for bucket index init
create a client api function for bucket index init.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:27:49 -07:00
Yehuda Sadeh
b03adda4d9 cls_rgw: add client api calls for index suggest
This is needed for unitest creation.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 12:27:43 -07:00
Sage Weil
1bf345ecd0 doc/config-cluster/*: update recently added/changed config options
Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-18 12:20:37 -07:00
John Wilkins
91f3893f0b :doc: Title change for clarity. Initial formatting cleanup.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:41:32 -07:00
John Wilkins
93323134c7 Merge branch 'master' of https://github.com/ceph/ceph 2012-09-18 11:23:55 -07:00
John Wilkins
88dd90e15e :doc: Removed old rec section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:23:22 -07:00
Yehuda Sadeh
2f73c07511 docs: add radosgw internals and usage testing section
Document the current tests that are being done on the
radosgw usage feature.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-18 11:17:22 -07:00
John Wilkins
3369ea3a1f :doc: Notes to add data directory. Notes to address key with escape char.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:15:37 -07:00
John Wilkins
0398b9f709 :doc: Added note to clarify use of host=shortname.
Fixes: #3155


Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:14:54 -07:00
John Wilkins
3d86194f9e :doc: Rewrote architecture paper. Still needs some work.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:08:23 -07:00
John Wilkins
fafb195169 :doc: Removing old control.rst. New one is in ceph/docs/cluster-ops.rst
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:05:30 -07:00
John Wilkins
f60ea3836b :doc: Removing old init section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-18 11:02:41 -07:00
Josh Durgin
4f3642c5f4 rbd: only open the destination pool for import
Otherwise importing into another pool when the default pool, rbd,
doesn't exist results in an error trying to open the rbd pool.

Reported-by: Sébastien Han <han.sebastien@gmail.com>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-09-18 10:10:50 -07:00
Gary Lowell
46883a5a0f ceph.spec.in: Add new rados libraries, libcls_kvs.so and libcls_refcount.so. 2012-09-17 22:21:15 -07:00
Sage Weil
cb49823569 osd: default journal size to 1 GB
The default is used to create or extend the journal when it is a file.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-17 18:23:39 -07:00
Sage Weil
6b0c9ffc8b filejournal: do not enforce that bdev size >= osd journal size
If the configure osd journal size is > the block device size, warn, but
do not generate an error and abort startup.  This makes it safe to have
a default 'osd journal size' value of, say, 1 GB without fear of breaking
existing clusters with smaller jouranl block devices.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Tommi Virtanen <tv@inktank.com>
2012-09-17 18:23:39 -07:00
Sage Weil
de930440e7 remove obsync
This now lives at http://github.com/ceph/obsync

Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-17 18:23:39 -07:00
John Wilkins
f1819f05dc :doc: Changed from bullet list to section titles.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-17 16:25:54 -07:00
John Wilkins
18cf1b6d61 :doc: Usage consistent with Tyler's notes.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-17 16:01:36 -07:00
John Wilkins
da1a9f914f :doc: Corrected a path.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-17 16:00:58 -07:00
John Wilkins
00e58a509d :doc: Addresses DOCUMENTATION #3155
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-09-17 15:59:45 -07:00
Josh Durgin
f46a5f1c79 Merge branch 'wip-rbd-import' 2012-09-17 08:27:03 -07:00
Yehuda Sadeh
4251d783db rgw: update cache interface
RGWRados::delete_obj() was updated in commit
93218aeab7, but we
failed to update the corresponding RGWCache api.
This commit fixes it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 15:21:56 -07:00
Yehuda Sadeh
ff2ce1cbd8 Merge branch 'wip-2923'
Conflicts:
	src/rgw/rgw_admin.cc
	src/rgw/rgw_log.cc
	src/rgw/rgw_log.h
	src/rgw/rgw_main.cc
	src/rgw/rgw_op.cc
	src/rgw/rgw_rados.cc
	src/rgw/rgw_rados.h

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 15:17:08 -07:00
Yehuda Sadeh
6756f9afef rgw-admin: add cluster info command
A new radosgw-admin command to dump cluster info

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:33:47 -07:00
Yehuda Sadeh
7fe1f0b11e rgw: simplify create_pool
create_pool should only create pool. A pool is not a bucket,
so we don't need to attach any attrs to it. Also, no reason
to make it exclusive.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:33:36 -07:00
Yehuda Sadeh
ff0f88a6ce rgw: store cluster params in a special object
We now have a cluster root pool that should hold the
cluster params. The cluster params are now read from
this object on startup, if object does not exist we
set its defaults and write it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:30:05 -07:00
Yehuda Sadeh
a16fa0492e Merge branch 'wip-rgw-refcount' 2012-09-15 13:02:54 -07:00
Yehuda Sadeh
a150cf3c1e rgw: document how refcount objclass works
Added some explanations in the client header.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:01:48 -07:00
Yehuda Sadeh
539a8c7e04 cls_refcount: add a unitest
test cls_refcount functionality.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:01:31 -07:00
Yehuda Sadeh
93218aeab7 rgw_rados: adjustments for refcount objclass
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 13:01:25 -07:00
Yehuda Sadeh
cfe4830516 rgw: set atomic context for copy operation src and dest
This is required so that we handle both src and dest atomically. We
also set the prefetch flag on the src object, so that we read the
first chunk along with its attrs.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 12:56:02 -07:00
Yehuda Sadeh
3cd17fb9dd rgw: copy of objects using refcount
For objects with manifest that have a tail, we
copy only the head, and the manifest, and increasing
the reference count on the tail objects.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 12:52:26 -07:00
Yehuda Sadeh
20c785278c rgw: use refcount put instead of obj delete where needed
Tie in new refcount objclass methods.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-09-15 12:51:35 -07:00
Josh Durgin
78d6a60c59 qa: test args for rbd import
Make sure that --pool/--dest-pool and --image/--dest all work
interchangeably.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-09-14 17:17:29 -07:00
Josh Durgin
fd4b294280 rbd: make --pool/--image args easier to understand for import
There's no need to set the default pool in set_pool_image_name - this
is done later, in a way that doesn't ignore --pool if --dest-pool
is not specified.

This means --pool and --image can be used with import, just like
the rest of the commands. Without this change, --dest and --dest-pool
had to be used, and --pool would be silently ignored for rbd import.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-09-14 17:17:29 -07:00
Josh Durgin
0bf7723733 doc: fix indentation
Before this the list of steps was split into two lists
after the conf example.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-09-13 16:58:13 -07:00
tamil
cf6899d859 Adding RBD CLI test automation script, RbdLib and cephscrub.sh
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
2012-09-13 13:54:23 -07:00
Sage Weil
539786a85a osd: move permissions check into pg op thread
The permission check examines the PG::pool struct.  Instead of adding
additional locking there, just push the check into the op thread.  This
makes life a bit simpler for the dispatch thread, which is particularly
hot.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-13 08:02:17 -07:00
Sage Weil
221fc788e7 osd: drop unnecessary can_discard_request() in PG::queue_op()
We perform the same check in PG::do_request(), and it is no longer safe to
do this at enqueue_op() time because we aren't holding PG::_lock (only
PG::_qlock).

Signed-off-by: Sage Weil <sage@inktank.com>
2012-09-13 08:02:17 -07:00
Andreas Bluemle
2223ac78e5 osd: introduce PG::_qlock to protect queuing
Taking the PG::_lock when queuing each op for the worker threads can intorduce
long delays that hold up subsequent operations on other PGs.  Use a separate
lock to protect the queuing.

Signed-off-by: Andreas Bluemle <andreas.bluemle@itxperts.de>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-09-13 08:02:17 -07:00
Samuel Just
d697b54c9b test/rados-api/aio.cc: use read operation for omap cmp test
The error code field is not filled in on writes.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-09-12 16:37:11 -07:00