Commit Graph

25967 Commits

Author SHA1 Message Date
Yehuda Sadeh
8f1da8f3d4 rgw: fix regionmap json decoding
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-26 22:39:07 -07:00
Yehuda Sadeh
af00f73348 rgw: automatic pool creation for placement pools
With the new pools configuration, now we auto create the
pools when needed (through bucket creation). Also, make
sure only to configure default placement in zone structure,
if old config hasn't been done yet.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 22:55:53 -07:00
Yehuda Sadeh
7a2566c60f rgw: remove test placement info
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 20:06:45 -07:00
Yehuda Sadeh
224130c9f7 rgw (test): remove some warnings
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 19:18:51 -07:00
Yehuda Sadeh
1b162ce662 rgw: initialize user system flag
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 17:59:37 -07:00
Yehuda Sadeh
7681c58e03 rgw: log in the same shard for bucket entry point and instance
We'd like to have bucket entry point and instance info at the same
log shard, so that we can process them in order.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 15:30:44 -07:00
Yehuda Sadeh
d4e39a7676 rgw: unlink/link don't always update entry point
Some operations already update the entry point, so no
need to do it again.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 14:59:49 -07:00
Yehuda Sadeh
6673b2d3aa rgw: tie metadata put to bucket link/unlink
and lot's of constifying

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 14:00:59 -07:00
Yehuda Sadeh
5c3df085c6 cls_rgw: cleanup
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 14:00:45 -07:00
Yehuda Sadeh
82db84bec5 rgw: some more internal api cleanups
Use of rgw_bucket when referring to the bucket instance,
use bucket name when referring to the bucket entry point.
Also, remove bucket input param where not needed (internally
was using the bucket structure from the bucket info).

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 12:56:25 -07:00
Yehuda Sadeh
c4be5a7057 rgw: unlink bucket from user on metadata rm bucket:< bucket>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 12:19:17 -07:00
Yehuda Sadeh
86c73c94ff rgw: fixes to object versioning tracking
There are a few different cases for setting the object version.
Either we need to create a new version, or we need to set the
version provided (one metadata put). We also need to make sure
that we log the correct previous version of the object. This
commit fixes a few cases.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 11:09:19 -07:00
Yehuda Sadeh
8bd31d42a2 rgw: filter read xattrs
We're only interested in object xattrs that have specific rgw.user
prefix.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 11:05:15 -07:00
Yehuda Sadeh
422bb6d0ac rgw: add str_startswith()
useful util

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 11:03:12 -07:00
Yehuda Sadeh
8db289f2e2 cls_ver: rename version xattr, add some more logging
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-25 10:36:00 -07:00
Yehuda Sadeh
63e81afeb8 rgw: multiple fixes related to metadata, bucket creation
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-24 23:43:50 -07:00
Yehuda Sadeh
7e41c1036d rgw: bucket entry point contains owner, other fix
We now keep the owner at the bucket entry point. Also, reading
bucket info at RGWCreateBucket::execute(), because previous changes
has made it so that we didn't have the required bucket info
initialized there.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-24 17:27:34 -07:00
Yehuda Sadeh
1e3161a84e rgw: add a system request param to select op bucket instance
Added rgwx-bucket-instance request param that selects the actual bucket
instance to use for this request.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 23:18:44 -07:00
Yehuda Sadeh
76228caf65 rgw: data log contains bucket instance info in key
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 22:34:24 -07:00
Yehuda Sadeh
5dd137ace4 rgw: format bucket.instance meta entries nicely
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 21:44:05 -07:00
Yehuda Sadeh
4e90c5ec29 rgw: use new instance handler for bucket operations
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 21:16:14 -07:00
Yehuda Sadeh
71869c4b9e rgw: create meta handler for bucket instance
Create utility functions for reading writing bucket entry
point and bucket instance. Add a separate meta handler for
bucket instance.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 21:00:00 -07:00
Yehuda Sadeh
00973dfdca rgw: put_bucket_info() uses objv_tracker in bucket info
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 19:42:17 -07:00
Yehuda Sadeh
dab57ef883 rgw: keep objv_tracker on bucket info
This ties it to the specific object instance info.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 18:52:39 -07:00
Yehuda Sadeh
e053d1718d rgw: refactor policy reading
reuse bucket info, no need to reread it again.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 15:38:09 -07:00
Yehuda Sadeh
d74cdad526 rgw: fix policy read
policy reading used to have exact same logic for both buckets
and objects, however, we now need to read bucket attributes
through get_bucket_info().

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 13:39:13 -07:00
Yehuda Sadeh
ad6406726f rgw: init member variable
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 13:12:59 -07:00
Yehuda Sadeh
b09d799f2c rgw: encode bucket info only after setting a flag
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 13:12:26 -07:00
Yehuda Sadeh
c3260b2768 rgw: initial work to separate bucket name and instance
Bucket info now resides in an instance specific object. The
old bucket info object now contains a pointer to the new
instance.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 01:15:56 -07:00
Yehuda Sadeh
c103031ec6 rgw: initialize system flag in RGWUserInfo
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 01:15:10 -07:00
Yehuda Sadeh
7e3493f872 rgw: various object put processor fixes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-23 01:04:24 -07:00
Yehuda Sadeh
5f43fa8140 rgw: fix broken obj put
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-22 23:26:55 -07:00
Yehuda Sadeh
d7af5e144c rgw: handle bucket creation with specified placement pool
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-21 21:59:53 -07:00
Yehuda Sadeh
2fcbf2bae7 rgw: user configuration for bucket placement
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-21 16:16:47 -07:00
Yehuda Sadeh
e5e924ca71 rgw: data structures for new data/index placement rules
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-21 16:06:51 -07:00
Yehuda Sadeh
241ad07394 rgw: make replica log object name configurable
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-21 11:53:49 -07:00
Greg Farnum
469900e62b RGWReplicaBucketLogger: store bucket replica logs in the .logs pool
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-21 11:47:09 -07:00
Yehuda Sadeh
409422be6d Merge branch 'wip-rgw-geo-2' into wip-rgw-geo
Conflicts:
	src/test/cli/radosgw-admin/help.t
2013-06-21 10:53:48 -07:00
Yehuda Sadeh
24e59b4377 rgw: buffer atomic put handler
Since we tied the atomic put handler to libcurl output
data, which uses much smaller chunks, we need to buffer
data, otherwise we'd end up with a huge amount of small
writes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-20 21:57:40 -07:00
Yehuda Sadeh
02de43ad73 rgw: tie opstate into intra-region copy operations
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-06-20 21:57:36 -07:00
Yehuda Sadeh
234fab0902 Merge branch 'wip-rgw-geo-enovance' into wip-rgw-geo-2 2013-06-20 14:42:17 -07:00
Greg Farnum
6d6dc42ee0 fixup "radosgw-admin: new commands to look at and clean up replica logs" 2013-06-20 14:41:43 -07:00
Greg Farnum
59bfdd0064 radosgw-admin: new commands to look at and clean up replica logs
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:35 -07:00
Greg Farnum
ab79ba46a4 cls_replica_log: integrate with RGWRados
We introduce an implementation class RGWReplicaLogger, and two user
classes RGWReplicaObjectLogger (for the data/metadata logs) and
RGWReplicaBucketLogger (for the bucket logs).

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:35 -07:00
Greg Farnum
36514731a5 cls_replica_log: add tests.
We do several good updates, a bad update, a good delete,
a bad delete, several good gets, and a bad get.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:35 -07:00
Greg Farnum
29df88af75 cls_replica_log: add client with user documentation
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:35 -07:00
Greg Farnum
e4ef5c6f96 cls_replica_log: add the actual class
This class can be applied to any object, and will keep track of
the bounds, matching times, and matching sets of in-progress items.
We're about to add a cls_replica_log_client that will provide a friendly
user interface and provide more documentation.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:35 -07:00
Greg Farnum
22a02e95ae cls_replica_log: add ops for new class
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:31 -07:00
Greg Farnum
d1c95943d8 cls_replica_log: add types for new class
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-06-20 14:10:26 -07:00
Babu Shanmugam
0deb6d4093 rgw: lock related modifications
1. zone-id inclusion in lock/unlock for mdlog and datalog
2. renewal of lock if the locker request lock again
3. modified lock_id param to locker-id

Signed-off-by: Babu Shanmugam <anbu@enovance.com>

Conflicts:
	src/rgw/rgw_rest_log.cc
2013-06-20 13:38:47 -07:00