Commit Graph

73171 Commits

Author SHA1 Message Date
Yehuda Sadeh
7a34a049ae rgw: split per-bucket resharding logic out from general resharding
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:43 -07:00
Yehuda Sadeh
0bc15671ee rgw: revert some reshard blocking logic
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:42 -07:00
Yehuda Sadeh
7d44601e83 cls/rgw: create an objclass method to guard in-progress resharding
This method will be called on the bucket index shard.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:42 -07:00
Orit Wasserman
4ddb785bb0 rgw: RGWReshard update to new cls API
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:41 -07:00
Yehuda Sadeh
a07e8eb6af cls/rgw: adjust types and ops
- don't keep data bufferlist in struct cls_rgw_bucket_instance_entry
 - remove rgw_get_bucket_instance

We need to keep the reshard info as part of the bucket index header (for
each source bucket shard), and not in a separate object. Otherwise it'll
be impossible to have the resharding check atomic as part of the regular
bucket index updates.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:41 -07:00
Yehuda Sadeh
04cf0ca074 cls/rgw: minor changes
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:41 -07:00
Yehuda Sadeh
44e46b35f0 cls/rgw: generate_reshard_key, change output param
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:40 -07:00
Yehuda Sadeh
f446b80e2a cls/rgw: return default header if not initialized
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-06-05 13:17:40 -07:00
Orit Wasserman
8282366704 rgw: cls_reshard_get remove unneeded check for bufferlist length
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:40 -07:00
Orit Wasserman
1ee15336da rgw: get_bucket_header will return -ENOENT when there is no header
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:39 -07:00
Orit Wasserman
f384708196 rgw: add guard class for bucket index lock
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:39 -07:00
Orit Wasserman
830ea53831 rgw: add reshard_pool_ctx
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:38 -07:00
Orit Wasserman
be156d086d rgw: check for exisiting entries in cls_rgw_reshard_add
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:38 -07:00
Orit Wasserman
0af96c178a rgw: use pointer for is_truncated
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:38 -07:00
Orit Wasserman
ea334d3fdf rgw: cls_rgw_reshard use helper function the generate key
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:37 -07:00
Orit Wasserman
ae6a5eac37 rgw: limit op.max and add extra entry to handle tuncation in cls_rgw_reshard_list
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:37 -07:00
Orit Wasserman
59c0c30ac0 rgw: use .rgw.log:reshard pool for resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:37 -07:00
Orit Wasserman
a7154a6a73 rgw: remove RGWReshard::get_head
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:36 -07:00
Orit Wasserman
5828d4a670 rgw: replace dout with ldout
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:36 -07:00
Orit Wasserman
822a5fbd47 rgw: remove cls_reshard_get_head
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:36 -07:00
Orit Wasserman
149bfc4337 rgw: store resharding information in bucket header
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:35 -07:00
Orit Wasserman
9e43e7f0bb rgw: handle correctly when there is no bucket or it doesn't have resharding flag
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:35 -07:00
Orit Wasserman
3b41ca025b rgw: block on bucket index operation when resharding the index
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:35 -07:00
Orit Wasserman
18c16f971e rgw: add RGWReshard object to RGWRados
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:34 -07:00
Orit Wasserman
c000a15fed rgw: Add lock/unlock bucket instance and block_while_resharding methods
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:34 -07:00
Orit Wasserman
2d23898b27 rgw: add cls_rgw_get_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:34 -07:00
Orit Wasserman
11e7cb1eb4 radosgw-admin: implement reshard execute command
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:33 -07:00
Orit Wasserman
3858171490 radsogw-admin add create_new_bucket_instance
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:32 -07:00
Orit Wasserman
8e30c37865 rgw: use const string for oid in get_bucket_instance_from_oid
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:32 -07:00
Orit Wasserman
8b497b06d7 rgw: add RGWReshard::clear_bucket_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:31 -07:00
Orit Wasserman
0f53c5dce5 rgw: add RGWReshard::set_bucket_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:31 -07:00
Orit Wasserman
98d85722a3 rgw: add missing handles for set_resharding, clear_resharding
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-05 13:17:14 -07:00
Orit Wasserman
7c5ec1d12b rgw: add cls _bucket_resharding_set and cls_bucket_resharding_clear
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:46:44 -07:00
Orit Wasserman
424dd65298 rgw: Add resharding flag and the new bucket instance id to RGWBucketInfo
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:46:43 -07:00
Orit Wasserman
9449095177 radosgw-admin: Add reshard add, list, execute and cancel commands
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:44:27 -07:00
Orit Wasserman
0806a00514 radosgw-admin: move resharding into a separate function
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:44:27 -07:00
Orit Wasserman
a34f6af726 rgw: add RGWReshard class
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:44:26 -07:00
Orit Wasserman
fd9db9e01e rgw: add reshard_pool
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:44:26 -07:00
Orit Wasserman
450ea1c4cf rgw: add cls_rgw_reshard ops
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2017-06-02 14:44:25 -07:00
Casey Bodley
bf72bd8dcb Merge pull request #15119 from ceph/wip-rgw-config-docs
doc: mention certain conf vars should be in global

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-02 14:09:33 -04:00
Yehuda Sadeh
ea911b7f48 Merge pull request #14351 from yehudasa/wip-rgw-mdsearch
rgw: metadata search part 2

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-06-02 09:16:07 -07:00
Sage Weil
ffced8cfa4 Merge pull request #15427 from tchaikov/wip-vstart-osd-id
vstart: print "start osd.$id" instead of "start osd$id"

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-02 10:34:45 -05:00
Sage Weil
b98ad7cadb Merge pull request #15217 from badone/wip-async-sleep-timer-fix
osd: Move scrub sleep timer to osdservice

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-06-02 10:32:15 -05:00
Sage Weil
6e406bab6a Merge pull request #15396 from jcsp/wip-ceph-status
mon: revise "ceph status" output

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-02 10:30:55 -05:00
Sage Weil
b735c5a939 Merge pull request #15011 from xiexingguo/wip-crush-tool
tools/crushtool: replicated-rule API support

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-02 10:20:44 -05:00
Sage Weil
cd4acd8d01 Merge pull request #15176 from SUSE/wip-mgr-fix-help
mgr,osd:  ceph-mgr --help, unify usage text of other daemons

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-02 10:20:27 -05:00
Sage Weil
6bedb4c35a Merge pull request #15395 from liewegas/wip-log-entity
common/LogEntry: include EntityName in log entries

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-02 10:19:58 -05:00
Sage Weil
ecef6fd839 Merge pull request #15398 from liewegas/wip-bluestore-leak
os/bluestore: fix buffers pinned by indefinitely deferred writes

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-06-02 10:19:26 -05:00
Sage Weil
773a0ac2d3 Merge pull request #15402 from liewegas/wip-bluestore-mempools
os/bluestore: improve mempool usage

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-06-02 10:19:03 -05:00
Sage Weil
a35cd1cb35 Merge pull request #15403 from liewegas/wip-mempool-dump
mempool: improve dump; fix buffer accounting bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-02 10:18:38 -05:00