RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-27 22:13:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
73,213 Commits 211 Branches 394 Tags 1.4 GiB
07b21866a1
Commit Graph

73213 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yehuda Sadeh
07b21866a1 rgw: update bucket instance with resharding details 
can be useful info for later in a cross zone sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:59 -07:00
Yehuda Sadeh
40faa8d952 radosgw-admin: reshard execute output adjustments 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:59 -07:00
Yehuda Sadeh
aa3f71d6b1 cls/rgw: remove unused field in reshard entry 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:58 -07:00
Yehuda Sadeh
1da369e1e8 rgw: reshard: renew lease and handle marker when listing reshard repo 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:58 -07:00
Yehuda Sadeh
7054308e85 cls/rgw: fix reshard_list truncation identification 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:58 -07:00
Yehuda Sadeh
6cb30f9c95 cls/rgw: expose api to generate reshard indexing key 
So that we could use it as a marker when doing reshard_list.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:57 -07:00
Yehuda Sadeh
81693efd78 rgw: remove unneeded and duplicate code 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:57 -07:00
Yehuda Sadeh
6c540bd5fe rgw: initialize bucket structure 
after reading bucket info

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:56 -07:00
Yehuda Sadeh
485a11fe59 rgw-admin: revive reshard execute command 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:56 -07:00
Yehuda Sadeh
ed72675ba2 rgw: bucket_id does not include bucket name 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:55 -07:00
Yehuda Sadeh
b516c92af5 rgw: bi_list() initialize is_truncated if -ENOENT 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:54 -07:00
Yehuda Sadeh
999cca4ed3 rgw: internal api cleanup 
rename method, minor change

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:54 -07:00
Yehuda Sadeh
436f151e55 cls/rgw: reshard add command doesn't read existing status 
No need to read existing status.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:54 -07:00
Yehuda Sadeh
2da8dc5d10 cls/rgw: fix remove_reshard objclass op 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:53 -07:00
Yehuda Sadeh
f677d0e960 cls/rgw: modify logshard key 
The logshard key shouldn't include bucket id.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:53 -07:00
Yehuda Sadeh
8e1bf1b9e8 rgw: multiple fixes and adjustments related to resharding scheduler 
still wip, but:
 - get rid of some unneeded rados locking
 - rename shards to logshards where relevant to avoid confusion
 - remove unused code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:53 -07:00
Yehuda Sadeh
b757e7e16e rgw: fix compilation following rebase 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:52 -07:00
Yehuda Sadeh
cac047a88b rgw: replace reshard blocking sleep with interruptible condition wait 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:52 -07:00
Orit Wasserman
7114124673 rgw: actually reshard an entry from the resharding queue 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:52 -07:00
Orit Wasserman
c7fc87f9f1 rgw: check if buckets needs resharding on all ops that add new objects 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:51 -07:00
Orit Wasserman
444b9c918c rgw: add bucket to resharding queue if needs resharding 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:51 -07:00
Orit Wasserman
e03b85e987 rgw: check for bucket resharding when adding an object 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:51 -07:00
Orit Wasserman
9a084497fd rgw: add a function to add a bucket to the resharding queue 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:50 -07:00
Orit Wasserman
553bf4b996 rgw: add a check_bucket_shards to the quota 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:50 -07:00
Orit Wasserman
1ed76a680c rgw: moved reshard_bucket into RGWReshard 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:49 -07:00
Orit Wasserman
dfdc2b2cc0 rgw: remove cct from RGWReshard and BucketIndexLockGuard 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:49 -07:00
Orit Wasserman
ad138a4d72 rgw: add resharding thread 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:49 -07:00
Yehuda Sadeh
806d47329e rgw: store new num shards in bucket index reshard entry 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:48 -07:00
Yehuda Sadeh
42b3ee12ea rgw: fix deletion of index completion object 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:48 -07:00
Yehuda Sadeh
94e3c380ab rgw: reshard complete handling, use multiple locks 
to reduce contention. Also, fix completions cleanup.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:47 -07:00
Yehuda Sadeh
da1f5bc910 rgw: clean up index completions on shutdown 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:47 -07:00
Yehuda Sadeh
60778b0214 rgw: update data log after index completion reshard handling 
Also rearrange code a bit.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:47 -07:00
Yehuda Sadeh
245cd0f9af rgw: guard resharding in async completions 
Catch return value from async index completions, resend operations
synchronously on a separate worker thread if needed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:46 -07:00
Yehuda Sadeh
e18e271490 rgw: reshard guard for versioned operations 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:46 -07:00
Yehuda Sadeh
9677f7d31c cls/rgw: changes to support reshard guard on olh bi ops 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:46 -07:00
Yehuda Sadeh
8fd589d1eb rgw: handle reshard wait logic in cancel() too 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:45 -07:00
Yehuda Sadeh
d250fab244 rgw: fix update_bucket_id, reshard blocking logic 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:45 -07:00
Yehuda Sadeh
9766f5b92d rgw_admin: add rgw-admin reshard status command and initial blocking logic 
A command that returns the reshard status field on the bucket index objects (all
shards).
Also, initial implementation of identifying and blocking while reshard in
progress.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:44 -07:00
Yehuda Sadeh
09ccfebf68 rgw: more detailed resharding status kept in bucket header 
So that we can differentiate between in-progress and complete status.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:44 -07:00
Yehuda Sadeh
e4d4aa8b86 rgw: a few fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:44 -07:00
Yehuda Sadeh
660a512635 rgw: mark source bucket shards when starting reshard 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:43 -07:00
Yehuda Sadeh
bb0cb4d7a5 rgw: move resharding code to BucketReshard class 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:43 -07:00
Yehuda Sadeh
7a34a049ae rgw: split per-bucket resharding logic out from general resharding 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:43 -07:00
Yehuda Sadeh
0bc15671ee rgw: revert some reshard blocking logic 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:42 -07:00
Yehuda Sadeh
7d44601e83 cls/rgw: create an objclass method to guard in-progress resharding 
This method will be called on the bucket index shard.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:42 -07:00
Orit Wasserman
4ddb785bb0 rgw: RGWReshard update to new cls API 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:41 -07:00
Yehuda Sadeh
a07e8eb6af cls/rgw: adjust types and ops 
- don't keep data bufferlist in struct cls_rgw_bucket_instance_entry
 - remove rgw_get_bucket_instance

We need to keep the reshard info as part of the bucket index header (for
each source bucket shard), and not in a separate object. Otherwise it'll
be impossible to have the resharding check atomic as part of the regular
bucket index updates.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:41 -07:00
Yehuda Sadeh
04cf0ca074 cls/rgw: minor changes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:41 -07:00
Yehuda Sadeh
44e46b35f0 cls/rgw: generate_reshard_key, change output param 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:40 -07:00
Yehuda Sadeh
f446b80e2a cls/rgw: return default header if not initialized 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:40 -07:00