RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-28 06:23:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
73,225 Commits 211 Branches 394 Tags 1.4 GiB
93106dac32
Commit Graph

73225 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yehuda Sadeh
93106dac32 rgw: increase max shard prime, consolidate logic 
Handle larger num of shards (max ~64k now). Use the same hash logic for shard ids
everywhere.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:18:03 -07:00
Yehuda Sadeh
ba1138d7a8 rgw: new num shards proportional to number of objects per shard 
and also limit number of shards to not exceed the max.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:18:03 -07:00
Yehuda Sadeh
3b7fd0ffd5 rgw: handle zones_trace in bi async completion handler 
needed following a rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:18:02 -07:00
Yehuda Sadeh
ef48cc4134 rgw: remove debug helper code 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:18:02 -07:00
Yehuda Sadeh
2417c2c0c6 rgw: ignore -ENOENT on bi_list() 
just treat it as success with empty result

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:18:02 -07:00
Orit Wasserman
dc04b20f1d rgw: run resharding thread only on master zone 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:18:01 -07:00
Orit Wasserman
6c0611136d rgw: add check to see if out is null 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:18:01 -07:00
Orit Wasserman
246c78ebfc rgw: update entry in resharding log when resharding has started 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:18:01 -07:00
Orit Wasserman
9b3c91cbd9 rgw: start resharding theard 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:18:00 -07:00
Yehuda Sadeh
f2b0eb0863 rgw: replace reshard blocking sleep with interruptible condition wait 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:18:00 -07:00
Orit Wasserman
45ad93ddee rgw: add bucket to resharding queue if needs resharding 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:18:00 -07:00
Yehuda Sadeh
c6dc81cf4f rgw-admin: radosgw-admin reshard execute -> reshard process 
to be in line with other radosgw-admin commands.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:59 -07:00
Yehuda Sadeh
07b21866a1 rgw: update bucket instance with resharding details 
can be useful info for later in a cross zone sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:59 -07:00
Yehuda Sadeh
40faa8d952 radosgw-admin: reshard execute output adjustments 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:59 -07:00
Yehuda Sadeh
aa3f71d6b1 cls/rgw: remove unused field in reshard entry 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:58 -07:00
Yehuda Sadeh
1da369e1e8 rgw: reshard: renew lease and handle marker when listing reshard repo 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:58 -07:00
Yehuda Sadeh
7054308e85 cls/rgw: fix reshard_list truncation identification 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:58 -07:00
Yehuda Sadeh
6cb30f9c95 cls/rgw: expose api to generate reshard indexing key 
So that we could use it as a marker when doing reshard_list.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:57 -07:00
Yehuda Sadeh
81693efd78 rgw: remove unneeded and duplicate code 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:57 -07:00
Yehuda Sadeh
6c540bd5fe rgw: initialize bucket structure 
after reading bucket info

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:56 -07:00
Yehuda Sadeh
485a11fe59 rgw-admin: revive reshard execute command 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:56 -07:00
Yehuda Sadeh
ed72675ba2 rgw: bucket_id does not include bucket name 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:55 -07:00
Yehuda Sadeh
b516c92af5 rgw: bi_list() initialize is_truncated if -ENOENT 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:54 -07:00
Yehuda Sadeh
999cca4ed3 rgw: internal api cleanup 
rename method, minor change

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:54 -07:00
Yehuda Sadeh
436f151e55 cls/rgw: reshard add command doesn't read existing status 
No need to read existing status.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:54 -07:00
Yehuda Sadeh
2da8dc5d10 cls/rgw: fix remove_reshard objclass op 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:53 -07:00
Yehuda Sadeh
f677d0e960 cls/rgw: modify logshard key 
The logshard key shouldn't include bucket id.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:53 -07:00
Yehuda Sadeh
8e1bf1b9e8 rgw: multiple fixes and adjustments related to resharding scheduler 
still wip, but:
 - get rid of some unneeded rados locking
 - rename shards to logshards where relevant to avoid confusion
 - remove unused code

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:53 -07:00
Yehuda Sadeh
b757e7e16e rgw: fix compilation following rebase 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:52 -07:00
Yehuda Sadeh
cac047a88b rgw: replace reshard blocking sleep with interruptible condition wait 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:52 -07:00
Orit Wasserman
7114124673 rgw: actually reshard an entry from the resharding queue 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:52 -07:00
Orit Wasserman
c7fc87f9f1 rgw: check if buckets needs resharding on all ops that add new objects 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:51 -07:00
Orit Wasserman
444b9c918c rgw: add bucket to resharding queue if needs resharding 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:51 -07:00
Orit Wasserman
e03b85e987 rgw: check for bucket resharding when adding an object 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:51 -07:00
Orit Wasserman
9a084497fd rgw: add a function to add a bucket to the resharding queue 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:50 -07:00
Orit Wasserman
553bf4b996 rgw: add a check_bucket_shards to the quota 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:50 -07:00
Orit Wasserman
1ed76a680c rgw: moved reshard_bucket into RGWReshard 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:49 -07:00
Orit Wasserman
dfdc2b2cc0 rgw: remove cct from RGWReshard and BucketIndexLockGuard 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:49 -07:00
Orit Wasserman
ad138a4d72 rgw: add resharding thread 
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
 2017-06-05 13:17:49 -07:00
Yehuda Sadeh
806d47329e rgw: store new num shards in bucket index reshard entry 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:48 -07:00
Yehuda Sadeh
42b3ee12ea rgw: fix deletion of index completion object 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:48 -07:00
Yehuda Sadeh
94e3c380ab rgw: reshard complete handling, use multiple locks 
to reduce contention. Also, fix completions cleanup.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:47 -07:00
Yehuda Sadeh
da1f5bc910 rgw: clean up index completions on shutdown 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:47 -07:00
Yehuda Sadeh
60778b0214 rgw: update data log after index completion reshard handling 
Also rearrange code a bit.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:47 -07:00
Yehuda Sadeh
245cd0f9af rgw: guard resharding in async completions 
Catch return value from async index completions, resend operations
synchronously on a separate worker thread if needed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:46 -07:00
Yehuda Sadeh
e18e271490 rgw: reshard guard for versioned operations 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:46 -07:00
Yehuda Sadeh
9677f7d31c cls/rgw: changes to support reshard guard on olh bi ops 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:46 -07:00
Yehuda Sadeh
8fd589d1eb rgw: handle reshard wait logic in cancel() too 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:45 -07:00
Yehuda Sadeh
d250fab244 rgw: fix update_bucket_id, reshard blocking logic 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:45 -07:00
Yehuda Sadeh
9766f5b92d rgw_admin: add rgw-admin reshard status command and initial blocking logic 
A command that returns the reshard status field on the bucket index objects (all
shards).
Also, initial implementation of identifying and blocking while reshard in
progress.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-06-05 13:17:44 -07:00