RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-03-11 02:39:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,457 Commits 219 Branches 396 Tags 1.4 GiB
e73eac1bbd
Commit Graph

131457 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Shilpa Jagannath
e73eac1bbd rgw: update bucket sync status after bucket shards finishes current gen 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
94fdadd356 rgw: reshard preserves old index in multisite 
if the old index is still referenced by an InIndex log layout, we can't
call clean_index() to remove the index objects yet. log trimming will do
that later, once the bilogs are no longer needed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
190c260b3e rgw: BILog_List handles requests for generation=0 
previous logic treated requests for generation=0 as the latest gen

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
d7750eeb13 rgw: zero-initialize rgw_bucket_sync_status::incremental_gen 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
3f86289f1b rgw: handle older/newer generations after reading bucket sync status 
wait until we've read the bucket sync status and found that we're in
incremental sync before we start using incremental_gen for comparison

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Adam C. Emerson
cdfc44006c rgw: Handle entries of the wrong generation 
Drop entries from past generations.

Send entries of future generations to the error repo for retry.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
fda49c4772 rgw: rgw_data_change can decode v1 format if gen was 0 
but if gen>0, require decoders to understand the v2 format. this way,
old clients can't decode entries with gen>0, so they won't be able to
serve them to other zones

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
56c7f37b55 rgw: rename rgw_data_change::gen_id 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Adam C. Emerson
6f83f07d7f rgw: add gen parameter to RGWDataChangesLog::add_entry 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
 2022-05-27 15:47:32 -04:00
Adam C. Emerson
2c94978c29 rgw: Add gen_id to rgw_data_change 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
 2022-05-27 15:47:32 -04:00
Adam C. Emerson
a0d1290503 rgw: vector<rgw_data_change_log_entry> not list 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
ccd68c668d rgw: add json encoding of bucket layout types 
adds a "layout" section to RGWBucketInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
9fd4db2bf7 json: add encode_json() overload for string_view 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
5c7b68606e rgw: refactor per-entry reshard logic into separate function 
this cuts down on nesting and avoids the need for goto

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
1c1bd6447c rgw: reshard adds a log layout for the new index 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
122da8f14a radosgw-admin: try reshard even if bucket is resharding 
allow reshard in case a previous reshard failed. if the reshard is
actually still in progress, we'll fail to get the reshard lock

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
fa59327279 test: fix threading for FaultInjector death tests 
addresses test timeout and warning message:

[WARNING] /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest-death-test.cc:1121:: Death tests use fork(), which is unsafe particularly in a threaded context. For this test, Google Test detected 3 threads. See https://github.com/google/googletest/blob/master/googletest/docs/advanced.md#death-tests-and-threads for more explanation and suggested solutions, especially if this is the last message you see before your test times out.

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
6156f0475f radosgw-admin: remove fault injection options from usage 
these are only used for testing, not administration

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
d99f1fd2a5 rgw: RGWBucketReshard doesn't need a friend 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
ed40166b1d rgw: fix decode of cls_rgw reshard types 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
91d5ba628f rgw: BucketReshardManager takes layouts 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
824fdc91af rgw: clean up uses of BucketShard::init() without info 
the rgw_bucket overload of BucketShard::init() has to look up the bucket
info. use the RGWBucketInfo overload when we have one

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:32 -04:00
Casey Bodley
6e6f59b945 qa/rgw: disable coredumps for reshard fault injection 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
2a51bc6ce1 test/rgw: add test_bucket_reshard() for fault injection testing 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
8c2bac38f8 test/rgw: test_rgw_reshard.py exec_cmd() can return error code 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
1cb18ba018 radosgw-admin: 'bucket reshard' returns positive error codes 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
b7041fff3f rgw: reshard first cleans up old-style reshards 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
0f9870ba39 rgw: refactor reshard init/cleanup with fault injection 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
e4c3bbe3ae rgw: add typedef for ReshardFaultInjector 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
38bb228934 rgw: BucketReshardManager stores BucketReshardShards by value 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
2ef324bb19 rgw: remove unused attrs from RGWBucketReshard 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
eb304f4bc5 rgw: reshard tests with fault injection 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Casey Bodley
69fc20f0c8 common: add generic FaultInjector 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
9424b1b8b2 rgw: Better error handling 
- function update_bucket() handles updating bucket state

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
cb376a4c44 rgw: use a helper function to handle repetitive num_shards check 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
d0214ab7f7 rgw: code organization fixes and error handling 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
6a922e10c4 rgw: clean up fixes for extra spaces and indentation. 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
eb8c6b7585 rgw: helper function to set reshard status from rgw::BucketReshardState. 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
a4af6ea82a rgw: minor fixes 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
3d3ecb2a1e rgw: failing to reshard, restore old indexes 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
fbe6842165 rgw: allow clean_index to take const bucket_index_layout_generation& 
fix bi_get() to get objects after being resharded

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
7df7b275b0 rgw: - take non-optional gen_id in svc_bi_rados functions 
- call init_index() on target layout during reshard process.
       Takes const rgw::bucket_index_layout_generation&

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Manjarabad Jagannath
112aea9e19 rgw: make RGWBucketInfo non-const in functions using guard_reshard() 
- remove unused 'instance_id' from cls_rgw_reshard_entry
 - other minor fixes

Signed-off-by: Shilpa Manjarabad Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Manjarabad Jagannath
680bfdcfeb rgw: target_layout to be optional 
Signed-off-by: Shilpa Manjarabad Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
f78b1a2ead rgw: use rgw::BucketReshardState for reshard state change 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
9f68c54fc1 rgw: fetch updated bucketinfo rather than bucket_id in guard_reshard 
To be able to read the updated layout.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
bd0b0e5d98 rgw: use BucketReshardState states in appropriate places 
This replaces cls_rgw_reshard_status.

Remove unused parameters in cls_rgw_bucket_instance_entry.
Other minor cleanup fixes.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
b6c91c506c rgw/dynamic-resharding: Allow clean_index to take generation number 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
7348a8397a rgw/dynamic-resharding: reshard bucket using the same instance id 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00
Shilpa Jagannath
faf5b79876 rgw/dynamic-resharding: remove creation of new bucket instance 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
 2022-05-27 15:47:31 -04:00