RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-02-20 17:37:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
107,090 Commits 4 Branches 395 Tags 1.3 GiB
68d3c86106
Commit Graph

107090 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yehuda Sadeh
ce962f3470 rgw: bucket sync: invalidate hints data if raced with another writer 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 11:36:41 -08:00
Sage Weil
14c4dbb03f osd: dispatch_context and queue split finish on early bail-out 
If we bail out of advance_pg early because there is an upcoming merge, we
still need to dispatch_context() on rctx before we drop the PG lock.  And
the rctx that we submit needs to include the on_applied finisher comit
to call _finish_splits.

This is noticeable (at least) when there is a split and merge that are
both known.  When we process the split, the new child is added to new_pgs.
When we get to the merge epoch, we stop early and take the bail-out
path.

Fix by adding a dispatch_context call for this path.  And further make sure
that both dispatch_context callers in this function queue up the
new_pgs event.

Fixes: https://tracker.ceph.com/issues/43825
Signed-off-by: Sage Weil <sage@redhat.com>
 2020-01-28 13:33:53 -06:00
Ali Maredia
7cf2af6e5c qa: remove rados/basic/tasks/rgw_snaps.yml 
rgw_snaps tasks should not be running in the rados suite.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
 2020-01-28 14:29:27 -05:00
Yehuda Sadeh
b972a35f66 rgw: fix compilation after rebase 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:34:02 -08:00
Neha
8aa3c65b56 mon/MgrMonitor.cc: add always_on_modules to the output of "ceph mgr module ls" 
Makes it easier to know what the always_on_modules for each release are, without
having to look at the code.

Signed-off-by: Neha Ojha <nojha@redhat.com>
 2020-01-28 18:24:55 +00:00
Yehuda Sadeh
b6b30cada3 rgw: don't include empty buckets in potential related buckets 
Being used by the sync hint system

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
9ed02f9d36 rgw: keep sync_policy as optional member inside RGWBucketInfo 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
d6cafc35ae rgw: move a few basic types into rgw_basic_types.{h,cc} 
To help with the fight against circular dependencies

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
485ba08192 rgw: bucket sync: initiate bucket meta sync if can't find sync handler 
If bucket sync handler returns -ENOENT, try to get bucket info. This can
trigger bucket meta sync.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
768dc94f62 rgw: trim bilog: send bucket param instead of source-bucket param 
For backward compatibility, so that older rgws could handle it appropriately.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
36e0c75e2f rgw: bucket sync: don't allow replication api on legacy config 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
50f13fd54b PendingReleaseNotes: rgw, bucket granularity 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
2d416ad527 rgw: clarify api 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
ed95206e22 rgw: bucket sync: get_hint_entities(), call read_bucket_info once 
once per bucket

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
fbec3aeff6 rgw: bucket sync: no create param in find directional/symmetrical 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
98dd994356 rgw: bucket sync: don't use optional for some members 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
db1a7335b1 rgw: fix a potential leak 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
f0ecbc3a23 rgw: remove return std::move() 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
6a4e529b55 rgw: fetch_remote_obj() create default acls if not found 
Create default acls if the original object did not have acl header.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
bd7247c184 rgw: remove superfluous alias 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
0c69aa00ff rgw: bucket sync: doc fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
3a8eea48b7 rgw: bucket sync: find first key with longest matching prefix 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
d0250e22c1 rgw-admin: fix bucket sync markers for non-trivial case 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
a44f6ace22 rgw: bucket autotrim: fetch aggregated bilog info of all bucket targets 
Add a new option to the bilog status request that allows returning the
min status for all buckets in a zone that sync from a specific bucket.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
bd2fd97ff9 rgw: bucket trim: only fetch status from relevant zones 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
8f6be2f1e1 rgw: bucket sync: use list instead of vector 
Keeping pointers into entries inside the container, vector might be
reallocated.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
caac422445 rgw: sync policy doc 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
82e44d9c8c ceph_argparse: binary flag value can be separate arg 
Resurrect old behavior where a binary flag could be detached from the
argument, e.g., "--rgw-cache-enabled false".

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:40 -08:00
Yehuda Sadeh
a657ec2f3f rgw-admin: fix use of opt_storage_class 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
d04ea3aab5 qa: s3a-hadoop: fix user creation 
This breaks with the new more strict radosgw-admin command line parsing

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
4d5e294dfc rgw-admin: source zone, dest zone converted to zone id 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
873d530dab rgw: pipe map in pipe_set should be multimap 
Can hold multiple entries for the same pipe id (if flow defines
more than a single entry).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
bb797d0cb1 rgw: flatten binary encoding for rgw_sync_pipe_params 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
5386f987c1 rgw: implement s3 get bucket replication api 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
c1199c101e rgw: implement s3 delete bucket replication api 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
4a587952df rgw: put bucket replication extension 
Now able to set specific source and destination zones

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
d2c70d9297 rgw: implement s3 put-bucket-replication api 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
dc742e466f rgw: fix pipes zone wildcard match logic 
and improve logging

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
6988fd7b44 rgw-admin: bucket sync status related fixes and improvements 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
99caf86cc7 rgw: use negative shard id when creating sync pair 
negative shard id means the bucket has a single shard. This
is to be backward compatible.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
16be6ecb20 rgw: remove unneeded #warnings 
and other cleanups

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
9a7a9895b4 rgw: sync: only update data log if bucket exports data 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
58efd3f9ad rgw: pubsub: user source bucket owner for event, not dest 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
e87533d2e1 rgw: remove a #warning with a comment 
It's a nice to have, not crucial

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
33c6e09e73 rgw: sync: object sync info oid depends on pipe 
The object sync info object (used in cloud sync module) depends on pipe. Different
oid is crafted if source and destination buckets are not the same.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
009608dd50 rgw: data sync: guard against racing source object change 
Source object might have changed since we tried to sync it,
and the decisions we made originally might be irrelevant.
After reading object header, check if the original dest params
are the same dest params we find now for this object, otherwise
need to retry.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
a8ded23d8e rgw: sync: override dest object owner if needed 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
a4550801cb rgw: data sync: check user permission in user level sync 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
b62311b3f2 rgw: auth: api to create auth identity without req_state 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00
Yehuda Sadeh
1bc51e2fc8 rgw: api to check bucket and object permissions without need for req_state 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2020-01-28 10:20:39 -08:00