RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-01-02 00:52:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
85,157 Commits 212 Branches 394 Tags 1.5 GiB
73c2ddf932
Commit Graph

85157 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yehuda Sadeh
73c2ddf932 doc/radosgw: cloud sync docs 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 16:08:40 -07:00
Yehuda Sadeh
7ece026713 rgw: rename aws tier type to 'cloud' 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:41 -07:00
Yehuda Sadeh
a5cbddb1be rgw: etag fixes 
Use string instead of bufferlist to avoid potential issues.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
b2a2b959ba rgw: fixes following code review 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
d49a1362e4 test/rgw/test_multi: fix a few tests to only iterate over rw zones 
Some of the tests require at least two read-write (regular rgw) zones

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
6df7a54198 test/rgw: zone_cloud: deal with key representation and other fixes 
Needed to present a key to the tests that reflected its original name
and version_id (and etag), so that the callers don't need to be modified.
However, this can only be achieved if we get the key, which doesn't work
if the caller was just listing the bucket objects. Created a new CloudKey
class to deal with the different issues there.
Also, other test related fixes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
4a4a18f9d6 rgw: streaming put also stores content_type and other fields 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Chang Liu
e0e6510665 rgw: fix parse_tier_config_param function 
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
9b283915e3 rgw: don't call http data callbacks under lock 
There is no need to hold req_data->lock when calling into client
callbacks. This removes an unneeded lock dependency (that is a
problem when cancelliing coroutines stack).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
057aac4516 rgw: coroutines: cancel stacks on teardown 
If we don't cancel stacks, ops might not be destructed, so ops callbacks
could still be active.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
319d0387e8 rgw: don't store etag with extra null character at the end 
head objects etag attr doesn't need to store an extra null char.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:40 -07:00
Yehuda Sadeh
4ba4bc9b8e rgw: cloud sync: store source object info in destination object 
store extra meta params on target object (original name, version_id, etag,
etc.)

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
7572335cf4 rgw: cloud sync: store versioned epoch in target object 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
340005c1f1 test/rgw: initial work on cloud sync test 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
5953965193 rgw: api adjustment following rebase 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
43ee30813f rgw: rework cloud sync configuration 
Change the configuration structure to separate between
connections and acl mappings, and other changes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
7aadf29f2d rgw: fixes following rebase 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
bcb4644c57 rgw: use different io id when waking up interval wait 
Can't use a zero channel id as a wildcard.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
d5d2d3208c rgw: fix compilation warning 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
b09fcde5b3 vstart: fixes for mstart 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:39 -07:00
Yehuda Sadeh
8069d61550 rgw: aws sync: apply acl mappings 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
9daa49e915 rgw: aws sync: sync acls 
still not transforming acls according to configuration

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
74281d5d47 rgw: aws sync: acl_mappings config handling 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
08451d7910 rgw: aws sync: remove bucket suffix configurable 
No need to have a special suffix, target path can be used instead.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
6e83d24dde rgw: aws sync: use configurable target path 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
d97b9fb8d3 rgw: aws sync: dump config 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
28bc88b7cd formatter: add two utility classes 
For automatic section scoping

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
d130b59f33 rgw: aws sync: configure different targets 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
ce1e91541f rgw: aws sync: new config structure, support multiple connections 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
d9fb18d0a6 formattable: more fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:38 -07:00
Yehuda Sadeh
e154b4b8cb rgw: sync module tier config fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:37 -07:00
Yehuda Sadeh
0c71d8bb64 formattable: fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:37 -07:00
Yehuda Sadeh
b279f9c93d rgw: cloud sync, sync object meta 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:37 -07:00
Yehuda Sadeh
feb67c9bd5 formattable: adjust array access interface 
appending new entry by providing empty index, setting existing entries
by providing negative index.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:37 -07:00
Yehuda Sadeh
76b79cb8cb ceph_json: formattable, set, erase, unitest 
Extend the formattable api to provide a mechanism to set and erase
entities by a string key that references them. E.g., "foo.bar[123].xyz"

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:37 -07:00
lvshanchun
056738d863 rgw: handle the situation that get a 409 response from S3 correctly 
pass the http body and use it when creating a bucket

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:37 -07:00
lvshanchun
cbe5a86427 rgw: avoid use Chunked transfer encoding in RGW's request 
Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:37 -07:00
lvshanchun
0d4c927455 rgw: add a bucket-suffix tier-config 
Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:37 -07:00
lvshanchun
2d8f21c7e9 rgw: virtual hosted-style support 
add host-style field in tier-config to specify the related
zone's hosted-style used in request from RGW, if this config
is not specified, path hosted-style will be used as default.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:37 -07:00
lvshanchun
a87e413561 rgw: RGWRESTStreamRWRequest::send_prepare(), url encode dest 
encode the resource in send_prepare(), as a result, all the callers
to do_send_prepare() have do url_encode before calling it.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:37 -07:00
lvshanchun
c43d497376 rgw: add encode_slash param in url_encode 
when sent request to S3, we should not encode the forward
slash character('/') in the object key name, so we need add
a encode_slash param in url_encode to decide whether to encode
the slash or not.

Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:36 -07:00
lvshanchun
425b328726 rgw: use GMT time format defined by RFC1123 in http request header 
Signed-off-by: lvshanchun <lvshanchun@gmail.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
7f2496ee7f rgw: sync modules: adapt to formattable based config 
still need to fix rgw-admin

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
a2530980e8 test/ceph_json_formattable: add unitest 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
854dea8243 ceph_json: formattable fix, api changes 
add implicit casts

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
32e77797c7 ceph_json: refine Formattable api 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
d8dc2be393 rgw: sync modules, use JSONFormattable 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
297a1b725b rgw: rgw_str_to_bool() should also accept "true" 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:36 -07:00
Yehuda Sadeh
41eeca5ce3 common/formattable: initial work 
initial work on a dict-like api

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-04-12 15:38:32 -07:00
Chang Liu
b754a28aa5 rgw/cleanup: fix compile warnning 
1. overloaded virtual function
2. virtual functions but no virtual destructors

Signed-off-by: Chang Liu <liuchang0812@gmail.com>
 2018-04-10 08:05:39 -07:00