RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-30 07:23:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
92,892 Commits 210 Branches 394 Tags 1.4 GiB
6b7b023350
Commit Graph

92892 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yehuda Sadeh
6b7b023350 docs/rgw: document pubsub 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
31b352fcfb rgw: pubsub: trivial push notifications 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
eb213bb302 rgw: pubsub: filter events 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
72e8e1c76e common/str_list: get_str_set(), add second param for template 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
4f47d987cd ceph_json: define second param for std::set encode_json() template 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
cfea5451ec rgw: pubsub: fix events distribution 
check was located in the wrong place

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
5f353e5f15 rgw: pubsub: remove warning 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
fa13c31155 rgw: pubsub: fix encoding of json config 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
27ec9c3dda rgw-admin: pubsub topic get command 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
239a53d3cb rgw: pubsub: dest bucket registers with lifecycle 
So that events expire after X days

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
10ac6692ac rgw: more lifecycle code cleanup and tooling 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
d80dc32967 rgw: lifecycle utility coroutine 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
89f773db34 rgw: pubsub: have sub name as event oid prefix 
dest should be unique for each subscription

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
dc47ce9f23 rgw-admin: add a new lc get command 
Useful for debugging lifecycle issues

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
36a1895d8f rgw: lifecycle code cleanup 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
fcc8d05d96 rgw: pubsub: create notif via REST api sets dest bucket 
This got lost somewhere in the refactoring work

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
eac1dcf228 rgw: coroutine set_state() also sets retcode 
Generally recode is needed for the caller, but current might need it too.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
b72879d469 rgw: pubsub: adjust event encoding 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
36470620e0 test_json_formattable: extend for formattable encoding 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:43 -08:00
Yehuda Sadeh
fa553e1b46 common/ceph_json: formattable encoding fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
66ee1038d2 rgw: pubsub: events hold a generic formattable 
Instead of encoding specific object store events, make it
hold a generic formattable struct, so that pubsub can be
used in the future for registering other event types.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
e47a228a86 common/formatter: json formattable is now also a formatter 
This way we can construct a formattable structure by encoding
objects into it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
19cb24acdf rgw-admin: pubsub: use effective conf 
the effective conf has the defaults set if needed

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
f94f597931 rgw: cleanups and data structure changes, api changes 
split user topics and bucket topics, can have multiple buckets per topic
and multiple topics per bucket. Buckets don't hold subscription info.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
26b55f960f rgw: pubsub: override basic rest handler callbacks 
these shouldn't do anything there.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
f6ccc6d7bd rgw: pubsub: doesn't support quota 
Check if handler supports quota, pubsub doesn't.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
e702343306 rgw: sync module instance specifies whether user writes supported 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
b85f9f9037 rgw: pubsub: REST api 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
2489cb76e3 rgw: add args.get_int() 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
2a9b315a71 rgw-admin: pubsub: pull sub, rm event 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
a0f0a25e48 rgw: pubsub: store event in bucket index 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
92b278d86c rgw: pubsub config fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
95d94cd59f rgw: pubsub: multiple fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
b78470627a rgw: data sync: add start_sync() callback 
The existing init_sync() is only called when the sync is first initialized,
need a call for whenever sync runs.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
17d4e96a6f rgw: pubsub: fetch user subs 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
cd37480373 rgw-admin: pubsub related new commands 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
a47d07cc03 rgw: define pubsub user structures 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
d5371502cd rgw: fetch topics via a cr, other changes 
this will allow fetching topics info from rados, so that we
don't need to hold in in memory.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
23ef624568 rgw: pubsub: more events coverage 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
8bf3648d25 rgw: introduce cr singleton, use for subscriptions init 
Prevent concurrent initalizations of the same subscription

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
a10a95fabd rgw: pubsub fixes 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
85de19385d rgw: pubsub config cleanup 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
ce99093760 rgw: register pubsub sync module 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
6cff97f871 rgw: distribution of events to subscriptions 
more work required

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
bdd1a690a6 rgw: pubsub: message storage functionality 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
3bd6661082 rgw: new cr for simple object put 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
0fb443aa35 rgw_admin: new object put command 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
fad214eb4b rgw: easy data access helper class 
Only does put object right now.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
702f7f42b4 rgw: pubsub: subscription init and more 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00
Yehuda Sadeh
b7a9f21762 rgw: new utility crs 
- get user info
 - create bucket

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2018-12-11 00:10:42 -08:00