RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-29 06:52:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
73,225 Commits 1 Branch 394 Tags 1.4 GiB
93106dac32
Commit Graph

73225 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yehuda Sadeh
6d5555accd rgw: expose conf_get(), conf_get_int(), conf_get_bool() 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:45 -07:00
Yehuda Sadeh
f1e681bf83 common/ceph_json: map type can handle 2 or 3 template arguments 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:45 -07:00
Yehuda Sadeh
874223b251 rgw: es: handle get config api in es zone 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:45 -07:00
Yehuda Sadeh
603863a46e rgw: es: api to delete bucket mdsearch config 
DELETE /bucket?mdsearch

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:44 -07:00
Yehuda Sadeh
acba916125 rgw: es: api to retrieve bucket mdsearch config 
GET /bucket?mdsearch

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:44 -07:00
Yehuda Sadeh
7b61c992b7 rgw: es: also return custom int and date results 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:44 -07:00
Yehuda Sadeh
e4e6e1e51c rgw: es: limit indexing to only configured user meta 
only metadata keys that are configured on user's bucket are allowed
to be indexed. Also, use the type of meta that was configured.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:44 -07:00
Yehuda Sadeh
8249f76b80 rgw: new api to configure bucket's custom keys for mdsearch 
POST /bucket?mdsearch
x-rgw-meta-search: <x-amz-meta-key>[;<str|int|date>][,...]

note that x-amz-meta-search header can also be used.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:43 -07:00
Yehuda Sadeh
3f62f17f94 rgw: guard restricted fields in es query 
don't allow users to use the 'permissions' field.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:43 -07:00
Yehuda Sadeh
de5bd34eb2 rgw: add dout defs 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:43 -07:00
Yehuda Sadeh
484fd2b8cb rgw: support field aliases in es compiler 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:43 -07:00
Yehuda Sadeh
21ee7c11b5 rgw: support more meaningful compilation error string 
in es module

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:42 -07:00
Yehuda Sadeh
42746ac9d2 rgw: support for generic entities types in es 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:42 -07:00
Yehuda Sadeh
d026a23d1f rgw: initial support for multiple types in es requests 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:42 -07:00
Yehuda Sadeh
1e886475bb rgw: metadata search pagination 
Leveraging elasticsearch "size" and "from" for doing pagination. This
is not optimal, but sadly it's currently the way to go as elasticsearch
does not have an efficient way to retrieve ordered entries (other than
using stateful api). This should be revisited in the future. A preferable
way would be to hold a unique identifier for each object doc, and have
the query sort by that unique identifier (that could be used as marker).
At the moment sorting by anything would require elasticseatch to load
all data into memory.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:42 -07:00
Yehuda Sadeh
99b49acc2e rgw: add permissions and bucket to es query 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:41 -07:00
Yehuda Sadeh
0180650d09 rgw: also dump custom meta entries 
when doing metadata search

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:41 -07:00
Yehuda Sadeh
86fe1223c9 rgw: send request to elasticsearch and parse response 
and send back response to user

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:41 -07:00
Yehuda Sadeh
698763fb13 rgw: extend interfaces to allow sending data 
when retrieving REST resource

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:41 -07:00
Yehuda Sadeh
b6fe05d9a4 rgw: move code around 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:40 -07:00
Yehuda Sadeh
952e72d979 rgw: can send data in RGWRESTStreamRWRequest::send_request() 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:40 -07:00
Yehuda Sadeh
9f494ff473 rgw: make key param in RGWRESTStreamRWRequest::send_request() optional 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:40 -07:00
Yehuda Sadeh
c9d6882275 rgw: rename a few methods 
just rename calls

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:40 -07:00
Yehuda Sadeh
e515bf3b15 rgw: meta search rest handler can access es module 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:39 -07:00
Yehuda Sadeh
bb46f19e44 rgw: work on REST handler for es module 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:39 -07:00
Yehuda Sadeh
42130df362 rgw: rename rgw_rest_es.cc to rgw_es_query.cc 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:38 -07:00
Yehuda Sadeh
0fdd24ff80 rgw: simplify es compile interface 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:38 -07:00
Yehuda Sadeh
86ea424313 rgw: move code into class 
just cleaning up

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:38 -07:00
Yehuda Sadeh
6de586ca74 rgw: handle nested fields in es queries 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:38 -07:00
Yehuda Sadeh
67bfdb4541 rgw: implement init_sync() callback in es module 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:24:37 -07:00
Yehuda Sadeh
49b27f4f90 rgw: define es index mapping 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:18:58 -07:00
Yehuda Sadeh
6d4733c58b rgw: add init callback to sync modules 
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:18:58 -07:00
Yehuda Sadeh
8fd55cc2e0 rgw: initial implementation of mdsearch query compiler 
convert infix queries that look as follows:

[(]<name> <operator> <value> [)] [<and|or> ...]

into a prefix structure that is understood by elasticsearch.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:17:18 -07:00
Yehuda Sadeh
760c5e4f86 Merge pull request #15184 from cbodley/wip-qa-rgw-cleanup 
qa/rgw: remove apache/fastcgi and radosgw-agent tests

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-05-30 13:09:31 -07:00
Casey Bodley
6049d97740 Merge pull request #13851 from zhangsw/optimize-datasync 
rgw: optimize data sync. Add zones_trace in log to avoid needless sync.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2017-05-30 14:16:07 -04:00
David Zafman
1dbb15d340 Merge pull request #15368 from dzafman/wip-20089 
osd: When scrub finds an attr error mark shard inconsistent

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-05-30 10:46:23 -07:00
Sage Weil
e88cccf969 Merge pull request #15350 from mrbojangles3/master 
libradosstriper: Add example code
 2017-05-30 12:32:06 -05:00
Sage Weil
3f99839b41 Merge pull request #15356 from joscollin/wip-test-msgr-warning 
test/msgr: silence warnings from -Wsign-compare
 2017-05-30 12:29:51 -05:00
Sage Weil
b38495c84e Merge pull request #15362 from andihit/doc-rados-xattr-commands 
doc: add rados xattr commands to manpage

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-05-30 12:28:34 -05:00
Sage Weil
57bef92ba1 Merge pull request #15366 from joscollin/wip-cleanup-test-remove-commented-code 
test/old: Removed commented code

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-05-30 12:25:39 -05:00
David Zafman
44736aac74 osd: When scrub finds an attr error mark shard inconsistent 
Fixes: http://tracker.ceph.com/issues/20089

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-05-30 09:32:53 -07:00
Patrick Donnelly
acde67b36a
mds: check export pin during replay 
Fixes: http://tracker.ceph.com/issues/20039

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-05-30 09:09:22 -07:00
Patrick Donnelly
76335b0e0f
qa: improve debug message for subtree wait 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-05-30 09:08:27 -07:00
Patrick Donnelly
2cc1fb04df
mds: use generic name for export dir wait class 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-05-30 09:08:24 -07:00
Sage Weil
3d7a3d7ab8 Merge pull request #14971 from SUSE/wip-mgr-py-sub-interpreter 
mgr: load modules in separate python sub-interpreters

Reviewed-by: John Spray <john.spray@redhat.com>
 2017-05-30 11:02:27 -05:00
Sage Weil
496d63a163 os/bluestore: bluestore_debug_fsck_abort 
Abort fsck early to get a massif result.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-05-30 11:54:28 -04:00
Yuri Weinstein
805acce791 Merge pull request #15259 from zealoussnow/wip-0524 
build: remove ceph-disk-udev entirely 

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
 2017-05-30 08:44:47 -07:00
Yuri Weinstein
7a2ca3323b Merge pull request #15285 from smithfarm/wip-kill-fcgi 
build/ops: drop libfcgi build dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-05-30 08:42:55 -07:00
Jos Collin
1f32a9c5e1 test/old: Removed commented code 
There were lot of commented code inside src/test/old/testcrush.cc. This was found while doing
the PR#15364. Removed them.

Signed-off-by: Jos Collin <jcollin@redhat.com>
 2017-05-30 21:01:22 +05:30
Sage Weil
aefd56b450 Merge pull request #15359 from liewegas/wip-kraken-x 
qa/suites/upgrade/kraken-x: enable experimental for bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-05-30 09:38:31 -05:00