Commit Graph

84958 Commits

Author SHA1 Message Date
Yehuda Sadeh
d05bf3ca97 rgw: better api for metadata mutation
Unlike the regular user and bucket metadata that are plain
raw rados objects, the otp info is controlled by object
class operations. The metadata manager mutate operation
will deal with the metadata related work that is needed
to happen (objv_tracker, update meta log), and call to
the operation that modifies the otp info.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
fbe7c01e00 rgw: tie otp meta get/put to meta log, objv_tracker
still need to have the radosgw-admin mfa tools to go through meta
instrumentation.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
45ee591d14 rgw: trivial implementation of otp meta handlers
Still not there yet. Need to tie into objv_tracker, and metadata log
tooling.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
0bf1fab731 cls/otp: client api to set multiple keys
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
6ec41bb95b cls/otp: update header anyway, don't use omap header
Update header even if it might have existed. If running within a compound op
we might not identify it correctly.
Also don't use omap header api.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
bba8c519c6 rgw: initial work to add otp as metadata provider
To allow transparent multisite sync

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
2d5d024f27 rgw: fix mfa setting, verify on object deletion
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
0c0046e259 rgw: S3 get bucket versioning returns MFA status
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
981f194355 rgw: functionality to modify mfa entries in user metadata
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
3ab463f9f5 rgw-admin: new mfa control commands
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
956997cff9 rgw: add otp pool, create otp service functionality
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
af36c08871 cls/otp: implement otp_info_t::dump()
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
63924ef625 cls_otp: treat empty header as -ENOENT
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
4fe9f176c0 rgw: send totp check to the otp objclass
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
f8ab2fc1fc rgw: req_state members init refactor
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
9fe808b683 cls/otp: add client interfaces
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
a774884baf cls/otp: implement totp check logic
Currently checking for bogus results, still need to integrate with
totp library.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
0c604eea56 cls/otp: initial work
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Yehuda Sadeh
c3a6f7588a rgw: mfa - initial work
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Alfredo Deza
9808407737
Merge pull request #20758 from ceph/wip-cv-doc-typos
ceph-volume doc updates

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2018-04-09 09:58:56 -04:00
Mykola Golub
2791d15455
Merge pull request #21249 from dillaman/wip-18844
rbd: improve 'import-diff' corrupt input error messages

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-09 16:22:03 +03:00
Alfredo Deza
d451a75c49
Merge pull request #21196 from jeffrey4l/fix_typo
ceph-volume: fix typo in ceph-volume lvm prepare help

Reviewed-by: Wido den Hollander <wido@42on.com> 
Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 07:02:50 -04:00
Lenz Grimmer
0d700d014f
Merge pull request #21270 from Devp00l/ceph-table-action-fix
mgr/dashboard: Fix the data table action selector

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-04-09 12:34:25 +02:00
Boris Ranto
f29f45e135
Merge pull request #21269 from b-ranto/wip-mgr-expose-rgw
mgr: Expose rgw perf counters

Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-09 10:40:42 +02:00
Mykola Golub
46df695b05
Merge pull request #21248 from dillaman/wip-23548
librbd: potential race between discard and writeback

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-08 23:22:38 +03:00
Nathan Cutler
fa5e8f9376
Merge pull request #17531 from smithfarm/wip-calamari-tests
tests: cleanup: drop calamari tasks

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
2018-04-08 08:54:27 +02:00
Xie Xingguo
1cdc9161ff
Merge pull request #21245 from ifed01/wip-ifed-shard-bug-logging
os/bluestore: be more verbose when hitting unloaded shard in extent map

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-04-08 13:37:00 +08:00
Kefu Chai
8d87f1c055
Merge pull request #20866 from jan--f/mgr-fix-metadata-return
mon/MonCommands: mgr metadata - improve parameter naming consistency

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-07 17:31:04 +08:00
Kefu Chai
4cfc09d1b5
Merge pull request #21217 from tchaikov/wip-cmp-hobject
common/hobject: preserve the order of hobject

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-07 17:29:57 +08:00
Kefu Chai
8c7f8bd433
Merge pull request #21211 from tchaikov/wip-clang-without-map-merge
cmake,mds: detect std::map::merge() before using it.

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-04-07 17:29:15 +08:00
Kefu Chai
9bfb1f6246
Merge pull request #21161 from cbodley/wip-mon-config-obs
config: notify config observers on set_mon_vals()

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-07 17:27:15 +08:00
Kefu Chai
9fda4d2b5e
Merge pull request #21151 from xiexingguo/wip-prune-olr
osd/PGBackend: drop input "snapid_t" from objects_list_range()

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-07 17:25:48 +08:00
Kefu Chai
6f6b70025b
Merge pull request #21150 from xiexingguo/wip-large-omap-obj-counting
osd/PGBackend: fix large_omap_objects checking

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
2018-04-07 17:24:56 +08:00
Kefu Chai
52012a069c
Merge pull request #21119 from Liuchang0812/refine-optracker-history
common, osd: duplicated "start" event in OpTracker, improve OpTracker::dump_ops

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-07 17:23:35 +08:00
Kefu Chai
008aefd723
Merge pull request #21095 from wido/ceph-metadata-version
common: Add metadata with only Ceph version number and release

Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-07 17:22:01 +08:00
Kefu Chai
7e8d490f7d
Merge pull request #21055 from tchaikov/wip-pr-20994
filestore,rgw: fix types/casts making clang on 32-Bit working

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-07 17:21:06 +08:00
Willem Jan Withagen
3263b90e1f mds: Clang does not (yet) support std::map::merge
@tchaikov found:
    libc++ still does not support "Splicing Maps and Sets",
    see https://libcxx.llvm.org/cxx1z_status.html ,
    search for "p0083r3" .

Make this case the more explicit coded.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-04-07 13:25:22 +08:00
Kefu Chai
2e45598d8a cmake: detect if std::map::merge() is supported
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-07 13:25:19 +08:00
Kefu Chai
9e840c4382
Merge pull request #21274 from dzafman/wip-cot-config
tools: Use --no-mon-config so ceph_objectstore_tool.py test doesn't hang

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-07 11:59:28 +08:00
Sage Weil
43978fc68d Merge PR #20760 into wip-sage-testing-20180407.023110
* refs/pull/20760/head:
	mon/ConfigMonitor: add 'config reset' command
	mon/ConfigMonitor: add 'config log' command
	mon/ConfigMap: add ConfigChangeSet
	mon/ConfigMonitor: avoid no-op updates
	mon/ConfigMonitor: maintain history of all config changes
2018-04-06 21:31:19 -05:00
Sage Weil
5d0c06d5fe Merge PR #20516 into wip-sage-testing-20180407.023003
* refs/pull/20516/head:
	os/filestore: advertise vdo in filestore metadata
	os/filestore: implement thin provisioning utilization stats for VDO
	os/bluestore/KernelDevice: advertise vdo and physical_size in metadata
	os/bluestore/KernelDevice: implement thin utilization stats for VDO
	os/bluestore: hook to expose utilization of thinly-provisioned device
	common/blkdev: identify VDO devices beneath LVM
	common/blkdev: implement VDO helper functions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-06 21:30:18 -05:00
Sage Weil
d7bca5d19f Merge PR #20512 into wip-sage-testing-20180407.022843
* refs/pull/20512/head:
	osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-06 21:29:32 -05:00
Sage Weil
aec47bc652 githubmap: update contributors
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 21:28:58 -05:00
Sage Weil
f67f3f87b7 Merge PR #21033 into wip-sage-testing-20180407.022623
* refs/pull/21033/head:
	buffer: kill push_front
	tools/RadosDump: avoid bufferlist::push_front
	os/filestore/FileJournal: avoid push_front
	erasure-code: avoid bufferlist::push_front
	buffer: kill claim_prepend
	unittest_compression: avoid claim_prepend
	os/bluestore: avoid claim_prepend
	osdc/Striper: rewrite assemble_result without push_front or claim_prepend

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
2018-04-06 21:27:20 -05:00
Sage Weil
f38bb6577d githubmap: update contributors
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 21:27:19 -05:00
Nathan Cutler
bdb674639d
Merge pull request #19594 from smithfarm/wip-doc-make-check
doc: expand developer documentation of unit tests

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2018-04-06 22:19:03 +02:00
David Zafman
a8d26122dc tools: Use --no-mon-config so ceph_objectstore_tool.py test doesn't hang
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-06 11:52:10 -07:00
Gregory Farnum
6d2e4c9b7b
Merge pull request #19973 from liewegas/wip-peering-fast-dispatch
osd: fast dispatch of peering events and pg_map + osd sharded wq refactor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-04-06 11:48:11 -07:00
Sage Weil
577737d007 osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max
The _max isn't used.  Drop the _min suffix.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 11:00:14 -05:00
Sage Weil
8f8f65594b
Merge pull request #21250 from liewegas/wip-recovery-slow-ops
qa/suites/rados/singleton/all/recovery_preemption: whitelist SLOW_OPS
2018-04-06 10:59:23 -05:00