Commit Graph

91386 Commits

Author SHA1 Message Date
Erwan Velu
d8fdf0b753 ceph_volume: Adding Device.is_valid()
A block device can be filtered-out/ignored because it have features that
doesn't match Ceph's expectations.

As of today, the current code was rejected removable devices but it was
pretty hidden from the user, and implicit in the get_devices() function.

This patch is creating a new is_valid() function to perform all the
rejection tests and returns if this device can be used in the Ceph
context or not.

If is_valid() is returning False, the 'rejected_reasons' list reports all
the reasons why that devices got rejected.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-10-22 11:56:14 +02:00
Sage Weil
98fc7ebc99 Merge PR #24184 into master
* refs/pull/24184/head:
	mgr/DaemonServer: remove any upmaps on merging PGs
	mgr/DaemonServer: prevent merge if either pg is remapped|upmap
	mgr/DaemonServer: move pending merge check for more consistent code
	qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
	mgr/DaemonServer: add option to bypass careful throttling for thrasher
	PendingReleaseNotes: note about mgr/balancer/max_misplaced change
	mgr/DaemonServer: remove stale/misleading check
	mgr/DaemonServer: throttle pgp_num changes based on misplaced %
	mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
	mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
	mon/PGMap: add get_recovery_stats()
	mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
	pybind/mgr/mgr_module: add get_option()
	mgr/DaemonServer: allow pg_num increases that abort pending merges
	mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
	mon/OSDMonitor: make pgp_num track pg_num more consistently

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-20 16:40:22 -05:00
Sage Weil
0db59b6c09 Merge PR #24654 into master
* refs/pull/24654/head:
	osd: remove unused parameter 'dev' in OSD::mkfs function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-20 15:39:56 -05:00
Sage Weil
ba3679c60b mgr/DaemonServer: remove any upmaps on merging PGs
Remove any pg_upmap[_items] on pgs that are merging to ensure that they
land on the same OSDs.

This is a bit sloppy: we *could* set the source upmap to match the target
upmap (vs potentially moving both PGs to a third location, and/or then
having the balancer move the resulting PG somewhere else again), but for
now assume upmaps are not a common case and Keep It Simple.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
550dcd53bb mgr/DaemonServer: prevent merge if either pg is remapped|upmap
Remapping means they could be on different OSDs.

Fixes: http://tracker.ceph.com/issues/36166
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
5bb9820b2d mgr/DaemonServer: move pending merge check for more consistent code
No functional change, but this makes the code simpler to read.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
86ae8fb6b8 qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
Thrash such that we still exercise the careful throttling in the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
70ec5bda23 mgr/DaemonServer: add option to bypass careful throttling for thrasher
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Josh Durgin
ba3252544c
Merge pull request #20581 from chrone81/patch-1
doc: Fix EC k=3 m=2 profile overhead calculation example.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-19 15:23:55 -07:00
Gregory Farnum
c61acc5b31
Merge pull request #22349 from gregsfortytwo/wip-24368-osd-restarts
systemd: only restart 3 times in 30 minutes, as fast as possible

Reviewed-by:  Sage Weil <sage@redhat.com>
2018-10-19 13:00:07 -07:00
Mykola Golub
cb48963385
Merge pull request #24646 from dillaman/wip-36500
rbd-mirror: always attempt to restart canceled status update task

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-10-19 18:19:56 +03:00
Casey Bodley
61f7c3f7d7
Merge pull request #24670 from falcon78921/wip-doc-multisite-link
doc: updated wording from federated to multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-19 11:06:35 -04:00
Casey Bodley
9312c59174
Merge pull request #24595 from cbodley/wip-24505
rgw: user stats account for resharded buckets

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-10-19 10:00:42 -04:00
Jason Dillaman
d0c6c6be78
Merge pull request #24673 from tchaikov/wip-librbd-test-os-cleanup
librbd,test: remove unused context_cb() function, silence GCC warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-10-19 09:47:54 -04:00
Matt Benjamin
598185a95d
Merge pull request #24645 from cbodley/wip-36496
cls/user: cls_user_remove_bucket writes modified header
2018-10-19 09:39:03 -04:00
Sage Weil
0708e73c26 Merge PR #24611 into master
* refs/pull/24611/head:
	mgr/diskprediction: Solve predictor model file not backward compatibility

Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-19 08:31:46 -05:00
Sage Weil
a9b4ed6c01 Merge PR #24623 into master
* refs/pull/24623/head:
	mgr: Mutex::Locker -> std::lock_guard
	kv: Mutex::Locker -> std::lock_guard
	librados: Mutex::Locker -> std::lock_guard
	mon: Mutex::Locker -> std::lock_guard
	osd: Mutex::Locker -> std::lock_guard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2018-10-19 08:31:29 -05:00
Kefu Chai
ccb0489895
Merge pull request #24605 from tchaikov/wip-use-ccache-the-old-way
cmake: fix compilation with distcc and other compiler wrappers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-19 20:57:04 +08:00
Kefu Chai
80376d32be
Merge pull request #21155 from runsisi/wip-auth-cleanup
auth/cephx: minor code cleanup

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-19 19:21:17 +08:00
Tim Serong
d08f290bcb
Merge pull request #24672 from SUSE/wip-orchestrator-result-property
mgr/orchestrator: use result property in Completion classes
2018-10-19 21:19:10 +10:00
Lenz Grimmer
94aefee3b0
Merge pull request #24314 from rhcs-dashboard/dashboards
mgr/dashboard: Grafana dashboard updates and additions

Reviewed-by: Boris Ranto <branto@redhat.com>
2018-10-19 12:42:23 +02:00
Lenz Grimmer
203e3d2e60
Merge pull request #24628 from Devp00l/wip-issue-36445
mgr/dashboard: Add missing test requirement "werkzeug"

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2018-10-19 11:57:12 +02:00
Kefu Chai
3797d1f654
Merge pull request #24669 from falcon78921/wip-docs-36527
doc: add spacing to subcommand references

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-19 17:32:59 +08:00
Kefu Chai
39fcba396b
Merge pull request #19386 from ganeshmaharaj/qat-ceph-crypto
crypto: QAT based Encryption for RGW

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-19 17:02:56 +08:00
Kefu Chai
7200b16544 librbd: remove unused context_cb() function
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-19 16:33:42 +08:00
Kefu Chai
533064aa58 test/objectstore: silence "-Wsign-compare" warnings
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-19 16:33:42 +08:00
Tim Serong
76d3f0d537 mgr/orchestrator: use result property in Completion classes
The _Completion class defined a get_result() method, but the orchestrator
CLI and Rook module both just expected a result property, so let's run with
the property version.

Signed-off-by: Tim Serong <tserong@suse.com>
2018-10-19 19:28:04 +11:00
runsisi
f1d741c823 auth/cephx: remove unused member method
Signed-off-by: runsisi <runsisi@zte.com.cn>
2018-10-19 13:55:15 +08:00
runsisi
55ef6cf5d7 auth: remove unused forward declaration
Signed-off-by: runsisi <runsisi@zte.com.cn>
2018-10-19 13:48:36 +08:00
Jos Collin
6572cd9ee3
Merge pull request #24618 from CharlotteRose/wip-number-eviction-events
doc: numbered eviction situations

Reviewed-by: Jos Collin <jcollin@redhat.com>
2018-10-19 10:43:51 +05:30
Ganesh Maharaj Mahalingam
1193805185 crypto: Merge compilation of crypto accelerators under a single cmake.
- Control ISA-L and QAT under a crypto makefile
- Allow conditional compile of QAT.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2018-10-18 21:48:18 -07:00
Ganesh Maharaj Mahalingam
bf5f83decc crypto: QAT Wrapper for Ceph Encryption
- Crypto plugin created and currently handles only AES_256_CBC
- Contents
-- Poller thread for QAT HW
-- Contiguous memory allocator using QAT's usdm driver
-- Do in-place crypto operation reducing mem usage.
-- Enable multi-QAT instance to allow parallel operations.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2018-10-18 21:48:18 -07:00
James McClune
f0447b29ad
doc: updated wording from federated to multisite
Updated the terminology from federated architecture
to multisite configuration. Updated the reference link for multisite
configuration.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-10-19 00:45:33 -04:00
James McClune
8372e85f52
doc: add spacing to subcommand references
Added appropriate spacing for subcommand
code references.

Fixes: https://tracker.ceph.com/issues/36527

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-10-19 00:34:54 -04:00
Alfredo Deza
496f27eac2
Merge pull request #24531 from alfredodeza/wip-rm36386
ceph-volume remove version reporting from help menu

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-10-18 15:43:44 -03:00
Matt Benjamin
f79e95d5d5
Merge pull request #24453 from cbodley/wip-rgw-putobjprocessor
rgw: refactor PutObjProcessor stack
2018-10-18 13:09:00 -04:00
Gregory Farnum
f9d66bb080
Merge pull request #23304 from smithfarm/wip-23776
tools: ceph-authtool: report correct number of caps when creating keyring

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-10-18 09:10:16 -07:00
Scoots Hamilton
169824444e doc: numbered eviction situations
Signed-off-by: Scoots Hamilton <scoots@redhat.com>
2018-10-18 11:57:09 -04:00
Casey Bodley
bdd71142c0
Merge pull request #19571 from Fusl/patch-1
doc: doc/radosgw/s3.rst: Adding AWS S3 `Storage Class` as `Not Supported`

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-18 11:30:12 -04:00
Kefu Chai
d4b564480e
Merge pull request #24660 from tchaikov/wip-crimson-cleanup
crimson: crimson/mon: remove timeout support from mon::Client::authenticate()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-18 21:57:56 +08:00
jtlayton
dcac4a6a53
Merge pull request #24626 from jtlayton/wip-mgr-urlparse
mgr/rook: fix urljoin import
2018-10-18 08:32:17 -04:00
Jason Dillaman
f3c843c715
Merge pull request #24604 from yangdongsheng/token_fix
common/Throttle: TokenBucketThrottle: use reference to m_blockers.front()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-10-18 07:30:02 -04:00
Kefu Chai
671804a3e7
Merge pull request #24653 from mynaramana/claydoc
doc: erasure-code-clay fixes typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-18 12:37:56 +08:00
Kefu Chai
2358cf154c
Merge pull request #22162 from liuzhanhao/master
doc: Update cpp.rst to accommodate the new APIs in libs3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-18 12:29:07 +08:00
Yao Guotao
9aab195eaa osd: remove unused parameter 'dev' in OSD::mkfs function
Signed-off-by: Yao Guotao <yaoguot@gmail.com>
2018-10-18 10:35:39 +08:00
Myna V
105fbb658c doc: erasure-code-clay fixes typos
Signed-off-by: Myna <mynaramana@gmail.com>
2018-10-18 07:09:49 +05:30
Nathan Cutler
aef4fc8786 tests/ceph-authtool: add reproducer for "with 0 caps" issue
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-10-18 00:18:29 +02:00
Nathan Cutler
b8c78bd44e tools: ceph-authtool: report correct number of caps when creating
Fixes: https://tracker.ceph.com/issues/23776
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-10-18 00:13:14 +02:00
Jason Dillaman
d3363e6139 rbd-mirror: always attempt to restart canceled status update task
Fixes: http://tracker.ceph.com/issues/36500
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-17 17:21:16 -04:00
Casey Bodley
dac26de1fa cls/user: cls_user_remove_bucket writes modified header
if cls_user_remove_bucket() modifies the header with dec_header_stats(),
it should write the updated header back

Fixes: http://tracker.ceph.com/issues/36496

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-17 14:50:19 -04:00