Commit Graph

4300 Commits

Author SHA1 Message Date
Lenz Grimmer
720e0d4bfd
Merge pull request #24900 from zmc/wip-minimal-health
mgr/dashboard: Replace dashboard service

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-11-28 10:56:40 +01:00
Zack Cerza
50b7d42fe5 mgr/dashboard: Replace dashboard service
This splits out the collection of health and log data from the
/api/dashboard/health controller into /api/health/{full,minimal} and
/api/logs/all.

/health/full contains all the data (minus logs) that /dashboard/health
did, whereas /health/minimal contains only what is needed for the health
component to function. /logs/all contains exactly what the logs portion
of /dashboard/health did.

By using /health/minimal, on a vstart cluster we pull ~1.4KB of data
every 5s, where we used to pull ~6KB; those numbers would get larger
with larger clusters. Once we split out log data, that will drop to
~0.4KB.

Fixes: http://tracker.ceph.com/issues/36675

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 16:08:53 -07:00
Patrick Donnelly
b76f14569d
Merge PR #24886 into master
* refs/pull/24886/head:
	qa: fix delay type config name

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-11-27 13:58:26 -08:00
Zack Cerza
a90b5ac7c7 Allow specifying a schema for unknown JObj keys
Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 12:47:26 -07:00
Lenz Grimmer
55e2ee34f0
Merge pull request #25235 from sebastian-philipp/dashboard-fix-type-annotation-error
qa/mgr/dashboard: Fix type annotation error.

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-11-27 18:19:27 +01:00
Kefu Chai
9c0141caae
Merge pull request #25268 from smithfarm/wip-37392
qa/workunits/rados/test_librados_build.sh: grab files from explicit git branch

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-27 23:22:51 +08:00
Casey Bodley
b54b1db0f4
Merge pull request #25073 from oshynsong/master
rgw: add list user admin OP API

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-27 09:08:01 -05:00
Sebastian Wagner
4fe1344696 qa/mgr/dashboard: Fix type annotation error.
Also removed unused import

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-11-27 11:45:22 +01:00
Nathan Cutler
6c6ae9e30d qa/workunits/rados/test_librados_build.sh: grab files from explicit git branch
Fixes: https://tracker.ceph.com/issues/37392
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-27 00:34:17 +01:00
Sage Weil
630a36a509 Merge PR #24798 into master
* refs/pull/24798/head:
	fix python collections module warning for v3.7 and above

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-26 11:25:19 -06:00
Sage Weil
825930b79b Merge PR #17526 into master
* refs/pull/17526/head:
	qa/tasks/ceph_manager: avoid test_map_discontinuity stall with too few up osds

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2018-11-25 21:21:02 -06:00
Sage Weil
c9d275a0e2 Merge PR #21444 into master
* refs/pull/21444/head:
	qa: Replace 'ceph' with cluster name in restart()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-25 21:20:44 -06:00
Lenz Grimmer
60d4fa608e
Merge pull request #25221 from tspmelo/wip-37354
mgr/dashboard: Fix test_remove_not_expired_trash qa test

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-11-23 13:03:45 +01:00
Lenz Grimmer
dc8237e94d
Merge pull request #24757 from votdev/issue_36480
mgr/dashboard: Can't handle user editing when tenants are specified

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-11-23 12:36:26 +01:00
Tiago Melo
0f6e8e5f3d mgr/dashboard: Fix test_remove_not_expired_trash qa test
Fixes: http://tracker.ceph.com/issues/37354

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-11-22 11:39:24 +00:00
songshuangyang
b7580932c0 rgw: add list user admin OP API unit tests
Signed-off-by: Oshyn Song <dualyangsong@gmail.com>
2018-11-21 20:39:37 +08:00
Sage Weil
2ec2c5ef9e Merge PR #24795 into master
* refs/pull/24795/head:
	qa/tasks/ceph: gather crash dumps
2018-11-20 17:34:54 -06:00
Sage Weil
28aaca58e7 qa/tasks/ceph_manager: avoid test_map_discontinuity stall with too few up osds
Some tests have m=2,k=2 and this will break them.  Sometimes even if we
have 5 up osds, we end up with 4 and CRUSH gets picky, so build in a
buffer and only do this if we have 6 up.

We don't have an easy way from here to see what the min up osds for healthy
is...  basically this map discontinuity test just sucks.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-20 17:12:43 -06:00
Lenz Grimmer
6e95664ead
Merge pull request #25149 from p-na/pna-fix-safe-to-destroy
mgr/dashboard: Fix failing QA test: test_safe_to_destroy

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-11-20 14:33:42 +01:00
Sage Weil
4c0632df91 qa/tasks/ceph: gather crash dumps
Gather crash dumps as part of the teuthology run.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-20 06:49:20 -06:00
Volker Theile
945e790cab mgr/dashboard: Can't handle user editing when tenants are specified
Fixes: https://tracker.ceph.com/issues/36480

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-11-20 09:07:33 +01:00
Sage Weil
ba49aacee2 Merge PR #25111 into master
* refs/pull/25111/head:
	test: Add test for requested scrub priority
	osd: Prioritize user specified scrubs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-19 13:15:21 -06:00
Patrick Nawracay
09f2bb4c9a mgr/dashboard: Fix failing QA test: test_safe_to_destroy
The behavior of `safe-to-destroy` has changed in
432f194355 (PR#24799) and the backend
needs to be adapted accordingly.

Fixes: http://tracker.ceph.com/issues/37290

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-11-19 14:02:14 +01:00
Shilpa Jagannath
a7e0c4c8ca qa: Replace 'ceph' with cluster name in restart()
Enabled ctx.managers to take cluster name from config in restart() method instead of default 'ceph'.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2018-11-19 12:10:27 +05:30
Sage Weil
d69e8d8de8 Merge PR #14092 into master
* refs/pull/14092/head:
	mgr/DaemonServer: fix session leak
	mon/MonClient: ignore new mon commands while stopping
	mgr/DeviceState: fix DeviceState initial refcount
	qa/suites: valgrind ceph-mgr too

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-16 07:11:44 -06:00
Lenz Grimmer
34a5ac0b19
Merge pull request #25084 from s0nea/wip-dashboard-add-missing-test-suites
mgr/dashboard/qa: add missing dashboard suites

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2018-11-16 11:16:42 +01:00
Sage Weil
13d1f62538 Merge PR #24973 into master
* refs/pull/24973/head:
	mgr: Separate diskprediction cloud plugin from the diskprediction plugin
2018-11-16 00:23:14 -06:00
Sage Weil
d941b28089 Merge PR #19987 into master
* refs/pull/19987/head:
	qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-16 00:22:17 -06:00
hsiang41
9799eb67eb mgr: Separate diskprediction cloud plugin from the diskprediction plugin
Separate diskprediction local cloud from the diskprediction plugin.
Devicehealth invoke device prediction function related on the global
configuration "device_failure_prediction_mode".

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
2018-11-16 00:15:41 -06:00
Tatjana Dehler
ef62c5eb4e mgr/dashboard/qa: add missing dashboard suites
Fixes: https://tracker.ceph.com/issues/37152
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-11-15 11:45:56 +01:00
David Zafman
1841928e28 test: Add test for requested scrub priority
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-14 23:57:20 -08:00
Kefu Chai
a04691031c
Merge pull request #24858 from jecluis/wip-pr-19983-tests
qa/cephtool: test bounds on pool's `hit_set_*`

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-15 14:12:09 +08:00
Kefu Chai
60dc1c2a32 qa: patch rocksdb the right way
quote from patch(1)

>       but usually just
>
>       patch -pnum <patchfile

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 13:03:00 +08:00
Kefu Chai
2dc0f86227 qa: use FOUND_VAR to be backward compatible with cmake 2.8.12
before this change, we assume that the variable set if rados::radospp is
found will be radospp_FOUND, but this is a feature cmake 3, see
https://cmake.org/cmake/help/v3.3/module/FindPackageHandleStandardArgs.html

while the cmake shipped by centos is cmake 2.8.12, where the variable
name will be <UPPERCASED_NAME>_FOUND, see
https://cmake.org/cmake/help/v2.8.12/cmake.html#module:FindPackageHandleStandardArgs

in the test of test_envlibrados_for_rocksdb.sh, we are using cmake not
the cmake3 offered by EPEL7, so RADOSPP_FOUND will be set instead. that's why
executable env_librados_test will fail to link against rados::radospp.
as rados::radospp won't be defined if radospp_FOUND is not defined/set.

after this change, the 2nd mode of FIND_PACKAGE_HANDLE_STANDARD_ARGS()
is used instead to ensure that radospp_FOUND is defined even if cmake
2.8.12 is used.

also, the message() commands for debugging purpose are removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:57:52 +08:00
Kefu Chai
ae36a61446 qa/suites: add librados2 to "extra_packages" for upgrade tests
we use the playbook of "testnodes.yml" defined by ceph-cm-ansible for
initializing test nodes, and the role of "testnode" is used by
testnodes.yml. "testnode" requires "qemu-system-x86" or "qemu-kvm"
package to be installed. the qemu in turn depends on librbd1 and
librados2.

before librados3 was introduced, this worked perfectly. because in ceph
repo, qa/packages/packages.yaml defines the default set of packages the
"install" tasks should install. and in that yaml file, librados2 was
listed. so the package management system will overwrite the librados2
installed by ansible playbook with the version specified by the
"install" task, as apt/yum thinks this is what user requires explicitly,
so it's fine to install a different version of librados2.

after librados3 was introduced, librados2 was removed from
qa/packages/packages.yaml. because, by default, we need to install
librados3 instead of librados2 for ready a nautilus cluster. but the
problem is, the packge list also applies to "install" tasks installing
releases before nautilus, where we still need to replace the librados2
installed by ansible.

so, to address this issue, "librados2" is added to "extra_packages" of
the "install" tasks of tests installing old releases to install
librados2 explicitly instead of as a dependency of other ceph packages
like librbd1.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:56:24 +08:00
Nathan Cutler
0185abfeb4 tests: make ceph-admin-commands.sh log what it does
Fixes: http://tracker.ceph.com/issues/37089
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-13 19:46:24 +01:00
Kefu Chai
c89e0715cb
Merge pull request #25037 from tchaikov/wip-qa-upgrade
qa: add librados3 to exclude_packages for ugprade tests

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-13 06:38:21 +08:00
Yuri Weinstein
5127d0166e qa/tests: added "-n 7" to make sure mimic-x runs on built master branch
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-10 10:18:49 -08:00
Kefu Chai
f10fc004bc qa: add librados3 to exclude_packages for ugprade tests
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-10 07:17:24 -08:00
Jason Dillaman
6ac41ccbaf
Merge pull request #24214 from iridescent-rsy/bursting_io
librbd: reduce the TokenBucket fill cycle and support bursting io configuration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 19:53:55 -05:00
Josh Durgin
fd2a4c5733
Merge pull request #22476 from dzafman/wip-23875
Removal of snapshot with corrupt replica crashes osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-09 15:15:01 -08:00
Lenz Grimmer
5b0a957961
Merge pull request #24627 from Devp00l/wip-manage-ec-profiles
mgr/dashboard: Adds ECP management to the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 16:29:43 +01:00
Sage Weil
03908113b4 qa/suites: valgrind ceph-mgr too
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-09 08:52:07 -06:00
Lenz Grimmer
3ba874004e
Merge pull request #24489 from ricardoasmarques/wip-saml2
mgr/dashboard: SSO - SAML 2.0 support

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 13:11:22 +01:00
Stephan Müller
3a7f85809a mgr/dashboard: Adds ECP info endpoint
The new info endpoint will provide the frontend with the necessary
information it needs to create new profiles.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-11-09 09:40:39 +01:00
David Zafman
a159f162c5 test: osd-scrub-snaps.sh: After snapshot removal wait for snaptrim to complete
Due to deliberate corruptions snaptrim_error means snaptrim is done

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:20 -08:00
David Zafman
e37f95ac27 test: osd-scrub-snaps.sh: Testing with new --rmtype in ceph-objectstore-tool
Use --rmtype snapmap with new obj16 to remove snapmap only, check for repair message
Use --rmtype nosnapmap to remove obj5 while leaving snapmap behind

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:20 -08:00
David Zafman
f43faf4ad7 test: cleanup: Remove redundant cat of log and handle errors in create_scenario()
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:19 -08:00
Sage Weil
c8a8dc21fd Merge PR #24828 into master
* refs/pull/24828/head:
	qa/osd-bluefs-volume-ops: use ceph-bluestore-tool for fsck
	qa/osd-bluefs-volume-ops: reduce space usage for the test case

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-11-08 16:26:52 -06:00
Kefu Chai
0966f43d98
Merge pull request #24896 from tchaikov/wip-librados-cleanup
librados,rpm,deb: various fixes to address librados3 transition and cleanups in librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-08 10:19:57 -08:00