Commit Graph

932 Commits

Author SHA1 Message Date
Sage Weil
09a8e5bce0 qa/suites/upgrade/mimic-x: add missing .qa
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 08:17:31 -06:00
Sage Weil
c7940db6b6 Merge PR #25596 into master
* refs/pull/25596/head:
	qa/suites/upgrade: fix wrt librados3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-18 07:24:03 -06:00
Sage Weil
5612b6714c qa/suites/upgrade: fix wrt librados3
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-17 13:25:34 -06:00
Sage Weil
dce1623db9 qa/rados/upgrade: align thrashing with upgrade suite, don't import/export pgs
Don't import/export between versions

Fixes: http://tracker.ceph.com/issues/37665
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-14 07:04:56 -06:00
Kefu Chai
1d973c1e90 qa: downgrade librados2,librbd1 for thrash-old-clients tests
librados2 and librbd1 are installed as a dependency of qemu-kvm.
qemu-kvm is installed by ceph-cm-ansible, see [1].

in thrash-old-clients, jewel packages are installed, but yum does
not allow downgrade unless it's required explicitly. in this change,
we downgrade librbd1 and librados2 to address this issue.

currently, the ceph packages shipped by CentOS/RHEL 7 are still an old
version of jewel. so this issue only kicks in when we try to install
hammer.

this change should address failures like

Command failed on smithi136 with status 1: '\n sudo yum -y install
rbd-fuse\n '

found in rados/thrash-old-clients tests.

---
[1]
3db1cbdc22 (diff-f2b05d775fedff6c5c6689f564b32f1c)

Fixes: http://tracker.ceph.com/issues/37618
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-13 10:49:37 +08:00
Stephan Müller
19b039c28e mgr/dashboard/qa: Fix ECP creation test
The current solution fails on our CI-system as some outputs can have
more values and some parameters like 'w' can vary in different
environments.

As this was only tested before in a vstart cluster environment it
worked.

Through this commit only the given attributes we know to be there,
will be tested.

Fixes: https://tracker.ceph.com/issues/37275
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-12-10 12:37:03 +01:00
Patrick Donnelly
4432aa5f26
Merge PR #24748 into master
* refs/pull/24748/head:
	qa: use 6h timeout for pjd test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-12-07 10:50:57 -08:00
Sage Weil
9ee3ce1ecd Merge PR #25345 into master
* refs/pull/25345/head:
	qa/suites: fix bluestore links
	qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-12-06 08:42:04 -06:00
Sage Weil
269910fc8b qa/suites: fix bluestore links
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-05 10:30:14 -06:00
Kefu Chai
105ca218ee qa/suites/rados/upgrade: set require-osd-release to nautilus
* add qa/releases/nautilus.yaml so it can be reused.
* use releases/nautilus.yaml in luminous-x upgrade test, so
  test_librbd_python.sh is able to use the feature introduced in
  nautilus.

Fixes: http://tracker.ceph.com/issues/37432
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-29 14:35:17 +08:00
Sage Weil
706197a7c7 Merge PR #25272 into master
* refs/pull/25272/head:
	qa: add simple test-volumes.sh workunit and run it from fs/basic_functional
	vstart.sh: create default fs via 'fs volume create'
	mgr/volumes: fix oremote
	mon/MgrMonitor: enable volumes module by default
	mgr: create `volumes` module
	mgr: cleaner constructor for CommandResult
	mgr: block for latest osdmap after command execution
	mgr: add MgrModule.mon_command helper
	ceph_volume_client: enable using existing rados inst
	mon: give ceph-mgr access to 'fs' commands

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-11-28 11:34:57 -06:00
Sage Weil
43bf12e12d qa: add simple test-volumes.sh workunit and run it from fs/basic_functional
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-28 08:54:29 -06:00
Lenz Grimmer
720e0d4bfd
Merge pull request #24900 from zmc/wip-minimal-health
mgr/dashboard: Replace dashboard service

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-11-28 10:56:40 +01:00
Zack Cerza
50b7d42fe5 mgr/dashboard: Replace dashboard service
This splits out the collection of health and log data from the
/api/dashboard/health controller into /api/health/{full,minimal} and
/api/logs/all.

/health/full contains all the data (minus logs) that /dashboard/health
did, whereas /health/minimal contains only what is needed for the health
component to function. /logs/all contains exactly what the logs portion
of /dashboard/health did.

By using /health/minimal, on a vstart cluster we pull ~1.4KB of data
every 5s, where we used to pull ~6KB; those numbers would get larger
with larger clusters. Once we split out log data, that will drop to
~0.4KB.

Fixes: http://tracker.ceph.com/issues/36675

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 16:08:53 -07:00
Patrick Donnelly
b76f14569d
Merge PR #24886 into master
* refs/pull/24886/head:
	qa: fix delay type config name

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-11-27 13:58:26 -08:00
Sage Weil
d69e8d8de8 Merge PR #14092 into master
* refs/pull/14092/head:
	mgr/DaemonServer: fix session leak
	mon/MonClient: ignore new mon commands while stopping
	mgr/DeviceState: fix DeviceState initial refcount
	qa/suites: valgrind ceph-mgr too

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-16 07:11:44 -06:00
Lenz Grimmer
34a5ac0b19
Merge pull request #25084 from s0nea/wip-dashboard-add-missing-test-suites
mgr/dashboard/qa: add missing dashboard suites

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2018-11-16 11:16:42 +01:00
Tatjana Dehler
ef62c5eb4e mgr/dashboard/qa: add missing dashboard suites
Fixes: https://tracker.ceph.com/issues/37152
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-11-15 11:45:56 +01:00
Kefu Chai
ae36a61446 qa/suites: add librados2 to "extra_packages" for upgrade tests
we use the playbook of "testnodes.yml" defined by ceph-cm-ansible for
initializing test nodes, and the role of "testnode" is used by
testnodes.yml. "testnode" requires "qemu-system-x86" or "qemu-kvm"
package to be installed. the qemu in turn depends on librbd1 and
librados2.

before librados3 was introduced, this worked perfectly. because in ceph
repo, qa/packages/packages.yaml defines the default set of packages the
"install" tasks should install. and in that yaml file, librados2 was
listed. so the package management system will overwrite the librados2
installed by ansible playbook with the version specified by the
"install" task, as apt/yum thinks this is what user requires explicitly,
so it's fine to install a different version of librados2.

after librados3 was introduced, librados2 was removed from
qa/packages/packages.yaml. because, by default, we need to install
librados3 instead of librados2 for ready a nautilus cluster. but the
problem is, the packge list also applies to "install" tasks installing
releases before nautilus, where we still need to replace the librados2
installed by ansible.

so, to address this issue, "librados2" is added to "extra_packages" of
the "install" tasks of tests installing old releases to install
librados2 explicitly instead of as a dependency of other ceph packages
like librbd1.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:56:24 +08:00
Patrick Donnelly
fdf6b38a49
qa: use 6h timeout for pjd test
Apparently needs this long for some configurations.

Fixes: https://tracker.ceph.com/issues/36594
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-13 12:57:49 -08:00
Kefu Chai
f10fc004bc qa: add librados3 to exclude_packages for ugprade tests
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-10 07:17:24 -08:00
Jason Dillaman
6ac41ccbaf
Merge pull request #24214 from iridescent-rsy/bursting_io
librbd: reduce the TokenBucket fill cycle and support bursting io configuration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 19:53:55 -05:00
Sage Weil
03908113b4 qa/suites: valgrind ceph-mgr too
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-09 08:52:07 -06:00
Kefu Chai
0966f43d98
Merge pull request #24896 from tchaikov/wip-librados-cleanup
librados,rpm,deb: various fixes to address librados3 transition and cleanups in librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-08 10:19:57 -08:00
Kefu Chai
0e1ec8dc20 qa: install libradospp-dev for librados_hello_world.yaml
libradospp-{dev,devel} is necessary for compiling sources in
examples/librados/hello_world.cc

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-07 14:13:48 -08:00
David Zafman
02964703de
Merge pull request #24749 from dzafman/wip-36474
Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-07 14:05:55 -08:00
Shiyang Ruan
fd613b1c32 librbd: make the minimum of schedule tick for QoS configurable.
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
2018-11-07 13:28:56 +08:00
Shiyang Ruan
761393a955 librbd: support burst limit configuration.
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
2018-11-07 13:25:53 +08:00
Patrick Donnelly
0e4ea92d19
qa: fix delay type config name
Setting as-is is a no-op.

Fixes: http://tracker.ceph.com/issues/36676

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-01 13:01:26 -07:00
Kefu Chai
b4a5f76243 qa: add librados3 to exclude list of pre-nautilus install tasks
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
David Zafman
3f621a1190 test: Set any value for osd_delete_sleep to guarantee we are testing even SSD
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-10-30 11:42:05 -07:00
Sage Weil
a350131d16 Merge PR #24698 into master
* refs/pull/24698/head:
	Merge PR #24697 into nautilus
	ceph_test_msgr: fix authorizer behavior
	Merge pull request #24667 from liewegas/wip-ec-thrash-full
	Merge PR #24689 into nautilus
	qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 22:46:23 -05:00
Sage Weil
ae583f5dde Merge PR #24689 into master
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
	Merge PR #24625 into nautilus
	qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:19:46 -05:00
Sage Weil
98fc7ebc99 Merge PR #24184 into master
* refs/pull/24184/head:
	mgr/DaemonServer: remove any upmaps on merging PGs
	mgr/DaemonServer: prevent merge if either pg is remapped|upmap
	mgr/DaemonServer: move pending merge check for more consistent code
	qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
	mgr/DaemonServer: add option to bypass careful throttling for thrasher
	PendingReleaseNotes: note about mgr/balancer/max_misplaced change
	mgr/DaemonServer: remove stale/misleading check
	mgr/DaemonServer: throttle pgp_num changes based on misplaced %
	mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
	mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
	mon/PGMap: add get_recovery_stats()
	mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
	pybind/mgr/mgr_module: add get_option()
	mgr/DaemonServer: allow pg_num increases that abort pending merges
	mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
	mon/OSDMonitor: make pgp_num track pg_num more consistently

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-20 16:40:22 -05:00
Sage Weil
86ae8fb6b8 qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
Thrash such that we still exercise the careful throttling in the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
c22d5a0fce qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection
For EC pools we have a lot of shards, and 30% probability on each one
means we are very like to repeatedly fail backfill reservations.. long
enough that teuthology gives up waiting.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-18 17:17:41 -05:00
Sage Weil
f8b6cbda34 Merge PR #24359 into master
* refs/pull/24359/head:
	qa/tests: update ansible version to 2.6 for master branch testing.
	qa/tests: use lvm as default for ceph-ansible testing, this should also work with raw devices

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-10-17 10:06:42 -05:00
Sage Weil
b833d35d9b qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'
Avoid failures like

"2018-10-16 20:36:00.437153 mgr.y (mgr.25609) 6 : cluster [SEC] foo bar security" in cluster log

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-17 07:09:15 -05:00
Sage Weil
9db328f2ab Merge PR #24204 into master
* refs/pull/24204/head:
	qa/suites/rgw/tempest: valgrind on centos only

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2018-10-12 16:29:44 -05:00
Jason Dillaman
c1f950236b qa/tasks: qemu task now uses a relative path in suite repo for test
This makes it easier to re-run tests against a suite branch without
requiring a full ceph-ci build and repo.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-11 20:23:39 -04:00
Ilya Dryomov
ccf5f07a0d
Merge pull request #24354 from idryomov/wip-krbd-kfsx-fancy-striping
qa/suites/krbd: more fsx tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-10-02 09:32:57 +02:00
Kefu Chai
7e90989214
Merge pull request #24291 from tchaikov/wip-23964
erasure-code: add clay codes

Reviewed-by: Myna Vajha <mynaramana@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-10-02 10:48:21 +08:00
Vasu Kulkarni
4c371daf99 qa/tests: update ansible version to 2.6 for master branch testing.
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-10-01 15:10:49 -07:00
Vasu Kulkarni
b0fc647e3f qa/tests: use lvm as default for ceph-ansible testing, this should also work with raw devices
and using the batch mode should auto select nvme/ssd for journal.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-10-01 12:05:50 -07:00
Ilya Dryomov
870e42ac6a qa/suites/krbd: more fsx tests
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-10-01 16:48:47 +02:00
Patrick Donnelly
bdd2ddcfd8
qa: use default timeout in fs workunits
Six hours is unnecessarily long.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-30 18:10:05 -07:00
Sage Weil
bc86b06a67 qa/suites/rados/thrash-erasure-code: add clay 4+2
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-27 18:56:21 +08:00
Nathan Cutler
a38af571c2
Merge pull request #23131 from smithfarm/wip-15100
qa: add test that builds example librados programs

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
2018-09-27 00:17:48 +02:00
Mykola Golub
78881934c0
Merge pull request #22043 from dillaman/wip-24139
rbd-mirror: bootstrap needs to handle local image id collision

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-26 22:49:02 +03:00
Casey Bodley
8622d8a241
Merge pull request #24011 from cbodley/wip-qa-rgw-multisite-speedup
qa/rgw: reduce number of multisite log shards

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-25 16:39:51 -04:00