Commit Graph

4359 Commits

Author SHA1 Message Date
Kefu Chai
a0352c83fe
Merge pull request #25653 from rhcs-dashboard/37717-update-health-test
mgr/dashboard: updated API test

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-31 22:30:13 +08:00
alfonsomthd
4ac2e3f864 mgr/dashboard: updated health controller & API test
Updated health controller & test to reflect changes introduced
in 'df' payload.

Return 'total_used_raw_bytes' instead of 'total_used_bytes'
to match CLI 'bin/rados df' used/avail summary in
Landing Page (frontend component).

Do not return 'stats_by_class' to save bandwidth as they are
not needed (right now) in the dashboard.

Fixes: https://tracker.ceph.com/issues/37717

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-24 14:45:34 +01:00
Kefu Chai
94cb394d62
Merge pull request #25675 from dzafman/wip-36525
test: Disable duplicate request command test during scrub testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-24 00:16:00 +08:00
Sage Weil
f3e4e56fa5 Merge PR #25618 into master
* refs/pull/25618/head:
	mgr/dashboard: Updated API pool tests

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-22 16:04:55 -06:00
David Zafman
554ea73cb5 test: Disable duplicate request command test during scrub testing
Scrub testing requires an orderly control of scrubbing.  Most but not
all the time, the duplicate scrub request is ignored because the first
request hasn't finished.  Teuthology enables this environment variable
in the workunit handling.

Fixes: https://tracker.ceph.com/issues/36525

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-21 18:28:23 -08:00
Sebastian Wagner
933b2cfc28 mgr/orchestrator: Add test orchestrator
1. To be able to run the cli without an external orchestrator.
2. Run the CLI in Teuthology.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-12-20 10:56:49 +01:00
Sage Weil
9f3cf00b79 Merge PR #25360 into master
* refs/pull/25360/head:
	qa/workunits/mon/pg_autoscaler: clean up pools afterwards
	qa/suites/rados/singletone/all/pg-autoscaler: whitelist health warnings
	qa/tasks/ceph: wait for splits/merges before final scrub
	mon/OSDMonitor: be tidy with target_size_ratio and pre-nautilus code
	mgr/pg_autoscaler: simplify conditions
	qa/suites/rados: add simple pg-autoscaler test
	qa/workunits/cephtool/test.sh: pg_autoscale_mode=off while testing pg_num etc
	doc/rados/operations: document autoscaler and its health warnings
	mgr/pg_autoscaler: add pg autoscaler module
	pybind/mgr/mgr_util: move format_ helpers out of status module
	mon/OSDMonitor: accept optional target_size_{bytes,ratio} to 'osd pool create'
	mon/OSDMonitor: remove max_split_count configurable
	osd/osd_types: pool_opts_t: int -> int64_t
	osd/osd_types: pool_opts: fix whitespace
	osd/osd_types: pool_opts_t: make encoding feature-dependent
	mgr/devicehealth: pg_num_min 1 for device_health_metrics pool
	mon/OSDMonitor: accept optional pg_num_min to 'osd pool create'
	mon/OSDMonitor: apply osd_pool_default_pg_autoscale_mode to new pools
	pybind/mgr/mgr_module: some accessors
	mon/MgrMonitor: enable progress module by default
	osd/osd_types: add pool pg_autoscale_mode, pg_num_min, target_size_{bytes,ratio} properties
	osdc/Objecter: revise get_latest_version locking
	os/memstore: ignore OP_COLL_SET_BITS
	qa: generalise REQUIRE_MEMSTORE
	mgr: drop GIL in get_config
	mon: add 'size' arg to `osd pool create`
	mon: use pg_num_target for checks during creation
	mgr: revise locking in getter paths
	common/options: add `mon_target_pg_per_osd`
	mgr: expose OSDMap.pool_raw_used_rate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-12-19 21:22:35 -06:00
Sage Weil
b255e49259 qa/workunits/mon/pg_autoscaler: clean up pools afterwards
This avoids a huge pg merge from 100s to 4, which takes a long time and
makes the teuthology scrub cleanup time out.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-19 14:37:01 -06:00
Sage Weil
b8d45b262c qa/suites/rados/singletone/all/pg-autoscaler: whitelist health warnings
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-19 14:37:01 -06:00
Sage Weil
6f06e394e4 qa/tasks/ceph: wait for splits/merges before final scrub
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-19 14:37:01 -06:00
Sage Weil
2cd1ca6625 qa/suites/rados: add simple pg-autoscaler test
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 13:30:54 -06:00
Sage Weil
87828e5cdd qa/workunits/cephtool/test.sh: pg_autoscale_mode=off while testing pg_num etc
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 13:30:54 -06:00
Sage Weil
4d3407ba93 mon/OSDMonitor: remove max_split_count configurable
This isn't really relevant or useful now that the mgr is throttling the
actual pg_num adjustment based on pg_num_target, % misplaced, etc.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 13:30:54 -06:00
John Spray
e5fb5a1ddd qa: generalise REQUIRE_MEMSTORE
Move it up into CephTestCase so that mgr tests can
use it too, and pick it up in vstart_runner.py so
that these tests will work neatly there.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-12-18 13:30:54 -06:00
Sage Weil
20caf99856 Merge PR #25613 into master
* refs/pull/25613/head:
	qa/suites/upgrade/mimic-x: add missing .qa

Reviewed-by: João Eduardo Luís <joao@suse.de>
2018-12-18 12:21:25 -06:00
alfonsomthd
5822effd1f mgr/dashboard: Updated API pool tests
Updated API pool tests
as stats are enabled by default when retrieving the pool list
since:
https://github.com/ceph/ceph/pull/25489

Fixes: https://tracker.ceph.com/issues/37704

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-18 18:13:42 +01:00
Sage Weil
81160bc202 Merge PR #25190 into master
* refs/pull/25190/head:
	mgr/prometheus: adjust to new 'df' fields
	mon/Monitor: fix newline between df section
	doc: update docs for new ceph df output
	mon/PGMap: break down RAW usage by device class
	mon/PGMap: tweak df headers
	mon/PGMap: GLOBAL -> RAW STORAGE in 'df' output
	mon/PGMap: dump_fs_stats -> dump_cluster_stats

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-18 10:47:52 -06:00
Sage Weil
09a8e5bce0 qa/suites/upgrade/mimic-x: add missing .qa
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 08:17:31 -06:00
Sage Weil
c7940db6b6 Merge PR #25596 into master
* refs/pull/25596/head:
	qa/suites/upgrade: fix wrt librados3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-18 07:24:03 -06:00
Sage Weil
5612b6714c qa/suites/upgrade: fix wrt librados3
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-17 13:25:34 -06:00
David Zafman
278fac1141
Merge pull request #25483 from dzafman/wip-hashinfo
ceph-objectstore-tool: Dump hashinfo

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-12-17 09:38:42 -08:00
Sage Weil
f9ab61209b Merge PR #25564 into master
* refs/pull/25564/head:
	qa/rados/upgrade: align thrashing with upgrade suite, don't import/export pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-12-17 08:20:11 -06:00
Mykola Golub
b29b4f0c71 qa/workunits/rbd: add trash move/restore mirror test
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-12-14 16:47:15 +00:00
Sage Weil
dce1623db9 qa/rados/upgrade: align thrashing with upgrade suite, don't import/export pgs
Don't import/export between versions

Fixes: http://tracker.ceph.com/issues/37665
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-14 07:04:56 -06:00
Kefu Chai
1d973c1e90 qa: downgrade librados2,librbd1 for thrash-old-clients tests
librados2 and librbd1 are installed as a dependency of qemu-kvm.
qemu-kvm is installed by ceph-cm-ansible, see [1].

in thrash-old-clients, jewel packages are installed, but yum does
not allow downgrade unless it's required explicitly. in this change,
we downgrade librbd1 and librados2 to address this issue.

currently, the ceph packages shipped by CentOS/RHEL 7 are still an old
version of jewel. so this issue only kicks in when we try to install
hammer.

this change should address failures like

Command failed on smithi136 with status 1: '\n sudo yum -y install
rbd-fuse\n '

found in rados/thrash-old-clients tests.

---
[1]
3db1cbdc22 (diff-f2b05d775fedff6c5c6689f564b32f1c)

Fixes: http://tracker.ceph.com/issues/37618
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-13 10:49:37 +08:00
David Zafman
975dbc5841 test: Minor improvement to create_ec_pool()
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-10 20:16:01 -08:00
Kefu Chai
3a83bb2ed5
Merge pull request #25298 from smanjara/wip-rgw-import
qa/rgw: add missing import line

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-11 09:35:22 +08:00
Lenz Grimmer
2dd184346a
Merge pull request #25120 from Devp00l/wip-ecp-api-test-failure
mgr/dashboard/qa: Fix ECP creation test

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-10 17:25:10 +01:00
Lenz Grimmer
a8d23bc970
Merge pull request #25363 from zmc/wip-fix-dash-ecp-test
mgr/dashboard: Fix a broken ECP controller test

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-10 16:19:05 +01:00
Stephan Müller
19b039c28e mgr/dashboard/qa: Fix ECP creation test
The current solution fails on our CI-system as some outputs can have
more values and some parameters like 'w' can vary in different
environments.

As this was only tested before in a vstart cluster environment it
worked.

Through this commit only the given attributes we know to be there,
will be tested.

Fixes: https://tracker.ceph.com/issues/37275
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-12-10 12:37:03 +01:00
Sage Weil
738789b057 mon/PGMap: GLOBAL -> RAW STORAGE in 'df' output
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-07 13:28:07 -06:00
Patrick Donnelly
9789f37fa5
Merge PR #24940 into master
* refs/pull/24940/head:
	qa: add test for getfattr ceph.dir.pin
	client: support getfattr ceph.dir.pin extended attribute

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-12-07 10:53:44 -08:00
Patrick Donnelly
4432aa5f26
Merge PR #24748 into master
* refs/pull/24748/head:
	qa: use 6h timeout for pjd test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-12-07 10:50:57 -08:00
Igor Fedotov
1991b70d3a qa/mgr: rename raw_bytes_used field reference.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
79fd227639 qa: replace raw_bytes_used field access in QA test cases
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
25664c4ccf test: add test coverage for rados df reports
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Sage Weil
9ee3ce1ecd Merge PR #25345 into master
* refs/pull/25345/head:
	qa/suites: fix bluestore links
	qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-12-06 08:42:04 -06:00
Volker Theile
ec3cae8fed
Merge pull request #24968 from s0nea/wip-dashboard-osd-recv-speed-profiles
mgr/dashboard: add profiles to set cluster's rebuild performance

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
2018-12-06 13:00:30 +01:00
Jason Dillaman
6db4c152a6
Merge pull request #25266 from trociny/wip-24608
rbd-ggate: support namespaces

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-05 20:25:54 -05:00
Jason Dillaman
4bc7e0aa84
Merge pull request #25260 from trociny/wip-24609
rbd-nbd: support namespaces

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-05 13:24:36 -05:00
Sage Weil
269910fc8b qa/suites: fix bluestore links
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-05 10:30:14 -06:00
Sage Weil
f390704b46 qa/objectstore: bluestore -> bluestore-{bitmap,stupid}
The new default is bitmap, so we were testing bitmap twice.  Instead,
explicitly call out stupid and bitmap cases so a future default change
won't break coverage.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-04 14:05:49 -06:00
Tatjana Dehler
f71e4f15e5 mgr/dashboard: add config option bulk_set
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-12-04 15:15:19 +01:00
Casey Bodley
70242bf7b5
Merge pull request #25305 from cbodley/wip-37440
qa/rgw: fix invalid syntax error in radosgw_admin_rest.py

Reviewed-by: Volker Theile <vtheile@suse.com>
2018-12-03 09:00:39 -05:00
Zack Cerza
b22d8106d1 mgr/dashboard: Fix a broken ECP controller test
The change in b3e69a9609 broke the test's assumption that the endpoint
wouldn't be readable by block-manager. It doesn't looks as though that's
actually problematic for the ECP controller, so just update the test to
use rgw-manager instead.

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-30 18:00:03 -07:00
Sage Weil
5f7e083678 Merge PR #25308 into master
* refs/pull/25308/head:
	osd/OSD: OSD::mkfs asserts when reusing disk with existing superblock.
	os/bluestore: add main device expand capability.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-30 13:19:42 -06:00
Casey Bodley
2217c0f018 qa/rgw: radosgw_admin_rest.py uses /admin/user?list for user listing
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-11-30 12:22:23 -05:00
Kefu Chai
8f98c6e5e2
Merge pull request #25314 from tchaikov/wip-37432
qa/suites/rados/upgrade: set require-osd-release to nautilus

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-11-30 01:49:40 +08:00
James Page
20aa3ad17b Correct usage of collections.abc
Some classes should still be imported directly from collections;
only OrderedDict, Iterable and Callable (in the context of the
ceph codebase) are found in collections.abc.

The current code works due to the fallback support for Python 2.

Signed-off-by: James Page <james.page@ubuntu.com>
2018-11-29 09:48:43 +00:00
Igor Fedotov
d07c10dfc0 os/bluestore: add main device expand capability.
One can do that via ceph-bluestore-tool's bluefs-bdev-expand command

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-11-29 12:48:20 +03:00