Commit Graph

1579 Commits

Author SHA1 Message Date
David Zafman
02686e4cc8
Merge pull request #35514 from dzafman/wip-41564
mon: Warn when too many reads are repaired on an OSD

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-06-17 10:18:04 -07:00
David Zafman
661996d434 mgr: Warn when too many reads are repaired on an OSD
Include test case
Configurable by setting mon_osd_warn_num_repaired (default 10)
Ignore new health warning with random eio injection test

Fixes: https://tracker.ceph.com/issues/41564

Signed-off-by: David Zafman <dzafman@redhat.com>
2020-06-16 17:45:27 -07:00
Kefu Chai
f1072fcadc qa/suites/krbd: whitelist MON_DOWN health warning
see also

- 93de19adcf
- 608e002195

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-11 08:36:45 +08:00
J. Eric Ivancich
94f53f513c
Merge pull request #33492 from prasakri/multisite-sync-check
RGW:Multisite: Verify if the synced object is identical to source

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-06-08 19:49:46 -04:00
Sebastian Wagner
115492f28f
Merge pull request #35069 from sebastian-philipp/qa-cephadm-add-rgw-and-iscsi
qa/cephadm: Add RGW

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-06-03 09:50:04 +02:00
Neha Ojha
7cd88b31e8 qa/suites/rgw/multisite, qa/suites/rgw/crypt: whitelist PG_AVAILABILITY
This follows b162541ac2.
The balancer was turned on by default in
d4fbaf7, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.

Fixes: https://tracker.ceph.com/issues/45802
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-06-02 17:15:22 +00:00
Kefu Chai
8f5c832915 qa/tasks/tempest: use user/pass to authenticate
instead of using admin token use "admin" user to authenticate,
as admin token is not suggested anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
739751ef7a qa/suites/rgw/tempest: update unsupported tests of tempest
after rerunning tempest with lastest radosgw, remove the supported
tests from the blacklist, and add the ones which are not supported
yet. now we can pass 123 tests in total.

also enable discoverity for better testing coverage, since it's
supported now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
a53c69943c qa/tasks/keystone: use "keystone-manage bootstrap"
* qa/tasks/keystone.py:
  instead of prefilling keystone manually, use "keystone-manage bootstrap"
  instead. it helps to setup the admin user, a "Default" domain with
  "default" id, and wire them up with the expected role and a "admin" project,
  etc. as id of the admin domain is known to be "default", we can just use it
  in our tests without querying openstack for the id of "Default"
  domain. this is very handy.
* qa/suites/rgw/tempest/tasks/rgw_tempest.yaml:
  use "Default" for domain name. as "Default" is the name of the domain
  created by bootstrap, while "default" is its id.
* qa/suites/rgw/crypt/2-kms/barbican.yaml:
  remove settings to bootstrap keystone

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
8cb3f0411d qa/suites/rgw/tempest: use the latest tempest supporting py3.5
in case we need to use ubuntu xenial for testing, xenial only had python
3.5 packaged. and tempest 23.0 was the last version which supports
python3.5 and python2.7.

also do not replace link in tox.ini, as it is reachable.

to address the issues of

- pallets/markupsafe#116
- pypa/setuptools#2017

MarkupSafe is installed by
https://opendev.org/openstack/requirements/raw/branch/stable/pike/upper-constraints.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
9bd3e0ff40 qa/suites/rgw/tempest: bump up keystone to 17.0.0
* also generate a sample conf file following the document at
  https://github.com/openstack/keystone/tree/17.0.0.0rc2/etc
* use "projects" instead of "tenants" to match the terminology used by
  openstack identify API 3.0.
* test API 3.0 instead of API 2.0, by changing
  `rgw_keystone_api_version` from "2" to "3"
* explicitly specify a domain "default" for project to be created,
  otherwise a POST request will fail with:
```
{"error":{"code":400,"message":"You have tried to create a resource using the admin token. As this token is not within a domain you must explicitly include a domain for this resource to belong
to.","title":"Bad Request"}}
````
* create "default" domain, and use it, othewise a GET request fails
  like:
```
2020-05-28T11:17:28.751 INFO:teuthology.orchestra.run.smithi092.stderr:http://smithi092.front.sepia.ceph.com:35357 "GET /v3/domains/default HTTP/1.1" 404 87
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP: [404] Content-Length: 87 Content-Type: application/json Date: Thu, 28 May 2020 11:17:28 GMT Server: WSGIServer/0.2
CPython/3.6.9 Vary: X-Auth-Token x-openstack-request-id: req-bc33796f-2bc3-411c-a7fb-1208918e0dbd
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP BODY: {"error":{"code":404,"message":"Could not find domain: default.","title":"Not Found"}}
```
* add user to "default" domain when creating it.
* use "type" as the positional argument, per
  https://docs.openstack.org/keystone/pike/admin/cli-keystone-manage-services.html
  otherwise we will have failures like:
```
2020-05-28T13:38:24.867 INFO:teuthology.orchestra.run.smithi198.stderr:openstack service create: error: unrecognized arguments: --type keystone
```
* update `create_endpoint()` to use the V3 API,
  see
  https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html

Fixes: https://tracker.ceph.com/issues/45692
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Sebastian Wagner
ffef64d046 qa/cephadm: Add RGW
Fixes: https://tracker.ceph.com/issues/43911

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-29 13:33:05 +02:00
Varsha Rao
ab41951b04 qa/cephfs: Add tests for nfs
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-29 14:47:32 +05:30
Neha Ojha
b162541ac2 qa/suites/rgw/verify/tasks/cls.yaml: whitelist PG_AVAILABILITY
The balancer was turned on by default in
d4fbaf7ea9, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.

Fixes: https://tracker.ceph.com/issues/45619
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-28 19:30:46 +00:00
Neha Ojha
794f25affd qa/*/tasks/rados_cls_all.yaml: whitelist PG_AVAILABILITY
The balancer was turned on by default in
d4fbaf7ea9, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.

Fixes: https://tracker.ceph.com/issues/45619
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-28 19:21:37 +00:00
Patrick Donnelly
41b6120399
qa: run install task only once
Fixes: https://tracker.ceph.com/issues/45612
Fixes: 8d77d134c7
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-19 15:28:49 -07:00
Sebastian Wagner
5200b6478a
Merge pull request #34980 from sebastian-philipp/cephadm-workunits-rm-ubuntu_2020
qa/cephadm: rm ubuntu_latest for now

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-19 13:59:09 +02:00
Patrick Donnelly
356c64a3ff
Merge PR #34596 into master
* refs/pull/34596/head:
	test: add a new program for testing ino_release_cb
	client: add a new inode release request callback
	client: expose ceph_ll_register_callbacks via libcephfs
	client: move callback typedefs and arg struct into ceph_ll_client.h
	client: rename ceph_statx.h to ceph_ll_client.h
	client: make client_dentry_callback_t more friendly for C
	client: only override umask_cb with non-NULL values

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-05-18 15:07:43 -07:00
Jason Dillaman
7f1df0d18a
Merge pull request #34789 from trociny/wip-rbd-nbd-quiesce
rbd-nbd: add quiesce/unquiesce hooks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-15 11:12:50 -04:00
Yuri Weinstein
6c9d990697 qa/tests: run nautilus-x on pacific on ubuntu 18.04
...as we don't build `nauiltus` on `20.04`

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-05-14 12:35:20 -07:00
Sebastian Wagner
3742a28415 qa/cephadm: rm ubuntu_latest for now
Fixes: https://tracker.ceph.com/issues/45462
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-11 11:53:11 +02:00
Mykola Golub
a834e7115e qa/suites/rbd: enable random distro for nbd tests
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-05-09 16:36:49 +01:00
Patrick Donnelly
d791daa732
qa: move fsx deps into common cephfs install yaml
The double install task causes errors during unwind.

Fixes: fe3961fc36
Fixes: https://tracker.ceph.com/issues/45387
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-05 09:35:44 -07:00
J. Eric Ivancich
43c948a4ab qa/rgw: integration test for rgw-orphan-list & radosgw-admin radoslist
Add teuthology test for `rgw-orphan-list` in a new tool suite under
rgw. It only needs to be tested under one configuration. And the new
tool sub-suite can be used by other tooling int he
future. radosgw-admin `radoslist` is tested indirectly through
`rgw-orphan-list` and therefore does not need its own test.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-04 13:55:00 -04:00
Casey Bodley
67415ea07b
Merge pull request #34286 from cbodley/wip-qa-rgw-cls-2pc-queue
qa/rgw: run ceph_test_cls_2pc_queue in verify suite

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-05-04 12:29:14 -04:00
Josh Durgin
beaa4b04bc
Merge pull request #34880 from neha-ojha/wip-balancer-cleanup
qa/suites/rados/thrash*: remove upmap.yaml and off.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-05-01 13:15:58 -07:00
Neha Ojha
d46ea00320 qa/suites/rados/thrash*: add on.yaml
To test with balancer in upmap mode, which is the default.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-01 18:49:36 +00:00
Neha Ojha
cb5611e0d5 qa/suites/rados/thrash*: remove upmap.yaml and off.yaml
The balancer is always on, so we don't need upmap.yaml.
off.yaml doesn't make sense anymore.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-01 18:31:01 +00:00
Jeff Layton
c0db8a01f0 test: add a new program for testing ino_release_cb
Create a bunch of files and get their inode numbers. Remount, look them
all up by inode number and hold references. Stop looking up inodes as
soon as we get a callback from libcephfs. If we got the callback, return
success. Fail otherwise.

Since this has the same cluster setup as the other client_trim_caps
testcase, we can piggyback onto that task.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2020-04-30 16:20:01 -04:00
Casey Bodley
6c1f3cbb24
Merge pull request #34352 from smanjara/wip-dynamic-resharding
rgw: more multisite dynamic resharding pieces.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-30 10:02:12 -04:00
Shilpa Jagannath
71b657b9d2 minor fix: run tests in python venv
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-30 12:39:50 +05:30
Tatjana Dehler
18875ef337 mgr/dashboard: add telemetry report component
Add a telemetry component in order to give the user the
possibility to configure the telemetry module in a more
guided fashion. The component offers broader explanations,
shows a preview of the generated report and asks the user
to accept the license before enabling the module.

Fixes: https://tracker.ceph.com/issues/43956
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-04-29 16:34:00 +02:00
Yuri Weinstein
c272749ed4
Merge pull request #34748 from neha-ojha/wip-jewel-cleanup
qa/suites/rados: jewel cleanup

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-27 17:05:59 -07:00
Shilpa Jagannath
0270dc4aaa qa/rgw: some basic reshard tests
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-27 13:16:09 +05:30
Kefu Chai
d07960ef7a
Merge pull request #34578 from lxbsz/volume_client_py3
qa/volume_client: be python3 compatible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-25 18:58:43 +08:00
Neha Ojha
c258ff2c15 qa/suites/rados/thrash-old-clients/1-install: remove hammer
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:45:19 +00:00
Neha Ojha
4a6361d93e qa/suites/rados/thrash/workloads: use default mon osd initial require min compat client
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:29:06 +00:00
Neha Ojha
5ac2f54701 qa/suites/rados/thrash-old-clients/1-install: remove jewel
Since we've changed mon_osd_initial_require_min_compat_client to
luminous in 986c271b75, we can remove jewel from
mix.

Fixes: https://tracker.ceph.com/issues/45242
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:29:06 +00:00
Casey Bodley
3466890e50
Merge pull request #34613 from cbodley/wip-44661
rgw: use DEFER_DROP_PRIVILEGES flag unconditionally

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2020-04-22 10:06:06 -04:00
Xiubo Li
0ed507dfd4 qa/volume_client: fix TestVolumeClient failing under new py3 runtime
The pybind now has dropped the WITH_PYTHON2 option, and for now only
py3 supported.

Fixes: https://tracker.ceph.com/issues/45103
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-20 07:04:52 -04:00
Kefu Chai
a2d1eaeb3f qa/suites/*/rados_cls_all.yaml: load all classes
the intention to add the whitelist was to test "sdk" class, but if we
add new classes to the list, and add tests exercising them, the tests
fail if we fail to update these `rados_cls_all.yaml` accordingly.

so in this change, the list is now '*' which allows OSD to load all
classes found in the specified directory

Fixes: https://tracker.ceph.com/issues/45113
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-20 18:58:35 +08:00
Casey Bodley
94ca8f4b81 qa/rgw: add setuser/group to test DEFER_DROP_PRIVILEGES
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-17 10:45:15 -04:00
Casey Bodley
0f44806c53 qa/rgw: run ceph_test_cls_2pc_queue in verify suite
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-14 11:43:05 -04:00
Jason Dillaman
267af5fcae qa/suites/rbd: tweak 'require_osd_release' to 'pacific'
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-04-14 08:43:29 -04:00
Casey Bodley
577f211391 test: add ceph_test_cls_cmpomap
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-13 09:23:44 -04:00
Sage Weil
d3d1ea5507 Merge PR #34272 into master
* refs/pull/34272/head:
	mgr/PyModuleRegistry: ignore 'obsolete' modules
	qa/suites/fs/upgrade: start with msgr2
	qa/suites/fs/upgrade: s/mimic/nautilus/
	qa/suites/upgrade: whitelist when orchestrator_cli can't be loaded
	qa/standalone/mon/msgr-v2-transition: remove test
	mon/MgrMonitor: disable orchestrator_cli if >= octopus
	qa/suites/ugprade: remove mimic -> pacific upgrade tests
	qa/standalone/mon/misc: update TEST_mon_features
	mon/MgrMap: use always_on_modules from most recent release
	mon/MonCommands: require-osd-release pacific
	qa/suites/rados/thrash-old-clients: adjust octopus package list for el7
	doc/dev/release-kickoff-checklist: include ceph-build change
	qa/workunits/cephtool/test.sh: fix require-osd-release test
	mon/MgrMonitor: define mgr modules for pacific
	doc/dev/release-checklists: generalize
	doc/dev/release-kickoff-checklist: quotes
	qa/suites/rados/thrash-old-clients/1-install/octopus
	qa/releases/pacific.yaml
	mds/cephfs_features: CEPHFS_CURRENT_RELEASE
	doc/dev/release-kickoff-checklist: a few other items
	osd: remove redundant check
	mon/OSDMonitor: make pacific changes
	mon/mon_types: add mon pacific feature
	include/ceph_features: define SERVER_PACIFIC
	include/ceph_features: fix whitespace
	mon: persistent INCOMPAT_PACIFIC item
	ceph-volume: __release__ == pacific
	script: update backport scripts for now-released octopus
	common: pacific release constants
	doc/dev/release-kickoff-checklist
	src/ceph_release: pacific, 16, dev
	CMakeLists: 16.0.0

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-04-12 17:33:57 -05:00
Kefu Chai
8c63b26fe8
Merge pull request #34264 from tchaikov/wip-qa-py3
qa/tasks: be python3 compatible

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-09 10:08:44 +08:00
Sage Weil
8aa9c88f3e qa/suites/fs/upgrade: start with msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 15:46:36 -05:00
Sage Weil
985c203eec qa/suites/fs/upgrade: s/mimic/nautilus/
Also rename 3-compat_client/mimic -> pacific to match the min_client
setting (the naming here confuses me...).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:10:32 -05:00
Sage Weil
be5f525df0 qa/suites/upgrade: whitelist when orchestrator_cli can't be loaded
This happens because mgrs may start before mons, or before mons have
published a new mgrmap that disables orchestrator_cli.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:10:32 -05:00