Commit Graph

6630 Commits

Author SHA1 Message Date
Jason Dillaman
e04422f2c6 rbd-mirror: switch to json_spirit formatter for journal image status
The free-form journal replay status description is now JSON-encoded. The
"master"/"mirror" designators have been changed to "primary"/"non_primary"
to better align with RBD terminology.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-04-03 15:24:53 -04:00
Laura Paduano
4d5fa9317d
Merge pull request #34232 from bk201/wip-44743
qa/tasks/mgr/dashboard/test_rbd: wait longer when purging

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-04-01 10:15:41 +02:00
Kiefer Chang
1c3d6539d9
qa/tasks/mgr/dashboard/test_rbd: wait longer when purging
Fixes: https://tracker.ceph.com/issues/44743
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
Co-authored-by: Kefu Chai <kefu@redhat.com>
Co-authored-by: Volker Theile <vtheile@suse.com>
2020-03-31 19:48:58 +08:00
Casey Bodley
c52cbe212f
Merge pull request #34253 from cbodley/wip-qa-rgw-too-many-shards
qa/rgw: remove 1999-shard tests from rgw/verify

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2020-03-30 10:06:11 -04:00
Sage Weil
803f05f254 Merge PR #34248 into master
* refs/pull/34248/head:
	qa/tasks/cephadm: no need to explicitly reconfig
	qa/tasks/cephadm: fetch final ceph.conf
	qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-30 08:24:59 -05:00
Casey Bodley
01516061ec qa/rgw: remove 1999-shard tests from rgw/verify
causing test failures because the host runs out of disk space

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-27 17:43:12 -04:00
Gregory Farnum
60504bd048
Merge pull request #33279 from rishabh-d-dave/fs-move-run_shell-and-related-methods
qa/cephfs: move run_shell to mount.py and add methods for negative testing
2020-03-27 12:59:43 -07:00
Gregory Farnum
ca24037277
Merge pull request #33740 from lxbsz/qa_test_timeout_failed
qa/test_config_session_timeout: fix options conflicting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-03-27 12:55:20 -07:00
Sage Weil
65ad142d59 qa/tasks/cephadm: no need to explicitly reconfig
cephadm does this magically now.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-27 11:49:29 -05:00
Sage Weil
29d2d7d64b qa/tasks/cephadm: fetch final ceph.conf
Otherwise we'll distribute the initial ceph.conf, which only has the
bootstrap mon in it.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-27 11:49:29 -05:00
Sage Weil
4c78dfe26f qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes
Revert part of 96220c0c05 so that we still
distribute a *final* ceph.conf and admin keyring to all nodes, right after
all of the mons are up.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-27 10:43:18 -05:00
Nathan Cutler
0e1e85047d
Merge pull request #34235 from toabctl/add-sle-15-sp2
qa/distros: add SLE-15-SP2

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-03-27 09:33:50 +01:00
Thomas Bechtold
3261e24636 qa/distros: add SLE-15-SP2
Ceph octopus is known to run on SLE-15-SP2 so add it to
qa/distros/all.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-27 08:17:13 +01:00
Kefu Chai
14722092fd cmake: add import-tasks to run-tox-qa test
to ensure that qa suites are python3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-27 14:54:37 +08:00
Kefu Chai
7d37226548 qa/tasks/mgr: use relative import
for better readability, and to ease the pain of developer to track back
to the top level python package for referencing a submodule

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-27 14:51:24 +08:00
Kefu Chai
4b4f393aec qa/tasks/cephfs: fix print() to be py3 compatible
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-27 14:51:24 +08:00
Kefu Chai
a105dea117
Merge pull request #33949 from toabctl/import-tasks-with-py3
qa: Add a tox env that can test importing files

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-27 14:49:46 +08:00
Kefu Chai
1032494ad2
Merge pull request #33873 from lxbsz/qa_vs_failed
qa/tasks/vstart_runner: fix sh to make it compatible with teth

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-26 23:26:53 +08:00
Kefu Chai
8e9621e8b9
Merge pull request #34195 from tchaikov/wip-tasks-mgr-submodule
qa/tasks/mgr: partial revert of 'import with full path'

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-26 23:23:43 +08:00
Yan, Zheng
4b79a2ecce
Merge pull request #29443 from simon-rock/metatool_dev
tool: add cephfs-meta-injection to show and amend info of inode
2020-03-26 17:10:28 +08:00
Kefu Chai
a16df762ae
Merge pull request #31055 from smithfarm/wip-42357
tests: qa/tasks/ceph.py: fail if any osd role doesn't get a device

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-26 16:41:07 +08:00
Thomas Bechtold
cb8d906af5 qa: Add a tox env that can test importing files
While switching to python3, we need to make sure that we can import
the qa/tasks (and others, but this starts with qa/tasks) on a python3
environment.
To test this, we need to install teuthology into the test
venv. Currently, teuthology is not py3 ready so this will fail.

To test the current state of the qa/tasks directory with the ongoing
work for python3 within teuthology, you can now do:

TEUTHOLOGY_GIT=git+https://github.com/kshtsk/teuthology.git@wip-py3-compat \
    tox -eimport-tasks

This is using the current branch from
https://github.com/ceph/teuthology/pull/1362 which does the work to
make teuthology python3 ready.

NOTE: This tox env is not activated by default. It's currently failing
but it provides a way to iterate over the failures and once we have
them fixed, we can activate the tox env during make-check.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-26 16:16:31 +08:00
Kefu Chai
b0bdbc3414 qa/tasks/mgr: partial revert of 'import with full path'
this change partially reverts #34139

using relative import helps with readability and ease the pain to write
down the full parent module name

in #34139, all relative imports were replaced with full path, because we
were using following code to verify if the code is python3 compatible:

```
  mod_spec = importlib.util.spec_from_file_location(mod_name, path)
  mod = importlib.util.module_from_spec(mod_spec)
  mod_spec.loader.exec_module(mod)
```

but this does not work with submodule which can import using relative
import without specifying the name of the package and its parent module.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-26 14:37:06 +08:00
Xie Xingguo
b7a959c1cf
Merge pull request #33910 from xiexingguo/wip-44532-2
osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-03-26 08:10:22 +08:00
Kefu Chai
a8d244707f
Merge pull request #34138 from bk201/wip-44725
qa/tasks/mgr/dashboard/test_health: update mdsmap schema

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-25 19:56:02 +08:00
Kefu Chai
b80d794473
Merge pull request #34139 from tchaikov/wip-qa-test-py3
qa,test: py3 compatible changes and fix

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-25 19:48:33 +08:00
Nathan Cutler
bc76b39a30 qa/tasks/ceph.py: fail test if osd devices not found
Fixes: https://tracker.ceph.com/issues/42357
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-25 09:35:01 +01:00
Nathan Cutler
9abebf28a8 qa/tasks/ceph.py: use .format to log dicts
The ".format" builtin logs dicts nicely right out of the box.

Also, some of the log messages were too cryptic - fixed them in this commit as
well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-25 09:33:44 +01:00
Nathan Cutler
ad477be286 qa/tasks/ceph.py: drop roles_to_journals and remote_to_roles_to_journals
These do not seem to get any use anymore.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-25 09:33:44 +01:00
Nathan Cutler
1393317129 qa/tasks/ceph.py: drop block_journal, tmpfs_journal
I looked, but did not find any tests that actually use these options.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-25 09:33:44 +01:00
Nathan Cutler
51c714d9b2 qa/tasks/ceph.py: cleanup: stop calling get_wwn_id_map()
Nowadays, get_wwn_id_map is essentially a noop - it does:

    return dict((d, d) for d in devs)

This reverts another bit of 8f720454cb from 2013.

References: https://tracker.ceph.com/issues/42313
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-25 09:33:44 +01:00
Kiefer Chang
afce413014
qa/tasks/mgr/dashboard/test_health: ignore fsmap.standbys
Fixes: https://tracker.ceph.com/issues/44725
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-25 11:17:17 +08:00
Kefu Chai
d4a04809fd qa/suites/perf-basic: only test on bionic
because centos8/rhel8 does not package collectl or pdsh anymore. but
these packages are required by CBT for collecting performance stats.
so instead of testing on all supported distros, let's run the perf tests
only on distros offering these packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-25 09:54:06 +08:00
Jan Fajerski
ed0db2445f qa/workunits/cephadm: create lvs before calling cephadm
Fixes: https://tracker.ceph.com/issues/44682

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-03-24 16:37:27 +01:00
Kefu Chai
496d7e2132 qa/tasks/cephfs: fix syntax error
it's a regression introduced by 9f6c764f10

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-24 18:27:55 +08:00
Kefu Chai
80b71ef461 qa: import py3 compatible modules using six
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-24 18:27:55 +08:00
Kefu Chai
947a74349d qa: import with full path
to be py3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-24 18:27:55 +08:00
Kiefer Chang
933df97cab
qa/tasks/mgr/dashboard/test_health: update mdsmap schema
Fixes: https://tracker.ceph.com/issues/44725
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-24 18:14:05 +08:00
simon gao
53ba9089d8 qa : add case to test cephfs-meta-injection
Signed-off-by: simon gao <simon29rock@gmail.com>
2020-03-24 16:39:02 +08:00
Sage Weil
04e0b9c2f8 Merge PR #34126 into master
* refs/pull/34126/head:
	qa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-23 13:55:16 -05:00
Neha
cfebec1b12 qa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length
It is possible for the pg dump to not be the latest when we check for newprimary
in _common_test(). This is because mgr_stats_period is 5 seconds, and we may not
have fetched the latest stats just yet. This causes the test to look at the same
stats before and after wait_for_clean.

Fixes: https://tracker.ceph.com/issues/43807 (2)
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-03-23 15:37:12 +00:00
Sage Weil
de20c7bc61 Merge PR #34105 into master
* refs/pull/34105/head:
	Merge PR #34042 into octopus
	Merge PR #33959 into octopus
	Merge PR #34067 into octopus
	mgr/DaemonServer: add explicit check that acting matches for merge
	Merge pull request #34040 from dillaman/wip-44396-partial-fix
	Merge PR #34098 into octopus
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	Merge PR #34082 into octopus
	Merge PR #34068 into octopus
	cephadm: relabel /etc/ganesha mount
	Merge PR #34046 into octopus
	Merge PR #34092 into octopus
	Merge pull request #33719 from ukernel/wip-44416
	rbd-mirror: leader watcher should not cancel get locker if locker is invalid
	rbd-mirror: snapshot sync request needs to check for interruption
	librbd: request exclusive lock when moving to trash
	rbd-mirror: basic integration with sync throttling
	rbd-mirror: don't prematurely finish snapshot replay loop
	rbd-mirror: pass InstanceWatcher to snapshot Replayer
	doc/releases/octopus.rst: add note about ec recovery below min_size
	mgr/cephadm: configure rgw_frontends for rgw service
	cephadm: switch grafana image to the ceph repo
	Merge PR #34034 into octopus
	qa/suites/rados/cephadm/upgrade: update starting version
	Merge PR #33540 into octopus
	Merge PR #34023 into octopus
	Merge PR #34044 into octopus
	Merge PR #34030 into octopus
	doc/orchestrator: update rgw creation
	mgr/cephadm: clean up client.crash.* container_image settings after upgrade
	cephadm: make add-repo --release and --version independent
	cephadm: env over last used
	mgr/orch: accept port and ssl flags to 'apply rgw'
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	remove ceph_test_rados_watch_notify
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors
	mgr/rook: fix 'orch ps' for osds
	qa: fix all the fsx.sh-invoking yaml files to install dependencies
	mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-23 08:24:06 -05:00
Lenz Grimmer
b819847556
Merge pull request #34063 from s0nea/wip-dashboard-crush-rule-suite
mgr/dashboard: add crush rule test suite

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-23 13:11:50 +01:00
Sage Weil
2740349122 Merge PR #33959 into octopus
* refs/pull/33959/head:
	qa: fix all the fsx.sh-invoking yaml files to install dependencies

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-22 10:56:31 -05:00
Kefu Chai
71f6db5f6b
Merge pull request #34066 from mgfritch/cephadm-mon-b-test
qa/workunits/cephadm/test_cephadm.sh: fix mon.b failure

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-21 14:34:28 +08:00
Kefu Chai
11b8e974a9
Merge pull request #33869 from mgfritch/cephadm-osd-create-test
qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-21 14:30:23 +08:00
Kefu Chai
b0dca75a59
Merge pull request #34056 from xiexingguo/wip-44662
qa/*/osd-markdown.sh: propagate map to osd before testing its reaction

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-21 14:27:51 +08:00
Kefu Chai
f617e10612
Merge pull request #33903 from tchaikov/wip-rados-object-locator
tools/rados: use object-locator in user-visible outputs

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-21 10:30:08 +08:00
Sage Weil
1bc2853d2f Merge PR #34046 into octopus
* refs/pull/34046/head:
	qa/suites/rados/cephadm/upgrade: update starting version
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-20 14:50:42 -05:00
Michael Fritch
26b7a3f819
qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume
instead of via the orchestrator

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-20 07:55:26 -06:00