Commit Graph

1548 Commits

Author SHA1 Message Date
Casey Bodley
6c1f3cbb24
Merge pull request #34352 from smanjara/wip-dynamic-resharding
rgw: more multisite dynamic resharding pieces.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-30 10:02:12 -04:00
Shilpa Jagannath
71b657b9d2 minor fix: run tests in python venv
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-30 12:39:50 +05:30
Tatjana Dehler
18875ef337 mgr/dashboard: add telemetry report component
Add a telemetry component in order to give the user the
possibility to configure the telemetry module in a more
guided fashion. The component offers broader explanations,
shows a preview of the generated report and asks the user
to accept the license before enabling the module.

Fixes: https://tracker.ceph.com/issues/43956
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-04-29 16:34:00 +02:00
Yuri Weinstein
c272749ed4
Merge pull request #34748 from neha-ojha/wip-jewel-cleanup
qa/suites/rados: jewel cleanup

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-27 17:05:59 -07:00
Shilpa Jagannath
0270dc4aaa qa/rgw: some basic reshard tests
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-27 13:16:09 +05:30
Kefu Chai
d07960ef7a
Merge pull request #34578 from lxbsz/volume_client_py3
qa/volume_client: be python3 compatible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-25 18:58:43 +08:00
Neha Ojha
c258ff2c15 qa/suites/rados/thrash-old-clients/1-install: remove hammer
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:45:19 +00:00
Neha Ojha
4a6361d93e qa/suites/rados/thrash/workloads: use default mon osd initial require min compat client
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:29:06 +00:00
Neha Ojha
5ac2f54701 qa/suites/rados/thrash-old-clients/1-install: remove jewel
Since we've changed mon_osd_initial_require_min_compat_client to
luminous in 986c271b75, we can remove jewel from
mix.

Fixes: https://tracker.ceph.com/issues/45242
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-24 18:29:06 +00:00
Casey Bodley
3466890e50
Merge pull request #34613 from cbodley/wip-44661
rgw: use DEFER_DROP_PRIVILEGES flag unconditionally

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2020-04-22 10:06:06 -04:00
Xiubo Li
0ed507dfd4 qa/volume_client: fix TestVolumeClient failing under new py3 runtime
The pybind now has dropped the WITH_PYTHON2 option, and for now only
py3 supported.

Fixes: https://tracker.ceph.com/issues/45103
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-20 07:04:52 -04:00
Kefu Chai
a2d1eaeb3f qa/suites/*/rados_cls_all.yaml: load all classes
the intention to add the whitelist was to test "sdk" class, but if we
add new classes to the list, and add tests exercising them, the tests
fail if we fail to update these `rados_cls_all.yaml` accordingly.

so in this change, the list is now '*' which allows OSD to load all
classes found in the specified directory

Fixes: https://tracker.ceph.com/issues/45113
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-20 18:58:35 +08:00
Casey Bodley
94ca8f4b81 qa/rgw: add setuser/group to test DEFER_DROP_PRIVILEGES
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-17 10:45:15 -04:00
Jason Dillaman
267af5fcae qa/suites/rbd: tweak 'require_osd_release' to 'pacific'
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-04-14 08:43:29 -04:00
Casey Bodley
577f211391 test: add ceph_test_cls_cmpomap
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-13 09:23:44 -04:00
Sage Weil
d3d1ea5507 Merge PR #34272 into master
* refs/pull/34272/head:
	mgr/PyModuleRegistry: ignore 'obsolete' modules
	qa/suites/fs/upgrade: start with msgr2
	qa/suites/fs/upgrade: s/mimic/nautilus/
	qa/suites/upgrade: whitelist when orchestrator_cli can't be loaded
	qa/standalone/mon/msgr-v2-transition: remove test
	mon/MgrMonitor: disable orchestrator_cli if >= octopus
	qa/suites/ugprade: remove mimic -> pacific upgrade tests
	qa/standalone/mon/misc: update TEST_mon_features
	mon/MgrMap: use always_on_modules from most recent release
	mon/MonCommands: require-osd-release pacific
	qa/suites/rados/thrash-old-clients: adjust octopus package list for el7
	doc/dev/release-kickoff-checklist: include ceph-build change
	qa/workunits/cephtool/test.sh: fix require-osd-release test
	mon/MgrMonitor: define mgr modules for pacific
	doc/dev/release-checklists: generalize
	doc/dev/release-kickoff-checklist: quotes
	qa/suites/rados/thrash-old-clients/1-install/octopus
	qa/releases/pacific.yaml
	mds/cephfs_features: CEPHFS_CURRENT_RELEASE
	doc/dev/release-kickoff-checklist: a few other items
	osd: remove redundant check
	mon/OSDMonitor: make pacific changes
	mon/mon_types: add mon pacific feature
	include/ceph_features: define SERVER_PACIFIC
	include/ceph_features: fix whitespace
	mon: persistent INCOMPAT_PACIFIC item
	ceph-volume: __release__ == pacific
	script: update backport scripts for now-released octopus
	common: pacific release constants
	doc/dev/release-kickoff-checklist
	src/ceph_release: pacific, 16, dev
	CMakeLists: 16.0.0

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-04-12 17:33:57 -05:00
Kefu Chai
8c63b26fe8
Merge pull request #34264 from tchaikov/wip-qa-py3
qa/tasks: be python3 compatible

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-09 10:08:44 +08:00
Sage Weil
8aa9c88f3e qa/suites/fs/upgrade: start with msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 15:46:36 -05:00
Sage Weil
985c203eec qa/suites/fs/upgrade: s/mimic/nautilus/
Also rename 3-compat_client/mimic -> pacific to match the min_client
setting (the naming here confuses me...).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:10:32 -05:00
Sage Weil
be5f525df0 qa/suites/upgrade: whitelist when orchestrator_cli can't be loaded
This happens because mgrs may start before mons, or before mons have
published a new mgrmap that disables orchestrator_cli.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:10:32 -05:00
Sage Weil
3c7d5291de qa/suites/ugprade: remove mimic -> pacific upgrade tests
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:10:32 -05:00
Sage Weil
858b3a9c55 qa/suites/rados/thrash-old-clients: adjust octopus package list for el7
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:09:41 -05:00
Sage Weil
80fb39e469 qa/suites/rados/thrash-old-clients/1-install/octopus
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:09:41 -05:00
Kefu Chai
83c632099b mgr/telegraf: catch FileNotFoundError exception
in tasks/module_selftest.yaml, `TestModuleSelftest.test_telegraf()` is
called. but we fail to prepare a unix domain socket to which the telegraf
module can send stats. and telegraf module does not catch
FileNotFoundError exception, so the exception is populated to ceph-mgr
and is found by the test, hence the test is marked a failure whenever
telegraf is tested.

in this change,

* catch this exception, so it won't be caught by ceph-mgr
* whitelist the error message, so the test can pass

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-08 21:07:07 +08:00
Sebastian Wagner
58cc88dd14
Merge pull request #34406 from liewegas/fix-cephadm-upgrade-start
qa/suites/rados/cephadm/upgrade: start from v15.2.0

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-07 11:59:00 +02:00
Brad Hubbard
9c9be1b695
Merge pull request #34310 from badone/wip-restful-api-dont-pass-empty-dict-to-get
qa/suites/rados/rest: don't pass empty dict as data arg

Reviewed-by: Sage Weil <sage@redhat.com>
2020-04-07 08:14:03 +10:00
Brad Hubbard
ab569c3818
Merge pull request #34302 from badone/wip-specify-rule-for-pool-creation
Wip specify rule for pool creation

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2020-04-07 07:44:30 +10:00
Gregory Farnum
13fc503b0f
Merge pull request #34281 from vshankar/wip-44677
mgr: force purge normal ceph entities from service map

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-03 12:29:19 -07:00
Sage Weil
44ca20017a qa/suites/rados/cephadm/upgrade: start from v15.2.0
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-03 14:15:01 -05:00
Sage Weil
76b559598a qa/suites/rados/cephadm/smoke-roleless: add smoke test
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 23:16:42 +00:00
Brad Hubbard
f7c49a4063 qa/suites/rados/rest: Test against all supported distros
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-03-31 17:43:10 +10:00
Venky Shankar
3c531197a6 qa: whitelist cluster warning message for removed MDS
Fixes: http://tracker.ceph.com/issues/44677
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-03-31 02:41:00 -04:00
Brad Hubbard
c9317ea2ae qa/ceph-ansible: CA master requires Ansible 2.9
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-03-30 19:46:19 +10:00
Brad Hubbard
a12353815d qa/ceph-ansible: rule_name is required for pool creation
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-03-30 16:21:14 +10:00
Xie Xingguo
b7a959c1cf
Merge pull request #33910 from xiexingguo/wip-44532-2
osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-03-26 08:10:22 +08:00
Kefu Chai
d4a04809fd qa/suites/perf-basic: only test on bionic
because centos8/rhel8 does not package collectl or pdsh anymore. but
these packages are required by CBT for collecting performance stats.
so instead of testing on all supported distros, let's run the perf tests
only on distros offering these packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-25 09:54:06 +08:00
Sage Weil
de20c7bc61 Merge PR #34105 into master
* refs/pull/34105/head:
	Merge PR #34042 into octopus
	Merge PR #33959 into octopus
	Merge PR #34067 into octopus
	mgr/DaemonServer: add explicit check that acting matches for merge
	Merge pull request #34040 from dillaman/wip-44396-partial-fix
	Merge PR #34098 into octopus
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	Merge PR #34082 into octopus
	Merge PR #34068 into octopus
	cephadm: relabel /etc/ganesha mount
	Merge PR #34046 into octopus
	Merge PR #34092 into octopus
	Merge pull request #33719 from ukernel/wip-44416
	rbd-mirror: leader watcher should not cancel get locker if locker is invalid
	rbd-mirror: snapshot sync request needs to check for interruption
	librbd: request exclusive lock when moving to trash
	rbd-mirror: basic integration with sync throttling
	rbd-mirror: don't prematurely finish snapshot replay loop
	rbd-mirror: pass InstanceWatcher to snapshot Replayer
	doc/releases/octopus.rst: add note about ec recovery below min_size
	mgr/cephadm: configure rgw_frontends for rgw service
	cephadm: switch grafana image to the ceph repo
	Merge PR #34034 into octopus
	qa/suites/rados/cephadm/upgrade: update starting version
	Merge PR #33540 into octopus
	Merge PR #34023 into octopus
	Merge PR #34044 into octopus
	Merge PR #34030 into octopus
	doc/orchestrator: update rgw creation
	mgr/cephadm: clean up client.crash.* container_image settings after upgrade
	cephadm: make add-repo --release and --version independent
	cephadm: env over last used
	mgr/orch: accept port and ssl flags to 'apply rgw'
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	remove ceph_test_rados_watch_notify
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors
	mgr/rook: fix 'orch ps' for osds
	qa: fix all the fsx.sh-invoking yaml files to install dependencies
	mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-23 08:24:06 -05:00
Lenz Grimmer
b819847556
Merge pull request #34063 from s0nea/wip-dashboard-crush-rule-suite
mgr/dashboard: add crush rule test suite

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-23 13:11:50 +01:00
Sage Weil
2740349122 Merge PR #33959 into octopus
* refs/pull/33959/head:
	qa: fix all the fsx.sh-invoking yaml files to install dependencies

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-22 10:56:31 -05:00
Sage Weil
ec1b0e33d8 qa/suites/rados/cephadm/upgrade: update starting version
Need upgrade command to match!

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-20 08:11:58 -05:00
Tatjana Dehler
04ee0178a5 mgr/dashboard: add crush rule test suite
https://tracker.ceph.com/issues/44679
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-03-19 16:33:22 +01:00
Sage Weil
41dcb3c814 mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 20:56:44 -05:00
Samuel Just
ceb8e457e7 suites/rados/thrash: add rados-high-concurrency
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-03-19 08:15:02 +08:00
Sage Weil
7c19c1534b qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
baeb051910 qa/suites/rados/verify/validater/valgrind: less bluestore logging
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
4fda9d50f0 qa/suites/rados/verify/validater: increase heartbeat grace
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
0bd14ab080 Revert "qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60"
This reverts commit 4f742f200d.

This was in the wrong file.. see valgrind.yaml

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
12105ed9d7 Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"
This reverts commit 65e81e6eb4.

This slows things down too much with valgrind.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
40a7bcea70 qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits
When running under valgrind (and thrashing) things can be slow.  Tell
tests in case they need to tolerate timeouts.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 18:24:12 -05:00
Sage Weil
d7de2abc2f cephadm: add allow_ptrace option to enable SYS_PTRACE
In any environments it is helpful to have SYS_PTRACE so that you can
gdb attach or strace a daemon.

Leave this off by default so that the container is more secure.

Enable this in teuthology and vstart.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-16 21:04:22 -05:00