Commit Graph

6582 Commits

Author SHA1 Message Date
Sage Weil
de20c7bc61 Merge PR #34105 into master
* refs/pull/34105/head:
	Merge PR #34042 into octopus
	Merge PR #33959 into octopus
	Merge PR #34067 into octopus
	mgr/DaemonServer: add explicit check that acting matches for merge
	Merge pull request #34040 from dillaman/wip-44396-partial-fix
	Merge PR #34098 into octopus
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	Merge PR #34082 into octopus
	Merge PR #34068 into octopus
	cephadm: relabel /etc/ganesha mount
	Merge PR #34046 into octopus
	Merge PR #34092 into octopus
	Merge pull request #33719 from ukernel/wip-44416
	rbd-mirror: leader watcher should not cancel get locker if locker is invalid
	rbd-mirror: snapshot sync request needs to check for interruption
	librbd: request exclusive lock when moving to trash
	rbd-mirror: basic integration with sync throttling
	rbd-mirror: don't prematurely finish snapshot replay loop
	rbd-mirror: pass InstanceWatcher to snapshot Replayer
	doc/releases/octopus.rst: add note about ec recovery below min_size
	mgr/cephadm: configure rgw_frontends for rgw service
	cephadm: switch grafana image to the ceph repo
	Merge PR #34034 into octopus
	qa/suites/rados/cephadm/upgrade: update starting version
	Merge PR #33540 into octopus
	Merge PR #34023 into octopus
	Merge PR #34044 into octopus
	Merge PR #34030 into octopus
	doc/orchestrator: update rgw creation
	mgr/cephadm: clean up client.crash.* container_image settings after upgrade
	cephadm: make add-repo --release and --version independent
	cephadm: env over last used
	mgr/orch: accept port and ssl flags to 'apply rgw'
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	remove ceph_test_rados_watch_notify
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors
	mgr/rook: fix 'orch ps' for osds
	qa: fix all the fsx.sh-invoking yaml files to install dependencies
	mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-23 08:24:06 -05:00
Lenz Grimmer
b819847556
Merge pull request #34063 from s0nea/wip-dashboard-crush-rule-suite
mgr/dashboard: add crush rule test suite

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-23 13:11:50 +01:00
Sage Weil
2740349122 Merge PR #33959 into octopus
* refs/pull/33959/head:
	qa: fix all the fsx.sh-invoking yaml files to install dependencies

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-22 10:56:31 -05:00
Kefu Chai
71f6db5f6b
Merge pull request #34066 from mgfritch/cephadm-mon-b-test
qa/workunits/cephadm/test_cephadm.sh: fix mon.b failure

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-21 14:34:28 +08:00
Kefu Chai
11b8e974a9
Merge pull request #33869 from mgfritch/cephadm-osd-create-test
qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-21 14:30:23 +08:00
Kefu Chai
b0dca75a59
Merge pull request #34056 from xiexingguo/wip-44662
qa/*/osd-markdown.sh: propagate map to osd before testing its reaction

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-21 14:27:51 +08:00
Kefu Chai
f617e10612
Merge pull request #33903 from tchaikov/wip-rados-object-locator
tools/rados: use object-locator in user-visible outputs

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-21 10:30:08 +08:00
Sage Weil
1bc2853d2f Merge PR #34046 into octopus
* refs/pull/34046/head:
	qa/suites/rados/cephadm/upgrade: update starting version
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-20 14:50:42 -05:00
Michael Fritch
26b7a3f819
qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume
instead of via the orchestrator

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-20 07:55:26 -06:00
Sage Weil
f64de8f475 Merge PR #34034 into octopus
* refs/pull/34034/head:
	cephadm: make add-repo --release and --version independent

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-20 08:12:50 -05:00
Sage Weil
ec1b0e33d8 qa/suites/rados/cephadm/upgrade: update starting version
Need upgrade command to match!

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-20 08:11:58 -05:00
Michael Fritch
90cb2feec3
qa/workunits/cephadm/test_cephadm.sh: fix mon.b failure
and is_available check to ensure quorum

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-19 14:16:01 -06:00
Sage Weil
d75c7e79bf Merge PR #34044 into octopus
* refs/pull/34044/head:
	remove ceph_test_rados_watch_notify

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-19 13:47:29 -05:00
Sage Weil
6cb39e4aa8 Merge PR #34060 into master
* refs/pull/34060/head:
	Merge PR #34027 into octopus
	Merge PR #34045 into octopus
	Merge pull request #34035 from dillaman/wip-rbd-permissions
	mgr/progress: fix duration strings
	Merge PR #34014 into octopus
	Merge PR #34001 into octopus
	Merge PR #34011 into octopus
	qa/workunits/rbd: use context managers to control Rados lifespan
	Merge pull request #34032 from dillaman/wip-rbd-octopus-docs
	doc/releases/octopus: add additional RBD improvements
	qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
	mgr/cephadm: do not reconfig unmanaged services
	Merge PR #33981 into octopus
	Merge pull request #34018 from ajarr/octopus-subvolume-clone-cancel
	qa/workunits/cephadm/test_cephadm: output file for pub key
	Merge PR #33866 into octopus
	Merge PR #34005 into octopus
	Merge PR #34013 into octopus
	mgr/cephadm: pytest: Enable SpecStore
	mgr/orchestrator: add test for default implementation for apply()
	python-common: validate ServiceSpec.service_type
	fixup mgr/cephadm: Fix ceph orch apply -i
	mgr/dashbaord: orchestrator service: Revert wait_api_result to a single completion
	mgr/orchestrator: `orch daemon add` accepts a yaml
	mgr/cephadm: apply_drivegroups() returns a single Completion
	mgr/cephadm: remove `trivial_result()`
	mgr/cephadm: Fix `ceph orch apply -i`
	Merge pull request #33994 from dillaman/wip-librbd-poll-event-race
	doc: document `clone cancel` command
	test: add `clone cancel` tests
	mgr/volumes: introduce "clone cancel" volume command
	mgr/volumes: allow canceling a single asynchronous job for a volume
	mgr/volumes: helper for looking up a clone entry index
	mgr/volumes: periodically check if clone operations should be canceled
	mgr/volumes: periodically check if copy operations should be canceled
	mgr/volumes: introduce 'canceled' state in clone op state machine
	qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS
	qa/suites/rados/verify/validater/valgrind: less bluestore logging
	qa/suites/rados/verify/validater: increase heartbeat grace
	Revert "qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60"
	Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"
	ceph_test_watch_notify: try notify 10x if ALLOW_TIMEOUTS is set
	ceph_test_rados_api_misc: ShutdownRace timeout if ALLOW_TIMEOUTS is set
	qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits
	doc/install: edits
	doc/cephadm: more edits
	doc/cephadm/install: edits
	doc/cephadm/adoption: improvements
	doc/cephadm/install: a few edits
	doc/cephadm/install: do not install ceph-common on host (by default)
	doc/cephadm: drop os recs link
	doc/cephadm/upgrade: improvements
	doc/cephadm/upgrade: document upgrade
	doc/cephadm/install: revamp install docs
	doc: reorganize cephadm docs
	doc/cephadm/administration: update docs on customizing SSH config
	doc/cephadm/administration: add a note about the 'removed' dir
	mgr/balancer: tolerate pgs outside of target weight map
	qa/workunits/cephadm/test_cephadm: --skip-monitoring-stack
	Merge PR #33974 into octopus
	Merge PR #33442 into octopus
	Merge PR #33997 into octopus
	Merge PR #34000 into octopus
	use quay octopus tip until 15.2 tag is available
	python-common: reduce output of ServiceSpec.to_json()
	python-common,mgr/cephadm: move assert_valid_host to service_spec
	mgr/cephadm: add HostAssignment.validate()
	mgr/dashboard: adapt create_osds interface change
	mon/MgrMonitor: make 'mgr fail' work with no arguments
	cephadm: add allow_ptrace option to enable SYS_PTRACE
	update default container images
	mgr/cephadm: limit number of times check host is performed in the serve loop
	Merge PR #33961 into octopus
	Merge PR #33952 into octopus
	Merge PR #33990 into octopus
	Merge PR #33955 into octopus
	Merge PR #33936 into octopus
	mgr/orch: add --all-available-devices to 'orch apply osd'
	qa/workunits/cephadm: --skip-mon-network when using 127.0.0.1
	cephadm: add tests
	qa/tasks/cephadm: pass -v to bootstrap
	mgr/cephadm: only try to place mons on hosts matching public_network
	mgr/cephadm: keep track of host networks, ips
	cephadm: automatically infer mon public_network, if we can
	cephadm: add list-networks command
	cephadm: bootstrap: deploy monitoring stack by default
	librbd: defer event socket completion until after callback issued
	cephadm: add-repo: add --version
	mgr/cephadm: respect 'unmanaged' flag in spec
	mgr/orch: orch ls: show <no spec> or <unmanaged> as appropriate
	mgr/orch: orch ls: rename SPEC -> PLACEMENT
	mgr/orch: add 'unmanaged' property to ServiceSpec
	cephadm: rename distro args in repo methods
	mgr/orch: combine 'orch daemon add <type> ...' into one command
	mgr/orch: combine 'orch apply <type> [<placement>]' into one command

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-19 11:07:16 -05:00
Tatjana Dehler
04ee0178a5 mgr/dashboard: add crush rule test suite
https://tracker.ceph.com/issues/44679
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-03-19 16:33:22 +01:00
Sage Weil
3ed3f34b98 Merge PR #34027 into octopus
* refs/pull/34027/head:
	qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
	mgr/cephadm: do not reconfig unmanaged services
	qa/workunits/cephadm/test_cephadm: output file for pub key

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-19 08:22:40 -05:00
Sage Weil
e1c39fbfec cephadm: make add-repo --release and --version independent
Specify either --release name (to get the latest) or --version x.y.z to
get a specific version.

Adapt to updated locations on download.ceph.com so that we don't need to
know the release name for a specific x.y.z release.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-19 08:09:21 -05:00
Sage Weil
1ee25c2c6a Merge PR #34047 into master
* refs/pull/34047/head:
	qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-19 08:02:06 -05:00
xie xingguo
afdff0cd3f qa/*/osd-markdown.sh: propagate map to osd before testing its reaction
Mon might fail to share the newest map with any of up osds, e.g.,
due to an injected broken pipe. Since we don't have any client
activities during the osd-markdown tests, osds might be unaware of
the map changes made through CLI. Make sure osds have pulled the
newest map down before we can test its reaction correctly.

Fixes: https://tracker.ceph.com/issues/44662
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-19 18:17:28 +08:00
Mykola Golub
bff92cab55
Merge pull request #34035 from dillaman/wip-rbd-permissions
qa/workunits/rbd: use context managers to control Rados lifespan

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-03-19 10:43:57 +02:00
Sage Weil
41dcb3c814 mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 20:56:44 -05:00
Neha
6edd1cb686 qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change
Fixes: https://tracker.ceph.com/issues/44518
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-03-18 22:57:41 +00:00
Sage Weil
282932867a remove ceph_test_rados_watch_notify
This is an old test, we have good watch/notify coverage in the newer
tests, and it is buggy.

Fixes: https://tracker.ceph.com/issues/43861
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 17:45:17 -05:00
Sage Weil
11d607d0b7 Merge PR #34011 into octopus
* refs/pull/34011/head:
	qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS
	qa/suites/rados/verify/validater/valgrind: less bluestore logging
	qa/suites/rados/verify/validater: increase heartbeat grace
	Revert "qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60"
	Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"
	ceph_test_watch_notify: try notify 10x if ALLOW_TIMEOUTS is set
	ceph_test_rados_api_misc: ShutdownRace timeout if ALLOW_TIMEOUTS is set
	qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-18 12:12:54 -05:00
Jason Dillaman
477570a974 qa/workunits/rbd: use context managers to control Rados lifespan
There is a potential race between the expected exceptions being
thrown and Python shutting down racing with librados background
threads. Ensure that librados is properly shut down prior to
exiting Python.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-18 12:56:14 -04:00
Sage Weil
acbda9d19d qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
We are deploying containers manually.  Mark them unmanaged so that we
do not fight against mgr/cephadm cleaning up orphan daemons.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 10:44:42 -05:00
Ramana Raja
e3bf5e7c7b
Merge pull request #34018 from ajarr/octopus-subvolume-clone-cancel
mgr/volumes: add FS subvolume clone cancel

 Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-03-18 19:22:06 +05:30
Sage Weil
ee108d4395 qa/workunits/cephadm/test_cephadm: output file for pub key
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 08:44:10 -05:00
Sage Weil
1e25680552 Merge PR #34013 into octopus
* refs/pull/34013/head:
	qa/workunits/cephadm/test_cephadm: --skip-monitoring-stack

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-18 08:13:56 -05:00
Venky Shankar
ba4b9358e1 test: add clone cancel tests
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-03-18 11:03:17 +05:30
Sage Weil
7c19c1534b qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
baeb051910 qa/suites/rados/verify/validater/valgrind: less bluestore logging
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
4fda9d50f0 qa/suites/rados/verify/validater: increase heartbeat grace
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
0bd14ab080 Revert "qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60"
This reverts commit 4f742f200d.

This was in the wrong file.. see valgrind.yaml

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
12105ed9d7 Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"
This reverts commit 65e81e6eb4.

This slows things down too much with valgrind.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 19:32:42 -05:00
Sage Weil
40a7bcea70 qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits
When running under valgrind (and thrashing) things can be slow.  Tell
tests in case they need to tolerate timeouts.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 18:24:12 -05:00
Sage Weil
8630b1a814 qa/workunits/cephadm/test_cephadm: --skip-monitoring-stack
We're deploying these things manually later, and they use fixed ports.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-17 14:53:58 -05:00
Sage Weil
126c4446ac Merge PR #33974 into octopus
* refs/pull/33974/head:
	use quay octopus tip until 15.2 tag is available
	update default container images

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-17 14:52:28 -05:00
Sage Weil
fff8d11496 Merge PR #33442 into octopus
* refs/pull/33442/head:
	cephadm: add allow_ptrace option to enable SYS_PTRACE

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-17 12:05:46 -05:00
Kiefer Chang
6009d2d654
mgr/dashboard: adapt create_osds interface change
Support the parameter change of create_osds since
https://github.com/ceph/ceph/pull/33922.

Fixes: https://tracker.ceph.com/issues/44632
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-17 16:07:45 +08:00
Sage Weil
d7de2abc2f cephadm: add allow_ptrace option to enable SYS_PTRACE
In any environments it is helpful to have SYS_PTRACE so that you can
gdb attach or strace a daemon.

Leave this off by default so that the container is more secure.

Enable this in teuthology and vstart.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-16 21:04:22 -05:00
Sage Weil
a9b15c7e1a update default container images
- For tests, use bleeding-edge octopus branch
- For production defaults, use ceph/ceph:v15.2 tag
- For bootstrap, grab cephadm script from latest octopus branch

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-16 20:53:57 -05:00
Sage Weil
ccf85a1fef Merge PR #33952 into octopus
* refs/pull/33952/head:
	qa/workunits/cephadm: --skip-mon-network when using 127.0.0.1
	cephadm: add tests
	qa/tasks/cephadm: pass -v to bootstrap
	mgr/cephadm: only try to place mons on hosts matching public_network
	mgr/cephadm: keep track of host networks, ips
	cephadm: automatically infer mon public_network, if we can
	cephadm: add list-networks command

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-16 17:36:43 -05:00
Sage Weil
3a8f964d7d Merge PR #33936 into octopus
* refs/pull/33936/head:
	cephadm: bootstrap: deploy monitoring stack by default

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-03-16 15:17:48 -05:00
Sage Weil
8ab7a91615 qa/workunits/cephadm: --skip-mon-network when using 127.0.0.1
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-16 14:38:22 -05:00
Sage Weil
1a54da0b01 qa/tasks/cephadm: pass -v to bootstrap
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-16 14:38:22 -05:00
Sage Weil
7f0c3c4f37 Merge remote-tracking branch 'gh/octopus' 2020-03-16 12:47:25 -05:00
Sage Weil
7957655227 cephadm: bootstrap: deploy monitoring stack by default
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-16 12:39:44 -05:00
Sage Weil
dd20ffe2f5 Merge PR #33979 into octopus
* refs/pull/33979/head:
	cephadm: implement 'install' command

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-16 12:34:16 -05:00
Lenz Grimmer
cf01ccc2fd
Merge pull request #33856 from rhcs-dashboard/44545-fix-vstart-runner
vstart_runner.py: fix OSError when checking if non-existent path is mounted

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-16 17:27:28 +01:00