Commit Graph

109766 Commits

Author SHA1 Message Date
Avan
0ae0a4abec
Merge pull request #33529 from rhcs-dashboard/wip-42979-fix-hover-issues
mgr/dashboard: add popover list of Stand-by Managers & Metadata Servers (MDS) in landing page
2020-03-20 20:02:52 +05:30
Kefu Chai
37180bf349
Merge pull request #34043 from adamemerson/wip-boost-valgrind
cmake: Add WITH_BOOST_VALGRIND option

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-20 22:17:46 +08:00
Michael Fritch
26b7a3f819
qa/workunits/cephadm/test_cephadm.sh: move osd test to ceph-volume
instead of via the orchestrator

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-20 07:55:26 -06:00
Michael Fritch
bac530f06a
cephadm: allow config and keyring args for ceph-volume
easier to script via the cephadm tests

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-20 07:55:26 -06:00
Paul Cuzner
8b76181e17 cephadm: switch grafana image to the ceph repo
Pull the pre-built grafana image from the ceph org on
docker.io

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-03-20 08:27:06 -05:00
Sage Weil
f64de8f475 Merge PR #34034 into octopus
* refs/pull/34034/head:
	cephadm: make add-repo --release and --version independent

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-20 08:12:50 -05:00
Sage Weil
ec1b0e33d8 qa/suites/rados/cephadm/upgrade: update starting version
Need upgrade command to match!

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-20 08:11:58 -05:00
Sebastian Wagner
f47033ad1e
Merge pull request #34080 from sebastian-philipp/doc-osd-create-osd-apply
doc/cephadm: replace `osd create` with `apply osd`

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-03-20 13:13:00 +01:00
Sebastian Wagner
cb12a3177c doc/cephadm: replace osd create with apply osd
Fixes: https://tracker.ceph.com/issues/44692

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-20 12:33:35 +01:00
Kefu Chai
884c5bb037 tests: update Dockerfile to support fc-31
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-20 18:28:31 +08:00
Yingxin Cheng
2092003ca2 crimson/net: allow mark_down() inside ms_handle_reset()
Although it is not necessary to mark_down the connection in its
ms_handle_reset() event, but it can be more convenient to allow it.
And Heartbeat already encounters this assertion failure.

So move the assertion to close_clean() which will help identify problems
if we happen to make ms_handle_reset() wait for messenger shutdown.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-20 16:07:49 +08:00
Yingxin Cheng
434cdd4666 crimson/net: change close() to mark_down()
* be explicit that mark_down() won't trigger reset event;
* return void so no deadlock is possible and memory is still safe
guarded by Messenger::shutdown();
* related changes in crimson/osd;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-20 16:07:49 +08:00
Yingxin Cheng
6c51c7aa82 crimson/net: cleanup dispatches with connection gate
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-20 16:07:48 +08:00
Yingxin Cheng
9cb4832410 crimson/net: enforce strict reset/accept order
When a new connection tries to replace the old one, the event order
should be like:
1. reset(old);
2. accept(new);

This means we cannot just reschedule the reset event asynchronously. And
we still need to make sure the internal state is integral when reset.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-20 16:07:48 +08:00
Yingxin Cheng
0bae2d3d89 crimson/net: fix incorrect reset events according to async-msgr
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-20 16:07:48 +08:00
Yingxin Cheng
22fe7ccc0b crimson/net: close() with ms_handle_reset()
* ms_handle_reset() should not be able to contaminate the internal
atomic messenger status, so make it an asynchronous event along
with close();

* add is_closed_clean() for messenger unit test, because the reset event
now happens after connection closed.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-20 16:07:48 +08:00
Adam C. Emerson
7f573bb334 build: Add WITH_BOOST_VALGRIND option
To build Boost.Context (and other libraries) with support to allow
them to be valground usefully, and to include the define to link
against them.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-20 00:51:49 -04:00
bangmingcheng
74e8d04576 doc: fix a spelling error at /doc/radosgw/dynamicresharding.rst
Signed-off-by: bangmingcheng <bangmingcheng@gmail.com>
2020-03-20 10:23:01 +08:00
Sage Weil
eeabe7d926 doc/cephadm: some troubleshooting tips
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-19 15:51:08 -05:00
Michael Fritch
90cb2feec3
qa/workunits/cephadm/test_cephadm.sh: fix mon.b failure
and is_available check to ensure quorum

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-19 14:16:01 -06:00
Sage Weil
06ecd31e39 Merge PR #33540 into octopus
* refs/pull/33540/head:
	doc/orchestrator: update rgw creation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-19 14:56:27 -05:00
Sage Weil
eab56bf5ef Merge PR #34023 into octopus
* refs/pull/34023/head:
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-19 14:33:35 -05:00
Sage Weil
d75c7e79bf Merge PR #34044 into octopus
* refs/pull/34044/head:
	remove ceph_test_rados_watch_notify

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-19 13:47:29 -05:00
Sage Weil
5f8450f091 Merge PR #34030 into octopus
* refs/pull/34030/head:
	cephadm: env over last used
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2020-03-19 12:21:52 -05:00
Sage Weil
6cb39e4aa8 Merge PR #34060 into master
* refs/pull/34060/head:
	Merge PR #34027 into octopus
	Merge PR #34045 into octopus
	Merge pull request #34035 from dillaman/wip-rbd-permissions
	mgr/progress: fix duration strings
	Merge PR #34014 into octopus
	Merge PR #34001 into octopus
	Merge PR #34011 into octopus
	qa/workunits/rbd: use context managers to control Rados lifespan
	Merge pull request #34032 from dillaman/wip-rbd-octopus-docs
	doc/releases/octopus: add additional RBD improvements
	qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
	mgr/cephadm: do not reconfig unmanaged services
	Merge PR #33981 into octopus
	Merge pull request #34018 from ajarr/octopus-subvolume-clone-cancel
	qa/workunits/cephadm/test_cephadm: output file for pub key
	Merge PR #33866 into octopus
	Merge PR #34005 into octopus
	Merge PR #34013 into octopus
	mgr/cephadm: pytest: Enable SpecStore
	mgr/orchestrator: add test for default implementation for apply()
	python-common: validate ServiceSpec.service_type
	fixup mgr/cephadm: Fix ceph orch apply -i
	mgr/dashbaord: orchestrator service: Revert wait_api_result to a single completion
	mgr/orchestrator: `orch daemon add` accepts a yaml
	mgr/cephadm: apply_drivegroups() returns a single Completion
	mgr/cephadm: remove `trivial_result()`
	mgr/cephadm: Fix `ceph orch apply -i`
	Merge pull request #33994 from dillaman/wip-librbd-poll-event-race
	doc: document `clone cancel` command
	test: add `clone cancel` tests
	mgr/volumes: introduce "clone cancel" volume command
	mgr/volumes: allow canceling a single asynchronous job for a volume
	mgr/volumes: helper for looking up a clone entry index
	mgr/volumes: periodically check if clone operations should be canceled
	mgr/volumes: periodically check if copy operations should be canceled
	mgr/volumes: introduce 'canceled' state in clone op state machine
	qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS
	qa/suites/rados/verify/validater/valgrind: less bluestore logging
	qa/suites/rados/verify/validater: increase heartbeat grace
	Revert "qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60"
	Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"
	ceph_test_watch_notify: try notify 10x if ALLOW_TIMEOUTS is set
	ceph_test_rados_api_misc: ShutdownRace timeout if ALLOW_TIMEOUTS is set
	qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits
	doc/install: edits
	doc/cephadm: more edits
	doc/cephadm/install: edits
	doc/cephadm/adoption: improvements
	doc/cephadm/install: a few edits
	doc/cephadm/install: do not install ceph-common on host (by default)
	doc/cephadm: drop os recs link
	doc/cephadm/upgrade: improvements
	doc/cephadm/upgrade: document upgrade
	doc/cephadm/install: revamp install docs
	doc: reorganize cephadm docs
	doc/cephadm/administration: update docs on customizing SSH config
	doc/cephadm/administration: add a note about the 'removed' dir
	mgr/balancer: tolerate pgs outside of target weight map
	qa/workunits/cephadm/test_cephadm: --skip-monitoring-stack
	Merge PR #33974 into octopus
	Merge PR #33442 into octopus
	Merge PR #33997 into octopus
	Merge PR #34000 into octopus
	use quay octopus tip until 15.2 tag is available
	python-common: reduce output of ServiceSpec.to_json()
	python-common,mgr/cephadm: move assert_valid_host to service_spec
	mgr/cephadm: add HostAssignment.validate()
	mgr/dashboard: adapt create_osds interface change
	mon/MgrMonitor: make 'mgr fail' work with no arguments
	cephadm: add allow_ptrace option to enable SYS_PTRACE
	update default container images
	mgr/cephadm: limit number of times check host is performed in the serve loop
	Merge PR #33961 into octopus
	Merge PR #33952 into octopus
	Merge PR #33990 into octopus
	Merge PR #33955 into octopus
	Merge PR #33936 into octopus
	mgr/orch: add --all-available-devices to 'orch apply osd'
	qa/workunits/cephadm: --skip-mon-network when using 127.0.0.1
	cephadm: add tests
	qa/tasks/cephadm: pass -v to bootstrap
	mgr/cephadm: only try to place mons on hosts matching public_network
	mgr/cephadm: keep track of host networks, ips
	cephadm: automatically infer mon public_network, if we can
	cephadm: add list-networks command
	cephadm: bootstrap: deploy monitoring stack by default
	librbd: defer event socket completion until after callback issued
	cephadm: add-repo: add --version
	mgr/cephadm: respect 'unmanaged' flag in spec
	mgr/orch: orch ls: show <no spec> or <unmanaged> as appropriate
	mgr/orch: orch ls: rename SPEC -> PLACEMENT
	mgr/orch: add 'unmanaged' property to ServiceSpec
	cephadm: rename distro args in repo methods
	mgr/orch: combine 'orch daemon add <type> ...' into one command
	mgr/orch: combine 'orch apply <type> [<placement>]' into one command

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-19 11:07:16 -05:00
Tatjana Dehler
04ee0178a5 mgr/dashboard: add crush rule test suite
https://tracker.ceph.com/issues/44679
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-03-19 16:33:22 +01:00
Yehuda Sadeh
1833797488 doc/orchestrator: update rgw creation
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2020-03-19 08:11:50 -07:00
Lenz Grimmer
4bdde0f1bd
Merge pull request #33925 from tspmelo/wip-update-npm
mgr/dashboard: Update Angular and fix npm audit

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-19 15:45:09 +01:00
Sage Weil
967a573b0a mgr/cephadm: clean up client.crash.* container_image settings after upgrade
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-19 09:29:20 -05:00
Sage Weil
3ed3f34b98 Merge PR #34027 into octopus
* refs/pull/34027/head:
	qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
	mgr/cephadm: do not reconfig unmanaged services
	qa/workunits/cephadm/test_cephadm: output file for pub key

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-19 08:22:40 -05:00
Sage Weil
e1c39fbfec cephadm: make add-repo --release and --version independent
Specify either --release name (to get the latest) or --version x.y.z to
get a specific version.

Adapt to updated locations on download.ceph.com so that we don't need to
know the release name for a specific x.y.z release.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-19 08:09:21 -05:00
Sage Weil
74fd8e7aa3 Merge PR #34045 into octopus
* refs/pull/34045/head:
	mgr/progress: fix duration strings

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-19 08:07:05 -05:00
Sage Weil
1ee25c2c6a Merge PR #34047 into master
* refs/pull/34047/head:
	qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-19 08:02:06 -05:00
Sage Weil
bad788d584 cephadm: env over last used
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-19 07:54:34 -05:00
Kefu Chai
e8db5eb5cc
Merge pull request #34017 from xxhdx1985126/wip-crimson-osd-report-failure
crimson/os/heartbeat: make Heartbeat::send_failures() safe

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-19 18:32:44 +08:00
xie xingguo
afdff0cd3f qa/*/osd-markdown.sh: propagate map to osd before testing its reaction
Mon might fail to share the newest map with any of up osds, e.g.,
due to an injected broken pipe. Since we don't have any client
activities during the osd-markdown tests, osds might be unaware of
the map changes made through CLI. Make sure osds have pulled the
newest map down before we can test its reaction correctly.

Fixes: https://tracker.ceph.com/issues/44662
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-19 18:17:28 +08:00
Mykola Golub
bff92cab55
Merge pull request #34035 from dillaman/wip-rbd-permissions
qa/workunits/rbd: use context managers to control Rados lifespan

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-03-19 10:43:57 +02:00
Xuehan Xu
c93e6d7db0 crimson/os/heartbeat: make Heartbeat::send_failures() safe
Currently, Heartbeat::send_failures() invokes monc.send_message() in a
continuation which may be run asynchronously, risking involving a daggling
"monc" reference when OSD shuts down and MonClient is destroyed.

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-03-19 15:04:13 +08:00
Kefu Chai
48942dc3bd
Merge pull request #34016 from tchaikov/wip-44658
ceph.spec.in: add build deps for Debug build of seastar

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-03-19 10:10:28 +08:00
Sage Weil
ac98ed009c mgr/orch: accept port and ssl flags to 'apply rgw'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 22:00:11 -04:00
Sage Weil
41dcb3c814 mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 20:56:44 -05:00
Samuel Just
ceb8e457e7 suites/rados/thrash: add rados-high-concurrency
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-03-19 08:15:02 +08:00
Samuel Just
d4b1cc61e6 PrimaryLogPG: add assert that we aren't trimming past last_update_ondisk
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-03-19 08:14:53 +08:00
Sage Weil
7bd3424bd7 cephadm: fall back to default for infer_image
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 19:10:20 -05:00
Sage Weil
a0a51c0874 cephadm: remove outdated check
Either the user passed their own image or we filled in the (monitoring)
default--no need for this check.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 19:10:20 -05:00
Sage Weil
a8d7f871b1 cephadm: consolidate default image logic
Merge the _parse_args() logic into @default_image.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 19:10:20 -05:00
Sage Weil
555926714c mgr/progress: fix duration strings
- simplify the code to just calculate the durations when we need them
(I'm not sure why we had those temporary strings!)
- use a nicer time delta format

Fixes: https://tracker.ceph.com/issues/44672
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 19:04:14 -05:00
Neha
6edd1cb686 qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change
Fixes: https://tracker.ceph.com/issues/44518
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-03-18 22:57:41 +00:00
Sage Weil
282932867a remove ceph_test_rados_watch_notify
This is an old test, we have good watch/notify coverage in the newer
tests, and it is buggy.

Fixes: https://tracker.ceph.com/issues/43861
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 17:45:17 -05:00
Sage Weil
2e7f7efc85 Merge PR #34014 into octopus
* refs/pull/34014/head:
	mgr/balancer: tolerate pgs outside of target weight map

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-18 17:42:48 -05:00