Commit Graph

108943 Commits

Author SHA1 Message Date
Ramana Raja
746a2a9180
Merge pull request #33788 from ajarr/wip-44438
test_volumes: fix _verify_clone_attrs call

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-03-08 23:06:50 +05:30
Yuval Lifshitz
e66b911299
Merge pull request #33686 from yuvalif/fix_data_corruption_in_cls_queue_head
cls/queue: fix data corruption in urgent data
2020-03-08 13:17:02 +02:00
Sage Weil
0360bea127 Merge PR #33742 into master
* refs/pull/33742/head:
	msg/async: s/nowait/always_async/ in EventCenter::submit_to().
	msg/async: perform recv reset immediately if called inside EC.

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-07 20:18:23 -06:00
Sage Weil
b38771f45f Merge PR #33706 into master
* refs/pull/33706/head:
	qa/suites/rados/cephadm/upgrade: adjust starting version
	mgr/orch: from_strings -> from_string; do not accept a list
	mgr/volumes: pass placement as string, not list
	qa/tasks/mgr/test_orchestrator_cli: adjust placement args
	qa/tasks/cephadm: pass apply placement as a single arg
	mgr/orch: PlacementSpec: allow 'count:123'
	mgr/orch: PlacementSpec: may pretty_str() match input
	mgr/orch: take single placement argument
	mgr/orch: PlacementSpec.from_strings: take a string *or* a list

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-07 13:45:16 -06:00
Kefu Chai
cb12759b81
Merge pull request #33625 from sebastian-philipp/python-common-drive-groups-and
python-common: Make Drive Group filter by AND, instead of OR

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-08 01:47:42 +08:00
Sage Weil
053fd8f816 Merge PR #33794 into master
* refs/pull/33794/head:
	pybind/mgr/mgr_util: fix pretty time delta

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-07 09:21:32 -06:00
Sage Weil
f0cc50725f Merge PR #33752 into master
* refs/pull/33752/head:
	mgr: update metadata if an osd just joins
	mgr: update "hostname" when we already have the daemon state from the same entity

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-07 08:25:22 -06:00
Sage Weil
a9bb16b40f Merge PR #33747 into master
* refs/pull/33747/head:
	doc/mgr/telemetry: add 'send' command
	mgr/telemetry: force --license when sending while opted-out

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-07 07:07:41 -06:00
Sage Weil
bfb03adbe6 qa/suites/rados/cephadm/upgrade: adjust starting version
This version understands how to apply a mgr spec like '2;host=x' with a
semicolon.  This particular test build does.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 06:42:07 -06:00
Sage Weil
d3ba225f2a mgr/orch: from_strings -> from_string; do not accept a list
Normalize all callers to pass a single string.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 06:42:07 -06:00
Sage Weil
dee0634a1d mgr/volumes: pass placement as string, not list
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 06:42:07 -06:00
Kefu Chai
f7e57709a7
Merge pull request #33780 from xxhdx1985126/wip-crimson-osd-pg-creation-issue
crimson: solve the problem that crimson-osd's created pgs stuck in "unknown" state

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-07 15:53:47 +08:00
Xuehan Xu
8d6cdc4e4e crimson: fix the problem that crimson-osd can't establish stable connection with mgr
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-03-07 11:59:55 +08:00
Xuehan Xu
2a51a3dcf9 crimson: decouple mgr client reconnect and connect reset handling
As of now, the following invocation sequence triggers deadlock when
closing crimson-osd's connection with mgr:
  ProtocolV2::dispatch_reset() --> crimson::mgr::Client::ms_handle_reset
--> crimson::mgr::Client::reconnect --> crimson::net::SocketConnection::close
--> crimson::net::Protocol::close()

In the above invocation sequence, ProtocalV2::dispatch_reset() enters the gate
"pending_dispatch" the leaving of which would wait for the complete of crimson::\
net::Protocal::close() which further wait for the complete of the gate's close().

This commit decouples this waiting chain.

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-03-07 11:59:45 +08:00
Sage Weil
51b60a212e qa/tasks/mgr/test_orchestrator_cli: adjust placement args
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
6e764e1a36 qa/tasks/cephadm: pass apply placement as a single arg
Use ';' as a separator, although ' ' would also work.  ',' would not
when msgr v2 addrs are in use.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
92d5af3b74 mgr/orch: PlacementSpec: allow 'count:123'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
cf0df5469a mgr/orch: PlacementSpec: may pretty_str() match input
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
42cf2e787f mgr/orch: take single placement argument
This is maybe a wash on the 'ceph orch ...' portion of the CLI.  However,
it means that elsewhere, like 'ceph fs volume ...', we can be consistent
and have placement be (1) optional and (2) a single arg so that it is
easier to use both positionally and as a flag (--placement=all:true).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
e334da4b09 mgr/orch: PlacementSpec.from_strings: take a string *or* a list
The string can be ' ', ';', or ',' separated.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
7f9ed79dc6 Merge PR #33700 into master
* refs/pull/33700/head:
	mgr/cephadm: point dashboard at grafana automatically
	doc/cephadm/monitoring: document process to set up monitoring with cephadm

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-03-06 21:19:49 -06:00
Sage Weil
63086e387d Merge PR #33785 into master
* refs/pull/33785/head:
	qa/suites/rados/cephadm: deploy all monitoring components
	qa/tasks/cephadm: deploy other monitoring components

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-06 21:19:34 -06:00
Sage Weil
f0f50e4f24 Merge PR #33602 into master
* refs/pull/33602/head:
	mgr/cephadm/osd: call synchronous _remove_daemon()
	mgr/cephadm/osd: Add pytest for osd removal
	mgr/cephadm/osd: delete transitional attributes
	mgr/cephadm: Move remove osd code to osd.py

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-06 21:18:26 -06:00
Sage Weil
822ee344fa pybind/mgr/mgr_util: fix pretty time delta
We were off by a factor of 10 with teh 'hours' values.  Oops!

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 16:24:32 -06:00
Sage Weil
4e7ab6f2e6 qa/suites/rados/cephadm: deploy all monitoring components
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 15:51:45 -06:00
Sage Weil
6232b9efd9 mgr/cephadm/osd: call synchronous _remove_daemon()
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 12:33:09 -06:00
Sebastian Wagner
493b9586fb mgr/cephadm/osd: Add pytest for osd removal
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-06 12:29:49 -06:00
Sebastian Wagner
b32695dd23 mgr/cephadm/osd: delete transitional attributes
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-06 12:29:15 -06:00
Sebastian Wagner
d3122fbf48 mgr/cephadm: Move remove osd code to osd.py
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-06 12:29:15 -06:00
Sage Weil
b2384e0291 mgr/cephadm: point dashboard at grafana automatically
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 11:30:18 -06:00
Sage Weil
739c23da1b doc/cephadm/monitoring: document process to set up monitoring with cephadm
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 11:30:18 -06:00
Sage Weil
234b4ebd41 Merge PR #33772 into master
* refs/pull/33772/head:
	mgr/cephadm: upgrade all ceph daemon types
	mgr/cephadm: do not specify --image arg for non-ceph daemons; fix upgrade
	mgr/cephadm: fix error-path return values for _run_cephadm
	mgr/orch: fix RGWSpec loading
	mgr/cephadm: fix 'orch rm'

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-06 11:30:01 -06:00
Sage Weil
97c54ff9c5 Merge PR #33779 into master
* refs/pull/33779/head:
	mgr/orch: fix ServiceSpec deserialization error

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-06 11:26:58 -06:00
Sage Weil
92c3989fc4 Merge PR #33614 into master
* refs/pull/33614/head:
	mgr/cephadm: enable custom TLS certificates for grafana
	mgr: enable verification of TLS certs without files
	mgr/cephadm: dump config to JSON only once when creating daemons

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-06 11:26:47 -06:00
Ramana Raja
d2e327c41c test_volumes: fix _verify_clone_attrs call
... to verify the attributes of clone and source subvolume belonging
to different subvolume groups.

Introduced in e22d546beb

Fixes: https://tracker.ceph.com/issues/44438
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-03-06 21:17:26 +05:30
Casey Bodley
11ce160b50
Merge pull request #33696 from cbodley/wip-common-spliterator
common: add iterator-based string splitter

Reviewed-by: Adam Emersen <aemerson@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-03-06 10:31:59 -05:00
Casey Bodley
ea10d4259e
Merge pull request #33754 from cbodley/wip-rgw-formatter-plain-segfault
rgw: fix string_view formatting in RGWFormatter_Plain

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-03-06 10:28:43 -05:00
Sage Weil
cab7eca926 Merge PR #33784 into master
* refs/pull/33784/head:
	mgr/cephadm: update type annotation

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-06 09:21:53 -06:00
Kefu Chai
e0265681af mgr/cephadm: update type annotation
the method signature was changed in
ea987a0e56

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-06 22:02:02 +08:00
Sage Weil
6fbd5d21e8 qa/tasks/cephadm: deploy other monitoring components
Grafana and alertmanager.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 13:32:57 +00:00
Sage Weil
31a7a46138 Merge PR #33757 into master
* refs/pull/33757/head:
	qa/valgrind.supp: abstract from ceph::buffer's symbol versioning.

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-06 07:17:45 -06:00
Sage Weil
b21a2874b6 Merge PR #33755 into master
* refs/pull/33755/head:
	mgr/cephadm: make osd create on an existing LV idempotent

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-06 06:39:50 -06:00
Kiefer Chang
880c0b7591
mgr/orch: fix ServiceSpec deserialization error
cephadm fails to load Service spec data when deserializing
HostPlacementSpec resource. Fix the issue by loading it properly.

Renaming the call from_dict to from_json for consistency.

This change also makes `spec` filed visible in JSON output of
ServiceSpec list.

Fixes: https://tracker.ceph.com/issues/44491
Fixes: https://tracker.ceph.com/issues/44392

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-06 19:51:57 +08:00
Lenz Grimmer
17e55269bb
Merge pull request #33081 from bk201/wip-43166
mgr/dashboard: improve device selection modal for creating OSDs

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-03-06 12:30:50 +01:00
Ramana Raja
6b45b3b6c9
Merge pull request #33775 from tchaikov/wip-44454
qa/tasks/ceph.py: quote "<kind>" in command line

Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-03-06 12:31:12 +05:30
Patrick Donnelly
e20ad96dbf
Merge PR #33674 into master
* refs/pull/33674/head:
	mgr/volumes: OpSmException.error -> OpSmException.errno
	mgr/volumes: remove unneeded assignment to `NoneType`
	mgr/volumes: add missing OpSmException import
	mgr/volumes: add missing error code
	mgr/volumes: remove stray return
	mgr/volumes: fixup format string args
	mgr/volumes: add missing `mgr` param
	mgr/volumes: assert self.fs
	mgr/volumes: skip type checking on RTimer class
	mgr/volumes: fixup threading.Timer import
	mgr/volumes: fix positional str formatting
	mgr/volumes: place getters and setters next to each other
	mgr/volumes: reverse params passed to `isinstace()`
	mgr/volumes: fix arg typo '.' -> ','
	mgr/volumes: add `deque` type
	mgr/volumes: `threading.Event.isSet` no longer exists in py3
	mgr/volumes: assert JobThread class
	mgr/volumes: import VolumeException
	mgr/volumes: use `SafeConfigParser` with py2
	mgr/volumes: use python version during configparser import
	mgr/volumes: add `Dict` type
	mgr/volumes: import errno
	mgr/volumes: add `mgr/volumes` to the mypy checks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-03-05 20:52:46 -08:00
Kefu Chai
da736c22c5 qa/tasks/ceph.py: quote "<kind>" in command line
otherwise bash will intepret "kind" as a file when handling command like
```
sudo zgrep <kind> /var/log/ceph/valgrind/* /dev/null | sort | uniq
```
and try to feed its content to zgrep, and write the output of zgrep
to /var/log/ceph/valgrind/*. this is not the intended behavior. what we
what to do is to pass "<kind>" as an argument to zgrep, along with
the globbed files names which matches "/var/log/ceph/valgrind/*".

in this change, "<kind>" is quoted as in the command line. it's also
what `pipes.quote()` does before the change of
35cf5131e7.

this addresses the regression introduced by
35cf5131e7.

Fixes: https://tracker.ceph.com/issues/44454
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-06 12:17:42 +08:00
Sage Weil
dfecbb19f5 mgr/cephadm: upgrade all ceph daemon types
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 21:28:36 -06:00
Sage Weil
725990f9d3 mgr/cephadm: do not specify --image arg for non-ceph daemons; fix upgrade
If we are calling the cephadm script for a non-ceph daemon (prometheus,
etc), do not specify the --image argument, and do not pull it out of
the config db from sections that don't exist.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 21:28:36 -06:00
Sage Weil
4e829f1df0 mgr/cephadm: fix error-path return values for _run_cephadm
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 21:23:12 -06:00