Commit Graph

108971 Commits

Author SHA1 Message Date
Samuel Just
d4b1cc61e6 PrimaryLogPG: add assert that we aren't trimming past last_update_ondisk
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-03-19 08:14:53 +08:00
xie xingguo
a9412ba25c qa/osd-recovery: pass osd_pg_log_trim_min = 0 to exercise short pg logs
we have osd_min_pg_log_entries to 2 (good) but not osd_pg_log_trim_min
which defaults to 100. Thus, even on those tests we're only rarely vulnerable.
Reset osd_min_pg_log_entries to 0 to make sure we really
would keep a minimal pg log in hand.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-13 08:49:58 +08:00
xie xingguo
b73b93c6f0 qa/short_pg_log: pass osd_pg_log_trim_min = 0 to exercise short pg logs
we have osd_min_pg_log_entries to 2 (good) but not osd_pg_log_trim_min
which defaults to 100. Thus, even on those tests we're only rarely vulnerable.
Reset osd_min_pg_log_entries to 0 to make sure we really
keep a minimal pg log in hand.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-13 08:49:32 +08:00
xie xingguo
eef1928e01 osd/PeeringState: do not trim pg log past last_update_ondisk
Trimming past last_update_ondisk would be really bad, e.g.,
a new interval change would cancel&redo a previous op, and if
we trim past last_update_ondisk, there could be potential
object inconsistencies as log merging won't necessarily be able
to find all divergent entries later (we lost track of the unfinished
op that should really be reverted).

Fixes: https://tracker.ceph.com/issues/44532
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-12 19:26:40 +08:00
Sage Weil
78b24f5783 Merge PR #33776 into master
* refs/pull/33776/head:
	test: Add flush_pg_stats to avoid race with getting num_shards_repaired

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-08 22:05:52 -05:00
Sage Weil
ab24887736 Merge PR #33804 into master
* refs/pull/33804/head:
	cephadm: ls: warn if daemon type (version) is not supported
	cephadm: report grafana version
	cephadm: report prometheus, node-exporter, alertmanager versions
	cephadm: use None (not '<no value>') for monitoring daemon version

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 19:57:06 -05:00
Sage Weil
f4ff4f96ec Merge PR #33792 into master
* refs/pull/33792/head:
	doc/cephadm: fix formatting for osd section
	doc/cephadm: update 'adding mons' section to suggest/prefer 'apply'
	doc/cephadm: fix formatting, typos
	mgr/cephadm: implement apply_mon
	mgr/cephadm: allow mon creation without explicit ip or addr
	mgr/cephadm: allow _apply_service to delete mon daemon's data
	mgr/cephadm: remove mon from monmap before removing mon
	mgr/cephadm: do not remove mon if it breaks quorum

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 17:29:00 -05:00
Sage Weil
4ba3e5d3b0 cephadm: ls: warn if daemon type (version) is not supported
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 17:24:27 -05:00
Sage Weil
02f2ddf3dc cephadm: report grafana version
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 17:23:24 -05:00
Sage Weil
6b0463cae5 Merge PR #33802 into master
* refs/pull/33802/head:
	mgr/cephadm: sanity check upgrade version
	mgr/cephadm: only need to invalidate once here
	mgr/cephadm: upgrade requires root mode for now

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 16:49:38 -05:00
Sage Weil
9fcad77bb5 Merge PR #33800 into master
* refs/pull/33800/head:
	mgr/cephadm: fix prom config generation when hosts have no labels or addrs

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 16:38:28 -05:00
Sage Weil
6d5c758117 Merge PR #33795 into master
* refs/pull/33795/head:
	mgr/orch: collapse 'SPEC' and 'PLACEMENT' columns in 'orch ls'

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 16:38:15 -05:00
Ramana Raja
746a2a9180
Merge pull request #33788 from ajarr/wip-44438
test_volumes: fix _verify_clone_attrs call

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-03-08 23:06:50 +05:30
Yuval Lifshitz
e66b911299
Merge pull request #33686 from yuvalif/fix_data_corruption_in_cls_queue_head
cls/queue: fix data corruption in urgent data
2020-03-08 13:17:02 +02:00
Sage Weil
5bd44d4338 mgr/cephadm: fix prom config generation when hosts have no labels or addrs
The inventory for a host might be {}, which evaluates as false.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 21:45:29 -06:00
Sage Weil
0360bea127 Merge PR #33742 into master
* refs/pull/33742/head:
	msg/async: s/nowait/always_async/ in EventCenter::submit_to().
	msg/async: perform recv reset immediately if called inside EC.

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-07 20:18:23 -06:00
Sage Weil
47d7c75731 cephadm: report prometheus, node-exporter, alertmanager versions
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 16:49:38 -06:00
Sage Weil
6ef96b4881 cephadm: use None (not '<no value>') for monitoring daemon version
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 16:34:08 -06:00
Sage Weil
b38771f45f Merge PR #33706 into master
* refs/pull/33706/head:
	qa/suites/rados/cephadm/upgrade: adjust starting version
	mgr/orch: from_strings -> from_string; do not accept a list
	mgr/volumes: pass placement as string, not list
	qa/tasks/mgr/test_orchestrator_cli: adjust placement args
	qa/tasks/cephadm: pass apply placement as a single arg
	mgr/orch: PlacementSpec: allow 'count:123'
	mgr/orch: PlacementSpec: may pretty_str() match input
	mgr/orch: take single placement argument
	mgr/orch: PlacementSpec.from_strings: take a string *or* a list

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-07 13:45:16 -06:00
Kefu Chai
cb12759b81
Merge pull request #33625 from sebastian-philipp/python-common-drive-groups-and
python-common: Make Drive Group filter by AND, instead of OR

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-08 01:47:42 +08:00
Sage Weil
82fa8f261a doc/cephadm: fix formatting for osd section
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
136c49d9b8 doc/cephadm: update 'adding mons' section to suggest/prefer 'apply'
It's nicer for users to specify the cluster/mon subnet once and let
cephadm scale mons.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
e0161132bc doc/cephadm: fix formatting, typos
No need for [monitor 1] when accessing the CLI--this can happy from
any node or container that has a working CLI.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
e025445232 mgr/cephadm: implement apply_mon
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
6b1d6b3cd4 mgr/cephadm: allow mon creation without explicit ip or addr
Allow mons to be created if the public_network option is defined in the
config database.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
f0dcf03d5e mgr/cephadm: allow _apply_service to delete mon daemon's data
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
d78ae6c04d mgr/cephadm: remove mon from monmap before removing mon
Check for force flag early so we don't update the monmap if the daemon
remove is going to fail anyway.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
b6b2d45dea mgr/cephadm: do not remove mon if it breaks quorum
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 11:22:55 -06:00
Sage Weil
26de97d093 mgr/cephadm: sanity check upgrade version
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 10:06:40 -06:00
Sage Weil
1ab85b7fde mgr/cephadm: only need to invalidate once here
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 09:55:24 -06:00
Sage Weil
a09e01fd21 mgr/cephadm: upgrade requires root mode for now
See https://tracker.ceph.com/issues/44429

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 09:55:24 -06:00
Sage Weil
053fd8f816 Merge PR #33794 into master
* refs/pull/33794/head:
	pybind/mgr/mgr_util: fix pretty time delta

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-07 09:21:32 -06:00
Sage Weil
f0cc50725f Merge PR #33752 into master
* refs/pull/33752/head:
	mgr: update metadata if an osd just joins
	mgr: update "hostname" when we already have the daemon state from the same entity

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-07 08:25:22 -06:00
Sage Weil
a9bb16b40f Merge PR #33747 into master
* refs/pull/33747/head:
	doc/mgr/telemetry: add 'send' command
	mgr/telemetry: force --license when sending while opted-out

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-07 07:07:41 -06:00
Sage Weil
bfb03adbe6 qa/suites/rados/cephadm/upgrade: adjust starting version
This version understands how to apply a mgr spec like '2;host=x' with a
semicolon.  This particular test build does.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 06:42:07 -06:00
Sage Weil
d3ba225f2a mgr/orch: from_strings -> from_string; do not accept a list
Normalize all callers to pass a single string.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 06:42:07 -06:00
Sage Weil
dee0634a1d mgr/volumes: pass placement as string, not list
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 06:42:07 -06:00
Kefu Chai
f7e57709a7
Merge pull request #33780 from xxhdx1985126/wip-crimson-osd-pg-creation-issue
crimson: solve the problem that crimson-osd's created pgs stuck in "unknown" state

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-07 15:53:47 +08:00
Xuehan Xu
8d6cdc4e4e crimson: fix the problem that crimson-osd can't establish stable connection with mgr
Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-03-07 11:59:55 +08:00
Xuehan Xu
2a51a3dcf9 crimson: decouple mgr client reconnect and connect reset handling
As of now, the following invocation sequence triggers deadlock when
closing crimson-osd's connection with mgr:
  ProtocolV2::dispatch_reset() --> crimson::mgr::Client::ms_handle_reset
--> crimson::mgr::Client::reconnect --> crimson::net::SocketConnection::close
--> crimson::net::Protocol::close()

In the above invocation sequence, ProtocalV2::dispatch_reset() enters the gate
"pending_dispatch" the leaving of which would wait for the complete of crimson::\
net::Protocal::close() which further wait for the complete of the gate's close().

This commit decouples this waiting chain.

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>
2020-03-07 11:59:45 +08:00
Sage Weil
51b60a212e qa/tasks/mgr/test_orchestrator_cli: adjust placement args
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
6e764e1a36 qa/tasks/cephadm: pass apply placement as a single arg
Use ';' as a separator, although ' ' would also work.  ',' would not
when msgr v2 addrs are in use.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
92d5af3b74 mgr/orch: PlacementSpec: allow 'count:123'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
cf0df5469a mgr/orch: PlacementSpec: may pretty_str() match input
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
42cf2e787f mgr/orch: take single placement argument
This is maybe a wash on the 'ceph orch ...' portion of the CLI.  However,
it means that elsewhere, like 'ceph fs volume ...', we can be consistent
and have placement be (1) optional and (2) a single arg so that it is
easier to use both positionally and as a flag (--placement=all:true).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
e334da4b09 mgr/orch: PlacementSpec.from_strings: take a string *or* a list
The string can be ' ', ';', or ',' separated.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Sage Weil
cdff256afe mgr/orch: collapse 'SPEC' and 'PLACEMENT' columns in 'orch ls'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:22 -06:00
Sage Weil
7f9ed79dc6 Merge PR #33700 into master
* refs/pull/33700/head:
	mgr/cephadm: point dashboard at grafana automatically
	doc/cephadm/monitoring: document process to set up monitoring with cephadm

Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-03-06 21:19:49 -06:00
Sage Weil
63086e387d Merge PR #33785 into master
* refs/pull/33785/head:
	qa/suites/rados/cephadm: deploy all monitoring components
	qa/tasks/cephadm: deploy other monitoring components

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-06 21:19:34 -06:00
Sage Weil
f0f50e4f24 Merge PR #33602 into master
* refs/pull/33602/head:
	mgr/cephadm/osd: call synchronous _remove_daemon()
	mgr/cephadm/osd: Add pytest for osd removal
	mgr/cephadm/osd: delete transitional attributes
	mgr/cephadm: Move remove osd code to osd.py

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-06 21:18:26 -06:00