Commit Graph

108971 Commits

Author SHA1 Message Date
Thomas Bechtold
46e22c422b qa: Enable basic mypy support for qa/ directory
A first step to do more automatic code checks on the qa/
directory. This is useful while transitioning to python3.

Also use log_exc to top-level to not run into:

error: Argument 1 to "log_exc" has incompatible type
  "Callable[[OSDThrasher], Any]"; expected "OSDThrasher"

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-05 06:54:56 +01:00
Thomas Bechtold
f5e77561e9 qa: Fix problems detected by mypy
This is a first step to enable mypy on the qa/ directory.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-05 06:53:31 +01:00
Kefu Chai
539c07818c
Merge pull request #33657 from tchaikov/wip-clang
client,common,mgr,rbd: clang related cleanups

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Patrick Donelly <pdonell@redhat.com>
2020-03-05 13:26:20 +08:00
Patrick Donnelly
16f2495471
Merge PR #33662 into master
* refs/pull/33662/head:
	mds: handle ceph_assert on blacklisting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-03-04 20:01:39 -08:00
Sage Weil
25b2190d43 mgr/cephadm: upgrade crash collectors too
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 21:58:58 -06:00
Sage Weil
44cc052a88 mgr/cephadm: fix service rm when there are no daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 21:57:36 -06:00
Kefu Chai
18f59107ac
Merge pull request #33707 from guits/guits-quick_fix_doc
doc: update erasure-code-profile.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-05 09:42:03 +08:00
Sage Weil
53120a6115 Merge PR #33730 into master
* refs/pull/33730/head:
	cephadm/vstart-smoke.sh: a smoke test to run against vstart
	mgr/cephadm: fix prom reconfig loop when dashboard is disabled
	mgr/cephadm: clean up log messages a bit
	mgr/cephadm: force remove daemons when removing service
	mgr/cephadm: do not error out on service rm|action if no daemons
	mgr/orchestrator: allow 'orch rm mon|mgr --force'

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-04 19:20:27 -06:00
Kefu Chai
38930d3df5
Merge pull request #33731 from wjwithagen/wjw-fix-test-compressor-gtest
src/test/compressor: Add missing gtest

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-05 07:50:41 +08:00
Sage Weil
d1ad1a9a69 mgr/cephadm: do not refresh daemon and device inventory as often
The device inventory rarely changes.

A daemon inventory of 60s may trigger every time around the loop when
doing upgrades, which is pretty slow (esp since it's currently done
synchronously).  We're doing a good job (I think?) of queueing refreshes,
so the only reason to time this data out is to catch daemon stops that
are caused by external factors.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 17:46:20 -06:00
Sage Weil
d999822be1 mgr/cephadm: if we had no record of deps, and deps are [], do not reconfig
None != []

This avoids a reconfig of every daemon on upgrade, among other things.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 17:17:08 -06:00
Guillaume Abrioux
25b6667459 doc: update erasure-code-profile.rst
Update this part of the documentation to reflect the change made by
3e86be7d50

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-03-05 00:13:50 +01:00
Sage Weil
050f5346bd Merge PR #33663 into master
* refs/pull/33663/head:
	mgr/cephadm: reduce boilerplate for unittests

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-04 16:54:10 -06:00
Willem Jan Withagen
a11c72d6d6 src/test/compressor: Add missing gtest
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2020-03-04 22:50:44 +01:00
Joshua Schmid
179b24bd93 mgr/cephadm: reduce boilerplate for unittests
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-03-04 15:13:38 -06:00
Sage Weil
762f0bbb70 cephadm/vstart-smoke.sh: a smoke test to run against vstart
This provides decent coverage of cephadm in practice, and can run against
a vstart cluster without having to go build containers.

vstart-cleanup.sh will clean up an aborted test reasonably well without
requiring sudo.

Caveats:
- there is no coverage for osds at this point.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 14:25:31 -06:00
Sage Weil
e855c74912 mgr/cephadm: fix prom reconfig loop when dashboard is disabled
We need to consider the mgrs a dep even if the prometheus module isn't
enabled, so that we are consistent with _calc_daemon_deps() and don't
reconfig prom in a loop.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 14:25:31 -06:00
Sage Weil
e28148c43e mgr/cephadm: clean up log messages a bit
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 14:25:31 -06:00
Sage Weil
f5be23f3a1 mgr/cephadm: force remove daemons when removing service
The rm CLI checks for --force based on type, so it isn't passed down
through the implementations.  So, pass it through.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 14:25:31 -06:00
Sage Weil
35a043caec mgr/cephadm: do not error out on service rm|action if no daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 14:25:31 -06:00
Sage Weil
150ee17f0d mgr/orchestrator: allow 'orch rm mon|mgr --force'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 14:25:31 -06:00
Sage Weil
54e4d2b663 Merge PR #33704 into master
* refs/pull/33704/head:
	mgr/cephadm: avoid racing updates to spec dict
	mgr/cephadm: make _do_upgrade work synchronously
	mgr/cephadm/tests/test_cephadm: remove broken rgw_update_fail test
	mgr/cephadm: make _apply_service move services
	mgr/cephadm: log when we create, destroy daemons
	mgr/cephadm: drop async_map_completion decorators from create funcs
	mgr/cephadm: _create_osd -> _create_osds
	mgr/cephadm: _remove_daemon -> _remove_daemons

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-04 14:25:05 -06:00
Nathan Cutler
11d0d3861a doc/releases: Octopus is not stable yet
Remove Octopus from list of active stable releases (it's not stable yet)
and put the 15.1.0 link in the Development column.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-04 20:52:45 +01:00
Sage Weil
04d92eec5b mgr/cephadm: avoid racing updates to spec dict
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:51 -06:00
Sage Weil
cd56b91021 mgr/cephadm: make _do_upgrade work synchronously
It mostly was, but it was futzing with completions when it didn't need to.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:51 -06:00
Sage Weil
8330d2f2bd mgr/cephadm/tests/test_cephadm: remove broken rgw_update_fail test
This was trying to show that we won't put 2 RGWs from different zones
on the same host.  In reality, the test was only passing because it was
specifying an invalid placement spec (count=3 in a 2-host cluster).

It's true that we shouldn't do that because they won't be able to bind,
but the HostAssignment code doesn't handle this case currently.

See https://tracker.ceph.com/issues/44397

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:50 -06:00
Sage Weil
58148c85c7 mgr/cephadm: make _apply_service move services
This transitions us to a synchronous mode of operation.

Fixes: https://tracker.ceph.com/issues/44167
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:50 -06:00
Sage Weil
cae9d5471d mgr/cephadm: log when we create, destroy daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:50 -06:00
Sage Weil
87849b069b mgr/cephadm: drop async_map_completion decorators from create funcs
So we can call them synchronously, if we chose.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:50 -06:00
Sage Weil
01edb5955c mgr/cephadm: _create_osd -> _create_osds
So we could do a single daemon synchronously, if we chose.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:50 -06:00
Sage Weil
bfdf3f2aa8 mgr/cephadm: _remove_daemon -> _remove_daemons
So we can remove a single daemon synchronously, if we choose.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 12:57:50 -06:00
Sage Weil
c4d5c430c9 Merge PR #33565 into master
* refs/pull/33565/head:
	os/bluestore: log allocation stats on a daily basis.

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-04 12:40:37 -06:00
Casey Bodley
2a3f16a5fd
Merge pull request #33539 from cbodley/wip-rgw-doc-reshard
doc/rgw: add design doc for multisite resharding

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-03-04 11:51:57 -05:00
Lenz Grimmer
411b023916
Merge pull request #33698 from rhcs-dashboard/vstart-set-prometheus-port-for-each-mgr
vstart.sh: set prometheus port for each mgr.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-04 16:21:38 +00:00
Sage Weil
86a27e5a9a Merge PR #33047 into master
* refs/pull/33047/head:
	qa/rgw: exercise DeleteRange in test_bucket_index_log_trim

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
2020-03-04 09:58:45 -06:00
Sage Weil
966c050df6 Merge PR #33287 into master
* refs/pull/33287/head:
	rgw: move frontends initial init to after global_init()
	rgw: ssl: don't try to init certificate if not needed
	rgw: frontend: add rgw_frontend_defaults configurable
	rgw: beast ssl: enable use of meta variable for cert config
	rgw: beast ssl: improve output
	rgw: beast frontend: handle default ssl configurables
	rgw: update docs about ssl config through config-key
	rgw: allow beast ssl frontend cert config via mon config-key
	rgw: create config-key svc
	rgw: svc/rados: new mon_command call

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-03-04 09:57:36 -06:00
Sage Weil
7dcb492bd3 ceph_test_cls_hello: set RETURNVEC on the expected EINVAL request
We are sending a request that returns an error *and* a data payload.  In order
to reliably get the payload, we need to set the RETURNVEC flag so that it is
recorded in the pg log.  Otherwise, we might get unlucky and resend the request
and end up in the dup handler and get the error code but no payload.

Fixes: https://tracker.ceph.com/issues/44385
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-04 09:48:38 -06:00
Lenz Grimmer
fd6fe6a413
Merge pull request #33612 from votdev/issue_44322_rgw_user_max_buckets
mgr/dashboard: Not able to restrict bucket creation for new user

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-03-04 15:38:58 +00:00
Abhishek L
5cf77765dd
Merge pull request #33721 from smithfarm/wip-nautilus_14_2_8
doc/releases: add 14.2.8 to release timeline

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-03-04 16:29:54 +01:00
Casey Bodley
006ee9e463 doc/rgw: add design doc for multisite resharding
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-04 10:27:37 -05:00
Sage Weil
c8981f9657 Merge PR #33705 into master
* refs/pull/33705/head:
	qa/suites/upgrade/nautilus-x/parallel: restart mgr.x before mons

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-04 08:53:24 -06:00
Kefu Chai
faf701d33a
Merge pull request #33723 from tchaikov/wip-seastar-on-demand
run-make-check.sh: WITH_SEASTAR on demand

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-03-04 22:50:24 +08:00
Kefu Chai
72a0ed4275 run-make-check.sh: WITH_SEASTAR on demand
don't compile WITH_SEASTAR=ON unless required

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-04 22:18:50 +08:00
Nathan Cutler
e935f02752 doc/releases: add 14.2.8 to release timeline
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-04 14:29:12 +01:00
Abhishek L
bb3a4f2270
Merge pull request #33670 from theanalyst/doc/releases/v14.2.8
doc: releases: v14.2.8 release notes

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Patrick Donelly <pdonell@redhat.com>
2020-03-04 14:21:22 +01:00
Kefu Chai
bd3dccd6fa
Merge pull request #33709 from tchaikov/wip-ceph-py-py3-rados
qa/tasks: py3 compat (tasks exercised by rados suites)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-04 21:16:21 +08:00
Alfonso Martínez
b591cfba5c vstart.sh: set prometheus port for each mgr.
Fixes: https://tracker.ceph.com/issues/44417

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-03-04 14:04:11 +01:00
Kefu Chai
00af2eddf5
Merge pull request #33690 from rhcs-dashboard/44237-remove-config-opt-read-from-system-roles
mgr/dashboard: remove 'config-opt: read' perm. from system roles.

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-03-04 20:54:42 +08:00
Kefu Chai
95a77f1497
Merge pull request #33699 from liewegas/fix-44078
common/ceph_time: tolerate mono time going backwards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-03-04 20:52:06 +08:00
Kefu Chai
6ac4348712
Merge pull request #33701 from tchaikov/wip-mgr-orch-exception
mgr/orch: try harder when pickle fails to marshal an exception

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-04 20:47:44 +08:00