Commit Graph

119228 Commits

Author SHA1 Message Date
Sage Weil
604d2636d7 mgr/cephadm/upgrade: scale down MDS cluster(s) for major version upgrades
For octopus -> pacific, as with other recent releases, we need to scale
down the MDS cluster(s) to a single daemon before upgrading.  (This is
because the MDS intra-cluster protocols aren't fully versioned.)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 17:20:14 -06:00
Sage Weil
1e60a10370 mgr/cephadm: fix capitalization, level; drop elipses of log msgs
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 17:20:14 -06:00
Sage Weil
fe72946729 mgr/cephadm/upgrade: match against any repo_digest, not image_id
The image id can vary across hosts and (most notably) docker vs podman.
Instead, use the repo_digest as an image identifier.

Unfortunately, a single image may have multiple digests, even within the
same registry, so keep a list of the digests for the image we are
upgrading to, and ensure that each container has a digest that matches at
least one of them.

This allows upgrade to proceed in mixed docker+podman clusters.  However,
it does not yet address a cluster with mixed CPU architectures, because
the container image will have different digest(s) for each architecture
build.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 17:20:14 -06:00
Sage Weil
e9333972c1 cephadm: return repo_digests (plural) in pull/inspect output
When we inspect a (pulled) image, return all of the repo digests.  Update
the mgr/cephadm code accordingly to match.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 17:20:14 -06:00
Sage Weil
9f61c94ae7 mgr/cephadm: include container_image_digests in inventory
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
0826c45e0c cephadm: include image_digests list in 'ls' output
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
6741082eb3 vstart.sh: only extract first container digest
Otherwise the container_image value includes the same hash twice, separated
by a newline, causing all sorts of confusion.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
43ac324c38 mgr/cephadm: move release -> major translation to helper
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
a3c9df9197 mgr/cephadm/upgrade: tolerate old upgrade_state.target_versoin
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
8abd45c4c9 mgr/cephadm/upgrade: set require-osd-release when done with OSDs
After we've upgraded all of the osds, complete the major version upgrade
for core RADOS by setting the OSD cluster min.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
ceac086143 mgr: add lookup_release_name(int) to mgr interface
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
5147d0f930 mgr/cephadm: verify container image version after we pull it
Apply the version checks to make sure this upgrade/downgrade is possible.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
dcaf514bbf mgr/cephadm: only save version portion of version string
We don't want to 'ceph version ' prefix or sha1 suffix.  This matches us
up with 'orch upgrade start'

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
d31bed7941 cephadm: fix 'inspect' and 'pull'
With podman 2.2.1 I would get

ERRO[0000] Error printing inspect output: template: all inspect:1: function "json" not defined

Removing the 'json' portion of the query resolves it.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Sage Weil
95dd015bce mgr/cephadm/upgrade: implement N-2 version checks on upgrade start
- Prevent major upgrades that span > 2 releases
- Prevent downgrades to an rc or dev release

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-01 16:23:02 -06:00
Josh Durgin
25651e57b5
Merge pull request #38805 from kamoltat/wip-mgr-pg-autoscaler-improve-out-of-box-experience
pybind/mgr/pg_autoscaler: avoid scale-down until there is pressure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-02-01 11:33:55 -08:00
Kamoltat
daeb6f6ac0 mgr/pg_autoscaler: avoid scale-down until there is pressure
The autoscaler will start out with scaling each
pools to have a full complements of pgs from the start
and will only decrease it when pools need more due to
increased usage.

Introduced a unit test that tests only the
function get_final_pg_target_and_ratio() which
deals with the distrubtion of pgs amongst the
pools

Edited workunit script to reflect the change
of how pgs are calculated and distrubted.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-02-01 13:05:54 +00:00
Sebastian Wagner
4203a7f93d
Merge pull request #39200 from mitsu-ko/mypatch
doc/mgr/orchestrator: Unify the content of command and yaml

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-01 13:22:50 +01:00
Kefu Chai
65285f8f08
Merge pull request #39201 from tchaikov/wip-sphinx-openapi
doc: silence warnings from openapi sphinx extension

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-02-01 20:15:23 +08:00
Kefu Chai
8a01bad75f
Merge pull request #39194 from tchaikov/wip-dashboard-enum
pybind/mgr/dashboard: do not install enum34

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-02-01 19:11:32 +08:00
Mitsumasa KONDO
53639c2b01 doc/mgr/orchestrator: Unify the content of command and yaml
Signed-off-by: Mitsumasa KONDO <kondo.mitsumasa@gmail.com>
2021-02-01 18:13:12 +08:00
Kefu Chai
1debd98565 doc: silence warnings from openapi sphinx extension
it prints out

skipping non-JSON example generation.

if an API does not contains example while ":example:" is specified for
openapi extension, and if this API is not a GET request, openapi
extention complains.

see also
9dbae9c9a6/sphinxcontrib/openapi/openapi30.py (L191)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-01 17:57:57 +08:00
Sebastian Wagner
639fac3564
Merge pull request #39198 from tchaikov/wip-mypy.ini
mypy.ini: fix the unresolve conflict

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-01 09:48:04 +01:00
zdover23
e9a74d6951
Merge pull request #39196 from badone/wip-doc-python3-api
doc/rados/api: Update code samples to python3

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2021-02-01 17:18:35 +10:00
Kefu Chai
0c926d26cd mypy.ini: fix the unresolved conflict
it was introduced by a0930a6310, i failed
to resolve this conflict when resolving the merge confliction.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-01 15:11:56 +08:00
Brad Hubbard
0060cc52f9 doc/rados/api: Update code samples to python3
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-02-01 11:14:16 +10:00
Kefu Chai
d6c6669032
Merge pull request #39106 from mgfritch/cephadm-apt-get
cephadm: use `apt-get` for package install/update

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2021-01-31 17:23:10 +08:00
Kefu Chai
15a53419ca
Merge pull request #39131 from sebastian-philipp/rook-disable-untyped
mgr/rook: disallow_untyped_defs = True

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-01-31 17:22:20 +08:00
Kefu Chai
9d0023ba90 pybind/mgr/dashboard: do not install enum34
enum was introduced by python 3.4, but we require python3.6 and up.
so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-31 12:04:23 +08:00
Sebastian Wagner
a3555a22c8 doc: add rook_client._helper to mock_imports
This is due to `mgr/rook` now also includes `._helper`

Co-authored-by Kefu Chai <kchai@redhat.com>

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-31 11:19:01 +08:00
Sebastian Wagner
a0930a6310 mgr/rook: disallow_untyped_defs = True
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-31 11:18:58 +08:00
Sebastian Wagner
ea51f6990c mgr/rook: Fix possibly infinite loop in _execute_blight_job
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-31 11:18:12 +08:00
Kefu Chai
79b91ea62d
Merge pull request #39143 from adk3798/ha-rgw-data-dir
cephadm: fix get_data_dir calls for ha-rgw daemons

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Daniel-Pivonka <dpivonka@redhat.com>
2021-01-31 10:34:46 +08:00
Kefu Chai
383af836d7
Merge pull request #39081 from ifed01/wip-ifed-perpg-instrumentation
os/bluestore: be more verbose when fsck detects stray per-pg omaps

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-31 10:31:13 +08:00
Kefu Chai
78cdd1e5a2
Merge pull request #39095 from tchaikov/wip-cephadm-fsid
cephadm: add fsid if --name is not specified

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-31 10:29:59 +08:00
Kefu Chai
461b384aeb
Merge pull request #39056 from sebastian-philipp/alerts-percent-d
mgr/alerts: interval can be None

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-01-31 10:28:54 +08:00
Kefu Chai
aca5ed1442
Merge pull request #38930 from vbashkirtsev/master
client/fuse_ll.cc: use uint64_t for fuse_ll_forget() nlookup argument

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-01-31 10:28:13 +08:00
Kefu Chai
510b0e067e
Merge pull request #39132 from rzarzynski/wip-blk-ptr_node-for-aio
blk: avoid temporary bptrs on aio paths; use ptr_node instead.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-31 10:26:38 +08:00
Kefu Chai
11a8d4fbd5
Merge pull request #39151 from batrick/radosstriper-refactor
libradosstriper: make default layout const and static

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-01-31 10:25:46 +08:00
Kefu Chai
6527ec533c
Merge pull request #39116 from tchaikov/wip-crimson-pgstat
crimson/osd: keep a local copy of pg_stat and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-01-31 10:24:36 +08:00
Kefu Chai
1e0f52a713
Merge pull request #39158 from tchaikov/wip-mgr-snap-stat-ann
pybind/mgr/{snap-schedule,status,iostat}: add type annotations

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-01-31 10:17:54 +08:00
Kefu Chai
0b5d8e1928
Merge pull request #39187 from tchaikov/wip-standalone-mon-misc.sh
qa/standalone/mon/misc: verify that len(monmap.features.persistent) == 9

Reviewed-by: Sage Weil <sage@redhat.com>
2021-01-31 10:09:27 +08:00
Sage Weil
b2651236b9 Merge PR #39097 into master
* refs/pull/39097/head:
	qa/workunits/cephadm/test_cephadm: fix argument
	qa/workunits/cephadm/test_cephadm: show exporter output
	qa/workunits/cephadm/test_cephadm: don't bother pulling octopus
	qa/workunits/cephadm/test_cephadm: behave on terminal
	qa/workunits/cepham/test_cephadm: pull only recent images, from quay.ceph.io
	qa/workunits/cephadm/test_cephadm: don't respawn for python versions

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-01-30 10:14:43 -06:00
Sage Weil
b224ba3135 Merge PR #39172 into master
* refs/pull/39172/head:
	mgr/orchestrator: remove 'host' arg for 'orch ls'

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-01-30 10:12:43 -06:00
Sage Weil
576a89fdaf Merge PR #39173 into master
* refs/pull/39173/head:
	mgr/devicehealth: make CLI commands error when pool doesn't exist
	mgr/devicehealth: only create pool when we have some osds

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-30 10:09:48 -06:00
Kefu Chai
8dc097ff46 qa/standalone/mon/misc: verify that len(monmap.features.persistent) == 9
in beb62c029a, FEATURE_QUINCY was added to
ceph::features::mon::get_persistent(), so update the test accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-30 22:45:20 +08:00
Matt Benjamin
c26eaed15a
Merge pull request #38892 from ceph/update_s3select_add_zero_segment_log
s3select: update s3select submodule to last commit ; add zero segment log
2021-01-29 15:50:18 -05:00
Samuel Just
d972c3e1fe
Merge pull request #38767 from myoungwon/wip-manifest-chunk-scrub
osd: add has_manifest_chunk to count chunks in snapshot

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-01-29 12:25:38 -08:00
Sage Weil
713f79fdf4 Merge PR #39039 into master
* refs/pull/39039/head:
	src/test/cli/monmaptool: adjust for new monmap features
	qa/tasks/cephadm: allow custom git_url for cephadm_branch pull
	qa/suites/rados/upgrade: include pacific-x
	qa/suites/upgrade/pacific-x/parallel
	qa/suites: some clean up for quincy
	mon: updates for quincy
	mon: update for quincy ondisk features
	script: add pacific
	doc/dev/release-checklist: we tagged v17.0.0
	ceph-volume: change to quincy
	include/ceph_features: retire MON_SINGLE_PAXOS
	include/ceph_features: define FEATURE_SERVER_QUINCY
	mon/MgrMonitor: add always_on_modules for quincy
	add feature/release name quincy
	kickoff v17 quincy
	doc/dev/release-checklists: uncheck everything!

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-29 14:01:26 -06:00
Sage Weil
8c625c077b qa/workunits/cephadm/test_cephadm: fix argument
Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-29 13:34:44 -06:00