Commit Graph

104165 Commits

Author SHA1 Message Date
Mykola Golub
63b66a1133 test/librbd: fix mock warnings in TestMockIoImageRequest
(introduced in 081d28ae)

Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-11-08 21:49:51 +02:00
Sage Weil
05f5f0dd7f Merge PR #31493 into master
* refs/pull/31493/head:
	ceph-daemon: 'profile ...' not 'allow profile ...'
	mgr/ssh: 'profile ...' not 'allow profile ...'
	mgr/orchestrator_cli: rearrange things a bit
	doc/mgr/orchestrator_cli: remove irrelevant line
	mgr/ssh: learn to deploy rbd-mirror daemons
	mgr/orchestrator: add rbd-mirror commands and hooks
	ceph-daemon: learn to deploy rbd-mirror daemon
	mgr/ssh: handle lack of node hints more gracefully
	mgr/ssh: factor out update_{rgw,mds} into common helper
	mgr/ssh: fix update_rgw, update_mgr

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-11-08 13:28:45 -06:00
Patrick Donnelly
19a08227fb
Merge PR #30890 into master
* refs/pull/30890/head:
	mgr: invoke plugin shutdown on SIG{TERM,INT} signals.
	mgr/volumes: guard volume delete by waiting for pending ops
	mgr/volumes: cleanup libcephfs handles when stopping
	Revert "qa/suites/rados/mgr/tasks/module_selftest: whitelist mgr client getting backlisted"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:43:46 -08:00
Patrick Donnelly
a12b9a21e5
Merge PR #31115 into master
* refs/pull/31115/head:
	mds: Reorganize class members in MDSTableClient header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:41:47 -08:00
Patrick Donnelly
1b4e210969
Merge PR #31400 into master
* refs/pull/31400/head:
	mds: establish session with mgr only after added to FSMap
	mds: do not register as a service daemon
	mds: do not try to diagnose cause of MDSMap removal
	mds: fix handling of initial MDS states
	mds: remove unnecessary const qualifier
	mds: cleanup type decl and map iteration
	mds: define stream operator for mds_info_t

Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-08 10:06:08 -08:00
Patrick Donnelly
e765f2d533
mds: establish session with mgr only after added to FSMap
Note that we now sub to the mgrmap after init because the MgrClient
connection to the mgr is driven by receipt of the MgrMap.

This is important so that the MDS does not have metadata with the mgr
when the mons are ignoring the MDS otherwise due to CompatSet
incompatibilities.

Fixes: https://tracker.ceph.com/issues/41538
Fixes: https://tracker.ceph.com/issues/42635
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:12 -08:00
Patrick Donnelly
046137f819
mds: do not register as a service daemon
This commit undoes the service daemon registration for the MDS. It doesn't look
absolutely necessary and it causes the MDS to be listed twice in the `ceph
versions` output:

    $ ceph versions
        ...
        "mds": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 3
        },
        "mds": {
            "unknown": 3
        },
        "overall": {
            "ceph version v15.0.0-6915-g0143b904676 (0143b9046763ea1801efa8358a0c033ec862cea9) octopus (dev)": 10,
            "unknown": 3
        }

Fixing that requires looking for duplicates or ignoring MDSs in the
service daemons when the mon processes `ceph versions`. I have a feeling
that it wasn't actually designed to be used by the MDS this way however.
Additionally, the reason for "unknown" version is because the metadata
sent to the mgr does not include "ceph_version".

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:12 -08:00
Patrick Donnelly
c385178b0c
mds: do not try to diagnose cause of MDSMap removal
The Monitors send an empty MDSMap to an MDS it is removing. The MDS
can't diagnose the cause. Instead suggest looking at the cluster/monitor
logs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:11 -08:00
Patrick Donnelly
26a08df2ad
mds: fix handling of initial MDS states
Few things here:

- Make explicit the check for getting removed from the MDSMap. This was
  only done before by checking if MDS held a rank which does not check the
  case where a standby is removed from the FSMap.

- Use mds_info_t::dump to simplify various debug output.

- Add a few sanity asserts for invalid state transitions.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:11 -08:00
Patrick Donnelly
3355581a29
mds: remove unnecessary const qualifier
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:11 -08:00
Patrick Donnelly
bafa3b731f
mds: cleanup type decl and map iteration
This is a trivial refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:11 -08:00
Patrick Donnelly
1c56632e88
mds: define stream operator for mds_info_t
This also cleans up the output to be more readable/useful in debug
output.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-08 10:02:09 -08:00
Sage Weil
a07d9299aa Merge PR #29437 into master
* refs/pull/29437/head:
	mgr/diskprediction_local: Reverted dependencies, added HGST models
	mgr/diskprediction_local: Updated dependencies in ceph.spec.in, debian/control to match requirements.txt
	mgr/diskprediction_local: Updated Red Hat developed prediction model. Updated module options to choose between Red Hat and ProphetStor models.
	mgr/diskprediction_local: Updated prediction models to use only supported python packages.
	mgr/diskprediction_local: Replaced old models and updated predictor.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-08 11:49:00 -06:00
Sage Weil
723b93df25 ceph-daemon: 'profile ...' not 'allow profile ...'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 11:29:29 -06:00
Sage Weil
e97259aa6a mgr/ssh: 'profile ...' not 'allow profile ...'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 11:29:12 -06:00
Sage Weil
5122d323c4 mgr/orchestrator_cli: rearrange things a bit
Group services together

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 11:26:49 -06:00
Sage Weil
b0e1ff5efc doc/mgr/orchestrator_cli: remove irrelevant line
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 11:25:32 -06:00
Sage Weil
5588b69c29 mgr/ssh: learn to deploy rbd-mirror daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 11:25:32 -06:00
Sage Weil
c480810426 mgr/orchestrator: add rbd-mirror commands and hooks
This is somewhat different from the other services in that the name is
basically unused: we have a single pool of rbd-mirror daemons for the
whole cluster.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 11:25:32 -06:00
Jan Fajerski
26f14d1511 ceph-volume: add proper size attribute to partitions
Fixes: https://tracker.ceph.com/issues/42710

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-11-08 17:55:33 +01:00
Sage Weil
282068b2e0 ceph-daemon: learn to deploy rbd-mirror daemon
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 10:47:14 -06:00
Sage Weil
c57027c7ef mgr/ssh: handle lack of node hints more gracefully
RuntimeError: Remote method threw exception: TypeError: object of type 'NoneType' has no len()

isn't friendly

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 10:47:14 -06:00
Sage Weil
e316fa5211 mgr/ssh: factor out update_{rgw,mds} into common helper
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 10:12:34 -06:00
Sage Weil
782f00eccd mgr/ssh: fix update_rgw, update_mgr
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 10:12:34 -06:00
Sage Weil
25e26d27cf Merge PR #31467 into master
* refs/pull/31467/head:
	ceph-daemon: Add basic unittest infrastructure and cmake integration
	ceph-daemon: Move ceph-daemon executable to own directory

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-08 10:10:46 -06:00
Thomas Bechtold
1109bf9bcd ceph-daemon: Add basic unittest infrastructure and cmake integration
To be able to write unittests, add a tox.ini (for testing in venvs)
and a first single test case as example.
Tests can be executed with eg:

$ tox

Or by running the usual:

$ ./run-make-check.sh

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-11-08 17:05:57 +01:00
Thomas Bechtold
4258c4772a ceph-daemon: Move ceph-daemon executable to own directory
Moving ceph-daemon into src/ceph-daemon/ makes it simpler to add extra
code (eg. tox.ini, README, unittests, ...) specific to ceph-daemon.
That way related files are in a single directory.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-11-08 17:05:57 +01:00
Ricardo Marques
1a9cd92d23 mgr/dashboard: RBD tests must use pools with power-of-two pg_num
Fixes: https://tracker.ceph.com/issues/42709

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-11-08 15:43:58 +00:00
Sage Weil
457b9560a3 Merge PR #31477 into master
* refs/pull/31477/head:
	ceph-daemon: unmount osd data dir during `adopt`

Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-08 09:15:50 -06:00
Nathan Cutler
dc73147240 ceph-backport.sh/interactive_setup: abort if Step 3 fails
Fixes confusing error message reported by a user:

Setup step 3 of 4 - remote repos
---------------------------------------------------------------------
Searching "git remote -v" for remote repos

ceph-backport.sh: ERROR: Cannot auto-determine fork remote
ceph-backport.sh: (Could not find GitHub user callithea's fork of ceph/ceph in
"git remote -v")
ceph-backport.sh: ERROR: (internal error) fork_remote not set, even after
completing Steps 1-3 of interactive setup
ceph-backport.sh: This could be reported as a bug!

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-11-08 15:14:10 +01:00
Ricardo Marques
2e59781100 mgr/dashboard: Manage PG autoscaling
Fixes: https://tracker.ceph.com/issues/38227

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-11-08 13:53:14 +00:00
Sage Weil
a5eb253105 Merge PR #31478 into master
* refs/pull/31478/head:
	ceph-daemon: add --skip-pull
	ceph-daemon: add explicit pull at bootstrap start

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-08 07:22:59 -06:00
Sage Weil
3e9a2d8065 Merge PR #31177 into master
* refs/pull/31177/head:
	doc: update with zone bucket and straw2 addition

Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-08 07:22:05 -06:00
Sage Weil
ef97d90b95 Merge PR #31476 into master
* refs/pull/31476/head:
	ceph-daemon: make mon container privileged

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
2019-11-08 07:11:20 -06:00
Sage Weil
60c891d48e ceph-daemon: add --skip-pull
It occurs to me there might be cases where the user *doesn't* want to pull
the latest image (e.g., because it is a partially disconnected enviroment,
and they know the image is already in the local registry).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-08 07:10:53 -06:00
Sage Weil
0b7956c6e5 Merge PR #31464 into master
* refs/pull/31464/head:
	ceph-daemon: help users find the shell/CLI too
	ceph-daemon: enable the dashboard during bootstrap
	ceph-daemon: add CLI helper to bootstrap

Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-08 07:08:46 -06:00
Jan Fajerski
037fe019b4
Merge pull request #31479 from rishabh-d-dave/cv-dd-sync
ceph-volume: use fsync for dd command
2019-11-08 14:01:29 +01:00
Jan Fajerski
01904346a9
Merge pull request #29041 from jan--f/c-v-too-small-disks-unavailable
ceph-volume: reject disks smaller then 5GB in inventory
2019-11-08 13:52:45 +01:00
Jason Dillaman
4a6c1ec14b
Merge pull request #31260 from trociny/wip-42405
rbd-mirror: removing non-primary trash snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-11-08 07:35:41 -05:00
Jason Dillaman
97db9473fc
Merge pull request #31188 from zy751713126/rbd_features
librbd: features converting bitmask and string API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-11-08 07:35:05 -05:00
Sebastian Wagner
1e9be56d49
Merge pull request #31457 from jschmid1/ssh_orch_dev_env
mgr/ssh: update ssh-orch bootstrap guide (Vagrantfile & docs)

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-11-08 13:20:49 +01:00
Lenz Grimmer
60406199b4
mgr/dashboard: do not show non-pool data in pool details (#31456)
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-11-08 11:36:22 +00:00
Lenz Grimmer
9880140ced
mgr/dashboard: Fix grafana dashboards (#31219)
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2019-11-08 11:08:11 +00:00
Lenz Grimmer
178ea7b854
Merge pull request #31189 from bk201/wip-42075
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-11-08 10:58:38 +00:00
Sebastian Wagner
3b31eea7fd
Merge pull request #31021 from sebastian-philipp/python-common-make-drive-selection
python-common: Add drive selection

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-11-08 11:20:50 +01:00
Nathan Cutler
556cfea4b4 ceph-backport.sh: increment version number
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-11-08 08:20:04 +01:00
Nathan Cutler
51250372b9 ceph-backport.sh: fix interactive setup brokenness
A user did "ceph-backport.sh --setup" and entered the interactive setup routine.
After entering a valid GitHub personal access token, she got the following
message:

ceph-backport.sh: my GitHub username is callithea
ceph-backport.sh: You must provide a valid GitHub personal access token
ceph-backport.sh: ERROR: problem detected in your setup
ceph-backport.sh: Run "ceph-backport.sh --setup" to fix

and the script aborted. This commit fixes the issue.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-11-08 08:20:04 +01:00
Nathan Cutler
1df9d5547b ceph-backport.sh: allow cherry-picking from non-master (via --force)
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-11-08 08:20:04 +01:00
Nathan Cutler
7b40d1a83d ceph-backport.sh --existing-pr: do not clobber existing labels
Fixes: https://tracker.ceph.com/issues/42657
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-11-08 08:20:00 +01:00
Nathan Cutler
50b21702b7 ceph-backport.sh: --existing-pr ... --force should update PR body
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-11-08 08:19:48 +01:00