Commit Graph

1897 Commits

Author SHA1 Message Date
Sage Weil
bb126cf9ab qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests
The ps output names daemons like 'type.foo', e.g., 'mgr.x'.  Now that
the test_orchestrator impl is less bonkers this needs to be adjusted to
match reality.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 08:00:42 -06:00
Kiefer Chang
55adeb93b3 mgr/dashboard: list services and daemons
- Display services and daemons in the cluster/services page.
- Display daemons in the cluster/hosts/host-detail page (Daemons tab).

This PR also partially addresses https://tracker.ceph.com/issues/43165:
The endpoint `/api/orchestrator/service` is removed.

Create new endpoints:
  - `/api/service`: listing all services in the Ceph cluster.
  - `/api/service/<service_name>/daemons`: listing daemons for a
    service. e.g. daemons of OSD.
  - `/api/host/<hostname>/daemons`: listing daemons of a host.

Fixes: https://tracker.ceph.com/issues/44221

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-02 10:20:33 +08:00
Sage Weil
d3305e94d1 Merge PR #33541 into master
* refs/pull/33541/head:
	mgr/test_orchestrator: add daemon_action
	qa/tasksmgr/test_orchestrator_cli: fix test_service_action
	qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
	mgr/test_orchestrator: update_foo -> apply_foo
	mgr/test_orchestrator: add force flag to remove_daemons
	qa/tasks/mgr/test_orchestrator_cli: update
	qa/tasks/mgr/test_orchestrator_cli: fix test_load_data
	mgr/test_orchestrator: fix list_daemons

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-01 16:55:06 -06:00
Sage Weil
114ef48739 qa/tasksmgr/test_orchestrator_cli: fix test_service_action
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 12:24:04 -06:00
Sage Weil
9639acfefe qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
I'm not sure what behavior we want here...

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 11:15:51 -06:00
Sage Weil
1428f54490 qa/tasks/mgr/test_orchestrator_cli: update
Most of these were broken due to CLI changes weeks ago.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-29 17:37:18 -06:00
Sage Weil
5cf9676199 qa/tasks/mgr/test_orchestrator_cli: fix test_load_data
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-27 17:15:28 -06:00
David Zafman
53158dbdd2
Merge pull request #32554 from dzafman/wip-dump-perf
Improve internal python to c++ interface

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-02-27 13:48:29 -08:00
David Zafman
4841100b4e test: Disable self-test of diskprediction_cloud since it isn't loaded
See qa/packages/packages.yaml

Signed-off-by: David Zafman <dzafman@redhat.com>
2020-02-27 13:12:45 -08:00
Sage Weil
f83f38a26e Merge PR #33552 into master
* refs/pull/33552/head:
	mgr/dashboard: Enhance user create CLI command to force password change

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-02-26 17:13:02 -06:00
Volker Theile
f1f7df3d33 mgr/dashboard: Enhance user create CLI command to force password change
Fixes: https://tracker.ceph.com/issues/44301

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-02-26 15:50:53 +01:00
Patrick Donnelly
a7d273cef7
Merge PR #31200 into master
* refs/pull/31200/head:
	qa/cephfs: test case for auto reconnect after blacklisted

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-26 06:26:13 -08:00
Patrick Donnelly
e5841cefb4
Merge PR #33263 into master
* refs/pull/33263/head:
	qa/vstart_runner.py: make run()'s interface same as teuthology's run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-25 14:45:49 -08:00
Patrick Donnelly
aebc88c31e
Merge PR #33427 into master
* refs/pull/33427/head:
	qa/cephfs: rewrite a bit of code xfstests_dev.py
	qa/cephfs: update xfstests-dev deps for RHEL 8

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-25 11:05:57 -08:00
Patrick Donnelly
1d3ed17dcb
Merge PR #32657 into master
* refs/pull/32657/head:
	test: query using mds id, not rank
	mgr: re-enable mds `scrub status` info in ceph status
	mon: filter out ceph normal ceph entity types when dumping service metadata
	mgr: filter out normal ceph services when processing service map
	mgr: helper function to check if a service is a normal ceph service

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-25 11:04:06 -08:00
Patrick Donnelly
c0c0b5a265
Merge PR #33078 into master
* refs/pull/33078/head:
	qa: avoid subtree rep in test_version_splitting

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-02-25 09:14:37 -08:00
Patrick Donnelly
0908d4812d
Merge PR #33413 into master
* refs/pull/33413/head:
	test: verify purge queue w/ large number of subvolumes
	test: pass timeout argument to mount::wait_for_dir_empty()
	mgr/volumes: access volume in lockless mode when fetching async job

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-24 18:18:19 -08:00
Sage Weil
1cb0da0005 Merge PR #33496 into master
* refs/pull/33496/head:
	mgr/cephadm: combine get_daemons_by_daemon -> get_daemons_by_service
	mgr/cephadm: remove apply_mon support
	mgr/cephadm: use generics for add_mon
	mgr/cephadm: use _apply_service for mgrs
	mgr/cephadm: refactor most daemon add methods
	mgr/cephadm: refactor _update_service and all apply methods
	mgr/cephadm: fix get_unique_name when name in use

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-02-24 14:40:19 -06:00
Patrick Donnelly
a80921307e
qa: avoid subtree rep in test_version_splitting
Root subtree may be replicated which would open client sessions early.

Fixes: https://tracker.ceph.com/issues/43796
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-24 12:28:15 -08:00
Sage Weil
3510e8eac6 Merge PR #33477 into master
* refs/pull/33477/head:
	qa/tasks/mgr/test_orchestrator_cli: fix test_host_ls

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-02-24 14:04:04 -06:00
Sage Weil
a29a78e4e9 mgr/cephadm: remove apply_mon support
Our apply method doesn't support removing mons at this point.  And using
it for adding mons is just an awkward version of 'daemon add'.

Update docs and cephadm.py task accordingly.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-24 10:46:10 -06:00
Venky Shankar
92b2008936 test: verify purge queue w/ large number of subvolumes
Fixes: http://tracker.ceph.com/issues/44207
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-02-24 04:27:02 -05:00
Venky Shankar
5ec09a228c test: pass timeout argument to mount::wait_for_dir_empty()
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-02-24 04:27:02 -05:00
Sage Weil
3e85a09ad2 Merge PR #33328 into master
* refs/pull/33328/head:
	osd/OSD: Log slow ops/types to cluster logs

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-02-23 15:12:48 -06:00
Sage Weil
5afec0fbfb Merge PR #33091 into master
* refs/pull/33091/head:
	qa/suites/rados: disable device scraping
	qa/standalone/ceph-helpers: disable device monitoring
	qa/tasks/ceph.py: add pre-mgr-commands option for ceph task
	mgr/devicehealth: set default monitoring to 'on'

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-22 12:05:55 -06:00
Sage Weil
52bb5b84ac qa/tasks/mgr/test_orchestrator_cli: fix test_host_ls
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-21 19:16:06 -06:00
Sebastian Wagner
ada3a3468d
Merge pull request #33258 from sebastian-philipp/orchestrator-get_hosts-HostSpec
mgr/orchestrator: get_hosts return `HostSpec` instead of `InventoryDevice`

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-21 16:48:08 +01:00
Lenz Grimmer
fcfe1da315
Merge pull request #32552 from Devp00l/wip-42617-new-tree
mgr/dashboard: Reload all CephFS directories

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-02-21 15:46:40 +00:00
Lenz Grimmer
54d32ce144
Merge pull request #32680 from votdev/wip-force-pwd
mgr/dashboard: Enforce password change upon first login

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-02-21 15:45:35 +00:00
Sebastian Wagner
32295039d7 qa/tasks/cephadm: orch host ls now returns hostnames
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-21 10:24:31 +01:00
Sage Weil
24f4094b73 Merge PR #33426 into master
* refs/pull/33426/head:
	mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-20 20:01:55 -06:00
Nathan Cutler
652243e71e qa/cephadm: "ls -l" cephadm after downloading it
For better debugging when the download goes awry.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-02-20 17:26:07 +01:00
Nathan Cutler
a0dd01bc5e qa/cephadm.py: better check for cephadm download success
When the URL passed to "curl --silent ..." does not exist, the
resulting file will be populated with the string:

    404: Not Found

If that (or something similar) happens, the file size will be
suspiciously low, like < 1000 bytes. Fail the test immediately in this
case.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-02-20 17:22:20 +01:00
Nathan Cutler
dcbdb28317 qa/tasks/cephadm.py: lop off .git more robustly
The line

    rest.replace('.git/', '/')

was added to accommodate weird folks who run teuthology-suite
with an option like this:

    --suite-repo https://github.com/ceph/ceph.git/

but they might just as well give the option like this:

    --suite-repo https://github.com/ceph/ceph.git

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-02-20 17:22:17 +01:00
Rishabh Dave
0c1dd0a51d qa/cephfs: rewrite a bit of code xfstests_dev.py
The variable storing the major version number plays an important role
while updating deps, therefore use a better name that is more
descriptive and makes spotting thereby easier.

Also, add an explanation for why we we have list of deps for fedora and
remove a redundant line of code.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-02-20 20:13:06 +05:30
Rishabh Dave
0f88a21267 qa/cephfs: update xfstests-dev deps for RHEL 8
btrfs-progs-devel is not availble on RHEL 8 anymore.

Fixes: https://tracker.ceph.com/issues/43964
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-02-20 20:10:11 +05:30
Rishabh Dave
061d972278 qa/vstart_runner.py: make run()'s interface same as teuthology's run
Arguments accepted by method run in teuthology should be exactly same as
that of arguments accepted by vstart_runner.py's method run to prevent
test failure with teuthology due to silly argument mismatch.
Teuthology's entry method run expects **kwargs, therefore so should
vstart_runner.py's run.

Fixes: https://tracker.ceph.com/issues/44117
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-02-20 16:01:39 +05:30
Alfonso Martínez
7cbee4d725 mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite
Fixes: https://tracker.ceph.com/issues/44201
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-02-20 10:47:38 +01:00
Sage Weil
3a10b54a6a qa/tasks/ceph.py: add pre-mgr-commands option for ceph task
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-19 15:31:26 -06:00
Sage Weil
5db4bddfaf Merge PR #33123 into master
* refs/pull/33123/head:
	qa/tasks/cephadm: deploy node-exporter
	mgr/orch: deploy node-exporter

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-02-19 14:50:20 -06:00
Sage Weil
7c9efa6bdf mgr/test_orchestrator: describe_service -> list_daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-19 11:01:13 -06:00
Sage Weil
93d53c05b0 qa/tasks/mgr/dashboard/test_orchestrator: disable services test
We need to either fill out the list_daemons APIs in the dashboard and test
that, or redesign and reimplement the services abstractions in the
orchestration layer.  Until then, disable this test.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-19 11:01:13 -06:00
Volker Theile
b37079a6fc mgr/dashboard: Enforce password change upon first login - Part 2
Introduce the following:
- A new layout component for the login pages.
- A new route called /login-change-password.
- A guard that checks if a user must change the password (ChangePasswordGuardService). If this is true, redirect to /login-change-password.
- Added LoginPasswordFormComponent (extends UserPasswordFormComponent) for the password form but (looks similar the login page).

Fixes: tracker.ceph.com/issues/24655

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-02-19 15:54:42 +01:00
Tatjana Dehler
310f787687 mgr/dashboard: Enforce password change upon first login
Fixes: https://tracker.ceph.com/issues/24655
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-02-19 15:54:42 +01:00
Sage Weil
e8520eba52 qa/tasks/cephadm: deploy node-exporter
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-19 15:24:52 +01:00
Sridhar Seshasayee
d20f57000b osd/OSD: Log slow ops/types to cluster logs
In addition to logging slow ops in mon and osd specific log files,
re-introduce logging the same information along with slow op type
details to cluster logs as well. The objective is to make debugging
slow ops easier.

Modify the log whitelisting string to "slow request" within qa suites in
order to make the search for the new warning log message within the
cluster log successful. This should not cause any issue as it's a
substring of the earlier string.

Fixes: https://tracker.ceph.com/issues/43975
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2020-02-19 14:31:48 +05:30
Sebastian Wagner
b473a84989
Merge pull request #33360 from liewegas/fix-prom
qa/suites/rados/cephadm/smoke: disable rgw role for now

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-18 15:14:03 +01:00
Sebastian Wagner
ae5d2c6ab2
Merge pull request #32817 from sebastian-philipp/rename-orchestrator_cli-orchestrator
pybind/mgr: Rename orchestrator_cli to orchestrator

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-18 10:05:30 +01:00
Venky Shankar
fb4cd42cbc test: query using mds id, not rank
Fixes: http://tracker.ceph.com/issues/42835
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-02-18 02:23:13 -05:00
Stephan Müller
e753d987e5 mgr/dashboard: Reload all CephFS directories
Now angular tree is used instead of ng2-tree, as it provides a better
way to dynamically load children and it provides a way to update all
children without losing all track of everything.

The loading icon will rotate now on any fetch.

The tree will detect new directories and removed directories.

It's also now possible to select the root directory of a CephFS in order
to create snapshots of the whole FS.

Fixes: https://tracker.ceph.com/issues/42617
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-02-17 16:35:29 +01:00