Commit Graph

450 Commits

Author SHA1 Message Date
Sage Weil
ac8f51b2e7 Merge remote-tracking branch 'gh/octopus' 2020-03-13 06:55:55 -05:00
Volker Theile
b0b188da57 mgr/dashboard: Create bucket with x-amz-bucket-object-lock-enabled
Fixes: https://tracker.ceph.com/issues/43446

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-03-12 11:56:21 +01:00
Sage Weil
603383605f Merge PR #33885 into master
* refs/pull/33885/head:
	Merge pull request #33848 from mchangir/octopus-tests-remove-suprious-whitespace
	Merge PR #33746 into octopus
	Merge PR #33830 into octopus
	Merge PR #33732 into octopus
	Merge PR #33620 into octopus
	Merge pull request #33876 from tchaikov/octopus-cephadm-mypy
	cephadm: add "assert foo is not None" for mypy check
	Merge pull request #33067 from tspmelo/wip-rbd-delete-with-snapshot
	cephadm: add grafana adopt
	Merge PR #33771 into octopus
	Merge PR #33850 into octopus
	Merge PR #33853 into octopus
	Merge PR #33857 into octopus
	Merge PR #32990 into octopus
	Merge PR #33713 into octopus
	Merge PR #33838 into octopus
	qa/tasks/cephadm: no default mon|mgr|crash service specs
	qa/suites/rados/cephadm/upgrade: upgrade start point that supports the no-spec option
	Merge PR #33832 into octopus
	cephadm: bootstrap: wait for mgr to restart after enabling a module
	mgr: add 'mgr_status' tell command
	Merge pull request #33839 from rhcs-dashboard/44538-fix-rgw-grafana-get-put-latencies
	Merge pull request #33743 from votdev/issue_43869_fix_qa_test
	cephadm: create initial mon and mgr service specs too
	cephadm: no need to pregenerate a crash key for the bootstrap host
	mgr/cephadm: do not complain when we don't have enough hosts
	mgr/cephadm: remove orphan daemons
	mgr/cephadm: report size=0 for fabricated ServiceDescription
	mgr/cephadm: safety check to prevent removing all mon|mgr daemons
	mgr/cephadm: prevent scaling mon|mgr below count=1
	mgr/cephadm: do not remove daemons from remove_service
	Merge pull request #33805 from tchaikov/wip-44500
	spec: Podman (temporarily) requires apparmor-abstractions on suse
	mgr/cephadm: Make sure we don't co-locate the same daemon
	monitoring: fix RGW grafana chart 'Average GET/PUT Latencies'
	tests: remove spurious whitespace
	mgr/cephadm: fix service list filtering
	Merge PR #33825 into octopus
	Merge PR #33811 into octopus
	Revert "Merge pull request #33673 from cbodley/wip-denc-enum"
	mgr/cephadm: fix upgrade order
	Merge PR #33801 into octopus
	Merge PR #33822 into octopus
	cephadm: bootstrap: tolerate error return from -h
	Merge PR #33809 into octopus
	Merge PR #32678 into octopus
	cephadm: use `sh` instead of `bash` during enter
	ceph.in: only shut down rados on clean exit
	common/ceph_timer: Pass reference to waited time on stack
	common/ceph_timer: Add test
	common/ceph_timer: Use unique_function, allowing noncopyable events
	common/ceph_timer: Couple cleanups
	common/ceph_timer: Fix namespaces
	common/ceph_timer: Add missing includes
	common/ceph_timer.h: Don't indent contents of a namespace
	mgr/dashboard: Crush rule modal
	mgr/dashboard: Preserve rule selection on pool type change
	mgr/dashboard: Crush rule is only send during replicated pool creation
	mgr/dashboard: Explicit returns in pool form
	mgr/dashboard: Removes fork join in pool form
	mgr/dashboard: Hide ECP actions during ec pool edit
	mgr/dashboard: Pool form erasure/replicated boolean
	mgr/dashboard: Change pool info API endpoint
	mgr/dashboard: Moves ECP info endpoint to UI-API
	mgr/cephadm: add _remove_osds_bg back to main loop
	mgr/cephadm/osd: update removal report immediately
	qa/tasks/ceph_manager: use StringIO for capturing COT output
	qa/standalone/scrub/osd-scrub-repair: force osdmap prop to osds
	qa/standalone/scrub/osd-scrub-test: wait longer for update
	qa/tasks/ceph_manager: capture stderr for COT
	qa/suites/rados/ceph: drop opensuse for now
	mon/MonClient: send logs to mon on separate schedule than pings
	mgr/dashboard: Fix missing ImageSpec usage
	mgr/dashboard: Allow removing RBD with snapshots
	mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user
	mgr/dashboard: support multiple DriveGroups when creating OSDs
	mon/MonClient: send logs to mon even if we have no keelalive2
	cephadm: flag dashboard user to change password

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-11 17:38:59 -05:00
Kefu Chai
8fd63d2384
Merge pull request #32262 from kshtsk/wip-ceph-py-py3
qa/tasks: py3 compat

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-11 23:22:04 +08:00
Sage Weil
07aba4deb9 Merge PR #33620 into octopus
* refs/pull/33620/head:
	mgr/dashboard: Crush rule modal
	mgr/dashboard: Preserve rule selection on pool type change
	mgr/dashboard: Crush rule is only send during replicated pool creation
	mgr/dashboard: Explicit returns in pool form
	mgr/dashboard: Removes fork join in pool form
	mgr/dashboard: Hide ECP actions during ec pool edit
	mgr/dashboard: Pool form erasure/replicated boolean
	mgr/dashboard: Change pool info API endpoint
	mgr/dashboard: Moves ECP info endpoint to UI-API

Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-03-11 07:12:11 -05:00
Lenz Grimmer
108cc47c49
Merge pull request #33067 from tspmelo/wip-rbd-delete-with-snapshot
mgr/dashboard: Allow deletion of RBD with snapshots

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-03-11 10:56:01 +01:00
Lenz Grimmer
1c7449414f
Merge pull request #33743 from votdev/issue_43869_fix_qa_test
mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-10 15:25:50 +01:00
Sage Weil
cf38372782 Merge PR #32678 into octopus
* refs/pull/32678/head:
	mgr/dashboard: support multiple DriveGroups when creating OSDs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-03-09 14:09:00 -05:00
Stephan Müller
f26b4b23fd mgr/dashboard: Crush rule modal
Now a crush rule can be created and deleted through the pool form,
similar to the ECP profile.

The creation form is somewhat more intelligent as it checks the crush
map to help create a usable rule, with only a few clicks
through preselections.

Fixes: https://tracker.ceph.com/issues/43260
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-03-09 12:35:58 +01:00
Stephan Müller
2ff8540c81 mgr/dashboard: Change pool info API endpoint
Moves the "_info" endpoint of pool into an equivalent
UI-API call with the name "info".

Added three more attributes to the info dict which enables the dashboard
to only call info to get all the needed data, currently three calls will
be used to do that.

Removed pool_name parameter as the outcome was not used.

Updated the tests and related angular files accordingly.

Fixes: https://tracker.ceph.com/issues/44371
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-03-09 12:35:58 +01:00
Stephan Müller
3e400f6a0b mgr/dashboard: Moves ECP info endpoint to UI-API
Moves the "_info" endpoint of erasure code profile into an equivalent
UI-API call with the name "info".

The serialization of the profile was outsourced into "ceph-service" as
it's used somewhere else (follow up commit).

Removed unused methods in angular service and REST controller.

Fixed path in angular service.

Fixes: https://tracker.ceph.com/issues/44371
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-03-09 12:35:57 +01:00
Sage Weil
51b60a212e qa/tasks/mgr/test_orchestrator_cli: adjust placement args
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-06 21:27:48 -06:00
Tiago Melo
5627919c01 mgr/dashboard: Allow removing RBD with snapshots
Fixes: https://tracker.ceph.com/issues/36404

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-03-06 13:18:15 -01:00
Kyr Shatskyy
e7473b631d qa/tasks/mgr: fix imports for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Volker Theile
4cc2e1a15b mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user
* Improve @DashboardTestCase.RunAs decorator
* Use command line to create the test user in the 'test_pwd_expiration_date_update' test case.
* Test return codes of various REST API calls.

Fixes: https://tracker.ceph.com/issues/43869

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-03-06 09:24:16 +01:00
Kiefer Chang
320bbeec11
mgr/dashboard: support multiple DriveGroups when creating OSDs
The `create_osds` call in orchestrator uses multiple named DriveGroups as
the parameter. Adapt the change in Dashboard.

Some minor polishes:
- Use task manager to wrap the operation.
- The submit button in Preview modal is changed from `Add` to `Create`.
- POST `/api/osd` to create OSDs:
  - Bare OSDs for OSD service container
    {
        "method": "bare",
        "data": {
            "uuid": "xxxx",
            "svc_id": 5
        }
    }

  - OSDs with devices (DriveGroups)

    {
        "method": "drive_groups",
        "data": {
            < drive group spec here>
        }
    }
- `/orchestrator/osd` endpoint is removed.

Fixes: https://tracker.ceph.com/issues/43615
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-06 11:15:18 +08:00
Sage Weil
8a84ddeaa8 Merge PR #33531 into master
* refs/pull/33531/head:
	qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests
	mgr/test_orchestrator: refactor listing services/daemons
	mgr/dashboard: list services and daemons

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-03-05 13:20:21 -06:00
Alfonso Martínez
eaa0d84a13 mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite
Fixes: https://tracker.ceph.com/issues/44405
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-03-05 16:22:55 +01:00
Sage Weil
bb126cf9ab qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests
The ps output names daemons like 'type.foo', e.g., 'mgr.x'.  Now that
the test_orchestrator impl is less bonkers this needs to be adjusted to
match reality.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 08:00:42 -06:00
Thomas Bechtold
f5e77561e9 qa: Fix problems detected by mypy
This is a first step to enable mypy on the qa/ directory.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-05 06:53:31 +01:00
Kefu Chai
bd3dccd6fa
Merge pull request #33709 from tchaikov/wip-ceph-py-py3-rados
qa/tasks: py3 compat (tasks exercised by rados suites)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-04 21:16:21 +08:00
Kefu Chai
00af2eddf5
Merge pull request #33690 from rhcs-dashboard/44237-remove-config-opt-read-from-system-roles
mgr/dashboard: remove 'config-opt: read' perm. from system roles.

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-03-04 20:54:42 +08:00
Kyr Shatskyy
eb10276c1c qa/tasks/mgr: get rid of itervalues for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-04 13:09:16 +08:00
Kefu Chai
e26a1cdee8 qa/tasks/test_orchestrator_cli: enable test_osd_create exception check
this reverts 9639acfefe, as the test does
make sense. what fails this test is the machinary to marshal/unmarshal
exception fails to handle un-picklable exceptions. the previous commit
is supposed to use a fallback to handle them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-04 01:52:02 +08:00
Alfonso Martínez
ad8dbefff8 mgr/dashboard: remove 'config-opt: read' perm. from system roles.
Fixes: https://tracker.ceph.com/issues/44237
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-03-03 12:34:11 +01:00
Lenz Grimmer
04e2058c6a
Merge pull request #31922 from rhcs-dashboard/42094-bucket-mfa-delete
mgr/dashboard: enable/disable MFA Delete on RGW bucket

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-02 13:55:52 +00:00
Kiefer Chang
55adeb93b3 mgr/dashboard: list services and daemons
- Display services and daemons in the cluster/services page.
- Display daemons in the cluster/hosts/host-detail page (Daemons tab).

This PR also partially addresses https://tracker.ceph.com/issues/43165:
The endpoint `/api/orchestrator/service` is removed.

Create new endpoints:
  - `/api/service`: listing all services in the Ceph cluster.
  - `/api/service/<service_name>/daemons`: listing daemons for a
    service. e.g. daemons of OSD.
  - `/api/host/<hostname>/daemons`: listing daemons of a host.

Fixes: https://tracker.ceph.com/issues/44221

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-02 10:20:33 +08:00
Sage Weil
d3305e94d1 Merge PR #33541 into master
* refs/pull/33541/head:
	mgr/test_orchestrator: add daemon_action
	qa/tasksmgr/test_orchestrator_cli: fix test_service_action
	qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
	mgr/test_orchestrator: update_foo -> apply_foo
	mgr/test_orchestrator: add force flag to remove_daemons
	qa/tasks/mgr/test_orchestrator_cli: update
	qa/tasks/mgr/test_orchestrator_cli: fix test_load_data
	mgr/test_orchestrator: fix list_daemons

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-01 16:55:06 -06:00
Sage Weil
114ef48739 qa/tasksmgr/test_orchestrator_cli: fix test_service_action
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 12:24:04 -06:00
Sage Weil
9639acfefe qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
I'm not sure what behavior we want here...

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 11:15:51 -06:00
Sage Weil
1428f54490 qa/tasks/mgr/test_orchestrator_cli: update
Most of these were broken due to CLI changes weeks ago.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-29 17:37:18 -06:00
Alfonso Martínez
5b6cf814b9 mgr/dashboard: enable/disable MFA Delete on RGW bucket
Fixes: https://tracker.ceph.com/issues/42094
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-02-28 11:35:57 +01:00
Sage Weil
5cf9676199 qa/tasks/mgr/test_orchestrator_cli: fix test_load_data
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-27 17:15:28 -06:00
David Zafman
53158dbdd2
Merge pull request #32554 from dzafman/wip-dump-perf
Improve internal python to c++ interface

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-02-27 13:48:29 -08:00
David Zafman
4841100b4e test: Disable self-test of diskprediction_cloud since it isn't loaded
See qa/packages/packages.yaml

Signed-off-by: David Zafman <dzafman@redhat.com>
2020-02-27 13:12:45 -08:00
Volker Theile
f1f7df3d33 mgr/dashboard: Enhance user create CLI command to force password change
Fixes: https://tracker.ceph.com/issues/44301

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-02-26 15:50:53 +01:00
Sage Weil
52bb5b84ac qa/tasks/mgr/test_orchestrator_cli: fix test_host_ls
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-21 19:16:06 -06:00
Lenz Grimmer
fcfe1da315
Merge pull request #32552 from Devp00l/wip-42617-new-tree
mgr/dashboard: Reload all CephFS directories

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-02-21 15:46:40 +00:00
Lenz Grimmer
54d32ce144
Merge pull request #32680 from votdev/wip-force-pwd
mgr/dashboard: Enforce password change upon first login

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-02-21 15:45:35 +00:00
Alfonso Martínez
7cbee4d725 mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite
Fixes: https://tracker.ceph.com/issues/44201
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-02-20 10:47:38 +01:00
Sage Weil
7c9efa6bdf mgr/test_orchestrator: describe_service -> list_daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-19 11:01:13 -06:00
Sage Weil
93d53c05b0 qa/tasks/mgr/dashboard/test_orchestrator: disable services test
We need to either fill out the list_daemons APIs in the dashboard and test
that, or redesign and reimplement the services abstractions in the
orchestration layer.  Until then, disable this test.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-19 11:01:13 -06:00
Volker Theile
b37079a6fc mgr/dashboard: Enforce password change upon first login - Part 2
Introduce the following:
- A new layout component for the login pages.
- A new route called /login-change-password.
- A guard that checks if a user must change the password (ChangePasswordGuardService). If this is true, redirect to /login-change-password.
- Added LoginPasswordFormComponent (extends UserPasswordFormComponent) for the password form but (looks similar the login page).

Fixes: tracker.ceph.com/issues/24655

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-02-19 15:54:42 +01:00
Tatjana Dehler
310f787687 mgr/dashboard: Enforce password change upon first login
Fixes: https://tracker.ceph.com/issues/24655
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-02-19 15:54:42 +01:00
Sebastian Wagner
ae5d2c6ab2
Merge pull request #32817 from sebastian-philipp/rename-orchestrator_cli-orchestrator
pybind/mgr: Rename orchestrator_cli to orchestrator

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-18 10:05:30 +01:00
Stephan Müller
e753d987e5 mgr/dashboard: Reload all CephFS directories
Now angular tree is used instead of ng2-tree, as it provides a better
way to dynamically load children and it provides a way to update all
children without losing all track of everything.

The loading icon will rotate now on any fetch.

The tree will detect new directories and removed directories.

It's also now possible to select the root directory of a CephFS in order
to create snapshots of the whole FS.

Fixes: https://tracker.ceph.com/issues/42617
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-02-17 16:35:29 +01:00
Lenz Grimmer
9269f550b3
Merge pull request #33013 from s0nea/wip-dashboard-43912
mgr/dashboard: wait for PG unknown state to be cleared

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
2020-02-17 15:34:11 +00:00
Sebastian Wagner
f2c5472286 mgr/orchestrator_cli: rename to mgr/orchestrator
* Move `mgr/orchestrator.py` to `orchestrator/_interface.py`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-17 10:24:01 +01:00
Kefu Chai
1d22ff29fb
Merge pull request #33347 from tchaikov/wip-orch-addr
mgr/orchestrator: "addr" is optional for constructing InventoryNode

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-15 22:16:50 +08:00
Kefu Chai
e470481d71
Merge pull request #33325 from tchaikov/wip-super-setup
qa/tasks: call super class's setUp()

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-15 18:46:51 +08:00