Commit Graph

199 Commits

Author SHA1 Message Date
Lenz Grimmer
7fe4820c60
Merge pull request #25791 from rhcs-dashboard/36740-pool-stats-read-write-columns
mgr/dashboard: add columns to Pools table

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2019-01-10 09:58:05 +01:00
alfonsomthd
d9404d6813 mgr/dashboard: add columns to Pools table
* Added columns:
Usage, Read bytes, Write bytes, Read ops, Write ops.

* TableKeyValueComponent:
Bugfix in method: _insertFlattenObjects

* Updated jest global mock:
Now window.getComputedStyle returns an object
that has getPropertyValue defined
(otherwise jest pool tests involving chart.js threw error). See:
https://developer.mozilla.org/en-US/docs/Web/API/Window/getComputedStyle

Fixes: https://tracker.ceph.com/issues/37717
Fixes: https://tracker.ceph.com/issues/34320

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-09 08:59:47 +01:00
alfonsomthd
dcf46cfe94 mgr/dashboard: updated health API test
Updated test_full_health to reflect changes in
mgr_map / mon_status payloads.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-07 12:49:25 +01:00
Ricardo Marques
695976705f
Merge pull request #25210 from dillaman/wip-dashboard-rbd-mirroring
dashboard: support configuring block mirroring pools and peers
2019-01-04 10:39:09 +00:00
Kefu Chai
407750d16e
Merge pull request #25761 from tchaikov/wip-qa-health-schema
mgr/dashboard: updated API test to reflect changes in ModuleInfo

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-03 23:15:45 +08:00
Kefu Chai
9a92825489
Merge pull request #25635 from rhcs-dashboard/36740-pool-stats-disabled-by-default
mgr/dashboard: pool stats not returned by default

Reviewed-by: Volker Theile <vtheile@suse.com>
2019-01-03 22:45:54 +08:00
Kefu Chai
486c6a48d9 qa/mgr: add "module_options" to module_info_schema
it was introduced in 280c23ba. so update the schema test accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-03 20:44:46 +08:00
Kefu Chai
b0a83097eb qa/mgr: extract module_info_schema and reused it
both standby mgr and active mgr have this sub-schema. so extract it out
and reuse it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-03 11:01:04 +08:00
Kefu Chai
a0352c83fe
Merge pull request #25653 from rhcs-dashboard/37717-update-health-test
mgr/dashboard: updated API test

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-31 22:30:13 +08:00
alfonsomthd
4ac2e3f864 mgr/dashboard: updated health controller & API test
Updated health controller & test to reflect changes introduced
in 'df' payload.

Return 'total_used_raw_bytes' instead of 'total_used_bytes'
to match CLI 'bin/rados df' used/avail summary in
Landing Page (frontend component).

Do not return 'stats_by_class' to save bandwidth as they are
not needed (right now) in the dashboard.

Fixes: https://tracker.ceph.com/issues/37717

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-24 14:45:34 +01:00
alfonsomthd
1d8bb72683 mgr/dashboard: pool stats not returned by default
* All pool controller methods with same default value
for stats flag.
* Stats requested explicitly by frontend service.
* Updated API tests accordingly.

Fixes: https://tracker.ceph.com/issues/36740

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-24 08:48:38 +01:00
Sage Weil
f3e4e56fa5 Merge PR #25618 into master
* refs/pull/25618/head:
	mgr/dashboard: Updated API pool tests

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-22 16:04:55 -06:00
Sebastian Wagner
933b2cfc28 mgr/orchestrator: Add test orchestrator
1. To be able to run the cli without an external orchestrator.
2. Run the CLI in Teuthology.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-12-20 10:56:49 +01:00
alfonsomthd
5822effd1f mgr/dashboard: Updated API pool tests
Updated API pool tests
as stats are enabled by default when retrieving the pool list
since:
https://github.com/ceph/ceph/pull/25489

Fixes: https://tracker.ceph.com/issues/37704

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-18 18:13:42 +01:00
Jason Dillaman
fa03702009 mgr/dashboard: backend REST APIs for block mirroring management
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-12-13 12:38:47 -05:00
Lenz Grimmer
2dd184346a
Merge pull request #25120 from Devp00l/wip-ecp-api-test-failure
mgr/dashboard/qa: Fix ECP creation test

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-10 17:25:10 +01:00
Lenz Grimmer
a8d23bc970
Merge pull request #25363 from zmc/wip-fix-dash-ecp-test
mgr/dashboard: Fix a broken ECP controller test

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-10 16:19:05 +01:00
Stephan Müller
19b039c28e mgr/dashboard/qa: Fix ECP creation test
The current solution fails on our CI-system as some outputs can have
more values and some parameters like 'w' can vary in different
environments.

As this was only tested before in a vstart cluster environment it
worked.

Through this commit only the given attributes we know to be there,
will be tested.

Fixes: https://tracker.ceph.com/issues/37275
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-12-10 12:37:03 +01:00
Igor Fedotov
1991b70d3a qa/mgr: rename raw_bytes_used field reference.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Tatjana Dehler
f71e4f15e5 mgr/dashboard: add config option bulk_set
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-12-04 15:15:19 +01:00
Zack Cerza
b22d8106d1 mgr/dashboard: Fix a broken ECP controller test
The change in b3e69a9609 broke the test's assumption that the endpoint
wouldn't be readable by block-manager. It doesn't looks as though that's
actually problematic for the ECP controller, so just update the test to
use rgw-manager instead.

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-30 18:00:03 -07:00
James Page
20aa3ad17b Correct usage of collections.abc
Some classes should still be imported directly from collections;
only OrderedDict, Iterable and Callable (in the context of the
ceph codebase) are found in collections.abc.

The current code works due to the fallback support for Python 2.

Signed-off-by: James Page <james.page@ubuntu.com>
2018-11-29 09:48:43 +00:00
Zack Cerza
50b7d42fe5 mgr/dashboard: Replace dashboard service
This splits out the collection of health and log data from the
/api/dashboard/health controller into /api/health/{full,minimal} and
/api/logs/all.

/health/full contains all the data (minus logs) that /dashboard/health
did, whereas /health/minimal contains only what is needed for the health
component to function. /logs/all contains exactly what the logs portion
of /dashboard/health did.

By using /health/minimal, on a vstart cluster we pull ~1.4KB of data
every 5s, where we used to pull ~6KB; those numbers would get larger
with larger clusters. Once we split out log data, that will drop to
~0.4KB.

Fixes: http://tracker.ceph.com/issues/36675

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 16:08:53 -07:00
Zack Cerza
a90b5ac7c7 Allow specifying a schema for unknown JObj keys
Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 12:47:26 -07:00
Sebastian Wagner
4fe1344696 qa/mgr/dashboard: Fix type annotation error.
Also removed unused import

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-11-27 11:45:22 +01:00
Sage Weil
630a36a509 Merge PR #24798 into master
* refs/pull/24798/head:
	fix python collections module warning for v3.7 and above

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-26 11:25:19 -06:00
Lenz Grimmer
60d4fa608e
Merge pull request #25221 from tspmelo/wip-37354
mgr/dashboard: Fix test_remove_not_expired_trash qa test

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-11-23 13:03:45 +01:00
Lenz Grimmer
dc8237e94d
Merge pull request #24757 from votdev/issue_36480
mgr/dashboard: Can't handle user editing when tenants are specified

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-11-23 12:36:26 +01:00
Tiago Melo
0f6e8e5f3d mgr/dashboard: Fix test_remove_not_expired_trash qa test
Fixes: http://tracker.ceph.com/issues/37354

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-11-22 11:39:24 +00:00
Volker Theile
945e790cab mgr/dashboard: Can't handle user editing when tenants are specified
Fixes: https://tracker.ceph.com/issues/36480

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-11-20 09:07:33 +01:00
Patrick Nawracay
09f2bb4c9a mgr/dashboard: Fix failing QA test: test_safe_to_destroy
The behavior of `safe-to-destroy` has changed in
432f194355 (PR#24799) and the backend
needs to be adapted accordingly.

Fixes: http://tracker.ceph.com/issues/37290

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-11-19 14:02:14 +01:00
hsiang41
9799eb67eb mgr: Separate diskprediction cloud plugin from the diskprediction plugin
Separate diskprediction local cloud from the diskprediction plugin.
Devicehealth invoke device prediction function related on the global
configuration "device_failure_prediction_mode".

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
2018-11-16 00:15:41 -06:00
Lenz Grimmer
5b0a957961
Merge pull request #24627 from Devp00l/wip-manage-ec-profiles
mgr/dashboard: Adds ECP management to the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 16:29:43 +01:00
Lenz Grimmer
3ba874004e
Merge pull request #24489 from ricardoasmarques/wip-saml2
mgr/dashboard: SSO - SAML 2.0 support

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 13:11:22 +01:00
Stephan Müller
3a7f85809a mgr/dashboard: Adds ECP info endpoint
The new info endpoint will provide the frontend with the necessary
information it needs to create new profiles.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-11-09 09:40:39 +01:00
Ricardo Marques
04f4d5053e mgr/dashboard: SAML 2.0 support
Fixes: https://tracker.ceph.com/issues/24268

Signed-off-by: Ricardo Dias <rdias@suse.com>
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-11-08 15:27:37 +00:00
Volker Theile
e6f130d470 mgr/dashboard: backend api tests: tasks.mgr.dashboard.test_osd.OsdTest failures
- Fix bug in Dashboard QA unit test framework. Don't set the application type header manually, this is done by the requests library if required.
- Enhance QA unit test helper: Print the response of the API request if it fails. This should help to identify the problem more easily.
- Fix bug in the OSD controller. A parameter needs to be converted to integer.
- Take care that the params of the request object are not modified.

The issue was introduced by PR https://github.com/ceph/ceph/pull/24475. The CherryPy json_in plugin disclosed the errorneous unit test helper implementation.

Fixes: https://tracker.ceph.com/issues/36708

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-11-07 17:36:53 +01:00
Lenz Grimmer
88719afc73
Merge pull request #22833 from rjfd/wip-dashboard-jwt
mgr/dashboard: JWT authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-10-30 10:50:20 +01:00
Ricardo Dias
2f5e7c3392
mgr/dashboard: backend: JWT based authentication
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-10-29 15:47:14 +00:00
Ganesh Maharaj Mahalingam
625868974b fix python collections module warning for v3.7 and above
Python 3.7 now shows a warning as below.

/usr/bin/ceph:128: DeprecationWarning: Using or importing the ABCs from
'collections' instead of from 'collections.abc' is deprecated, and in
3.8 it will stop working
  import rados

This patch addresses the that particular issue.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2018-10-28 23:32:22 -07:00
Zack Cerza
03de8f9557 mgr/dashboard: Enable gzip compression
This is related to http://tracker.ceph.com/issues/36453. It is far from
a complete solution, but seems like a positive move.

I tested this change by first disabling my browser cache, and then used
the /docs endpoint to query /api/dashboard/health. Before compression:
  Content-Length: 60748
  Time: 615ms
After:
  Content-Length: 7505
  Time: 92ms

Then, I logged into the dashboard as normal and reloaded the page once I
was in. Some values for the reload operation before compression:
  Total page load time: 58.48s
  vendor.js Content-Length: 6486025
  vendor.js time: 48.09s
After:
  Total page load time: 14.55s
  vendor.js Content-Length: 1143178
  vendor.js time: 4.50s

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-10-24 16:04:37 -06:00
Zack Cerza
bd09bc0462 DashboardTestCase: add assertHeaders()
Signed-off-by: Zack Cerza <zack@redhat.com>
2018-10-24 16:04:37 -06:00
Patrick Nawracay
5c0c122597 mgr/dashboard: Fix Python3 issue
Which results in a 500 error when trying to access the `Performance
Counter` tab on the OSD list.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-10-17 14:02:12 +02:00
John Spray
5a39f0a9aa
Merge pull request #23970 from noahdesu/mgr-always-on-mods-v2
mgr: always on modules v2

Reviewed-by: John Spray <john.spray@redhat.com>
2018-10-10 12:17:41 +01:00
John Spray
17778d217c
Merge pull request #24239 from votdev/python_ceph_audit_log
mgr: Add ability to trigger a cluster/audit log message from Python

Reviewed-by: John Spray <john.spray@redhat.com>
2018-10-10 11:47:49 +01:00
Ricardo Marques
13af4b52f7
Merge pull request #21614 from Devp00l/pool-creation-form
mgr/dashboard: Pool management

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-10-09 17:29:09 +01:00
Stephan Müller
8e56421bcd mgr/dashboard: Update PG update test
It is now commented out like it was before,
but I've added a comment what happened during this test with the QA
system. The problem was that even with only a increase of 1 PG the QA
cluster went into a cluster warning state and did not recover in time.
The QA coverage timeout is 2 minutes.

I could not reproduce this behavior with a local cluster, but I've
added a loop to wait until pgp and pg number are equal and the cluster
is in a healthy state again. This can take locally about 5 seconds.
The internal loop has a timeout of 3 minutes.

Fixes: https://tracker.ceph.com/issues/36362
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-10-09 15:56:54 +02:00
Stephan Müller
c98c8055c2 mgr/dashboard: Unset compression arguments for pools
The dashboard backend can now unset all set compression arguments if the
compression mode is switched to 'unset'. In the case of 'unset' Ceph
itself will only delete the 'compression_mode' argument, not all other
set arguments. The other arguments that should be removed, too, are
added to the update arguments in order to delete all set arguments.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-10-09 15:56:47 +02:00
Tatjana Dehler
18d91e2419 mgr/dashboard: add backend method to create config options
Fixes: http://tracker.ceph.com/issues/24455

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-10-09 14:50:00 +02:00
Tatjana Dehler
2a630be872 mgr/dashboard: refactor '_get_mon_allow_pool_delete_config'
Refactor '_get_mon_allow_pool_delete_config' method to be a little bit
more general. The method can now be used to get the value of every
config option known to the cluster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-10-09 14:47:24 +02:00