Add support for RBD Image Format v1:
- This format lacks ID field, required for dashboard. Instead,
RBD image `block_name_prefix` is used as unique ID (together with pool
id and namespace)
- Additionally, `image_format` is now exposed.
- In the front-end side:
- Copy action on a v1 image will cause the image to be copied to v2
format.
- List doesn't allow Move to Trash on v1 images,
- Details section now shows `image_format` for images,
- Edit Form disables flags not supported for v1 (`deep-flatten`,
`layering`, `exclusive-lock`).
- Protect does not work on v1 images or v2 images created from v1
ones.
Fixes: https://tracker.ceph.com/issues/36354
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Currently the max size is determined by the number of OSDs, which is
compared with the maximum of the current crush rule.
The problem with that is, that this is wrong for every crush rule that
doesn't have OSDs as failure domain and that don't have the root of the
cluster set as root of the crush rule.
Now the crush map will be used to determine how many failure domains are
really available in the cluster and how many can really be used in the
end. This number now defines the maximum size you can enter.
The crush detail view will now the new attribute usable_size and hide
the redundant information steps, ruleset, type and rule_name.
Fixes: https://tracker.ceph.com/issues/44620
Signed-off-by: Stephan Müller <smueller@suse.com>
* RGW sync perf. counters are now exposed through grafana panels.
* Sync Performance tab is only shown if rgw realm is detected.
* Prometheus module: added metrics suitable for prometheus consumption (from existing ones, not replacing for backward compatibility).
Fixes: https://tracker.ceph.com/issues/45310
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
qa/tasks/mgr: always add "application" label to pool
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
otherwise monitor complains like
```
health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': '1 pool(s) do not have an application enabled', 'count': 1}, 'detail': [{'message': "application not
enabled on pool 'dashboard_pool3'"}, {'message': "use 'ceph osd pool application enable <pool-name> <app-name>', where <app-name> is 'cephfs', 'rbd', 'rgw', or freeform for custom applications."}],
'muted': False, 'type': 'POOL_APP_NOT_ENABLED'}], 'mutes': []}
```
and if test checks for the healthy status, the test fails like:
```
2020-04-30T17:29:54.498 INFO:tasks.cephfs_test_runner: self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health))
2020-04-30T17:29:54.498 INFO:tasks.cephfs_test_runner:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK'
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
Add a telemetry component in order to give the user the
possibility to configure the telemetry module in a more
guided fashion. The component offers broader explanations,
shows a preview of the generated report and asks the user
to accept the license before enabling the module.
Fixes: https://tracker.ceph.com/issues/43956
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
Now the crush node class preselects root, failure domain and device
class in the erasure code profile modal.
Like for crush rule, now also if you try to delete an used ECP you can't
and the info box will show you, what pool is using the profile.
Fixes: https://tracker.ceph.com/issues/44621
Signed-off-by: Stephan Müller <smueller@suse.com>
in python2, dict.values() and dict.keys() return lists. but in python3,
they return views, which cannot be indexed directly using an integer index.
there are three use cases when we access these views in python3:
1. get the first element
2. get all the elements and then *might* want to access them by index
3. get the first element assuming there is only a single element in
the view
4. iterate thru the view
in the 1st case, we cannot assume the number of elements, so to be
python3 compatible, we should use `next(iter(a_dict))` instead.
in the 2nd case, in this change, the view is materialized using
`list(a_dict)`.
in the 3rd case, we can just continue using the short hand of
```py
(first_element,) = a_dict.keys()
```
to unpack the view. this works in both python2 and python3.
in the 4th case, the existing code works in both python2 and python3, as
both list and view can be iterated using `iter`, and `len` works as
well.
Signed-off-by: Kefu Chai <kchai@redhat.com>
for better readability, and to ease the pain of developer to track back
to the top level python package for referencing a submodule
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change partially reverts #34139
using relative import helps with readability and ease the pain to write
down the full parent module name
in #34139, all relative imports were replaced with full path, because we
were using following code to verify if the code is python3 compatible:
```
mod_spec = importlib.util.spec_from_file_location(mod_name, path)
mod = importlib.util.module_from_spec(mod_spec)
mod_spec.loader.exec_module(mod)
```
but this does not work with submodule which can import using relative
import without specifying the name of the package and its parent module.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/34060/head:
Merge PR #34027 into octopus
Merge PR #34045 into octopus
Merge pull request #34035 from dillaman/wip-rbd-permissions
mgr/progress: fix duration strings
Merge PR #34014 into octopus
Merge PR #34001 into octopus
Merge PR #34011 into octopus
qa/workunits/rbd: use context managers to control Rados lifespan
Merge pull request #34032 from dillaman/wip-rbd-octopus-docs
doc/releases/octopus: add additional RBD improvements
qa/workunits/cephadm/test_cephadm: mark services unmanaged for test
mgr/cephadm: do not reconfig unmanaged services
Merge PR #33981 into octopus
Merge pull request #34018 from ajarr/octopus-subvolume-clone-cancel
qa/workunits/cephadm/test_cephadm: output file for pub key
Merge PR #33866 into octopus
Merge PR #34005 into octopus
Merge PR #34013 into octopus
mgr/cephadm: pytest: Enable SpecStore
mgr/orchestrator: add test for default implementation for apply()
python-common: validate ServiceSpec.service_type
fixup mgr/cephadm: Fix ceph orch apply -i
mgr/dashbaord: orchestrator service: Revert wait_api_result to a single completion
mgr/orchestrator: `orch daemon add` accepts a yaml
mgr/cephadm: apply_drivegroups() returns a single Completion
mgr/cephadm: remove `trivial_result()`
mgr/cephadm: Fix `ceph orch apply -i`
Merge pull request #33994 from dillaman/wip-librbd-poll-event-race
doc: document `clone cancel` command
test: add `clone cancel` tests
mgr/volumes: introduce "clone cancel" volume command
mgr/volumes: allow canceling a single asynchronous job for a volume
mgr/volumes: helper for looking up a clone entry index
mgr/volumes: periodically check if clone operations should be canceled
mgr/volumes: periodically check if copy operations should be canceled
mgr/volumes: introduce 'canceled' state in clone op state machine
qa/suites/rados/verify/validater/valgrind: tolerate SLOW_OPS
qa/suites/rados/verify/validater/valgrind: less bluestore logging
qa/suites/rados/verify/validater: increase heartbeat grace
Revert "qa/suites/rados/verify: debug_ms = 1, osd_heartbeat_grace = 60"
Revert "qa/suites/rados/verify/validator/valgrind: debug refs = 5"
ceph_test_watch_notify: try notify 10x if ALLOW_TIMEOUTS is set
ceph_test_rados_api_misc: ShutdownRace timeout if ALLOW_TIMEOUTS is set
qa/suites/rados/verify: set ALLOW_TIMEOUTS for workunits
doc/install: edits
doc/cephadm: more edits
doc/cephadm/install: edits
doc/cephadm/adoption: improvements
doc/cephadm/install: a few edits
doc/cephadm/install: do not install ceph-common on host (by default)
doc/cephadm: drop os recs link
doc/cephadm/upgrade: improvements
doc/cephadm/upgrade: document upgrade
doc/cephadm/install: revamp install docs
doc: reorganize cephadm docs
doc/cephadm/administration: update docs on customizing SSH config
doc/cephadm/administration: add a note about the 'removed' dir
mgr/balancer: tolerate pgs outside of target weight map
qa/workunits/cephadm/test_cephadm: --skip-monitoring-stack
Merge PR #33974 into octopus
Merge PR #33442 into octopus
Merge PR #33997 into octopus
Merge PR #34000 into octopus
use quay octopus tip until 15.2 tag is available
python-common: reduce output of ServiceSpec.to_json()
python-common,mgr/cephadm: move assert_valid_host to service_spec
mgr/cephadm: add HostAssignment.validate()
mgr/dashboard: adapt create_osds interface change
mon/MgrMonitor: make 'mgr fail' work with no arguments
cephadm: add allow_ptrace option to enable SYS_PTRACE
update default container images
mgr/cephadm: limit number of times check host is performed in the serve loop
Merge PR #33961 into octopus
Merge PR #33952 into octopus
Merge PR #33990 into octopus
Merge PR #33955 into octopus
Merge PR #33936 into octopus
mgr/orch: add --all-available-devices to 'orch apply osd'
qa/workunits/cephadm: --skip-mon-network when using 127.0.0.1
cephadm: add tests
qa/tasks/cephadm: pass -v to bootstrap
mgr/cephadm: only try to place mons on hosts matching public_network
mgr/cephadm: keep track of host networks, ips
cephadm: automatically infer mon public_network, if we can
cephadm: add list-networks command
cephadm: bootstrap: deploy monitoring stack by default
librbd: defer event socket completion until after callback issued
cephadm: add-repo: add --version
mgr/cephadm: respect 'unmanaged' flag in spec
mgr/orch: orch ls: show <no spec> or <unmanaged> as appropriate
mgr/orch: orch ls: rename SPEC -> PLACEMENT
mgr/orch: add 'unmanaged' property to ServiceSpec
cephadm: rename distro args in repo methods
mgr/orch: combine 'orch daemon add <type> ...' into one command
mgr/orch: combine 'orch apply <type> [<placement>]' into one command
Reviewed-by: Laura Paduano <lpaduano@suse.com>
* refs/pull/33885/head:
Merge pull request #33848 from mchangir/octopus-tests-remove-suprious-whitespace
Merge PR #33746 into octopus
Merge PR #33830 into octopus
Merge PR #33732 into octopus
Merge PR #33620 into octopus
Merge pull request #33876 from tchaikov/octopus-cephadm-mypy
cephadm: add "assert foo is not None" for mypy check
Merge pull request #33067 from tspmelo/wip-rbd-delete-with-snapshot
cephadm: add grafana adopt
Merge PR #33771 into octopus
Merge PR #33850 into octopus
Merge PR #33853 into octopus
Merge PR #33857 into octopus
Merge PR #32990 into octopus
Merge PR #33713 into octopus
Merge PR #33838 into octopus
qa/tasks/cephadm: no default mon|mgr|crash service specs
qa/suites/rados/cephadm/upgrade: upgrade start point that supports the no-spec option
Merge PR #33832 into octopus
cephadm: bootstrap: wait for mgr to restart after enabling a module
mgr: add 'mgr_status' tell command
Merge pull request #33839 from rhcs-dashboard/44538-fix-rgw-grafana-get-put-latencies
Merge pull request #33743 from votdev/issue_43869_fix_qa_test
cephadm: create initial mon and mgr service specs too
cephadm: no need to pregenerate a crash key for the bootstrap host
mgr/cephadm: do not complain when we don't have enough hosts
mgr/cephadm: remove orphan daemons
mgr/cephadm: report size=0 for fabricated ServiceDescription
mgr/cephadm: safety check to prevent removing all mon|mgr daemons
mgr/cephadm: prevent scaling mon|mgr below count=1
mgr/cephadm: do not remove daemons from remove_service
Merge pull request #33805 from tchaikov/wip-44500
spec: Podman (temporarily) requires apparmor-abstractions on suse
mgr/cephadm: Make sure we don't co-locate the same daemon
monitoring: fix RGW grafana chart 'Average GET/PUT Latencies'
tests: remove spurious whitespace
mgr/cephadm: fix service list filtering
Merge PR #33825 into octopus
Merge PR #33811 into octopus
Revert "Merge pull request #33673 from cbodley/wip-denc-enum"
mgr/cephadm: fix upgrade order
Merge PR #33801 into octopus
Merge PR #33822 into octopus
cephadm: bootstrap: tolerate error return from -h
Merge PR #33809 into octopus
Merge PR #32678 into octopus
cephadm: use `sh` instead of `bash` during enter
ceph.in: only shut down rados on clean exit
common/ceph_timer: Pass reference to waited time on stack
common/ceph_timer: Add test
common/ceph_timer: Use unique_function, allowing noncopyable events
common/ceph_timer: Couple cleanups
common/ceph_timer: Fix namespaces
common/ceph_timer: Add missing includes
common/ceph_timer.h: Don't indent contents of a namespace
mgr/dashboard: Crush rule modal
mgr/dashboard: Preserve rule selection on pool type change
mgr/dashboard: Crush rule is only send during replicated pool creation
mgr/dashboard: Explicit returns in pool form
mgr/dashboard: Removes fork join in pool form
mgr/dashboard: Hide ECP actions during ec pool edit
mgr/dashboard: Pool form erasure/replicated boolean
mgr/dashboard: Change pool info API endpoint
mgr/dashboard: Moves ECP info endpoint to UI-API
mgr/cephadm: add _remove_osds_bg back to main loop
mgr/cephadm/osd: update removal report immediately
qa/tasks/ceph_manager: use StringIO for capturing COT output
qa/standalone/scrub/osd-scrub-repair: force osdmap prop to osds
qa/standalone/scrub/osd-scrub-test: wait longer for update
qa/tasks/ceph_manager: capture stderr for COT
qa/suites/rados/ceph: drop opensuse for now
mon/MonClient: send logs to mon on separate schedule than pings
mgr/dashboard: Fix missing ImageSpec usage
mgr/dashboard: Allow removing RBD with snapshots
mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user
mgr/dashboard: support multiple DriveGroups when creating OSDs
mon/MonClient: send logs to mon even if we have no keelalive2
cephadm: flag dashboard user to change password
Reviewed-by: Sebastian Wagner <swagner@suse.com>
* refs/pull/33620/head:
mgr/dashboard: Crush rule modal
mgr/dashboard: Preserve rule selection on pool type change
mgr/dashboard: Crush rule is only send during replicated pool creation
mgr/dashboard: Explicit returns in pool form
mgr/dashboard: Removes fork join in pool form
mgr/dashboard: Hide ECP actions during ec pool edit
mgr/dashboard: Pool form erasure/replicated boolean
mgr/dashboard: Change pool info API endpoint
mgr/dashboard: Moves ECP info endpoint to UI-API
Reviewed-by: Tiago Melo <tmelo@suse.com>
mgr/dashboard: Allow deletion of RBD with snapshots
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Now a crush rule can be created and deleted through the pool form,
similar to the ECP profile.
The creation form is somewhat more intelligent as it checks the crush
map to help create a usable rule, with only a few clicks
through preselections.
Fixes: https://tracker.ceph.com/issues/43260
Signed-off-by: Stephan Müller <smueller@suse.com>
Moves the "_info" endpoint of pool into an equivalent
UI-API call with the name "info".
Added three more attributes to the info dict which enables the dashboard
to only call info to get all the needed data, currently three calls will
be used to do that.
Removed pool_name parameter as the outcome was not used.
Updated the tests and related angular files accordingly.
Fixes: https://tracker.ceph.com/issues/44371
Signed-off-by: Stephan Müller <smueller@suse.com>
Moves the "_info" endpoint of erasure code profile into an equivalent
UI-API call with the name "info".
The serialization of the profile was outsourced into "ceph-service" as
it's used somewhere else (follow up commit).
Removed unused methods in angular service and REST controller.
Fixed path in angular service.
Fixes: https://tracker.ceph.com/issues/44371
Signed-off-by: Stephan Müller <smueller@suse.com>
* Improve @DashboardTestCase.RunAs decorator
* Use command line to create the test user in the 'test_pwd_expiration_date_update' test case.
* Test return codes of various REST API calls.
Fixes: https://tracker.ceph.com/issues/43869
Signed-off-by: Volker Theile <vtheile@suse.com>
