Commit Graph

127529 Commits

Author SHA1 Message Date
Samuel Just
92d0a7ca3d
Merge pull request #43592 from rzarzynski/wip-crimson-mount_ertr
crimson: errorate the FuturizedStore::mount() paths.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-10-19 21:55:23 -07:00
Neha Ojha
e70c7de4b5
Merge pull request #42839 from giladmaya/wip-on-wire-compression-documentation
doc: Added msgr on-wire-compression

 Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-19 14:55:16 -07:00
Neha Ojha
1a2403afc1
Merge pull request #43577 from ThomasLamprecht/docs-glossary-typo
docs: fix case and spelling of Proxmox VE

Reviewed-by: Laura Flores <lflores@redhat.com>
2021-10-19 14:54:10 -07:00
Radoslaw Zarzynski
677bce0e97 crimson: errorate the FuturizedStore::mount() paths.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-19 20:11:55 +00:00
Sage Weil
9e70065a73 Merge PR #43556 into master
* refs/pull/43556/head:
	qa/suites/orch/cephadm/osds: add osd rm/zap/add test

Reviewed-by: Cory Snyder <csnyder@iland.com>
2021-10-19 16:04:36 -04:00
Radoslaw Zarzynski
f94cab0c53 crimson/os: move singleton_ec up to let mount() reuse it.
No other changes in this commit.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-19 19:25:00 +00:00
Sage Weil
a4fef1dabd Merge PR #43569 into master
* refs/pull/43569/head:
	mgr/orchestrator: clean up 'orch device ls' output

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 10:58:31 -04:00
Casey Bodley
9b806156f5
Merge pull request #43486 from cbodley/wip-rgw-maintainers
rgw: start on MAINTAINERS.md

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-10-19 08:53:01 -04:00
Sebastian Wagner
a755ffa678
Merge pull request #43514 from mgfritch/cephadm-agent-binary
cephadm: use the current cephadm binary for the agent

Reviewed-by: Adam King <adking@redhat.com>
2021-10-19 13:33:43 +02:00
Ernesto Puerta
005b5b4b18
Merge pull request #43465 from rhcs-dashboard/radosgw-grafana-dashboards
mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-19 13:30:06 +02:00
Gabriel BenHanokh
d1f5ca7d8d remove allocation file after rollback to real-fm
Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
2021-10-19 12:48:14 +03:00
Gabriel BenHanokh
35b4dd06bf bug-fix close db when umount failed to store allocation file
Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
2021-10-19 12:48:12 +03:00
Gabriel BenHanokh
585d3f6636 Bug-Fix: When restoring allocation from file use a temp allocator and only copy the allocation to the shared-allocator after the file was verified and all extents were cleared
Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
2021-10-19 12:48:07 +03:00
Sebastian Wagner
84d37ec78f
Merge pull request #43370 from liewegas/fix-rook-rgw-multisite
mgr/rook: describe_services: name rgw svc appropriately

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 11:34:46 +02:00
Sebastian Wagner
f1b93ec882
Merge pull request #43550 from liewegas/rook-nvme
qa/suites/orch/rook/smoke: use nvme-loop

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 11:30:40 +02:00
Ernesto Puerta
f5fddd6121
Merge pull request #42526 from liewegas/dashboard-nfs
mgr/dashboard: consume mgr/nfs

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-19 11:17:17 +02:00
Pere Diaz Bou
be88fca941 cephadm/box: improve robustness of setup_loop.sh
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-19 09:44:39 +02:00
Aashish Sharma
ed954b0e6c mgr/dashboard: monitoring: grafonnet refactoring for cephfs dashboards
This PR intends to refactor cephfs dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:36:31 +05:30
Aashish Sharma
e490e2f3ab mgr/dashboard: monitoring: grafonnet refactoring for osds dashboards
This PR intends to refactor osds dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:13:50 +05:30
Aashish Sharma
8c48821c21 mgr/dashboard: monitoring: grafonnet refactoring for pools dashboards
This PR intends to refactor pools dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:10:56 +05:30
Aashish Sharma
e737aaa000 mgr/dashboard: monitoring: grafonnet refactoring for rbd dashboards
This PR intends to refactor rbd dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:09:04 +05:30
Aashish Sharma
eb01954cd9 mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards
This PR intends to refactor radosgw dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 11:57:28 +05:30
Joseph Sawaya
22631ec76d mgr/rook: implement orch device zap in rook orchestrator
This commit implements orch device zap by creating a pod on the target
host that mounts the /dev directory and runs either overwrites the first
few blocks of the device with zeros if it's a raw device or if it's not
a raw device it will use `ceph-volume lvm zap`.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-10-18 15:06:03 -04:00
Sage Weil
96319a3d39 qa/suites/orch/cephadm/osds: add osd rm/zap/add test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 14:05:24 -05:00
Sage Weil
c576184cd8 Merge PR #43504 into master
* refs/pull/43504/head:
	mgr/nfs: add some test coverage for cluster info, cluster config
	mgr/nfs: add 'nfs cluster config get'

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-18 15:03:33 -04:00
Sage Weil
4a63e3eafd Merge PR #43510 into master
* refs/pull/43510/head:
	qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
	mgr/cephadm: make upgrade ls output structured
	mgr/cephadm: add 'orch upgrade ls' to list available versions

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:03:24 -04:00
Sage Weil
75b2cc5c1b Merge PR #43549 into master
* refs/pull/43549/head:
	mgr/rook: fix mds service list

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-18 13:56:39 -04:00
Casey Bodley
50e4f2b702 rgw: add src/rgw/MAINTAINERS.md
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-18 12:51:21 -04:00
Casey Bodley
1aee987ec8 rgw: fix lock scope in ObjectCache::get()
in the touch_lru() case, we promote the shared_lock to a unique_lock.
but because the unique_lock is in a nested scope, the lock drops with
its scope and we continue accessing the map without any protection

this moves the unique_lock up to function scope, where it's
constructed as unlocked with std::defer_lock. after promotion, this
lock will be held until the function returns

Fixes: https://tracker.ceph.com/issues/52800

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-18 12:04:26 -04:00
Sebastian Wagner
c1c1e961f6
mgr/cephadm: test: Move test_monitoring_ports to TestMonitoring
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 17:47:18 +02:00
Sebastian Wagner
f4c1544633
mgr/cephadm: Monitoring: Verify config file generation
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 17:47:18 +02:00
Ernesto Puerta
19535b1d0e
Merge pull request #43469 from rhcs-dashboard/hosts-grafana-dashboards
mgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-18 17:14:03 +02:00
Pere Diaz Bou
e5e4e6569d cephadm/box: cleanups
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 17:12:55 +02:00
Pere Diaz Bou
87aeb1c8ae cephadm/box: add cluster ssh key to hosts
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 17:12:55 +02:00
Pere Diaz Bou
772db8c1b6 cephadm/box: create osds with cephadm and cleanups
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 17:12:55 +02:00
Pere Diaz Bou
6f87be64f1 cephadm/box: Wrap ceph docker and add some util scripts
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 17:12:55 +02:00
Ernesto Puerta
b787fa4c75 cephadm: add cephadm sandbox (Docker in Docker)
Fixes:
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-10-18 17:12:55 +02:00
Sage Weil
8900775841 qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 10:05:32 -04:00
Sebastian Wagner
406734a142
mgr/cephadm: Monitoring: Prometheus: Fix IPv6
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:05:43 +02:00
Sebastian Wagner
d65ccf3a0f
mgr/cephadm: Monitoring: Alertmanager: Fix IPv6
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:05:27 +02:00
Sebastian Wagner
bb2cb3ea3a
mgr/cephadm: Monitoring: Grafana: Fix IPv6
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:04:46 +02:00
Ilya Dryomov
cf8b6dc972
Merge pull request #42760 from ideepika/wip-iscsi-testing
qa/suites/rbd: switch iscsi tests to cephadm

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-18 14:25:36 +02:00
Sebastian Wagner
cd2a476fa1
Merge pull request #43564 from sebastian-philipp/doc-cephadm-host-location
doc/cephadm: Hosts: Add "initial CRUSH location"

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-18 13:58:55 +02:00
Alfonso Martínez
58a6ab2147 mgr/dashboard: NFS exports: API + UI: integration with mgr/nfs; cleanups
mgr/dashboard: move NFS_GANESHA_SUPPORTED_FSALS to mgr_module.py

Importing from nfs module throws AttributeError because as a side effect the dashboard module is impersonating the nfs module.
https://gist.github.com/varshar16/61ac26426bbe5f5f562ebb14bcd0f548

mgr/dashboard: 'Create NFS export' form: list clusters from nfs module

mgr/dashboard: frontend+backend cleanups for NFS export

Removed all code and references related to daemons. UI cleanup and adopted unit-testing for
nfs-epxort create form for CEPHFS backend. Cleanup for export list/get/create/set/delete endpoints.

mgr/dashboard: rm set-ganesha ref + update docs

Remove existing set-ganesha-clusters-rados-pool-namespace references as
they are no longer required. Moreover, nfs doc in dashboard doc is
updated accordingly to the current nfs status.

mgr/dashboard: add nfs-export e2e test coverage

mgr/dashboard: 'Create NFS export' form: remove RGW user id field.

- Improve bucket typeahead behavior.
- Increase version for bucket list endpoint.
- Some refactoring.

mgr/dashboard: 'Create NFS export' form: allow RGW backend only when default realm is selected.

When RGW multisite is configured, the NFS module can only handle buckets in the default realm.

mgr/dashboard: 'Create service' form: fix NFS service creation.

After https://github.com/ceph/ceph/pull/42073, NFS pool and namespace are not customizable.

mgr/dashboard: 'Create NFS export' form: add bucket validation.

- Allow only existing buckets.
- Refactoring:
  - Moved bucket validator from bucket form to cd-validators.ts
  - Split bucket validator into 2: bucket name validator and bucket existence (that checks either existence or non-existence).

mgr/dashboard: 'Create NFS export' form: path validation refactor: allow only existing paths.

Fixes: https://tracker.ceph.com/issues/46493
Fixes: https://tracker.ceph.com/issues/51479
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 12:58:54 +02:00
Thomas Lamprecht
023c32c99b docs: fix case and spelling of Proxmox VE
There's no such thing as ProxMox, the hypervisor product with build
in ceph server management and RBD client access is named `Proxmox VE`
and the company behind it is named Proxmox Server Solutions GmbH

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-18 12:10:26 +02:00
Varsha Rao
64178e1f5b mgr/dashboard: directly use ExportMgr and NFSCluster objects
Using the objects directly provides access to other methods and helps in
avoiding repeatition.

mgr/dashboard/nfsganesha: remove tag

Since NFS v3 is no longer supported. We can remove tag.

mgr/nfs: define global constant to list supported FSALs

mgr/dashboard: directly list nfs clusters by directly importing available_cluster() method

The current dashboard api returns a list of following dictionary

{
   'pool': 'nfs-ganesha',
   'namespace': cluster_id,
   'type': 'orchestrator',
   'daemon_conf': None
}

None of these values are required for listing nfs cluster by mgr/nfs module.
Instead directly list available cluster names

mgr/dashboard: add comment to remove listing of daemons

As the configs are per cluster. There is no need to list daemons per cluster.

mgr/dashboard/controllers/nfsganesha: Add comments to update/remove status endpoint

This endpoint can be updated in suggested way or even removed. As it was
initially[1] introduced to check if dashboard pool and namespace configuration was
set.

[1] 824726393b

mgr/nfs: remove fetch_cluster_obj()

There is no need to fetch NFSCluster class object. Directly
available_clusters() can be imported to list nfs clusters.

mgr/dashboard/controllers/nfsganesha: list exports based on cluster id

As mgr/nfs module lists based on cluster id.

mgr/dashboard/nfs: get and delete export by export id

Fixes: https://tracker.ceph.com/issues/46493
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-10-18 12:00:53 +02:00
Sage Weil
6e5a1eefd0 mgr/dashboard: consume mgr/nfs via mgr.remote()
Stop using the dashboard version of the Ganesha config classes; consume
mgr/nfs instead via remote().

mgr/nfs/export: return Export from _apply_export

Future callers will want this.

mgr/nfs: new module methods for dashboard consumption

Add some new methods that are easy for the dashboard API to consume.  These
are very similar to the CLI methods but do now have the @CLICommand and
related decorators, and have slightly different interfaces (e.g., returning
the created/modified Export dict).

mgr/dashboard: remove old ganesha code (and tests)

Fixes: https://tracker.ceph.com/issues/46493
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 11:58:35 +02:00
Deepika Upadhyay
b09c6ca2cc src/test/cli-integration: correct hostname for iscsi gateway
iscsi accepts first word of hostname as local hostname[0] instead of
smithi061.front.sepia.ceph.com we use: smithi061 otherwise gateway
creation will error out with:
```The first gateway defined must be the local machine```

[0] ff3e430b70/gwcli/utils.py (L36)

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
3e9b8c8f8b src/test/cli-integration/rbd: update gwcli_* to use podman exec
* since iscsi now resides inside the container, for testing we use the
  hack:
  - find the iscsi container/
  - use found container and podman exec command to test gateway creation
    deletion from host itself

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
cbd2c71398 qa/tasks: adapt ceph_iscsi.py task to ceph_iscsi_client
* we use setup_iscsi_client.py to deploy iscsi client services,
  configuring intiator and mulitpath this is done by qa task
  ceph_iscsi_client
* qa/cephadm: adds remotes ip addresses to iscsi gateway,
* rename poolname: iscsi >> datapool, which we usually use for tests and
  expresses type of pool more clearly.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30