Commit Graph

127282 Commits

Author SHA1 Message Date
Adam C. Emerson
447564e4db build: Remove LevelDB support
Whereas new LevelDB packages are build with -fno-rtti and break our
attempts to compile against them,

and whereas LevelDB has been deprecated for some time, with the Ceph
team expending great effort to migrating OSDs off of it,

let it be therefore removed.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2021-11-01 12:42:17 -04:00
Ronen Friedman
352399d6a4
Merge pull request #43596 from ronen-fr/wip-rf-test_info
tests: modify osd-scrub-repair to match PR #43239 changes

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-10-20 18:03:22 +03:00
Neha Ojha
9fda06aa6a
Merge pull request #43572 from trociny/wip-qa-backfill-toofull-compress
qa/tasks/backfill_toofull: make test work when compression on

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-20 06:57:06 -07:00
Sebastian Wagner
a1ac0ec720
Merge pull request #42772 from Daniel-Pivonka/cephadm-add-existing-host
mgr/cephadm: update host when adding host that already exists

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-20 11:49:33 +02:00
Sebastian Wagner
2ac749e7e0
Merge pull request #43354 from adk3798/daemon-failed-agent
mgr/cephadm: update CEPHADM_FAILED_DAEMON after receiving agent metadata

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-20 11:48:42 +02:00
Sebastian Wagner
f2da4ec9b4
Merge pull request #43366 from adk3798/log-list-servers
mgr/cephadm: put list_servers output in debug logs

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-20 11:48:09 +02:00
Sebastian Wagner
8435ff07e0
Merge pull request #43367 from adk3798/handle-use-agent
mgr/cephadm: move handling of use_agent setting into agent.py

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-20 11:47:41 +02:00
Sebastian Wagner
99985b970d
Merge pull request #43489 from adk3798/clear-agent-down
mgr/cephadm: update CEPHADM_AGENT_DOWN when agent reports in

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-20 11:46:17 +02:00
Sebastian Wagner
5397d15bb8
Merge pull request #43552 from adk3798/dont-log-credentials
cephadm: do not log commands sent to "call" function

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-20 11:45:29 +02:00
Ronen Friedman
52e9fa16ef tests: modify osd-scrub-repair to match PR #43239 changes
PR #43239 has modified ECBackend::get_hash_info() behavior.
Modified the standalone scrub test to match.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-10-20 06:42:51 +00:00
Samuel Just
92d0a7ca3d
Merge pull request #43592 from rzarzynski/wip-crimson-mount_ertr
crimson: errorate the FuturizedStore::mount() paths.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-10-19 21:55:23 -07:00
Neha Ojha
e70c7de4b5
Merge pull request #42839 from giladmaya/wip-on-wire-compression-documentation
doc: Added msgr on-wire-compression

 Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-19 14:55:16 -07:00
Neha Ojha
1a2403afc1
Merge pull request #43577 from ThomasLamprecht/docs-glossary-typo
docs: fix case and spelling of Proxmox VE

Reviewed-by: Laura Flores <lflores@redhat.com>
2021-10-19 14:54:10 -07:00
Radoslaw Zarzynski
677bce0e97 crimson: errorate the FuturizedStore::mount() paths.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-19 20:11:55 +00:00
Sage Weil
9e70065a73 Merge PR #43556 into master
* refs/pull/43556/head:
	qa/suites/orch/cephadm/osds: add osd rm/zap/add test

Reviewed-by: Cory Snyder <csnyder@iland.com>
2021-10-19 16:04:36 -04:00
Radoslaw Zarzynski
f94cab0c53 crimson/os: move singleton_ec up to let mount() reuse it.
No other changes in this commit.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-19 19:25:00 +00:00
Sage Weil
a4fef1dabd Merge PR #43569 into master
* refs/pull/43569/head:
	mgr/orchestrator: clean up 'orch device ls' output

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 10:58:31 -04:00
Casey Bodley
9b806156f5
Merge pull request #43486 from cbodley/wip-rgw-maintainers
rgw: start on MAINTAINERS.md

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-10-19 08:53:01 -04:00
Sebastian Wagner
a755ffa678
Merge pull request #43514 from mgfritch/cephadm-agent-binary
cephadm: use the current cephadm binary for the agent

Reviewed-by: Adam King <adking@redhat.com>
2021-10-19 13:33:43 +02:00
Ernesto Puerta
005b5b4b18
Merge pull request #43465 from rhcs-dashboard/radosgw-grafana-dashboards
mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-19 13:30:06 +02:00
Sebastian Wagner
84d37ec78f
Merge pull request #43370 from liewegas/fix-rook-rgw-multisite
mgr/rook: describe_services: name rgw svc appropriately

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 11:34:46 +02:00
Sebastian Wagner
f1b93ec882
Merge pull request #43550 from liewegas/rook-nvme
qa/suites/orch/rook/smoke: use nvme-loop

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 11:30:40 +02:00
Ernesto Puerta
f5fddd6121
Merge pull request #42526 from liewegas/dashboard-nfs
mgr/dashboard: consume mgr/nfs

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-19 11:17:17 +02:00
Aashish Sharma
ed954b0e6c mgr/dashboard: monitoring: grafonnet refactoring for cephfs dashboards
This PR intends to refactor cephfs dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:36:31 +05:30
Aashish Sharma
e490e2f3ab mgr/dashboard: monitoring: grafonnet refactoring for osds dashboards
This PR intends to refactor osds dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:13:50 +05:30
Aashish Sharma
8c48821c21 mgr/dashboard: monitoring: grafonnet refactoring for pools dashboards
This PR intends to refactor pools dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:10:56 +05:30
Aashish Sharma
e737aaa000 mgr/dashboard: monitoring: grafonnet refactoring for rbd dashboards
This PR intends to refactor rbd dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 12:09:04 +05:30
Aashish Sharma
eb01954cd9 mgr/dashboard: monitoring: grafonnet refactoring for radosgw dashboards
This PR intends to refactor radosgw dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-19 11:57:28 +05:30
Sage Weil
96319a3d39 qa/suites/orch/cephadm/osds: add osd rm/zap/add test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 14:05:24 -05:00
Sage Weil
c576184cd8 Merge PR #43504 into master
* refs/pull/43504/head:
	mgr/nfs: add some test coverage for cluster info, cluster config
	mgr/nfs: add 'nfs cluster config get'

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-18 15:03:33 -04:00
Sage Weil
4a63e3eafd Merge PR #43510 into master
* refs/pull/43510/head:
	qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
	mgr/cephadm: make upgrade ls output structured
	mgr/cephadm: add 'orch upgrade ls' to list available versions

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:03:24 -04:00
Sage Weil
75b2cc5c1b Merge PR #43549 into master
* refs/pull/43549/head:
	mgr/rook: fix mds service list

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-18 13:56:39 -04:00
Casey Bodley
50e4f2b702 rgw: add src/rgw/MAINTAINERS.md
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-18 12:51:21 -04:00
Ernesto Puerta
19535b1d0e
Merge pull request #43469 from rhcs-dashboard/hosts-grafana-dashboards
mgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-18 17:14:03 +02:00
Sage Weil
8900775841 qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 10:05:32 -04:00
Ilya Dryomov
cf8b6dc972
Merge pull request #42760 from ideepika/wip-iscsi-testing
qa/suites/rbd: switch iscsi tests to cephadm

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-18 14:25:36 +02:00
Sebastian Wagner
cd2a476fa1
Merge pull request #43564 from sebastian-philipp/doc-cephadm-host-location
doc/cephadm: Hosts: Add "initial CRUSH location"

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-18 13:58:55 +02:00
Alfonso Martínez
58a6ab2147 mgr/dashboard: NFS exports: API + UI: integration with mgr/nfs; cleanups
mgr/dashboard: move NFS_GANESHA_SUPPORTED_FSALS to mgr_module.py

Importing from nfs module throws AttributeError because as a side effect the dashboard module is impersonating the nfs module.
https://gist.github.com/varshar16/61ac26426bbe5f5f562ebb14bcd0f548

mgr/dashboard: 'Create NFS export' form: list clusters from nfs module

mgr/dashboard: frontend+backend cleanups for NFS export

Removed all code and references related to daemons. UI cleanup and adopted unit-testing for
nfs-epxort create form for CEPHFS backend. Cleanup for export list/get/create/set/delete endpoints.

mgr/dashboard: rm set-ganesha ref + update docs

Remove existing set-ganesha-clusters-rados-pool-namespace references as
they are no longer required. Moreover, nfs doc in dashboard doc is
updated accordingly to the current nfs status.

mgr/dashboard: add nfs-export e2e test coverage

mgr/dashboard: 'Create NFS export' form: remove RGW user id field.

- Improve bucket typeahead behavior.
- Increase version for bucket list endpoint.
- Some refactoring.

mgr/dashboard: 'Create NFS export' form: allow RGW backend only when default realm is selected.

When RGW multisite is configured, the NFS module can only handle buckets in the default realm.

mgr/dashboard: 'Create service' form: fix NFS service creation.

After https://github.com/ceph/ceph/pull/42073, NFS pool and namespace are not customizable.

mgr/dashboard: 'Create NFS export' form: add bucket validation.

- Allow only existing buckets.
- Refactoring:
  - Moved bucket validator from bucket form to cd-validators.ts
  - Split bucket validator into 2: bucket name validator and bucket existence (that checks either existence or non-existence).

mgr/dashboard: 'Create NFS export' form: path validation refactor: allow only existing paths.

Fixes: https://tracker.ceph.com/issues/46493
Fixes: https://tracker.ceph.com/issues/51479
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 12:58:54 +02:00
Thomas Lamprecht
023c32c99b docs: fix case and spelling of Proxmox VE
There's no such thing as ProxMox, the hypervisor product with build
in ceph server management and RBD client access is named `Proxmox VE`
and the company behind it is named Proxmox Server Solutions GmbH

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-18 12:10:26 +02:00
Varsha Rao
64178e1f5b mgr/dashboard: directly use ExportMgr and NFSCluster objects
Using the objects directly provides access to other methods and helps in
avoiding repeatition.

mgr/dashboard/nfsganesha: remove tag

Since NFS v3 is no longer supported. We can remove tag.

mgr/nfs: define global constant to list supported FSALs

mgr/dashboard: directly list nfs clusters by directly importing available_cluster() method

The current dashboard api returns a list of following dictionary

{
   'pool': 'nfs-ganesha',
   'namespace': cluster_id,
   'type': 'orchestrator',
   'daemon_conf': None
}

None of these values are required for listing nfs cluster by mgr/nfs module.
Instead directly list available cluster names

mgr/dashboard: add comment to remove listing of daemons

As the configs are per cluster. There is no need to list daemons per cluster.

mgr/dashboard/controllers/nfsganesha: Add comments to update/remove status endpoint

This endpoint can be updated in suggested way or even removed. As it was
initially[1] introduced to check if dashboard pool and namespace configuration was
set.

[1] 824726393b

mgr/nfs: remove fetch_cluster_obj()

There is no need to fetch NFSCluster class object. Directly
available_clusters() can be imported to list nfs clusters.

mgr/dashboard/controllers/nfsganesha: list exports based on cluster id

As mgr/nfs module lists based on cluster id.

mgr/dashboard/nfs: get and delete export by export id

Fixes: https://tracker.ceph.com/issues/46493
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-10-18 12:00:53 +02:00
Sage Weil
6e5a1eefd0 mgr/dashboard: consume mgr/nfs via mgr.remote()
Stop using the dashboard version of the Ganesha config classes; consume
mgr/nfs instead via remote().

mgr/nfs/export: return Export from _apply_export

Future callers will want this.

mgr/nfs: new module methods for dashboard consumption

Add some new methods that are easy for the dashboard API to consume.  These
are very similar to the CLI methods but do now have the @CLICommand and
related decorators, and have slightly different interfaces (e.g., returning
the created/modified Export dict).

mgr/dashboard: remove old ganesha code (and tests)

Fixes: https://tracker.ceph.com/issues/46493
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 11:58:35 +02:00
Deepika Upadhyay
b09c6ca2cc src/test/cli-integration: correct hostname for iscsi gateway
iscsi accepts first word of hostname as local hostname[0] instead of
smithi061.front.sepia.ceph.com we use: smithi061 otherwise gateway
creation will error out with:
```The first gateway defined must be the local machine```

[0] ff3e430b70/gwcli/utils.py (L36)

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
3e9b8c8f8b src/test/cli-integration/rbd: update gwcli_* to use podman exec
* since iscsi now resides inside the container, for testing we use the
  hack:
  - find the iscsi container/
  - use found container and podman exec command to test gateway creation
    deletion from host itself

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
cbd2c71398 qa/tasks: adapt ceph_iscsi.py task to ceph_iscsi_client
* we use setup_iscsi_client.py to deploy iscsi client services,
  configuring intiator and mulitpath this is done by qa task
  ceph_iscsi_client
* qa/cephadm: adds remotes ip addresses to iscsi gateway,
* rename poolname: iscsi >> datapool, which we usually use for tests and
  expresses type of pool more clearly.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
a7952949a8 qa/suites/rbd: remove baremetal based setup needed for iscsi testing
* replace ceph baremetal deployment with cephadm based deployment

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
8a35516294 doc: add iSCSI initiator client configuration section for cephadm
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:43 +05:30
Maya Gilad
3f5dca4977 doc added msgr on-wire-compression
Signed-off-by: Maya Gilad <ms.maya.gilad@gmail.com>
2021-10-18 10:03:30 +03:00
Mykola Golub
429ac06cbb qa/tasks/backfill_toofull: make test work when compression on
The osd backfill reservation does not take compression into account so
we need to operate with "uncompressed" bytes when calculating nearfull
ratio.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2021-10-16 15:28:53 +03:00
Kefu Chai
5db11d3988
Merge pull request #43571 from tchaikov/wip-bl-prepare-iovs
common/bl: bl::prepare_iovs() related cleanups

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-16 10:57:05 +08:00
Kefu Chai
4e81734247 common/bl: mark bl::prepare_iovs() const
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-16 08:31:08 +08:00