Commit Graph

8788 Commits

Author SHA1 Message Date
Marcus Watts
454cc8a18c Fix vault token file access.
Put the vault token file in a location that ceph can read.
Make it readable only by ceph.

On rhel8 (and indeed, any vanilla rhel machine), $HOME is liable to be
mode 700.  This means the ceph user can't read things in that user's
directory.  This causes radosgw to emit the confusing message "ERROR:
Vault token file ... not found" even though the teuthology log will
plainly show it was created and made readable by ceph.

Fixes: http://tracker.ceph.com/issues/51539
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2021-10-28 14:14:10 -04:00
Prasanna Kumar Kalever
2adfe55bcc qa/workunits/rbd: test map/attach with --show-cookie/--cookie options
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-10-26 19:24:13 +05:30
Mykola Golub
8c646721c2
Merge pull request #41696 from MrFreezeex/fix-rbd-mirror-remove
rbd-mirror: fix mirror image removal

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-10-25 18:12:38 +03:00
Ilya Dryomov
6278a04ac2 qa/suites/rbd: whitelist POOL_FULL due to quota for test_librbd.sh
RemoveFullTry tests fill up the pool and expect EDQUOT.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-23 22:04:58 +05:30
Patrick Donnelly
94b712bdab
Merge PR #43566 into master
* refs/pull/43566/head:
	qa: reduce frag split confs for dir_split counter test

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-10-21 10:26:43 -04:00
Patrick Donnelly
04aabf8bee
Merge PR #38752 into master
* refs/pull/38752/head:
	qa: enable dynamic debug support to kclient

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-20 22:24:32 -04:00
Ronen Friedman
352399d6a4
Merge pull request #43596 from ronen-fr/wip-rf-test_info
tests: modify osd-scrub-repair to match PR #43239 changes

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-10-20 18:03:22 +03:00
Neha Ojha
9fda06aa6a
Merge pull request #43572 from trociny/wip-qa-backfill-toofull-compress
qa/tasks/backfill_toofull: make test work when compression on

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-20 06:57:06 -07:00
Ronen Friedman
52e9fa16ef tests: modify osd-scrub-repair to match PR #43239 changes
PR #43239 has modified ECBackend::get_hash_info() behavior.
Modified the standalone scrub test to match.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-10-20 06:42:51 +00:00
Sage Weil
9e70065a73 Merge PR #43556 into master
* refs/pull/43556/head:
	qa/suites/orch/cephadm/osds: add osd rm/zap/add test

Reviewed-by: Cory Snyder <csnyder@iland.com>
2021-10-19 16:04:36 -04:00
Sebastian Wagner
f1b93ec882
Merge pull request #43550 from liewegas/rook-nvme
qa/suites/orch/rook/smoke: use nvme-loop

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-19 11:30:40 +02:00
Ernesto Puerta
f5fddd6121
Merge pull request #42526 from liewegas/dashboard-nfs
mgr/dashboard: consume mgr/nfs

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-19 11:17:17 +02:00
Sage Weil
96319a3d39 qa/suites/orch/cephadm/osds: add osd rm/zap/add test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 14:05:24 -05:00
Sage Weil
4a63e3eafd Merge PR #43510 into master
* refs/pull/43510/head:
	qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
	mgr/cephadm: make upgrade ls output structured
	mgr/cephadm: add 'orch upgrade ls' to list available versions

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:03:24 -04:00
Sage Weil
8900775841 qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 10:05:32 -04:00
Ilya Dryomov
cf8b6dc972
Merge pull request #42760 from ideepika/wip-iscsi-testing
qa/suites/rbd: switch iscsi tests to cephadm

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-18 14:25:36 +02:00
Alfonso Martínez
58a6ab2147 mgr/dashboard: NFS exports: API + UI: integration with mgr/nfs; cleanups
mgr/dashboard: move NFS_GANESHA_SUPPORTED_FSALS to mgr_module.py

Importing from nfs module throws AttributeError because as a side effect the dashboard module is impersonating the nfs module.
https://gist.github.com/varshar16/61ac26426bbe5f5f562ebb14bcd0f548

mgr/dashboard: 'Create NFS export' form: list clusters from nfs module

mgr/dashboard: frontend+backend cleanups for NFS export

Removed all code and references related to daemons. UI cleanup and adopted unit-testing for
nfs-epxort create form for CEPHFS backend. Cleanup for export list/get/create/set/delete endpoints.

mgr/dashboard: rm set-ganesha ref + update docs

Remove existing set-ganesha-clusters-rados-pool-namespace references as
they are no longer required. Moreover, nfs doc in dashboard doc is
updated accordingly to the current nfs status.

mgr/dashboard: add nfs-export e2e test coverage

mgr/dashboard: 'Create NFS export' form: remove RGW user id field.

- Improve bucket typeahead behavior.
- Increase version for bucket list endpoint.
- Some refactoring.

mgr/dashboard: 'Create NFS export' form: allow RGW backend only when default realm is selected.

When RGW multisite is configured, the NFS module can only handle buckets in the default realm.

mgr/dashboard: 'Create service' form: fix NFS service creation.

After https://github.com/ceph/ceph/pull/42073, NFS pool and namespace are not customizable.

mgr/dashboard: 'Create NFS export' form: add bucket validation.

- Allow only existing buckets.
- Refactoring:
  - Moved bucket validator from bucket form to cd-validators.ts
  - Split bucket validator into 2: bucket name validator and bucket existence (that checks either existence or non-existence).

mgr/dashboard: 'Create NFS export' form: path validation refactor: allow only existing paths.

Fixes: https://tracker.ceph.com/issues/46493
Fixes: https://tracker.ceph.com/issues/51479
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 12:58:54 +02:00
Deepika Upadhyay
cbd2c71398 qa/tasks: adapt ceph_iscsi.py task to ceph_iscsi_client
* we use setup_iscsi_client.py to deploy iscsi client services,
  configuring intiator and mulitpath this is done by qa task
  ceph_iscsi_client
* qa/cephadm: adds remotes ip addresses to iscsi gateway,
* rename poolname: iscsi >> datapool, which we usually use for tests and
  expresses type of pool more clearly.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
a7952949a8 qa/suites/rbd: remove baremetal based setup needed for iscsi testing
* replace ceph baremetal deployment with cephadm based deployment

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Patrick Donnelly
abd19aa215
qa: reduce frag split confs for dir_split counter test
Fixes: https://tracker.ceph.com/issues/52949
Fixes: a5675535ba
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-17 21:52:23 -04:00
Mykola Golub
429ac06cbb qa/tasks/backfill_toofull: make test work when compression on
The osd backfill reservation does not take compression into account so
we need to operate with "uncompressed" bytes when calculating nearfull
ratio.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2021-10-16 15:28:53 +03:00
Patrick Donnelly
e55381c105
Merge PR #43430 into master
* refs/pull/43430/head:
	qa: skip c-v install on pacific start

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-15 11:16:25 -04:00
Patrick Donnelly
5af9882f94
Merge PR #43426 into master
* refs/pull/43426/head:
	qa/cephfs: update xfstests_dev for centos stream

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:15:41 -04:00
Patrick Donnelly
a8e77365b1
Merge PR #43425 into master
* refs/pull/43425/head:
	qa: import CommandFailedError from exceptions not run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2021-10-15 11:13:55 -04:00
Patrick Donnelly
7aea7f48ba
Merge PR #43420 into master
* refs/pull/43420/head:
	qa: skip internal metadata directory when scanning ceph debugfs directory

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:13:05 -04:00
Sage Weil
e4e9477dd9 qa/suites/orch/rook/smoke: use nvme-loop
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-14 12:58:38 -05:00
Kefu Chai
70b049ffdb
Merge pull request #43239 from trociny/wip-48959
osd: handle inconsistent hash info during backfill and deep scrub gracefully

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-14 22:43:16 +08:00
Kefu Chai
b5d2548ceb
Merge pull request #43463 from Zhiwei-Dai/wip-enhance-qa-python3-compatibility
qa/tasks: replace iterkeys() with keys() for Python 3

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:38:39 +08:00
Sebastian Wagner
d4783f5a65
Merge pull request #43214 from batrick/i52654
pybind/mgr/cephadm: set allow_standby_replay during CephFS upgrade

Reviewed-by: Sage Weil <sage@newdream.net>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:28:30 +02:00
Ernesto Puerta
90bbcab09f
Merge pull request #42557 from ceph/feature-50336-cluster-creation-wizard
mgr/dashboard: Cluster Creation/Expansion Wizard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Volker Theile <vtheile@suse.com>
2021-10-14 15:12:42 +02:00
Sage Weil
e94ba142f0 Merge PR #43049 into master
* refs/pull/43049/head:
	mgr/rook: apply mds using placement spec and osd_pool_default_size
	mgr/rook: factor out replica/failureDomain calc

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-10-13 17:08:20 -04:00
Avan Thakkar
6644a00a2c mgr/dashboard: introduce gather facts in host list
Fixes: https://tracker.ceph.com/issues/52017
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2021-10-13 16:02:51 +05:30
Nizamudeen A
59cbf97e6c mgr/dashboard: Cluster Creation Add Host Section and e2es
Add host section of the cluster creation workflow.

1. Fix bug in the modal where going forward one step on the wizard and coming back opens up the add host modal.
2. Rename Create Cluster to Expand Cluster as per the discussions
3. A skip confirmation modal to warn the user when he tries to skip the
   cluster creation
4. Adapted all the tests
5. Did some UI improvements like fixing and aligning the styles,
   colors..
- Used routed modal for host Additon form
- Renamed the Create to Add in Host Form

Fixes: https://tracker.ceph.com/issues/51517
Fixes: https://tracker.ceph.com/issues/51640
Fixes: https://tracker.ceph.com/issues/50336
Fixes: https://tracker.ceph.com/issues/50565
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-13 15:55:23 +05:30
Avan Thakkar
b9f38cadc4 mgr/dashboard: Create Cluster Workflow welcome screen and e2e tests
A module option called CLUSTER_STATUS has two option. INSTALLED
AND POST_INSTALLED. When CLUSTER_STATUS is INSTALLED it will allow to show the
create-cluster-wizard after login the initial time.  After the cluster
creation is succesfull this option is set to POST_INSTALLED
Also has the e2e codes for the Review Section

Fixes: https://tracker.ceph.com/issues/50336
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-13 15:52:14 +05:30
Zack Cerza
b57539dc94 Revert "qa: support isal ec test for aarch64"
This commit has been causing scheduled jobs to request e.g. aarch64
smithi machines, which don't exist. The dispatcher then tries to find them forever, requiring the dispatcher to be killed and restarted. The queue
will sit idle until someone notices the problem.

Signed-off-by: Zack Cerza <zack@redhat.com>
2021-10-12 12:53:58 -06:00
Joseph Sawaya
1a67d3e559 mgr/rook: apply mds using placement spec and osd_pool_default_size
This commit changes the apply_mds command in the rook orchestrator
to support some placement specs and also sets the replica size according
to the osd_pool_default_size ceph option.

This commit also adds `orch apply mds` to the QA to test if the command
runs.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-10-11 12:06:37 -05:00
Sebastian Wagner
100a466e3f
Merge pull request #43471 from liewegas/fix-rook-1-node
qa/suites/orch/rook: replicate across osds for 1-node clusters

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-11 18:47:35 +02:00
Sebastian Wagner
1be6dc174f
Merge pull request #43455 from liewegas/qa-nvme-loop
qa: use nvme_loop devices for (some) cephadm tests

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-11 16:43:10 +02:00
Dai Zhiwei
00e5e5d5cd qa/tasks: replace iterkeys() with keys() in Python 3
Python 2.7 reached the end of its lifetime, the pr fixes teuthology task
error in Python 3.x

Fixes: https://tracker.ceph.com/issues/52878
Signed-off-by: Dai Zhiwei <daizhiwei3@huawei.com>
2021-10-09 23:10:03 +08:00
Sage Weil
d4a1ec2d06 qa/tasks/nvme_loop: loop until 'nvme list' shows new devs
Sometimes this doesn't happen immediately.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
6436b7e05f qa/suites/roch/cephadm/smoke: use nvme_loop
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
4206b28dd7 qa/suites/orch/cephadm/smoke-roleless: no osd roles, use nvme loop
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
8d4b870316 qa/overrides/nvme_loop: reusable fragment to use nvme_loop devs
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
dda15a7924 qa/tasks/cephadm: wait for osds to start explicitly
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
d3c9486ed9 qa/tasks/cephadm: if no osd roles, --all-available-devices
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
65cf69c6ff qa/tasks/nvme_loop: set up nvme_loop on scratch_devs
Using an nvme loop device makes the LVs look like "real" disks,
which means we can exercise all of the normal code paths for
provisioning, deprovisioning, and zapping.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Patrick Donnelly
c6c9734a91
Merge PR #43163 into master
* refs/pull/43163/head:
	qa: fsync dir for asynchronous creat on stray tests
	qa: refactor and generalize create_n_files
	qa: only set frag confs for workloads
	mds: improve debugging for fragment size check

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-10-08 14:38:35 -04:00
Sage Weil
0144e3a4d7 qa/suites/orch/rook: replicate across osds for 1-node clusters
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 11:00:48 -05:00
Dai Zhiwei
eaa385f3da qa: support isal ec test for aarch64
modified:   qa/standalone/erasure-code/test-erasure-code-plugins.sh
	new file:   qa/suites/rados/thrash-erasure-code-isa/arch/aarch64.yaml

Signed-off-by: Dai Zhiwei <daizhiwei3@huawei.com>
2021-10-08 14:37:25 +08:00
Patrick Donnelly
d33debc643
qa: fsync dir for asynchronous creat on stray tests
Use the enhanced create_n_files to dedup code. Also split the large test
into three.

Fixes: https://tracker.ceph.com/issues/52606
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00