Commit Graph

2291 Commits

Author SHA1 Message Date
David Galloway
805d2320b3
Merge pull request #46385 from ceph/inclusive-rename-yaml
Rename/re-symlink whitelist_*.yaml
2022-05-24 17:04:50 -04:00
Zack Cerza
0666dd34cf Rename/re-symlink whitelist_*.yaml
whitelist_health.yaml -> ignorelist_health.yaml
whitelist_wrongly_marked_down.yaml -> ignore_wrongly_marked_down.yaml

This was mostly addressed in
2ee9365d0b,
but the rename wasn't done there.

Signed-off-by: Zack Cerza <zack@cerza.org>
2022-05-24 14:14:04 -06:00
Adam King
7dbddc008c
Merge pull request #46324 from ceph/dgalloway-nodockerio
qa: Pull from quay.io not docker.io

Reviewed-by: Adam King <adking@redhat.com>
2022-05-24 10:27:03 -04:00
Neha Ojha
f0aeb2ec29
Merge pull request #46346 from ljflores/wip-lflores-testing-recovery
qa/suites/rados/thrash-erasure-code-big/thrashers: add `osd max backfills` setting to mapgap and pggrow

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-23 16:26:27 -07:00
Casey Bodley
51969f86a2
Merge pull request #45985 from soumyakoduri/wip-skoduri-dbstore-tests
rgw/qa: Run s3tests on dbstore backend in teuthology

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-23 12:55:41 -04:00
Soumya Koduri
a7397100aa rgw/qa: Run s3tests on dbstore backend in teuthology
Add a new test-suite to run s3tests on dbstore backend.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-21 23:49:47 +05:30
Laura Flores
40062676c2 qa/suites/rados/thrash-erasure-code-big/thrashers: add osd max backfills setting to mapgap and pggrow
All `rados/thrash-erasure-code-big` tests that die due to the “wait_for_recovery” timeout have one thing in common: They contain either `thrashers/pggrow` or `thrashers/mapgap`.

The difference between pggrow and mapgap vs. all other non-offending thrashers (default, careful, fastread, and morepggrow) is that they lack an override setting for `osd max backfills`. `osd max backfills` is the max number of backfill operations allowed to/from an OSD. The higher the number, the quicker the recovery. By default, this value is 1. On all of the non-offending thrashers (default, careful, fastread, and morepggrow), the default 1 value gets overridden in their .yaml files with a value > 1. This is not the case for pggrow and mapgap, however, as they lack an `osd max backfills` override setting.

The mclock op scheduler is known to override `osd max backfills` with a high value, but all of the thrash-erasure-code-big thrashers have their op queue set to “debug_random”, which chooses randomly between op queues (the debug_random op queue is set to override the default mclock_scheduler in qa/config/rados.yaml). So, coupled with the “debug_random” op queue, the low `osd max backfill` setting is causing some tests to time out in recovery.

WITHOUT `osd max backfills`, as they are now, “mapgap” and “pggrow” tests die due to timed-out recovery about 17/100 times, as seen here with a pggrow test: http://pulpito.front.sepia.ceph.com/lflores-2022-05-18_14:24:29-rados:thrash-erasure-code-big-master-distro-default-smithi/

WITH `osd max backfills` specified, as I have suggested in this PR, 99/100 tests passed, with one test failing for a different reason:
http://pulpito.front.sepia.ceph.com/lflores-2022-05-17_22:40:27-rados:thrash-erasure-code-big-master-distro-default-smithi/

I also scheduled 145 tests WITH `osd max backfills` that are a mix of pggrow and mapgap thrashers. 144/145 tests passed, with one test failing for a different reason. http://pulpito.front.sepia.ceph.com/lflores-2022-05-17_15:27:54-rados:thrash-erasure-code-big-master-distro-default-smithi/

Fixes: https://tracker.ceph.com/issues/51076
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-05-19 18:29:00 -05:00
Adam King
791e1d20b3 qa/suites/orch/cephadm: staggered upgrade test
Signed-off-by: Adam King <adking@redhat.com>
2022-05-19 17:24:56 -04:00
Soumya Koduri
aafa3d173f
Merge pull request #46008 from soumyakoduri/wip-skoduri-cloudtier-tests
rgw/qa: enable s3-tests related to cloud-transition feature

Reviewed-by: casey Bodley <cbodley@redhat.com>
Reviewed-by: Maredia, Ali <amaredia@redhat.com>
2022-05-19 15:53:54 +05:30
Soumya Koduri
fa86d2ea42 rgw/qa: Run tests on multiple cloudtier config
Run cloudtier tests with parameter 'retain_head_object'
set to true and false.

However having multiple cloudtier storage classes in the same task
is increasing the transition time and resulting in spurious failures.
Hence until there is a consistent way of running the tests, without
having to depend on lc_debug_interval, disabled one of the config for
now.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-19 13:57:46 +05:30
Casey Bodley
0b7ab95c81
Merge pull request #46232 from cbodley/wip-qa-rgw-sse
qa/rgw: use 'with-sse-s3' override for s3tests

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-05-18 16:44:28 -04:00
David Galloway
ae2e93f5f1 qa: Pull from quay.io not docker.io
We are pushing to quay.io now due to docker.io rate limiting.

Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-05-18 14:22:52 -04:00
Soumya Koduri
f846f6ff02 rgw/qa: Add separate task for cloudtier tests
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-17 20:54:58 +05:30
Soumya Koduri
d0ac12dc38 rgw/qa: enable s3-tests related to cloud-transition feature
Added a new test-suite to configure cloudtier storage class and run related s3-tests.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-17 20:54:58 +05:30
Venky Shankar
44a0199114
Merge pull request #45834 from ajarr/wip-53996
qa/suites/fs/upgrade: change octopus installations and references

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:14:53 +05:30
Casey Bodley
98ddd40859 qa/rgw: use 'with-sse-s3' override for s3tests
don't rely on the ceph manager task to parse a config file. each rgw
could be using a different config. instead, revert to an s3tests
override called 'with-sse-s3'

this way, the only job that enables sse-s3, vault_transit.yaml, contains
both the 'rgw crypt sse s3' configurables, and the flag to enable the
associated test cases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 14:49:51 -04:00
Ali Maredia
d634f0690b
Merge pull request #44494 from mdw-at-linuxbox/wip-master-mankey
rgw/crypt: sse-s3 changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-05-02 16:57:55 -04:00
Ramana Raja
898240c8ee qa/suites/fs/upgrade: change octopus installations and references
... to pacific.

Fixes: https://tracker.ceph.com/issues/53996
Signed-off-by: Ramana Raja <rraja@redhat.com>
2022-05-02 14:53:18 -04:00
Adam King
9268e32047
Merge pull request #45899 from rkachach/fix_issue_53527
mgr/cephadm: fixing upgrade candidate versions listing

Reviewed-by: Adam King <adking@redhat.com>
2022-04-26 15:09:37 -04:00
Redouane Kachach
671442c4b4
mgr/cephadm: fixing upgrade candidate verions listing
Fixes: https://tracker.ceph.com/issues/53527

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-04-22 15:54:57 +02:00
Adam King
5720cca2f8
Merge pull request #45920 from adk3798/stop-mgr-nfs-upgrade-fail
qa/suites/orch/cephadm: stop upgrade tests if failures are seen

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-04-22 09:28:35 -04:00
Marcus Watts
20cb9805c9 qa/rgw - run sse-s3 test cases by default
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2022-04-19 17:35:40 -04:00
Casey Bodley
065b9aad86 qa/rgw: run sse-s3 test cases conditionally
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4e644832b0)
2022-04-19 17:35:39 -04:00
Casey Bodley
e346a75399 qa/rgw: configure sse-s3 in vault_transit.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1b679e0828)
2022-04-19 17:35:39 -04:00
Ionut Balutoiu
8054199787 qa: add basic Ceph on Windows integration test
Due to lack of Windows support in the Teuthology, the test case adopts
the following workaround:

* Deploy baremetal machine with `ubuntu_latest.yaml` and
  configure it with libvirt KVM.
* Create a libvirt VM and provision it with Windows Server 2019, using
  the official ISO from Microsoft.
* Configure SSH in the Windows VM, and run the tests remotely via SSH.

The implementation of the test case consists of workunit scripts.

`qa/workunits/windows/test_rbd_wnbd.py` is the main Python script
to test Ceph on Windows basic functionality. This is executed in the
libvirt VM configured with Windows Server 2019.

Co-authored-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-authored-by: Daniel Vincze <dvincze@cloudbasesolutions.com>

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2022-04-19 17:21:52 +03:00
Adam King
017aa9cfe8 qa/suites/orch/cephadm: stop upgrade tests if failures are seen
Otherwise the tests may run forever. This was already done for
mds upgrade sequence, justadding it in the other two places here

Related to: https://tracker.ceph.com/issues/53939

Signed-off-by: Adam King <adking@redhat.com>
2022-04-17 12:21:44 -04:00
Laura Flores
bd82d21943
Merge pull request #45749 from josephsawaya/fix-rook-tests
Remove orchestrator from rook task and suite
2022-04-08 16:36:38 -05:00
David Galloway
fcd512b50b qa: Latest RHEL8 -> RHEL8.5
Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-04-01 15:00:01 -04:00
Joseph Sawaya
043a330ed0 Remove orchestrator from rook task and suite
This commit removes orchestrator commands from the
Rook task and the Rook test suite because the Rook
orchestrator is not being maintained, and the Rook
orchestrator CLI is obsolete. This should also
clarify the issue:

https://tracker.ceph.com/issues/53680

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2022-04-01 11:37:55 -04:00
Adam King
6e4dd0e082
Merge pull request #45361 from adk3798/mds-upgrade-test
qa/suites/fs: stop looping in mds upgrade test if upgrade failed

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-03-30 18:56:45 -04:00
Ilya Dryomov
f0ade57458 qa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-22 21:42:41 +01:00
Teoman ONAY
de8b3c2676 cephadm: remove containers pids-limit
The default pids-limit (docker 4096/podman 2048) prevent some
customization from working (http threads on RGW) or limits the number
of luns per iscsi target.

Fixes: https://tracker.ceph.com/issues/52898
Signed-off-by: Teoman ONAY <tonay@redhat.com>
2022-03-22 21:41:26 +01:00
Adam King
37019aad80 qa/suites/fs: stop looping in mds upgrade test if upgrade failed
Signed-off-by: Adam King <adking@redhat.com>
2022-03-22 08:05:31 -04:00
Samuel Just
987465eea0
Merge pull request #45176 from liu-chunmei/crimson-teuthology
crimson: teuthology suite for seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-16 12:06:04 -07:00
chunmei-liu
3cca3ed552 qa/suites/crimson-rados: add fixed-1 for seastore
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-03-08 21:29:04 -08:00
chunmei-liu
815300a5bd qa/suites/crimson-rados: add seastore test case
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-03-07 01:35:08 -08:00
Kamoltat
1bc51f057f upgrade/pacific-x/parallel: Added mds.a and mds.b
Added mds daemons so that it can create
cephFS pools and set options using
`do_set_pool()` in FSCommand.cc. Such that
we can cover corner cases like that in

https://tracker.ceph.com/issues/54263

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-03-02 22:34:21 +00:00
Christopher Hoffman
19d46b9181 qa/suites/krbd: add rbd_default_map_options override coverage
Add coverage to test precedence, override, and option merge on rbd map.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>
2022-02-18 17:19:45 +01:00
Ilya Dryomov
7f391c5688 qa/suites/krbd: rename rxbounce subsuite
A new job that doesn't want ms_mode to be set underneath it is about to
be added.  Rename rxbounce to ms_modeless to make this purpose obvious.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-02-18 17:19:45 +01:00
Ilya Dryomov
8f0fd0af3d qa/suites/rbd: make sure block-rbd.so is installed
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-02-16 12:20:44 +01:00
Adam King
c1b48ce400
Merge pull request #44917 from guits/fix_rook_smoke_nvme_loop
qa/orch: use hwe kernel in rook/smoke tests (ubuntu 20.04)

Reviewed-by: Neha Ojha nojha@redhat.com
Reviewed-by: Yuri Weinstein yweinste@redhat.com
2022-02-14 10:55:36 -05:00
Venky Shankar
43b3a7c845
Merge pull request #39315 from shenhang/fetchkey
mds: fetch single dentry instead of complete dirfrag for lookup

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-14 09:53:10 +05:30
Patrick Donnelly
e883dc3b82
Merge PR #42000 into master
* refs/pull/42000/head:
	qa: update rhel kclient to setup container tools
	qa: stop overriding distro for k-testing
	qa: only use RHEL for workload testing
	qa: convert fs:workload to use cephadm
	qa: split fs begin task
	qa/tasks/cephadm: setup CephManager when OSDs are provisioned
	qa/tasks/cephadm: setup file system if MDS are provisioned

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 09:34:49 -05:00
shenhang
d7feb89823 mds: add an option to decide whether prefetching entire dir or not.
Accessing one single dentry could be fastened by set this option to
false, when dir is not in the memory.
Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:46 +08:00
Venky Shankar
b7af2a94a4
Merge pull request #42549 from ajarr/wip-add-volume-rename
mgr/volumes: Add `fs volume rename` command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 11:34:32 +05:30
Guillaume Abrioux
b252c6495c qa/orch: use hwe kernel in rook/smoke tests (ubuntu 20.04)
Use the override in ./src/qa/distros/container-hosts/ubuntu_20.04.yaml
in order to use hwe kernel for Ubuntu 20.04

This is because the ubuntu 20.04 kernel (5.4) has a bug that prevents
from using nvme-loop.

see https://lkml.org/lkml/2020/9/21/1456

Fixes: https://tracker.ceph.com/issues/54094

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-02-08 10:58:02 +01:00
Ali Maredia
8de2884e06
Merge pull request #44856 from cbodley/wip-54103
qa/rgw: install s3cmd with extra_system_packages

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-02-07 16:02:40 -05:00
Ilya Dryomov
bad21fa497
Merge pull request #44842 from idryomov/wip-krbd-rxbounce-option
rbd: recognize rxbounce map option

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-02-06 20:37:31 +01:00
Venky Shankar
0e37e6a7be
Merge pull request #44851 from batrick/i54081
mds: add inline flag to MDS bootstrap compat in MDSMap

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-05 18:04:29 +05:30
Neha Ojha
8a8945e640
Merge pull request #44868 from neha-ojha/wip-move-to-stream
qa/distros: remove centos8

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-04 11:56:08 -08:00