Commit Graph

2307 Commits

Author SHA1 Message Date
Venky Shankar
27f4729256
Merge pull request #45556 from mchangir/qa-add-subvolume-option-flavors
qa: add subvolume option flavors

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-13 12:29:43 +05:30
Venky Shankar
45c9fd6187
Merge pull request #46078 from kotreshhr/fuse-directory-dacs-issue
client/fuse: Fix directory DACs overriding for root

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-06-07 22:45:05 +05:30
Ilya Dryomov
9324b49ebb
Merge pull request #46045 from idryomov/wip-qa-rwl-on-tmpfs
qa/suites/rbd: place cache file on tmpfs for xfstests

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-06-07 10:39:31 +02:00
Ilya Dryomov
23759e0034 qa/suites/rbd: place cache file on tmpfs for xfstests
The RWL mode needs DAX and is dog slow otherwise -- qemu_xfstests.yaml
job always hits the 6 hour max_job_time limit.

As our tmpfs instance is limited and qemu_xfstests.yaml opens three
images at the same time, reduce the "big cache" size to 5G.  This facet
was added to iron out 32-bit head/tail pointer issues and 5G still does
the job there.

Going through the loop device is needed because tmpfs doesn't support
O_DIRECT.

Fixes: https://tracker.ceph.com/issues/55400
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-03 19:20:37 +02:00
Ilya Dryomov
3475f9ef07 qa/suites/rbd: refactor persistent-writeback-cache suite
Rename to pwl-cache, introduce home subdirectory and 4-cache-path.yaml.
No functional changes.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-03 00:06:35 +02:00
Casey Bodley
63f84c50e0
Merge pull request #39002 from ceph/wip-rgw-multisite-reshard
rgw multisite: bucket reshard work in progress

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2022-06-02 16:04:30 -04:00
Casey Bodley
8e51eb8c05 test/rgw/multisite: enable zonegroup resharding feature
qa/tasks/rgw_multisite.py uses 'zonegroup set' to create zonegroups from
their json format. this doesn't enable any of the supported zonegroup
features by default, so this adds the 'enabled_features' field to the
json representations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-06-01 14:10:26 -04:00
0xavi0
895ee7921c
rgw/dbstore: change default value of dbstore_db_dir to /var/lib/ceph/radosgw
Changes a few NULL to nullptr.

Adds std::filesystem for path building so they're platform independant.

Fixes a bug for DBStoreManager's second constructor not creating the DB.

Adds unit tests to test DB path and prefix.

Fixes: https://tracker.ceph.com/issues/55731

Signed-off-by: 0xavi0 <xavi.garcia@suse.com>
2022-05-31 09:01:56 +02:00
Casey Bodley
8521fb2be0 qa/rgw: temporarily disable multisite reshard tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:32 -04:00
Casey Bodley
e7f3ac92a7 qa/rgw: disable multisite tests for 'bucket sync disable'
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-27 15:47:31 -04:00
Patrick Donnelly
6606d91ace
Merge commit 'b76b6ea16899861d6c266797b3c11d9f68f924d7' into main
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-05-25 16:02:14 -04:00
Venky Shankar
b76b6ea168
Merge pull request #44486 from batrick/fs-workload-kclient-switches
qa: reduce fs:workload use of random selection

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-25 06:54:53 +05:30
David Galloway
805d2320b3
Merge pull request #46385 from ceph/inclusive-rename-yaml
Rename/re-symlink whitelist_*.yaml
2022-05-24 17:04:50 -04:00
Zack Cerza
0666dd34cf Rename/re-symlink whitelist_*.yaml
whitelist_health.yaml -> ignorelist_health.yaml
whitelist_wrongly_marked_down.yaml -> ignore_wrongly_marked_down.yaml

This was mostly addressed in
2ee9365d0b,
but the rename wasn't done there.

Signed-off-by: Zack Cerza <zack@cerza.org>
2022-05-24 14:14:04 -06:00
Adam King
7dbddc008c
Merge pull request #46324 from ceph/dgalloway-nodockerio
qa: Pull from quay.io not docker.io

Reviewed-by: Adam King <adking@redhat.com>
2022-05-24 10:27:03 -04:00
Neha Ojha
f0aeb2ec29
Merge pull request #46346 from ljflores/wip-lflores-testing-recovery
qa/suites/rados/thrash-erasure-code-big/thrashers: add `osd max backfills` setting to mapgap and pggrow

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-05-23 16:26:27 -07:00
Casey Bodley
51969f86a2
Merge pull request #45985 from soumyakoduri/wip-skoduri-dbstore-tests
rgw/qa: Run s3tests on dbstore backend in teuthology

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-05-23 12:55:41 -04:00
Soumya Koduri
a7397100aa rgw/qa: Run s3tests on dbstore backend in teuthology
Add a new test-suite to run s3tests on dbstore backend.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-21 23:49:47 +05:30
Laura Flores
40062676c2 qa/suites/rados/thrash-erasure-code-big/thrashers: add osd max backfills setting to mapgap and pggrow
All `rados/thrash-erasure-code-big` tests that die due to the “wait_for_recovery” timeout have one thing in common: They contain either `thrashers/pggrow` or `thrashers/mapgap`.

The difference between pggrow and mapgap vs. all other non-offending thrashers (default, careful, fastread, and morepggrow) is that they lack an override setting for `osd max backfills`. `osd max backfills` is the max number of backfill operations allowed to/from an OSD. The higher the number, the quicker the recovery. By default, this value is 1. On all of the non-offending thrashers (default, careful, fastread, and morepggrow), the default 1 value gets overridden in their .yaml files with a value > 1. This is not the case for pggrow and mapgap, however, as they lack an `osd max backfills` override setting.

The mclock op scheduler is known to override `osd max backfills` with a high value, but all of the thrash-erasure-code-big thrashers have their op queue set to “debug_random”, which chooses randomly between op queues (the debug_random op queue is set to override the default mclock_scheduler in qa/config/rados.yaml). So, coupled with the “debug_random” op queue, the low `osd max backfill` setting is causing some tests to time out in recovery.

WITHOUT `osd max backfills`, as they are now, “mapgap” and “pggrow” tests die due to timed-out recovery about 17/100 times, as seen here with a pggrow test: http://pulpito.front.sepia.ceph.com/lflores-2022-05-18_14:24:29-rados:thrash-erasure-code-big-master-distro-default-smithi/

WITH `osd max backfills` specified, as I have suggested in this PR, 99/100 tests passed, with one test failing for a different reason:
http://pulpito.front.sepia.ceph.com/lflores-2022-05-17_22:40:27-rados:thrash-erasure-code-big-master-distro-default-smithi/

I also scheduled 145 tests WITH `osd max backfills` that are a mix of pggrow and mapgap thrashers. 144/145 tests passed, with one test failing for a different reason. http://pulpito.front.sepia.ceph.com/lflores-2022-05-17_15:27:54-rados:thrash-erasure-code-big-master-distro-default-smithi/

Fixes: https://tracker.ceph.com/issues/51076
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-05-19 18:29:00 -05:00
Adam King
791e1d20b3 qa/suites/orch/cephadm: staggered upgrade test
Signed-off-by: Adam King <adking@redhat.com>
2022-05-19 17:24:56 -04:00
Soumya Koduri
aafa3d173f
Merge pull request #46008 from soumyakoduri/wip-skoduri-cloudtier-tests
rgw/qa: enable s3-tests related to cloud-transition feature

Reviewed-by: casey Bodley <cbodley@redhat.com>
Reviewed-by: Maredia, Ali <amaredia@redhat.com>
2022-05-19 15:53:54 +05:30
Soumya Koduri
fa86d2ea42 rgw/qa: Run tests on multiple cloudtier config
Run cloudtier tests with parameter 'retain_head_object'
set to true and false.

However having multiple cloudtier storage classes in the same task
is increasing the transition time and resulting in spurious failures.
Hence until there is a consistent way of running the tests, without
having to depend on lc_debug_interval, disabled one of the config for
now.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-19 13:57:46 +05:30
Casey Bodley
0b7ab95c81
Merge pull request #46232 from cbodley/wip-qa-rgw-sse
qa/rgw: use 'with-sse-s3' override for s3tests

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-05-18 16:44:28 -04:00
David Galloway
ae2e93f5f1 qa: Pull from quay.io not docker.io
We are pushing to quay.io now due to docker.io rate limiting.

Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-05-18 14:22:52 -04:00
Soumya Koduri
f846f6ff02 rgw/qa: Add separate task for cloudtier tests
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-17 20:54:58 +05:30
Soumya Koduri
d0ac12dc38 rgw/qa: enable s3-tests related to cloud-transition feature
Added a new test-suite to configure cloudtier storage class and run related s3-tests.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-05-17 20:54:58 +05:30
Kotresh HR
2e1f43c99b client/fuse: Fix directory DACs overriding for root
DACs are overridable for directories. For files,
Read/write DACs are always overridable but executable
DACs are overridable when there is at least one exec bit
set.

The files and directory DACS overriding were handled the
same way for root which is incorrect. This patch fixes
DACs overriding as described above for the root.

Fixes: https://tracker.ceph.com/issues/55313
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-05-17 12:15:41 +05:30
Venky Shankar
44a0199114
Merge pull request #45834 from ajarr/wip-53996
qa/suites/fs/upgrade: change octopus installations and references

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-12 15:14:53 +05:30
Casey Bodley
98ddd40859 qa/rgw: use 'with-sse-s3' override for s3tests
don't rely on the ceph manager task to parse a config file. each rgw
could be using a different config. instead, revert to an s3tests
override called 'with-sse-s3'

this way, the only job that enables sse-s3, vault_transit.yaml, contains
both the 'rgw crypt sse s3' configurables, and the flag to enable the
associated test cases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-11 14:49:51 -04:00
Ali Maredia
d634f0690b
Merge pull request #44494 from mdw-at-linuxbox/wip-master-mankey
rgw/crypt: sse-s3 changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-05-02 16:57:55 -04:00
Ramana Raja
898240c8ee qa/suites/fs/upgrade: change octopus installations and references
... to pacific.

Fixes: https://tracker.ceph.com/issues/53996
Signed-off-by: Ramana Raja <rraja@redhat.com>
2022-05-02 14:53:18 -04:00
Adam King
9268e32047
Merge pull request #45899 from rkachach/fix_issue_53527
mgr/cephadm: fixing upgrade candidate versions listing

Reviewed-by: Adam King <adking@redhat.com>
2022-04-26 15:09:37 -04:00
Redouane Kachach
671442c4b4
mgr/cephadm: fixing upgrade candidate verions listing
Fixes: https://tracker.ceph.com/issues/53527

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-04-22 15:54:57 +02:00
Adam King
5720cca2f8
Merge pull request #45920 from adk3798/stop-mgr-nfs-upgrade-fail
qa/suites/orch/cephadm: stop upgrade tests if failures are seen

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-04-22 09:28:35 -04:00
Marcus Watts
20cb9805c9 qa/rgw - run sse-s3 test cases by default
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2022-04-19 17:35:40 -04:00
Casey Bodley
065b9aad86 qa/rgw: run sse-s3 test cases conditionally
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4e644832b0)
2022-04-19 17:35:39 -04:00
Casey Bodley
e346a75399 qa/rgw: configure sse-s3 in vault_transit.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1b679e0828)
2022-04-19 17:35:39 -04:00
Ionut Balutoiu
8054199787 qa: add basic Ceph on Windows integration test
Due to lack of Windows support in the Teuthology, the test case adopts
the following workaround:

* Deploy baremetal machine with `ubuntu_latest.yaml` and
  configure it with libvirt KVM.
* Create a libvirt VM and provision it with Windows Server 2019, using
  the official ISO from Microsoft.
* Configure SSH in the Windows VM, and run the tests remotely via SSH.

The implementation of the test case consists of workunit scripts.

`qa/workunits/windows/test_rbd_wnbd.py` is the main Python script
to test Ceph on Windows basic functionality. This is executed in the
libvirt VM configured with Windows Server 2019.

Co-authored-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-authored-by: Daniel Vincze <dvincze@cloudbasesolutions.com>

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2022-04-19 17:21:52 +03:00
Adam King
017aa9cfe8 qa/suites/orch/cephadm: stop upgrade tests if failures are seen
Otherwise the tests may run forever. This was already done for
mds upgrade sequence, justadding it in the other two places here

Related to: https://tracker.ceph.com/issues/53939

Signed-off-by: Adam King <adking@redhat.com>
2022-04-17 12:21:44 -04:00
Milind Changire
8d9bf97a58 qa: replace subvolume.sh with test_subvolume.py
qa/workunits/fs/misc/subvolume.sh is getting in the way of fs:workload
testing with subvolumes. Hence moved this script to a python test.

Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-04-10 21:35:23 +05:30
Laura Flores
bd82d21943
Merge pull request #45749 from josephsawaya/fix-rook-tests
Remove orchestrator from rook task and suite
2022-04-08 16:36:38 -05:00
Milind Changire
f6c68c32aa qa: add test fragments to test subvols
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-04-07 14:18:06 +05:30
David Galloway
fcd512b50b qa: Latest RHEL8 -> RHEL8.5
Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-04-01 15:00:01 -04:00
Joseph Sawaya
043a330ed0 Remove orchestrator from rook task and suite
This commit removes orchestrator commands from the
Rook task and the Rook test suite because the Rook
orchestrator is not being maintained, and the Rook
orchestrator CLI is obsolete. This should also
clarify the issue:

https://tracker.ceph.com/issues/53680

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2022-04-01 11:37:55 -04:00
Adam King
6e4dd0e082
Merge pull request #45361 from adk3798/mds-upgrade-test
qa/suites/fs: stop looping in mds upgrade test if upgrade failed

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-03-30 18:56:45 -04:00
Ilya Dryomov
f0ade57458 qa/suites/orch/cephadm: restrict test_iscsi_pids_limit to CentOS
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-22 21:42:41 +01:00
Teoman ONAY
de8b3c2676 cephadm: remove containers pids-limit
The default pids-limit (docker 4096/podman 2048) prevent some
customization from working (http threads on RGW) or limits the number
of luns per iscsi target.

Fixes: https://tracker.ceph.com/issues/52898
Signed-off-by: Teoman ONAY <tonay@redhat.com>
2022-03-22 21:41:26 +01:00
Adam King
37019aad80 qa/suites/fs: stop looping in mds upgrade test if upgrade failed
Signed-off-by: Adam King <adking@redhat.com>
2022-03-22 08:05:31 -04:00
Samuel Just
987465eea0
Merge pull request #45176 from liu-chunmei/crimson-teuthology
crimson: teuthology suite for seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-16 12:06:04 -07:00
chunmei-liu
3cca3ed552 qa/suites/crimson-rados: add fixed-1 for seastore
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-03-08 21:29:04 -08:00