Commit Graph

10072 Commits

Author SHA1 Message Date
Yuri Weinstein
95627c9673
Merge pull request #51909 from NitzanMordhai/wip-nitzan-divergent-priors-test-3-pick-osd
qa/standalone/osd/divergent-prior.sh: Divergent test 3 with pg_autoscale_mode on pick divergent osd

Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2023-07-12 14:36:45 -04:00
Nizamudeen A
add6bf99b0
Merge pull request #52342 from ljflores/wip-whitelist-api
qa/tasks: set defer_client_eviction_on_laggy_osds=false in api tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-07-12 10:21:45 +05:30
Laura Flores
fe6ae275f0
Merge pull request #52345 from ljflores/wip-tracker-59057
qa/suites/rados/singleton/all: remove test_envlibrados_for_rocksdb
2023-07-11 17:04:32 -05:00
Laura Flores
2322d2c8e0 qa/tasks: set defer_client_eviction_on_laggy_osds=false in api tests
We expect laggy OSDs in this testing environment,
so it makes sense to disable this warning.

Fixes: https://tracker.ceph.com/issues/61907
Signed-off-by: Laura Flores <lflores@redhat.com>
2023-07-11 15:33:12 -05:00
Yuri Weinstein
6e02660f10
Merge pull request #51275 from mchangir/mon-block-osd-pool-mksnap-for-fs-pools
mon: block osd pool mksnap for fs pools


Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-07-07 17:09:07 -04:00
Adam King
b985fadc48
Merge pull request #52301 from adk3798/cephadm-fix-rgw-migration
mgr/cephadm: fix rgw spec migration with simple specs

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-07-07 14:33:11 -04:00
Laura Flores
2fca433c71 qa/suites/rados/singleton/all: remove test_envlibrados_for_rocksdb
In rocksdb 7.0, all envlibrados files were moved to a separate repository (ref: https://github.com/facebook/rocksdb/pull/9206).
The new repo is temporary and serves as an example before it is finalized where and who to host RADOS support.

Since this new repo is outside of the rocksdb repo and in an unceratin state, we should remove support for it in main
and Reef test suites. Quincy and below still use rocksdb 6.0, so the same does not apply.

Fixes: https://tracker.ceph.com/issues/59057
Signed-off-by: Laura Flores <lflores@redhat.com>
2023-07-06 12:14:05 -05:00
Casey Bodley
744c8b0841
Merge pull request #52216 from cbodley/wip-61838
qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2023-07-06 09:59:31 -04:00
Adam King
1860ef8387 mgr/cephadm: fix rgw spec migration with simple specs
The rgw spec migration code, intended to formalize
the rgw_frontend_type spec option, doesn't work with
simple specs i.e.

service_type: rgw
service_id: rgw.1
service_name: rgw.rgw.1
placement:
  label: rgw

because the migration code assumes there will always be
a "spec" section inside the spec. This is the case for
more involved rgw specs such as

service_type: rgw
service_id: foo
placement:
  label: rgw
  count_per_host: 2
spec:
  rgw_realm: myrealm
  rgw_zone: myzone
  rgw_frontend_type: "beast"
  rgw_frontend_port: 5000

which is what the migration is actually concerned about
(verification of the rgw_frontend_type in these specs).

In the case where the spec is more simple, we should
just leave the spec alone and move on. Unfortunately
the current code assumes the field will always be
there and hits an unhandled KeyError when trying to
migrate the more simple specs. This causes the
cephadm module to crash shortly after starting an
upgrade to a version that includes this migration
and it's very difficult to find the root cause. This
can be worked around by adding fields to the rgw
spec before upgrade so the "spec" field exists in
the spec and the migration works as intended.

This commit fixes the migration in the simple
case as well as adding testing for that case to
both the unit tests and orch/cephadm teuthology
upgrade tests

Fixes: https://tracker.ceph.com/issues/61889

Signed-off-by: Adam King <adking@redhat.com>
2023-07-05 11:01:25 -04:00
Matan Breizman
780f6f55cd qa/tasks/cbt.py: Install collectl manually
See: d4a04809fd

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-04 11:42:34 +00:00
Matan Breizman
abdbfa2275 qa/suites/crimson-rados: Introduce performance testing
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-04 11:42:34 +00:00
J. Eric Ivancich
fb9d481188
Merge pull request #50667 from ivancich/wip-add-reupload-testing
qa/rgw: test that multipart re-upload does not leave any orphans

Reviewed-by: Cory Snyder <csnyder@iland.com>
2023-06-30 11:30:27 -04:00
Venky Shankar
809d475814 Merge PR #49971 into main
* refs/pull/49971/head:
	doc/cephfs: document MDS_CLIENTS_LAGGY health warning
	qa: ignore warnings
	qa: add test cases to check client eviction if an OSD is laggy
	mds,messages: enable beacon to report clients lagginess
	mds: do not evict client on laggy osds
	common: add new config option to defer client eviction
	osd: add method to check for laggy osds

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-06-28 10:23:54 +05:30
Casey Bodley
7d981bfa1a qa/rgw/sts: rename tasks, don't duplicate s3tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-27 17:05:38 -04:00
Casey Bodley
17486bddda qa/rgw/sts: move overrides into overrides.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-27 17:05:38 -04:00
Casey Bodley
09e10793e1 qa/s3tests: filter on 'sts_tests' and 'webidentity_tests'
qa/tasks/s3tests.py was adding `not test_of_sts` and `not
webidentity_test` by default, and the rgw/sts suites were adding
those attrs back in `extra_attrs`

when `extra_attrs` was changed to be additive, this started causing
InvocationError failures

instead of using `extra_attrs` to control these filters,
qa/tasks/s3tests.py now uses the `sts_tests` and `webidentity_tests`
flags from the sts yaml files to control whether or not the `not
test_of_sts`/`not webidentity_test` attrs are added to the pytest
command line

Fixes: https://tracker.ceph.com/issues/61838

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-27 17:05:24 -04:00
Venky Shankar
f370b581f6 qa: assign file system affinity for replaced MDS
Otherwise, the MDS that just got replaced can transition to a rank
for another file system and the test cannot deterministically infer
which MDS needs to checked.

Fixes: http://tracker.ceph.com/issues/61764
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-06-27 09:23:14 +05:30
Ali Maredia
26fda85af2
Merge pull request #52156 from cbodley/wip-qa-rgw-s3tests-extra-attrs
qa/rgw/s3tests: make extra_attrs additive
2023-06-22 16:15:02 -04:00
Casey Bodley
39e5f096d1
Merge pull request #52085 from alimaredia/wip-rgw-tempest-keystone-fix
Tempest tests Keystone Fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-06-22 13:46:46 -04:00
Ali Maredia
703effd45b qa: rgw/tempest suite only runs on ubuntu latest
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2023-06-22 11:59:59 -04:00
Tobias Urdin
fffa119424 qa: update rgw openstack versions
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2023-06-22 11:59:53 -04:00
Adam King
54413fd570
Merge pull request #51738 from phlogistonjohn/jjm-cephadm-new-deploy
cephadm: create a new version of the deploy command

Reviewed-by: Adam King <adking@redhat.com>
2023-06-22 11:39:49 -04:00
Casey Bodley
596e32f542 qa/s3tests: remove stale comment about rfc2616
this comment was about the old fails_strict_rfc2616 attr that was
removed in 6238eaaf0b

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-22 08:54:33 -04:00
Casey Bodley
1e65b50f60 qa/s3tests: clean up extra_attrs and extra_args logic
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-22 08:54:33 -04:00
Casey Bodley
1c006c92e1 qa/s3tests: make extra_attrs additive
the s3tests.py task is filtering out several attrs by default. but
when dbstore uses `extra_attrs` to add 'not fails_on_dbstore', it
overwrites those other filters

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-22 08:54:14 -04:00
Yuri Weinstein
d05da0b4d1
Merge pull request #52072 from NitzanMordhai/wip-nitzan-add-ec-clay-teuthology-tests
qa: adding clay test to thrash erasure code big

Reviewed-by: Neha Ojha <nojha@redhat.com>
2023-06-21 17:16:00 -04:00
Yuri Weinstein
b1ed114a66
Merge pull request #51450 from batrick/i59716
pybind/rados: keep byte representation if decode fails

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-06-21 17:12:08 -04:00
John Mulligan
f890d82478 qa/workunits/cephadm: update script use use cephadm _orch deploy
cephadm _orch deploy will eventually be the only deploy method so
we need to update the test script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-06-21 13:19:16 -04:00
Ilya Dryomov
ca20127c70
Merge pull request #52109 from idryomov/wip-61616
librbd: avoid object map corruption in snapshots taken under I/O

Reviewed-by: Mykola Golub <mgolub@suse.com>
2023-06-21 11:43:26 +02:00
Ilya Dryomov
acb270a3dd qa/workunits/rbd: make continuous export-diff test actually work
The current version is pretty useless:

- "rbd bench" writes the same byte (0xff) over and over again, so
  almost all checksumming is in vain
- snapshots are taken in a steady state (i.e. not under I/O), so no
  race conditions can get exposed
- even with these caveats, it's not wired up into the suite

Redo this workunit to be a reliable reproducer for the issue fixed
in the previous commit and wire it up for both krbd and rbd-nbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-06-20 22:14:39 +02:00
Yuri Weinstein
5ae95880bf
Merge pull request #48209 from kamoltat/wip-ksirivad-fix-tracker-57570
osd/OSDMap: Check for uneven weights & != 2 buckets post stretch mode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2023-06-19 13:29:21 -04:00
Rishabh Dave
67b1935a18
Merge pull request #51132 from lxbsz/wip-59349
qa: wait for 100 seconds to make sure the quota to be enforced

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-06-16 17:52:00 +05:30
NitzanMordhai
293f13ed49 qa: adding clay test to thrash erasure code big
Currently we don't have any clay test in the erasure code big tests
adding also clay tests.

Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2023-06-16 06:58:32 +00:00
Adam King
8e5a983a73
Merge pull request #51267 from phlogistonjohn/jjm-cephadm-versioning
Support cephadm "binary" versioning

Reviewed-by: Adam King <adking@redhat.com>
2023-06-14 14:23:27 -04:00
Casey Bodley
212745267d
Merge pull request #52012 from cbodley/wip-61428
qa: suppress SyscallParam error during startup on jammy

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2023-06-13 13:42:05 -04:00
Matan
eeae19fba3
Merge pull request #51973 from Matan-B/wip-matan-lsan-supp
qa/lsan.supp: update heap_profiler suppression and ASAN_OPTIONS

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-06-13 09:11:14 +02:00
Xiubo Li
6183c992d7
Merge pull request #51703 from lxbsz/wip-59683
xfstests_dev: install extra packages from powertools repo for xfsprogs
2023-06-13 09:44:24 +08:00
Xiubo Li
4a60f6749a
Merge pull request #50728 from lxbsz/wip-59195
qa: switch to use the merge fragment for fscrypt
2023-06-13 07:39:00 +08:00
Casey Bodley
522251642e qa: suppress SyscallParam error during startup on jammy
Fixes: https://tracker.ceph.com/issues/61428

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-12 16:25:56 -04:00
Xiubo Li
dedf3aae65 xfstests_dev: install extra packages from powertools repo for xfsprogs
Centos Stream 8 has removed the 'device-mapper-devel', 'libedit-devel'
and 'userspace-rcu-devel' packages from the mirrors and we need to
install it from powertools repo.

Fixes: https://tracker.ceph.com/issues/59683
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-06-12 15:08:28 +08:00
Casey Bodley
4167b5c797
Merge pull request #48879 from samarahu/wip-d4n
RGW: WIP D4N

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-06-08 11:44:55 -04:00
Matan Breizman
d074e910df qa/lsan.supp: update heap_profiler suppression and ASAN_OPTIONS
In continuation to: 8c099a5340

Regsiter singleton leak seems to be replaced by:
```
-----------------------------------------------------
Suppressions used:
  count      bytes template
      1          8 ^InitModule
-----------------------------------------------------
```

In similarity to Crimson's suppression: 6ed8d839b4

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-06-08 14:16:01 +00:00
Nizamudeen A
098f175d71
Merge pull request #51955 from rhcs-dashboard/teuth-e2e-fix
qa/workunits: fix test_dashboard_e2e.sh: no spec files found

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-06-08 14:18:14 +05:30
Ronen Friedman
6620562d85
Merge pull request #51911 from ronen-fr/wip-rf-recovery-updates
qa/standalone: osd-recovery-scrub: fix slow updates and recovery concurrency

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
2023-06-08 11:43:59 +03:00
Nizamudeen A
bec3140b20 qa/workunits: fix test_dashboard_e2e.sh: no spec files found
Fixes: https://tracker.ceph.com/issues/61578
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-06-07 14:04:33 +05:30
Patrick Donnelly
1d3a549ec2
qa: test first-damage.py with binary file name
Specifically, one which cannot be decoded as utf8.

Fixes: https://tracker.ceph.com/issues/59716
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-06-06 21:39:42 -04:00
Patrick Donnelly
b6867bb255
qa: add option to set python executable
To support systems like mine with multiple python installs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-06-06 21:39:41 -04:00
Laura Flores
8c6374e8da
Merge pull request #51927 from ljflores/wip-rook-tests
qa/suites/rados: remove rook coverage from the rados suite
2023-06-06 13:35:35 -05:00
Samarah
cd1d249678 QA: Add D4N teuthology suite
Signed-off-by: Samarah <samarah.uriarte@ibm.com>
2023-06-06 14:04:47 -04:00
Nitzan Mordechai
13c640b5a8 test: Divergent test 3 with pg_autoscale_mode on pick divergent osd
When creating new pool, the current code pick the divergent osd by
the first pg out of pg dump pgs, that can be in "unknown" status
which means the up_primary = -1 and that will fail the test.
We need to wait unitl the first pg is active+clean

Fixes: https://tracker.ceph.com/issues/56034
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2023-06-06 05:54:39 +00:00