Commit Graph

192 Commits

Author SHA1 Message Date
Casey Bodley
1942095a32 qa/rgw: rgw/multisite uses supported-random-distro$
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-12 12:06:29 -04:00
Casey Bodley
8f27c845be qa/rgw: move rgw/multisite valgrind config into valgrind.yaml.disabled
the valgrind coverage wasn't enabled, and it will take a lot more work
before we can reenable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-12 12:05:16 -04:00
Yuval Lifshitz
c43878d0bd
Merge pull request #40625 from TRYTOBE8TME/wip-rgw-notifications
rgw/test: Changing force-branch to master
2021-04-08 08:32:45 +03:00
Kalpesh
42d87989e9 rgw/test: Changing force-branch to master
This PR includes 2 things:
1. Changing force-branch to master and removing the git-remote. This change was forgetten for PR #39139.
2. Proper cleanup/removal after completion of commands more precisely removing the kafka logs directory.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-04-08 01:37:29 +05:30
Casey Bodley
b1830e43d2 qa/rgw: move ignore-pg-availability.yaml out of suites/rgw
this was added in https://github.com/ceph/ceph/pull/39820 to
avoid repeating the override in rgw subsuites, but teuthology
was picking up suites/rgw/ignore-pg-availability.yaml as a job
of its own, and led to 'list index out of range' failures

Fixes: https://tracker.ceph.com/issues/50147

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-05 13:09:52 -04:00
Yuval Lifshitz
b903b2e011
Merge pull request #39139 from TRYTOBE8TME/wip-rgw-bucket-tests-separation-new
Wip rgw bucket tests separation new
2021-03-18 20:33:00 +02:00
Casey Bodley
29f4bbb5ee qa/rgw: notifications suite runs single job
pin to the beast frontend, default bluestore, replicated pools, and run
against a random distro

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-03-18 11:06:40 +05:30
Casey Bodley
2bc016b52c
Merge pull request #39820 from cbodley/wip-qa-rgw-ignore-pg-avail
qa/rgw: put PG_AVAILABILITY ignorelist override in its own file

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-03-15 15:09:29 -04:00
Marcus Watts
e8ff0464d4 rgw/kms/vault - s3tests for both old and new test logic.
Test both "old" and "new" transit logic with s3tests.  Does not test
migration - that will need to be done separately.  I've added
a "flavor" parameter so the test logic can tell the difference
between the "old" engine and the "new" engine.  The vault
keys creation logic now has options to determine whether
the keys created are exportable (needed for the old transit
engine), or not (should be the case going forward with the
new transit engine.)

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2021-03-05 23:05:13 -05:00
Matt Benjamin
d2c30d7130
Merge pull request #33996 from mdw-at-linuxbox/wip-master-kmip
rgw / kmip kms
2021-03-05 16:20:18 -05:00
Sage Weil
9002400301 qa/suites/rgw/singleton: add trivial cephadm case
Make sure we can deploy an rgw and not generate any warnings.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-05 09:47:19 -05:00
Marcus Watts
809677d991 rgw/kms/kmip - rgw / kmip test integration.
Actually add kmip to the kms crypt suite.

This also makes some ssl certs which is required for use of kmip.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2021-03-03 19:14:10 -05:00
root
948a00b1d8 qa: Kafka task files for bucket notification tests
This commit consists of 3 things:
1. Files required for setting up new directory (in order to run the task in teuthology)
2. Kafka task file
3. The new files conataining tests and it's infrastructure for separation of bucket notification tests from pubsub tests

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-03-03 22:04:04 +05:30
Casey Bodley
4fe1b138ff qa/rgw: put PG_AVAILABILITY ignorelist override in its own file
now covers the sts and multifs subsuites

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-03-03 11:29:18 -05:00
Sage Weil
568eb9e3f3 Merge PR #39561 into master
* refs/pull/39561/head:
	qa/valgrind.supp: suppress free[] warnings
	qa/suites: do not use notcmalloc flavor

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-02-19 13:56:33 -05:00
Sage Weil
dc64ccf063 qa/suites: do not use notcmalloc flavor
teuthology now knows how to run valgrind against a tcmalloc binary

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-18 10:26:28 -06:00
Casey Bodley
6f2af91518 qa/rgw: add 'branch' option to rgw-multisite-tests task
Fixes: https://tracker.ceph.com/issues/49330

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-17 09:01:05 -05:00
Casey Bodley
c706740dbe test/rgw: re-enable valgrind in rgw verify suite
this only enables it for radosgw itself, not the other daemons

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-09 14:26:19 -05:00
Casey Bodley
58e0da8fc9 qa/rgw: move valgrind config into valgrind.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-09 14:26:19 -05:00
Casey Bodley
85641ac3e6 qa/rgw: add librgw_file unit tests to rgw/verify task
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-01-25 16:46:04 -05:00
Casey Bodley
e2f8f23d21 qa/rgw: don't add a certificate for nonexistent rgw.client.1
this override to the rgw task is causing the rgw/verify suite to start
an extra radosgw

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-01-13 10:58:42 -05:00
Kalpesh Pandya
64db3a9aca qa: STS Web Identity task files
This commit comprises of 3 things:
1) Keycloak task (to bring up keycloak in teuthology inorder to execute Assume_Role_With_Web_Identity tests).
2) The required changes for this in s3tests.py which consists of writing some parameters to configuration file.
3) The yaml files which executes these tasks.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2020-11-29 04:52:19 +05:30
Kalpesh Pandya
60904c907e qa : STS Testing Files (AssumeRole and GetSessionToken)
Added required files for testing of AssumeRole and GetSessionToken API's and modified s3tests.py to handle the same.

(cherry picked from commit c2c90eaf524087925b8a0993098a87132d4940bd)

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2020-09-15 11:58:59 +05:30
Sage Weil
2ee9365d0b qa: log-whitelist -> log-ignorelist
Signed-off-by: Sage Weil <sage@newdream.net>
2020-08-24 19:53:08 +00:00
Sage Weil
dfd01d7653 blacklist -> blocklist
Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Daniel Gryniewicz
00ccbc74bb RGW - Turn on debug logging for tempest tests
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2020-08-17 12:30:58 -04:00
J. Eric Ivancich
94f53f513c
Merge pull request #33492 from prasakri/multisite-sync-check
RGW:Multisite: Verify if the synced object is identical to source

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-06-08 19:49:46 -04:00
Neha Ojha
7cd88b31e8 qa/suites/rgw/multisite, qa/suites/rgw/crypt: whitelist PG_AVAILABILITY
This follows b162541ac21e965a304ee6ffe604c43f22fa96c4.
The balancer was turned on by default in
d4fbaf7, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.

Fixes: https://tracker.ceph.com/issues/45802
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-06-02 17:15:22 +00:00
Kefu Chai
8f5c832915 qa/tasks/tempest: use user/pass to authenticate
instead of using admin token use "admin" user to authenticate,
as admin token is not suggested anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
739751ef7a qa/suites/rgw/tempest: update unsupported tests of tempest
after rerunning tempest with lastest radosgw, remove the supported
tests from the blacklist, and add the ones which are not supported
yet. now we can pass 123 tests in total.

also enable discoverity for better testing coverage, since it's
supported now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
a53c69943c qa/tasks/keystone: use "keystone-manage bootstrap"
* qa/tasks/keystone.py:
  instead of prefilling keystone manually, use "keystone-manage bootstrap"
  instead. it helps to setup the admin user, a "Default" domain with
  "default" id, and wire them up with the expected role and a "admin" project,
  etc. as id of the admin domain is known to be "default", we can just use it
  in our tests without querying openstack for the id of "Default"
  domain. this is very handy.
* qa/suites/rgw/tempest/tasks/rgw_tempest.yaml:
  use "Default" for domain name. as "Default" is the name of the domain
  created by bootstrap, while "default" is its id.
* qa/suites/rgw/crypt/2-kms/barbican.yaml:
  remove settings to bootstrap keystone

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
8cb3f0411d qa/suites/rgw/tempest: use the latest tempest supporting py3.5
in case we need to use ubuntu xenial for testing, xenial only had python
3.5 packaged. and tempest 23.0 was the last version which supports
python3.5 and python2.7.

also do not replace link in tox.ini, as it is reachable.

to address the issues of

- pallets/markupsafe#116
- pypa/setuptools#2017

MarkupSafe is installed by
https://opendev.org/openstack/requirements/raw/branch/stable/pike/upper-constraints.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
9bd3e0ff40 qa/suites/rgw/tempest: bump up keystone to 17.0.0
* also generate a sample conf file following the document at
  https://github.com/openstack/keystone/tree/17.0.0.0rc2/etc
* use "projects" instead of "tenants" to match the terminology used by
  openstack identify API 3.0.
* test API 3.0 instead of API 2.0, by changing
  `rgw_keystone_api_version` from "2" to "3"
* explicitly specify a domain "default" for project to be created,
  otherwise a POST request will fail with:
```
{"error":{"code":400,"message":"You have tried to create a resource using the admin token. As this token is not within a domain you must explicitly include a domain for this resource to belong
to.","title":"Bad Request"}}
````
* create "default" domain, and use it, othewise a GET request fails
  like:
```
2020-05-28T11:17:28.751 INFO:teuthology.orchestra.run.smithi092.stderr:http://smithi092.front.sepia.ceph.com:35357 "GET /v3/domains/default HTTP/1.1" 404 87
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP: [404] Content-Length: 87 Content-Type: application/json Date: Thu, 28 May 2020 11:17:28 GMT Server: WSGIServer/0.2
CPython/3.6.9 Vary: X-Auth-Token x-openstack-request-id: req-bc33796f-2bc3-411c-a7fb-1208918e0dbd
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP BODY: {"error":{"code":404,"message":"Could not find domain: default.","title":"Not Found"}}
```
* add user to "default" domain when creating it.
* use "type" as the positional argument, per
  https://docs.openstack.org/keystone/pike/admin/cli-keystone-manage-services.html
  otherwise we will have failures like:
```
2020-05-28T13:38:24.867 INFO:teuthology.orchestra.run.smithi198.stderr:openstack service create: error: unrecognized arguments: --type keystone
```
* update `create_endpoint()` to use the V3 API,
  see
  https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html

Fixes: https://tracker.ceph.com/issues/45692
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Neha Ojha
b162541ac2 qa/suites/rgw/verify/tasks/cls.yaml: whitelist PG_AVAILABILITY
The balancer was turned on by default in
d4fbaf7ea9, as a result of which we might see
PG_AVAILABILITY health warnings when pg-upmap-items are applied.

Fixes: https://tracker.ceph.com/issues/45619
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-28 19:30:46 +00:00
J. Eric Ivancich
43c948a4ab qa/rgw: integration test for rgw-orphan-list & radosgw-admin radoslist
Add teuthology test for `rgw-orphan-list` in a new tool suite under
rgw. It only needs to be tested under one configuration. And the new
tool sub-suite can be used by other tooling int he
future. radosgw-admin `radoslist` is tested indirectly through
`rgw-orphan-list` and therefore does not need its own test.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-04 13:55:00 -04:00
Casey Bodley
67415ea07b
Merge pull request #34286 from cbodley/wip-qa-rgw-cls-2pc-queue
qa/rgw: run ceph_test_cls_2pc_queue in verify suite

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-05-04 12:29:14 -04:00
Shilpa Jagannath
71b657b9d2 minor fix: run tests in python venv
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-30 12:39:50 +05:30
Shilpa Jagannath
0270dc4aaa qa/rgw: some basic reshard tests
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-27 13:16:09 +05:30
Casey Bodley
94ca8f4b81 qa/rgw: add setuser/group to test DEFER_DROP_PRIVILEGES
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-17 10:45:15 -04:00
Casey Bodley
0f44806c53 qa/rgw: run ceph_test_cls_2pc_queue in verify suite
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-14 11:43:05 -04:00
Casey Bodley
577f211391 test: add ceph_test_cls_cmpomap
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-04-13 09:23:44 -04:00
Prasad Krishnan
fa54225978 RGW:Multisite: Convert is_mpu_obj into an enum SourceObjType
Signed-off-by: Prasad Krishnan <prasad.krishnan@flipkart.com>
2020-03-20 20:01:35 +00:00
Sage Weil
86a27e5a9a Merge PR #33047 into master
* refs/pull/33047/head:
	qa/rgw: exercise DeleteRange in test_bucket_index_log_trim

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
2020-03-04 09:58:45 -06:00
Casey Bodley
08e0a4289d qa/rgw: verify suite selects a random bucket sharding configuration
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-02-28 14:38:13 -05:00
Casey Bodley
2aa526b96a qa/rgw: exercise DeleteRange in test_bucket_index_log_trim
test_bucket_index_log_trim doesn't create enough keys to exceed the
default threshold, so set it to 0

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-02-03 11:46:59 -05:00
Casey Bodley
d486b5bc45 qa/rgw: test with non-default rgw-obj-stripe-size
each job will select one of the striping strategies at random

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-02-03 11:24:52 -05:00
Sage Weil
acd0e31177 qa/suites/rgw/website: run test on ubuntu
The dnsmasq package on centos 8.0 is broken, see
https://tracker.ceph.com/issues/43744

For now, run this test on ubuntu.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-22 15:17:57 -06:00
Ali Maredia
2552974fe7 qa: pin rgw/verify to 8.0
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-01-21 13:02:42 -05:00
Sage Weil
5c1e0e5f1f qa/suite/rgw/verify: valgrind on centos again!
Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-19 18:47:13 -06:00
Kefu Chai
7b0973e33b
Merge pull request #32624 from alimaredia/wip-s3-tests-python-3
qa/suites: use s3-tests with python3 support

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-17 23:47:51 +08:00