Commit Graph

45 Commits

Author SHA1 Message Date
Kalpesh Pandya
60904c907e qa : STS Testing Files (AssumeRole and GetSessionToken)
Added required files for testing of AssumeRole and GetSessionToken API's and modified s3tests.py to handle the same.

(cherry picked from commit c2c90eaf52)

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2020-09-15 11:58:59 +05:30
Kyr Shatskyy
552e6a771b qa/tasks/s3tests: use remote.write_file instead of misc.write_file
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-09-04 00:02:15 +02:00
Casey Bodley
f0cc5a8c50
Merge pull request #35910 from galsalomon66/add_s3select_filter
adding filter for s3select

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-07-08 08:52:22 -04:00
Kefu Chai
5d56234b45 qa/tasks/s3tests: drop py2 support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-05 10:58:28 +08:00
gal salomon
873871d6f1 adding filter for s3select
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2020-07-03 06:43:28 +03:00
Kyr Shatskyy
478083e731 qa/tasks/s3tests: py3 compat
- use string.ascii_uppercase instead string.uppercase
- use six.ensure_str for bytes when required
- use six.ensure_binary if needed
- get rid of dict.itervalues in favor of dict.values
- get rid of cStringIO.StringIO in favor io.BytesIO

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kefu Chai
4c6a5798c2
Merge pull request #32222 from toabctl/qa-flake8-py3
qa: Run flake8 on python2 and python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-24 10:47:07 +08:00
Abhishek L
c195b78049
Merge pull request #31127 from theanalyst/rgw/qa-s3-user-caps
qa: rgw: add user-policy caps for the s3tests users

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Pritha Srivastava <prsivas@redhat.com>
2019-12-20 18:13:03 +01:00
Sage Weil
3268ec7ac8 Merge PR #32252 into master
* refs/pull/32252/head:
	qa/cephfs/begin: libaio-devel on el8
	qa/tasks: nosetests -> python -m nose
	qa/tasks/rbd_fio: fio 2.21 -> 3.16
	src/test/cli-integration/rbd/snap-diff.t: python -> python
	qa/workunits: use nose 3
	qa/tasks/cbt: install python3 deps
	qa/tasks/ceph_manager.py: do not use python to write a file
	test/pybind/test_rados: execute takes a bytes (not str) payload
	qa/packages/packages: python[3]-ceph is no more
	qa: use python3 for venvs etc
	packaging: remove python3-ipaddres, as it is part of the stdlib in py3
	qa/packages: python-ceph -> python3-ceph
	qa/distros: centos7 -> centos8, rhel7 -> rhel8
	spec: remove _python_buildid in favor of python3_pkgversion macro
	spec: remove python2 packages and conditions
	debian: remove python >= 2.7 requirement
	debian: add mgr python versions
	debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
	debian: update control file to use python3 dependency names
	debian: remove all python2 overrides and declarations
	debian: remove all python2 install files

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-12-17 15:23:27 -06:00
Sage Weil
6476517def qa/tasks: nosetests -> python -m nose
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-17 12:57:57 -06:00
Thomas Bechtold
bdcc94a1d1 qa: Run flake8 on python2 and python3
To be able to catch problems with python2 *and* python3, run flake8
with both versions. From the flake8 homepage:

It is very important to install Flake8 on the correct version of
Python for your needs. If you want Flake8 to properly parse new
language features in Python 3.5 (for example), you need it to be
installed on 3.5 for Flake8 to understand those features. In many
ways, Flake8 is tied to the version of Python on which it runs.

Also fix the problems with python3 on the way.
Note: This requires now the six module for teuthology. But this is
already an install_require in teuthology itself.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-13 09:24:20 +01:00
Thomas Bechtold
0127cd1e88 qa: Enable flake8 tox and fix failures
There were a couple of problems found by flake8 in the qa/
directory (most of them fixed now). Enabling flake8 during the usual
check runs hopefully avoids adding new issues in the future.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-12 10:21:01 +01:00
Kefu Chai
0e52da069b
Merge pull request #31839 from alimaredia/wip-s3-test-branch-refactor
qa: update s3-test download code for s3-test tasks

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-12-11 09:28:58 +08:00
Ali Maredia
f43c0cf454 qa: update s3-test download code for s3-test tasks
- Ensure the download code for all tasks running
s3-tests is consistent.
- Simplify download code to only use the config
variable 'force-branch' for the branch being
cloned.
- make ceph-master the force-branch for all
suites using s3-tests.

Fixes: https://tracker.ceph.com/issues/43077

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2019-12-10 12:39:28 -05:00
Abhishek Lekshmanan
ac8bc39d54 qa: rgw: add user-policy caps for the s3tests users
This allows user-policy based user tests to be written

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-12-04 14:32:21 +01:00
Sergio de Carvalho
f9d7b685c1 rgw: improvements to SSE-KMS with Vault
* Minor improvements to Vault documentation
* Add teuthology tests for Transit secrets engine
* Add unit tests for KV secrets engine, minor improvements to Transit
  secrets engine
* use string_view::npos instead of string::npos

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-11-28 09:19:39 +00:00
Kyr Shatskyy
50b1823fac qa: get rid of iteritems for python3 compatibility
Fixes: https://tracker.ceph.com/issues/42267
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2019-10-15 11:27:39 +02:00
Kyr Shatskyy
5f95b532aa qa: get rid of iterkeys for py3 compatibility
Fixes: https://tracker.ceph.com/issues/42287

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2019-10-11 18:54:29 +02:00
Andrea Baglioni
955d3b517a rgw: add SSE-KMS with Vault using token auth
Restructure SSE-KMS tests which now has 3 scenarios for each KMS
backend: Barbican, Vault, and testing (keys stored in ceph.conf).

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-10-02 16:38:01 +01:00
Casey Bodley
97f1be5d3a qa/rgw: rgw/website suite uses subdomain calling format
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-17 08:24:52 -04:00
Casey Bodley
71312246ad qa/rgw: s3test tasks sets s3website_domain to match dns-s3website-name
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-17 08:24:51 -04:00
Casey Bodley
7a84a00b6b qa/s3tests: use correct endpoint from rgw_server
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-17 08:22:21 -04:00
Casey Bodley
1feca42f4c qa/rgw: fix for extra_args in s3tests task
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-17 08:19:28 -04:00
Adam Kupczyk
a5ec971402 tasks/barbican: Added task to deploy and configure barbican
Fixed keystone, rgw and s3tests tasks to work with it.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2019-09-13 15:04:43 -04:00
Casey Bodley
09e992ff01 qa/rgw: extra s3tests tasks use rgw endpoint configuration
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-19 07:50:35 -04:00
Enming Zhang
ad6edd4dd2 qa: fix for s3tests.functional.test_s3.test_bucket_get_location failed
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
2018-06-23 17:09:54 +08:00
Abhishek Lekshmanan
f8ed817ccc qa: drop rfc2616 checking on civetweb
Since newer versions of civetweb are also strict on rfc2616 checks let's enforce
strict rfc2616 checks in s3tests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-04-13 19:19:13 +02:00
Yehuda Sadeh
9569cc8278 qa/tasks/s3tests: configure mfa tests
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:02:17 -07:00
Casey Bodley
2aec7e1f1d qa/rgw: disable s3tests for lifecycle_expiration
the tests are dependent on timing and often fail in teuthology

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-03-13 16:16:06 -04:00
Casey Bodley
9641b97a32 qa/rgw: s3test task overrides REQUESTS_CA_BUNDLE to use system certs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-03-09 16:31:33 -05:00
Casey Bodley
aee4924e32 qa/rgw: s3test task conditionally sets is_secure
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-03-09 16:31:33 -05:00
Casey Bodley
d4c33f0177 qa/rgw: add class for rgw endpoints
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-03-09 16:31:32 -05:00
Sage Weil
acf4ea1cad
Merge branch 'master' into lc-time-disable 2017-12-29 13:38:37 -06:00
Casey Bodley
607b72f553 qa/rgw: add beast frontend to some rgw suites
added a qa/rgw_frontend directory for civetweb.yaml and the new
beast.yaml. the rgw suites for multifs and singleton now symlink
rgw_frontend/civetweb.yaml. the multisite, tempest and verify suites
symlink rgw_frontend to test both. this doubles the number of jobs in
those suites

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-27 11:20:52 -04:00
Abhishek Lekshmanan
29de857c63 qa: s3: re enable LC tests on ceph-master
Needs to go in after https://github.com/ceph/s3-tests/pull/187 gets
merged

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-09-20 10:02:37 +02:00
Casey Bodley
0debf4dc6e qa/rgw: disable lifecycle tests because of expiration failures
lifecycle expiration tests are too reliant on timing, and have been
failing consistently for a long time

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-08-02 11:06:35 -04:00
Pritha Srivastava
5e94a9852c rgw: Changes for s3test config file, to add user under a tenant.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2017-06-20 12:57:24 +05:30
Casey Bodley
8c74c8a639 qa/rgw: remove apache/fastcgi
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-19 16:05:36 -04:00
Casey Bodley
762e15fbb3 qa/rgw: remove radosgw-agent config from s3tests task
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-19 15:53:37 -04:00
Casey Bodley
a31aa6f65c qa/rgw: add cluster name to path when s3tests scans rgw log
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-27 14:48:40 -04:00
Casey Bodley
a4fc5c38e5 qa/rgw: don't scan radosgw logs for encryption keys on jewel upgrade test
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-20 14:49:04 -04:00
Casey Bodley
da7acc4211 Merge pull request #13597 from cbodley/wip-s3tests-crypto
qa/rgw: add configuration for server-side encryption tests

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-04-18 12:28:37 -04:00
Ali Maredia
c5956790e6 rgw: multisite enabled over multiple clusters
Added '--cluster' to all necessary commands
ex: radosgw-admin, rados, ceph, made sure
necessary checks were in place so that clients
can be read with our without a cluster_name
preceeding them

Made master_client defined in the config for
radosgw-admin task

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-04-13 12:15:50 -04:00
Casey Bodley
9730fec922 qa: s3test task scans radosgw logs for leaked encryption keys
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-03 10:44:58 -04:00
Sage Weil
c01f2ee0e2 move ceph-qa-suite dirs into qa/ 2016-12-14 11:29:55 -06:00