Commit Graph

1690 Commits

Author SHA1 Message Date
Kefu Chai
f12dbaeaa3
Merge pull request #32427 from tchaikov/qa-devices-schema-with-path
qa: add "path" to "device" output schema

Reviewed-by: Sage Weil <sage@redhat.com>
2019-12-28 12:18:15 +08:00
Kefu Chai
5fe27c2267 qa/tasks/vstart_runner: write string to StringIO
we use `six.StringIO` for stdout when writing output messages from
`self.subproc.communicate()`, and `six.StringIO` accepts strings, so we
need to decode the output before sending them to `six.StringIO`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-28 01:11:54 +08:00
Kefu Chai
570f18f4c8 qa: add "path" to "device" output schema
"path" was added by 2c0fd7d868, so update
the test accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-27 00:25:51 +08:00
Kefu Chai
4c6a5798c2
Merge pull request #32222 from toabctl/qa-flake8-py3
qa: Run flake8 on python2 and python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-24 10:47:07 +08:00
Sage Weil
5d8635d0d3 Merge PR #32377 into master
* refs/pull/32377/head:
	qa/suites/rados/thrash-old-clients: configure mons in terms of addrvecs
	qa/suites/rados/thrash-old-clients: hammer: fix package list
	qa/tasks/cephadm: set .conf to cluster config object
	qa/tasks/cephadm: archive /var/log/ceph logs too (not just cluster dir)
	qa/tasks/cephadm: client keyring
	qa/tasks/cephadm: setup thrashers ctx item
	qa/tasks/ceph_manager: asok commands via cephadm shell
	qa/suites/rados/thrash-old-clients: stick to el7
	qa/tasks/cephadm: check cluster log; support log-whitelist
	qa/suites/rados/thrash-old-clienets: python-foo to python3-foo
	qa/suites/rados/thrash-old-clients: add new exclude_packages
	qa/suites/rados/thrash-old-clients: use cephadm
	mon/ConfigMonitor: make legacy mon addr/port parseable by legacy code

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-12-23 17:54:30 -06:00
Sage Weil
31ba596a63 qa/tasks/cephadm: set .conf to cluster config object
ceph.py does this and ceph_manager.py kill_osd relies on it.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 16:35:09 -06:00
Sage Weil
272225c4be qa/tasks/cephadm: archive /var/log/ceph logs too (not just cluster dir)
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 16:35:09 -06:00
Sage Weil
62806d4df2 qa/tasks/cephadm: client keyring
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 16:35:09 -06:00
Sage Weil
c5a3d67962 qa/tasks/cephadm: setup thrashers ctx item
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 13:59:26 -06:00
Sage Weil
4e57785e74 qa/tasks/ceph_manager: asok commands via cephadm shell
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 13:59:26 -06:00
Sage Weil
e07330c084 mgr/cephadm: add 'reconfig' service action
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 10:50:23 -06:00
Sage Weil
65b4025635 qa/tasks/cephadm: check cluster log; support log-whitelist
Mostly just lifted from ceph.py

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Sage Weil
2b2c122fd8 qa/tasks/mgr/test_orchestrator_cli: fix device ls test
No more "$host:" formatting as of 594179f1aa

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 16:14:34 -06:00
Samuel Just
bdc3ed252e
Merge pull request #32381 from athanatos/sjust/wip-read-from-replica-py2
osd: propagate mlcod to replicas and fix problems with read from replica

Reviewed-by: Sage Weil <sage@redhat.com>
2019-12-20 13:07:56 -08:00
Sage Weil
7e19a507f9 Merge PR #32375 into master
* refs/pull/32375/head:
	qa/tasks/cephadm: ceph.git branches are now pushed to quay.io

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-12-20 14:57:00 -06:00
Abhishek L
c195b78049
Merge pull request #31127 from theanalyst/rgw/qa-s3-user-caps
qa: rgw: add user-policy caps for the s3tests users

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Pritha Srivastava <prsivas@redhat.com>
2019-12-20 18:13:03 +01:00
Sage Weil
4a566bdc2d Merge PR #32362 into master
* refs/pull/32362/head:
	qa/packages/packages: el8 has granular -debuginfo
	qa/tasks/cbt: include py2 deps on ubuntu for now
	src/test: misc python -> python3
	qa/suites/rados/singleton-flag/valgrind-leaks: run on latest centos
	qa/workunits: env python -> env python3
	qa/suites/rados/dashboard/tasks/dashboard: whitelist OSDMAP_FLAGS
	qa/suites/rados/verify: ping to specific centos
	qa/workunits/rados/test_envlibrados_rocksdb: enable el8 PowerTools
	qa/workunits/rados/test_pool_access.sh: python -> python3
	mgr/crash: fix signature for py3

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-12-20 09:29:47 -06:00
Sage Weil
895a5e1427 qa/tasks/cephadm: ceph.git branches are now pushed to quay.io
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 08:56:09 -06:00
Kefu Chai
130e3eddb9 qa/tasks/vstart_runner.py: use current python executable
python3 is not always the path to executable used by current python
session.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-20 21:39:49 +08:00
Sage Weil
3f2270a1c4 qa/tasks/cbt: include py2 deps on ubuntu for now
...until cbt is converted to py3

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 07:17:10 -06:00
Sage Weil
e689caa880 qa/tasks/rados: add {localize,balance}_reads field
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 17:35:36 -08:00
Sage Weil
cfef72ede7 Merge PR #32359 into master
* refs/pull/32359/head:
	qa/workunits: python -> python3 throughout
	qa/workunits/fs/misc/direct_io.py: fix for py3
	qa/tasks/cephfs: os.write takes bytes, not str

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-19 16:21:44 -06:00
Sage Weil
f9bd3d7673 Merge PR #32357 into master
* refs/pull/32357/head:
	qa/tasks/swift: remove swift tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-12-19 16:17:06 -06:00
Patrick Donnelly
49a093d79a
Merge PR #32168 into master
* refs/pull/32168/head:
	test_cephfs_shell: fix test_du_works_for_hardlinks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-19 11:54:36 -08:00
Sage Weil
aa9272d872 qa/tasks/swift: remove swift tests
- This is an ancient swift version
- The tempest tests are newer and show provide similar coverage
- It somehow broke with the py3 transition

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 12:21:21 -06:00
Sage Weil
05eb2f647d qa/tasks/cephfs: os.write takes bytes, not str
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 11:12:04 -06:00
Sage Weil
1a624493c7 Merge PR #32335 into master
* refs/pull/32335/head:
	qa/tasks/s3tests_java: move to gradle 6.0.1

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2019-12-19 10:16:10 -06:00
Sage Weil
3fb581c86f Merge PR #32339 into master
* refs/pull/32339/head:
	qa/tasks/cephfs: python3
	qa/workunits/fs: python3
	qa/tasks/cephfs/fuse_mount: use python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-19 08:13:27 -06:00
Kefu Chai
79584862f6 src,qa: install python bindings into lib/cython_modules/lib.3
instead of checking for the WITH_PYTHON* options, just hardwire to lib.3

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:54 +08:00
Sage Weil
3c751fea7f qa/tasks/cephfs: python3
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-18 21:51:52 -06:00
Sage Weil
1de47907c3 qa/tasks/cephfs/fuse_mount: use python3
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-18 17:13:57 -06:00
Sage Weil
f71bb6c731 qa/tasks/s3tests_java: move to gradle 6.0.1
Also switch to using a stable symlink so that the version is determined by the branch.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-18 17:04:20 -06:00
Sage Weil
3268ec7ac8 Merge PR #32252 into master
* refs/pull/32252/head:
	qa/cephfs/begin: libaio-devel on el8
	qa/tasks: nosetests -> python -m nose
	qa/tasks/rbd_fio: fio 2.21 -> 3.16
	src/test/cli-integration/rbd/snap-diff.t: python -> python
	qa/workunits: use nose 3
	qa/tasks/cbt: install python3 deps
	qa/tasks/ceph_manager.py: do not use python to write a file
	test/pybind/test_rados: execute takes a bytes (not str) payload
	qa/packages/packages: python[3]-ceph is no more
	qa: use python3 for venvs etc
	packaging: remove python3-ipaddres, as it is part of the stdlib in py3
	qa/packages: python-ceph -> python3-ceph
	qa/distros: centos7 -> centos8, rhel7 -> rhel8
	spec: remove _python_buildid in favor of python3_pkgversion macro
	spec: remove python2 packages and conditions
	debian: remove python >= 2.7 requirement
	debian: add mgr python versions
	debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
	debian: update control file to use python3 dependency names
	debian: remove all python2 overrides and declarations
	debian: remove all python2 install files

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-12-17 15:23:27 -06:00
Sage Weil
6476517def qa/tasks: nosetests -> python -m nose
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-17 12:57:57 -06:00
Patrick Donnelly
b31a020008
Merge PR #31884 into master
* refs/pull/31884/head:
	qa/tasks: Fix the volume ls in test_volume_rm
	qa/tasks: tests for 'fs volume create' and 'fs volume ls'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-16 16:30:45 -08:00
Patrick Donnelly
4ef0310742
qa: note timeout in debug message
Some tests set this to a dynamic value, it'd be helpful to know how long
a test is planning to wait.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-16 15:59:25 -08:00
Sage Weil
fc6840933b qa/tasks/rbd_fio: fio 2.21 -> 3.16
2.21 doesn't build on el8; 3.16 does.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-16 17:03:06 -06:00
Sage Weil
299375852a qa/tasks/cbt: install python3 deps
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-16 10:46:13 -06:00
Lenz Grimmer
1143de2ff8
mgr/dashboard: CephFS quota management (#31811)
mgr/dashboard: CephFS quota management

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-12-16 09:55:36 +00:00
Lenz Grimmer
efdcbcc6e1
mgr/dashboard: add password expiration date (#30939)
mgr/dashboard: add password expiration date

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-12-16 09:49:16 +00:00
Kefu Chai
0a83d9003e
Merge pull request #31815 from tchaikov/wip-mgr-dashboard-gevent
mgr/dashboard: install teuthology using pip

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
2019-12-16 17:30:08 +08:00
Sage Weil
d4f4a2cbd8 qa/tasks/ceph_manager.py: do not use python to write a file
/usr/bin/python dne on el8, /usr/bin/python3 dne on el7.  But
all we need to do is write a file--we can do that with tee.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-13 12:44:42 -06:00
Sage Weil
240154f114 qa: use python3 for venvs etc
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-13 08:47:28 -06:00
Stephan Müller
16bae1a3e3 mgr/dashboard: CephFS quota management
Now both CephFS quotas can be changed with a validation against the next
tree maximum in the file tree, that prevents setting the quotas in a way
that would not be usable.

Fixes: https://tracker.ceph.com/issues/38287
Signed-off-by: Stephan Müller <smueller@suse.com>
2019-12-13 15:15:34 +01:00
Thomas Bechtold
c150befff8 qa: Fix flake8 for qa/tasks/cephfs/test_volumes.py
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-13 09:24:25 +01:00
Thomas Bechtold
bdcc94a1d1 qa: Run flake8 on python2 and python3
To be able to catch problems with python2 *and* python3, run flake8
with both versions. From the flake8 homepage:

It is very important to install Flake8 on the correct version of
Python for your needs. If you want Flake8 to properly parse new
language features in Python 3.5 (for example), you need it to be
installed on 3.5 for Flake8 to understand those features. In many
ways, Flake8 is tied to the version of Python on which it runs.

Also fix the problems with python3 on the way.
Note: This requires now the six module for teuthology. But this is
already an install_require in teuthology itself.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-13 09:24:20 +01:00
Tatjana Dehler
0221241909 mgr/dashboard: add password expiration date
Add a 'pwd_expiration_date' field to the User object to be able
to set password expiration date per user.

There are two options to set a password expiration date for a
user:

1. Set the 'USER_PWD_EXPIRATION_SPAN' setting. If defined
   the expiration date of the password will be calculated
   automatically based on the given value. It will also be
   updated automatically when the user changes the password.
2. Set an expiration date by setting the 'pwdExpirationDate'
   field when creating a user.

Add two more settings in addition: USER_PWD_EXPIRATION_WARNING_1
and USER_PWD_EXPIRATION_WARNING_2. These two settings are
defining the amount of days to notify the user that his password
will expiration soon.

It's not possible to set the 'pwd_expiration_date' field to a
date in the past.

If the password of a user is already expired he is no longer
allowed to log into the dashboard.

Fixes: https://tracker.ceph.com/issues/40329
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-12-13 08:53:05 +01:00
Rishabh Dave
4e11b5b5e7 test_cephfs_shell: fix test_du_works_for_hardlinks
ln command needs superuser privileges, oddly that test had run
successfully before.

Fixes: https://tracker.ceph.com/issues/43250
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-13 12:15:27 +05:30
Kefu Chai
a9906b8cd2 vstart_runner: be python3 compatible
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-12 20:38:35 +08:00
Thomas Bechtold
0127cd1e88 qa: Enable flake8 tox and fix failures
There were a couple of problems found by flake8 in the qa/
directory (most of them fixed now). Enabling flake8 during the usual
check runs hopefully avoids adding new issues in the future.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-12 10:21:01 +01:00