Commit Graph

119 Commits

Author SHA1 Message Date
Patrick Donnelly
8a0caa435e
Merge PR #34962 into master
* refs/pull/34962/head:
	vstart_runner.py: use tuple instead of set

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-11 19:35:18 -07:00
Rishabh Dave
cb0bfc8fab vstart_runner.py: use tuple instead of set
Using python3 leads to an error that says
"TypeError: unhashable type: 'Raw'".

Fixes: https://tracker.ceph.com/issues/45446
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-11 18:01:58 +05:30
Xiubo Li
028e214c97 qa/tasks/vstart_runner: fix TypeError of string argument expected, got 'bytes'
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-28 01:30:25 -04:00
Gregory Farnum
0a9612bcde
Merge pull request #34595 from rishabh-d-dave/fs-move-run_shell
qa/cephfs: move run_shell to mount.py and add methods for negative testing

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-04-27 16:35:20 -07:00
Xiubo Li
dd4fc808a8 qa/tasks/cephfs: switch to StringIO to fix byte-like errors
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:55 -04:00
Xiubo Li
bf10478294 qa/tasks/vstart_runner: remove the stdin str type check
For the "ceph-dencoder" it will use the byte type data.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:54 -04:00
Xiubo Li
624944503e qa/tasks/vstart_runner: make remote.sh return str as default
To address the issue in https://github.com/ceph/teuthology/pull/1459.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-24 03:05:34 -04:00
Xiubo Li
704ce2c969 qa/vstart_runner.py: add os and arch properties support
Fixes: https://tracker.ceph.com/issues/44456
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-24 03:05:34 -04:00
Xiubo Li
c541bae6b1 qa/vstart_runner: remove the debug info
Fixes: https://tracker.ceph.com/issues/45175
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-23 20:55:35 -04:00
Gregory Farnum
6fae077e9e
Merge pull request #33576 from lxbsz/fuse
unshare-ns-mount: add isolating netns for mount helper support

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-04-21 22:12:37 -07:00
Kefu Chai
bedea85019 qa/tasks/vstart_runner.py: be python3 compatible
differentiate `str` and `bytes` instances, and drop python2 support from
vstart_runner.py, as we've moved to python3 already

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-20 19:41:09 +08:00
Gregory Farnum
2e84275947
Merge pull request #32612 from rishabh-d-dave/fs-qa-UnicodeDecodeError
test_cephfs_shell: fix UnicodeDecodeError

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-04-17 08:21:42 -07:00
Gregory Farnum
b274265acd
Merge pull request #33711 from lxbsz/qa_umount_failed
qa/tasks/cephfs: umount the mountpoints when tearDown

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-04-17 08:20:30 -07:00
Rishabh Dave
7b15a8437e qa/cephfs: move run_shell and related methods to mount.py
LocalFuseMount and LocalKernelMount can directly inherit these methods
from CephFSMount via FuseMount and KernelMount respectively. Moving
would avoid duplication and would make these methods more accessible
for reusing via inheritance.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-04-17 00:27:41 +05:30
Xiubo Li
f0c67256b9 qa/vstart_runner: add unsharing network namespace support
This will isolate the network namespace for each mount point with
a private ip address and iptables, etc.

For the kill() stuff it will just do DOWN the veth interface instead
of sending ipmi request for kernel mount and kill the fuse processes
for the fuse mount. This could avoid sending the socket FIN to the
ceph cluster.

Fixes: https://tracker.ceph.com/issues/44044
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-14 07:47:03 -04:00
Kefu Chai
f1375de3eb
Merge pull request #33945 from rishabh-d-dave/fs-qa-vstart_runner.LocalRemote.sh
qa/vstart_runner: update vstart_runner.LocalRemote.sh

Reviewed-by: "Yan, Zheng" <zyan@redhat.com>
2020-04-08 09:53:32 +08:00
Greg Farnum
81ce2b240a Revert "qa/cephfs: move run_shell and related methods to mount.py"
This reverts commit 751d432a8696c7fdceb9d9707c2b3331878342ea.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 02:24:48 +00:00
Rishabh Dave
43a0b64596 qa/vstart_runner: filter out non-ASCII chars
...from stdout and stderr of a command issued using LocalRemoteProcess.

Fixes: https://tracker.ceph.com/issues/43567
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-03-31 13:17:16 +05:30
Rishabh Dave
2cc0ee709c qa/vstart_runner: update vstart_runner.LocalRemote.sh
Commit 9f6c764f10 replaces remote.run calls by remote.sh without
updating the definition of vstart_runner.LocalRemote.sh which breaks the
cephfs tests when executed locally.

Fixes: https://tracker.ceph.com/issues/44579
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-03-30 11:46:12 +05:30
Gregory Farnum
60504bd048
Merge pull request #33279 from rishabh-d-dave/fs-move-run_shell-and-related-methods
qa/cephfs: move run_shell to mount.py and add methods for negative testing
2020-03-27 12:59:43 -07:00
Gregory Farnum
ca24037277
Merge pull request #33740 from lxbsz/qa_test_timeout_failed
qa/test_config_session_timeout: fix options conflicting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-03-27 12:55:20 -07:00
Kefu Chai
1032494ad2
Merge pull request #33873 from lxbsz/qa_vs_failed
qa/tasks/vstart_runner: fix sh to make it compatible with teth

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-26 23:26:53 +08:00
Yan, Zheng
4b79a2ecce
Merge pull request #29443 from simon-rock/metatool_dev
tool: add cephfs-meta-injection to show and amend info of inode
2020-03-26 17:10:28 +08:00
Kefu Chai
947a74349d qa: import with full path
to be py3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-24 18:27:55 +08:00
simon gao
53ba9089d8 qa : add case to test cephfs-meta-injection
Signed-off-by: simon gao <simon29rock@gmail.com>
2020-03-24 16:39:02 +08:00
Rishabh Dave
751d432a86 qa/cephfs: move run_shell and related methods to mount.py
LocalFuseMout and LocalKernelMount can directly inherit these methods
from CephFSMount via FuseMount and KernelMount respectively. Moving
would avoid duplication and would make these methods more accessible
for reusing via inheritance.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-03-20 02:11:54 +05:30
Xiubo Li
4d95517df3 qa/tasks/vstart_runner: fix sh to make it compatible with teth
Fixes: https://tracker.ceph.com/issues/44555
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-03-11 04:59:26 -04:00
Xiubo Li
95820bf48f qa/tasks/vstart_runner: use parent's umount methods
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-03-11 01:52:18 -04:00
Alfonso Martínez
3b9f99c549 vstart_runner.py: fix OSError when checking if non-existent path is mounted
Fixes: https://tracker.ceph.com/issues/44545
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-03-10 15:06:35 +01:00
Xiubo Li
1bcd319496 qa/vstart_runner: fix mutiple 'timeout' option conflicting
The 'timeout' option in the environment may will conflict with the
ones in some ceph commands, like:

$ timeout 120 ./bin/ceph daemon mds.b session config 8718 timeout 45

And the old code will also give us incorrect result like:

['adjust-ulimits', 'ceph-coverage', 'timeout', '120', 'ceph', 'fs', 'dump']
will be transfered to:
['adjust-ulimits', 'ceph-coverage', '120', 'ceph', 'fs', 'dump']

The '120' is left behind.

Fixes: https://tracker.ceph.com/issues/44437
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-03-06 18:29:02 -05:00
Thomas Bechtold
f5e77561e9 qa: Fix problems detected by mypy
This is a first step to enable mypy on the qa/ directory.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-03-05 06:53:31 +01:00
Kyr Shatskyy
49592af400 qa/tasks/vstart_runner: use io.BytesIO for py3 compat
Use io.BytesIO instead of StringIO for py3 compatibility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-04 13:09:17 +08:00
Patrick Donnelly
61681cffd2
Merge PR #33506 into master
* refs/pull/33506/head:
	client: add client_fs mount option support

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-03-02 16:58:02 -08:00
Xiubo Li
1a58e64299 client: add client_fs mount option support
"client_fs" is one alias for "client_mds_namespace=" and it will be
cleaner and be more user-friendly to use. "client_mds_namespace="
will be kept and backwards compatibility used.

Update the documents at the same time.

Fixes: https://tracker.ceph.com/issues/44212
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-02-27 19:30:31 -05:00
Patrick Donnelly
a7d273cef7
Merge PR #31200 into master
* refs/pull/31200/head:
	qa/cephfs: test case for auto reconnect after blacklisted

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-26 06:26:13 -08:00
Patrick Donnelly
e5841cefb4
Merge PR #33263 into master
* refs/pull/33263/head:
	qa/vstart_runner.py: make run()'s interface same as teuthology's run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-25 14:45:49 -08:00
Rishabh Dave
061d972278 qa/vstart_runner.py: make run()'s interface same as teuthology's run
Arguments accepted by method run in teuthology should be exactly same as
that of arguments accepted by vstart_runner.py's method run to prevent
test failure with teuthology due to silly argument mismatch.
Teuthology's entry method run expects **kwargs, therefore so should
vstart_runner.py's run.

Fixes: https://tracker.ceph.com/issues/44117
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-02-20 16:01:39 +05:30
Patrick Donnelly
2c9642131f
qa: add tests for mds_join_fs cluster affinity
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-13 07:51:10 -08:00
Patrick Donnelly
8729281121
qa: manage config changes through mons
This provides a generic framework for modifying Ceph configuration
changes in tests through the monitors rather than the asok interface or
local ceph.conf changes. Any changes are reverted during test teardown.

A future patch will convert existing tests manipulating the local
ceph.conf or admin socket.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-13 07:51:09 -08:00
Yan, Zheng
261a456439 qa/cephfs: test case for auto reconnect after blacklisted
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/42085
2020-01-15 15:43:17 +08:00
Rishabh Dave
b7d740ede7 vstart_runnner: add sh method to LocalRemote
The method is a wrapper of teuthology.misc.sh

Fixes: https://tracker.ceph.com/issues/43496
Signed-off-by: Rishabh Dave <ridave@gmail.com>
2020-01-13 10:22:56 +05:30
Kefu Chai
5fe27c2267 qa/tasks/vstart_runner: write string to StringIO
we use `six.StringIO` for stdout when writing output messages from
`self.subproc.communicate()`, and `six.StringIO` accepts strings, so we
need to decode the output before sending them to `six.StringIO`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-28 01:11:54 +08:00
Kefu Chai
4c6a5798c2
Merge pull request #32222 from toabctl/qa-flake8-py3
qa: Run flake8 on python2 and python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-24 10:47:07 +08:00
Kefu Chai
130e3eddb9 qa/tasks/vstart_runner.py: use current python executable
python3 is not always the path to executable used by current python
session.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-20 21:39:49 +08:00
Kefu Chai
79584862f6 src,qa: install python bindings into lib/cython_modules/lib.3
instead of checking for the WITH_PYTHON* options, just hardwire to lib.3

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:54 +08:00
Sage Weil
3268ec7ac8 Merge PR #32252 into master
* refs/pull/32252/head:
	qa/cephfs/begin: libaio-devel on el8
	qa/tasks: nosetests -> python -m nose
	qa/tasks/rbd_fio: fio 2.21 -> 3.16
	src/test/cli-integration/rbd/snap-diff.t: python -> python
	qa/workunits: use nose 3
	qa/tasks/cbt: install python3 deps
	qa/tasks/ceph_manager.py: do not use python to write a file
	test/pybind/test_rados: execute takes a bytes (not str) payload
	qa/packages/packages: python[3]-ceph is no more
	qa: use python3 for venvs etc
	packaging: remove python3-ipaddres, as it is part of the stdlib in py3
	qa/packages: python-ceph -> python3-ceph
	qa/distros: centos7 -> centos8, rhel7 -> rhel8
	spec: remove _python_buildid in favor of python3_pkgversion macro
	spec: remove python2 packages and conditions
	debian: remove python >= 2.7 requirement
	debian: add mgr python versions
	debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
	debian: update control file to use python3 dependency names
	debian: remove all python2 overrides and declarations
	debian: remove all python2 install files

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-12-17 15:23:27 -06:00
Kefu Chai
0a83d9003e
Merge pull request #31815 from tchaikov/wip-mgr-dashboard-gevent
mgr/dashboard: install teuthology using pip

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
2019-12-16 17:30:08 +08:00
Sage Weil
240154f114 qa: use python3 for venvs etc
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-13 08:47:28 -06:00
Thomas Bechtold
bdcc94a1d1 qa: Run flake8 on python2 and python3
To be able to catch problems with python2 *and* python3, run flake8
with both versions. From the flake8 homepage:

It is very important to install Flake8 on the correct version of
Python for your needs. If you want Flake8 to properly parse new
language features in Python 3.5 (for example), you need it to be
installed on 3.5 for Flake8 to understand those features. In many
ways, Flake8 is tied to the version of Python on which it runs.

Also fix the problems with python3 on the way.
Note: This requires now the six module for teuthology. But this is
already an install_require in teuthology itself.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-13 09:24:20 +01:00
Kefu Chai
a9906b8cd2 vstart_runner: be python3 compatible
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-12 20:38:35 +08:00