Commit Graph

2402 Commits

Author SHA1 Message Date
Kefu Chai
fe6c9b5487 qa/tasks/ragweed: flake8 fixes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-23 23:01:27 +08:00
Kefu Chai
2c2b5e0c63 qa/tasks/mgr/test_orchestrator_cli: flake8 fixes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-23 23:01:27 +08:00
Kefu Chai
04290960f4 tasks/mgr/dashboard: flake8 fixes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-23 23:01:27 +08:00
Kefu Chai
2fa726b88c qa/tasks: flake8 fixes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-23 23:00:56 +08:00
Patrick Donnelly
0e1dc163db
Merge PR #35646 into master
* refs/pull/35646/head:
	qa/tasks/cephfs/nfs: Poll for max 60 seconds to ensure removal of ganesha services
	pybind/mgr/volumes/nfs: Remove unused cephfs module
	mgr/volumes/nfs: Ignore if ganesha disconnects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-06-23 06:44:08 -07:00
Xiubo Li
de62c5393b qa/tasks/cephfs/cephfs_test_case.py: skip cleaning the core dumps
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h

If the first character of this file is a pipe symbol (|), then the
remainder of the line is interpreted as the command-line for a
user-space program (or script) that is to be executed. More detail,
please see: http://man7.org/linux/man-pages/man5/core.5.html

Here will just skip cleaning the core dumps in this case.

Fixes: https://tracker.ceph.com/issues/45530
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-06-22 19:22:43 -04:00
Aashish Sharma
4a2fcc54ab mgr/dashboard/api: To increase test coverage in API controllers
Added test cases for the missing endpoints

Fixes: https://tracker.ceph.com/issues/45901
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2020-06-22 12:35:05 +05:30
Varsha Rao
0caa54bae8 qa/tasks/cephfs/nfs: Poll for max 60 seconds to ensure removal of ganesha services
Fixes: https://tracker.ceph.com/issues/46104
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-06-22 12:13:11 +05:30
Rishabh Dave
90727ee474 qa: add omit_sudo=False for commands ran with sudo
Commands that cleanup FUSE and kernel mount and that setup and
teardown/cleanup network namespaces are that ones that use sudo. Set
omit_sudo to False while running these commands.

Fixes: https://tracker.ceph.com/issues/46101
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-19 14:21:04 +05:30
Rishabh Dave
1ff990c2a1 vstart_runner.py: improve a conditional in _do_run
Before checking for "/" in args[0], check if it's an instance of Raw.
Since Raw instances aren't iterable, the execution would crash here.

Fixes: https://tracker.ceph.com/issues/46100
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-19 12:13:57 +05:30
Patrick Donnelly
62777f653d
qa: fix type error in stderr processing
Fixes: 422d736568
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-18 07:12:08 -07:00
Rishabh Dave
23f279d7a9 qa/cephfs: don't pass cmd args from run_as_user as str
Passing command arguments from run_as_user() to run_shell() as string
can be problematic since command argument to be passed to -c option of
sudo should be a single argument (i.e. 'ls dir' instead of
['ls', 'dir']).

Fixes: https://tracker.ceph.com/issues/46057
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-17 23:13:05 +05:30
Rishabh Dave
4c9811aae3 qa/cephfs: refactor run_as_root() to user run_as_user()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-17 23:12:59 +05:30
Patrick Donnelly
f1174475e3
Merge PR #32111 into master
* refs/pull/32111/head:
	test_cephfs_shell: run cephfs-shell with conf file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-06-17 09:42:21 -07:00
Patrick Donnelly
af98e6afbe
Merge PR #35539 into master
* refs/pull/35539/head:
	vstart_runner: set omit_sudo to True by default

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-17 09:38:36 -07:00
Rishabh Dave
58c7c7e8f4 test_cephfs_shell: run cephfs-shell with conf file
And set colors to False and debug to True in this conf file.

stdout captured by Python code in this testsuite can contain the colour
codes along with the expected string output. Setting colors to False in
cephfs-shell.conf should fix this.

Setting debug to False should help with analyzing test failures since
it enables printing messages that helps debugging (stacktrace for
cephfs-shell code, for example).

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-17 17:21:31 +05:30
Kefu Chai
48ee2b8441
Merge pull request #35395 from kshtsk/wip-registry-conf
qa/tasks/cephadm: setup site based container registry

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-06-17 19:49:20 +08:00
Rishabh Dave
cc8f15818a vstart_runner: set default values of stdout and stderr to None
Not doing so leads to tests run successfully with vstart_runner.py but
crash when triggered with teuthology since the default values of these
variables there is None.

Fixes: https://tracker.ceph.com/issues/45815
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-17 14:42:53 +05:30
Patrick Donnelly
0e74b92258
Merge PR #32288 into master
* refs/pull/32288/head:
	test_cephfs_shell: remove test_every_shell_cmd_at_invocation
	test_cephfs_shell: catch specific exception instead of catching all
	cephfs-shell: set return value before exiting
	test_cephfs_shell: re-arrange TestCephFSShell's methods
	cephfs-shell: re-arrange code under __main__
	test_cephfs_shell: add new test for shell cmd at invocation
	cephfs-shell: launch shell only if there are no args following

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-06-16 19:28:48 -07:00
Patrick Donnelly
4b54ab690b
Merge PR #34716 into master
* refs/pull/34716/head:
	mgr/volumes: Add snapshot info command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2020-06-12 14:10:14 -07:00
Patrick Donnelly
7952dc58b2
Merge PR #35261 into master
* refs/pull/35261/head:
	qa/vstart_runner.py: ignores when source and destination are same

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-06-12 14:08:13 -07:00
Neha Ojha
04553e8822
Merge pull request #35544 from neha-ojha/wip-reduce-noise
qa/tasks/ceph_manager.py: dump more useful info before failing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-06-11 15:40:46 -07:00
Neha Ojha
4deba4e8bd qa/tasks/ceph_manager.py: dump more useful info before failing
Add helpers that dump information only about PGs that haven't reached
the desired state when we fail. Previously we dumped the output of
"ceph pg dump" before failing, which prints a lot of unnecessary information
about PGs that are not responsible for the failure, making debugging harder.

Also, try to make the failure messages distinct.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-06-11 15:22:04 +00:00
Rishabh Dave
e68736961f vstart_runner: set omit_sudo to True by default
Let's set omit_sudo to True by default since we don't to run
commands with sudo on developer's machine as far as possible.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-11 17:05:40 +05:30
Kefu Chai
b010a137c7
Merge pull request #35521 from sebastian-philipp/dashboard-mgr-module-option-default-value-none-for-it-or-float
mgr/dashboard: fix None value for ints and floats

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-06-11 14:17:15 +08:00
Kefu Chai
4fb37f881f
Merge pull request #35486 from jdurgin/wip-radosbench-qa
qa/tasks/radosbench: use long form of option for compatibility

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2020-06-11 09:28:11 +08:00
Kyr Shatskyy
d8edde0c53 qa/tasks/cephadm: setup site based container registry
Add containers registry mirror for 'docker.io' to the config
file /etc/containers/registries.conf

Since we need site based config, so each teuthology instance
could have own containers' mirror, store corresponding data
in /etc/teuthology.yaml under 'overrides/cephadm' task specific
section.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-06-10 17:03:58 +02:00
Sebastian Wagner
f6f9b3fb99 mgr/dashboard: fix None value for ints and floats
Fixes: https://tracker.ceph.com/issues/45963

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-06-10 15:43:54 +02:00
Lenz Grimmer
11ddc9c8de
Merge pull request #35007 from rhcs-dashboard/fix-36354-master
mgr/dashboard: work with v1 RBD images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-10 13:32:37 +02:00
Josh Durgin
9673d37f1b qa/tasks/radosbench: use long form of option for compatibility
Since the short version of --object-size changed from -o to -O, it
does not work with upgrade tests.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-06-09 13:13:55 -07:00
Josh Durgin
a78781d82e Revert "qa/tasks/radosbench: only set object size if != block size"
This reverts commit fb991fa5f6.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Conflicts:
	qa/tasks/radosbench.py - extra context from -t, trivial resolution
2020-06-09 13:13:41 -07:00
Josh Durgin
5a8c7f7d7f Revert "qa/tasks/radosbench: fix usage of -O"
This reverts commit da19fd14db.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Conflicts:
	qa/tasks/radosbench.py - concurrency arg added in context, trival resolution
2020-06-09 13:12:34 -07:00
Rishabh Dave
99d944ce67 test_cephfs_shell: remove test_every_shell_cmd_at_invocation
The reason behind adding and removing this test in different commits of
the same PR is that test_every_shell_cmd_at_invocation is to keep it in
case it's needed later. It reruns all tests in this testsuite by issuing
all cephfs shell commands as arguments to cephfs-shell instead of
issuing them on stdin.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:41:17 +05:30
Rishabh Dave
a11b3c0177 test_cephfs_shell: catch specific exception instead of catching all
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:41:17 +05:30
Rishabh Dave
9b2077ff10 cephfs-shell: set return value before exiting
Code for each command has a return value set yet cephfs-shell itself
doesn't exit with proper return value. This makes cephfs-shell quit with
zero return value all the time.

Also, many tests execute cephfs-shell commands expecting the command to
fail but without expecting the method issuing the command to fail. Fix
these test accordingly by adding a new method (negtest_cephfs_shell_cmd)
that expects the command to fail and allows verifying return value and
error messages.

Fixes: https://tracker.ceph.com/issues/44113
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:41:16 +05:30
Rishabh Dave
c4bd10bdb9 test_cephfs_shell: re-arrange TestCephFSShell's methods
Re-arrange in the order of their usefulness and use keyword arguments
instead of positional arguments to reduce chances of errors, especially
since test_every_cephfs_shell_cmd_at_invocation overrides these methods.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:31:40 +05:30
Rishabh Dave
d0be1f9f07 test_cephfs_shell: add new test for shell cmd at invocation
Test passing each CephFS shell command at invocation of shell. The test
replaces run_cephfs_shell_cmd (and other related methods) by it's own
version of the same method that runs each shell command as
"cephfs-shell -c ceph.conf shellcmd" instead of
"cephfs-shell -c ceph.conf -- shellcmd" and then run each
test_cephfs_shell.py.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:28:03 +05:30
Patrick Donnelly
6be968228e
Merge PR #35263 into master
* refs/pull/35263/head:
	test_cephfs_shell: use StringIO instead of BytesIO

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-04 16:08:58 -07:00
Rishabh Dave
b67ce004ad test_cephfs_shell: use StringIO instead of BytesIO
Code in qa/ uses both StringIO and BytesIO. Let's use StringIO
exclusively (unless necessary) for uniformity. The reason for using
StringIO over BytesIO is that tests mostly need stdout as string than
as bytes and StringIO is used more frequently used in qa/ code at this
point.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 00:02:36 +05:30
Ernesto Puerta
119851ec89
mgr/dashboard: work with RBD images v1
Add support for RBD Image Format v1:
- This format lacks ID field, required for dashboard. Instead,
RBD image `block_name_prefix` is used as unique ID (together with pool
id and namespace)
- Additionally, `image_format` is now exposed.
- In the front-end side:
  - Copy action on a v1 image will cause the image to be copied to v2
format.
  - List doesn't allow Move to Trash on v1 images,
  - Details section now shows `image_format` for images,
  - Edit Form disables flags not supported for v1 (`deep-flatten`,
`layering`, `exclusive-lock`).
  - Protect does not work on v1 images or v2 images created from v1
ones.

Fixes: https://tracker.ceph.com/issues/36354
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2020-06-04 16:55:06 +02:00
Patrick Donnelly
4729688792
Merge PR #35202 into master
* refs/pull/35202/head:
	qa: return a string via getfattr

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-06-03 07:43:08 -07:00
Patrick Donnelly
f3d52ec619
Merge PR #35328 into master
* refs/pull/35328/head:
	qa/tasks/cephfs/test_scrub.py: use umount_wait to avoid ceph-fuse stuck

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-03 07:41:09 -07:00
Lenz Grimmer
b6dcd15ac4
Merge pull request #35051 from Devp00l/wip-44620
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-06-03 15:39:06 +02:00
Kefu Chai
bfb08d7b14
Merge pull request #35352 from tchaikov/wip-vstart-runner-create-cluster-only
qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-…

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-06-03 17:56:00 +08:00
Sebastian Wagner
115492f28f
Merge pull request #35069 from sebastian-philipp/qa-cephadm-add-rgw-and-iscsi
qa/cephadm: Add RGW

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-06-03 09:50:04 +02:00
Kefu Chai
954e9a6fa6 qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-only"
otherwise we could be removing a "None" directory when tearing down the cluster,
and have following failure:

Exception ignored in: <bound method LocalContext.__del__ of <__main__.LocalContext object at 0x7f99fd4a6cc0>>
Traceback (most recent call last):
  File "../qa/tasks/vstart_runner.py", line 1189, in __del__
    shutil.rmtree(self.teuthology_config['test_path'])
  File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 477, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/tmp/tmp.mmM2ugspuR/venv/lib/python3.6/shutil.py", line 475, in rmtree
    orig_st = os.lstat(path)
TypeError: lstat: path should be string, bytes or os.PathLike, not NoneType

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-03 15:24:16 +08:00
Patrick Donnelly
340d2a828f
Merge PR #34288 into master
* refs/pull/34288/head:
	mds: flag backtrace scrub failures for new files as okay

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-02 18:57:28 -07:00
Patrick Donnelly
cc36113ad4
Merge PR #34776 into master
* refs/pull/34776/head:
	mgr/volumes: Create subvolume with isolated rados namespace

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-02 17:44:21 -07:00
Patrick Donnelly
3d75b5a55f
Merge PR #34839 into master
* refs/pull/34839/head:
	qa/cephfs: add FUSE module before running mount -t fusectl

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-06-02 17:42:59 -07:00
Patrick Donnelly
0c6a92f5c0
Merge PR #35193 into master
* refs/pull/35193/head:
	qa/tasks/mds_thrash: s/random.sample/random.choice/

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-02 17:38:09 -07:00
Patrick Donnelly
182985455a
Merge PR #34838 into master
* refs/pull/34838/head:
	vstart_runner: don't use namespaces by default
	qa/cephfs: run nsenter commands with superuser privileges
	qa/cephfs: look for mountpoint in cmdline file

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-02 12:01:52 -07:00
Patrick Donnelly
963bb62ec9
Merge PR #34782 into master
* refs/pull/34782/head:
	qa/tasks/cephfs/mount.py: remove netns name parsing in mountpoint setter
	qa/tasks/vstart_runner.py: add kwargs parameter to ignore the ones it does not understand

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-02 07:40:39 -07:00
Rishabh Dave
49e9faee67 qa/vstart_runner.py: ignores when source and destination are same
put_file() in vstart_runner.py should ignore the error when source and
destination paths supplied are the same. Source and destination paths
for put_file() method represents different locations for teuthology
since source and destination are on the different machines.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-02 17:29:19 +05:30
Rishabh Dave
a4040bf0d2 vstart_runner: don't use namespaces by default
And add option --usens to vstart_runner.py to use namespaces.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-02 17:10:56 +05:30
Rishabh Dave
04d501d6ff qa/cephfs: run nsenter commands with superuser privileges
And add a method that sets self.fuse_daemon.subproc.pid to the PID of
the process that doesn't have sudo in its arguments. For example, when
"sudo ceph-fuse /mnt/cephfs" is run on the shell, it launches process
with arguments "ceph-fuse /mnt/cephfs". The added method gets PID of
latter/child process and sets that as the fuse daemon's PID. Not doing
so kills the former/parent process but the not the child process.

Also, since we are around cleanup this method a bit.

Fixes: https://tracker.ceph.com/issues/45339
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-02 17:07:49 +05:30
Sebastian Wagner
cca6533da2
Merge pull request #35313 from sebastian-philipp/qa-py2-filenotfound
qa/cephadm: FileNotFoundError doesn't exist in py2

Reviewed-by: Tim Serong <tserong@suse.com>
2020-06-02 12:58:25 +02:00
Stephan Müller
c76926bcd2 mgr/dashboard: Use right size in pool form
Currently the max size is determined by the number of OSDs, which is
compared with the maximum of the current crush rule.

The problem with that is, that this is wrong for every crush rule that
doesn't have OSDs as failure domain and that don't have the root of the
cluster set as root of the crush rule.

Now the crush map will be used to determine how many failure domains are
really available in the cluster and how many can really be used in the
end. This number now defines the maximum size you can enter.

The crush detail view will now the new attribute usable_size and hide
the redundant information steps, ruleset, type and rule_name.

Fixes: https://tracker.ceph.com/issues/44620
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-06-02 10:38:06 +02:00
Milind Changire
be650fe47d mds: flag backtrace scrub failures for new files as okay
New, unwritten files, fail when backtracing during scrub.
This is not necessarily bad. So flag such failures as okay and continue
with other entries.

Fixes: https://tracker.ceph.com/issues/43543
Signed-off-by: Milind Changire <mchangir@redhat.com>
2020-06-02 07:41:40 +05:30
Kefu Chai
b51139fce7 qa/tasks/tempest: use --black-regex for blacklisting tests
simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
60269764b4 qa/keystone: unpin python-openstackclient
this reverts 2dac5c5012,
as we don't use `--os-url` anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
8f5c832915 qa/tasks/tempest: use user/pass to authenticate
instead of using admin token use "admin" user to authenticate,
as admin token is not suggested anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
a53c69943c qa/tasks/keystone: use "keystone-manage bootstrap"
* qa/tasks/keystone.py:
  instead of prefilling keystone manually, use "keystone-manage bootstrap"
  instead. it helps to setup the admin user, a "Default" domain with
  "default" id, and wire them up with the expected role and a "admin" project,
  etc. as id of the admin domain is known to be "default", we can just use it
  in our tests without querying openstack for the id of "Default"
  domain. this is very handy.
* qa/suites/rgw/tempest/tasks/rgw_tempest.yaml:
  use "Default" for domain name. as "Default" is the name of the domain
  created by bootstrap, while "default" is its id.
* qa/suites/rgw/crypt/2-kms/barbican.yaml:
  remove settings to bootstrap keystone

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
8cb3f0411d qa/suites/rgw/tempest: use the latest tempest supporting py3.5
in case we need to use ubuntu xenial for testing, xenial only had python
3.5 packaged. and tempest 23.0 was the last version which supports
python3.5 and python2.7.

also do not replace link in tox.ini, as it is reachable.

to address the issues of

- pallets/markupsafe#116
- pypa/setuptools#2017

MarkupSafe is installed by
https://opendev.org/openstack/requirements/raw/branch/stable/pike/upper-constraints.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
9bd3e0ff40 qa/suites/rgw/tempest: bump up keystone to 17.0.0
* also generate a sample conf file following the document at
  https://github.com/openstack/keystone/tree/17.0.0.0rc2/etc
* use "projects" instead of "tenants" to match the terminology used by
  openstack identify API 3.0.
* test API 3.0 instead of API 2.0, by changing
  `rgw_keystone_api_version` from "2" to "3"
* explicitly specify a domain "default" for project to be created,
  otherwise a POST request will fail with:
```
{"error":{"code":400,"message":"You have tried to create a resource using the admin token. As this token is not within a domain you must explicitly include a domain for this resource to belong
to.","title":"Bad Request"}}
````
* create "default" domain, and use it, othewise a GET request fails
  like:
```
2020-05-28T11:17:28.751 INFO:teuthology.orchestra.run.smithi092.stderr:http://smithi092.front.sepia.ceph.com:35357 "GET /v3/domains/default HTTP/1.1" 404 87
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP: [404] Content-Length: 87 Content-Type: application/json Date: Thu, 28 May 2020 11:17:28 GMT Server: WSGIServer/0.2
CPython/3.6.9 Vary: X-Auth-Token x-openstack-request-id: req-bc33796f-2bc3-411c-a7fb-1208918e0dbd
2020-05-28T11:17:28.752 INFO:teuthology.orchestra.run.smithi092.stderr:RESP BODY: {"error":{"code":404,"message":"Could not find domain: default.","title":"Not Found"}}
```
* add user to "default" domain when creating it.
* use "type" as the positional argument, per
  https://docs.openstack.org/keystone/pike/admin/cli-keystone-manage-services.html
  otherwise we will have failures like:
```
2020-05-28T13:38:24.867 INFO:teuthology.orchestra.run.smithi198.stderr:openstack service create: error: unrecognized arguments: --type keystone
```
* update `create_endpoint()` to use the V3 API,
  see
  https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html

Fixes: https://tracker.ceph.com/issues/45692
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
a620587ecf qa/tasks/keystone.py: support multiple positional args
it's required when creating endpoint, see
https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/endpoint.html,
where we need to pass <service>, <interface>, and <url>

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
b7152e8e94 qa/tasks/tox: use the latest stable tox
in hope to avoid any ancient bugs

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Kefu Chai
874f8b79e5 qa/tasks/tox: use python3 for tox tests
both tempest and keystone have dropped python2 support in their tox
based tests.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-02 09:48:32 +08:00
Patrick Donnelly
9442abd152
Merge PR #34672 into master
* refs/pull/34672/head:
	qa/tasks/cephfs: Enable multiple exports tests
	mgr/nfs: Instead of 'auth del' use 'auth rm'
	qa/tasks/cephfs: Don't enable cephadm in TestNFS
	qa/tasks/cephfs: Add tests for nfs exports
	mgr/volumes/nfs: Fix idempotency of cluster and export commands
	mgr/volumes/nfs: Fix incorrect read only access_type value
	mgr/fs/nfs: Use check_mon_command() instead of mon_command()
	qa/cephfs: Add tests for nfs
	mgr/volumes/nfs: Remove type option from export create interface
	vstart: Instead of CACHEINODE use MDCACHE
	mgr/volumes: Rearrange nfs export interface
	mgr/volumes/nfs: Delete common config object on cluster deletion
	mgr/volumes/nfs: Delete all exports on cluster deletion
	mgr/volumes: Make nfs create export interface idempotent
	vstart: Add watch url for conf-nfs object
	mgr/volumes/nfs: Delete user on removing export
	mgr/volumes: Create user for given path and fs
	vstart: Ensure cephadm and NFS does not conflict
	vstart: Update details about ganesha packages
	mgr/volumes/nfs: Add delete cephfs export command
	mgr/volumes/nfs: Add RADOS notify for common config object
	mgr/volumes/nfs: Pass cluster_id directly to NFSCluster {create, update, delete} methods
	mgr/volumes: Add nfs cluster delete interface
	mgr/volumes: Add nfs cluster update interface
	vstart: Enable test_orchestrator in start_ganesha()
	mgr/volumes: Add placement option to create nfs cluster interface
	mgr/volumes/nfs: Change common ganesha config object name to 'conf-nfs.ganesha-<cluster_id>'
	mgr/volumes/nfs: Call orch nfs apply

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-01 11:23:03 -07:00
Rishabh Dave
ea1432810c qa/cephfs: look for mountpoint in cmdline file
Look for self.mountpoint in the contents of /proc/<pid>/cmdline file
when finding asok file for the client so that vstart_runner.py won't end
up picking asok file for a client not created in current run.

This usually never happens so far because PID of newly created processes
is higher than that of previously created processes and list of asok
files returned by "glob.glob(asok_path)" in find_socket() is in
descending order of PIDs.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-01 23:53:02 +05:30
Xiubo Li
a7423b2286 qa/tasks/cephfs/test_scrub.py: use umount_wait to avoid ceph-fuse stuck
If the ceph-fuse client need to flush the caps and does sync wait,
the umount() will just return successfully, then the netns container
will be destroyed and the network will not be reachable, but the
ceph-fuse daemon is still stucked and waiting for the flush caps ack.

This will cause the ceph-fuse daemon get stuck forever and if the
mds daemons get restarted, it will try to reconnect the clients,
but the stucked ceph-fuse daemnon won't reply to it, because it is
not reachable any more.

Fixes: https://tracker.ceph.com/issues/45665
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-06-01 09:24:02 -04:00
Varsha Rao
b2adff10b5 qa/tasks/cephfs: Enable multiple exports tests
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-29 23:36:34 +05:30
Sebastian Wagner
eddb6354e1 qa/cephadm: FileNotFoundError doesn't exist in py2
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-29 16:04:20 +02:00
Sebastian Wagner
ffef64d046 qa/cephadm: Add RGW
Fixes: https://tracker.ceph.com/issues/43911

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-29 13:33:05 +02:00
Varsha Rao
1477c98764 qa/tasks/cephfs: Don't enable cephadm in TestNFS
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-29 15:24:03 +05:30
Varsha Rao
af3b925dfd qa/tasks/cephfs: Add tests for nfs exports
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-29 14:56:34 +05:30
Varsha Rao
ab41951b04 qa/cephfs: Add tests for nfs
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-29 14:47:32 +05:30
Georgios Kyratsas
fabcbeab2c qa/tasks/cephadm: Add check fox already set image
Adding a check for already bootstrapped clusters where the image is
already set to avoid overriding it.

Signed-off-by: Georgios Kyratsas <gkyratsas@suse.com>
2020-05-28 17:15:59 +02:00
Xiubo Li
9bc1a74342 qa/tasks/cephfs/mount.py: remove netns name parsing in mountpoint setter
We can get the correct netns name always by using the netns_name
getter.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-05-27 21:34:02 -04:00
Xiubo Li
9f757c18a8 qa/tasks/vstart_runner.py: add kwargs parameter to ignore the ones it does not understand
INFO:__main__:test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) ... ERROR
INFO:__main__:Stopped test: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) in 22.251901s
INFO:__main__:
INFO:__main__:======================================================================
INFO:__main__:ERROR: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush)
INFO:__main__:----------------------------------------------------------------------
INFO:__main__:Traceback (most recent call last):
INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/cephfs_test_case.py", line 145, in setUp
INFO:__main__:    self.mounts[i].mount_wait()
INFO:__main__:  File "/data/ceph/qa/tasks/cephfs/mount.py", line 420, in mount_wait
INFO:__main__:    mount_options=mount_options)
INFO:__main__:TypeError: mount() got an unexpected keyword argument 'mountpoint'
INFO:__main__:
INFO:__main__:----------------------------------------------------------------------
INFO:__main__:Ran 1 test in 22.253s
INFO:__main__:
INFO:__main__:FAILED (errors=1)
INFO:__main__:

Fixes: https://tracker.ceph.com/issues/45300
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-05-27 21:18:53 -04:00
Sebastian Wagner
ab953a4151
Merge pull request #35273 from sebastian-philipp/qa-cephadm-mirror-ubuntu
qa/cephadm: Docker doesn't ship a registries.conf

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-27 18:01:17 +02:00
Patrick Donnelly
b63ae14927
Merge PR #35267 into master
* refs/pull/35267/head:
	vstart_runner: set mounted to True at the end of mount()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-05-27 08:32:59 -07:00
Kefu Chai
b01b58c45f
Merge pull request #35012 from tchaikov/wip-45147
qa/tasks/mgr: skip test_diskprediction_local on python>=3.8

Reviewed-by: Yuri Weinstein <yuriw@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-05-27 21:59:37 +08:00
Kefu Chai
8ffdde3188
Merge pull request #35198 from varshar16/wip-cephadm-module-error
qa/tasks/mgr: Don't disable cephadm module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-27 21:49:27 +08:00
Sebastian Wagner
9310b6e278 qa/cephadm: Docker doesn't ship a registries.conf
I guess not using the mirror for docker based tests is ok for now.

For adding docker support, we need to:

1. change the docker config
2. restart the docker daemon

Note that Docker's config only supports to mirror the docker.io registry.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-27 13:17:58 +02:00
Rishabh Dave
418dea32c2 vstart_runner: set mounted to True at the end of mount()
Not doing so prevents cleanup methods from running.

Fixes: https://tracker.ceph.com/issues/45723
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-27 14:45:59 +05:30
Lenz Grimmer
86d81d4204 Merge pull request #34863 from rhcs-dashboard/45310-add-grafana-rgw-sync-info
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-05-27 10:01:28 +02:00
Sebastian Wagner
d6edd8199f qa/cephadm: Add local registry mirror
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-26 17:38:39 +02:00
Kotresh HR
7396880d40 mgr/volumes: Create subvolume with isolated rados namespace
1. Add --namespace-isolated option to 'subvolume create' command
   to create subvolume in a separate RADOS namespace
2. Add "pool_namespace" field to 'subvolume info' command
   which displays the rados namespace if set else empty string

Fixes: https://tracker.ceph.com/issues/45289
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2020-05-26 12:00:53 +05:30
Kefu Chai
bb2c587435 qa/tasks/mds_thrash: s/random.sample/random.choice/
* use list comprehension instead of concatenating two ranges for
  better readablity -- we want to skip current max_mds for changing
  it. this helps reader to understand the goal of thrashing
* random.sample() is replaced with random.choice(). the latter is a
  better alternative, if the number of samples is 1.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-23 11:29:53 +08:00
Patrick Donnelly
481b575491
Merge PR #35090 into master
* refs/pull/35090/head:
	mgr/volumes: Fix subvolume create idempotency

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-05-22 14:23:57 -07:00
Patrick Donnelly
dbef451083
Merge PR #35063 into master
* refs/pull/35063/head:
	cephfs: allow pool names with '-' and '.'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-22 14:21:30 -07:00
Patrick Donnelly
3c1330795a
qa: return a string via getfattr
Fixes: https://tracker.ceph.com/issues/45666
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-22 14:15:19 -07:00
Patrick Donnelly
22517f8fd1
Merge PR #35109 into master
* refs/pull/35109/head:
	qa: use py3 compat list from range

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-22 10:34:19 -07:00
Varsha Rao
261c769333 qa/tasks/mgr: Don't disable cephadm module
Fixes: https://tracker.ceph.com/issues/45596
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-22 11:50:50 +00:00
Alfonso Martínez
cf4ff7d2f0 mgr/dashboard: grafana panels for rgw multisite sync performance
* RGW sync perf. counters are now exposed through grafana panels.
* Sync Performance tab is only shown if rgw realm is detected.
* Prometheus module: added metrics suitable for prometheus consumption (from existing ones, not replacing for backward compatibility).

Fixes: https://tracker.ceph.com/issues/45310
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-05-22 13:36:10 +02:00
Kotresh HR
db8706aa68 mgr/volumes: Fix subvolume create idempotency
After subvolume is created, it can be resized
using subvolume create command. But it was
broken and the same is fixed.

Fixes: https://tracker.ceph.com/issues/45398
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2020-05-22 16:28:35 +05:30
Rishabh Dave
2d7c32997b qa/cephfs: add FUSE module before running mount -t fusectl
Also, change timeout from 15 minutes to 30 seconds for the command
"mount -t fusectl xxx xxx" since 15 minutes is too much as per Zheng.

Fixes: https://tracker.ceph.com/issues/45304
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-22 15:26:37 +05:30
Kefu Chai
fdb2d1e62c
Merge pull request #35104 from varshar16/wip-fix-roleless-mon-issue
qa/cephadm: Add roleless mon names to ctx.cluster.remotes role dict

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Georgios Kyratsas <gkyratsas@suse.com>
2020-05-22 12:44:09 +08:00
Rishabh Dave
1abfdd161c cephfs-shell: support cephfs-shell.conf
cephfs-shell options should reside in cephfs-shell.conf and not in
ceph.conf. Please note, unlike before, -c option of cephfs-shell must
take path to cephfs-shell.conf instead of path to ceph.conf.

This commit also updates the docs and the tests for cephfs-shell accordingly
and renames the variable config_path in cephfs-shell to shell_conf_path so
that it's easy to distinguish.

Fixes: https://tracker.ceph.com/issues/44127
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-20 21:32:45 +05:30
Patrick Donnelly
3a26099d05
Merge PR #33264 into master
* refs/pull/33264/head:
	test_cephfs_shell: refactor test_put_and_get_without_target_directory

Reviewed-by: Varsha Rao <varao@redhat.com>
2020-05-20 08:27:34 -07:00
Kefu Chai
d67b0f1e3a qa/tasks/cephadm: ignore tarfile.ReadError
there is chance that /var/log/ceph (and other directories) is not
created when cephadm fails, but we always try to collect /var/log/ceph
by tar the directory on the test node and transfer it back for archiving
it. when we fail to do so, an exception is thrown, and this exception is
in turn logged as the cause of the failure instead of the one which
fails the test.

in this change, `tarfile.TarError`s is caught and ignored if it is
thrown by `misc.pull_directory()`, as normally, we don't consider it a fatal
failure when `misc.pull_directory()` fails because of `tar`. the
failures for the test are.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-19 19:10:56 +08:00
Varsha Rao
58eaeb53e4 qa/cephadm: Add roleless mon names to ctx.cluster.remotes role dict
Fixes: https://tracker.ceph.com/issues/45584
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-05-19 09:57:27 +00:00
Ramana Raja
d9f8f8e3d0 cephfs: allow pool names with '-' and '.'
... to be set as pool layouts for files and directories.

Fixes: https://tracker.ceph.com/issues/45521
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-05-19 13:34:49 +05:30
Patrick Donnelly
076d04654f
Merge PR #35062 into master
* refs/pull/35062/head:
	qa/tasks/cephfs/fuse_mount.py: retry when the admin socket is not ready

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-05-18 15:14:08 -07:00
Patrick Donnelly
5ea3a9a1f7
Merge PR #34951 into master
* refs/pull/34951/head:
	qa/cephfs: run() cleanup whether FS was mounted or not

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2020-05-18 14:57:20 -07:00
Patrick Donnelly
0825b9b97d
Merge PR #35058 into master
* refs/pull/35058/head:
	qa/tasks: logger.warn() -> logger.warning()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-18 14:34:28 -07:00
Patrick Donnelly
8d51b33e5d
qa: use py3 compat list from range
Fixes: https://tracker.ceph.com/issues/45590
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-18 12:57:43 -07:00
Sebastian Wagner
71c7f959f7
Merge pull request #34900 from gekios/wip-cephadm-bootstrapped
qa/tasks/cephadm.py: Support already bootstrapped clusters

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-18 20:46:47 +02:00
Patrick Donnelly
6f14713aed
qa: do not save/restore ip_forward sysctl
We don't really care if this is left on during the course of the mount.
These settings don't persist across tests anyway.

Fixes: https://tracker.ceph.com/issues/45525
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-14 20:15:38 -07:00
Xiubo Li
422d736568 qa/tasks/cephfs/fuse_mount.py: retry when the admin socket is not ready
Fixes: https://tracker.ceph.com/issues/45552
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-05-14 22:00:35 -04:00
Georgios Kyratsas
35e158605e qa/tasks/cephadm: Add support for raw disks
Currently tests on raw disks are failing due to stripping '/dev/' from
device name. (ie https://tracker.ceph.com/issues/45383#change-165721)

Signed-off-by: Georgios Kyratsas <gkyratsas@suse.com>
2020-05-14 16:54:57 +02:00
Georgios Kyratsas
a002b28bc1 qa/tasks/cephadm.py: Support bootstrapped clusters
Add the possibility of being able to run cephadm.py on clusters that
were already bootstrapped by some other task/way so you can deploy the
rest of the daemons/services.

Signed-off-by: Georgios Kyratsas <gkyratsas@suse.com>
2020-05-14 16:54:30 +02:00
Rishabh Dave
e57c86fab3 qa/cephfs: run() cleanup whether FS was mounted or not
In case the mount command in mount() fails, it would still have created
the mountpoint and network namespace for the FS's mount. Therefore, run
cleanup() and cleanup_netns() in umount() and umount_wait() even when
self.mounted is set to False.

Also, move the call to cleanup_netns() in cleanup().

Fixes: https://tracker.ceph.com/issues/45430
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-14 10:06:21 +05:30
Michael Fritch
44bcee8d1a
qa/tasks: logger.warn() -> logger.warning()
logger.warn() is the same as logger.warning(), but was removed in py3.3

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-05-13 17:18:13 -06:00
Kotresh HR
171930ef77 mgr/volumes: Add snapshot info command
The following command is added

"ceph fs subvolume snapshot info <vol_name> <sub_name> <snap_name> [<group_name>]"

The output is in json format with following fields

    created_at: time of creation of snapshot in the format "YYYY-MM-DD HH:MM:SS:ffffff"
    data_pool: data pool the snapshot belongs to
    has_pending_clones: "yes" if snapshot clone is in progress otherwise "no"
    protected: "yes" if snapshot is protected otherwise "no"
    size: snapshot size in bytes

Fixes: https://tracker.ceph.com/issues/45237
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2020-05-13 15:23:36 +05:30
Kefu Chai
b23de0c4b0
Merge pull request #34956 from tchaikov/wip-qa-tasks-cephfs-py3
qa/tasks/cephfs: remove sys.exc_clear() calls

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-13 15:21:41 +08:00
Kefu Chai
2941bbd7bc
Merge pull request #34857 from tchaikov/wip-qa-tasks-no-filter
qa/tasks: use list comprehension for checking the length

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-13 14:10:24 +08:00
Patrick Donnelly
01aea7f7e1
Merge PR #34978 into master
* refs/pull/34978/head:
	qa/tasks/cephfs/mount.py: always setup the NAT rules
	qa/tasks/cephfs/mount.py: fall back to use ip command to setup the bridge

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-05-12 11:59:51 -07:00
Kefu Chai
31704b3ecc qa/tasks/mgr: skip test_diskprediction_local on python>=3.8
See-also: https://tracker.ceph.com/issues/45147
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-12 11:54:23 +08:00
Xiubo Li
2e6e3916a0 qa/tasks/cephfs/mount.py: always setup the NAT rules
If the last test failed just after the 'ceph-brx' bridge is created
but with setuping the NAT rules in iptables, so if we run the test
case again, it will just skip seting the 'ceph-brx' and the NAT rules.
Then deleting the NAT rules will get the errors like:

iptables: Bad rule (does a matching rule exist in that chain?).

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-05-11 23:20:42 -04:00
Xiubo Li
f7b49f5bab qa/tasks/cephfs/mount.py: fall back to use ip command to setup the bridge
CentOS/RHEL8 have abandoned the bridge-utils package so the brctl
cmd is none avalible, and on Ubuntu 18.04 and 20.04 the nmcli is
buggy to setup the bridge. So this will fall back to use ip cmd
to setup the bridge stuff.

Fixes: https://tracker.ceph.com/issues/45459
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-05-11 22:51:29 -04:00
Patrick Donnelly
8a0caa435e
Merge PR #34962 into master
* refs/pull/34962/head:
	vstart_runner.py: use tuple instead of set

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-11 19:35:18 -07:00
Rishabh Dave
cb0bfc8fab vstart_runner.py: use tuple instead of set
Using python3 leads to an error that says
"TypeError: unhashable type: 'Raw'".

Fixes: https://tracker.ceph.com/issues/45446
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-11 18:01:58 +05:30
Kefu Chai
bcbb8abb96
Merge pull request #34834 from sidharthanup/qa-test-exports-fix
qa/test_exports: fix TestExports failure under new py3 compability changes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-09 14:53:10 +08:00
Kefu Chai
0d06671898
Merge pull request #34871 from tchaikov/wip-qa-tasks-mgr-set-application
qa/tasks/mgr: always add "application" label to pool

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-05-09 14:50:00 +08:00
Kefu Chai
9f2ac621e9
Merge pull request #34875 from tchaikov/wip-qa-tasks-qemu
qa/tasks/qemu: open file in text mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-05-09 14:41:04 +08:00
Patrick Donnelly
9859cd0f25
Merge PR #34941 into master
* refs/pull/34941/head:
	qa/cephfs: use StringIO instead of BytesIO

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2020-05-08 21:28:04 -07:00
Kefu Chai
702de9590c qa/tasks/cephfs: remove sys.exc_clear() calls
sys.exc_clear() was removed in Python3, see
https://docs.python.org/3/whatsnew/3.0.html#index-22. so we should not
call it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-08 11:21:22 +08:00
Rishabh Dave
71b823daee qa/cephfs: use StringIO instead of BytesIO
Fixes: https://tracker.ceph.com/issues/45425
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-05-07 12:13:53 +05:30
Volker Theile
2fa69eb688 qa: mgr/dashboard: Replace Telemetry module in REST API test
Fixes: https://tracker.ceph.com/issues/45109

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-05-05 09:35:24 +02:00
J. Eric Ivancich
89654d0477 qa/rgw: allow the rgw teuthology task to capture/set dns names
A teuthology workunit might want to use the rgw task, setting the
rgw-dns-name and/or rgw-dns-s3website-name configuration options to
the fully-qualified domain name. Existing code implies that setting
these configuration options to the empty string will do that. However
the current logic does not support that given it has Python
conditionals that treat the empty string as false. This fixes that.

Now the following teuthology tasks YAML will work as expected:

  tasks:
  - rgw:
      client.0:
        dns-name: ''
        dns-s3website-name: ''

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-04 13:55:00 -04:00
Lenz Grimmer
b7b46a9691
Merge pull request #34448 from Devp00l/wip-44621
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-05-04 10:51:18 +02:00
Kefu Chai
0b21c597d4 qa/tasks/qemu: open file in text mode
in python3, if a file is opened in 'rb' mode, we will have bytes when
reading from it, as a result, we have following failure like:
```
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks/qemu.py", line 124, in generate_iso
    test_setup = ''.join(f.readlines())
TypeError: sequence item 0: expected str instance, bytes found
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-01 16:55:45 +08:00
Kefu Chai
fcf222a589
Merge pull request #34657 from lxbsz/py3_com
qa/tasks/vstart_runner.py: to be python3 compatible

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-01 13:35:46 +08:00
Kefu Chai
a75d43efbf qa/tasks/mgr: always add "application" label to pool
otherwise monitor complains like
```
 health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': '1 pool(s) do not have an application enabled', 'count': 1}, 'detail': [{'message': "application not
enabled on pool 'dashboard_pool3'"}, {'message': "use 'ceph osd pool application enable <pool-name> <app-name>', where <app-name> is 'cephfs', 'rbd', 'rgw', or freeform for custom applications."}],
'muted': False, 'type': 'POOL_APP_NOT_ENABLED'}], 'mutes': []}
```

and if test checks for the healthy status, the test fails like:
```
2020-04-30T17:29:54.498 INFO:tasks.cephfs_test_runner:    self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health))
2020-04-30T17:29:54.498 INFO:tasks.cephfs_test_runner:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK'
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-01 12:20:15 +08:00
Kefu Chai
726c59be58 qa/tasks: use list comprehension for checking the length
instead of using filter(), use `sum()` for counting its
length, as in Python3, `filter()` actually returns a `filter` object
instead of a list.

in this change, `filter()` calls are replaced with `sum()`
for Python3 compatibility.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-04-30 23:18:03 +08:00
Sidharth Anupkrishnan
b98f3e388b qa/test_exports: fix TestExports failure under new python3 compability changes
self.mount_a.client_remote.sh() returns an 'str' object rather than a StringIO object. Hence the p.stdout.getvalue() produces an error. This commit fixes this and also fix str and byte mismatch as byte and string were the same object in Python2 but this is not the case in Python3.

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-04-30 15:49:32 +05:30
Lenz Grimmer
dc9973a7fb
Merge pull request #33462 from s0nea/wip-dashboard-telemetry-report
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-29 17:22:50 +02:00
Tatjana Dehler
18875ef337 mgr/dashboard: add telemetry report component
Add a telemetry component in order to give the user the
possibility to configure the telemetry module in a more
guided fashion. The component offers broader explanations,
shows a preview of the generated report and asks the user
to accept the license before enabling the module.

Fixes: https://tracker.ceph.com/issues/43956
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-04-29 16:34:00 +02:00
Ilya Dryomov
4e3160a019 qa/tasks/rbd_fio: update NamedTemporaryFile usage for python3
Open the file in text mode to avoid 'bytes' objects.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-04-28 23:38:05 +02:00
Ilya Dryomov
aba9910388 qa/tasks/rbd: update NamedTemporaryFile usage for python3
- open the file in text mode to avoid 'bytes' objects

- 'bufsize' is now spelled 'buffering' and switching buffering off
  with 'buffering=0' is only allowed in binary mode.  As we need the
  file to be either in the page cache or on disk before we copy it
  to the remote, call flush().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-04-28 23:22:17 +02:00
Stephan Müller
94a32d1a12 mgr/dashboard: Use of crush node class in ECP modal
Now the crush node class preselects root, failure domain and device
class in the erasure code profile modal.

Like for crush rule, now also if you try to delete an used ECP you can't
and the info box will show you, what pool is using the profile.

Fixes: https://tracker.ceph.com/issues/44621
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-04-28 17:45:28 +02:00
Lenz Grimmer
56a2a9ab9f
Merge pull request #34656 from rhcs-dashboard/44935-custom-cherryPy-header
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen <nia@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-28 10:52:53 +02:00
Xiubo Li
028e214c97 qa/tasks/vstart_runner: fix TypeError of string argument expected, got 'bytes'
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-28 01:30:25 -04:00
Gregory Farnum
0a9612bcde
Merge pull request #34595 from rishabh-d-dave/fs-move-run_shell
qa/cephfs: move run_shell to mount.py and add methods for negative testing

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-04-27 16:35:20 -07:00
Rishabh Dave
39db91ea19 test_cephfs_shell: refactor test_put_and_get_without_target_directory
The refactor intends to simplify the test and make it more readable,

It also changes how this test creates directory -- tempfile.mkdtemp is
used instead of coreutils mkdir to avoid bug reported here -
https://tracker.ceph.com/issues/43567.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-04-27 14:28:03 +05:30
Kefu Chai
d8084806fb
Merge pull request #34752 from tchaikov/fix-test-perf-counters-mgr-get
mgr/dashboard: fix API tests to be py3 compatible

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2020-04-27 12:00:40 +08:00
Kefu Chai
28c7ba68ae
Merge pull request #34732 from lxbsz/fs_fix_new
qa: misc fixes to run for cephfs qa test

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-26 01:46:30 +08:00
Kefu Chai
d07960ef7a
Merge pull request #34578 from lxbsz/volume_client_py3
qa/volume_client: be python3 compatible

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-25 18:58:43 +08:00
Xiubo Li
dd4fc808a8 qa/tasks/cephfs: switch to StringIO to fix byte-like errors
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-04-25 06:21:55 -04:00