Commit Graph

2479 Commits

Author SHA1 Message Date
Rishabh Dave
9a36a0abd0 qa/cephfs: allow not aborting execution when mount command fails
This commit adds a new argument check_status to mount methods of
KernelMount, FuseMount, LocalKernelMount and LocalFuseMount. When value
of this argument is False, these methods would catch the
CommandFailedError exception and would return a tuple consisting of the
exception itself, and stdout and stderr of the mount command. This
allows reusing these mount methods while running negative tests for
commands.

The name "check_status" is selected so since teuthology's run() and
vstart_runner's run() use a variable with same name for the very same
purpose.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
07e493ffb5 qa/cephfs: allow reusing mount objects and add remount method
This commit introduces following two set of changes -

First, make client keyring path, mountpoint on host FS and CephFS and
CephFS's name attributes of the object representing the mount
and update all the mount object creation calls accordingly. Also,
rewrite all the mount object creation to use keyword arguments instead
of positional arguments to avoid mistakes, especially since a new
argument was added in this commit.

Second, add remount method to mount.py so that it's possible to unmount
safely, modify the attributes of the object representing the mount and
mount again based on new state of the object *in a single call*. The
method is placed in mount.py to avoid duplication.

This change has two leads to two more changes: upgrading interface of
mount() and mount_wait() and upgrading testsuites to adapt to these
change.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Kefu Chai
e5b9b08cc4
Merge pull request #36962 from tchaikov/wip-qa-py3-cleanup
qa: py3 cleanups

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-10 09:39:20 +08:00
Patrick Donnelly
465eba6d14
Merge PR #36546 into master
* refs/pull/36546/head:
	vstart_runner: log commands in a more usable form

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-09-03 10:08:42 -07:00
Sebastian Wagner
71da1d457e
Merge pull request #36894 from pcuzner/device-ls-with-lsm
orchestrator: intergrate LSM information with device ls
2020-09-03 14:45:06 +02:00
Kefu Chai
eda90040ad qa: always use subprocess.{DEVNULL,check_output}
no need to check for their existence, and prepare a replacement.
because we've migrated to python3. and we only support python3.6 and up.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-03 13:09:16 +08:00
Paul Cuzner
8691f94104 mgr/orchestrator: add test command testcase for "device ls"
Quick test to ensure changes that break wide mode are picked
up

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-09-03 14:53:22 +12:00
Rishabh Dave
464ec3dbdc vstart_runner: log commands in a more usable form
Currently commands are printed as ['arg1', 'arg2', 'arg3']. Instead, log
them as '> arg1 arg2 arg3' so that it's simpler to copy and run them
manually. The reason behind prepending '> ' to these logs entries is
just to follow the practice followed by teuthology logs.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-02 22:52:27 +05:30
Lenz Grimmer
4e71a942ce
Merge pull request #35802 from rhcs-dashboard/wip-46160-fix_cephfs_api_inconsistency
mgr/dashboard: Address API inconsistencies in Cephfs endpoint

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-02 12:29:59 +02:00
Lenz Grimmer
9e76d73a13
Merge pull request #34644 from rhcs-dashboard/26954-histogram
mgr/dashboard: Remove Histogram Component

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-02 08:28:49 +02:00
Fabrizio D'Angelo
871df3671c mgr/dashboard: Address API inconsistencies
Cephfs endpoints are inconsistencies with other endpoints.

* Change rm endpoints from GET to DELETE requests
* Change set_quotas from POST to PUT
* Renames endpoints accordingly
* Changes related angular files
* update a few lines related to mypy
* remove future import as the python2 compatibility is not needed

Fixes: https://tracker.ceph.com/issues/46160
Signed-off-by: Fabrizio D'Angelo <fdangelo@redhat.com>
(cherry picked from commit 222286b08bde3600d50a6deb25e7425c17e9e482)
2020-08-31 23:45:49 +05:30
Patrick Donnelly
d33c6157a3
qa: test for statfs on deleted file system
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-29 10:22:12 -07:00
Patrick Donnelly
72d6f0413e
qa: move run_shell args to kwargs
To avoid listing all run args.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-29 10:22:12 -07:00
Kefu Chai
786b87cdb2
Merge pull request #36718 from ShraddhaAg/add-dir-to-archive-in-logs
qa/tasks/ceph.py: add ceph logs directory in job's info.yaml

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-08-27 22:26:01 +08:00
Laura Paduano
836e1d4202
Merge pull request #36736 from ceph/47052-fix-base-uri-api-tests
mgr/dashboard: fix API tests: Invalid URL: None/api/auth

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-08-27 14:27:08 +02:00
anurag
ff7c7ae40d mgr/dashboard: Remove histogram component
This commit is intended for code cleanup of Histogram component.

Fixes: https://tracker.ceph.com/issues/26954
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2020-08-27 14:10:42 +05:30
Patrick Donnelly
d2a3fcc728
Merge PR #36773 into master
* refs/pull/36773/head:
	mgr/volumes: Prevent subvolume recreate if trash is not-empty
	mgr/volumes: Disallow subvolume group level snapshots
	mgr/volumes: Add test case to ensure subvolume is marked
	mgr/volumes: handle idempotent subvolume marks
	mgr/volumes: Tests amended and added to ensure subvolume trash functionality
	mgr/volumes: Mark subvolume root with the vxattr ceph.dir.subvolume
	mgr/volumes: Move incarnations for v2 subvolumes, to subvolume trash
	mgr/volumes: maintain per subvolume trash directory
	mgr/volumes: make subvolume_v2::_is_retained() object property

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-26 10:46:07 -07:00
Shyamsundar Ranganathan
00e4ab054f mgr/volumes: Prevent subvolume recreate if trash is not-empty
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:33 -04:00
Shyamsundar Ranganathan
f97e57c28c mgr/volumes: Disallow subvolume group level snapshots
Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:26 -04:00
Shyamsundar Ranganathan
3527711a4a mgr/volumes: Add test case to ensure subvolume is marked
with the vxattr ceph.dir.subvolume set to true.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:21 -04:00
Shyamsundar Ranganathan
88cffe6ea9 mgr/volumes: Tests amended and added to ensure subvolume trash functionality
Amended a few test cases to ensure created subvolumes and snaps
are removed, and trash stays empty at the end of the test.

Further added one test case for create errors in a retained
v2 subvolume, to ensure metadata is sane, and created incarnation
is not present.

Fixes: https://tracker.ceph.com/issues/47154
Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2020-08-26 13:41:08 -04:00
Neha Ojha
fb44c2f500
Merge pull request #36527 from neha-ojha/wip-whitelist-blacklist
Replace 'whitelist' and 'blacklist' language

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2020-08-25 12:24:36 -07:00
Sebastian Wagner
f82058f06b
Merge pull request #36738 from ShraddhaAg/cephadm-archive-logs
qa/tasks/cephadm.py: add ceph logs directory in job's info.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-25 15:18:58 +02:00
Patrick Donnelly
d83dc481dc
Merge PR #36684 into master
* refs/pull/36684/head:
	qa/tasks/nfs: Test mounting of export created with nfs command
	qa/tasks/nfs: Add helper method to check nfs cluster status
	qa/tasks/nfs: Cleanup created filesystem
	qa/tasks/nfs: Remove unused port status function and 'stdin' keyword argument

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 13:18:13 -07:00
Patrick Donnelly
c4f5de0058 *: convert EBLACKLISTED to EBLOCKLISTED
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 19:53:08 +00:00
Sage Weil
2ee9365d0b qa: log-whitelist -> log-ignorelist
Signed-off-by: Sage Weil <sage@newdream.net>
2020-08-24 19:53:08 +00:00
Sage Weil
dfd01d7653 blacklist -> blocklist
Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-08-24 19:53:08 +00:00
Patrick Donnelly
02813988a6
Merge PR #36394 into master
* refs/pull/36394/head:
	qa: add method run ceph cluster command with better interface
	cephfs: allow only "good" characters

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-24 11:33:03 -07:00
Kefu Chai
103fd4af21
Merge pull request #36760 from tchaikov/wip-qa-optional-args
qa/tasks/workunit: allow passing optional args to workunit

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-24 20:03:04 +08:00
Varsha Rao
f56a83f806 qa/tasks/nfs: Test mounting of export created with nfs command
Fixes: https://tracker.ceph.com/issues/46989
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-08-24 17:04:45 +05:30
Varsha Rao
5d6742b6b4 qa/tasks/nfs: Add helper method to check nfs cluster status
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-08-24 17:04:40 +05:30
Varsha Rao
e94a250cb0 qa/tasks/nfs: Cleanup created filesystem
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-08-24 17:00:42 +05:30
Varsha Rao
91e89a4df5 qa/tasks/nfs: Remove unused port status function and 'stdin' keyword argument
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-08-24 17:00:42 +05:30
Kefu Chai
12212a40db
Merge pull request #36751 from tchaikov/wip-redirect-stderr
qa/tasks/ceph: create a log file before redirecting to it

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-22 16:59:14 +08:00
Kefu Chai
56c86bbb23 qa/tasks/workunit: allow passing optional args to workunit
* add comment to _run_tests()
* use `os.path.commonpath()` instead using string matching directly
  for matching given workunit spec with executables.
* allow passing optional args to workunit

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-22 15:26:54 +08:00
Kefu Chai
e6eabeeeb2 qa/tasks/ceph: create a log file before redirecting to it
as it is shell who interprets ">>" and redirect the stderr to given
file, but the shell process is launched by ubuntu:ububunt without using
sudo, so the command fails with "Permission denied" failure. to address
this issue, in this change, a file with proper priviledges is created
beforehand using `install`, so shell is able to write to it.

also, instead of creating this file in `maybe_redirect_stderr()`, it
returns the command to create the log file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-22 11:33:12 +08:00
Patrick Donnelly
3fb857cd04
Merge PR #36131 into master
* refs/pull/36131/head:
	doc: document cephfs mirroring dev work
	test: add tests for `ceph fs mirror` family of commands
	mds: track filesystem mirror peers in fsmap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-21 18:40:42 -07:00
Rishabh Dave
530982129e qa: add method run ceph cluster command with better interface
This new method should allow better control on the process launched by
the passed command. This is achieved by allowing arguments provided by
teuthology.orchestra.run.run().

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-08-21 22:16:21 +05:30
Alfonso Martínez
28ecbbde65 mgr/dashboard: fix API tests: Invalid URL: None/api/auth
Fixes: https://tracker.ceph.com/issues/47052
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-08-21 17:22:21 +02:00
Sebastian Wagner
fc9da48226
Merge pull request #36740 from varshar16/wip-fix-nfs-bug-47009
qa/tasks/nfs: Change the mds name and delete created nfs clusters

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.
2020-08-21 09:58:12 +02:00
Kefu Chai
ec73fea99e
Merge pull request #36680 from tchaikov/wip-mgr-disk-predict
pybind/mgr/diskprediction_cloud: remove mgr-diskprediction-cloud

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-08-21 09:46:54 +08:00
Varsha Rao
5c4e659067 qa/tasks/nfs: Instead of 'orch ls' use 'orch ps'
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-08-20 15:18:31 +00:00
Varsha Rao
049afe5a7e qa/tasks/nfs: Delete nfs cluster after exports are created
Fixes: https://tracker.ceph.com/issues/47009
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-08-20 14:27:25 +00:00
Ramana Raja
f84d22832a
Merge pull request #35756 from ShyamsundarR/tr-46163
mgr/volumes: Use snapshot root directory attrs when creating clone root

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2020-08-20 16:53:11 +05:30
Rishabh Dave
4022e2ba7d cephfs: allow only "good" characters
Allow only [A-Za-z0-9-_.] characters for FS, volume, subvolume and
subvolume group names and add test for the same.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-08-20 15:34:02 +05:30
Ernesto Puerta
64ec1112bb
Revert "vstart_runner: rotate logs between testcases"
Fixes: https://tracker.ceph.com/issues/47032
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2020-08-19 19:25:22 +02:00
Shraddha Agrawal
d2ff5c5eb6 qa/tasks/cephadm.py: add ceph logs directory in job's info.yaml
This commit adds the file path of ceph log directories to the job's
info.yaml log file. The motivation behind this is, in case of job
timeout, the logs would still be tranferred to teuthology host
before nuking test machines using these ceph log directory paths in
job's info.yaml log file.

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>
2020-08-19 17:52:06 +05:30
Shraddha Agrawal
e991f04b59 qa/tasks/ceph.py: add ceph logs directory in job's info.yaml
This commit adds the file path of ceph log directories to the job's
info.yaml log file. The motivation behind this is, in case of job
timeout, the logs would still be tranferred to teuthology host
before nuking test machines using these ceph log directory paths in
job's info.yaml log file.

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>
2020-08-19 17:34:26 +05:30
Kefu Chai
dc977d543d qa/tasks/mgr: drop commented code
the test for diskprediction_cloud is never enabled, and the used
cloud-based service is not reachable anymore. let's just remove the dead
code.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-19 11:08:39 +08:00
Rishabh Dave
bb325e332c vstart_runner.py: rotate logs between testcases
Fixes: https://tracker.ceph.com/issues/46059
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-08-18 14:47:21 +05:30