Commit Graph

1229 Commits

Author SHA1 Message Date
Rishabh Dave
fec9be184d qa/cephfs: add 'rhel' to family of RH OS in xfstest_dev.py
The string returned by get_system_type() for RHEL 8.4 recently wasn't
one of the strings that xfstests_dev.py expects. The string returned
this time was instead "rhel". Add this string to the list of strings
that xfstests_dev.py expects for Red Hat OS family so that this failure
won't occur again in future. Also log the detected distro and version so
that it is easier to debug when this issue occurs next time.

Fixes: https://tracker.ceph.com/issues/58726
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-23 22:31:27 +05:30
Rishabh Dave
4017bfa2c5
Merge pull request #46574 from rishabh-d-dave/xfstests-dev-improvements
qa/cephfs: improvements for xfstests_dev.py

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-23 17:08:53 +05:30
Venky Shankar
da7928adab Merge PR #50196 into main
* refs/pull/50196/head:
	qa: export DIFF_LENGTH env parameter to print the full git diff

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-23 11:16:21 +05:30
Rishabh Dave
92aaaf7b57 qa/cephfs: save results dir from xfstests-dev repo
The "results" directory in xfstests-dev repo contains logs that are
created by it during execution of its tests. Copying this directory to
Ceph log directory saves it from being erased during tear down conducted
by teuthology test runner. These logs allow deeper inspection in to the
cause of failure of tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:44:29 +05:30
Rishabh Dave
053a1fb851 qa/cephfs: don't create symlinks for xfstests-dev
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
18465b8f56 qa/cephfs: modify user creation code for xfstests-dev
Create user "fsgqa2" and, since xfstests-dev's README uses option "-m"
while creating user "fsgqa", do the same in our codebase.

Creating user "fsgqa2" is optional along with other users according to
xfstests-dev's README. Create all these users is a preventive step since
tests needing user/users maybe skipped during execution.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
392e278749 qa/cephfs: log xfstests-dev local.config
Log contents of xfstests-dev repo config file "local.config" to make
debugging easier.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
4d75730200 qa/cephfs: minor changes in xfstests_dev.py
In xfstests-dev repo, local.config options "SCRATCH_DEV" and
"SCRATCH_MNT" were enabled in commit
2ba6b87105 but TODO comment for the
wasn't deleted. Let's delete that and use format string instead of
format function on regular string.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
49c7a79f8a qa/cephfs: make it easier to use xfstests-dev using vstart_runner
The intention of this re-arrangement is to make it easier for developers
to use an already cloned and built xfstests-dev repo  with
vstart_runner.py.

The re-arrangements are -

* move tearDown() closer to setUp() (so that devs can
  enable/disable parts of them easily),

* carve out separate methods (so that these codeblocks can be disabled
  in a single go) for -
  ** deleting groups and users,
  ** deleting xfstests-dev and xfsprogs-dev repo and
  ** building xfstests-dev repo

* carve out method for deleting xfstests-dev and xfsprogs-dev repos

* add self.xfstests_repo_path comment (for devs to set path to
  xfstests-dev repo), and,

* add a note showing how to use already cloned and built xfstests-dev
  repo

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:07:48 +05:30
Rishabh Dave
4a00b2bbb1 qa/cephfs: use standard name for logging variable
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:03:05 +05:30
Venky Shankar
ae6f1b18be Merge PR #50175 into main
* refs/pull/50175/head:
	cephfs: upgrade cephfs-shell's path wherever necessary

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-02-22 19:47:43 +05:30
Xiubo Li
0f9d3f2d3d qa: export DIFF_LENGTH env parameter to print the full git diff
From the xfstests-dev's README doc set DIFF_LENGTH to 0 will print
the full diff instead of the default 10.

This will be very helpful to debug the test failures, or we will
lose many valuable debug infomation.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-22 14:15:14 +08:00
Venky Shankar
86b90286cd Merge PR #49934 into main
* refs/pull/49934/head:
	qa: add test_fscrypt_dummy_encryption test case support
	qa: add 'options' parameter support for write_local_config
	qa: add ceph.exclude file to exclude individual tests
	qa: add require_kernel_mount helper support
	qa: rename test_fscrypt to test_fscrypt_encrypt

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-22 11:41:44 +05:30
Rishabh Dave
48ef044477 cephfs: upgrade cephfs-shell's path wherever necessary
Commit dc69033763 moves cephfs-shell from
"<CEPH-REPO-ROOT>/src/tools/cephfs/" to
"<CEPH-REPO-ROOT>/src/tools/cephfs/shell" but cephfs-shell's location in
src/vstart.sh and qa/tasks/cephfs/test_cephfs_shell.py is left
un-updated. This produces a broken vstart_environment.sh and broken
export command in test_cephfs_shell.py.

Introduced-by: dc69033763
Fixes: https://tracker.ceph.com/issues/58795
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-21 19:16:27 +05:30
Xiubo Li
b420db8dba qa: add test_fscrypt_dummy_encryption test case support
This will test around 130 test cases with the test_dummy_encryption
option enabled.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-21 16:15:12 +08:00
Xiubo Li
306b8bea4a qa: add 'options' parameter support for write_local_config
This will allow the fscrypt test case to add dedicated options.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-21 12:41:33 +08:00
Xiubo Li
002d45b589 qa: add ceph.exclude file to exclude individual tests
Skip some tests, which will take to much time or will fail, for now.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-21 12:41:30 +08:00
Jos Collin
ddd20d4d4d
qa: add tests for cephfs-top
* Updated the existing tests
* Added new qa tests for --dump and --dumpfs options
* Updated the cluster configuration to 2 mds, 2 clients

Fixes: https://tracker.ceph.com/issues/57014
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-02-16 15:13:19 +05:30
Xiubo Li
0ee2f80367 qa: add require_kernel_mount helper support
This will simplify the code later.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:43:39 +08:00
Xiubo Li
b699307a34 qa: rename test_fscrypt to test_fscrypt_encrypt
The test_fscrypt_encrypt will only run the 'encrypt' related test
cases without 'test_dummy_encryption' option enabled. This will
test the filename and content verification.

After this I will add the whole test cases with 'test_dummy_encryption'
option.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:43:39 +08:00
Rishabh Dave
44d7ab943e
Merge pull request #47753 from mchangir/mon-disable-snap-id-allocation-for-fsmap-pools
mon: disable snap id allocation for fsmap pools

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-01-30 16:35:04 +05:30
John Mulligan
e03331e7c4 qa/test_nfs: fix test failure when cluster does not exist
The patches that add object formatting / decorators to the nfs module
also made error handling more generic when accessing an nfs cluster and
now returns a nonzero exit code. A test was after the PR adding the
object format support that only checked an error message.
Update the test to match the new nfs module behavior as well as fixing a
typo.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 15:43:54 -05:00
John Mulligan
57e94dd502 cephfs/test_nfs: adapt _test_list_cluster to expect JSON output
Previously, the '... nfs cluster ls' command emitted newline separated
list of cluster names. For consistency with other module commands
reporting via JSON the cluster ls mgr function now emits JSON. Update
the test to match the new behavior of the module.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 13:44:11 -05:00
Venky Shankar
7443a451f4 Merge PR #49172 into main
* refs/pull/49172/head:
	qa/test_nfs: added testcase test_non_existent_cluster
	mgr/nfs: handle bad cluster name during info command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-12-22 22:19:58 +05:30
Venky Shankar
754799097d test: verify that snapshot items are accounted when deciding to split/merge a directory
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-12-22 20:01:36 +05:30
dparmar18
b91aff831d qa/test_nfs: added testcase test_non_existent_cluster
Fixes: https://tracker.ceph.com/issues/58138
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-22 19:44:17 +05:30
Venky Shankar
8e6fae4f13 Merge PR #48450 into main
* refs/pull/48450/head:
	qa: add test_mdsdir_scrub_backtrace
	mds: avoid ~mdsdir's scrubbing and reporting damage health status

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-12 19:42:22 +05:30
Venky Shankar
f80d1ad238 Merge PR #48642 into main
* refs/pull/48642/head:
	qa: Fix test_subvolume_snapshot_info_if_orphan_clone

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-12 19:41:47 +05:30
Kotresh HR
87b50f3ff1 qa: Fix test_subvolume_snapshot_info_if_orphan_clone
Fixes the usage of the grep command. It was failing as below.

smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep '"clone' 'snaps"' \
-A3 ./volumes/_nogroup/subvolume_0000000000856603/.meta)
INFO:teuthology.orchestra.run.smithi008.stderr:grep: snaps": No such file or directory

Fixes: https://tracker.ceph.com/issues/57446
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-12-02 15:48:02 +05:30
Xiubo Li
496bf662ef qa: remove unused 'teuthology.orchestra.run' in xfstests_dev.py
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 17:42:13 +08:00
Xiubo Li
383add444b qa: add xfstests fscrypt test cases support
This will test the common fscrypt features such as the set_encpolicy,
get_encpolicy,etc only.

Will run the IO path test in another test suite.

Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 16:33:10 +08:00
Xiubo Li
4cc5bcd602 qa: install the dependency packages required by xfsprogs-dev repo
We need to install the xfsprogs-dev from source to make the xfs_io
command to support the 'add_enckey'.

Currently we couldn't install 'userspace-rcu-devel', 'libedit-devel'
and 'device-mapper-devel' from rhel distos. So the file encyrpt test
cases will skip the rhel distos for now. And it works well on centos
and ubuntu.

Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 16:33:10 +08:00
Xiubo Li
d5cb1cf529 qa: install xfsprogs-dev from source to support encrypt
Currently the xfsprogs packages won't support the 'add_enckey' feature
in xfs_io command, we need to install it from source.

Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 16:33:10 +08:00
Xiubo Li
b86450136d qa: switch TEST_FS_MOUNT_OPTS to CEPHFS_MOUNT_OPTIONS
Both the test_dev and scratch_dev could use the CEPHFS_MOUNT_OPTIONS.
Without this the xfstests tests will just be aborted without any error
logs when testing the encrypt and it's very hard to debug.

Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 16:33:10 +08:00
Xiubo Li
2ba6b87105 qa: enable SCRATCH_DEV for file encryption tests
ceph file encryption test cases need the SCRATCH_DEV.

Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 16:33:10 +08:00
Xiubo Li
4c97a9e469 qa: switch to https protocol for ffsb and xfstests-dev repos
Since the git protocol is not reachable any more, just switch it
to https.

Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-12-01 16:33:10 +08:00
Kefu Chai
2413da8c85 qa/tasks/cephfs: annotate variable without comment
to silence the warning like
```
flake8 run-test: commands[0] | flake8 --select=F,E9 --exclude=venv,.tox
./tasks/cephfs/test_full.py:6:5: F401 'typing.Optional' imported but unused
```

since we've migrated to python3.6 and up, there is no need to be
compatible with Python 2 anymore.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-11-24 14:06:24 +08:00
Neeraj Pratap Singh
014582bfb2 qa: add test_mdsdir_scrub_backtrace
Fixes: https://tracker.ceph.com/issues/58030
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-11-22 12:13:08 +05:30
Milind Changire
d8c55edaf3 qa: test snap id failures for fs pools
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-11-21 16:14:53 +05:30
Venky Shankar
82f46b1fdc Merge PR #43284 into main
* refs/pull/43284/head:
	doc: describe bal_rank_mask configuration
	qa/cephfs: add bal_rank_mask test cases
	mds: add bal_rank_mask option for ceph fs set

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-10-26 11:28:25 +05:30
Yongseok Oh
10fb8f1377 qa/cephfs: add bal_rank_mask test cases
Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
2022-10-13 01:39:34 +00:00
Venky Shankar
296ba0641a
Merge pull request #48196 from neesingh-rh/wip-human-readable-volume-info
mgr/volumes: Add human-readable flag to volume info command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-10-11 16:48:19 +05:30
Venky Shankar
bc5e1a6b7b
Merge pull request #47812 from rishabh-d-dave/json-decode-error
qa/cephfs: handle non-numeric values for json.loads()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-11 09:32:42 +05:30
Rishabh Dave
adc65d6bb4
Merge pull request #48182 from lxbsz/wip-57580
qa: make mount to be compatible with nautilus for blocklist

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2022-10-11 00:05:55 +05:30
Venky Shankar
ee858598e3
Merge pull request #48218 from mchangir/qa-test_scrub_pause_and_resume_with_abort-wait-for-scrub-to-finish
qa: wait for scrub to finish

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-10-04 11:45:24 +05:30
Venky Shankar
8981012ad9
Merge pull request #46821 from batrick/i55470
qa: postgresql test suite workunit

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-03 10:08:05 +05:30
Venky Shankar
8b7537964b
Merge pull request #48179 from batrick/i57598
qa: use correct recovery procedure for alternate meta pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-03 10:07:13 +05:30
Venky Shankar
187d73bf4c
Merge pull request #48174 from batrick/i57597
qa: output debugging for cephfs-journal-tool/cephfs-data-scan

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-09-29 18:14:23 +05:30
Rishabh Dave
fb25b0b17f
Merge pull request #47991 from kotreshhr/qa-fix-57446
qa: Fix test_subvolume_snapshot_info_if_orphan_clone

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-09-29 17:56:35 +05:30
Rishabh Dave
2ff34d1d60
Merge pull request #47985 from kotreshhr/qa-fix-57205
qa: Fix test_subvolume_group_ls_filter_internal_directories

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-09-29 17:54:24 +05:30