Commit Graph

157 Commits

Author SHA1 Message Date
Rishabh Dave
437f2c75f5 qa/cephfs: don't import entire module needlessly
Importing entire module ceph_manager.py is pointless since only
ceph_manager.CephManager is required in qa/tasks/cephfs/filesystem.py.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-06-24 19:18:05 +05:30
Venky Shankar
7b2968570a Merge PR #49691 into main
* refs/pull/49691/head:
	qa: add test for opening a file via a hard link that is not in the same mds as the inode
	mds: rdlock_path_xlock_dentry supports returning auth target inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-05-18 12:49:33 +05:30
Patrick Donnelly
06c90a6c48
qa: check each fs for health
Fixes: https://tracker.ceph.com/issues/59425
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-04-11 10:16:53 -04:00
Patrick Donnelly
69565dbbf1
qa: load file system info if not created
Mounts may create a Filesystem object which does not create the file
system. This causes self.id to be Null when we try to wait for daemons
to be up. Load it just in time!

Fixes: https://tracker.ceph.com/issues/59332
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-04-05 18:24:22 -04:00
Zhansong Gao
9945f3b51f qa: add test for opening a file via a hard link that is not in the same mds as the inode
Signed-off-by: Zhansong Gao <zhsgao@hotmail.com>
2023-03-31 18:29:13 +08:00
Patrick Donnelly
293b90fe63
qa/tasks/cephfs: test damage to dentry's first is caught
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
3ba17390e9
qa/tasks/cephfs: use rank_asok and allow specifying rank
This will use the more efficient:

    ceph tell mds.<fsname>:<rank> ...

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
dparmar18
87cfd0b379 qa: added two testcases
- test_reject_client_session
- test_reject_client_session_on_reconnect

Also added function set_reject_client_session(bool) in filesystem.py

Fixes: https://tracker.ceph.com/issues/57090
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-22 20:35:54 +05:30
Venky Shankar
82f46b1fdc Merge PR #43284 into main
* refs/pull/43284/head:
	doc: describe bal_rank_mask configuration
	qa/cephfs: add bal_rank_mask test cases
	mds: add bal_rank_mask option for ceph fs set

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-10-26 11:28:25 +05:30
Yongseok Oh
10fb8f1377 qa/cephfs: add bal_rank_mask test cases
Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
2022-10-13 01:39:34 +00:00
Venky Shankar
bc5e1a6b7b
Merge pull request #47812 from rishabh-d-dave/json-decode-error
qa/cephfs: handle non-numeric values for json.loads()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-11 09:32:42 +05:30
Venky Shankar
8981012ad9
Merge pull request #46821 from batrick/i55470
qa: postgresql test suite workunit

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-03 10:08:05 +05:30
Venky Shankar
8b7537964b
Merge pull request #48179 from batrick/i57598
qa: use correct recovery procedure for alternate meta pools

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-03 10:07:13 +05:30
Patrick Donnelly
722b88328e
qa: raise error if fwd_scrub discovers damage
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:43 -04:00
Patrick Donnelly
8512bbc015
qa: simplify and use correct recovery procedure
Notably, this recovery procedure was missing scan_links.

Also, the test was oddly trying to recover the real file system in
addition to the recovery file system. I've removed that unnecessary
recovery.

Fixes: https://tracker.ceph.com/issues/57598
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-20 17:53:50 -04:00
Patrick Donnelly
acd395ace2
qa: output higher debugging for cephfs-journal-tool/cephfs-data-scan
Fixes: https://tracker.ceph.com/issues/57597
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-19 10:44:08 -04:00
Rishabh Dave
38fdf6df66 qa/cephfs: handle non-numeric values for json.loads()
Not handling values infinity, negative infinity and "NaN" properly leads
to JSONDecodeError. Replace "inf" by "Infinity" in string before loading
JSON and pass a method to json.load() to handle these values to fix this
issue.

Introduced-by: a8f138cd36
Fixes: https://tracker.ceph.com/issues/57299
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-08-26 17:06:25 +05:30
Xiubo Li
2a28060adb qa: make filesystem to be compatible with nautilus when creating pools
For old cephs, such as the nautilus, there is no --pg_num_min option.

Fixes: https://tracker.ceph.com/issues/56529
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-08-16 08:30:52 +08:00
Rishabh Dave
233cdea4e1
Merge pull request #47013 from rishabh-d-dave/qa-caps-method-authorize
qa/cephfs: upgrade method authorize in class Filesystem

Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-07-20 13:04:08 +05:30
Rishabh Dave
be4e39e4c2 qa/cephfs: upgrade method authorize in class Filesystem
Method authorize currently accepts caps as a list that should be as
follows -

(path1, perm2, path2, perm2)

Modify this method to also accepts caps arranged in following format -

((path1, perm1), (path2, perm2), (path3, perm3))

The latter format is pairs the path and the permission to be set on path
together. This makes the association more apparent and thus it less
prone to human error.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-07-07 21:18:48 +05:30
Venky Shankar
615f553cf4
Merge pull request #46560 from lxbsz/wip-55824
qa: wait rank 0 to become up:active state before mounting fuse client

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-06-20 09:52:32 +05:30
Neeraj Pratap Singh
4b26559858 qa: TestMDSMetrics.test_delayed_metrics failure
TestMDSMetrics.test_delayed_metrics is failing due to
the absence of omit_sudo parameter in the remote.run()
of set_inter_mds_block() in qa/tasks/cephfs/filesystem.py

Fixes: https://tracker.ceph.com/issues/56065
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-06-15 19:21:53 +05:30
Xiubo Li
72194627c1 qa: wait rank 0 to become up:active state before mounting fuse client
When setting the ec pool to the layout the filesystem may not be
ready, so when mounting a fuse client it will fail. To fix this we
need to wait at least the rank 0 to be in up:active state.

Fixes: https://tracker.ceph.com/issues/55824
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-06-14 09:20:45 +08:00
Venky Shankar
27f4729256
Merge pull request #45556 from mchangir/qa-add-subvolume-option-flavors
qa: add subvolume option flavors

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-13 12:29:43 +05:30
Jos Collin
47de5d79b8
qa: fix is_addr_blocklisted() to get blocklisted clients from 'osd dump'
By the introduction of range blocklist, the 'blocklist ls' command outputs
two lists. It's also straightforward to get the blocklisted clients directly
from 'osd dump' to avoid regression.

Fixes: https://tracker.ceph.com/issues/55516
Signed-off-by: Jos Collin <jcollin@redhat.com>
2022-05-06 13:50:32 +05:30
Rishabh Dave
ac5de474cf qa/cephfs: change default timeout from 900 secs to 300
15 minutes is unnecessarily large as a default value for timeout for a
command. Not having to wait unnecessarily on a crash of a command will
reduce teuthology's testing queue and will save individual developer's
time while running tests locally.

Whatever lines are modified for this purpose are also modified to follow
the stlye guideline, specfically wrapping at 80 characters.

Fixes: https://tracker.ceph.com/issues/54236
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-26 18:34:42 +05:30
Milind Changire
bf83eaa4e7 qa: enhancement for subvol creation and mounting
Fixes: https://tracker.ceph.com/issues/54317
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-04-07 14:15:56 +05:30
Venky Shankar
a612b3cb85
Merge pull request #43618 from kotreshhr/recover-symlink
mds: Store symlink target in first data object

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-01-04 14:39:22 +05:30
Patrick Donnelly
bf4168245d
qa: test that scrub causes reintegration
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-12-16 14:29:12 -05:00
Kotresh HR
75291459f2 qa/test_backtrace: Validate symlink xattr is stored
Validate the symlink xattr is stored on first data
object of the symlink along with backtrace if
'mds_symlink_recovery' option is enabled and
vice-versa.

Also add 'string_wrapper' class to decode bufferlist
to string. This helps 'ceph-dencoder' tool to decode
the symlink target stored, which is used in tests to
validate.

Signed-off-by: Kotresh HR <khiremat@redhat.com>
Fixes: https://tracker.ceph.com/issues/46166
2021-12-07 15:39:52 +05:30
Xiubo Li
3b44f20ac0 qa: move the optional client_config parameter to the end
Fixes: https://tracker.ceph.com/issues/53216
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-12-01 15:05:58 +08:00
Patrick Donnelly
c8810e46e8
qa: lengthen health warning wait
It's just a little too short!

Fixes: https://tracker.ceph.com/issues/52995
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-26 14:26:19 -04:00
Patrick Donnelly
f4a11a3290
qa: add test for standby-replay marking rank damaged
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-09-09 20:16:03 -04:00
Patrick Donnelly
295971b9c6 qa: add tests for compat manipulation and upgrade
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
665b36de4e Merge PR #42349 into master
* refs/pull/42349/head:
	mon/MDSMonitor: propose if FSMap struct_v is too old
	mon/MDSMonitor: give a proper error message if FSMap struct_v is too old
	mds/FSMap: use DECODE_OLDEST to gate FSMap version
	qa: add tests for fs dump of epoch and trimming
	qa: add file system support for dumping epoch
	mon/MDSMonitor: return mon_mds_force_trim_to even if equal to current epoch
	mon: add debugging for trimming methods
	mon: fix debug spacing
	qa: add nofs upgrade suite

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-07-28 10:45:08 -07:00
Patrick Donnelly
ee899d9a44 qa: add file system support for dumping epoch
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-28 07:07:05 -07:00
Xiubo Li
a448d1c3ee qa: wait pgs to be clean before using the pools
Or in some use cases, like the mds-full tests, we will hit the
"PG_AVAILABILITY" warning.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:54:03 +08:00
Xiubo Li
c1cea71299 qa: use the pg autoscale mode to calcuate the pg_num
Setting the pg_num to 8 is too small that some osds maybe not covered by the
pools, some osds maybe overloaded. Remove the hardcodeing pg_num here and let
the pg autoscale mode to calculate it as needed, and at the same time set the
pg_num_min to 64 to avoid the pg_num to small.

If ec pool is used, for the test cases most datas will go to the ec pool and
the primary replicated pool will store a small amount of metadata for all the
files only, so set the target size ratio to 0.05 should be enough.

Fixes: https://tracker.ceph.com/issues/45434
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Patrick Donnelly
4fc4ccb7a1
qa: convert mount calls to mount_wait
These tests want to immediately use the mount anyway. But the main
problem is, without waiting for the mount to complete, the command:

    chmod 1777 /path/to/mount

is not run so the mount cannot be written to by normal users without
sudo.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-01 10:56:10 -07:00
Patrick Donnelly
285e1547de
qa: remove obsolete deactivate routines
This is handled automatically since Mimic.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-21 12:52:10 -07:00
Patrick Donnelly
4702be4f47
qa: test standby-replay with fs:workloads
Fixes: https://tracker.ceph.com/issues/50045
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 11:37:56 -07:00
Patrick Donnelly
5292e88201
qa: reduce dependence on teuthology role list for mds
It's not yet possible to completely remove the dependency on
mds_ids/mds_daemons in the CephFS tests but this commit reduces it
enough for most code paths to work with cephadm.

The main change here is use of CephManager.do_rados, with some
improvements.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-21 10:35:07 -07:00
Patrick Donnelly
8e45af05c9
qa: remove get_lone_mds_id
This helps remove a dependence on the list of mds_daemons from
teuthology/cephadm.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-21 10:35:07 -07:00
Patrick Donnelly
6ca48f6330
qa: use mds_tell for getting mds status
mds_asok is broken with cephadm currently due to [1].

[1] https://tracker.ceph.com/issues/48019#note-5

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-21 10:35:06 -07:00
Patrick Donnelly
4c3c69b230
qa: remove fallback for non-multifs ceph version
This code was for jewel (IIRC) or older.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-21 10:35:06 -07:00
Patrick Donnelly
eccf961697
qa: add Filesystem.reset helper
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-21 10:35:06 -07:00
Sebastian Wagner
340281fe76 qa/tasks: some type annotations
Mostly for making my IDE aware of things

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-03-10 15:02:41 +01:00
Patrick Donnelly
fe31c23964
Merge PR #39814 into master
* refs/pull/39814/head:
	qa/cephfs: move is_blocklisted() to filesystem.CephCluster

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-09 13:36:40 -08:00
Rishabh Dave
4d0f56fcc5 qa/cephfs: move is_blocklisted() to filesystem.CephCluster
Using self.fs.mon_manager in mount.py can lead to a crash since self.fs
can be None. Move is_blocklisted() to tasks.filesystem.CephCluster where
it can get access to mon_manager without depending on objects
representing Ceph FSs.

Fixes: https://tracker.ceph.com/issues/49511
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-03-06 09:11:36 +05:30
Patrick Donnelly
13c50a9923
Merge PR #38684 into master
* refs/pull/38684/head:
	qa: add _check_scrub_status helper to simplify the code
	qa: add run_scrub helper in filesystem class
	qa: add get_scrub_status helper in filesystem class
	qa: wait the scrub task to complete
	qa: remove passed_validation check for test_damage
	qa: move wait_until_scrub_complete helper to filesystem class
	mds: simplify the C_MDS_EnqueueScrub finish code

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-04 13:27:45 -08:00