* refs/pull/49971/head:
doc/cephfs: document MDS_CLIENTS_LAGGY health warning
qa: ignore warnings
qa: add test cases to check client eviction if an OSD is laggy
mds,messages: enable beacon to report clients lagginess
mds: do not evict client on laggy osds
common: add new config option to defer client eviction
osd: add method to check for laggy osds
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Otherwise, the MDS that just got replaced can transition to a rank
for another file system and the test cannot deterministically infer
which MDS needs to checked.
Fixes: http://tracker.ceph.com/issues/61764
Signed-off-by: Venky Shankar <vshankar@redhat.com>
qa: wait for 100 seconds to make sure the quota to be enforced
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Centos Stream 8 has removed the 'device-mapper-devel', 'libedit-devel'
and 'userspace-rcu-devel' packages from the mirrors and we need to
install it from powertools repo.
Fixes: https://tracker.ceph.com/issues/59683
Signed-off-by: Xiubo Li <xiubli@redhat.com>
* refs/pull/50875/head:
mon/MDSMonitor: ignore extraneous up:boot messages
qa: add test case for mds sending multiple boot messages
qa: support checking for a log message that should not exist
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
* refs/pull/49691/head:
qa: add test for opening a file via a hard link that is not in the same mds as the inode
mds: rdlock_path_xlock_dentry supports returning auth target inode
Reviewed-by: Venky Shankar <vshankar@redhat.com>
* refs/pull/51251/head:
PendingReleaseNotes: add a note about deleting files from lost+found directory
qa: add checks that validate removal of entries from lost+found dir
mds: allow unlink operation under lost+found directory
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Running file system scrub is recommended post running filesystem
data and metadata recovery. Running scrub isn't covered in tests.
Fixes: http://tracker.ceph.com/issues/59527
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
- test_nfs_export_creation_at_filepath:
ENOTDIR is raised instead of EINVAL which is better
aligned with the nature of the failure
- test_nfs_export_creation_at_symlink:
ENOTDIR is raised instead of ENOENT since the code
can now check if the path is symlink but won't follow
it.
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
It actually didn't test the invalid path but still ended with
ENOENT(which is expected in case path is invalid) as the test
didn't create a fs, and it failed saying "FS nfs-cephfs not found"
which too raises ENOENT and thus it always passed.
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
Class CapTester contains two distinct immiscible group of methods: one
that tests MON caps and other that tests MDS caps. When using CapTester
for the former reason the instantiation neither needs mount object and
the path where files for testing will be created nor it needs to run the
method that creates files for testing rw permissions. When using
this class for latter the case is the exact opposite.
Create 2 separate classes for each of these purpose and class that
inherits both of these classes so that instantiating the class becomes
as simple as it can be.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The worst case in kclient the dirty caps will be held for 60 seconds,
and also the MDS may defer updating the directory rstat for 5 seconds,
which is per tick, maybe longer if needs to wait for mdlog to flush.
Fixes: https://tracker.ceph.com/issues/59349
Signed-off-by: Xiubo Li <xiubli@redhat.com>
When trying to filling the volume space by continuing filling multiple
files, and when flushing the dirty caps back to MDS the MDS will try
to skip updating the parent rstat in 'mds_dirstat_min_interval' to
avoid propagating more often than this. That means the quota changes
couldn't be broadcasted to the clients in time.
So after waiting for 20 seconds, and if we try to write the existing
files only the first file could successfully update the parent quota
realm in MDS, but this won't increase the total size.
Fixes: https://tracker.ceph.com/issues/59349
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Move get_mon_cap_from_keyring() and get_fsnmes_from_moncap() from class
CapTester to main namespace of caps_helper.py so that they can be
imported freely and reused by tests.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This method checks if the output of the command "ceph fs ls" for client
ID it receives is same as the output printed for client.admin. Don't do
so, limit the test to only checking if "ceph fs ls --id client.x -k
keyring_file" prints fs name for which client.x has permissions.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Improvement #1:
CapTester.write_test_files() not only creates the test file but also
does the following for every mount object it receives in parameters -
* carefully produces the path for the test file as per parameters
received
* generates the unique data for each test file on a CephFS mount
* creates a data structure -- list of lists -- that holds all this
information along with mount object itself for each mount object so
that tests can be conducted at a later point
Untangle this mess of code by splitting this method into 3 separate
methods -
1. To produce the path for test file (as per user's need).
2. To generate the data that will be written into the test file.
3. To actually create the test file on CephFS.
Improvement #2:
Remove the internal data structure used for testing -- self.test_set --
and use separate class attributes to store all the data required for
testing instead of a tuple. This serves two purpose -
One, it makes it easy to manipulate all this data from helper methods
and during debugging session, especially while using a PDB session.
And two, make it impossible to have multiple mounts/multiple "test sets"
within same CapTester instance for the sake of simplicity. Users can
instead create two instances of CapTester instances if needed.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Inheritting CephFSTestCase in CapTester just for methods assertEqual()
and assertIn() from class unittest.TestCase is odd and heavy-weight.
Don't inherit CephFSTestCase and use simple assert instead.
Reference: https://github.com/ceph/ceph/pull/50882#discussion_r1160611549.
To avoid code duplication, a couple of similar methods have been added
instead.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The tuple was not meant to be passed as a whole but its individual
members are to be passed as a list of positional arguments.
Introduced-by: 87025d15858aa88b22afa9702511ccc7120f8b0b
Signed-off-by: Rishabh Dave <ridave@redhat.com>
mgr: Add one finisher thread per module
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Mounts may create a Filesystem object which does not create the file
system. This causes self.id to be Null when we try to wait for daemons
to be up. Load it just in time!
Fixes: https://tracker.ceph.com/issues/59332
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
test_mutlifs_auth.TestMDSCaps._create_client() not only creates a client
but also generate caps strings for the client as per the parameter this
method receives and and then writes the keyring to all remote machines.
This creates confusion when reading code on test methods in TestMDSCaps.
Let's re-arrange this code such that this confusion is avoided.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/49460/head:
qa: fix issue with fn unable to fetch port and ip
qa: fix helper function _check_nfs_cluster_status()
qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
qa: fix cluster creation failure in test_nfs.py
qa: test export creation at filepath and symlink
qa: added test case test_nfs_export_with_invalid_path
mgr/nfs: disallow non-existent paths when creating export
mgr/nfs/tests: mock check_cephfs_path
mgr/nfs/utils: add helper func to check cephfs path
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
_get_port_ip_info() fails to fetch port and ip due to empty 'backend' key:
2023-02-24T20:49:09.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph nfs cluster info test
2023-02-24T20:49:09.471 INFO:teuthology.orchestra.run.smithi042.stdout:{
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "test": {
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "backend": [],
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: "virtual_ip": null
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout: }
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:}
it then raises:
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner: info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:IndexError: list index out of range
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
Comment in the code says to wait for two minutes as cluster
creation takes time but actually it's waiting for thirteen
minutes, it's not required to wait this long, i think a minute
here is more than enough, also switched to using safe_while().
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
