Wait for scrub to finish during test_scrub_pause_and_resume_with_abort
which otherwise races and fails with an incorrect assertion.
Fixes: https://tracker.ceph.com/issues/48812
Signed-off-by: Milind Changire <mchangir@redhat.com>
test_client_recovery was also using mntopts to specify additional
options to ceph-fuse. Because the two prior commits unify the behavior
of ceph-fuse and the kernel mount so that the "-o" option is available
for both, that changes breaks this test. Add a special set of args
available only for fuse (there is no equivalent on the kernel).
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Also, fix the type default to None. A list does not work in the code but
the default is not used in QA.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Only fuse would support the "mountpoint" yaml config. Move it to
mount.py so it works consistently with the kernel mount. Ditto for the
cephfs mount point.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Notably, this recovery procedure was missing scan_links.
Also, the test was oddly trying to recover the real file system in
addition to the recovery file system. I've removed that unnecessary
recovery.
Fixes: https://tracker.ceph.com/issues/57598
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
qa: ignore disk quota exceeded failure in test
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
The list of pending clones didn't match the expected.
The test fails occasionally. This is because the
clone would have finished before checking for this.
So increase the clone delay.
Fixes: https://tracker.ceph.com/issues/57446
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Quota enforcement takes some time. It is documented.
Test should ignore 'Disk Quota Exceeded' failure.
Fixes: https://tracker.ceph.com/issues/56632
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
Not handling values infinity, negative infinity and "NaN" properly leads
to JSONDecodeError. Replace "inf" by "Infinity" in string before loading
JSON and pass a method to json.load() to handle these values to fix this
issue.
Introduced-by: a8f138cd36
Fixes: https://tracker.ceph.com/issues/57299
Signed-off-by: Rishabh Dave <ridave@redhat.com>
mds/MDBalancer: add an arg to limit depth when dump loads for dirfrags
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
mgr/stats: change in structure of perf_stats o/p
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
mds/client: fail the request if the peer MDS doesn't support getvxattr op
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
test_client_metrics_and_metadataand other tests has been
updated as earlier it was checking according to the old structure
of perf stats o/p, which has been changed in this PR.
Fixes: https://tracker.ceph.com/issues/56162
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
Before unmounting check if the client has been evicted and, if so, run
"umount -f -l" for the mount point of the client and cleanup the mount
right after it.
Attempting to unmount, cleanup or operate in any way over mount point
of a evicted client will hang the operation (and thereby our Python
code too). Lazy-force unmount prevents such hangs for our Python code
and also frees the mount point.
This commit also adds code to gather session info for kernel mounts
after mounting is successful. This is a necessity since network address
of session is needed to check if it is blocked by Ceph cluster.
Fixes: https://tracker.ceph.com/issues/56476
Signed-off-by: Rishabh Dave <ridave@redhat.com>
For new OPs the old ceph may will crash the MDSs when receiving
unknow OPs. As a workaround the new clients will check the sessions
feature bits to decide to send the new OPs or not.
Fixes: https://tracker.ceph.com/issues/56529
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Old cephs, such as nautilus, will use "blacklist" instead of "blocklist".
Fixes: https://tracker.ceph.com/issues/56529
Signed-off-by: Xiubo Li <xiubli@redhat.com>
For old cephs, such as the nautilus, there is no --pg_num_min option.
Fixes: https://tracker.ceph.com/issues/56529
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Internal directories: '_nogroup', '_index', '_legacy', '_deleting'
1. Internal directories should be filtered in 'subvolmegroup ls' command.
2. Internal directories should not be accepted as a group name.
Fixes: https://tracker.ceph.com/issues/55762
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
Validate subvolume discover on upgrade from
legacy subvolume to v1. The handcrafted
`.meta' file on legacy subvolume root should
not be used for any subvolume apis like getpath,
authorize.
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Remove setup_test_env() because removing this methods removes an extra
layer of abstraction which makes tests more readable. Rename
_create_client_and_keyring_file() to just _create_client() and reverse
the order of parameters in remount_with_new_client() and set default
value of cephfs_mntpt to '/'.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
test_single_path_rootsquash instead of using helper methods duplicates
the code from those methods. This commit fixes that and also upgrade
this method since caps_helper was upgraded in previous commits.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Upgrade test_admin.FsAuthorize to repair its compatibility with
caps_helper.py since caps_hepler.py has been heavily modified in
previous commits.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
caps_helper.py since caps_helper.py has been heavily modified in
previous commits.
setup_fs_contents() is being deleted since it is not used anymore.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The intention is to make logs contain some information of what's being
done by caps_helper.py regardless of which test/method is calling it.
This should make logs more understandable and also add hints for
future debugging.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
With this commit, the design of run_mon_cap_tests() is now aligned with
rest of CapTester -- it's not meant to be inherited by test class
anymore and it is to be called by using CapTester instance.
This commit also changes working of this method. Now instead of
obtaining FS names from Python objects representing FSs, it obtains
those names from MON cap. This removes the need to pass the FS objects
around.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Let's change the file data to include the file path, the CephFS name
and the host FS and CephFS mountpoint so that the bugs where, let's say,
"cephfs2" is mounted instead of "cephfs1" (where obviously both the
CephFSs lie on the same Ceph cluster) is caught by the tests due to
uniqueness of every test file's content.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Simplify methods in CapsTester by adding a test set, which will be a
list of tuples. The first element in tuple will be the mount object,
the second will be the test file path and the third will be the test
file data. Thus instead of having three independent class variables
that are always used together now we have list of test sets making
management of multiple test sets simpler.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Modify caps_helper.py such that calling code does't have to store
returned values just to pass those values again to a method in
caps_helper.py. This is a common pattern where write_test_files()
return 3 values and all 3 passed as it is to run_cap_tests().
The easy way to do it is to upgrade caps_helper so that it can be used
an object and not just as a class supplying helper methods. The return
values will be stored by the object internally and thus resued. In case
of testing multiple FSs inside a single test method, we'll now need
multiple instances of this class to keep those values separate.
And since CapsHelper is not just a class supplying helper methods
anymore, it's being renamed to CapTester.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Concerned method: caps_helper.CapsHelper.write_test_file()
When CephFS mountpoint is changed inside a single test method, the test
files created at root are neither accessible nor useful. Therefore, add
an option to create the test files in the directory passed by the user.
This also increases general flexibility of the concerned method.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Instead of relying on value of a mutable variable, actually check if the
CephFS is mounted on the system. This will prevent bugs due to stale and
incorrect values.
Fixes: https://tracker.ceph.com/issues/54283
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Method authorize currently accepts caps as a list that should be as
follows -
(path1, perm2, path2, perm2)
Modify this method to also accepts caps arranged in following format -
((path1, perm1), (path2, perm2), (path3, perm3))
The latter format is pairs the path and the permission to be set on path
together. This makes the association more apparent and thus it less
prone to human error.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
A new path is obtained every iteration which is appended to command
arguments. After the command has been executed successfully, delete the
the path from command's arguments so that the command's arguments don't
have two paths in next iteration and it works as expected.
Fixes: https://tracker.ceph.com/issues/56416
Signed-off-by: Rishabh Dave <ridave@redhat.com>
If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output.
Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
Right now, run_shell() in mount.py accepts both "sudo" and "omit_sudo"
as parameters. It's better to accept only one of these two parameters.
A call to run_shell() where both are set to opposing values will be
buggy. Therefore, methods calling run_shell() must add "sudo" to command
arguments before call and set omit_sudo to False in call.
As a result of this change, methods like stat() and run_python() in
mount.py are now modified to add "sudo" to command arguments
and set omit_sudo to False within their own definitions.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Methods run_as_user() and run_python() don't set omit_sudo to False even
when command arguments contain sudo. This will cause vstart_runner.py to
delete "sudo" from command arguments which will/might lead to a bug.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Directory hierarchy may be deep for a large filesystem, dump loads would output
a lot and take a long time. So add an arg to limit depth when dump loads of dirfrags.
Fixes: https://tracker.ceph.com/issues/56058
Signed-off-by: haoyixing <haoyixing@kuaishou.com>
mon: verify data pool is already not in use by any file system
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
TestMDSMetrics.test_delayed_metrics is failing due to
the absence of omit_sudo parameter in the remote.run()
of set_inter_mds_block() in qa/tasks/cephfs/filesystem.py
Fixes: https://tracker.ceph.com/issues/56065
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
If --group_name=_nogroup is provided in the command then
throw error permission denied as it is internal group of ceph fs.
Fixes: https://tracker.ceph.com/issues/55759
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
When setting the ec pool to the layout the filesystem may not be
ready, so when mounting a fuse client it will fail. To fix this we
need to wait at least the rank 0 to be in up:active state.
Fixes: https://tracker.ceph.com/issues/55824
Signed-off-by: Xiubo Li <xiubli@redhat.com>
The 'size' shown in the output of snapshot info command relies on
rstats which is incorrect snapshot size. It tracks size of the
subvolume from the snapshot has been taken instead of the snapshot
itself. Hence having the 'size' field in the output of 'snapshot info'
doesn't make sense until the rstats is fixed.
Fixes: https://tracker.ceph.com/issues/55822
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
mds: fix crash when exporting unlinked dir
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Validates the subvolume removal is successful if the
corresponding group's quota is set.
Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
qa/cephfs: set omit_sudo False when sudo is set to True
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
qa/cephfs: fix minor bug in caps_helper.py's run_mon_cap_tests()
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
CephFSMount.get_key_from_keyfile() should raise an exception instead of
returning None if key is not found in keyring file.
Fixes: https://tracker.ceph.com/issues/50010
Signed-off-by: Rishabh Dave <ridave@redhat.com>
When an entity's MON cap contains no FS name in it, the entity has the
access to all the FSs on cluster and not just to the default FS of the
cluster.
Fixes: https://tracker.ceph.com/issues/55558
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Following test are added:
1. Set custom metadata for subvolume snapshot.
2. Set custom metadata for subvolume snapshot(Idempotency).
3. Get custom metadata for specified key.
4. Get custom metadata if specified key not exist (Expecting error ENOENT).
5. Get custom metadata if no any key-value is added means section not exist (Expecting error ENOENT).
6. Update value for existing key in custom metadata.
7. List custom metadata of subvolume snapshot.
8. List custom metadata of subvolume snapshot if no any key-value is added (Expect empty json/dictionary)
9. Remove custom metadata for specified key.
10. Remove custom metadata if specified key not exist (Expecting error ENOENT).
11. Remove custom metadata if no any key-value is added means section not exist (Expecting error ENOENT).
12. Remove custom metadata with --force option.
13. Remove custom metadata with --force option if specified key not exist (Expecting command to succeed because of '--force' option)
14. Remove subvolume snapshot and verify whether metadata for snapshot is removed or not
Fixes: https://tracker.ceph.com/issues/55401
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
