Commit Graph

131 Commits

Author SHA1 Message Date
Nikhilkumar Shelke
41b0cb70e0 qa: ignore disk quota exceeded failure in test
Quota enforcement takes some time. It is documented.
Test should ignore 'Disk Quota Exceeded' failure.

Fixes: https://tracker.ceph.com/issues/56632
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-29 13:42:11 +05:30
Venky Shankar
d1ca282f60
Merge pull request #45949 from neesingh-rh/feature-51434
mgr/volumes: Add volume info command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-23 21:33:09 +05:30
Kotresh HR
aece3b9b28 qa: Validate cleaning of the stale snapshot metadata
Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 18:27:54 +05:30
Nikhilkumar Shelke
7b585d4db9 qa: filter internal directories in 'subvolumegroup ls' command
Internal directories: '_nogroup', '_index', '_legacy', '_deleting'
1. Internal directories should be filtered in 'subvolmegroup ls' command.
2. Internal directories should not be accepted as a group name.

Fixes: https://tracker.ceph.com/issues/55762
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-05 14:05:49 +05:30
Neeraj Pratap Singh
eaaea3bea3 qa: test the fs volume info cmd
Fixes: https://tracker.ceph.com/issues/51434
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-08-05 13:00:25 +05:30
Nikhilkumar Shelke
c0c386b197 docs: minor doc fixes of showing in progress clones for a snapshot
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-04 19:16:39 +05:30
Venky Shankar
bcfc2e2089
Merge pull request #46876 from neesingh-rh/feature_55821
mgr/volumes: add interface to check the presence of subvolumegroups/subvolumes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-08-04 18:40:10 +05:30
Kotresh HR
5a33f7e6f6 qa: Fix use of 'sudo' args
https://tracker.ceph.com/issues/56727
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-07-30 05:33:23 +05:30
Neeraj Pratap Singh
f182ab4676 qa: add test cases for subvolumegroup/subvolume exist command
Fixes: https://tracker.ceph.com/issues/55821
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-07-26 16:33:46 +05:30
Kotresh HR
c8416c3f0e qa: Remove unused variable
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-07-22 19:20:33 +05:30
Kotresh HR
1d66216902 qa: validate subvolume discover on upgrade
Validate subvolume discover on upgrade from
legacy subvolume to v1. The handcrafted
`.meta' file on legacy subvolume root should
not be used for any subvolume apis like getpath,
authorize.

Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-07-22 19:20:09 +05:30
Rishabh Dave
17b6d4374d
Merge pull request #46191 from rishabh-d-dave/qa-cephfs-mount-rm-sudo
qa/cephfs: remove sudo parameter from run_shell()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-07-08 19:30:16 +05:30
Nikhilkumar Shelke
f4c6bdb3c6 qa: display in-progress clones for a snapshot
If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-07-04 13:31:40 +05:30
Rishabh Dave
5967592792 qa/cephfs: remove sudo parameter from run_shell()
Right now, run_shell() in mount.py accepts both "sudo" and "omit_sudo"
as parameters. It's better to accept only one of these two parameters.
A call to run_shell() where both are set to opposing values will be
buggy. Therefore, methods calling run_shell() must add "sudo" to command
arguments before call and set omit_sudo to False in call.

As a result of this change, methods like stat() and run_python() in
mount.py are now modified to add "sudo" to command arguments
and set omit_sudo to False within their own definitions.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-06-27 19:56:04 +05:30
Venky Shankar
c88a5c832e
Merge pull request #46635 from nmshelke/fix-55822
mgr/volumes: remove incorrect 'size' from output of 'snapshot info'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by:  Anthony D'Atri <anthony.datri@gmail.com>
2022-06-20 09:50:20 +05:30
Venky Shankar
0fb05df9ec
Merge pull request #46636 from nmshelke/fix-55759
mgr/volumes: subvolume ls command crashes if groupname as '_nogroup'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-20 09:48:35 +05:30
Nikhilkumar Shelke
dc4b0ee405 qa: subvolume ls command crashes if groupname as '_nogroup'
If --group_name=_nogroup is provided in the command then
throw error permission denied as it is internal group of ceph fs.

Fixes: https://tracker.ceph.com/issues/55759
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-06-14 12:34:51 +05:30
Nikhilkumar Shelke
9957a036df qa: remove incorrect 'size' from output of 'snapshot info'
The 'size' shown in the output of snapshot info command relies on
rstats which is incorrect snapshot size. It tracks size of the
subvolume from the snapshot has been taken instead of the snapshot
itself. Hence having the 'size' field in the output of 'snapshot info'
doesn't make sense until the rstats is fixed.

Fixes: https://tracker.ceph.com/issues/55822
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-06-12 17:17:03 +05:30
Kotresh HR
256c75e260 qa: Add tests which validates quota limitation for subvolumegroup quota
Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-06-06 12:40:46 +05:30
Kotresh HR
e9fe37b8e1 qa: Add subvolume removal tests when it's group quota exceeds
Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-06-06 12:40:46 +05:30
Kotresh HR
12e003f980 qa: Add tests for subvolumegroup idempotence creation
Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-06-06 12:40:46 +05:30
Kotresh HR
c006181803 qa: Add tests for subvolume removal on group quota set
Validates the subvolume removal is successful if the
corresponding group's quota is set.

Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-06-06 12:40:46 +05:30
Kotresh HR
dbe8cf1792 qa: Add tests for subvolumegroup resize
Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-06-06 12:40:46 +05:30
Kotresh HR
085700bf19 qa: Add subvolume group quota tests
Fixes: https://tracker.ceph.com/issues/53509
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-06-06 12:40:46 +05:30
Nikhilkumar Shelke
6fd28cc9d6 qa: set, get, list and remove custom metadata for snapshot
Following test are added:
1. Set custom metadata for subvolume snapshot.
2. Set custom metadata for subvolume snapshot(Idempotency).
3. Get custom metadata for specified key.
4. Get custom metadata if specified key not exist (Expecting error ENOENT).
5. Get custom metadata if no any key-value is added means section not exist (Expecting error ENOENT).
6. Update value for existing key in custom metadata.
7. List custom metadata of subvolume snapshot.
8. List custom metadata of subvolume snapshot if no any key-value is added (Expect empty json/dictionary)
9. Remove custom metadata for specified key.
10. Remove custom metadata if specified key not exist (Expecting error ENOENT).
11. Remove custom metadata if no any key-value is added means section not exist (Expecting error ENOENT).
12. Remove custom metadata with --force option.
13. Remove custom metadata with --force option if specified key not exist (Expecting command to succeed because of '--force' option)
14. Remove subvolume snapshot and verify whether metadata for snapshot is removed or not

Fixes: https://tracker.ceph.com/issues/55401
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-05-25 14:59:53 +05:30
Venky Shankar
3f13df8388
Merge pull request #45603 from nmshelke/feature-54472
mgr/volumes: set, get, list and remove metadata of subvolume

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-19 16:52:13 +05:30
Venky Shankar
1ac01c60df
Merge pull request #45816 from kotreshhr/clone-failure-status
mgr/volumes: Show clone failure reason in clone status command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-18 09:43:40 +05:30
Kotresh HR
916a5981cf qa: Add test for clone failure status
Fixes: https://tracker.ceph.com/issues/55190
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-04-13 14:26:20 +05:30
Nikhilkumar Shelke
d1f4ecd557 qa: set, get, list and remove custom metadata for subvolume
Following test are added:
1. Set custom metadata for subvolume.
2. Set custom metadata for subvolume(Idempotency).
3. Get custom metadata for specified key.
4. Get custom metadata if specified key not exist (Expecting error ENOENT).
5. Get custom metadata if no any key-value is added means section not exist (Expecting error ENOENT).
6. Update value for existing key in custom metadata.
7. List custom metadata of subvolume.
8. List custom metadata of subvolume if no any key-value is added (Expect empty json/dictionary)
9. Remove custom metadata for specified key.
10. Remove custom metadata if specified key not exist (Expecting error ENOENT).
11. Remove custom metadata if no any key-value is added means section not exist (Expecting error ENOENT).
12. Remove custom metadata with --force option.
13. Remove custom metadata with --force option if specified key not exist (Expecting command to succeed because of '--force' option)
14. Set and Get custom metadata for legacy subvolume.
15. List and Remove custom metadata from legacy subvolume.

Fixes: https://tracker.ceph.com/issues/54472
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-04-07 12:30:42 +05:30
Kotresh HR
15a2ab4e26 mgr/volumes: Fix idempotent subvolume rm
The subvolume deletion of a subvolume which is already deleted
with retain snapshots option fails with 'EAGAIN: clone in progress'
error. After subvolume deletion with retain snapshots, the subvolume
exists until the trash directory (resides inside subvolume) is
cleaned up. The subvolume deletion issued while the trash directory is not empty, should
pass. This patch fixes the same.

Credit: Issue discovery and fix suggestion to John Mulligan <jmulligan@redhat.com>
Fixes: https://tracker.ceph.com/issues/54625
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-03-29 10:57:47 +05:30
Nikhilkumar Shelke
cc64c57c5c qa: verify idempotent subvolume creation for 'mode' argument
Fixes: https://tracker.ceph.com/issues/54375
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-03-09 11:55:14 +05:30
Venky Shankar
fe3d543a0d
Merge pull request #44873 from kotreshhr/clone-inherit-quota-files
mgr/volumes: Inherit file quota attr to clone

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-21 14:37:44 +05:30
Venky Shankar
b8afe74b49
Merge pull request #44854 from kotreshhr/subvolumegroup_ls
mgr/volumes: Fix subvolumegroup ls

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-21 14:37:03 +05:30
Kotresh HR
95a83efcbe qa: Add test for subvolumegroup ls filter
Fixes: https://tracker.ceph.com/issues/54099
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-02-16 11:49:47 +05:30
Kotresh HR
96c7963404 qa: Validate file quota attrs on clone subvolume
Fixes: https://tracker.ceph.com/issues/54121
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-02-15 13:40:17 +05:30
Venky Shankar
b7af2a94a4
Merge pull request #42549 from ajarr/wip-add-volume-rename
mgr/volumes: Add `fs volume rename` command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 11:34:32 +05:30
Kotresh HR
7c0d31e52c qa: Add tests snapshot clone failure with quota
Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-01-17 12:14:33 +05:30
Ramana Raja
70697629bf mgr/volumes: Add fs volume rename command
The `fs volume rename` command renames the volume, i.e.,
orchestrator MDS service, file system, and the data and
metadata pool of the file system.

Fixes: https://tracker.ceph.com/issues/51162
Signed-off-by: Ramana Raja <rraja@redhat.com>
2022-01-13 10:36:46 -05:00
Kotresh HR
7440ef842a mgr/volumes: Fix permission during subvol creation with mode
The subvolume creation with specific mode leads to
creation of parent directories ('/volumes/_no_group') with
the same mode if it's not already created. Fixed the same.

Similarly, the subvolumegroup creation with specific mode
leads to creation of parent directory ('/volumes') with
same mode if it's not already created. Fixed the same.

Fixes: https://tracker.ceph.com/issues/51870
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-09-07 15:51:21 +05:30
Kotresh HR
103c7bdc70 mgr/volumes: Fail subvolume removal if it's in progress
Removing an in-progress subvolume clone with force doesn't
remove the clone index (tracker). This results in the cloner
thread to stuck in loop trying to clone the deleted one.

This patch addresses the issue by not allowing the subvolume clone
to be removed if it's not complete/cancelled/failed even with force option.
It throws the error EAGAIN, asking the user to cancel the pending clone
and retry.

Fixes: https://tracker.ceph.com/issues/51707
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-07-30 13:14:28 +05:30
Patrick Donnelly
e2b39f6c8c
qa: avoid using sudo for regular test artifacts
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-01 10:56:10 -07:00
Patrick Donnelly
4fc4ccb7a1
qa: convert mount calls to mount_wait
These tests want to immediately use the mount anyway. But the main
problem is, without waiting for the mount to complete, the command:

    chmod 1777 /path/to/mount

is not run so the mount cannot be written to by normal users without
sudo.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-01 10:56:10 -07:00
Kotresh HR
7588f98505 mgr/volumes: Add config to insert delay at the beginning of the clone
Added the config 'delay_snapshot_clone' to insert delay at the beginning
of the clone to avoid races in tests. The default value is set to 0.

Fixes: https://tracker.ceph.com/issues/48231
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-05-21 12:35:07 +05:30
Ramana Raja
433b70a483 qa/tasks/cephfs: create enough subvolumes
... in a subvolume group so that the desired number of ephmerally pinned
subtrees are created for the subvolume group, which has the distributed
ephemeral pin policy set on it.

Fixes: https://tracker.ceph.com/issues/48411
Signed-off-by: Ramana Raja <rraja@redhat.com>
2021-03-30 16:36:21 -04:00
Kotresh HR
92dc982318 mgr/volumes: Retain suid/guid bits in subvolume clone
Fixes: https://tracker.ceph.com/issues/49882
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-03-19 09:30:04 +05:30
Kotresh HR
818c7781ff mgr/volumes: Bump up AuthMetadataManager's version
With ceph_volume_client and mgr-volumes co-existing
for sometime, the version of both needs to be same.
The ceph_volume_client version <=5 can't decode
'subvolumes' key in auth-metadata file. Hence to
handle version in-compatibility, the version of
ceph_volume_client is bumped up to 6 and the same
needs to be done in mgr-volume's AuthMetadataManager

Fixes: https://tracker.ceph.com/issues/49374
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-02-21 12:59:40 +05:30
Kotresh HR
4952d2132f qa: Fix a few mgr/volume test cases
Recovering dirty auth metadata file might not retain the order,
fixed the comparison in 'test_recover_auth_metadata_during_authorize'
and 'test_recover_auth_metadata_during_deauthorize'.

Fixes: https://tracker.ceph.com/issues/49192
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-02-11 19:22:44 +05:30
Kotresh HR
269adcc8b8 mgr/volumes: Evict clients based on auth-IDs and subvolume mounted
Add subvolume evict command which evicts the subvolume mounts
which are mounted using particular auth-ID.

Fixes: https://tracker.ceph.com/issues/44928
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-01-25 15:53:14 +05:30
Patrick Donnelly
60510411be
Merge PR #38160 into master
* refs/pull/38160/head:
	doc/mgr/volumes: Document 'fs subvolume authorized_list' cli
	qa: Add tests for list auth-ids of a subvolume
	mgr/volumes: Update the 'volumes' key to 'subvolumes' in auth metadata file
	mgr/volumes: Optionally authorize existing auth-ids
	mgr/volumes: Preserve existing caps while authorize/deauthorize auth-id
	mgr/volumes: Disallow authorize existing auth_id
	mgr/volumes: Add subvolume authorized_list command
	mgr/volumes: Add tenant_id option to subvolume authorize
	mgr/volumes: Persist auth and subvolume metadata

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-15 11:53:06 -08:00
Kotresh HR
1b98e63e75 qa: Add tests for list auth-ids of a subvolume
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-01-15 10:10:22 +05:30