Before unmounting check if the client has been evicted and, if so, run
"umount -f -l" for the mount point of the client and cleanup the mount
right after it.
Attempting to unmount, cleanup or operate in any way over mount point
of a evicted client will hang the operation (and thereby our Python
code too). Lazy-force unmount prevents such hangs for our Python code
and also frees the mount point.
This commit also adds code to gather session info for kernel mounts
after mounting is successful. This is a necessity since network address
of session is needed to check if it is blocked by Ceph cluster.
Fixes: https://tracker.ceph.com/issues/56476
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Instead of relying on value of a mutable variable, actually check if the
CephFS is mounted on the system. This will prevent bugs due to stale and
incorrect values.
Fixes: https://tracker.ceph.com/issues/54283
Signed-off-by: Rishabh Dave <ridave@redhat.com>
15 minutes is unnecessarily large as a default value for timeout for a
command. Not having to wait unnecessarily on a crash of a command will
reduce teuthology's testing queue and will save individual developer's
time while running tests locally.
Whatever lines are modified for this purpose are also modified to follow
the stlye guideline, specfically wrapping at 80 characters.
Fixes: https://tracker.ceph.com/issues/54236
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Stop importing CommandFailedError from teuthology.orchestra.run, it is
actually defined in teuthology.exception.
Fixes: https://tracker.ceph.com/issues/51226
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Accepting "cwd" as a parameter in _create_mntpt() methods was a mistake
that happened commit 5e71e9b065.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Make FuseMount.mount() such that it can LocalFuseMount can reuse it
instead of duplicating it. For this, move subtasks required for mounting
Ceph FS (like creating mountpoint directory) to a different method, so
that these methods can be overriden in LocalFuseMount as per
requirement.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The use of chdir will muck up the use of nsenter with valgrind:
2021-03-03T02:13:49.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' --id 0 /home/ubuntu/cephtest/mnt.0
2021-03-03T02:13:49.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse
2021-03-03T02:13:49.914 INFO:teuthology.orchestra.run:Running command with timeout 30
2021-03-03T02:13:49.914 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections
2021-03-03T02:13:49.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi144.stderr:nsenter: failed to execute cd: No such file or directory
It's not necessary to chdir at all to do the mount, so don't.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This method is unused in the teuthology repo. The helper method better
belongs here where it is more easily modified.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/39607/head:
qa/cephfs: check for Ceph FS before mounting
qa/cephfs: remove parameter createfs from mount classes
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
For ceph.py this comes from the ceph.conf.template, but it's not there
for cephadm. Instead of inflicting this on the config inside the
container, just pass it to ceph-fuse incantation here.
Signed-off-by: Sage Weil <sage@newdream.net>
If the "ceph.cluster_fsid" and "ceph.client_id" vxattrs or the
"metric" debug file are not support yet, will assume the test
succeeds.
Fixes: https://tracker.ceph.com/issues/48053
Signed-off-by: Xiubo Li <xiubli@redhat.com>
The mount.cleanup method will remove the mount point. This `rm -rf` will
always fail (with exit status 0).
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Add testsuite for testing authorization on Ceph cluster with multiple
file systems and enable it to be executable with Teuthology framework.
Also add helper methods required to setup the test environment for
multi-FS tests.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This commit adds a new argument check_status to mount methods of
KernelMount, FuseMount, LocalKernelMount and LocalFuseMount. When value
of this argument is False, these methods would catch the
CommandFailedError exception and would return a tuple consisting of the
exception itself, and stdout and stderr of the mount command. This
allows reusing these mount methods while running negative tests for
commands.
The name "check_status" is selected so since teuthology's run() and
vstart_runner's run() use a variable with same name for the very same
purpose.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This commit introduces following two set of changes -
First, make client keyring path, mountpoint on host FS and CephFS and
CephFS's name attributes of the object representing the mount
and update all the mount object creation calls accordingly. Also,
rewrite all the mount object creation to use keyword arguments instead
of positional arguments to avoid mistakes, especially since a new
argument was added in this commit.
Second, add remount method to mount.py so that it's possible to unmount
safely, modify the attributes of the object representing the mount and
mount again based on new state of the object *in a single call*. The
method is placed in mount.py to avoid duplication.
This change has two leads to two more changes: upgrading interface of
mount() and mount_wait() and upgrading testsuites to adapt to these
change.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
fuse_mount.py. This isn't critical at all to vstart_runner.py runs but
this patch must dramatically reduce the time it takes in case the
command fails with "permission denied" due to lack of superuser
privileges since in this case the command is re-run 9 more times, each
separated by a sleep for 5 seconds.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/35664/head:
qa: add omit_sudo=False for commands ran with sudo
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Commands that cleanup FUSE and kernel mount and that setup and
teardown/cleanup network namespaces are that ones that use sudo. Set
omit_sudo to False while running these commands.
Fixes: https://tracker.ceph.com/issues/46101
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/34839/head:
qa/cephfs: add FUSE module before running mount -t fusectl
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Look for self.mountpoint in the contents of /proc/<pid>/cmdline file
when finding asok file for the client so that vstart_runner.py won't end
up picking asok file for a client not created in current run.
This usually never happens so far because PID of newly created processes
is higher than that of previously created processes and list of asok
files returned by "glob.glob(asok_path)" in find_socket() is in
descending order of PIDs.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Also, change timeout from 15 minutes to 30 seconds for the command
"mount -t fusectl xxx xxx" since 15 minutes is too much as per Zheng.
Fixes: https://tracker.ceph.com/issues/45304
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/35062/head:
qa/tasks/cephfs/fuse_mount.py: retry when the admin socket is not ready
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/34951/head:
qa/cephfs: run() cleanup whether FS was mounted or not
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
In case the mount command in mount() fails, it would still have created
the mountpoint and network namespace for the FS's mount. Therefore, run
cleanup() and cleanup_netns() in umount() and umount_wait() even when
self.mounted is set to False.
Also, move the call to cleanup_netns() in cleanup().
Fixes: https://tracker.ceph.com/issues/45430
Signed-off-by: Rishabh Dave <ridave@redhat.com>
INFO:teuthology.orchestra.run.smithi13a2:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2)
INFO:teuthology.orchestra.run.smithi132.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.2': Permission denied
DEBUG:teuthology.orchestra.run:got remote process result: 1
Here just wait and rety for 10 times.
Signed-off-by: Xiubo Li <xiubli@redhat.com>
If the network couldn't response due to some reasons, the 'stat' cmd
will stuck until the network recovery, the best case is it will stuck
forever.
Fixes: https://tracker.ceph.com/issues/44044
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This will isolate the network namespace for each mount point with
a private ip address and iptables, etc.
For the kill() stuff it will just do DOWN the veth interface instead
of sending ipmi request for kernel mount and kill the fuse processes
for the fuse mount. This could avoid sending the socket FIN to the
ceph cluster.
Fixes: https://tracker.ceph.com/issues/44044
Signed-off-by: Xiubo Li <xiubli@redhat.com>
* refs/pull/33636/head:
qa: add upgrade test for volume upgrade from legacy
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
This tests that volumes created using the ceph_volume_client.py library
continue to be accessible/function via the Nautilus/Octopus ceph-mgr
volumes plugin.
Fixes: https://tracker.ceph.com/issues/42723
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/33506/head:
client: add client_fs mount option support
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
"client_fs" is one alias for "client_mds_namespace=" and it will be
cleaner and be more user-friendly to use. "client_mds_namespace="
will be kept and backwards compatibility used.
Update the documents at the same time.
Fixes: https://tracker.ceph.com/issues/44212
Signed-off-by: Xiubo Li <xiubli@redhat.com>
