Before export (and ephemeral) pinned subtrees are stuck in cache
forever.
Add qa test for checking export pinned directories can be trimmed.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Pretty print output once. Use --format=json so the stdout on teuthology
is not pretty printed, taking hundreds of lines.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/35540/head:
qa/cephfs: don't pass cmd args from run_as_user as str
qa/cephfs: refactor run_as_root() to user run_as_user()
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
While volume deletion, the associated pools are not always
removed. The pools are removed only if the volume is created
using mgr plugin and not if created with custom osd pools.
This is because mgr plugin generates pool names with specific
pattern. Both create and delete volume relies on it. This
patch fixes the issue by identifying the pools of the volume
without relying on the pattern.
Fixes: https://tracker.ceph.com/issues/45910
Signed-off-by: Kotresh HR <khiremat@redhat.com>
* refs/pull/35646/head:
qa/tasks/cephfs/nfs: Poll for max 60 seconds to ensure removal of ganesha services
pybind/mgr/volumes/nfs: Remove unused cephfs module
mgr/volumes/nfs: Ignore if ganesha disconnects
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h
If the first character of this file is a pipe symbol (|), then the
remainder of the line is interpreted as the command-line for a
user-space program (or script) that is to be executed. More detail,
please see: http://man7.org/linux/man-pages/man5/core.5.html
Here will just skip cleaning the core dumps in this case.
Fixes: https://tracker.ceph.com/issues/45530
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Commands that cleanup FUSE and kernel mount and that setup and
teardown/cleanup network namespaces are that ones that use sudo. Set
omit_sudo to False while running these commands.
Fixes: https://tracker.ceph.com/issues/46101
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Passing command arguments from run_as_user() to run_shell() as string
can be problematic since command argument to be passed to -c option of
sudo should be a single argument (i.e. 'ls dir' instead of
['ls', 'dir']).
Fixes: https://tracker.ceph.com/issues/46057
Signed-off-by: Rishabh Dave <ridave@redhat.com>
And set colors to False and debug to True in this conf file.
stdout captured by Python code in this testsuite can contain the colour
codes along with the expected string output. Setting colors to False in
cephfs-shell.conf should fix this.
Setting debug to False should help with analyzing test failures since
it enables printing messages that helps debugging (stacktrace for
cephfs-shell code, for example).
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/32288/head:
test_cephfs_shell: remove test_every_shell_cmd_at_invocation
test_cephfs_shell: catch specific exception instead of catching all
cephfs-shell: set return value before exiting
test_cephfs_shell: re-arrange TestCephFSShell's methods
cephfs-shell: re-arrange code under __main__
test_cephfs_shell: add new test for shell cmd at invocation
cephfs-shell: launch shell only if there are no args following
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
* refs/pull/34716/head:
mgr/volumes: Add snapshot info command
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
The reason behind adding and removing this test in different commits of
the same PR is that test_every_shell_cmd_at_invocation is to keep it in
case it's needed later. It reruns all tests in this testsuite by issuing
all cephfs shell commands as arguments to cephfs-shell instead of
issuing them on stdin.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Code for each command has a return value set yet cephfs-shell itself
doesn't exit with proper return value. This makes cephfs-shell quit with
zero return value all the time.
Also, many tests execute cephfs-shell commands expecting the command to
fail but without expecting the method issuing the command to fail. Fix
these test accordingly by adding a new method (negtest_cephfs_shell_cmd)
that expects the command to fail and allows verifying return value and
error messages.
Fixes: https://tracker.ceph.com/issues/44113
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Re-arrange in the order of their usefulness and use keyword arguments
instead of positional arguments to reduce chances of errors, especially
since test_every_cephfs_shell_cmd_at_invocation overrides these methods.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Test passing each CephFS shell command at invocation of shell. The test
replaces run_cephfs_shell_cmd (and other related methods) by it's own
version of the same method that runs each shell command as
"cephfs-shell -c ceph.conf shellcmd" instead of
"cephfs-shell -c ceph.conf -- shellcmd" and then run each
test_cephfs_shell.py.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Code in qa/ uses both StringIO and BytesIO. Let's use StringIO
exclusively (unless necessary) for uniformity. The reason for using
StringIO over BytesIO is that tests mostly need stdout as string than
as bytes and StringIO is used more frequently used in qa/ code at this
point.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/34288/head:
mds: flag backtrace scrub failures for new files as okay
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/34839/head:
qa/cephfs: add FUSE module before running mount -t fusectl
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
* refs/pull/34838/head:
vstart_runner: don't use namespaces by default
qa/cephfs: run nsenter commands with superuser privileges
qa/cephfs: look for mountpoint in cmdline file
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/34782/head:
qa/tasks/cephfs/mount.py: remove netns name parsing in mountpoint setter
qa/tasks/vstart_runner.py: add kwargs parameter to ignore the ones it does not understand
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
New, unwritten files, fail when backtracing during scrub.
This is not necessarily bad. So flag such failures as okay and continue
with other entries.
Fixes: https://tracker.ceph.com/issues/43543
Signed-off-by: Milind Changire <mchangir@redhat.com>
Look for self.mountpoint in the contents of /proc/<pid>/cmdline file
when finding asok file for the client so that vstart_runner.py won't end
up picking asok file for a client not created in current run.
This usually never happens so far because PID of newly created processes
is higher than that of previously created processes and list of asok
files returned by "glob.glob(asok_path)" in find_socket() is in
descending order of PIDs.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
If the ceph-fuse client need to flush the caps and does sync wait,
the umount() will just return successfully, then the netns container
will be destroyed and the network will not be reachable, but the
ceph-fuse daemon is still stucked and waiting for the flush caps ack.
This will cause the ceph-fuse daemon get stuck forever and if the
mds daemons get restarted, it will try to reconnect the clients,
but the stucked ceph-fuse daemnon won't reply to it, because it is
not reachable any more.
Fixes: https://tracker.ceph.com/issues/45665
Signed-off-by: Xiubo Li <xiubli@redhat.com>
1. Add --namespace-isolated option to 'subvolume create' command
to create subvolume in a separate RADOS namespace
2. Add "pool_namespace" field to 'subvolume info' command
which displays the rados namespace if set else empty string
Fixes: https://tracker.ceph.com/issues/45289
Signed-off-by: Kotresh HR <khiremat@redhat.com>
After subvolume is created, it can be resized
using subvolume create command. But it was
broken and the same is fixed.
Fixes: https://tracker.ceph.com/issues/45398
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Also, change timeout from 15 minutes to 30 seconds for the command
"mount -t fusectl xxx xxx" since 15 minutes is too much as per Zheng.
Fixes: https://tracker.ceph.com/issues/45304
Signed-off-by: Rishabh Dave <ridave@redhat.com>
cephfs-shell options should reside in cephfs-shell.conf and not in
ceph.conf. Please note, unlike before, -c option of cephfs-shell must
take path to cephfs-shell.conf instead of path to ceph.conf.
This commit also updates the docs and the tests for cephfs-shell accordingly
and renames the variable config_path in cephfs-shell to shell_conf_path so
that it's easy to distinguish.
Fixes: https://tracker.ceph.com/issues/44127
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/35062/head:
qa/tasks/cephfs/fuse_mount.py: retry when the admin socket is not ready
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/34951/head:
qa/cephfs: run() cleanup whether FS was mounted or not
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
We don't really care if this is left on during the course of the mount.
These settings don't persist across tests anyway.
Fixes: https://tracker.ceph.com/issues/45525
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
In case the mount command in mount() fails, it would still have created
the mountpoint and network namespace for the FS's mount. Therefore, run
cleanup() and cleanup_netns() in umount() and umount_wait() even when
self.mounted is set to False.
Also, move the call to cleanup_netns() in cleanup().
Fixes: https://tracker.ceph.com/issues/45430
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The following command is added
"ceph fs subvolume snapshot info <vol_name> <sub_name> <snap_name> [<group_name>]"
The output is in json format with following fields
created_at: time of creation of snapshot in the format "YYYY-MM-DD HH:MM:SS:ffffff"
data_pool: data pool the snapshot belongs to
has_pending_clones: "yes" if snapshot clone is in progress otherwise "no"
protected: "yes" if snapshot is protected otherwise "no"
size: snapshot size in bytes
Fixes: https://tracker.ceph.com/issues/45237
Signed-off-by: Kotresh HR <khiremat@redhat.com>
* refs/pull/34978/head:
qa/tasks/cephfs/mount.py: always setup the NAT rules
qa/tasks/cephfs/mount.py: fall back to use ip command to setup the bridge
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
If the last test failed just after the 'ceph-brx' bridge is created
but with setuping the NAT rules in iptables, so if we run the test
case again, it will just skip seting the 'ceph-brx' and the NAT rules.
Then deleting the NAT rules will get the errors like:
iptables: Bad rule (does a matching rule exist in that chain?).
Signed-off-by: Xiubo Li <xiubli@redhat.com>
CentOS/RHEL8 have abandoned the bridge-utils package so the brctl
cmd is none avalible, and on Ubuntu 18.04 and 20.04 the nmcli is
buggy to setup the bridge. So this will fall back to use ip cmd
to setup the bridge stuff.
Fixes: https://tracker.ceph.com/issues/45459
Signed-off-by: Xiubo Li <xiubli@redhat.com>
self.mount_a.client_remote.sh() returns an 'str' object rather than a StringIO object. Hence the p.stdout.getvalue() produces an error. This commit fixes this and also fix str and byte mismatch as byte and string were the same object in Python2 but this is not the case in Python3.
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
The refactor intends to simplify the test and make it more readable,
It also changes how this test creates directory -- tempfile.mkdtemp is
used instead of coreutils mkdir to avoid bug reported here -
https://tracker.ceph.com/issues/43567.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
INFO:teuthology.orchestra.run.smithi13a2:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2)
INFO:teuthology.orchestra.run.smithi132.stderr:chmod: changing permissions of '/home/ubuntu/cephtest/mnt.2': Permission denied
DEBUG:teuthology.orchestra.run:got remote process result: 1
Here just wait and rety for 10 times.
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Fix "Error EINVAL: invalid size option '4915200.0'".
In pytho2 int/2 will get a int type result, but in python3 it maybe
a float type.
Fixes: https://tracker.ceph.com/issues/45247
Signed-off-by: Xiubo Li <xiubli@redhat.com>
The pybind now has dropped the WITH_PYTHON2 option, and for now only
py3 supported.
Fixes: https://tracker.ceph.com/issues/45103
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Methods like run_shell effectively conduct positive test on the given
command. Add methods that runs given command expecting failure and then
verifies return value and error message with given one. Rewrite testcmd,
testcmd_as_user and testcmd_as_root to create these new methods for
negative testing since run_shell, run_as_user and run_as_root is
equivalent of running positive test.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
LocalFuseMount and LocalKernelMount can directly inherit these methods
from CephFSMount via FuseMount and KernelMount respectively. Moving
would avoid duplication and would make these methods more accessible
for reusing via inheritance.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
"ceph fs status" json format outputs to stderr instead of
stdout. This patch fixes the same.
Fixes: https://tracker.ceph.com/issues/44962
Signed-off-by: Kotresh HR <khiremat@redhat.com>
After creating a filesystem using the 'fs new' command, the value
of the 'data' and 'metadata' key of the datapool and metadatapool's
application tag 'cephfs' should be the filesystem's name. This
didn't happen when the data or metadata pool's application metadata
'cephfs' was enabled before the pool was used in the 'fs new' command.
Fix this during the handling of the 'fs new' command by setting the
value of the key of the pool's application metadata 'cephfs' to the
filesystem's name even when the application metadata 'cephfs' is
already enabled or set.
Fixes: https://tracker.ceph.com/issues/43761
Signed-off-by: Ramana Raja <rraja@redhat.com>
After making a RADOS pool a filesystem's data pool using the
'add_data_pool' command, the value of the 'data' key of the pool's
application metadata 'cephfs' should be the filesystem's name. This
didn't happen when the pool's application metadata 'cephfs' was
enabled before the pool was made the data pool. Fix this during the
handling of the 'add_data_pool' command by setting the value of
the 'data' key of the pool's application metadata 'cephfs' to the
filesystem's name even when the application metadata 'cephfs' is
already enabled or set.
Fixes: https://tracker.ceph.com/issues/43061
Signed-off-by: Ramana Raja <rraja@redhat.com>
The boolean self.mounted parameter is conflicted with mounted() helper,
so we will hit error like:
INFO:tasks.cephfs_test_runner:TypeError: 'bool' object is not callable
Fixes: https://tracker.ceph.com/issues/44044
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Mostly we should wait the mountpoint to get ready, especially for
the fuse mountpoint, sometimes it may take a few seconds to get
ready.
Fixes: https://tracker.ceph.com/issues/44044
Signed-off-by: Xiubo Li <xiubli@redhat.com>
