Some of the tests in Rados.sh can fail when trying to test
watch_list return size if we hit watch timeout.
increase the watch timeout for rados test
Fixes: https://tracker.ceph.com/issues/47025
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
The Windows rbd-wnbd python test performs various IO operations
against raw disks.
However, it can be useful to test overlaying filesystems as well.
For this reason, we're adding the following tests:
* RbdFsTest
* RbdFsFioTest
* RbdFsStampFioTest
To simplify the implementation, those tests reuse the existing
ones along with a mixin class (RbdFsTestMixin).
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
We'll make the following improvements to the Windows rbd-wnbd
Python test:
* expose fio write validation, defaulting to crc32c
* change the default fio operation to "rw"
* enable the disk and clear the "rw" flag only if required by the
test and if "--skip-enabling-disk" is not set (useful with custom
SAN policies). This operation can take a significant amount of
time under heavy load.
* print fio read and write results separately instead of
aggregating them, useful when running rw tests
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
marginal suite was probably used sometime back and seems obsolete
now. Remove it and its corresponding restart task.
Signed-off-by: Venky Shankar <vshankar@redhat.com>
The old way of using the git archive doesn't seem
to work anymore so trying another approach
Was getting "fatal: operation not supported by protocol"
when trying the git archive
Fixes: https://tracker.ceph.com/issues/58222
Signed-off-by: Adam King <adking@redhat.com>
* refs/pull/45312/head:
qa: new snapshot test for snapshot naming limits
mds: limit the snapshot names to 240 characters
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Since the git:// is not reachable any more and have switch to
https://.
The git archive does not support the https protocol, so we couldn't
user the git archive to retrieve the tar ball any more, will split
this into 3 steps:
1, clone the whole ceph repo
2, checkout the commit/tag/branch
3, then change directory to qa/workunits/.
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Set osd_mclock_override_recovery_settings option to true for tests that
modify recovery/backfill configuration options. This prevents logging of
the cluster warning when modifying recovery/backfill limits.
Fixes: https://tracker.ceph.com/issues/57529
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
- Consolidate all mclock standalone tests under
qa/standalone/misc/mclock-config.sh.
- Revert existing tests in ceph-helpers.sh that verified the earlier hard
override of recovery/backfill limits.
- Add new tests to verify the procedure to change the recovery/backfill
limits with mclock scheduler.
Fixes: https://tracker.ceph.com/issues/57529
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
Problem:
--mon-initial-members does nothing but causes monmap
to populate ``removed_ranks`` because the way we start
monitors in standalone tests uses ``run_mon $dir $id ..``
on each mon. Regardless of --mon-initial-members=a,b,c, if
we set --mon-host=$MONA,$MONB,$MONC (which we do every single tests),
everytime we run a monitor (e.g.,run mon.b) it will pre-build
our monmap with
```
noname-a=mon.noname-a addrs v2:127.0.0.1:7127/0,
b=mon.b addrs v2:127.0.0.1:7128/0,
noname-c=mon.noname-c addrs v2:127.0.0.1:7129/0,
```
Now, with --mon-initial-members=a,b,c we are letting
monmap know that we should have initial members name:
a,b,c, which we only have `b` as a match. So what
``MonMap::set_initial_members`` do is that it will
remove noname-a and noname-c which will
populate `removed_ranks`.
Solution:
remove all instances of --mon-initial-members
in the standalone test as it has no impact on
the nature of the tests themselves.
Fixes: https://tracker.ceph.com/issues/58132
Signed-off-by: Kamoltat <ksirivad@redhat.com>
py3 tries to import all python modules to ensure that they are
python3 compatible. but the installation fails on jenkins test node:
Resolved https://github.com/ceph/teuthology.git to commit 4da97cf64e542f347ec47b7bdbe5eca99759f9b7
Installing build dependencies: started
error: subprocess-exited-with-error
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
otherwise it is mising when running test, and we'd have following
failure:
py3: exit 2 (0.00 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/qa> pytest --assert=plain test_import.py
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
as ansible is using UTF-8 encoded characters in the file names, so,
to avoid failures like:
File "/home/jenkins-build/build/workspace/ceph-pull-requests/qa/.tox/py3/lib/python3.10/site-packages/pip/_internal/utils/unpacking.py", line 217, in untar_file
with open(path, "wb") as destfp:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 137-140: ordinal not in range(256)
we have to set a locale which is able to handle UTF-8.
see also https://github.com/ceph/teuthology/pull/1671
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Note that we are hitting https://tracker.ceph.com/issues/58160 here
because by the time we get to "rbd resize" RAW_DEV mapping owns the
lock (due to a write to /dev/mapper/cryptsetupdev being last).
While at it, resurrect the ability to easily run this script on
vstart clusters -- see commit f737c2855a19 ("qa/workunits/rbd: make
luks-encryption test work on vstart cluster").
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
If no --encryption-format specified at all, default to "luks" for each
specified --encryption-passphrase-file.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Since RBD_ENCRYPTION_FORMAT_LUKS1, RBD_ENCRYPTION_FORMAT_LUKS2
and RBD_ENCRYPTION_FORMAT_LUKS aren't treated the same when loading
encryption anymore, "luks1" and "luks2" formats need to be accepted
in addition to "luks" format.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
One of the stated goals is compatibility with standard LUKS tools,
in particular being able to load encryption on images formatted with
cryptsetup. cryptsetup doesn't do this and this really interferes
with randomly generated (binary) passphrases.
While at it, open passphrase files as binary -- it communicates the
intent if nothing else on POSIX.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Commit e0da2a4e8cae ("qa/workunits/rbd: Add test to list snapshots of
consistency group") added bash-specific syntax.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Fixes the usage of the grep command. It was failing as below.
smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep '"clone' 'snaps"' \
-A3 ./volumes/_nogroup/subvolume_0000000000856603/.meta)
INFO:teuthology.orchestra.run.smithi008.stderr:grep: snaps": No such file or directory
Fixes: https://tracker.ceph.com/issues/57446
Signed-off-by: Kotresh HR <khiremat@redhat.com>
For both io path and common test cases in xfstests. Currently only
support the centos and ubuntu distros. But for rhel we couldn't
install the 'userspace-rcu-devel', 'device-mapper-devel' and
'libedit-devel' dependency packages.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This will test the common fscrypt features such as the set_encpolicy,
get_encpolicy,etc only.
Will run the IO path test in another test suite.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This will allow us to create a encrypted test directory to do the
I/O test.
More coplex fscrypt tests we will use the xfstest-dev later.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
We need to install the xfsprogs-dev from source to make the xfs_io
command to support the 'add_enckey'.
Currently we couldn't install 'userspace-rcu-devel', 'libedit-devel'
and 'device-mapper-devel' from rhel distos. So the file encyrpt test
cases will skip the rhel distos for now. And it works well on centos
and ubuntu.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Currently the xfsprogs packages won't support the 'add_enckey' feature
in xfs_io command, we need to install it from source.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Both the test_dev and scratch_dev could use the CEPHFS_MOUNT_OPTIONS.
Without this the xfstests tests will just be aborted without any error
logs when testing the encrypt and it's very hard to debug.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Since the git protocol is not reachable any more, just switch it
to https.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
to silence the warning like
```
flake8 run-test: commands[0] | flake8 --select=F,E9 --exclude=venv,.tox
./tasks/cephfs/test_full.py:6:5: F401 'typing.Optional' imported but unused
```
since we've migrated to python3.6 and up, there is no need to be
compatible with Python 2 anymore.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rados.ObjectNotFound exception handler was referencing ioctx variable
which is assigned only if the pool exists and rados.open_ioctx() call
succeeds. This lead to a fatal error
mgr[rbd_support] Failed to locate pool mypool
mgr[rbd_support] execute_task: [errno 2] error opening pool 'b'mypool''
mgr[rbd_support] Fatal runtime error: local variable 'ioctx' referenced before assignment
and wedged the task queue. No other commands were processed until
ceph-mgr daemon restart.
Fixes: https://tracker.ceph.com/issues/52932
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Both these two cephfs tests are using 1-mds-2-client-micro.yaml:
roles:
- [mon.a, mon.b, mon.c, mgr.x, mgr.y, mds.a, mds.b, mds.c, osd.0, osd.1, osd.2, osd.3]
- [client.0]
- [client.1]
So all the mon/mgr/mds/osd daemons are running in the first single
node. It makes no sense to upgrade the same node twice.
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This workunit currently grabs files in html form, since
there is a redirect that occurs. If we specify `-L` in the
curl command, it handles redirects and graps the files
appropriately.
Fixes: https://tracker.ceph.com/issues/58046
Signed-off-by: Laura Flores <lflores@redhat.com>
It's an accident that some of these pass -- the tests provide some
coverage for inconsistencies that can arise, but don't really validate
that the objects actually get moved between tiers.
It's going to be some time before we implement cache or dedup in
crimson, and we'll probably want to disable the related commands for
pools that can map to crimson osds to prevent accidents.
Signed-off-by: Samuel Just <sjust@redhat.com>
Let's use crimson-rados-experimental for tests that don't yet
pass reliably. We can move these to crimson-rados as they
become reliable.
Signed-off-by: Samuel Just <sjust@redhat.com>
Reef cycle will not allow upgrades from octopus. However, the featureful
client tests still needs to be testes, therefore, upgrade to quincy (from
octopus) rather to the current cycle (reef).
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Tried to edit the permission of a subuser but once changed the permission, edited permission not seen in user info of UI(ceph dashboard) and from CLI as well
Fixes:https://tracker.ceph.com/issues/57805
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
qa: the default CEPH_ASOK_DIR is asok/ now
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
qa: make mount to be compatible with nautilus for blocklist
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
now that we officially have v18 in main the upgrade
will fail if upgrading from Octopus (v15) to main
because they are now considered 3 major releases apart
Additionally adding more debugging to the end
of the upgrade tests as something like `ceph health detail`
can be very useful to see at the end of failed
upgrade tests
Fixes: https://tracker.ceph.com/issues/57695
Signed-off-by: Adam King <adking@redhat.com>
fio 3.16 doesn't build against recent kernel headers due to removal of
linux/raw.h.
Fixes: https://tracker.ceph.com/issues/57766
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
The OS Type and OS Version for mixed-clients tests is left blank on teuthology and they appear to be trying to
run ubuntu jammy which is causing failures. We should specify what distros the tests should run on.
Fixes: https://tracker.ceph.com/issues/57280
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
Wait for scrub to finish during test_scrub_pause_and_resume_with_abort
which otherwise races and fails with an incorrect assertion.
Fixes: https://tracker.ceph.com/issues/48812
Signed-off-by: Milind Changire <mchangir@redhat.com>
Idea here is to force the MDS to flush metadata mutations out of the
journal. This may help expose any type of corruption seen with postgres.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
To exercise snapshot creation and deletion during workloads.
Fixes: https://tracker.ceph.com/issues/23724
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
So we can enforce an ordering of tasks and ensure that the fs/subvolume
is available before the task runs.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
test_client_recovery was also using mntopts to specify additional
options to ceph-fuse. Because the two prior commits unify the behavior
of ceph-fuse and the kernel mount so that the "-o" option is available
for both, that changes breaks this test. Add a special set of args
available only for fuse (there is no equivalent on the kernel).
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Also, fix the type default to None. A list does not work in the code but
the default is not used in QA.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Only fuse would support the "mountpoint" yaml config. Move it to
mount.py so it works consistently with the kernel mount. Ditto for the
cephfs mount point.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Since the default of None gets interpreted as 'lvm', the only useful
value would be 'raw'. This will enable us to use loop devices inside
containers for OSDs.
Signed-off-by: Zack Cerza <zack@redhat.com>
The test_orphan_scan test deliberately removes a dentry which will cause
rstat damage. Ignore it.
Fixes: https://tracker.ceph.com/issues/57657
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
In newer versions the policies is inside the code
so we don't need these files for default policy
anymore.
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
This adds a QA suite for the service token auth in
RGW. The suite runs a workunit that is a bash script
that spawns a fake Keystone server which is then used
as an auth backend for RGW to test the feature.
A python script is then executed that runs a battery
of tests against RadosGW which talks to the fake Keystone
server running in the background.
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
