py3 tries to import all python modules to ensure that they are
python3 compatible. but the installation fails on jenkins test node:
Resolved https://github.com/ceph/teuthology.git to commit 4da97cf64e542f347ec47b7bdbe5eca99759f9b7
Installing build dependencies: started
error: subprocess-exited-with-error
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
otherwise it is mising when running test, and we'd have following
failure:
py3: exit 2 (0.00 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/qa> pytest --assert=plain test_import.py
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
as ansible is using UTF-8 encoded characters in the file names, so,
to avoid failures like:
File "/home/jenkins-build/build/workspace/ceph-pull-requests/qa/.tox/py3/lib/python3.10/site-packages/pip/_internal/utils/unpacking.py", line 217, in untar_file
with open(path, "wb") as destfp:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 137-140: ordinal not in range(256)
we have to set a locale which is able to handle UTF-8.
see also https://github.com/ceph/teuthology/pull/1671
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Note that we are hitting https://tracker.ceph.com/issues/58160 here
because by the time we get to "rbd resize" RAW_DEV mapping owns the
lock (due to a write to /dev/mapper/cryptsetupdev being last).
While at it, resurrect the ability to easily run this script on
vstart clusters -- see commit f737c2855a ("qa/workunits/rbd: make
luks-encryption test work on vstart cluster").
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
If no --encryption-format specified at all, default to "luks" for each
specified --encryption-passphrase-file.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Since RBD_ENCRYPTION_FORMAT_LUKS1, RBD_ENCRYPTION_FORMAT_LUKS2
and RBD_ENCRYPTION_FORMAT_LUKS aren't treated the same when loading
encryption anymore, "luks1" and "luks2" formats need to be accepted
in addition to "luks" format.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
One of the stated goals is compatibility with standard LUKS tools,
in particular being able to load encryption on images formatted with
cryptsetup. cryptsetup doesn't do this and this really interferes
with randomly generated (binary) passphrases.
While at it, open passphrase files as binary -- it communicates the
intent if nothing else on POSIX.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Commit e0da2a4e8c ("qa/workunits/rbd: Add test to list snapshots of
consistency group") added bash-specific syntax.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
For both io path and common test cases in xfstests. Currently only
support the centos and ubuntu distros. But for rhel we couldn't
install the 'userspace-rcu-devel', 'device-mapper-devel' and
'libedit-devel' dependency packages.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This will test the common fscrypt features such as the set_encpolicy,
get_encpolicy,etc only.
Will run the IO path test in another test suite.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
This will allow us to create a encrypted test directory to do the
I/O test.
More coplex fscrypt tests we will use the xfstest-dev later.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
We need to install the xfsprogs-dev from source to make the xfs_io
command to support the 'add_enckey'.
Currently we couldn't install 'userspace-rcu-devel', 'libedit-devel'
and 'device-mapper-devel' from rhel distos. So the file encyrpt test
cases will skip the rhel distos for now. And it works well on centos
and ubuntu.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Currently the xfsprogs packages won't support the 'add_enckey' feature
in xfs_io command, we need to install it from source.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Both the test_dev and scratch_dev could use the CEPHFS_MOUNT_OPTIONS.
Without this the xfstests tests will just be aborted without any error
logs when testing the encrypt and it's very hard to debug.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Since the git protocol is not reachable any more, just switch it
to https.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
to silence the warning like
```
flake8 run-test: commands[0] | flake8 --select=F,E9 --exclude=venv,.tox
./tasks/cephfs/test_full.py:6:5: F401 'typing.Optional' imported but unused
```
since we've migrated to python3.6 and up, there is no need to be
compatible with Python 2 anymore.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
rados.ObjectNotFound exception handler was referencing ioctx variable
which is assigned only if the pool exists and rados.open_ioctx() call
succeeds. This lead to a fatal error
mgr[rbd_support] Failed to locate pool mypool
mgr[rbd_support] execute_task: [errno 2] error opening pool 'b'mypool''
mgr[rbd_support] Fatal runtime error: local variable 'ioctx' referenced before assignment
and wedged the task queue. No other commands were processed until
ceph-mgr daemon restart.
Fixes: https://tracker.ceph.com/issues/52932
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This workunit currently grabs files in html form, since
there is a redirect that occurs. If we specify `-L` in the
curl command, it handles redirects and graps the files
appropriately.
Fixes: https://tracker.ceph.com/issues/58046
Signed-off-by: Laura Flores <lflores@redhat.com>
It's an accident that some of these pass -- the tests provide some
coverage for inconsistencies that can arise, but don't really validate
that the objects actually get moved between tiers.
It's going to be some time before we implement cache or dedup in
crimson, and we'll probably want to disable the related commands for
pools that can map to crimson osds to prevent accidents.
Signed-off-by: Samuel Just <sjust@redhat.com>
Let's use crimson-rados-experimental for tests that don't yet
pass reliably. We can move these to crimson-rados as they
become reliable.
Signed-off-by: Samuel Just <sjust@redhat.com>
Reef cycle will not allow upgrades from octopus. However, the featureful
client tests still needs to be testes, therefore, upgrade to quincy (from
octopus) rather to the current cycle (reef).
Signed-off-by: Venky Shankar <vshankar@redhat.com>
