Otherwise the setting doesn't take effect. While at it, replace
home-grown stringify() with standard to_string().
Fixes: https://tracker.ceph.com/issues/58833
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
At the moment, the latency results are reported in nanoseconds.
In order to improve readability, we'll convert it to seconds.
While at it, we'll fix the fio duration report, which we're
wrongfully dividing by 1000 twice.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Added data regarding completion latency (ns) to the fio results
table, such as min, max, mean, stddev, etc.
Signed-off-by: Stefan Chivu <schivu@cloudbasesolutions.com>
"rbd feature disable" appears to reliably hang if the corresponding
remote request is proxied to rbd-nbd (because rbd-nbd happens to own
the exclusive lock after a series of blkdiscard calls) [1]. Work
around it here by enabling journaling before the image is mapped
and disabling it after the image is unmapped.
Also, don't assert on the output of "rbd journal inspect --verbose"
having a certain number of entries. This is racy: if the script gets
delayed after the last blkdiscard call for some reason, there may be
fewer entries present in the journal or none at all.
[1] https://tracker.ceph.com/issues/58740
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
The existing
xmlstarlet sel -t -v '//mirror/peers/peer[1]/uuid')" = ""
test is bogus since a tx-only peer gets added after the remote
rbd-mirror daemon pings the local cluster. It happened to pass most
of the time because xmlstarlet filter just failed on an empty peers
array, producing the wrongly expected empty string by accident.
Fixes: https://tracker.ceph.com/issues/58688
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Dedup_tier option needs to be reset whenever every test begins,
because dedup_tier(dedup_chunk_pool) is deleted at the end of each
test---if not, this causes error in case dedup_tier's pool id is
changed.
This commit adds a set command to refresh newly added dedup_tier
pool id.
fixes: https://tracker.ceph.com/issues/58587
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Previously, we only allotted 2 seconds for the telemetry
opt-in status to disappear. However, it sometimes takes longer
than this for the ceph status to update.
Here, we give the status up to 60 seconds to update.
Fixes: https://tracker.ceph.com/issues/58545
Signed-off-by: Laura Flores <lflores@redhat.com>
This commit fixes commit 7ca1bab90f by pushing properly aligned
discards back to m_image_extents, if corrected.
If discards are misaligned (off 0, len 4608, gran=4096), they are
corrected properly, but only in object_extents and not in
m_image_extents.
When journal_append_event is triggered it will only append from
m_image_extents and does not now about the alignment fixes. In
commit_io_events_extent it will log a message and return without
completing the io since the larger misaligned area was sent to the journal.
This will in turn break rbd journal mirroring since the local client will wait
indefinately on the commit to be completed, which it never does.
This does not effect rbd-mirror in any way, which may be confusing and
dangerous since it's only rbd-mirror that updates ceph health, and not
the local client.
Setting `rbd_skip_partial_discard = false` under client will restore the
pre 7ca1bab behaviour and thus not trigger the bug with journals growing.
This will set `rbd_discard_granularity_bytes = 0` internally. This
setting is only changed during startup of a client.
Fixes: 7ca1bab90f
Fixes: https://tracker.ceph.com/issues/57396
Signed-off-by: Josef Johansson <josef@oderland.se>
We have to iterate in chunks otherwise the get_omap_vals API returns
partial results even if the max count is large.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
We have a few Python rbd-wnbd tests that are invoked explicitly
by the ceph-build scripts [1].
There are a few issues with that:
* it's a separate repo that has to be updated whenever we add new
tests
* new tests that reside in the ceph repo will not be executed by
the PR check
* some tests may be missing in case of older branches
For this reason, we're adding a new script as part of the Ceph
repo that will take care of invoking the Windows rbd-wnbd tests.
The ceph-build script has already been updated accordingly [2].
[1] https://github.com/ceph/ceph-build/blob/main/scripts/ceph-windows/run_tests#L73-L80
[2] https://github.com/ceph/ceph-build/pull/2094
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-Authored-By: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
We're adding a test for the newly introduced live resize feature.
It will simply extend/shrink the image, wait for the new size to
be picked up and then run FIO tests to validate the resized image.
While at it, we're fixing two unrelated linter warnings:
E275 missing whitespace after keyword
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
first==CEPH_NOSNAP is never valid. Apparently somehow the MDS may set
this in some code path yet to be discovered, even without snapshots.
Note, we probably should not necessarily automate this fix via the MDS
because we do not know if this dentry (visible in HEAD) should also be
visible before NEXT_SNAP.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This test uses certain PS commands that attempt to display
a progress bar. However, this can cause issues when invoked
remotely (e.g. by the jenkins job).
For this reason, we're defining a helper (ps_execute) that runs
PS commands, disabling the progress bars and enabling the non
interactive mode.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Certain FS related operations can fail, especially under load
(e.g. initializing partitions, volume formatting, etc).
For this reason, we're going to introduce some retries.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
The following operations may fail right after a block device
is attached:
* retrieving the disk number (can return -1)
* opening the disk
* setting the disk online or writable
For this reason, we'll need to add some retries. For convenience,
we're moving the existing retry logic to a separate decorator.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Instead of trying to use the first partiton which may be reserved
by Windows, we'll fetch the first non-empty drive letter from
the disk that we've just mounted.
While at it, we're ensuring that the drive letter is actually a
letter and not a null character, which the Powershell command
returns in case of empty drive letters.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
The Windows rbd-wnbd python test performs various IO operations
against raw disks.
However, it can be useful to test overlaying filesystems as well.
For this reason, we're adding the following tests:
* RbdFsTest
* RbdFsFioTest
* RbdFsStampFioTest
To simplify the implementation, those tests reuse the existing
ones along with a mixin class (RbdFsTestMixin).
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
We'll make the following improvements to the Windows rbd-wnbd
Python test:
* expose fio write validation, defaulting to crc32c
* change the default fio operation to "rw"
* enable the disk and clear the "rw" flag only if required by the
test and if "--skip-enabling-disk" is not set (useful with custom
SAN policies). This operation can take a significant amount of
time under heavy load.
* print fio read and write results separately instead of
aggregating them, useful when running rw tests
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
* refs/pull/45312/head:
qa: new snapshot test for snapshot naming limits
mds: limit the snapshot names to 240 characters
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Since the git:// is not reachable any more and have switch to
https://.
The git archive does not support the https protocol, so we couldn't
user the git archive to retrieve the tar ball any more, will split
this into 3 steps:
1, clone the whole ceph repo
2, checkout the commit/tag/branch
3, then change directory to qa/workunits/.
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Note that we are hitting https://tracker.ceph.com/issues/58160 here
because by the time we get to "rbd resize" RAW_DEV mapping owns the
lock (due to a write to /dev/mapper/cryptsetupdev being last).
While at it, resurrect the ability to easily run this script on
vstart clusters -- see commit f737c2855a ("qa/workunits/rbd: make
luks-encryption test work on vstart cluster").
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
If no --encryption-format specified at all, default to "luks" for each
specified --encryption-passphrase-file.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Since RBD_ENCRYPTION_FORMAT_LUKS1, RBD_ENCRYPTION_FORMAT_LUKS2
and RBD_ENCRYPTION_FORMAT_LUKS aren't treated the same when loading
encryption anymore, "luks1" and "luks2" formats need to be accepted
in addition to "luks" format.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
One of the stated goals is compatibility with standard LUKS tools,
in particular being able to load encryption on images formatted with
cryptsetup. cryptsetup doesn't do this and this really interferes
with randomly generated (binary) passphrases.
While at it, open passphrase files as binary -- it communicates the
intent if nothing else on POSIX.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Commit e0da2a4e8c ("qa/workunits/rbd: Add test to list snapshots of
consistency group") added bash-specific syntax.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This will allow us to create a encrypted test directory to do the
I/O test.
More coplex fscrypt tests we will use the xfstest-dev later.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Since the git protocol is not reachable any more, just switch it
to https.
Fixes: https://tracker.ceph.com/issues/58133
Signed-off-by: Xiubo Li <xiubli@redhat.com>
rados.ObjectNotFound exception handler was referencing ioctx variable
which is assigned only if the pool exists and rados.open_ioctx() call
succeeds. This lead to a fatal error
mgr[rbd_support] Failed to locate pool mypool
mgr[rbd_support] execute_task: [errno 2] error opening pool 'b'mypool''
mgr[rbd_support] Fatal runtime error: local variable 'ioctx' referenced before assignment
and wedged the task queue. No other commands were processed until
ceph-mgr daemon restart.
Fixes: https://tracker.ceph.com/issues/52932
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This workunit currently grabs files in html form, since
there is a redirect that occurs. If we specify `-L` in the
curl command, it handles redirects and graps the files
appropriately.
Fixes: https://tracker.ceph.com/issues/58046
Signed-off-by: Laura Flores <lflores@redhat.com>
