This commits extends rbd API's to allow the user to load encryption
using either LUKS1 or LUKS2 formats, without explicitly specifying which.
Prior, the user had to specify explicitly LUKS1 or LUKS2.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
This commit fixes 2 issues in luks-encryption.sh:
1. Fix sporadic comparison failures due to stale data read from kernel buffer cache.
2. Fix test skipping condition (when journaling is enabled)
Additionally, sudo is removed from most commands.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Commit 96f05a7956 ("rbd: delay determination of default pool name")
broke "rbd perf image iostat" and "rbd perf image iotop" GLOBAL_POOL_KEY
support (the ability to blend all rbd pools together into a single
view).
Fixes: https://tracker.ceph.com/issues/56561
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Establishing a watch on rbd_mirroring object and skipping rescanning
image mirror snapshots on periodic refresh unless rbd_mirroring object
gets notified in the interim is flawed. rbd_mirroring object is
notified when mirroring is enabled or disabled on some image (including
when the image is removed), but it is not notified when images are
promoted or demoted. However, load_pool_images() discards images that
are not primary at the time of the scan. If the image is promoted
later, no snapshots are created even if the schedule is in place. This
happens regardless of whether the schedule is added before or after the
promotion.
This effectively reverts commit 69259c8d37 ("mgr/rbd_support: make
mirror_snapshot_schedule rescan only updated pools"). An alternative
fix could be to stop discarding non-primary images (i.e. drop
if not info['primary']:
continue
check added in commit d39eb283c5 ("mgr/rbd_support: mirror snapshot
schedule should skip non-primary images")), but that would clutter the
queue and therefore "rbd mirror snapshot schedule status" output with
bogus entries. Performing a rescan roughly every 60 seconds should be
manageable: currently it amounts to a single mirror_image_status_list
request, followed by mirror_image_get, get_snapcontext and snapshot_get
requests for each snapshot-based mirroring enabled image and concluded
by a single dir_list request. Among these, per-image get_snapcontext
and snapshot_get requests are necessary for determining primaryness.
Fixes: https://tracker.ceph.com/issues/53914
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Commit 08df6e0fd0 ("qa/workunits/rbd: expand LevelSpec parsing
coverage") didn't account for images with a separate data pool. This
was missed because of small-cache-pool.yaml breakage.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Invoke "rbd mirror snapshot schedule ls -R" and "rbd mirror snapshot
schedule status" commands on all levels, consistently. In particular,
make sure that an image level schedule is listed for a recursive query
at the pool level both before and after the schedule kicks in:
$ rbd create --size 1G --mirror-image-mode snapshot -p foo bar
$ rbd mirror snapshot schedule add -p foo --image bar 1m
$ rbd mirror snapshot schedule ls -p foo -R
POOL NAMESPACE IMAGE SCHEDULE
foo bar every 1m
<wait for schedule to become visible in status>
$ rbd mirror snapshot schedule ls -p foo -R
POOL NAMESPACE IMAGE SCHEDULE
foo bar every 1m
Also, make sure that pool and image level status queries work:
$ rbd mirror snapshot schedule status -p foo
SCHEDULE TIME IMAGE
2022-03-04 07:14:00 foo/bar
$ rbd mirror snapshot schedule status -p foo --image bar
SCHEDULE TIME IMAGE
2022-03-04 07:14:00 foo/bar
Both of these issues are fixed by the previous commit.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Commit fea6fdff4c ("mgr/rbd_support: level_spec passed to some
commands is not optional") is wrong. While it is true that a valid
level_spec is needed to create a LevelSpec instance, an empty string
is very much a valid level spec -- it signifies "all levels".
This wasn't caught because within Ceph these commands are wrapped by
rbd CLI which injects an empty string in get_level_spec_args().
Fixes: https://tracker.ceph.com/issues/54058
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
In the LoadRequest in the ImageMap class add initial cleanup to remove
stale entries. To cleanup the LoadRequest will query the mirror image
list and remove all the image_map that are notin the list.
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
This make sure that all images are deleted in the existing qa scripts
and checks if all rbd-mirror metadata in OMAP are correctly deleted.
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
As there is no inherent ordering, there may be multiple removable
images past the unremovable image. On top of that, removing a clone
may make its parent removable so perform an additional pass if any
image gets removed.
Fixes: https://tracker.ceph.com/issues/51021
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This patch includes twe new test cases:
a. map/unmap test with only image name and
b. map/unmap test after changing default pool which expects the image
to come from new default pool.
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
It doesn't work on Focal and already disabled on CentOS 7 and 8. More
importantly, it doesn't actually test rbd -- it always tests "file", no
matter which protocol is specified in IMGPROTO.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Currently Focal ends up with v2.2.0-rc3, which results in "WARNING:
Image format was not specified for ..." for all tests and all tests
fail.
Fixes: https://tracker.ceph.com/issues/50605
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Downstream Red Hat products do not support the older QCOW format. This
will allow the support for the legacy QCOW format to be disabled for the
new RBD import-only migration support.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
It appears that commit 6eb8f30a23 broke the test utility and
its failure was masked by the test case that expected a failure
due to a timeout force-killing the app.
Fixes: https://tracker.ceph.com/issues/49117
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
If the qemu-img command fails when attempting to use the QCOW (v1) format,
assume that the distro does not have support for the format enabled.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Utilize the COPIED bit within the cluster offset to quickly determine
if the corresponding cluster block has been copied.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Tweak the IO read path to now utilize the L1 table associated with
the specified snapshot id. This will cause the IO to properly read
from the specific snapshot.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
The initial implementation does not support backing files, compression,
snapshots, (deprecated) encryption, external data files, nor L2 subcluster
allocation. The former three features will be added in a future commit.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
The initial implementation does not support backing files, compression, nor
(deprecated) encryption. The former two features will be added in a future commit.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Since we cannot mark the source image read-only when running in import-only
migration mode, we should require the user to provide a snapshot to ensure
that data cannot change while the migration is running.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
bdi/stable_pages_required attribute was deprecated in 5.10 and now
always returns 0. The replacement is queue/stable_writes. (It is
also writeable, so we can simplify these test cases somewhat in the
future.)
Fixes: https://tracker.ceph.com/issues/48232
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Pending a larger suite of tests for instant-restore image migration,
this test provides a basic sanity check for both the native and
raw image formats -- including basic snapshot tests.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
For `detach` failing to find the process is fatal while unmap
will still try to send disconnect to the device.
Signed-off-by: Mykola Golub <mgolub@suse.com>
The commands allow to restart a daemon without destroying the nbd
device.
Now, if the netlink is used, a dead connection timeout is set on
the nbd device setup, so the device is not immediately released
if the rbd-nbd process terminates without disconnect (unmap).
The attach command just sends terminate signal to the rbd-nbd
process. The detach command starts a new process and connects to
the existing device.
Signed-off-by: Mykola Golub <mgolub@suse.com>
