Commit Graph

536 Commits

Author SHA1 Message Date
Mykola Golub
5e15e111d3
Merge pull request #33588 from dillaman/wip-rbd-snapshot-mirror
rbd-mirror: initial end-to-end test and associated bug fixes

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-03-03 15:19:55 +02:00
Jason Dillaman
d6f4ce2a38 qa: test cases for snapshot-based mirroring
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Mykola Golub
d68d41c366 qa/workunits/rbd: add `mirror snapshot schedule' cli test
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-02-25 09:52:22 +00:00
Mykola Golub
253733ab89 qa/workunits/rbd: add `trash purge schedule' cli test
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-02-25 09:52:22 +00:00
Jason Dillaman
41357d242b qa/workunits/rbd: added mgr 'profile rbd' caps to mirror user
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-02-12 20:22:30 -05:00
Jason Dillaman
14001eb17e qa/workunits/rbd: additional overrides for CEPH_ARGS
The 'ceph' CLI and 'rbd mirror pool/image status' commandsshould revert
to use the admin user so that it has proper credentials for the cluster.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-02-12 20:21:48 -05:00
Mykola Golub
0ec4f8f1b9
Merge pull request #33187 from dillaman/wip-44066
qa/workunits/rbd: override CEPH_ARGS when initializing the site name

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-02-11 16:27:35 +02:00
Jason Dillaman
57d373fa99 qa/workunits/rbd: override CEPH_ARGS when initializing the site name
The mirroring site name is stored in the MON config which requires
higher privledges than the standard "client.mirror" user.

Fixes: https://tracker.ceph.com/issues/44066
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-02-10 11:39:24 -05:00
Jason Dillaman
ffa791f540 qa: move existing rbd-mirror functional test to journal-specific test
A new functional test for snapshot-based mirroring will be created and
the other stress-tests should eventually be applied to both snapshot-
and journal-based mirroring.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-01-30 10:26:36 -05:00
Jason Dillaman
cefaa78186 qa/workunits/rbd: use more recent qemu-iotests that support EL8
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-01-17 08:44:44 -05:00
Ilya Dryomov
b2b7610d1a qa: krbd_exclusive_option.sh: fixup for json.tool ordering change
In Python 3.5 json.tool was changed to produce unsorted output and
--sort-keys option was added to compensate.  This wasn't caught by
4fe245cc2f2d ("qa: update krbd tests for python3") because it raced
with 50933b863a1d ("qa: krbd_exclusive_option.sh: update for recent
kernel changes").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-01-06 15:12:25 +01:00
Jason Dillaman
5116a77f5d qa/workunits/rbd: fixed python interpreter for EL8
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-23 15:23:49 -05:00
Sage Weil
253a557f10 qa/workunits: python -> python3 throughout
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 12:38:47 -06:00
Kefu Chai
79584862f6 src,qa: install python bindings into lib/cython_modules/lib.3
instead of checking for the WITH_PYTHON* options, just hardwire to lib.3

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:54 +08:00
Sage Weil
3268ec7ac8 Merge PR #32252 into master
* refs/pull/32252/head:
	qa/cephfs/begin: libaio-devel on el8
	qa/tasks: nosetests -> python -m nose
	qa/tasks/rbd_fio: fio 2.21 -> 3.16
	src/test/cli-integration/rbd/snap-diff.t: python -> python
	qa/workunits: use nose 3
	qa/tasks/cbt: install python3 deps
	qa/tasks/ceph_manager.py: do not use python to write a file
	test/pybind/test_rados: execute takes a bytes (not str) payload
	qa/packages/packages: python[3]-ceph is no more
	qa: use python3 for venvs etc
	packaging: remove python3-ipaddres, as it is part of the stdlib in py3
	qa/packages: python-ceph -> python3-ceph
	qa/distros: centos7 -> centos8, rhel7 -> rhel8
	spec: remove _python_buildid in favor of python3_pkgversion macro
	spec: remove python2 packages and conditions
	debian: remove python >= 2.7 requirement
	debian: add mgr python versions
	debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
	debian: update control file to use python3 dependency names
	debian: remove all python2 overrides and declarations
	debian: remove all python2 install files

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-12-17 15:23:27 -06:00
Jason Dillaman
6787219e71
Merge pull request #32280 from dillaman/wip-librbd-qos-test
qa/workunit/rbd: fixed QoS throughput unit parsing

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-12-17 12:30:52 -05:00
Jason Dillaman
7838d3e18e qa/workunit/rbd: fixed QoS throughput unit parsing
The 'rbd bench' command was recently modified to print IEC units
instead of bytes/sec. This broke the handling for QoS throughput
tests since it was incorrectly evaluating the available RBD
throughput. Additionally, the QoS tests should use a "<="
comparison operator since the QoS is the upper-bound limit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-17 09:12:43 -05:00
Sage Weil
889446a515 qa/workunits: use nose 3
Ubuntu is /usr/bin/nosetests3, el is /usr/bin/nosetests-3, so do neither.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-16 17:02:45 -06:00
Mykola Golub
678b53735b librbd: explicitly specify mode on mirror image enable
- don't use journaling feature to chose the mode;
- provide new API function mirror_image_enable2;
- return back the old behavior to automatically enable/disable
  journaling feature on enabling/disabling image mirroring.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-12-16 08:24:30 +00:00
Jason Dillaman
f6c18468ce
Merge pull request #30548 from trociny/wip-41557
rbd-mirror: snapshot mirror mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-12-10 16:41:28 -05:00
Mykola Golub
c9c8852f6e librbd: snapshot mirror mode
Enabling mirroring for an image that does not support journaling
assumes snapshot based mirroring, which is supported only when the
pool is in the "image" mirror mode.

Also for the pool in the "image" mirror mode disabling/enabling
journaling feature for a mirroring image will switch
snapshot/journal mirror mode.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-12-10 15:45:30 +00:00
Ilya Dryomov
0bc226f6e1
Merge pull request #32088 from idryomov/wip-krbd-exclusive-lock-fixups
qa: krbd_exclusive_option.sh: update for recent kernel changes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-12-09 22:59:13 +01:00
Ilya Dryomov
3aae140353 qa: kernel.sh: unlock before rolling back
"rbd snap rollback" expects an unlocked image, but we may get there
locked if object map is enabled (or if lock_on_read is specified in
rbd_default_map_options).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-12-09 17:29:13 +01:00
Ilya Dryomov
50933b863a qa: krbd_exclusive_option.sh: update for recent kernel changes
Since 5.3:
- a plain "rbd map" acquires the lock, so it's not different from
  "rbd map -o exclusive" in this regard
- if the lock is held by the exclusive peer, I/O is failed right away
  instead of blocking
- lock_timeout option is respected only by "rbd map" and not by I/O

Since 5.5:
- if the mapping is read-only, the lock isn't acquired

Added blacklisting test case, dropped lock_timeout test case.

Fixes: https://tracker.ceph.com/issues/43127
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-12-09 17:29:13 +01:00
Jason Dillaman
b27820cc73 rbd: incorporate rbd-mirror daemon status in mirror pool status
This will ensure that the Ceph dashbord's block mirroring page and
the CLI's 'mirror pool status' have matching health indications.

Fixes: https://tracker.ceph.com/issues/42748
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-04 08:14:06 -05:00
Ilya Dryomov
4fe245cc2f qa: update krbd tests for python3
There is no /usr/bin/python symlink in CentOS 8 even if python2 is
installed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-12-02 21:13:19 +01:00
Ilya Dryomov
6fb544045e qa: kernel.sh: update for read-only changes
Snapshot existence validation code was removed from krbd.  It was racy
and relied on having watch established for snapshots.

Fixes: https://tracker.ceph.com/issues/42916
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-11-20 21:32:45 +01:00
Jason Dillaman
4a6c1ec14b
Merge pull request #31260 from trociny/wip-42405
rbd-mirror: removing non-primary trash snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-11-08 07:35:41 -05:00
Mykola Golub
1dc0db74d4
Merge pull request #30912 from dillaman/wip-42265
mgr: add 'rbd' profiles to support 'rbd_support' module commands

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-01 13:41:28 +02:00
Mykola Golub
473695e3bc qa/workunits/rbd: test mirrored snap trash removal
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-10-31 15:58:18 +02:00
Jason Dillaman
f4d99f796a
Merge pull request #31161 from trociny/wip-42488
rbd-mirror: mirrored clone should be same format 

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-10-30 12:07:30 -04:00
Jason Dillaman
7cccdef25e qa/workunits/rbd: add permission tests for mgr profile
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-10-29 08:35:03 -04:00
Ilya Dryomov
b7a0e2adcb qa: add script to stress udev_enumerate_scan_devices()
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-10-25 22:05:38 +02:00
Mykola Golub
dea3d1b738 rbd-mirror: mirrored clone should be same format
Fixes: https://tracker.ceph.com/issues/42488
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-10-25 17:50:54 +03:00
Ilya Dryomov
898c113f93 qa: add script to test udev event reaping
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-10-18 21:56:30 +02:00
Jason Dillaman
eaec1786ee qa/workunits/rbd: tweak mirror test case to handle peer status
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-10-08 11:17:15 -04:00
Jason Dillaman
547ebccaf9 librbd: v1 clones are restricted to the same namespace
Therefore the v1 parent spec should not attempt to populate the
namespace.

Fixes: https://tracker.ceph.com/issues/41938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-10-04 13:27:50 -04:00
Jason Dillaman
85f98c01c0 qa/suites/rbd: test case for rbd-mirror bootstrap
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-09-20 08:09:21 -04:00
Ilya Dryomov
d63570c5a5
Merge pull request #30363 from idryomov/wip-krbd-huge-osdmap
qa: add script to test how libceph handles huge osdmaps

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-09-13 22:35:57 +02:00
Ilya Dryomov
81becbdc68 qa: add script to test how libceph handles huge osdmaps
That code will also handle moderately-sized osdmaps when the memory is
fragmented.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-09-13 19:21:54 +02:00
Mykola Golub
4b0fcc8cfb qa/workunits/rbd: add rbd-mirror namespace tests
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-09-05 13:57:47 +01:00
Mykola Golub
de66e0b89c qa/workunits/rbd: add 'remove mirroring pool' test
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-08-27 16:42:51 +01:00
Mykola Golub
0866a2456b qa/workunits/rbd: stress test rbd mirror pool status --verbose
Fixes: https://tracker.ceph.com/issues/40923
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-08-16 12:50:42 +01:00
Jason Dillaman
7adb17f831 pybind/mgr/rbd_support: use image ids to detect duplicate tasks
This helps to to avoid the case where new tasks were not being scheduled
when an image name was re-used after having a task created under the
same name.

Fixes: https://tracker.ceph.com/issues/41032
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-08-06 12:17:46 -04:00
Jason Dillaman
63a8f6c2c3 qa: test case for new rbd background tasks
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-07-25 08:58:37 -04:00
Mykola Golub
11e2951243 qa/suites/rbd: allow to test with require-min-compat-client octopus
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-06-23 10:06:45 +01:00
Mykola Golub
c99e26eff9 qa/workunits/rbd: tweak mirror tests
(after "image mirror disable" started to disable journaling)

Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-06-18 12:08:58 +01:00
Mykola Golub
92ee1c7b8b qa/workunits/rbd: fix compare_images and compare_image_snapshots
Due to pipe, sdiff return code was ignored. Also the compare functions
spent most of time in xxd, which was normally unnecessary.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-06-13 08:26:29 +01:00
Jason Dillaman
b36a1ad302 qa/workunits/rbd: remove 'fast-diff' from dynamic features test
Fixes: http://tracker.ceph.com/issues/39946
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-15 16:53:43 -04:00
Jason Dillaman
fb4f9a8a08 qa/workunits/rbd: use https protocol for devstack git operations
Fixes: http://tracker.ceph.com/issues/39656
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-10 11:09:54 -04:00