Commit Graph

9617 Commits

Author SHA1 Message Date
Ilya Dryomov
8780f602a9
Merge pull request #48618 from idryomov/rbd-clone-encryption-part2
librbd: add encryption format support for clones (part 2/2)

Reviewed-by: Mykola Golub <mgolub@suse.com>
Acked-by: Or Ozeri <oro@il.ibm.com>
2022-12-05 17:47:19 +01:00
Ilya Dryomov
86b60ebd70
Merge pull request #49235 from idryomov/wip-rbd-groups-bash
qa: rbd_groups.sh: change interpreter to bash

Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-12-05 15:59:29 +01:00
Matan
b583ed6ede
Merge pull request #49076 from Matan-B/wip-matanb-crimson-suite-adjust
crimson: Suite testing minor adjustment

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-05 12:19:36 +02:00
Ilya Dryomov
8d5d478532 qa/workunits/rbd: add encryption-aware resize test
Note that we are hitting https://tracker.ceph.com/issues/58160 here
because by the time we get to "rbd resize" RAW_DEV mapping owns the
lock (due to a write to /dev/mapper/cryptsetupdev being last).

While at it, resurrect the ability to easily run this script on
vstart clusters -- see commit f737c2855a ("qa/workunits/rbd: make
luks-encryption test work on vstart cluster").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:24:10 +01:00
Ilya Dryomov
a27ee2bdf8 rbd, rbd-nbd: make --encryption-format optional
If no --encryption-format specified at all, default to "luks" for each
specified --encryption-passphrase-file.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
e62e3b6613 rbd, rbd-nbd: accept "luks", "luks1" and "luks2" formats
Since RBD_ENCRYPTION_FORMAT_LUKS1, RBD_ENCRYPTION_FORMAT_LUKS2
and RBD_ENCRYPTION_FORMAT_LUKS aren't treated the same when loading
encryption anymore, "luks1" and "luks2" formats need to be accepted
in addition to "luks" format.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
d642f7804b rbd, rbd-nbd: don't strip trailing newline in passphrase files
One of the stated goals is compatibility with standard LUKS tools,
in particular being able to load encryption on images formatted with
cryptsetup.  cryptsetup doesn't do this and this really interferes
with randomly generated (binary) passphrases.

While at it, open passphrase files as binary -- it communicates the
intent if nothing else on POSIX.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
8f712733af qa: rbd_groups.sh: change interpreter to bash
Commit e0da2a4e8c ("qa/workunits/rbd: Add test to list snapshots of
consistency group") added bash-specific syntax.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 13:20:44 +01:00
Ilya Dryomov
9ca2ec704e
Merge pull request #48549 from pkalever/snap-list
cls/rbd: update last_read in group::snap_list

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-02 13:18:08 +01:00
Brad Hubbard
75447396c3
Merge pull request #48838 from badone/wip-ceph-ansible-suite-nightlies-disable
qa/tests: Remove octopus ceph-ansible tests

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-12-02 11:51:57 +10:00
Matan Breizman
d45bbaf6bf qa/suites/crimson-rados/thrash: do not wait for scrub
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-11-29 16:03:08 +00:00
Ilya Dryomov
af6ed506f2
Merge pull request #48680 from pkalever/snap-id
rbd: add --snap-id option to "rbd device map" to allow mapping arbitrary snapshots

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-11-27 14:10:31 +01:00
Ilya Dryomov
4a7150cd36 qa/workunits/rbd-nbd: clear DEV after detach tests
Otherwise we attempt to unmap it in cleanup(), needlessly.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-11-26 13:27:33 +01:00
Ilya Dryomov
f2f5ca5150
Merge pull request #49036 from idryomov/wip-52932
mgr/rbd_support: avoid wedging the task queue if pool is removed

Reviewed-by: Mykola Golub <mgolub@suse.com>
2022-11-25 11:48:42 +01:00
Kefu Chai
2413da8c85 qa/tasks/cephfs: annotate variable without comment
to silence the warning like
```
flake8 run-test: commands[0] | flake8 --select=F,E9 --exclude=venv,.tox
./tasks/cephfs/test_full.py:6:5: F401 'typing.Optional' imported but unused
```

since we've migrated to python3.6 and up, there is no need to be
compatible with Python 2 anymore.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-11-24 14:06:24 +08:00
Ilya Dryomov
5a425927ed mgr/rbd_support: avoid wedging the task queue if pool is removed
rados.ObjectNotFound exception handler was referencing ioctx variable
which is assigned only if the pool exists and rados.open_ioctx() call
succeeds.  This lead to a fatal error

  mgr[rbd_support] Failed to locate pool mypool
  mgr[rbd_support] execute_task: [errno 2] error opening pool 'b'mypool''
  mgr[rbd_support] Fatal runtime error: local variable 'ioctx' referenced before assignment

and wedged the task queue.  No other commands were processed until
ceph-mgr daemon restart.

Fixes: https://tracker.ceph.com/issues/52932
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-11-23 23:11:42 +01:00
Yuval Lifshitz
b524a430f3
Merge pull request #48758 from yuvalif/wip-yuval-kafka-tests
rgw/notification/test: add support for testing kafka security
2022-11-23 13:00:00 +02:00
Yuval Lifshitz
dcbc4afc21 rgw/notification/test: avoid kafka ssl tests in teuthology
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-11-21 11:56:11 +02:00
Laura Flores
bf30531f01 qa/workunits/rados: specify redirect in curl command
This workunit currently grabs files in html form, since
there is a redirect that occurs. If we specify `-L` in the
curl command, it handles redirects and graps the files
appropriately.

Fixes: https://tracker.ceph.com/issues/58046
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-11-18 00:31:59 +00:00
Brad Hubbard
532f7c9765 qa/tests: Remove octopus ceph-ansible tests
Ceph-ansible is no longer supported and octopus is EOL.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2022-11-11 12:05:12 +10:00
Prasanna Kumar Kalever
92480e6561 qa/workunits/rbd: added tests for --snap-id
Fixes: https://tracker.ceph.com/issues/57902
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-11-10 19:28:30 +05:30
Prasanna Kumar Kalever
e0da2a4e8c qa/workunits/rbd: Add test to list snapshots of consistency group
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-11-09 11:19:35 +05:30
Laura Flores
5573558a17
Merge pull request #48291 from ljflores/wip-upgrade-telemetry 2022-11-04 20:00:30 -05:00
Laura Flores
d1dbfae3b2
Merge pull request #48619 from ceph/wip-tracker-57883
qa/standalone/erasure-code: give osdmap 5 seconds to refresh
2022-10-26 09:43:09 -05:00
Venky Shankar
82f46b1fdc Merge PR #43284 into main
* refs/pull/43284/head:
	doc: describe bal_rank_mask configuration
	qa/cephfs: add bal_rank_mask test cases
	mds: add bal_rank_mask option for ceph fs set

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-10-26 11:28:25 +05:30
Laura Flores
c1e6f7c470 qa/standalone/erasure-code: give osdmap 5 seconds to refresh
Fixes: https://tracker.ceph.com/issues/57883
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-10-25 17:03:24 +00:00
Ilya Dryomov
0f93f745c6
Merge pull request #40363 from orozery/rbd-clone-encryption
librbd: add encryption format support for clones (part 1/2)

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-10-25 18:16:33 +02:00
Casey Bodley
5dc16bef2e
Merge pull request #46188 from Zhiwei-Dai/wip-rgw-crypto-stack-on-compress
rgw: support full object encryption stack on compression

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-10-24 11:52:26 -04:00
Kefu Chai
9bdf050b92 qa/tasks/vstart_runner: do not fail if stop.sh fails
stop.sh might fail, but let's keep moving on.

See-also: https://tracker.ceph.com/issues/57834
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-10-22 01:24:32 +08:00
Samuel Just
5a6199b7d7
Merge pull request #48515 from athanatos/sjust/wip-crimson-suites
qa/suites/crimson-rados: move seastore to crimson-rados-experimental, remove cache/dedup workloads

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2022-10-18 15:45:28 -07:00
Samuel Just
224891fe47 qa/suites/crimson-rados: remove cache and dedup related tests
It's an accident that some of these pass -- the tests provide some
coverage for inconsistencies that can arise, but don't really validate
that the objects actually get moved between tiers.

It's going to be some time before we implement cache or dedup in
crimson, and we'll probably want to disable the related commands for
pools that can map to crimson osds to prevent accidents.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-10-18 07:59:50 -07:00
Samuel Just
f9c58c0416 qa/suites: move seastore to crimson-rados-experimental suite
Let's use crimson-rados-experimental for tests that don't yet
pass reliably.  We can move these to crimson-rados as they
become reliable.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-10-18 07:59:47 -07:00
Venky Shankar
bfa5da4b2f qa: fs:upgrade featureful_client test upgrades to quincy
Reef cycle will not allow upgrades from octopus. However, the featureful
client tests still needs to be testes, therefore, upgrade to quincy (from
octopus) rather to the current cycle (reef).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-10-14 11:24:34 +05:30
Yongseok Oh
10fb8f1377 qa/cephfs: add bal_rank_mask test cases
Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
2022-10-13 01:39:34 +00:00
Samuel Just
2013ea32d0
Merge pull request #48406 from athanatos/sjust/wip-57798
qa/suites/crimson-rados: add CRIMSON_COMPAT to workunit env

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2022-10-11 11:36:03 -07:00
Venky Shankar
296ba0641a
Merge pull request #48196 from neesingh-rh/wip-human-readable-volume-info
mgr/volumes: Add human-readable flag to volume info command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-10-11 16:48:19 +05:30
Ilya Dryomov
9db96c33d2
Merge pull request #48367 from stefan-chivu/namespace-optarg-fix
rbd: device map/unmap --namespace handling fixes

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-10-11 11:58:15 +02:00
Marcus Watts
5deb213ae5
Merge pull request #45395 from tobias-urdin/rgw-keystone-service-token
rgw/auth: Add service token support for Keystone auth
2022-10-11 05:49:30 -04:00
Nizamudeen A
75868f12da
Merge pull request #48412 from rhcs-dashboard/fix-rgw-subuser-permissions
mgr/dashboard: Unable to change rgw subuser permission

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-10-11 15:11:07 +05:30
Aashish Sharma
0b3bdbbc4a mgr/dashboard: Unable to change rgw subuser permission
Tried to edit the permission of a subuser but once changed the permission, edited permission not seen in user info of UI(ceph dashboard) and from CLI as well

Fixes:https://tracker.ceph.com/issues/57805
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2022-10-11 12:22:00 +05:30
Venky Shankar
bc5e1a6b7b
Merge pull request #47812 from rishabh-d-dave/json-decode-error
qa/cephfs: handle non-numeric values for json.loads()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-10-11 09:32:42 +05:30
Samuel Just
436fc5d7e6 qa/suites/crimson-rados: add CRIMSON_COMPAT to workunit env
Fixes: https://tracker.ceph.com/issues/57798
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-10-10 16:45:33 -07:00
Rishabh Dave
ee321e8570
Merge pull request #47754 from batrick/i57248
qa: cleanup volumes on unwind

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2022-10-11 00:12:24 +05:30
Rishabh Dave
72a0b536ce
Merge pull request #48064 from lxbsz/asok_fs
qa: the default CEPH_ASOK_DIR is asok/ now

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-10-11 00:11:08 +05:30
Rishabh Dave
adc65d6bb4
Merge pull request #48182 from lxbsz/wip-57580
qa: make mount to be compatible with nautilus for blocklist

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2022-10-11 00:05:55 +05:30
Ilya Dryomov
edfcf4bd6b qa/workunits/rbd-nbd: also test map/unmap with --namespace option
The entire script is using specs, let's cover non-spec syntax as well.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-10-10 20:19:17 +02:00
Venky Shankar
e3a82a2cba
Merge pull request #48337 from batrick/first-python
tools/cephfs: convert first-damage.sh to python

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-10-10 20:58:47 +05:30
Laura Flores
968607a243 qa/workunits: interpret with bash instead of sh
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-10-08 06:50:25 +00:00
Laura Flores
f9e77f0286 qa/suites/upgrade: add telemetry coverage to existing upgrade tests
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-10-08 04:48:49 +00:00
Laura Flores
359435efb8 qa/suites/upgrade/telemetry-upgrade: add new telemetry upgrade tests
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-10-08 04:48:40 +00:00