Commit Graph

2986 Commits

Author SHA1 Message Date
Vallari Agrawal
fdc93ad15e
qa/suites/nvmeof: add mtls test
Add qa/workunits/nvmeof/mtls_test.sh which enables
mtls config and redeploy, then verify and disables
mtls config.

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2024-10-21 10:00:25 +05:30
Vallari Agrawal
531ffdb5bd
Merge pull request #60206 from VallariAg/wip-nvmeof-fix-test
qa/suites/nvmeof: Fix broken suite by using --no-group-append tag
2024-10-10 23:44:31 +05:30
Venky Shankar
43012083f4
Merge pull request #59987 from kotreshhr/licephfs-aio-nfsganesha
client: Fix libcephfs aio metadata corruption.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-10-10 17:41:49 +05:30
Vallari Agrawal
303f18b1af
qa/workunits/nvmeof/setup_subsystem.sh: use --no-group-append
In newer version of nvmeof cli, "subsystem add" needs
this tag to ensure subsystem name is value of --subsystem.
Otherwise, in newer cli version, the gateway group is appended
at the end of the subsystem name.

This fixes the teuthology nvmeof suite (currently all jobs fails
because of this).

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2024-10-10 15:38:51 +05:30
Ilya Dryomov
7e5ebb7934
Merge pull request #60109 from idryomov/wip-63184
librbd/crypto/LoadRequest: clone format for migration source image

Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-10-07 20:15:54 +02:00
Casey Bodley
af86cb51ba
Merge pull request #60001 from cbodley/wip-65654-debug
qa/rgw: add debug log to help diagnose 'bucket check unlinked' failure

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2024-10-07 09:12:05 -04:00
Kotresh HR
59b996f0ed qa: Add libcephfs client test with objectcacher disabled
Fixes: https://tracker.ceph.com/issues/68146
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2024-10-04 22:35:47 +05:30
Ilya Dryomov
7b2085daad librbd/crypto/LoadRequest: clone format for migration source image
Migration source and migration target images naturally have the same
encryption format, but the user shouldn't have to need to specify it
for the image that they can't even immediately see -- migration source
image gets moved to the RBD trash to avoid mistaken usage while
migration is in progress.

Fixes: https://tracker.ceph.com/issues/63184
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-10-03 18:32:17 +02:00
Casey Bodley
fe273a981a
Merge pull request #59745 from cbodley/wip-qa-rgw-multifs-combined
qa/rgw/multifs: add tasks/+ to concatenate tasks into same job

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2024-10-01 16:46:15 -04:00
Casey Bodley
fdecec7349 qa/rgw: add debug log to help diagnose 'bucket check unlinked' failure
relates to https://tracker.ceph.com/issues/65654

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-09-26 10:53:51 -04:00
Adam King
6442dea18f
Merge pull request #54334 from adk3798/cephadm-iscsi-teuth-extension
qa/cephadm: extend iscsi teuth test

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2024-09-23 11:08:25 -04:00
Ilya Dryomov
2ae674d09c
Merge pull request #59417 from nbalacha/wip-nbalacha-ns-mirroring
rbd-mirror: allow mirroring to a different namespace

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-09-18 15:13:31 +02:00
N Balachandran
3fdbc160bb rbd-mirror: allow mirroring to a different namespace
Allows a namespace in a pool to be mirrored to a differently named
namespace in the secondary cluster.

Signed-off-by: N Balachandran <nibalach@redhat.com>
2024-09-17 14:40:27 +05:30
Casey Bodley
ed09e85a89 qa/rgw: perl tests generate valid bucket names
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-09-12 14:15:19 -04:00
Ilya Dryomov
7f24e58932
Merge pull request #44640 from effi-ofer/live-migration-nbd
librbd/migration: add NBD stream

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-09-07 12:07:27 +02:00
Ilya Dryomov
90b55cfdc2 librbd/migration/NBDStream: switch to NBD URIs
This removes the constraint on the transport being TCP, allowing to
use a Unix domain socket or other options.  It also allows specifying
export names which a) are needed in case of serving different content
on different exports and b) some servers may require regardless.

Additionally, NBD URIs are future proof as all that NBDStream needs to
do is forward the string to libnbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-09-06 14:14:12 +02:00
Effi Ofer
dc8eed68ce librbd/migration: add nbd stream
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Effi Ofer <effio@il.ibm.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-09-06 14:14:12 +02:00
Casey Bodley
4ee8e591f3
Merge pull request #56597 from liangmingyuanneo/optimize-reshard
rgw reshard: optimize reshard process to minimum blocking time

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-09-05 13:21:47 -04:00
Ilya Dryomov
f7168600a8
Merge pull request #59551 from idryomov/wip-67845
librbd/migration: prune snapshot extents in RawFormat::list_snaps()

Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-09-04 13:03:04 +02:00
liangmingyuan
196a73cbd4 cls/rgw: add a helper function for calls to cls_cxx_map_remove_key()
Add some testing cases and do cleanup too.

Signed-off-by: Mingyuan Liang <liangmingyuan@baidu.com>
2024-09-04 09:49:18 +08:00
Vallari Agrawal
2ed818ebd8
qa: move nvmeof shell scripts to qa/workunits/nvmeof
Move all scripts qa/workunits/rbd/nvmeof_*.sh
to qa/workunits/nvmeof/*.sh

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2024-09-02 17:04:55 +05:30
Ilya Dryomov
d9192b5aca librbd/migration: prune snapshot extents in RawFormat::list_snaps()
list-snaps is exempt from clipping in ImageDispatcher::PreprocessVisitor
because it's considered to be an internal API.  Further, reads issued
by ObjectCopyRequest based on list-snaps results may also be exempt
because of READ_FLAG_DISABLE_CLIPPING.

Since RawFormat allows specifying a set of snapshots (possibly of
varying size!) to be imported, it needs to compensate for that in its
list-snaps implementation.  Otherwise, an out-of-bounds read will
eventually be submitted to the stream.

Fixes: https://tracker.ceph.com/issues/67845
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-09-01 19:37:30 +02:00
Vallari Agrawal
e5a9cda326
qa/suites/nvmeof/basic: add nvmeof_scalability test
Add test to upscale/downscale nvmeof
gateways.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2024-08-28 18:12:29 +05:30
Vallari Agrawal
58d8be9fd8
qa: Expand nvmeof thrasher and add nvmeof_namespaces.yaml job
1. qa/tasks/nvmeof.py: add other methods to stop nvmeof daemons
2. add qa/workunits/rbd/nvmeof_namespace_test.sh which adds and
   deletes new namespaces. It is run in nvmeof_namespaces.yaml
   job where fio happens to other namespaces in background.

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2024-08-28 18:12:28 +05:30
Patrick Donnelly
305235f11e
Merge PR #59095 into main
* refs/pull/59095/head:
	qa: wait for file creation before changing mode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-08-27 13:09:11 -04:00
Ilya Dryomov
4f309603ca qa: drop XMLSTARLET variable, use xmlstarlet directly
The variable was added in commit 9b6b7c35d0 ("Handle
differently-named xmlstarlet binary for *suse") but this
compatibility business is long outdated:

  Mon Oct 13 08:52:37 UTC 2014 - toms@opensuse.org

  - SPEC file changes
    - Added link from /usr/bin/xml to /usr/bin/xmlstarlet as other
      distributions do the same
    - Did the same for the manpage

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-08-25 13:22:08 +02:00
Ilya Dryomov
9ac05d9030
Merge pull request #44470 from orozery/rbd-external-migrate
librbd/migration: add external clusters support

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-08-23 10:20:47 +02:00
Ilya Dryomov
4a6800f146 qa/workunits/rbd: exercise snap_{name,id} parsing in test_import_native_format()
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-08-22 12:30:38 +02:00
NitzanMordhai
182d803ce2
Merge pull request #59165 from NitzanMordhai/wip-nitzan-test-rados-tools-newline-trim
test: test_rados_tools compare output without trimming newline
2024-08-20 15:07:21 +03:00
Ilya Dryomov
a4ff70396b
Merge pull request #59153 from ajarr/wip-67436
rbd: fix CLI output of `rbd group snap info` command when a group snapshot with no member images

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Sunil Angadi <Sunil.Angadi@ibm.com>
2024-08-20 12:19:23 +02:00
Yuri Weinstein
959a4c3017
Merge pull request #58961 from NitzanMordhai/wip-nitzan-dencoder-test-forward-incompat-fix
workunit/dencoder: dencoder test forward incompat fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-08-19 07:25:47 -07:00
Ramana Raja
9ea03123a1 rbd: fix CLI output of rbd group snap info command
... when a group snapshot has no member images.

A group snapshot can be created with no member images. For such a group
snapshot, omit the 'image snap' and 'images' fields from the
unformatted CLI output of `rbd group snap info` command so as to not
confuse the user. In the librbd C/C++ data structures representing a
group snapshot with no member images, set the 'image_snap_name' data
member to an empty string.

Fixes: https://tracker.ceph.com/issues/67436
Signed-off-by: Ramana Raja <rraja@redhat.com>
2024-08-13 17:24:42 -04:00
Nitzan Mordechai
0c423e1d81 test: test_rados_tools compare output without trimming newline
Fixes: https://tracker.ceph.com/issues/67419
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-08-12 11:28:41 +00:00
Ilya Dryomov
d5a061e68b
Merge pull request #44366 from orozery/rbd-crypto-migration
librbd/crypto: fix issue when live-migrating from encrypted export

Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-08-08 00:30:24 +02:00
Kamoltat (Junior) Sirivadhna
6a0d503a59
Merge pull request #56233 from kamoltat/wip-ksirivad-fix-64802
RADOS: Generalize stretch mode pg temp handling to be usable without stretch mode
Samuel Just <sjust@redhat.com>
2024-08-07 09:45:54 -04:00
Venky Shankar
ebf66af827 qa: wait for file creation before changing mode
Fixes: http://tracker.ceph.com/issues/67408
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2024-08-07 13:26:30 +00:00
Ilya Dryomov
0000c34474 librbd/migration: make ImageDispatch handle encryption for non-native formats
With NativeFormat now being handled via dispatch, handling encryption
for non-native formats (i.e. mapping to raw image extents and performing
decryption/mapping back on completion) in the migration layer is really
straightforward.

Note that alignment doesn't need to be performed in the migration layer
because it happens on the destination image -- the "align and resubmit"
logic in C_UnalignedObjectReadRequest should kick in before the call to
read_parent().

Fixes: https://tracker.ceph.com/issues/53674
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-08-07 12:35:34 +02:00
Ilya Dryomov
82d26909cb qa/workunits/rbd: perform cleanup in test_clone_encryption()
... so that RAW_DEV can be unmapped and future tests can reuse testimg
and other image names without bumping into watchers and older snapshots.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-08-07 12:35:34 +02:00
Ilya Dryomov
830cbee7a5 qa/workunits/rbd: no need to chmod in luks-encryption.sh
Most workunits expect the user to be a member of "disk" group, so we
can pretty much rely on that being the case at this point.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-08-07 12:35:34 +02:00
Patrick Donnelly
95dffdcbbd
Merge PR #58582 into main
* refs/pull/58582/head:
	test: add a test for `fs set down` on online cluster
	mon: fix `fs set down` to adjust max_mds only when cluster is not down

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-08-06 17:03:25 -04:00
Yuri Weinstein
1a2a617edd
Merge pull request #54671 from baum/ceph-nvmeof-mon
mon: add NVMe-oF gateway monitor and HA

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-07-31 09:05:41 -07:00
Nitzan Mordechai
30921272dd workunit/dencoder: fix corpus test for backword and forward compability
- changed the check for non-deterministic, return code 1 is also legit
- unneeded check for is_dir, if it exist
- limit the number of threads to prevent error

Fixes: https://tracker.ceph.com/issues/67263
Signed-off-by: NitzanMordhai <nmordech@redhat.com>
2024-07-31 11:23:05 +00:00
Vallari Agrawal
d0c4182bf5 qa/suites/rbd/nvmeof: add multi-subsystem setup and thrash test
1. qa/tasks/nvmeof.py:
    1.1. create multiple rbd images for all subsystems
    1.2. add NvmeofThrasher and ThrashTest
2. qa/tasks/mon_thrash.py: add 'switch_thrashers' option
3. nvmeof_setup_subsystem.sh: create multiple subsystems and enable HA
4. Restructure qa/suites/rbd/nvmeof: Create two sub-suites
   - "basic" (nvmeof_initiator job)
   - "thrash" (new: nvmeof_mon_thrash and nvmeof_thrash jobs)

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2024-07-31 08:52:05 +00:00
Casey Bodley
42ecd4c8d7
Merge pull request #58213 from benhanokh/ref_count
rgw/rgw_rados: fix server side-copy orphans tail-objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-07-30 18:20:24 +01:00
chungfengz
f8becaaedb test: add a test for fs set down on online cluster
Fixes: https://tracker.ceph.com/issues/66960
Signed-off-by: chungfengz <chungfengz@synology.com>
2024-07-26 22:41:41 +08:00
liangmingyuan
cec5e83d8e test/rgw: add test for logrecord reshard
Signed-off-by: Mingyuan Liang <liangmingyuan@baidu.com>
2024-07-26 08:50:50 +08:00
Ilya Dryomov
e25b81ea93
Merge pull request #58781 from idryomov/wip-67097
qa/suites/krbd: stress test for recovering from watch errors for -o exclusive

Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-07-25 14:45:18 +02:00
J. Eric Ivancich
155b49872e
Merge pull request #58670 from ivancich/wip-bolster-reshard-testing
test/rgw: address potential race condition in reshard testing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-07-24 11:03:38 -04:00
Ilya Dryomov
8fee41da8b qa/suites/krbd: stress test for recovering from watch errors for -o exclusive
This is based on a test added in commit 237aa221eb ("qa/suites/krbd:
stress test for recovering from watch errors") for regular mappings.

Fixes: https://tracker.ceph.com/issues/67097
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-07-24 08:48:47 +02:00
Ilya Dryomov
ed30494acb
Merge pull request #57797 from ajarr/wip-66011
rbd: add `group snap info` command

Reviewed-by: N Balachandran <nibalach@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-07-23 19:52:40 +02:00