Commit Graph

141722 Commits

Author SHA1 Message Date
Ramana Raja
b7aae5c3c5 qa: Add tests to validate syncing of images using rbd-mirror
Introduce functional tests to validate that the images under
workloads are correctly mirrored between two clusters using snapshot
based mirroring.

Run workload on a primary image using a krbd or nbd client. Take
mirror snapshots of the image under workload. Unmount the mapped image
and calculate its MD5 checksum before demoting it. After demotion,
wait for the mirror status of the image to be 'up+unknown' in both
the clusters. This is to make sure that the non-primary image in the
other cluster is ready to be promoted. Now promote the non-primary
image in the other cluster. Map the promoted image and calculate its
MD5 checksum. Verify that the checksums of the demoted and promoted
images in the two clusters are the same.

The above test is run as part of two different workunits:
 - a workunit that validates the syncing of multiple mirrored images
   with workloads running on them
 - another workunit that validates the syncing of a single mirrored
   image with workload running on it and the image is set as primary
   alternatively between the two clusters, as it happens during
   failover and failback scenarios.

Fixes: https://tracker.ceph.com/issues/61617
Signed-off-by: Ramana Raja <rraja@redhat.com>
Co-authored-by: Ilya Dryomov <idryomov@redhat.com>
Co-authored-by: Christopher Hoffman <choffman@redhat.com>
2024-02-22 11:44:36 -05:00
Ramana Raja
ea3a567f7f qa/workunits: make wait_for_status_in_pool_dir() reentrant
In rbd_mirror_helpers.sh, the `wait_for_status_in_pool_dir()` helper
stored `mirror image status` and `mirror pool status` command outputs
in files that could be shared over successive calls or calls from
multiple threads. Instead store the command outputs in local variables
to make `wait_for_status_in_pool_dir()` reentrant.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2024-02-22 11:44:28 -05:00
Casey Bodley
8cd25c3624
Merge pull request #54787 from cbodley/wip-versioned-variant-inline
common: use inline for monostate dencoders

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2023-12-06 01:55:57 +00:00
Casey Bodley
0179a8ea2f common: use inline for monostate dencoders
fix a 'multiple definition' error when included by multiple sources:

src/common/versioned_variant.h:31: multiple definition of `ceph::encode(std::monostate const&, ceph::buffer::v15_2_0::list&)';
rgw_main.cc.o:src/common/versioned_variant.h:31: first defined here

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-12-05 12:21:27 -05:00
Nizamudeen A
f64ad4406c
Merge pull request #54068 from rhcs-dashboard/rgw-roles-update
mgr/dashboard: add support for editing and deleting rgw roles 

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2023-12-05 13:39:00 +05:30
zdover23
b393ea035a
Merge pull request #54774 from sinashan/main
doc: Fixes two typos and grammatical errors. Signed-off-by: Sina Ahma…

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-12-04 20:41:46 +01:00
sinashan
bc39adceba doc: Fixes two typos and grammatical errors. Signed-off-by: Sina Ahmadi <sinatak1373@live.com> 2023-12-04 14:02:54 -05:00
Ilya Dryomov
d9147a14c4
Merge pull request #54205 from VallariAg/wip-nvmeof-test
qa: add rbd/nvmeof integration test

Reviewed-by: Zack Cerza <zack@redhat.com>
Reviewed-by: Aviv Caro <Aviv.Caro@ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-12-04 18:14:38 +01:00
Vallari Agrawal
42e121a42a
qa: add rbd/nvmeof test
A basic test for ceph-nvmeof[1] where
nvmeof initiator is created.
It requires use of a new task "nvmeof_gateway_cfg"
under cephadm which shares config information
between two remote hosts.

[1] https://github.com/ceph/ceph-nvmeof/

Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
2023-12-04 19:27:54 +05:30
Adam King
1680e466aa
Merge pull request #54441 from phlogistonjohn/jjm-cephadm-breakup
cephadm: break various daemon type classes out to smaller files in cephadmlib

Reviewed-by: Adam King <adking@redhat.com>
2023-12-04 07:32:52 -05:00
Adam King
a1a2252f7f
Merge pull request #54679 from Suyashd999/add-rgw
src/cephadm: Added rgw for listing in Daemons

Reviewed-by: Adam King <adking@redhat.com>
2023-12-04 07:26:13 -05:00
Nizamudeen A
4b7bfdaa00
Merge pull request #54076 from rhcs-dashboard/subvolume-snapshot
mgr/dashboard: cephfs subvolume list snapshots

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
2023-12-04 14:12:51 +05:30
Anthony D'Atri
e89bb53732
Merge pull request #54758 from zdover23/wip-doc-2023-12-03-rados-stretch-mode-repair-of-54689
doc/rados: repair stretch-mode.rst
2023-12-03 09:40:46 -05:00
Zac Dover
b73656787d doc/rados: repair stretch-mode.rst
Remove a section of doc/rados/operations/stretch-mode.rst that I wrongly
re-included after its removal. The request for this (re)-removal is
here: https://github.com/ceph/ceph/pull/54689#discussion_r1413007655.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-12-03 13:17:46 +01:00
Matan Breizman
67b75549b3
Merge pull request #54437 from Matan-B/wip-matanb-crimson-osdmap-trimming
crimson/osd: introduce osdmap trimming 

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-12-03 10:06:33 +02:00
Ronen Friedman
65945407ac
Merge pull request #53712 from ronen-fr/wip-rf-move-1
osd: fix a missing 'noexcept' on a move ctor

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-12-03 09:51:41 +02:00
Ronen Friedman
8ec47bba09
Merge pull request #54482 from ronen-fr/wip-rf-repl-hp
osd/scrub: decouple being reserved from handling scrub requests

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-12-03 09:45:18 +02:00
zdover23
28f26d87b3
Merge pull request #54689 from PC-Admin/update-stretch-mode-docs
doc/rados: Update stretch mode docs.

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2023-12-03 06:56:59 +01:00
PC-Admin
28551b41f8 doc/rados: update "stretch mode"
Update stretch mode docs, min_size and max_size are no longer defined in
the CRUSH map and the example rule given will fail to compile.

Specify that the tiebreaker data centre cannot be defined in CRUSH as
this produces an error.

Signed-off-by: Michael Collins <perthserverplus@gmail.com>
2023-12-03 06:45:16 +01:00
Ronen Friedman
084fb34971 osd: fix a missing 'noexcept' on a move ctor
as a non-default, non-noexcept move ctor is ignored by
stl containers.

See clang-tidy's performance-noexcept-move-constructor

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-12-02 16:36:30 +02:00
zdover23
171d2b5b38
Merge pull request #54752 from zdover23/wip-doc-2023-12-02-radosgw-multisite-formatting
doc/radosgw: fix formatting

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-12-02 07:22:12 +01:00
Zac Dover
08ac3114e9 doc/radosgw: fix formatting
Repair the formatting of a string that had a string inside backticks
that itself was inside double asterisks. The presence of the asterisks
around the entire string caused the backticks to appear in the rendered
documentation.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-12-02 06:38:28 +01:00
Casey Bodley
b27938f5f1
Merge pull request #54526 from cbodley/wip-rgw-acl-cleanup
rgw/acl: clean up RGWAccessControlPolicy and friends

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-12-01 16:46:33 +00:00
Casey Bodley
22a3208a1e
Merge pull request #54488 from cbodley/wip-versioned-variant
common: add versioned encodings for std::variant

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2023-12-01 16:02:20 +00:00
Ronen Friedman
6f1e0e6825 tests/scrub: deactivate osd-scrub-dump stand-alone test
as the scrub reservation changes had made it obsolete.
Note - it is not an issue of fixing the test, but rather
that the tested functionality is no longer there.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-12-01 08:48:49 -06:00
Casey Bodley
b7e1c1d59c common: add generate_test_instances() for std::variant
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-12-01 08:52:31 -05:00
Casey Bodley
8e92cbac3e common: add versioned encodings for std::variant
adds two encoding strategies for `std::variant<>` under the namespaces
`ceph::versioned_variant` and `ceph::converted_variant`

these versioned encodings allow the variant to be extended with new
types, provided that they're always added to the end without changing
or removing existing types. because of this requirement, no default
encoding is provided for `std::variant`. callers must opt in to one
namespace or the other

the `converted_variant` encoding requires the variant's first type T
to use versioned encoding, and guarantees that the variant's encoding
is backward-compatible with T's

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-12-01 08:52:31 -05:00
Nizamudeen A
4689c3d65e mgr/dashboard: fs snapshots e2e
Fixes: https://tracker.ceph.com/issues/63237
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-12-01 18:24:29 +05:30
Nizamudeen A
b35be54ed9 mgr/dashboard: cephfs subvolume list snapshots
Added a tab for displaying the subvolume snapshots
- this tab will show an info alert when there are no subvolumes present
- if the subvolume is present, then it'll be auto-selected by default

Implemented a filter to search the groups and subvolumes by its name.
Also added a scrollbar when there are too many items in the nav list

Modified the REST APIs to fetch only the names of the resources and
fetch the info when an API call is requesting for it.

Added unit tests

Fixes: https://tracker.ceph.com/issues/63237
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-12-01 18:24:28 +05:30
Nizamudeen A
122c6b2170
Merge pull request #54648 from rhcs-dashboard/fs-rename-fix
mgr/dashboard: fs rename only when fs is offline 

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
2023-12-01 18:21:49 +05:30
zdover23
a34821aaa4
Merge pull request #54731 from zdover23/wip-doc-2023-11-30-rados-tshooting-log-and-debug-note
doc/rados: improve formatting of log-and-debug.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-12-01 11:43:24 +01:00
zdover23
9a6c468bc3
Merge pull request #54730 from zdover23/wip-doc-2023-11-30-glossary-monitor-store
doc/glossary.rst: add "Monitor Store"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-12-01 10:44:41 +01:00
Zac Dover
e63fa697b7 doc/rados: improve formatting of log-and-debug.rst
Improve the arrangment of information in the section "Ceph Subsystems"
in doc/rados/troubleshooting/log-and-debug.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-12-01 10:35:42 +01:00
Ilya Dryomov
cb16d96e83
Merge pull request #54724 from ajarr/wip-63673
qa/workunits/rbd/cli_generic.sh: narrow race window

... when checking that rbd_support module command fails after blocklisting the module's client.

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-12-01 10:33:10 +01:00
Nizamudeen A
2c8faf256d mgr/dashboard: generalized code-block component
Fixes: https://tracker.ceph.com/issues/63608
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-12-01 13:49:48 +05:30
John Mulligan
ed6f171ba0 cephadm: black format daemons/tracing.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
b0f1c7fff3 cephadm: black format daemons/snmp.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
bc0c9d5422 cephadm: black format daemons/nvmeof.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
192ff2f6ef cephadm: black format daemons/nfs.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
8535d57b3b cephadm: black format daemons/monitoring.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
196ad1855e cephadm: black format daemons/iscsi.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
505624de9f cephadm: black format daemons/ingress.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
c791915164 cephadm: black format daemons/custom.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
a7f5b0c20b cephadm: black format daemons/ceph.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
ddee9492b4 cephadm: convert all deploy tests to use funkypatch fixture
During the refactor of various daemon type classes some of the tests had
been converted to funkypatch in order to deal with imports occuring over
multiple files. However, this conversion was done piece by piece in
order to make clear what was changing. This left the functions in this
file inconsistent. Change all the remaining function to use funkypatch
for consistency.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
13baad75f5 cephadm: sort imports import daemons/__init__.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
31ba507cc7 cephadm: move ceph classes to a new file
Move the ceph classes (Ceph, OSD, CephExporter) along with a few heavily
linked functions to a new ceph.py file under the daemons dir.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
63a14afb5c cephadm: move snmp class to a new file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
643dd5d0f6 cephadm: move monitoring class to a new file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00
John Mulligan
0a6a1dfa96 cephadm: move nfs class to a new file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-30 16:55:59 -05:00