Commit Graph

142301 Commits

Author SHA1 Message Date
Aashish Sharma
41ff2bdacd
Merge pull request #62087 from aaSharma14/wip-70252-reef
reef: mgr: fix subuser creation via dashboard

Reviewed-by: Nizamudeen A <nia@redhat.com>
2025-03-17 11:22:29 +05:30
Zac Dover
7f91f1e933
Merge pull request #62321 from zdover23/wip-doc-2025-03-15-backport-62319-to-reef
reef: doc/rados/troubleshooting: Improve troubleshooting-pg.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2025-03-15 13:23:53 +10:00
Anthony D'Atri
9ebc3a0c15 doc/rados/troubleshooting: Improve troubleshooting-pg.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 56a21cbc03)
2025-03-15 12:57:45 +10:00
Anthony D'Atri
ad78e0a9da
Merge pull request #62318 from zdover23/wip-doc-2025-03-15-backport-62316-to-reef
reef: doc/rados/operations: improve crush-map-edits.rst
2025-03-14 22:47:09 -04:00
Anthony D'Atri
2c10e50df6 doc/rados/operations: improve crush-map-edits.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 59a894713a)
2025-03-15 12:23:52 +10:00
Yuri Weinstein
5ea1c5072e
Merge pull request #62218 from idryomov/wip-66419-reef
reef: qa/workunits/rbd: wait for resize to be applied in rbd-nbd

Reviewed-by: Ramana Raja <rraja@redhat.com>
2025-03-14 13:23:42 -07:00
Laura Flores
476b47e6fc
Merge pull request #60615 from YiteGu/wip-68792-reef
reef: blk/KernelDevice: using join() to wait thread end is more safe
2025-03-14 15:20:25 -05:00
Anthony D'Atri
1d08f2afa2
Merge pull request #62298 from zdover23/wip-doc-2025-03-14-backport-62119-to-reef
reef: doc: fixup #58689 - document SSE-C iam condition key
2025-03-14 08:18:45 -04:00
dawg
5c7f8f28d5 doc: fixup #58689 - document SSE-C iam condition key
Signed-off-by: dawg <code@dawg.eu>

(cherry picked from commit 7b4ac88662)
2025-03-14 17:09:49 +10:00
Anthony D'Atri
c95c527527
Merge pull request #62266 from zdover23/wip-doc-2025-03-13-backport-62249-to-reef
reef: doc/monitoring: Improve index.rst
2025-03-13 08:08:13 -04:00
afreen23
7e3bc10c1d
Merge pull request #59697 from rhcs-dashboard/wip-67928-reef
reef: qa/mgr/dashboard: fix test race condition  

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2025-03-13 16:09:23 +05:30
Hannes Baum
2036e87b91 mgr: fix subuser creation via dashboard
Subusers couldn't be created through the dashboard, because the get call was overwritten with Python magic due to it being the function under the HTTP call.
The get function was therefore split into an "external" and "internal" function, whereas one
can be used by functions without triggering the magic. Since the user object was then returned correctly, json.loads could be removed.

Signed-off-by: Hannes Baum <hannes.baum@cloudandheat.com>
(cherry picked from commit 90e221d0b5)
2025-03-13 14:09:22 +05:30
Nizamudeen A
2e1544e6be qa/tests: retry the api call after making the request
based on the pointer from Bill in https://tracker.ceph.com/issues/62972#note-75

Fixes: https://tracker.ceph.com/issues/62972
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 1588712b55)
2025-03-13 11:31:10 +05:30
Nizamudeen A
5cc7fc6806 qa/dashboard: fix test_list_enabled_module failure
Check the ports availability and go for a new port if the current one is
not available

Fixes: https://tracker.ceph.com/issues/62972
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit a2a4a3415c)
2025-03-13 11:31:05 +05:30
Nizamudeen A
c0955dc2ad qa/dashboard: fix tasks.mgr.dashboard.test_health.HealthTest
as per: https://tracker.ceph.com/issues/47612#note-14

Fixes: https://tracker.ceph.com/issues/47612
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 79d61bbb58)
2025-03-13 11:31:01 +05:30
Nizamudeen A
afba83d8af qa/tests: fix test_list_enabled_modules timeout error
This test deals with enabling/disabling the modules. The assumption I
have is after enabling the
module test will wait for an active mgr but its not able to find it in
time and it fails. so taking inspiration from 6c7253be6f adding retries and logs to see if that's the case

Fixes: https://tracker.ceph.com/issues/62972
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit b2da7394ee)
2025-03-13 11:30:57 +05:30
Ernesto Puerta
bfee4c6b17 qa/mgr/dashboard: fix test race condition
Fixes: https://tracker.ceph.com/issues/66844

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 6c7253be6f)
2025-03-13 11:30:49 +05:30
Nizamudeen A
6992f57173
Merge pull request #62256 from rhcs-dashboard/wip-70424-reef
reef: mgr/dashboard: pin lxml to fix run-dashboard-tox-make-check failure
2025-03-13 11:20:30 +05:30
Anthony D'Atri
f4e5aaf4b3 doc/monitoring: Improve index.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 1bc67295c8)
2025-03-13 14:56:03 +10:00
Nizamudeen A
d56a8afd4e mgr/dashboard: pin lxml to fix run-dashboard-tox-make-check failure
xmlsec had an upgrade yesterday night and python3-saml might need to
adapt its library accordingly I suppose. Testing a fix by pinning lxml

Another approach is being tried out separately
https://github.com/ceph/ceph/pull/62239, but that is failing with some
other errors.

Fixes: https://tracker.ceph.com/issues/70411
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 1f84505f12)

 Conflicts:
	src/pybind/mgr/dashboard/requirements.txt
 - only kept the lxml pinning. didn't add the newer deps that are
   present in main
2025-03-12 22:47:26 +05:30
Yuri Weinstein
1f7fb81be4
Merge pull request #57590 from NitzanMordhai/wip-66141-reef
reef: common/pick_address: check if address in subnet all public address

Reviewed-by: Laura Flores <lflores@redhat.com>
2025-03-12 07:33:48 -07:00
Igor Fedotov
ac2c2a8a3d
Merge pull request #62124 from ifed01/wip-ifed-fragmentation-command-reef
reef: tool/ceph-bluestore-tool: fix wrong keyword for 'free-fragmentation' …

Reviewed-by: akupczyk@ibm.com
2025-03-12 12:19:06 +03:00
Aashish Sharma
564ec6bcc3
Merge pull request #62209 from aaSharma14/wip-67936-reef
reef: mgr/dashboard: Fix variable capitalization in embedded rbd-details panel 

Reviewed-by: Naman Munet <naman.munet@ibm.com>
2025-03-12 12:42:05 +05:30
Adam Kupczyk
4c1966645f
Merge pull request #61892 from k0ste/wip-70068-reef
reef: os/bluestore: fix the problem that _estimate_log_size_N calculates the log size incorrectly
2025-03-11 19:33:12 +01:00
Ilya Dryomov
59f2c3af87 qa/workunits/rbd: wait for resize to be applied in rbd-nbd
Implement the same logic as in commit 6f3d0f570f ("test/librbd/fsx:
wait for resize to propagate in krbd_resize()").

Fixes: https://tracker.ceph.com/issues/66419
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit bedc75fff2)
2025-03-11 10:56:15 +01:00
Juan Ferrer Toribio
ec0695c4dd mgr/dashboard: Fix variable capitalization in embedded rbd-details panel
Fix capitalization of image and pool variables in embedded grafana rbd-details panel

Fixes: https://tracker.ceph.com/issues/67849

Signed-off-by: Juan Ferrer Toribio <22457707+juan-ferrer-toribio@users.noreply.github.com>
(cherry picked from commit dfca044b64)
2025-03-11 11:40:02 +05:30
Adam King
f3093595a3
Merge pull request #62162 from phlogistonjohn/wip-70345-reef
reef: build-with-container: fixes and enhancements

Reviewed-by: Adam King <adking@redhat.com>
2025-03-10 16:02:05 -04:00
Yuri Weinstein
55fb540837
Merge pull request #62175 from idryomov/wip-64063-reef
reef: rbd-nbd: use netlink interface by default

Reviewed-by: Ramana Raja <rraja@redhat.com>
2025-03-10 11:04:56 -04:00
Venky Shankar
16b9de4637
Merge pull request #61379 from vshankar/wip-reef-client-secfix
reef: client: disallow unprivileged users to escalate root privileges

Reviewed-by: Milind Changire <mchangir@redhat.com>
2025-03-10 17:47:17 +05:30
Anthony D'Atri
58f50f1271
Merge pull request #62193 from zdover23/wip-doc-2025-03-10-backport-62176-to-reef
reef: doc/releases: Add ordering comment to releases.yml
2025-03-10 00:41:43 -04:00
Anthony D'Atri
c5801f7ff8 doc/releases: Add ordering comment to releases.yml
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 2290a904bd)
2025-03-10 13:30:01 +10:00
Dan Mick
1f324f52fe
Merge pull request #62065 from dmick/wip-70241-reef
reef: container/build.sh: remove local container images
2025-03-07 16:17:31 -08:00
Yuri Weinstein
7d3ec84213
Merge pull request #62129 from idryomov/wip-65720-reef
reef: librbd: add rbd_diff_iterate3() API to take source snapshot by ID

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>
2025-03-07 15:24:43 -05:00
Yuri Weinstein
27a78e16cb
Merge pull request #62127 from idryomov/wip-70190-reef
reef: librbd: fix a deadlock on image_lock caused by Mirror::image_disable()

Reviewed-by: Vinay Bhaskar Varada <vvarada@redhat.com>
2025-03-07 15:23:19 -05:00
Ilya Dryomov
a2d3f3b764 Revert "test/librbd/fsx: switch to netlink interface for rbd-nbd"
This reverts commit 1a128a8d8c.

With commit fcbf7367d2 ("rbd-nbd: map using netlink interface by
default") backported to reef, this reef-only fixup limited to fsx is no
longer needed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2025-03-07 21:09:13 +01:00
Ramana Raja
b70160ac4d rbd-nbd: map using netlink interface by default
Mapping rbd images to nbd devices using ioctl interface is not
robust. It was discovered that the device size or the md5 checksum
of the nbd device was incorrect immediately after mapping using
ioctl method. When using the nbd netlink interface to map RBD images
the issue was not encountered. Switch to using nbd netlink interface
for mapping.

Fixes: https://tracker.ceph.com/issues/64063
Signed-off-by: Ramana Raja <rraja@redhat.com>
(cherry picked from commit fcbf7367d2)

Conflicts:
	PendingReleaseNotes [ moved to >=18.2.5 section ]
2025-03-07 21:05:56 +01:00
David Galloway
cdc9635303
Merge pull request #62037 from ceph/template-reef
Links to Jenkins jobs in PR comment commands / Remove deprecated commands
2025-03-07 14:42:36 -05:00
Ilya Dryomov
8d014f171f test/pybind/rbd: fix read offset in write zeroes tests
Random data is written and write zeroes is invoked on 0~256, but the
read is done on 256~256.  This means that if write zeroes malfunctions
the test wouldn't catch it (especially in the thick provision case).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d41f0fa01f)
2025-03-07 11:57:53 +01:00
VinayBhaskar-V
dd64132044 librbd: add rbd_diff_iterate3() API to take source snapshot by ID
Allow a diff to start from a non-user snapshot.  This would be used by
"rbd du" command to account for non-user snapshots which are currently
just skipped potentially resulting in underreported space usage and in
other places.

Fixes: https://tracker.ceph.com/issues/65720
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Vinay Bhaskar Varada <vvarada@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 54f47cc28f)

Conflicts:
	src/include/rbd/librbd.h [ commit e5ccce14c4 ("rbd: add group
	  snap info command") not in reef ]
	src/test/pybind/test_rbd.py [ commit d7fd66ec99 ("librbd: add
	  rbd_clone4() API to take parent snapshot by ID") not in reef ]
2025-03-07 11:57:53 +01:00
John Mulligan
c5748f0871 doc: document the new container build tool and link to it in README
Add a new markdown file in the root of the tree, ContainerBuild.md, that
can serve as a basic introduction to the new container build tools
recently merged to ceph.
Add a small 'breadcrumb' section to the project README.md to help find
this new document.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 313546146c)
2025-03-06 16:14:28 -05:00
John Mulligan
895ffc9989 script/build-with-container: add support for overlay dir
The source dir (aka homedir, default /ceph) is mounted in the container
read-write. This is needed as the various ceph build scripts expect to
write things into the tree - often this is in the build directory - but
not always. This can lead to small messes and/or situations that are
confusing to debug, especially if one is jumping between distros often.
Add an option to use an overlay volume for the homedir - by default we
enable a persistent overlay with a supplied "upper dir" where files that
were written will appear. One can also enable a temporary overlay that
forgets the writes when the container exits - maybe useful when doing
experiments in 'interactive' mode.

To use this option run the command with the `--overlay=<dir>` option.
For example: `./src/script/build-with-container.py -b build.inner
--overlay-dir build.ovr`. This will create a directory
`build.ovr/content` automatically and all new files will appear there.
For example the build directory will appear at
`build.ovr/content/build.inner`.

To use the temporary overlay use a `-` as the directory name. For
example: `./src/script/build-with-container.py -b build.inner
--overlay-dir -`

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 794e3d0b25)
2025-03-06 16:14:28 -05:00
John Mulligan
08f7626d72 script/build-with-container: skip dnf cache dir volume mounts on docker
When using docker the --volume option is not available during build
(docker [buildx] build), unlike podman. Since passing these volumes must
be conditional on them being set up I see no way to handle this short of
just disabling the option on docker. Log the fact that it's being
skipped - the only other issue is that we pointlessly set up some dirs
and the build may be a bit slower.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 4208a73665)
2025-03-06 16:14:27 -05:00
John Mulligan
e56df19133 script/build-with-container: remove default --volume arg from ctr build
On the original github pr #59841 user fayak kindly informed us that the
--volume option was not supported by docker build. Since this section
was a leftover from a previous way of constructing the builder image and
was no longer needed we simply removed it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 612a9d6808)
2025-03-06 16:14:27 -05:00
John Mulligan
915f6210d7 script/build-with-container.py: build builder image with --pull=always
Construct the builder image using the --pull=always flag to initiate a
pull of the base image (centos, ubuntu, etc) in order to avoid using a
stale base image. Since the script automatically (by default) avoids
building if a matching tag is in local container storage it is handy to
use a fresh base when it *is* time to build something. Otherwise, you
end up in a situation like I sometimes do - using a months old base
unintentionally.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit f6e6188e30)
2025-03-06 16:14:27 -05:00
John Mulligan
de0d927944 script/build-with-container: add a common packages target
Add a `packages` target to build-with-container.py that requests a build
of packages, whatever package type is native to the distro selected.
For example `./src/script/build-with-container.py -d ubuntu22.04 -e
packages` will automatically select a deb packages build where
`./src/script/build-with-container.py -d centos9 -e packages` will
trigger rpm packages to be built. The underlying package-type specific
targets remain unchanged.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 37b7d509c5)
2025-03-06 16:14:27 -05:00
John Mulligan
5d7ff135f6 script/build-with-container: support custom tag suffixes
Previously, one could use the `--tag` option to completely override the
container tag generated by the script. However, there are cases where
one may want to add information to the tag rather than override it.
Allow the tag value to start with a plus (+) character that indicates
that the remainder of the string is to be suffixed to the generated tag.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 30836c4ed4)
2025-03-06 16:14:27 -05:00
John Mulligan
8a3e3af0fa script/build-with-container: add --base-branch cli option
Add a command line option --base-branch that allows the user to supply a
custom base branch name. git doesn't make determining this easy so we
always assume a base branch of 'main' by default - but this option lets
one change that.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit ff34bf7241)
2025-03-06 16:14:27 -05:00
John Mulligan
4304f848e6 src/script: rename CEPH_BRANCH to CEPH_BASE_BRANCH for build container
Previously, we were passing build argument of CEPH_BRANCH, but that was
a bit misleading as we expect the current branch to vary a bit (as users
will be using branches to develop and test the code). What we actually
care about is the base branch ('main', 'squid', etc) as that is fed into
our bootstrap script and we want the option to simple variations based
on the name of said base branch.
Rename CEPH_BRANCH to CEPH_BASE_BRANCH for clarity.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit a1d49d557c)
2025-03-06 16:14:27 -05:00
John Mulligan
98be912957 script/build-with-container: add --current-branch cli option
Add a new --current-branch argument that lets the user supply a name for
the current branch. This allows the automatic tag generation to avoid
calling git - something useful if the tree is not using a git checkout
(like a tarball). It also allows you to pull a temporary branch in git
but ignore it and act like the temporary branch is the base branch.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit c1713c5bc3)
2025-03-06 16:14:27 -05:00
John Mulligan
0adecbe761 script/build-with-container: add more distro aliases
Add a system to define distro name aliases and use that to define some
additional aliases, primarily to match ubuntu codenames rather than
version numbers. Requested by Zack.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 65f055f0d8)
2025-03-06 16:14:27 -05:00