Commit Graph

139815 Commits

Author SHA1 Message Date
Pedro Gonzalez Gomez
397bb61d0b mgr/dashboard: remove unnecessary failing hosts e2e
These host e2e test were failing, since we are already checking this on the Dashboard Cephadm e2e tests we can get rid of these ones.

Fixes: https://tracker.ceph.com/issues/62491
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-08-24 20:34:17 +02:00
Casey Bodley
4293d9bf11 qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist
Fixes: https://tracker.ceph.com/issues/62504

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-08-24 12:29:09 -04:00
Ali Masarwa
5b94a7d1af
Merge pull request #53039 from AliMasarweh/wip-alimasa-persistant-q-enhance-fix
RGW | Bucket Notification: Fix the issue of miscounting the entries to remove

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-08-24 18:39:45 +03:00
Nizamudeen A
b2c00675f4 mgr/dashboard: cephfs subvolume management e2e tests
includes subvolume and subvolume groups e2es

Also taking care of renaming of Volume to File Systems in the remaining
actions like Edit and Remove

Fixes: https://tracker.ceph.com/issues/62564
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-08-24 20:11:51 +05:30
Pedro Gonzalez Gomez
210af39b19
Merge pull request #53091 from rhcs-dashboard/fix-62453-main
mgr/dashboard: Create realm sets to default

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-24 16:33:03 +02:00
Patrick Donnelly
3a5f9dc37a
doc: expand and consolidate mds placement
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-08-24 08:55:47 -04:00
Venky Shankar
6639465452 Merge PR #52575 into main
* refs/pull/52575/head:
	ceph_fs.h: add separate owner_{u,g}id fields

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-08-24 17:36:53 +05:30
Venky Shankar
726e5d7dde Merge PR #52676 into main
* refs/pull/52676/head:
	mds/Server: mark a cap acquisition throttle event in the request

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-08-24 17:35:04 +05:30
avanthakkar
16e913f0ba mgr/dashboard: disable protect if layering is not enabled on the image
Fixes: https://tracker.ceph.com/issues/62498
Signed-off-by: avanthakkar <avanjohn@gmail.com>
2023-08-24 16:08:41 +05:30
Nizamudeen A
30b31083d4 mgr/dashboard: cleanup behave logics
Fixes: https://tracker.ceph.com/issues/62564
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-08-24 15:20:09 +05:30
Nizamudeen A
117d5566db
Merge pull request #52829 from rhcs-dashboard/cephfs-e2e-test
mgr/dashboard: add e2e tests for cephfs management 

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2023-08-24 15:17:38 +05:30
Aashish Sharma
00d3ead5ee mgr/dashboard: Create realm sets to default
In Multisite page, When we create a realm the realm sets to default even if some other realm is already default and default checkbox in unchecked as well while creating.

Fixes: https://tracker.ceph.com/issues/62453

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2023-08-24 14:30:52 +05:30
Nizamudeen A
62709d0a61
Merge pull request #52895 from cloudbehl/minor-improvements
mgr/dashboard: Added some minor usability improvements

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-24 13:59:04 +05:30
Yingxin
3e0185fa56
Merge pull request #52896 from cyx1231st/wip-seastar-multi-core-msgr-UT
crimson/net: consolidate messenger implementations and enable multi-shard UTs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2023-08-24 15:37:59 +08:00
Matan Breizman
89a571f7f3
Merge pull request #53083 from Matan-B/wip-matanb-crimson-centos9-supp
crimson/osd/lsan_suppressions: add MallocExtension::Register

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-08-24 10:05:19 +03:00
zdover23
293c640d74
Merge pull request #53117 from batrick/doc-name-fix
doc: correct option name

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-24 15:13:23 +10:00
Radoslaw Zarzynski
ce9771bbeb
Merge pull request #53104 from ceph/revert-49428-maintain_prefix_itr
Revert "osd/SnapMapper: Maintain the prefix_itr between calls to avoid search…"

Reviewed-by: Gabriel BenHanokh <gbenhano@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2023-08-24 00:36:46 +02:00
Patrick Donnelly
94129fc126
doc: correct option name
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-08-23 16:23:46 -04:00
Radoslaw Zarzynski
b77ebe8cd4 Revert "osd/SnapMapper: Maintain the prefix_itr between calls to avoid search…"
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-08-23 20:12:11 +02:00
Patrick Donnelly
4a6f6ed0f3
Merge PR #53053 into main
* refs/pull/53053/head:
	libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-08-23 13:57:25 -04:00
Gabriel Benhanokh
46974c9024
Merge pull request #49428 from benhanokh/maintain_prefix_itr
osd/SnapMapper: Maintain the prefix_itr between calls to avoid search…
2023-08-23 18:34:22 +03:00
Gabriel BenHanokh
690739e821 osd/SnapMapper:
Maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim() to prevent searching depleted prefixes.
We got 8 distinct hash prefixes used for searching objects owned by a given PG.
On each call to SnapMapper::get_next_objects_to_trim() we start from the first prefix even after all objects mapped to it were depleted.
This means that we will be searching for 1 non-existing prefix after the first prefix was depleted, 2 after the first two prefixes were depleted... and so on until we will search 7 non-existing prefixes after the first 7 prefixes were depleted.

This is a performance improvement PR only!
It maintains the existing behavior and does not try to fix/change any of the TRIM logic.
I added an extra step after the last object is trimmed doing a full scan of the DB and only if no object was found it will return ENOENT.
This should make the new code no-worse than existing code which returns ENOENT after a full scan found no object.
It should not impact performance in real life snaps as it should only happen once per-snap.

added snap-mapper tests to rados-test-suite
disabled osd_debug_trim_objects when running (SnapMapperTest, prefix_itr) to prevent asserts(as this code does illegal inserts into DELETED snaps)
Code beautifing

Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
2023-08-23 13:47:45 +00:00
zdover23
dbdf6861a0
Merge pull request #53076 from zdover23/wip-doc-2023-08-22-start-os-recommendations-chart
doc/start: refactor ABC test chart

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-23 21:47:51 +10:00
Venky Shankar
b8bf0c62b9 Merge PR #53077 into main
* refs/pull/53077/head:
	qa: add `POOL_APP_NOT_ENABLED` to ignorelist for cephfs tests

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-08-23 11:07:24 +05:30
Yingxin
fe07f64396
Merge pull request #53051 from xxhdx1985126/wip-crimson-omap-list
crimson/os/seastore/omap_manager: fix the entry leak issue in BtreeOMapManager::omap_list()

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 13:11:08 +08:00
Yingxin Cheng
e44ce1a892 test/crimson: increase smp settings for test_socket and test_messenger
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:09 +08:00
Yingxin Cheng
ec8722f212 test/crimson/test_messenger: add delay to suite markdown
markdown() needs delay to be propagated to the primary core.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:09 +08:00
Yingxin Cheng
290dd2d68d test/crimson/test_messenger: enable multi-core messenger in FailoverSuite
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:09 +08:00
Yingxin Cheng
d65ba2d018 test/crimson/test_messenger: support cross-core protocol-level tests
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:09 +08:00
Yingxin Cheng
bcf20d6771 crimson/net: support cross-core blocking for interceptor
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:09 +08:00
Yingxin Cheng
74e9098833 crimson/net: should not check shutdown from a different core
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:09 +08:00
Yingxin Cheng
6be5e630ed crimson/net: rework interceptor to support cross-core interceptions
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-23 12:49:03 +08:00
Patrick Donnelly
d94684c1ee
libcephsqlite: fill 0s in unread portion of buffer
Unfortunately, this code is filling 0s at the beginning of the short-read
buffer.

Fixes: https://tracker.ceph.com/issues/62492
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-08-22 20:53:11 -04:00
Yuri Weinstein
aa04f4a164
Merge pull request #52984 from tchaikov/cmake-python3
cmake: update FindPython3 module

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zack Cerza <zack@redhat.com>
2023-08-22 11:34:10 -07:00
Yuri Weinstein
b571b3c828
Merge pull request #52904 from ronen-fr/wip-rf-ost-fmt
commom,osd: support for default fmtlib formatters

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-22 11:32:16 -07:00
Yuri Weinstein
8394bd6c83
Merge pull request #52809 from sseshasa/wip-mclock-client-lim-62293
osd/scheduler/mClockScheduler: Use same profile and client ids for all clients to ensure allocated QoS limit consumption.

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-08-22 11:31:22 -07:00
Yuri Weinstein
cdddefd2f9
Merge pull request #52339 from Matan-B/wip-matanb-trim-maps-refactor
osd/OSD: Simplify trim_maps

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-08-22 11:31:05 -07:00
Yuri Weinstein
96dbb66c13
Merge pull request #51868 from NitzanMordhai/wip-nitzan-admin-socket-new-session-mon-shutdown
mon/Monitor: during shutdown don't accept new authentication and crea…


Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-08-22 11:30:37 -07:00
Yuri Weinstein
93961385d7
Merge pull request #50216 from rhcs-dashboard/log-runway
os/bluestore: expand BlueFS log if available space is insufficient

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2023-08-22 11:30:17 -07:00
Ronen Friedman
d493aea03a
Merge pull request #53038 from ronen-fr/wip-rf-max-utime
common: add a utime_t::max constant

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-08-22 21:16:50 +03:00
Adam King
cb742261f9
Merge pull request #50200 from adk3798/drain-leave-etc-ceph
mgr/cephadm: allow draining host without removing conf/keyring files

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-08-22 12:31:33 -04:00
Adam King
4aebb55ba8
Merge pull request #53008 from adk3798/cephadm-bind-to-specific-ip
mgr/cephadm: allow ingress services with same frontend port and different VIP

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-08-22 12:29:27 -04:00
Adam King
b2f133fd99 cephadm: have attempt_bind raise up OSErrors
Before it was always converting the OSError to
our self-defined "Error" class. This causes an issue
with the port_in_use function that has special handling
for OSError when the errno is EADDRNOTAVAIL or
EAFNOSUPPORT. Since the error being raised was no
longer an OSError it wasn't being caught and checked
properly in port_in_use.

This has the additional property of being necessary
to check port availability for haproxy on its VIP. If
we fail deployment when EADDRNOTAVAIL is raised, it becomes
difficult to deploy the ingress service. If we deploy
haproxy first it fails because the VIP isn't available
yet (since keepalive isn't up) and it fails saying the port
it wants to bind to is unavailable (specifically EADDRNOTAVAIL).
If we try to deploy keepalive first it fails because it
needs to know the location of the haproxy daemons in
order to build its config file. This has worked in the past
by just having the haproxy fail to bind at first and then
fix itself once the keepalive daemon is deployed. That
no longer works if the haproxy daemon fails to deploy
because cephadm is reporting the port it needs is
unavailable. Since EADDRNOTAVAIL when deploying
haproxy likely means the VIP is not up rather than
something else is taking up the port it needs, fixing
the handling of this allows ingress deployment to
work while also allowing multiple haproxy daemons
on the same host to use the same frontend port
bound to different VIPs.

Signed-off-by: Adam King <adking@redhat.com>
2023-08-22 10:05:21 -04:00
Adam King
640af15598 mgr/cephadm: only check haproxy frontend_port conflicts on VIP
If we know what IP the frontend_port will be binding
to, we can pass that down through the port_ips mapping
so cephadm will only check if that port on that specific
VIP if in use. This allows multiple haproxy daemons
to be bound to the same port on different VIPs on the
same host.

Note that you still must use a different monitor port
for the two different ingress services as that port
is bound to on the actual IP of the host. Only the
frontend port can be the same for haproxies on the
same host as long as the VIP is different.

Fixes: https://tracker.ceph.com/issues/57614

Signed-off-by: Adam King <adking@redhat.com>
2023-08-22 10:05:21 -04:00
Adam King
178cbdab33 cephadm: support for specifying IP a port will be bound to on
This is mostly for checking for port conflicts.
Currently, we just check if the port is bound to
on any IP on the host. This mechanism should allow
certain daemon types to specify a port -> IP mapping
that will be passed to the cephadm binary. That
mapping will then be used by cephadm to only
check for the port being bound to on that specific
IP rather than any IP on the host. The end result
is we could have daemons bound to the same
port on different IPs on the same node.

It's expected that daemon types will set this
up as part of their prepare_create or generate_config
functions where they may have more info about the
specific IPs and ports they need.

Signed-off-by: Adam King <adking@redhat.com>
2023-08-22 10:05:21 -04:00
Matan Breizman
bc190972fa crimson/osd/lsan_suppressions: add MallocExtension::Register
```
=================================================================
==80592==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x7f5c76eb6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x7f5c76a2fb81 in MallocExtension::Register(MallocExtension*) (/lib64/libtcmalloc.so.4+0x2fb81)

SUMMARY: AddressSanitizer: 8 byte(s) leaked in 1 allocation(s)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-08-22 13:14:48 +00:00
Ronen Friedman
403a073c55 common: add a utime_t::max constant
a convenient init value when searching for some early timepoint.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-08-22 16:09:28 +03:00
Aashish Sharma
267a9351ec
Merge pull request #52700 from cloudbehl/utilization-improvements
mgr/dashboard: Adding legends and fixing queries

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2023-08-22 18:32:28 +05:30
Ronen Friedman
e8ada7c65a
Merge pull request #53040 from ronen-fr/wip-rf-conf-minor
osd: minor changes to the descriptions of some configuration items

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-08-22 15:52:45 +03:00
Rishabh Dave
00e4e6996b
Merge pull request #53036 from ceph/revert-52416-ior-make-failures
Revert "qa/cephfs: fix ior build failure"

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-08-22 15:16:42 +05:30