Commit Graph

140326 Commits

Author SHA1 Message Date
Patrick Donnelly
0737936248
Merge PR #53149 into main
* refs/pull/53149/head:
	qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-09-20 08:31:17 -04:00
Venky Shankar
20d027eb9d doc: add note for removing (automatic) partitioning policy
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-09-20 15:49:19 +05:30
Guillaume Abrioux
ed2bd84014
Merge pull request #53309 from guits/bz2203397
ceph-volume: fix mpath device support
2023-09-20 09:32:49 +02:00
Ronen Friedman
c873c3973a osd/scrub: handle configuration changes in OsdScrub
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
0803780d5e osd/scrub: move initiate_a_scrub() to OsdScrub
Scrub initiation is now fully owned by OsdScrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
d54c579e18 osd/scrub: add ScrubQueue::ready_to_scrub()
At this phase of the refactoring:
this is the main interface from the scrub scheduler in OsdScrub
to the ScrubQueue. The ScrubQueue provides the ordered list of
all targets (for now - PGs) that are ready for scrubbing.

Scrub initiation code is modified to use the new interface.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
4959d4691b osd/scrub: move scrub_sleep_time() to OsdScrub
also scrub_time_permit().

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
0af4fc4261 osd/scrub: extract restrictions_on_scrubbing()
from ScrubQueue::select_pg_and_scrub().

Clearing the path to moving some ScrubQueue methods into
OscScrub. Starting here with the CPU load tracker.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
08d435e6dd osd/scrub: introduce OsdService::get_locked_pg()
which returns an RAII-wrapper around a locked PG.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
60deaa45f2 osd/scrub: move OSD::sched_scrub() to OsdScrub
... (as OsdScrub::initiate_scrub()).

The random backoff dice roller (scrub_random_backoff())
is moved as well.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
e0551a743c osd/scrub: declare OsdScrub, an OSD subobject
for all OSD scrub things.

For now: OsdScrub is mostly a forwarder to the ScrubQueue object
(which it now owns).
The resource counters moved into a separate object within OsdScrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
61af748a62 osd/scrub: unify dout macros across scrub scheduling code
to facilitate easy migration of code fragments between
related classes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
012db1ae0d osd/scrub: complete ScrubJob transition from within ScrubQueue
ScrubJob is now in the Scrub namespace.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
f3bd9b621c osd/scrub: set_reserving_now() signature modified
set_reserving_now() can now return a failure status, indicating
a race between two PGs to start scrubbing on the same OSD.

The scrubber FSM is modified to handle the failure.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
0bf1b29513 osd/scrub: select_pg_and_scrub() moved into osd_scrub.cc
No code changes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
5e4956cbae osd/scrub: renaming & fmt support for restrictions structure
Renaming ScrubPreconds, the collection of "environmental"
restrictions on possible scrubs, to OSDRestrictions.
Also - providing fmtlib support for that structure.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
46d291a7fc osd/scrub: introducing random_bool_with_probability()
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
de273640fb osd/scrub: moving the resources counters code into a separate file
No code changes in this commit.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
000bf250b7 osd/scrub: moving code as-is into osd_scrub.cc
Code from OSD.cc & osd_scrub_sched.cc moved as-is,
to be modified in followup commits.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
6db324700f osd/scrub: moving ScrubJob declaration as-is
from osd_scrub_sched.h into scrub_job.h.

A purely mechanical step. No code is changed.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Zhang Song
07577e3bc1 crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
2023-09-20 11:38:24 +08:00
John Mulligan
abaeed1a20 doc/cephadm: document new cephadm logging destination settings
Add docs for setting the binary's log destination at cephadm bootstrap
or on a running cluster.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:17:24 -04:00
John Mulligan
d5e67783b7 doc/cephadm: clarify what cephadm component writes to the cluster log channel
Clarify that the cephadm orchestrator module, a part of the ceph mgr,
logs to the cluster log channel. This prepares for adding a specific
section to cover logging for the cephadm "binary".

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:17:24 -04:00
John Mulligan
4d5502c7b4 cephadm: remember log destination used during bootstrap
Store the log destination(s) specified on the CLI for cephadm bootstrap
as the manager configuration, unless the configuration key is explicitly
set by the input config.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:17:24 -04:00
John Mulligan
f550ca9850 mgr/cephadm: add a module option for controlling cephadm log dest
Now that cephadm has multiple possible persistent logging destinations
we need a way to choose which one to use when the command is started by
the mgr. Add the option 'cephadm_log_destination' which can take one
of 'file', 'syslog', or 'file,syslog'. If left unset (empty string)
then the behavior is equivalent to 'file' and that is the same as
previous cephadm versions.

Fixes: https://tracker.ceph.com/issues/62233

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:17:24 -04:00
John Mulligan
bb7225f4c4 cephadm: add cli option to enable logging to syslog
Add the --log-dest option to cephadm. The --log-dest option can be
specified 0, 1 or more times. If unspecified, cephadm will log to
the default location, the log file. If specified one ore more times,
each instance will enable the named logging destination.
Example:

```
cephadm boostrap

cephadm --log-dest=syslog bootstrap

cephadm --log-dest=file bootstrap

cephadm --log-dest=syslog --log-dest=file bootstrap
```

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:17:24 -04:00
John Mulligan
81dedb100b cephadm: add support for logging to syslog/journal
Add support to logging.py for persistent logging to syslog and thus to
journald. This is accomplished by switching logging handlers depending
on the log_dest attribute of the context. Setting this value is left
for a future patch.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:17:16 -04:00
John Mulligan
8707879408 cephadm: move colored output support into logging.py
Rewrite cephadm's colored output support such that it abstracts away
the colorization into extra logging metadata. The new code will not
unconditionally put control characters into the log files. It will
only print the control chars if the stderr is a tty.
In theory this is probably more future proof as well, but it's only
got two callers so it is hard to say how useful it'll be.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-19 16:16:55 -04:00
Casey Bodley
04d9ea6daf rgw/crypt: don't deref null manifest_bl
with dbstore, the manifest_bl pointer was null; check for null before
dereferencing for read_manifest_parts()

Fixes: https://tracker.ceph.com/issues/62378

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-19 14:59:19 -04:00
Kamoltat (Junior) Sirivadhna
24d95fda23
Merge pull request #51921 from kamoltat/wip-ksirivad-fix-54136
pybind/mgr/pg_autoscaler: Use bytes_used for actual_raw_used
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2023-09-19 14:12:35 -04:00
Ilya Dryomov
523831912b
Merge pull request #53532 from zdover23/wip-doc-2023-09-19-man-ceph-monstore-tool
doc/man: s/kvstore-tool/monstore-tool/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-19 16:35:18 +02:00
Zac Dover
231afc8313 doc/man: s/kvstore-tool/monstore-tool/
s/kvstore-tool/monstore-tool/ in accordance with Ilya's remark here:
https://github.com/ceph/ceph/pull/53450#discussion_r1329804085

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-19 23:12:34 +10:00
Nizamudeen A
e068b2aabd
Merge pull request #53411 from rhcs-dashboard/align-charts
mgr/dashboard: align charts of landing page

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-09-19 18:08:29 +05:30
Guillaume Abrioux
28a12f8ef7 ceph-volume: fix mpath device support
commit [1] broke mpath devices support in `disk.is_device()`

[1] 4fc6bc394d

Fixes: https://tracker.ceph.com/issues/62722
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-09-19 14:25:00 +02:00
Yingxin
080bb4ffba
Merge pull request #53230 from myoungwon/fix-cbj-overflow-bug
crimson/os/seastore/cbj: fix a potential overflow bug on segment_seq

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2023-09-19 16:03:34 +08:00
Milind Changire
096e82c19b
Merge PR #52631 into main
* refs/pull/52631/head:
	mds: add debug logs to monitor ceph.dir.subvolume management
	mds: dump subvolume flag for inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-19 13:21:09 +05:30
Milind Changire
29eb1ca580
Merge PR #52892 into main
* refs/pull/52892/head:
	qa: add test to validate periodic checks by async threads
	mgr/volumes: periodically check for async work

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2023-09-19 12:53:29 +05:30
Milind Changire
7f0df59ac2
Merge PR #52755 into main
* refs/pull/52755/head:
	mds: adjust pre_segments_size for MDLog when trimming segments for standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2023-09-19 12:52:48 +05:30
Milind Changire
a94089bde2
Merge PR #52692 into main
* refs/pull/52692/head:
	qa/tasks/cephfs: reset the client_inject_fixed_oldest_tid after test

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-19 12:51:52 +05:30
zdover23
132cab7162
Merge pull request #53518 from likid0/patch-2
doc/dev: Fix typos in cephfs-mirroring.rst and  deduplication.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-09-19 09:04:02 +10:00
Daniel Parkes
06358e8d17 doc/dev: Fix typos in files cephfs-mirroring.rst and deduplication.rst
Typo Error in Doc cephfs-mirroring.rst , replace RAODS with RADOS
Typo Error in Doc deduplication.rst , replace RAODS with RADOS

Signed-off-by: Daniel Parkes <dparkes@redhat.com>
2023-09-19 00:39:40 +02:00
Casey Bodley
d8b0a9112c
Merge pull request #53502 from dang/wip-dang-cls-test
RGW - Fix cls test build on new gcc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-18 23:01:31 +01:00
Daniel Gryniewicz
38f11f28b0
Merge pull request #53508 from dang/wip-dang-posix-cache
RGW - Add wait backoff to posix bucket cache test

Reviewed-by Matt Benjamin <mbenjamin@redhat.com>
2023-09-18 14:36:58 -04:00
Daniel Gryniewicz
c7d2c740b0 RGW - Add wait backoff to posix bucket cache test
The CI appears to be really slow, and even a second of wait for inotify
sometimes fails.  Add an exponential backoff wait of up to ~25 seconds
to hopefully make the test pass reliably.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2023-09-18 11:47:54 -04:00
Casey Bodley
3ad17ed3b2 rgw/sal: get_placement_target_names() returns void
the function returned an integer error code, but two callers were
incorrectly testing the return value as a boolean

the function just returns placement ids that are in-memory, so none of
the drivers have a failure case; change the return value to void

Fixes: https://tracker.ceph.com/issues/62771

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-18 11:15:04 -04:00
Casey Bodley
9b46b9b0a0
Merge pull request #53478 from EdwardVitor/cuiming_chinamobile
auth:rectify a cmake compilation warning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-18 14:46:55 +01:00
Daniel Gryniewicz
8372e80d52 RGW - Fix cls test build on new gcc
The new encoder types broke building the cls test on newer gcc (13+) due
to undefined encoder/decoder.  Add the file that defines those to the
test.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2023-09-18 09:04:43 -04:00
Patrick Donnelly
6c993ee3bb
Merge PR #53175 into main
* refs/pull/53175/head:
	qa: increase the http postBuffer size and disable sslVerify

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-18 08:37:16 -04:00
cuiming_yewu
800c148d2e auth:rectify a cmake compilation warning
rectify src/auth/cephx/CephxProtocol.cc 1 warning
with the variable 'ch' Used before initialized
auth/cephx/CephxProtocol.cc:595:57: warning: '*((void*)& ch +8)' may be used uninitialized in this function [-Wmaybe-uninitialized]
     msg.server_challenge_plus_one = ch.server_challenge + 1;
                                     ~~~~~~~~~~~~~~~~~~~~^~~

Signed-off-by: cuiming <cuiming_yewu@cmss.chinamobile.com>
2023-09-18 15:30:06 +08:00
Anthony D'Atri
245f8af8c4
Merge pull request #53490 from zdover23/wip-doc-2023-09-17-architecture-6-of-x
doc/architecture: "Edit HA Auth" (one of several)
2023-09-17 10:40:26 -04:00