Commit Graph

140430 Commits

Author SHA1 Message Date
Yuval Lifshitz
94ac81981f
Merge pull request #52392 from yuvalif/wip-yuval-trace-name
rgw: rename request traces and change for tags

reviewed-by: cbodley
2023-09-21 20:12:01 +03:00
Patrick Donnelly
ab3e5ba744
Merge PR #50503 into main
* refs/pull/50503/head:
	mon: do not change pending if strategy is unchanged
	mon/MonmapMonitor: do not propose on error in prepare_update
	mon/MonmapMonitor: wait for commit before reply
	mon: use wait_for_commit to reply
	mon: add context list for commit wait
	mon: remove unused method
	test/mon: add commit benchmark script
	mon/MonClient: provide config to target specific rank

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-09-21 11:51:31 -04:00
Zac Dover
78804e1aad doc/architecture: "Edit HA Auth" (one of several)
Edit "High Availability Authentication" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-22 00:10:12 +10:00
Mark Kogan
03616b4330 valgrind: UninitCondition under __run_exit_handlers suppression
reqiered in CentOS / RHEL 9 & Ubuntu 22.04.1 LTS

Fixes: https://tracker.ceph.com/issues/62141

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2023-09-21 16:55:30 +03:00
John Mulligan
18e4552502 cephadm: black format container_types.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-21 09:47:34 -04:00
John Mulligan
934fe80d75 cephadm: black format systemd.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-21 09:47:34 -04:00
John Mulligan
6c7d043378 cephadm: move CephContainer/similar to new container_types.py
Part of general cephadm split-up refactoring. I am not happy with the
name 'container_types' but none of the alternatives I could think
of were much better.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-21 09:47:34 -04:00
John Mulligan
aa01e9d0b1 cephadm: move a pair of systemd unit status funcs to systemd.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-21 09:47:32 -04:00
John Mulligan
1a5a61d78f cephadm: fix unit tests executing FileLock type
The FileLock type doesn't play much of a role when running tests so
to prevent issues, always mock it out when using with_cephadm_ctx.

In particular, a future patch revealed a problem with the FileLock code
that I can not understand how it was not hit before, or why this simple
refactoring - not directly related to file locking - triggered it. But
in short, the FakeFilesystem mocking utility only covers some syscalls.
In fact, the fake filesystem was returning an fd that was then passed to
real calls (fcntl and os.close).  The latter then triggered issues when
pytest was trying to clean up after it applied it's magic to stdio
objects in sys. The fix is easy - understanding why it happens and how
was hard.  I still don't understand why it popped up when it did only
that this is necessary to implement the following patches.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-21 09:46:41 -04:00
Ronen Friedman
f7ddca67b6 osd/scrub: modify schedule_result_t to report error class
(which directly translates to the required followup action)
instead of reporting the exact failure. The specific of the failure
were never used by the scrub scheduler.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-21 07:57:58 -05:00
Ronen Friedman
693cdf3565 osd/scrub: scheduler refactoring - cleanups
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-21 07:57:58 -05:00
Rishabh Dave
8b62329195 qa/cephfs: add "set -x" in mdtest.yaml
Set the flag for printing the commands that will be executed so that
it's easier to go through teuthology.log

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-21 18:16:21 +05:30
Rishabh Dave
6a2e94b323 qa/cephfs: fix build failure for mdtest project
To fix the mdtest job failure (which happens because building mdtest
project fails) do -

1. Use ior projects intead of mdtest project bcecause latter was merged
   into former. See:
   https://github.com/MDTEST-LANL/mdtest/blob/master/README.md

2. Purge mpich package and then install it again. This is a vital step
   that's needed to build ior project on Ubuntu 22.04.

Fixes: https://tracker.ceph.com/issues/61574
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-21 18:15:16 +05:30
Adam King
0c8bd683b5
Merge pull request #52863 from batrick/i62326
pybind/mgr/cephadm/upgrade: stop disabling FSMap sanity checks

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-21 08:14:54 -04:00
Sridhar Seshasayee
401b30f19f PendingReleaseNotes: Note change to 'ceph config dump' pretty-print output.
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2023-09-21 17:10:10 +05:30
Ronen Friedman
631899a971 osd/scrub: scheduler: removing unused code
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-21 05:11:06 -05:00
Ronen Friedman
27e151414b osd/scrub: correct placement for some scheduler-related methods
Moving some member functions to their corresponding files.
Including ScrubQueue::dump_scrubs()
as it was moved in a previous commit,
and some ScrubJob code.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-21 05:11:05 -05:00
Yingxin
9f12b959b5
Merge pull request #53501 from zhscn/wip-lba-backref-node-size
crimson/os/seastore: create page aligned bufferptr in copy ctor of CachedExtent

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-09-21 17:20:28 +08:00
Xuehan Xu
88ca65776b crimson/os/seastore/cache: don't add EXIST_CLEAN extents to lru
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-09-21 17:15:43 +08:00
Redouane Kachach
6b8187b2ec
mgr/cephadm: removing double quotes from the generated nvmeof config
Fixes: https://tracker.ceph.com/issues/62838

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2023-09-21 10:40:17 +02:00
Nizamudeen A
b5bf9d70cb
Merge pull request #52150 from paulreece42/wip-grafana-quorum-fix
monitoring: grafana mons out of quorum should be count - sum

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-09-21 12:36:21 +05:30
Venky Shankar
dd33f1026e Merge PR #53028 into main
* refs/pull/53028/head:
	Update MDSDaemon.cc
	Update MDSRank.cc - Logoutput: Fix personal pronoun "I" to uppercase

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-21 06:11:54 +05:30
Venky Shankar
4859fef855 Merge PR #53176 into main
* refs/pull/53176/head:
	doc: add note for removing (automatic) partitioning policy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2023-09-21 06:03:42 +05:30
Adam King
d6b8162d89
Merge pull request #53467 from phlogistonjohn/jjm-cephadm-move-get_data_dir
cephadm: move get data dir function to daemonidentity method

Reviewed-by: Adam King <adking@redhat.com>
2023-09-20 14:20:09 -04:00
Adam King
d5555ce34c
Merge pull request #53415 from rkachach/fix_issue_62814
cephadm: fix cephadm binary mount when --shared_ceph_folder is used

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-09-20 14:19:02 -04:00
Adam King
58d331f618
Merge pull request #53298 from phlogistonjohn/jjm-logging-2
cephadm: enhance logging behavior

Reviewed-by: Adam King <adking@redhat.com>
2023-09-20 14:17:03 -04:00
Adam King
c579ff27f7
Merge pull request #52251 from rkachach/fix_issue_61856
mgr/cephadm: Adding sort-by support for ceph orch ps

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-09-20 14:14:23 -04:00
Ilya Dryomov
65a7a43bba
Merge pull request #52982 from batrick/backport-cp-simplify
script/ceph-backport: perform cherry-pick in single command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-20 19:58:40 +02:00
Pedro Gonzalez Gomez
b7f25e6317 mgr/dashboard: fix prometheus queries subscriptions
Fixes: https://tracker.ceph.com/issues/62868
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-09-20 16:48:02 +02:00
Patrick Donnelly
bf3051f458
Merge PR #53145 into main
* refs/pull/53145/head:
	mds: log message when exiting due to asok command

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-09-20 08:57:07 -04:00
Patrick Donnelly
0737936248
Merge PR #53149 into main
* refs/pull/53149/head:
	qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-09-20 08:31:17 -04:00
Venky Shankar
20d027eb9d doc: add note for removing (automatic) partitioning policy
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-09-20 15:49:19 +05:30
Guillaume Abrioux
ed2bd84014
Merge pull request #53309 from guits/bz2203397
ceph-volume: fix mpath device support
2023-09-20 09:32:49 +02:00
Ronen Friedman
c873c3973a osd/scrub: handle configuration changes in OsdScrub
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
0803780d5e osd/scrub: move initiate_a_scrub() to OsdScrub
Scrub initiation is now fully owned by OsdScrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
d54c579e18 osd/scrub: add ScrubQueue::ready_to_scrub()
At this phase of the refactoring:
this is the main interface from the scrub scheduler in OsdScrub
to the ScrubQueue. The ScrubQueue provides the ordered list of
all targets (for now - PGs) that are ready for scrubbing.

Scrub initiation code is modified to use the new interface.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
4959d4691b osd/scrub: move scrub_sleep_time() to OsdScrub
also scrub_time_permit().

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
0af4fc4261 osd/scrub: extract restrictions_on_scrubbing()
from ScrubQueue::select_pg_and_scrub().

Clearing the path to moving some ScrubQueue methods into
OscScrub. Starting here with the CPU load tracker.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
08d435e6dd osd/scrub: introduce OsdService::get_locked_pg()
which returns an RAII-wrapper around a locked PG.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
60deaa45f2 osd/scrub: move OSD::sched_scrub() to OsdScrub
... (as OsdScrub::initiate_scrub()).

The random backoff dice roller (scrub_random_backoff())
is moved as well.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
e0551a743c osd/scrub: declare OsdScrub, an OSD subobject
for all OSD scrub things.

For now: OsdScrub is mostly a forwarder to the ScrubQueue object
(which it now owns).
The resource counters moved into a separate object within OsdScrub.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
61af748a62 osd/scrub: unify dout macros across scrub scheduling code
to facilitate easy migration of code fragments between
related classes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
012db1ae0d osd/scrub: complete ScrubJob transition from within ScrubQueue
ScrubJob is now in the Scrub namespace.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
f3bd9b621c osd/scrub: set_reserving_now() signature modified
set_reserving_now() can now return a failure status, indicating
a race between two PGs to start scrubbing on the same OSD.

The scrubber FSM is modified to handle the failure.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
0bf1b29513 osd/scrub: select_pg_and_scrub() moved into osd_scrub.cc
No code changes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
5e4956cbae osd/scrub: renaming & fmt support for restrictions structure
Renaming ScrubPreconds, the collection of "environmental"
restrictions on possible scrubs, to OSDRestrictions.
Also - providing fmtlib support for that structure.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
46d291a7fc osd/scrub: introducing random_bool_with_probability()
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
de273640fb osd/scrub: moving the resources counters code into a separate file
No code changes in this commit.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
000bf250b7 osd/scrub: moving code as-is into osd_scrub.cc
Code from OSD.cc & osd_scrub_sched.cc moved as-is,
to be modified in followup commits.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Ronen Friedman
6db324700f osd/scrub: moving ScrubJob declaration as-is
from osd_scrub_sched.h into scrub_job.h.

A purely mechanical step. No code is changed.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00