Commit Graph

141597 Commits

Author SHA1 Message Date
John Mulligan
fdce99a6df cephadm: convert calls to get_container_{mounts,binds} to method calls
In get_container, convert calls to get_container_binds and
get_container_mounts to direct uses of the ContainerDaemonForm customize
methods, avoiding constructing said ContainerDaemonForm subclasses
redundantly.
This is all done in one pass because the _update_podman_mounts function
is called by get_container_mounts and trying to convert each type at a
time would result in extra calls to that function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-17 08:10:11 -05:00
John Mulligan
101dbb2f01 cephadm: handle the case where ctx may not have a config attribute
This prevents a test failure in future refactoring changes.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-17 08:10:11 -05:00
John Mulligan
b58369e916 cephadm: use CustomContainer.create classmethod like most other blocks
Use the CustomContainer.create classmethod like most other blocks of
code in the get_container function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-17 08:10:11 -05:00
Ilya Dryomov
54b3e35167
Merge pull request #54486 from pdvian/wip-app-release
PendingReleaseNotes: Add note for POOL_APP_NOT_ENABLED

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-17 11:35:44 +01:00
Aashish Sharma
70d8c5b565
Merge pull request #53650 from rhcs-dashboard/fix-62969-main
mgr/dashboard: Show the OSDs Out and Down panels as red whenever an OSD is in Out or Down state in Ceph Cluster grafana dashboard

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-11-17 11:24:45 +05:30
yu.wang
4efd9317b8 ceph-menv:fix typo in README
Signed-off-by: yu.wang <wangyu@wyu.space>
2023-11-17 12:39:14 +08:00
Nizamudeen A
cacc2eb70f mgr/dashboard: get port from ssl_endpoint
Fixes: https://tracker.ceph.com/issues/63564
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-11-17 09:50:15 +05:30
Adam King
4998173909
Merge pull request #54490 from ceph/nvmeof-controllerid-spec
mgr/cephadm: Expose nvmeof gateway configuration parameters through specifications

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Aviv Caro <Aviv.Caro@ibm.com>
2023-11-16 19:44:52 -05:00
Adam King
0f7c5526f1
Merge pull request #54487 from phlogistonjohn/jjm-cephadm-zipimporter-wrkaround
cephadm: workaround issues running cephadm with relative path

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-11-16 19:42:26 -05:00
Adam King
c37fd85050
Merge pull request #54403 from phlogistonjohn/jjm-cephadm-reorg-ctr-setup-methods
cephadm: reorganize the remaining container setup methods

Reviewed-by: Adam King <adking@redhat.com>
2023-11-16 19:41:26 -05:00
Adam King
91a16bca05
Merge pull request #54173 from phlogistonjohn/jjm-cephadm-jinja-2
cephadm: expand support for dependencies bundled into the zipapp

Reviewed-by: Adam King <adking@redhat.com>
2023-11-16 19:40:16 -05:00
Adam King
6091aa6f65
Merge pull request #53803 from adk3798/regex-host-pattern
mgr/cephadm: support for regex based host patterns

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-11-16 19:38:52 -05:00
Casey Bodley
3bef525b84
Merge pull request #54529 from cbodley/wip-osd-scrubber-format-to
osd/scrubber: fix ambiguous call to format_to()

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-16 22:12:01 +00:00
Prashant D
44886efc64 PendingReleaseNotes: Add note for POOL_APP_NOT_ENABLED
Adds release notes for the fix added in #47560

Signed-off-by: Prashant D <pdhange@redhat.com>
2023-11-16 12:13:12 -05:00
J. Eric Ivancich
b9c2fb61bd rgw: fix flight load_bucket call
The interface to the load_bucket call changed when tenant and bucket
name are specified and user is no longer required. This updates the
flight call to load_bucket to reflect those changes.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-11-16 11:45:31 -05:00
Casey Bodley
fa6ea9de7c osd/scrubber: fix ambiguous call to format_to()
new call to format_to() is ambiguous between std and fmt namespaces:

ceph/src/osd/scrubber/pg_scrubber.cc:654:26:   required from here
ceph/src/osd/scrubber/scrub_job.h:241:21: error: call of overloaded ‘format_to(fmt::v9::basic_format_context<fmt::v9::appender, char>::iterator, const char [60], const double&, double, const double&, const double&, const double&, const double&, const bool&)’ is ambiguous
...
ceph/src/fmt/include/fmt/core.h:3233:17: note: candidate: ‘OutputIt fmt::v9::format_to(OutputIt, format_string<T ...>, T&& ...) [with OutputIt = appender; T = {const double&, double, const double&, const double&, const double&, const double&, const bool&}; typename std::enable_if<detail::is_output_iterator<OutputIt, char>::value, int>::type <anonymous> = 0; format_string<T ...> = basic_format_string<char, const double&, double, const double&, const double&, const double&, const double&, const bool&>]’
 3233 | FMT_INLINE auto format_to(OutputIt out, format_string<T...> fmt, T&&... args)
      |                 ^~~~~~~~~
...
/usr/include/c++/13/format:3824:5: note: candidate: ‘_Out std::format_to(_Out, format_string<_Args ...>, _Args&& ...) [with _Out = fmt::v9::appender; _Args = {const double&, double, const double&, const double&, const double&, const double&, const bool&}; format_string<_Args ...> = basic_format_string<char, const double&, double, const double&, const double&, const double&, const double&, const bool&>]’
 3824 |     format_to(_Out __out, format_string<_Args...> __fmt, _Args&&... __args)
      |     ^~~~~~~~~

gcc (GCC) 13.2.1 20231011 (Red Hat 13.2.1-4)

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-16 11:03:45 -05:00
Casey Bodley
dc2a10a84d
Merge pull request #54171 from smanjara/wip-fix-archive-version-status
rgw/multisite: set versioning status in archive zone upon bucket creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-16 15:42:00 +00:00
Matan Breizman
84c5b6c578 crimson/osd/osd_operations/snaptrim_event: fix lifetime on finally()
Sanitized backtrace:
```
DEBUG 2023-11-14 15:23:50,871 [shard 0] osd - snaptrim_event(id=10610, detail=SnapTrimEvent(pgid=16.1a snapid=a needs_pause=0)): interrupted crimson::common::actingset_changed (acting set changed)

    #0 0x5653c613c071 in seastar::shared_mutex::unlock() (/usr/bin/ceph-osd+0x1ed27071)
    #1 0x5653c8670acf in auto seastar::futurize_invoke<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&>(crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&) (/usr/bin/ceph-osd+0x2125bacf)
    #2 0x5653c8670e22 in _ZN7seastar20noncopyable_functionIFNS_6futureIvEEvEE17direct_vtable_forIZNS2_4thenIZN7crimson23OrderedConcurrentPhaseTINS7_3osd13SnapTrimEvent9WaitSubopEE11ExitBarrierINSC_13BlockingEvent7TriggerISA_EEE4exitEvEUlvE_S2_EET0_OT_EUlDpOT_E_E4callEPKS4_ (/usr/bin/ceph-osd+0x2125be22)

freed by thread T1 here:
    #0 0x7f10628b73cf in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb73cf)
    #1 0x5653c8794bff in crimson::osd::SnapTrimEvent::~SnapTrimEvent() (/usr/bin/ceph-osd+0x2137fbff)

previously allocated by thread T1 here:
    #0 0x7f10628b6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/bin/ceph-osd+0x1ed27071) in seastar::shared_mutex::unlock()
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-16 09:42:54 +00:00
Guillaume Abrioux
3657ad5200
Merge pull request #54514 from guits/fix-cv-raw-list
ceph-volume: fix a regression in `raw list`
2023-11-16 10:29:15 +01:00
John Mulligan
4377fbd851 cephadm: workaround issues running cephadm with relative path
Implement a workaround for the jinja2 package loader not correctly
finding a template inside the cephadmlib package when run as
a zipapp. See docstring in the shim class for more details.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-15 13:32:48 -05:00
Guillaume Abrioux
e67dce7120 ceph-volume: fix a regression in raw list
'ceph-volume raw list' is broken for a specific use case (rook).

rook copies devices from /dev/ to /mnt for specific/internal needs.

when ceph-volume raw list is passed a device from /mnt then
ceph-volume ignores it and return an empty dict.
That prevent rook from creating OSDs properly.

https://tracker.ceph.com/issues/63545

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-11-15 17:58:22 +01:00
Matan Breizman
54fd676055 crimson/osd/pg: rep_tid as a faillure_func param
This change is crucial for the next commits,
submit_error_log and failure_func should share the same
rep_tid.

to be shared later with error_log call

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:49 +00:00
Matan Breizman
049e071b25 crimson/osd/pg: add logs and assert around log_entry_update_waiting_on
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:49 +00:00
Matan Breizman
c5146998b8 crimson/osd/pg: introduce PG::log_entry_version map
`submit_error_log()` was returning `version` to be used later in
`failure_func` call to `complete_write()`.

Maintain the version returned from `submit_error_log()` in a dedicated map
to avoid handling the lifetime of 'version'.

Note: This change is crucial to the following change that will
      return 'error_fut' separately.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
e1e8d1f3fd crimson/osd/pg: cleanup version usage submit_error_log
* '!log_entries.empty()' assert instead of if-case.
  log_entries entry is inserted right before.

* 'version != eversion_t()' assert instead of if-case.
  since op_info.may_write() is true, we should have a non-empty version.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
e3de7c0941 crimson/osd/pg: add logs around submit_error_log()
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
bf3845c05c crimson/osd/osd.cc: handle_update_log_missing* don't decode payload.
Payload is already decoded in IOHandler::read_message (decode_message).

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
7cd0aa0db0 crimson/osd/shard_services: add comment to next_tid initialization
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
e5aeade8f2 crimson/osd: remove do_osd_ops_success_func_t and do_osd_ops_failure_func_t
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Ronen Friedman
d9a4752cd8
Merge pull request #54363 from ronen-fr/wip-rf-force-scrub
osd/scrub: modify "scrub" & "deep_scrub" ASOK commands

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-11-15 18:12:22 +02:00
Ilya Dryomov
e0584935f9
Merge pull request #54489 from Suyashd999/main
qa/suites/rbd: Cleanup of MIRROR_IMAGE_MODE

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-15 16:49:57 +01:00
Milind Changire
49cd7587a6
Merge PR #54471 into main
* refs/pull/54471/head:
	qa: use correct imports to resolve fuse_mount and kernel_mount

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-11-15 21:15:15 +05:30
Ronen Friedman
fbb7d73e6f tests/standalone: fix scrub-related tests following command changes
Using
ceph tell $pgid [deep]-scrub
to initiate an 'operator initiated' scrub, and
ceph tell $pgid schedule[-deep]-scrub
for causing a 'periodic scrub' to be scheduled.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-15 03:14:15 -06:00
Alexander Indenbaum
ea3f155d90 mgr/cephadm: Expose nvmeof gateway configuration parameters through specifications
- min_controller_id, max_controller_id: Enable the specification of minimum and maximum controller IDs utilized by the SPDK.
  Having distinct controller IDs is vital for configuring multipath setups.
- enable_spdk_discovery_controller: Manage whether the SPDK or ceph-nvmeof discovery service is employed. The default value is set to False.

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
2023-11-15 07:16:49 +00:00
Samuel Just
ab230683cf
Merge pull request #53047 from xxhdx1985126/wip-crimson-interrupt-exception
crimson/common/interruptible_future: deal with exceptions thrown from seastar::future::get() and seastar:🧵:yield()

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-11-14 23:04:51 -08:00
zdover23
6d5127ee2a
Merge pull request #54497 from zdover23/wip-doc-2023-11-14-glossary-quorum
doc/glossary: add "Quorum" to glossary

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-15 16:11:41 +10:00
Zac Dover
c2f6a770bf doc/glossary: add "Quorum" to glossary
Add the term "Quorum" to the glossary and link to the part of
architecture.rst concerning Monitors. The sticky header at the top of
the docs.ceph.com website gets in the way of the location linked to in
this commit, but fatigue and disgust prevent me from spending time today
trial-and-erroring my way through the hostile and ill-documented
wilderness of scroll-margin so that the link goes where it should.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-15 15:45:11 +10:00
Venky Shankar
e23a69c1fa Merge PR #54259 into main
* refs/pull/54259/head:
	mds,client: update the oldest_client_tid via the renew caps
	mds: add trim_completed_request_list() helper

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-11-15 10:23:20 +05:30
John Mulligan
57b73442f8 cephadm: eliminate the _update_pids_limit function
Eliminate the _update_pids_limit function by moving the logic into the
various daemon type classes `customize_container_args` functions allow
each class to decide if it needs to run with unlimited-pids or not.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
a939d015c1 cephadm: use class methods when setting up ceph based daemon containers
Now that the ceph and ceph exporter classes have all the necessary
methods, rework the get_container function to use those class methods
in a common manner.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
a2bd2a435e cephadm: add container envs methods to ceph and ceph exporter classes
These, currently unused, methods will be used in a subsequent change to
move logic for setting up containers out of the get_containers function
into the daemon type classes in a common way.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
882073bc46 cephadm: add methods to ceph daemon type class
Add unused methods to the ceph class in order to prepare for moving
the logic for setting up the various ceph deaemon types to be
handled by the class.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
c423850fce cephadm: add more ContainerDeamonForm methods to the monitoring class
Add methods customize_{container_args,process_args} & default_entrypoint
to the monitoring daemon type class. Use those methods in the
get_container function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
c701621ad9 cephadm: add more ContainerDeamonForm methods to the tracing class
Add methods customize_{container_envs,process_args} & default_entrypoint
to the tracing daemon type class. Use those methods in the get_container
function.
This change tries to adapt to, but not fix, the rather strange
set_configuration static method that side-effects the class's(!)
property.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
9ffe0771f7 cephadm: add more ContainerDeamonForm methods to the nfs class
Add methods customize_{container_envs,process_args} & default_entrypoint
to the nfs daemon type class. Use those methods in the get_container
function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
a691e95df6 cephadm: add more ContainerDeamonForm methods to the ceph exporter class
Add methods customize_process_args & default_entrypoint to the ceph
exporter daemon type class. Use those methods in the get_container
function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
3d7aa101e4 cephadm: add more ContainerDeamonForm methods to the haproxy class
Add methods customize_{container_args,process_args} to the haproxy
daemon type class. Use those methods in the get_container function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
3098282c59 cephadm: add more ContainerDeamonForm methods to the keepalived class
Add methods customize_{container_args,container_envs} to the keepalived
daemon type class. Use those methods in the get_container function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
5bc8b8d80b cephadm: add more ContainerDeamonForm methods to the nvmeof class
Add method customize_container_args to the nvmeof daemon type class. Use
that method in the get_container function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00
John Mulligan
4f07a3b678 cephadm: add more ContainerDeamonForm methods to the iscsi class
Add method default_entrypoint to the iscsi daemon type class. Use that
method in the get_container function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-14 16:05:48 -05:00