Commit Graph

141366 Commits

Author SHA1 Message Date
Matan Breizman
45312902f2 crimson/osd/pg: introduce clear_log_entry_maps()
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 12:31:39 +00:00
Matan Breizman
1d98e8dab6 crimson/osd/pg: move submit_error_log to do_osd_ops_execute
Previously, submit_error_log was chained to failure_func
returned future.
Now submit_error_log is called from within do_osd_ops_execute

Fixes: https://tracker.ceph.com/issues/61651

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 12:27:22 +00:00
Matan Breizman
11d2c66757 crimson/osd/pg: add record_error bool to failure_func
```
submit_error_log records the result of an IO into the pg log so that we can return
the same error code if the client resends the request.
This should only be relevant for logical errors resulting from the target object state
-- for example, EEXIST returned on an exclusive create -- because there is application
logic built to rely on them.
In classic, the only such site is if the return value from do_osd_ops is negative
(or the transaction is empty) -- see PrimaryLogPG::prepare_transaction,
specifically where we set update_log_only to true.

We do not want to record space usage errors or errors specific to conditions on the primary
OSD such as IO errors -- submit_error_log isn't a catch-all error path.
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 09:58:50 +00:00
Matan Breizman
74965cb4dd crimson/osd/pg: do_osd_ops_execute assert error type handling
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 09:47:29 +00:00
Matan Breizman
4bd7b94f41 crimson/osd/pg: submit_error_log send messages to osd by order
Use chained futurized `send_to_osd()` instead of voided `send_cluster_message()`.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 09:46:00 +00:00
Matan Breizman
54fd676055 crimson/osd/pg: rep_tid as a faillure_func param
This change is crucial for the next commits,
submit_error_log and failure_func should share the same
rep_tid.

to be shared later with error_log call

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:49 +00:00
Matan Breizman
049e071b25 crimson/osd/pg: add logs and assert around log_entry_update_waiting_on
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:49 +00:00
Matan Breizman
c5146998b8 crimson/osd/pg: introduce PG::log_entry_version map
`submit_error_log()` was returning `version` to be used later in
`failure_func` call to `complete_write()`.

Maintain the version returned from `submit_error_log()` in a dedicated map
to avoid handling the lifetime of 'version'.

Note: This change is crucial to the following change that will
      return 'error_fut' separately.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
e1e8d1f3fd crimson/osd/pg: cleanup version usage submit_error_log
* '!log_entries.empty()' assert instead of if-case.
  log_entries entry is inserted right before.

* 'version != eversion_t()' assert instead of if-case.
  since op_info.may_write() is true, we should have a non-empty version.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
e3de7c0941 crimson/osd/pg: add logs around submit_error_log()
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
bf3845c05c crimson/osd/osd.cc: handle_update_log_missing* don't decode payload.
Payload is already decoded in IOHandler::read_message (decode_message).

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
7cd0aa0db0 crimson/osd/shard_services: add comment to next_tid initialization
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Matan Breizman
e5aeade8f2 crimson/osd: remove do_osd_ops_success_func_t and do_osd_ops_failure_func_t
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-15 16:12:48 +00:00
Samuel Just
ab230683cf
Merge pull request #53047 from xxhdx1985126/wip-crimson-interrupt-exception
crimson/common/interruptible_future: deal with exceptions thrown from seastar::future::get() and seastar:🧵:yield()

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-11-14 23:04:51 -08:00
zdover23
6d5127ee2a
Merge pull request #54497 from zdover23/wip-doc-2023-11-14-glossary-quorum
doc/glossary: add "Quorum" to glossary

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-15 16:11:41 +10:00
Zac Dover
c2f6a770bf doc/glossary: add "Quorum" to glossary
Add the term "Quorum" to the glossary and link to the part of
architecture.rst concerning Monitors. The sticky header at the top of
the docs.ceph.com website gets in the way of the location linked to in
this commit, but fatigue and disgust prevent me from spending time today
trial-and-erroring my way through the hostile and ill-documented
wilderness of scroll-margin so that the link goes where it should.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-15 15:45:11 +10:00
Venky Shankar
e23a69c1fa Merge PR #54259 into main
* refs/pull/54259/head:
	mds,client: update the oldest_client_tid via the renew caps
	mds: add trim_completed_request_list() helper

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-11-15 10:23:20 +05:30
Adam King
473eadaa90
Merge pull request #54401 from adk3798/osd-rm-stop-reweight
mgr/cephadm: fix reweighting of OSD when OSD removal is stopped

Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-11-14 15:21:55 -05:00
Adam King
7e49853742
Merge pull request #54399 from phlogistonjohn/jjm-cephadm-reorg-ctr-mounts
cephadm: reorganize container mounts functions

Reviewed-by: Adam King <adking@redhat.com>
2023-11-14 15:20:36 -05:00
zdover23
1b4842658d
Merge pull request #54496 from zdover23/wip-doc-2023-11-14-rados-troubleshooting-mon-initial-rewrite
doc/rados: edit "troubleshooting-mon"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-15 03:06:21 +10:00
zdover23
8a616b2f5b
Merge pull request #54499 from yuvalif/wip-yuval-63530
doc/rgw/notifications: specify which event types are enabled by default

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-11-15 02:57:46 +10:00
Zac Dover
fa8129a746 doc/rados: edit "troubleshooting-mon"
Edit the text in the "Initial Troubleshooting" section of
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-15 02:47:56 +10:00
Yuval Lifshitz
4ea87307e7 doc/rgw/notifications: specify which event types are enabled by default
Fixes: https://tracker.ceph.com/issues/63530

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-11-14 15:30:07 +00:00
Matan Breizman
c2091ef742
Merge pull request #54431 from Matan-B/wip-matanb-crimson-extend-snaptrimsubevent-lifetime
crimson/osd/osd_operations/snaptrim_event: fix lifetime management

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-11-14 17:15:48 +02:00
Matan Breizman
c98149a440
Merge pull request #54416 from Matan-B/wip-matanb-crimson-extend-pg-lifetime
crimson/osd/pg: extend pg lifetime on snap_trimq iteration

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-11-14 17:14:34 +02:00
Ilya Dryomov
070b69046a
Merge pull request #54375 from ajarr/wip-63421
pybind/mgr: remove __del__() of mgr modules

Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-14 12:21:09 +01:00
Anthony D'Atri
7325a8d317
Merge pull request #54479 from anthonyeleven/anthonyeleven/ceph
doc/start: fix typo in hardware-recommendations.rst
2023-11-13 11:38:13 -05:00
Ramana Raja
6d8e0a82f6 pybind/mgr: remove __del__() of mgr modules
It's strongly recommended for objects that have references to
external resources (e.g., files) to explicitly release them.
Python doesn't guarantee garbage collection of objects and hence
doesn't guarantee freeing of external resources that occur on
garbage collection.

The __del__() methods in the python mgr modules may not even be
called since garbage collection of objects is not guaranteed in python.
And some of the __del__() methods try to cleanup that seem redundant.

- In volumes/module.py, vc.shutdown() is called in Module.shutdown().
  No need to call it again in Module.__del__()

- In telegraf/basesocket.py, BaseSocker.close() is called in
  BaseSocket.__exit__(). No need to call it again in
  BaseSocket.__del__().

- In mgr_module.py, MgrModuleLoggingMixin._unconfigure_logging() is
  called in MgrModule.__init__() and MgrStandbyModule.__init__(). No
  need to call it in MgrModule.__del__() and
  MgrStandbyModule.__del__().|

- In dashboard/services/cephfs.py, the libcephfs mount is not
  shutdown explicitly by the mgr module. However, the cython libcephfs
  bindings has a LibCephFS.__dealloc__() finalizer method that calls
  LibCephFS.shutdown(). This should unmount and cleanup the ceph mount
  handle.

Remove the __del__() of the python mgr modules.

Fixes: https://tracker.ceph.com/issues/63421
Signed-off-by: Ramana Raja <rraja@redhat.com>
2023-11-13 11:07:40 -05:00
Matan Breizman
f5300331e5 crimson/osd/pg: PG::on_active_actmap() indention
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-13 16:06:00 +00:00
Matan Breizman
454a2b9285 crimson/osd/pg: extend pg lifetime on snap_trimq iteration
Beacuse the loop's returned future is ignored,
we should cover the scenario where the pg is removed and the
snap_trimq iteration didn't complete yet.

Fixes: https://tracker.ceph.com/issues/61653

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-13 16:05:36 +00:00
Yuri Weinstein
cc15663690
Merge pull request #53655 from ifed01/wip-ifed-peering-cleanup
osd: getting rid off redundant std::map/set::count() calls.

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-13 07:29:07 -08:00
Yuri Weinstein
47fce541de
Merge pull request #53597 from ifed01/wip-ifed-bluefs-perf-counters
os/bluestore: add more bluefs perf counters

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2023-11-13 07:28:39 -08:00
Yuri Weinstein
40c6bad894
Merge pull request #53530 from ifed01/wip-ifed-fast-shutdown-assert
osd: do not assert on fast shutdown timeout

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-13 07:27:39 -08:00
Yuri Weinstein
c2d216bbc1
Merge pull request #52855 from Matan-B/wip-matanb-monstore-osd_snap
tools/ceph_monstore_tool: additional commands and doc

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
2023-11-13 07:27:10 -08:00
Yuri Weinstein
df780ba1ce
Merge pull request #52406 from Matan-B/wip-matanb-mon-44069-commits
mon/Config*: Cleanups and ConfigMap unit test

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-13 07:26:05 -08:00
Anthony D'Atri
48fd7ff1e1 doc/start: fix typo in hardware-recommendations.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2023-11-13 08:58:44 -05:00
zdover23
3cf8e3b49b
Merge pull request #54473 from zdover23/wip-doc-2023-11-13-rados-troubleshooting-mon-initial
doc/rados: format "initial troubleshooting"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-13 23:53:15 +10:00
Zac Dover
c581018caf doc/rados: format "initial troubleshooting"
Format the steps in the "Initial Troubleshooting" section of
doc/rados/troubleshooting/troubleshooting-mon.rst. A near-future PR (not
this one) will add context to this section and explain that the steps
described here are the first steps that you should undertake when you
determine that you have an unresponsive or down Monitor. This PR is
merely for formatting.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-13 20:57:07 +10:00
Patrick Donnelly
474f9a5483
Merge PR #54338 into main
* refs/pull/54338/head:
	script: enable checkout of specific sha1 as base

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-11-12 10:25:07 -05:00
Matan Breizman
cdee9b928f crimson/osd/osd_operations/snaptrim_event: fix lifetime management
The operation's id and future returned when starting SnapTrimObjSubEvent
is emplaced into subop_blocker.
Later on, we await the completion of all the started operations futures.
Before this patch, we only stored the op id in the subop_blocker vector
which allowed `op` to go out of scope and lose all its references
(and get deleted) before exiting.

Storing the operation as a reference instead of the id
will maintain the SnapTrimObjSubEvent operation lifetime.

Fixes: https://tracker.ceph.com/issues/63299

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-12 14:12:39 +00:00
zdover23
011cf0e991
Merge pull request #54457 from zdover23/wip-doc-2023-11-12-rados-config-ceph-conf-config-ceph
doc/config: edit "ceph-conf.rst"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2023-11-13 00:00:40 +10:00
Anthony D'Atri
513c3f9cc1
Merge pull request #54458 from zdover23/wip-doc-2023-11-12-rados-troubleshooting-mon-parallel-structure
doc/rados: parallelize t-mon headings
2023-11-12 08:48:40 -05:00
Zac Dover
010e5a13b9 doc/rados: parallelize t-mon headings
Give parallel structure to the questions in the Q&A section of the "The
Cluster Has Quorum But At Least One Monitor Is Down" subsection of the
"Most Common Monitor Issues" section of
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-12 20:52:09 +10:00
Zac Dover
c886e05404 doc/config: edit "ceph-conf.rst"
Edit the first section of doc/rados/configuration/ceph-conf.rst.
Initially I just wanted to change "series" to "set", but once I got my
hands dirty I ended up simplifying some sentences.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-12 20:21:41 +10:00
John Mulligan
77ee6ab8d4 cephadm: use customize_container_mounts in get_container_mounts
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-11 08:10:49 -05:00
John Mulligan
3411e10c5c cephadm: change ceph & exporter to customize_container_mounts method
Unlike the other types Ceph and CephExporter share the underlying
method. There was no other use of get_container_mounts on the class
so it could be converted to be customize_container_mounts.
Because there's an extra arg that passes from get_container_mounts
top-level function to Ceph.get_ceph_mounts, that function was not
changed.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-11 08:10:49 -05:00
John Mulligan
44c8e55b5c cephadm: add customize_container_mounts method to monitoring class
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-11 08:10:49 -05:00
John Mulligan
8acdacd789 cephadm: add customize_container_mounts method to nfs class
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-11 08:10:49 -05:00
John Mulligan
7817a6f429 cephadm: add customize_container_mounts method to haproxy class
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-11 08:10:49 -05:00
John Mulligan
f76822a15e cephadm: add customize_container_mounts method to keepalived class
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-11 08:10:49 -05:00