Commit Graph

141457 Commits

Author SHA1 Message Date
Brad Hubbard
e877333f07
Merge pull request #54566 from badone/wip-python-version-fedora-39
do_cmake.sh: set python version for Fedora 39

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2023-11-22 09:44:22 +10:00
J. Eric Ivancich
eb4b542976
Merge pull request #54447 from ceph/wip-fix-flight-load-bucket
rgw: fix flight load_bucket call

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 16:36:53 -05:00
J. Eric Ivancich
0af251b856
Merge pull request #47208 from 5cs/fix-lambda-capture-by-ref
rgwlc: lock_lambda overwrites ret val

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-11-21 16:36:27 -05:00
zdover23
0bd8b17bfa
Merge pull request #54598 from zdover23/wip-doc-2023-11-22-rados-troubleshooting-mon-recovering-broken-monmap
doc/rados: edit "recovering broken monmap"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-22 05:56:33 +10:00
Zac Dover
6ccb9f3ca1 doc/rados: edit "recovering broken monmap"
Edit the section "Recovering a monitor's broken monmap" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-22 05:33:37 +10:00
Casey Bodley
b0b050d31c
Merge pull request #52813 from cbodley/wip-59424
qa/rgw: run s3tests against keystone ec2

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-21 13:33:58 +00:00
Guillaume Abrioux
bc868d9a4f
Merge pull request #53798 from asm0deuz/track_58812
ceph-volume: fixes fallback to stat in is_device and is_partition
2023-11-21 12:25:25 +01:00
Samuel Just
1ea87baab6
Merge pull request #54513 from Matan-B/wip-matanb-crimson-snaptrimevent-lifetime
crimson/osd/osd_operations/snaptrim_event: lifetime fixes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-11-20 19:16:37 -08:00
Brad Hubbard
838489f6b1 do_cmake.sh: set python version for Fedora 39
If do_cmake.sh is being exeuted on Fedora 39 set Python version to 3.12.
Remove versions for anything earlier than Fedora 37

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2023-11-21 10:44:33 +10:00
zdover23
40f55c30e5
Merge pull request #54574 from zdover23/wip-doc-2023-11-21-rados-troubleshooting-mon-understanding-mon-status
doc/rados: edit "understanding mon_status"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-21 04:42:49 +10:00
zdover23
d1c85afcb9
Merge pull request #54565 from zdover23/wip-doc-2023-11-20-radso-troubleshooting-mon-admin-socket
doc/rados: edit "Using the Monitor's Admin Socket"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-21 04:36:21 +10:00
Zac Dover
08c16aa113 doc/rados: edit "understanding mon_status"
Edit the section "Understanding mon_status" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-21 03:18:17 +10:00
Ilya Dryomov
7d4651e9d6
Merge pull request #54571 from lxbsz/wip-63586-debuglog
osd: log the number of extents for sparse read

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-20 16:36:59 +01:00
zdover23
df051c8917
Merge pull request #54561 from zdover23/wip-doc-2023-11-20-documenting-ceph-url
doc/start: update release names

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-21 01:24:27 +10:00
Xiubo Li
1cf5ecb93f osd: add more debug logs for sparse read
This will be very important to get to know what exactly has happened
when client get a very large number of extents.

URL: https://tracker.ceph.com/issues/63586
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-11-20 22:04:30 +08:00
Tongliang Deng
c094f1a909 rgwlc: lock_lambda overwrites ret val
`lock_lambda` captures `ret` by reference, it will overwrites
returned value of `bucket_lc_process` when `wait_backoff` is called.

Fixes: c069eb7ff0.

Signed-off-by: Tongliang Deng <dengtongliang@gmail.com>
2023-11-20 08:54:02 -05:00
barakda
06b3d30814
Merge pull request #54564 from barakda/nvmeof_bump_latest_version
nvmeof bump latest version
2023-11-20 15:19:08 +02:00
Teoman ONAY
52ca4a61d5 ceph-volume: fixes fallback to stat in is_device and is_partition
os.stat (or lstat) cannot distinguish a block device from
a partition.

Fixes: https://tracker.ceph.com/issues/58812

Signed-off-by: Teoman ONAY <tonay@ibm.com>
2023-11-20 09:50:05 +01:00
Aashish Sharma
39fea8f71c
Merge pull request #51340 from Javlopez/feature/12087-upgrade-and-generate-grafana-dashboards
monitoring: add new dashboards

Fixes: https://tracker.ceph.com/issues/63592

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2023-11-20 11:33:07 +05:30
Zac Dover
83ff8f2b67 doc/start: update release names
Update "Quincy" to "Reef" and "Pacific" to "Quincy" in the section
"Viewing Old Ceph Documentation" in /doc/start/documenting-ceph.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-20 06:44:46 +10:00
Zac Dover
f627445806 doc/rados: edit "Using the Monitor's Admin Socket"
Edit the section "Using the Monitor's Admin Socket" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-20 06:32:09 +10:00
barakda
e2e18a5722 nvmeof bump latest version
Signed-off-by: barakda <barak.davidov@gmail.com>
2023-11-19 22:24:09 +02:00
Yuval Lifshitz
329e2a1e04
Merge pull request #54528 from yuvalif/wip-yuval-63532
rgw/notifications: cleanup all coroutines after sending the notification

reviewed-by: cbodley
2023-11-19 21:25:39 +02:00
zdover23
b9b9ec8b7d
Merge pull request #54545 from zdover23/wip-doc-2023-11-17-start-intro-osd-glossary
doc/start: explain "OSD"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-19 23:48:12 +10:00
Matan Breizman
abceb16522 crimson/osd/osd_operations/snaptrim_event: lifetime fixes
```
    // SnapTrimEvent is a background operation,
    // it's lifetime is not guarnteed since the caller
    // returned future is being ignored. We should capture
    // a self reference thourhgout the entire execution
    // progress (not only on finally() continuations).
    // See: PG::on_active_actmap()
```

Sanitized backtrace:
```
DEBUG 2023-11-16 08:42:48,441 [shard 0] osd - snaptrim_event(id=21122, detail=SnapTrimEvent(pgid=3.1 snapid=3cb needs_pause=1)): interrupted crimson::common::actingset_changed (acting set changed

kernel callstack:
    #0 0x55e310e0ace7 in seastar::shared_mutex::unlock() (/usr/bin/ceph-osd+0x1edd0ce7)
    #1 0x55e313325d9c in auto seastar::futurize_invoke<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&>(crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&) (/usr/bin/ceph-osd+0x212ebd9c)
    #2 0x55e3133260ef in _ZN7seastar20noncopyable_functionIFNS_6futureIvEEvEE17direct_vtable_forIZNS2_4thenIZN7crimson23OrderedConcurrentPhaseTINS7_3osd13SnapTrimEvent9WaitSubopEE11ExitBarrierINSC_13BlockingEvent7TriggerISA_EEE4exitEvEUlvE_S2_EET0_OT_EUlDpOT_E_E4callEPKS4_ (/usr/bin/ceph-osd+0x212ec0ef)
0x61500013365c is located 92 bytes inside of 472-byte region [0x615000133600,0x6150001337d8)
freed by thread T2 here:
    #0 0x7fb345ab73cf in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb73cf)
    #1 0x55e313474863 in crimson::osd::SnapTrimEvent::~SnapTrimEvent() (/usr/bin/ceph-osd+0x2143a863)

previously allocated by thread T2 here:
    #0 0x7fb345ab6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)
    #1 0x55e31183ac18 in auto crimson::OperationRegistryI::create_operation<crimson::osd::SnapTrimEvent, crimson::osd::PG*, SnapMapper&, snapid_t const&, bool const&>(crimson::osd::PG*&&, SnapMapper&, snapid_t const&, bool const&) (/usr/bin/ceph-osd+0x1f800c18)
SUMMARY: AddressSanitizer: heap-use-after-free (/usr/bin/ceph-osd+0x1edd0ce7) in seastar::shared_mutex::unlock()
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 12:43:06 +00:00
Matan Breizman
3b162d38b2 crimson/osd: avoid refcount mutations
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-19 12:41:16 +00:00
Yuval Lifshitz
63e14893cc rgw/notifications: cleanup all coroutines after sending the notification
this is fixing a regression from: 6b6592f50b

Fixes: https://tracker.ceph.com/issues/63580

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-11-19 11:19:16 +00:00
Milind Changire
aca254a3ed
Merge PR #54414 into main
* refs/pull/54414/head:
	qa: run kernel_untar_build with newer tarball

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-11-19 08:58:14 +05:30
zdover23
35dfedef65
Merge pull request #54523 from zdover23/wip-doc-2023-11-16-man-ceph-osd-grammar
doc/man: edit ceph-osd description

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-19 04:12:16 +10:00
Zac Dover
a78fe85470 doc/start: explain "OSD"
Explain the initialism "OSD" and link to its definition in the glossary.
This PR is raised in response to an anonymous documentation bug that reads

  "Paragraph 2 uses the acronym OSD without any explanation.
   This makes it very difficult to understand this part of
   the documentation as there is no indication of what this
   acronym is until much further into the documentation. Replace
   first occurence of OSD with Object Storage Daemon (OSD) or
   link it to the glossary."
     -- https://pad.ceph.com/p/Report_Documentation_Bugs

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-19 04:09:20 +10:00
Zac Dover
2921dacf1e doc/man: edit ceph-osd description
Improve the English grammar and reduce the ambiguity and unclear writing
in the "Description" section of doc/man/8/ceph-osd.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-18 10:12:55 +10:00
Samuel Just
6adc03ac37
Merge pull request #52151 from myoungwon/wip-ceph-dedup-tool-memory
tools/ceph-dedup-tool: set memory threshold not to cause excessive  memory usage

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-11-17 10:40:11 -08:00
Anthony D'Atri
96cc4e3e65
Merge pull request #54530 from yuw1/fix_typo
ceph-menv:fix typo in README
2023-11-17 09:38:16 -05:00
Ilya Dryomov
54b3e35167
Merge pull request #54486 from pdvian/wip-app-release
PendingReleaseNotes: Add note for POOL_APP_NOT_ENABLED

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-17 11:35:44 +01:00
Aashish Sharma
70d8c5b565
Merge pull request #53650 from rhcs-dashboard/fix-62969-main
mgr/dashboard: Show the OSDs Out and Down panels as red whenever an OSD is in Out or Down state in Ceph Cluster grafana dashboard

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-11-17 11:24:45 +05:30
yu.wang
4efd9317b8 ceph-menv:fix typo in README
Signed-off-by: yu.wang <wangyu@wyu.space>
2023-11-17 12:39:14 +08:00
Adam King
4998173909
Merge pull request #54490 from ceph/nvmeof-controllerid-spec
mgr/cephadm: Expose nvmeof gateway configuration parameters through specifications

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Aviv Caro <Aviv.Caro@ibm.com>
2023-11-16 19:44:52 -05:00
Adam King
0f7c5526f1
Merge pull request #54487 from phlogistonjohn/jjm-cephadm-zipimporter-wrkaround
cephadm: workaround issues running cephadm with relative path

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-11-16 19:42:26 -05:00
Adam King
c37fd85050
Merge pull request #54403 from phlogistonjohn/jjm-cephadm-reorg-ctr-setup-methods
cephadm: reorganize the remaining container setup methods

Reviewed-by: Adam King <adking@redhat.com>
2023-11-16 19:41:26 -05:00
Adam King
91a16bca05
Merge pull request #54173 from phlogistonjohn/jjm-cephadm-jinja-2
cephadm: expand support for dependencies bundled into the zipapp

Reviewed-by: Adam King <adking@redhat.com>
2023-11-16 19:40:16 -05:00
Adam King
6091aa6f65
Merge pull request #53803 from adk3798/regex-host-pattern
mgr/cephadm: support for regex based host patterns

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-11-16 19:38:52 -05:00
Casey Bodley
3bef525b84
Merge pull request #54529 from cbodley/wip-osd-scrubber-format-to
osd/scrubber: fix ambiguous call to format_to()

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-16 22:12:01 +00:00
Prashant D
44886efc64 PendingReleaseNotes: Add note for POOL_APP_NOT_ENABLED
Adds release notes for the fix added in #47560

Signed-off-by: Prashant D <pdhange@redhat.com>
2023-11-16 12:13:12 -05:00
J. Eric Ivancich
b9c2fb61bd rgw: fix flight load_bucket call
The interface to the load_bucket call changed when tenant and bucket
name are specified and user is no longer required. This updates the
flight call to load_bucket to reflect those changes.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-11-16 11:45:31 -05:00
Casey Bodley
fa6ea9de7c osd/scrubber: fix ambiguous call to format_to()
new call to format_to() is ambiguous between std and fmt namespaces:

ceph/src/osd/scrubber/pg_scrubber.cc:654:26:   required from here
ceph/src/osd/scrubber/scrub_job.h:241:21: error: call of overloaded ‘format_to(fmt::v9::basic_format_context<fmt::v9::appender, char>::iterator, const char [60], const double&, double, const double&, const double&, const double&, const double&, const bool&)’ is ambiguous
...
ceph/src/fmt/include/fmt/core.h:3233:17: note: candidate: ‘OutputIt fmt::v9::format_to(OutputIt, format_string<T ...>, T&& ...) [with OutputIt = appender; T = {const double&, double, const double&, const double&, const double&, const double&, const bool&}; typename std::enable_if<detail::is_output_iterator<OutputIt, char>::value, int>::type <anonymous> = 0; format_string<T ...> = basic_format_string<char, const double&, double, const double&, const double&, const double&, const double&, const bool&>]’
 3233 | FMT_INLINE auto format_to(OutputIt out, format_string<T...> fmt, T&&... args)
      |                 ^~~~~~~~~
...
/usr/include/c++/13/format:3824:5: note: candidate: ‘_Out std::format_to(_Out, format_string<_Args ...>, _Args&& ...) [with _Out = fmt::v9::appender; _Args = {const double&, double, const double&, const double&, const double&, const double&, const bool&}; format_string<_Args ...> = basic_format_string<char, const double&, double, const double&, const double&, const double&, const double&, const bool&>]’
 3824 |     format_to(_Out __out, format_string<_Args...> __fmt, _Args&&... __args)
      |     ^~~~~~~~~

gcc (GCC) 13.2.1 20231011 (Red Hat 13.2.1-4)

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-16 11:03:45 -05:00
Casey Bodley
dc2a10a84d
Merge pull request #54171 from smanjara/wip-fix-archive-version-status
rgw/multisite: set versioning status in archive zone upon bucket creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-16 15:42:00 +00:00
Matan Breizman
84c5b6c578 crimson/osd/osd_operations/snaptrim_event: fix lifetime on finally()
Sanitized backtrace:
```
DEBUG 2023-11-14 15:23:50,871 [shard 0] osd - snaptrim_event(id=10610, detail=SnapTrimEvent(pgid=16.1a snapid=a needs_pause=0)): interrupted crimson::common::actingset_changed (acting set changed)

    #0 0x5653c613c071 in seastar::shared_mutex::unlock() (/usr/bin/ceph-osd+0x1ed27071)
    #1 0x5653c8670acf in auto seastar::futurize_invoke<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&>(crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::ExitBarrier<crimson::OrderedConcurrentPhaseT<crimson::osd::SnapTrimEvent::WaitSubop>::BlockingEvent::Trigger<crimson::osd::SnapTrimEvent> >::exit()::{lambda()#1}&) (/usr/bin/ceph-osd+0x2125bacf)
    #2 0x5653c8670e22 in _ZN7seastar20noncopyable_functionIFNS_6futureIvEEvEE17direct_vtable_forIZNS2_4thenIZN7crimson23OrderedConcurrentPhaseTINS7_3osd13SnapTrimEvent9WaitSubopEE11ExitBarrierINSC_13BlockingEvent7TriggerISA_EEE4exitEvEUlvE_S2_EET0_OT_EUlDpOT_E_E4callEPKS4_ (/usr/bin/ceph-osd+0x2125be22)

freed by thread T1 here:
    #0 0x7f10628b73cf in operator delete(void*, unsigned long) (/lib64/libasan.so.6+0xb73cf)
    #1 0x5653c8794bff in crimson::osd::SnapTrimEvent::~SnapTrimEvent() (/usr/bin/ceph-osd+0x2137fbff)

previously allocated by thread T1 here:
    #0 0x7f10628b6367 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6367)

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/bin/ceph-osd+0x1ed27071) in seastar::shared_mutex::unlock()
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-11-16 09:42:54 +00:00
Guillaume Abrioux
3657ad5200
Merge pull request #54514 from guits/fix-cv-raw-list
ceph-volume: fix a regression in `raw list`
2023-11-16 10:29:15 +01:00
John Mulligan
4377fbd851 cephadm: workaround issues running cephadm with relative path
Implement a workaround for the jinja2 package loader not correctly
finding a template inside the cephadmlib package when run as
a zipapp. See docstring in the shim class for more details.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-11-15 13:32:48 -05:00
Guillaume Abrioux
e67dce7120 ceph-volume: fix a regression in raw list
'ceph-volume raw list' is broken for a specific use case (rook).

rook copies devices from /dev/ to /mnt for specific/internal needs.

when ceph-volume raw list is passed a device from /mnt then
ceph-volume ignores it and return an empty dict.
That prevent rook from creating OSDs properly.

https://tracker.ceph.com/issues/63545

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-11-15 17:58:22 +01:00