Commit Graph

129985 Commits

Author SHA1 Message Date
Kefu Chai
d9cdd6175b crimson/osd: use reference when structured binding
to silence warning from clang13 with -std=c++20, like

ceph/src/crimson/osd/replicated_recovery_backend.cc:1098:21: warning: loop variable '[off, len]' creates a copy from type 'const std::pair<const unsigned long, unsigned long>' [-Wrange-loop-construct]
    for (const auto [off, len] : extents) {
                    ^
ceph/src/crimson/osd/replicated_recovery_backend.cc:1098:10: note: use reference type 'const std::pair<const unsigned long, unsigned long> &' to prevent copying
    for (const auto [off, len] : extents) {
         ^~~~~~~~~~~~~~~~~~~~~~~
                    &

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-02 00:31:14 +08:00
Kefu Chai
905969611d include/rados: define iterator without std::iterator<>
std::iterator<> is deprecated in C++17, so let's just
define the required traits directly.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-02 00:29:28 +08:00
Kefu Chai
c662ee698c crimson/os: define iterator without std::iterator<>
std::iterator<> is deprecated in C++17, so let's just
define the required traits directly.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-02 00:29:28 +08:00
Kefu Chai
fe0da04f1b crimson/os/seastore: remove default ctor for delta_t
otherwise the aggregate initialization for this type would fail to
work on C++20, as the user-declared ctor is defined for this struct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-02 00:29:28 +08:00
Kefu Chai
a8d119fc6f crimson/os: stop using seastar::metrics::label
seastar::metrics::label was dropped in a recent change. quote from
the commit message which removed this type:

>     In practice, it's an over-kill and just add information in the reporting
>    layer.

see also da395f31989a1ee333e489ffb251fc1152f12f0a in seastar repo

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-02 00:29:07 +08:00
Kefu Chai
7c381ba985 include/buffer: include <memory>
to address following FTBFS:

/usr/bin/ccache /usr/bin/clang++-13 -DBOOST_ALL_NO_LIB -DBOOST_ASIO_DISABLE_CONCEPTS -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_T$
In file included from /var/ssd/ceph/src/crimson/os/seastore/seastore_types.cc:4:
In file included from /var/ssd/ceph/src/crimson/os/seastore/seastore_types.h:14:
In file included from /var/ssd/ceph/src/include/denc.h:47:
/var/ssd/ceph/src/include/buffer.h:98:37: error: no template named 'unique_ptr' in namespace 'std'; did you mean 'boost::movelib::unique_ptr'?
struct unique_leakable_ptr : public std::unique_ptr<T, ceph::nop_delete<T>> {
                                    ^~~~~~~~~~~~~~~
                                    boost::movelib::unique_ptr
/opt/ceph/include/boost/move/unique_ptr.hpp:354:7: note: 'boost::movelib::unique_ptr' declared here
class unique_ptr
      ^

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-02 00:28:55 +08:00
Josh Durgin
c7225ee0e6
Merge pull request #45047 from ceph/wip-yuriw-release-15.2.16-master
doc: 15.2.16 Release Notes

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-03-01 08:05:55 -08:00
Yuval Lifshitz
e72b6510a4 rgw: prevent spurious/lost notifications in the index completion thread
this was happening when asyn completions happened during reshard.
more information about testing:
https://gist.github.com/yuvalif/d526c0a3a4c5b245b9e951a6c5a10517

we also add more logs to the completion manager.
should allow finding unhandled completions due to reshards.

Fixes: https://tracker.ceph.com/issues/54435

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-03-01 11:05:03 -05:00
John Mulligan
dd5a47f833 mgr/nfs: remove redundant check
Remove the extra check of the cluster id from _apply method. As _apply
is a "private" method that should be only called from other private
methods that have already validated the cluster_id. It also removes
a dependency on the orch-requiring func available_clusters.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-03-01 10:11:18 -05:00
John Mulligan
daa455cd16 mgr/nfs: fix unintentional recursion
The `exports` property of the ExportMgr exists to cache the exports
configuration found in the .nfs namespace. Using that property
within the property method is probably not intentional and is probably
only working due to the lucky construction of the _exports dict
immediately after the None check so that the _exports dict is returned
(and is a mutable type).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-03-01 10:11:18 -05:00
John Mulligan
4d09660dea mgr/nfs: add known_cluster_ids to generalize nfs cluster id fetching
The changes to the nfs module in 8c711afc are working but when I began
writing more test automation I found a few more places in the
export-configuration code path relying on the orchestration module
only. This change generalizes the logic to source nfs clusters from
orchestration when it's enabled but from the .nfs pool when
orchestration is disabled. It then uses that call when loading
the exports cache on the ExportMgr object.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-03-01 10:11:18 -05:00
Zac Dover
88d19c5e43 doc/start: alphabetize hardware-recs links
This PR alphabetizes the internal and external
references at the bottom of the Restructured
Text.

(This is really just a PR made so that I can
show how to make a documentation pull request.)

(In a video.)

(For people who prefer step-by-step instructions
in video form.)

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-03-01 23:53:29 +10:00
Ronen Friedman
ded84b8d5f
Merge pull request #44050 from ronen-fr/wip-rf-digest-update
osd/scrub: ignoring unsolicited DigestUpdate events

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-01 14:14:25 +02:00
chunmei-liu
3ffe2517a9 crimson/seastore: fix bad_alloc
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-02-28 21:10:22 -08:00
zdover23
327bbd762e
Merge pull request #45139 from zdover23/wip-doc-2022-02-24-hardware-recs-adatri-recs
doc/start: include A. D'Atri's hardware-recs recs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-03-01 14:41:20 +10:00
Adam King
fbe0c3fd23 mgr/cephadm: block removing last instance of _admin label
Fixes: https://tracker.ceph.com/issues/54425

Signed-off-by: Adam King <adking@redhat.com>
2022-02-28 20:23:10 -05:00
Laura Flores
5cdf8929e9
Merge pull request #45191 from neesingh-rh/cephfs-top_doc_fix
doc/cephfs: Typographical error.
2022-02-28 16:20:03 -06:00
Neha Ojha
6259641be7 doc/releases: couple of notable changes for core
also fix release date for v15.2.16

Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-02-28 22:09:29 +00:00
Adam King
9c8fd8a6c1
Merge pull request #45136 from adk3798/extra-all
mgr/cephadm: extend extra_container_args to other service types

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-02-28 16:58:38 -05:00
Adam King
a0d21c7108 mgr/cephadm: block draining last _admin host
Fixes: https://tracker.ceph.com/issues/54413

Signed-off-by: Adam King <adking@redhat.com>
2022-02-28 16:56:08 -05:00
Casey Bodley
df07e6d013
Merge pull request #45166 from pritha-srivastava/wip-rgw-bucket-sync-run
rgw: add OPT_BUCKET_SYNC_RUN to gc_ops_list, so that

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-28 16:16:03 -05:00
Neeraj Pratap Singh
d952916190 doc/cephfs: Typographical error.
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-02-28 15:28:24 +05:30
Zac Dover
07a1f7fe04 doc/start: include A. D'Atri's hardware-recs recs
This PR restores material about partition alignment
and material about separating OS and OSD data that
was removed in an earlier rewrite. The restoration
of this information was requested by Anthony D'Atri in
https://github.com/ceph/ceph/pull/45123/

This PR also includes several refinements to the language
that could not be made to this text until now, owing to my
(Zac's) ignorance and illiteracy.

I call upon Mark Nelson (and anyone else with sufficient
command of the current state of storage technology) to advise
me on whether the Ceph Foundation feels comfortable in the year
2022 referring to QLC as an emerging technology.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

(squash) more notes and revisions

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-02-28 19:19:00 +10:00
Pere Diaz Bou
dd1b5eb38c cephadm/box: default add hosts
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-02-28 09:48:36 +01:00
Pritha Srivastava
aa3006ea34 rgw: add OPT_BUCKET_SYNC_RUN to gc_ops_list, so that
gc is initialised and send_chain does not crash.

Also deleting objects inline in case gc is uninitialised.

Fixes: https://tracker.ceph.com/issues/54417
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2022-02-28 14:16:10 +05:30
Ronen Friedman
84fe3622b8 osd/scrub: cluster-logging unsolicited DigestUpdate events
... and not just to the OSD's log

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-26 13:06:24 +00:00
Ronen Friedman
0242368c2e
Merge pull request #44941 from ronen-fr/wip-rf-scrubBEv1_2
osd/scrub: limiting scrubber-backend external interfaces

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-26 14:56:36 +02:00
David Galloway
b2e517065c
Merge pull request #43957 from petrutlucian94/windows_ipv6
msg: Fix Windows IPv6 support
2022-02-25 15:56:30 -05:00
Adam King
922211851e
Merge pull request #44870 from phlogistonjohn/jjm-nfs-cleanups2
mgr/nfs: support managing exports without orchestration enabled 

Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-02-25 10:08:05 -05:00
Adam King
b39f36b121
Merge pull request #44845 from phlogistonjohn/jjm-nfs-cleanups
mgr/nfs: various minor cleanups

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-02-25 10:02:52 -05:00
Adam King
057089c90d
Merge pull request #45113 from adk3798/minimize-image
cephadm: still set container_image when --no-assimilate-config is provided in bootstrap

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-02-25 09:58:08 -05:00
Adam King
6818b7bb33
Merge pull request #45127 from rkachach/fix_issue_54026_part1
mgr/cephadm: Adding python natsort module

Reviewed-by: Adam King <adking@redhat.com>
2022-02-25 09:56:12 -05:00
Adam King
1211be6ff1
Merge pull request #45132 from melissa-kun-li/asyncssh_error_messages
mgr/cephadm: reduce log level for asyncssh error messages

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-02-25 09:54:15 -05:00
Adam King
d3c14a17dc mgr/cephadm: extend extra_container_args to other service types
Otherwise, without this change, this can only be used for mgr,
mon and crash (daemons without their own service spec class)

Fixes: https://tracker.ceph.com/issues/54390

Signed-off-by: Adam King <adking@redhat.com>
2022-02-25 08:03:51 -05:00
Samuel Just
8c54b697d9
Merge pull request #45041 from cyx1231st/wip-crimson-cleanup-epm-segment-writer
crimson/os/seastore: introduce SegmentAllocator for Journal and ExtentPlacementManager

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-24 14:23:12 -08:00
Casey Bodley
c209b50ec5
Merge pull request #45109 from cbodley/wip-rgw-dbstore-strings
rgw/dbstore: use string_view for string constants

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2022-02-24 13:57:31 -05:00
Sunny Kumar
05f2cfd9b5 mgr/rbd_support: cast pool_id from int to str when collecting LevelSpec
While collecting LevelSpec using class method from_name make sure to cast
pool_id from int to string. This is necessary to match the internal
representation of LevelSpec where pool_id is maintained as str.

Fixes: https://tracker.ceph.com/issues/53915

Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
2022-02-24 17:39:17 +00:00
Guillaume Abrioux
4804a74619
Merge pull request #45126 from guits/fix-bug-cv-bz2056719
ceph-volume: abort when passed devices have partitions
2022-02-24 17:25:25 +01:00
Ronen Friedman
fe3ec8be7b osd/scrub: fix some scrubber-backend formatting
no code changes.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-24 14:26:37 +00:00
Ronen Friedman
43c8d8434c osd/scrub: cleanup scrubber-be's usage of the size-of-disk service
Limit the amount of scrubber code that directly calls the
PG-Backend services - in this case, the logical-size to
on-disk-size translation.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-24 14:26:37 +00:00
Ronen Friedman
aabeaacd77 osd/scrub: clean-up scrubber-be interfaces
Esp - the shallow/deep-error counters.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-24 14:26:37 +00:00
Ronen Friedman
397992439f osd/scrub: make the scrubber-be the sole owner of the authoritative set
simplifying the interfaces used by the scrubber backend.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-24 14:26:37 +00:00
Ronen Friedman
2635929d13 osd/scrub: remove I/O (store) direct access from the Scrubber Backend
Target: improving Scrubber BE testability by reducing the number and
complexity of the external APIs it uses.

(note - first step of many)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-24 14:26:37 +00:00
Yingxin Cheng
5327997032 crimson/os/seastore/epm: integrate SegmentAllocator with Writer
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-24 20:55:45 +08:00
Yingxin Cheng
b0db4be190 crimson/os/seastore: introduce SegmentAllocator and integrate with Journal
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-24 20:55:37 +08:00
Yingxin Cheng
a1e5490758 crimson/os/seastore/journal: cleanup, move commit_to to RecordSubmitter
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-24 20:43:15 +08:00
Yingxin Cheng
d2e3bb6f7a crimson/os/seastore/epm: replace condition_variable by a shared_promise
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-24 20:39:39 +08:00
Yingxin Cheng
7ebf3590de crimson/os/seastore/epm: simplify gating writes for Writer
Dropped open_segment_wrapper_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-24 20:24:26 +08:00
Yingxin Cheng
65a294058f crimson/os/seastore/segment_cleaner: drop unnecessary future from get_segment()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-24 20:24:26 +08:00
Dan van der Ster
29545b617b osd: require osd_pg_max_concurrent_snap_trims > 0
If osd_pg_max_concurrent_snap_trims is zero, we mistakenly clear
the snaptrim queue. Require it to be > 0.

Fixes: https://tracker.ceph.com/issues/54396
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2022-02-24 09:42:00 +01:00