Commit Graph

124260 Commits

Author SHA1 Message Date
Yuri Weinstein
712e7a0e22 doc: 14.2.22 Release Notes
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-07-01 00:18:54 +02:00
Kefu Chai
87ab4d7afa
Merge pull request #41941 from tchaikov/wip-crimson-errorator-loop
crimson/common: extract parallel_for_each_state out

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-21 20:46:41 +08:00
Kefu Chai
596e1ca370
Merge pull request #41949 from tchaikov/wip-crimson-prometheus
crimson/osd: expose metrics using http server

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-21 16:36:20 +08:00
Kefu Chai
80961c27d1 crimson/osd: expose metrics using http server
so, we can query the metrics using HTTP API, like

http://localhost:9180/metrics?name=io*

or

http://192.168.2.8:9180/metrics?name=io_queue_delay

or

http://localhost:9180/metrics

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-21 15:41:04 +08:00
Kefu Chai
175bb4d724
Merge pull request #41934 from cyx1231st/wip-seastore-onode-logs
crimson/onode-staged-tree: improve logs to understand inconsistent load from seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-21 12:34:02 +08:00
Yingxin Cheng
6673bf88a4 crimson/onode-staged-tree: print NodeExtent with the header
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-21 10:08:46 +08:00
Yingxin Cheng
860ddba0f0 crimson/onode-staged-tree: validate node header when load
Add logs to detect corruptions when load nodes. assert() is not
informative enough to understand the context.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-21 10:08:44 +08:00
Yingxin Cheng
366efc403e crimson/onode-staged-tree: delete copy constructor of DummyNodeExtent
Dummy backend is used for unit tests without transactions, so there
should be no copy-on-write behavior.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-21 10:05:28 +08:00
Yingxin Cheng
512dac2c6e crimson/onode-staged-tree: add trace logs when start to load nodes
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-21 10:05:20 +08:00
Amnon Hanuhov
ac7ab31ef6
Merge pull request #41861 from AmnonHanuhov/wip-Refactor_crimson_internals
crimson/net: Complete the refactor to std::unique_ptr inside Messenger

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-20 21:30:36 +03:00
Kefu Chai
230a1c4113
Merge pull request #41921 from gregsfortytwo/wip-mon-stretch-crush-rule
mon: Sanely set the default CRUSH rule when creating pools in stretch…

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-19 22:57:07 +08:00
Amnon Hanuhov
bb71ebbb19 tools/crimson: Use crimson::make_message() in perf_crimson_msgr
Instead of ceph::make_message() because conn::send() in crimson expects
a std::unique_ptr and not boost::intrusive_ptr

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-19 17:56:13 +03:00
Kefu Chai
6733d767dd
Merge pull request #41845 from agayev/zoned-revise-per-zone-naming-scheme
os/bluestore: Revise the naming scheme for per-zone cleaning informat…

Reviewed-by: Igor Fedotov <ifedotov@suse,com>
2021-06-19 22:54:25 +08:00
Amnon Hanuhov
84265a695b test/crimson: Use crimson::make_message() in test_alien_echo
Instead of ceph::make_message() because conn::send() in crimson expects
a std::unique_ptr and not boost::intrusive_ptr

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-19 17:52:54 +03:00
Kefu Chai
e05f186abc
Merge pull request #41830 from tchaikov/wip-ceph-argparse-cleanup
pybind/ceph_argparse: cleanups preparing for type annotations

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-19 22:51:59 +08:00
Amnon Hanuhov
c9a96891a9 crimson/net: Use MessageURef in messenger internals
Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-19 17:03:08 +03:00
Amnon Hanuhov
85563a617e crimson/osd: Get rid of send_to_osd() overloading
Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-19 17:03:08 +03:00
Amnon Hanuhov
a6d7fefde3 osd: Overload send_osd_message() in PeeringState
To allow passing MessageURef from crimson-osd and MessageRef from
ceph-osd

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-19 17:01:43 +03:00
Amnon Hanuhov
cf6de63e89 crimson/osd: Move message to send_to_osd() in ShardServices
To avoid refcounting the underlying RefCountedObject

Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-19 17:01:43 +03:00
Kefu Chai
fe17b2b058
Merge pull request #41923 from liewegas/fix-51234
ceph_test_librados_service: wait longer for servicemap to update

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-19 21:11:32 +08:00
Kefu Chai
94ab1c8fbe
Merge pull request #41914 from lxbsz/wip-51092
os/memstore: make the used_bytes to atomic

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-19 21:03:04 +08:00
Kefu Chai
c5ff2450a5
Merge pull request #41896 from ifed01/wip-ifed-verbose-kernel-read
blk/KernelDevice: be more verbose on read errors.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-19 21:02:18 +08:00
Kefu Chai
6654d63393 crimson/common: specialize errorator<> for future<>
otherwise it always needs a return value.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-19 19:13:19 +08:00
Kefu Chai
108eaf795f crimson/common: extract parallel_for_each_state out
if `parallel_for_each_state` is defined as a nested class in errorator,
clang fails to compile it:

../src/crimson/common/errorator.h:716:47: error: no class named 'parallel_for_each_state' in 'errorator<AllowedErrors...>'
    friend class errorator<AllowedErrors...>::parallel_for_each_state;
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

and the forward declaration does not help. so we have to extract it
out of the errorator. to speed up the compilation, it is moved into
errorator-loop.h. its name mirrors `include/seastar/core/loop.h`.

we could extract the `errorator<>::parallel_for_each()` out as well,
as its return type can be deduced from the type of Iterator and Func.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-19 19:12:38 +08:00
Kefu Chai
a234acb1d2
Merge pull request #41920 from ljflores/patch-1
doc: fixed a small typo in Perf Counters documentation

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-19 16:51:58 +08:00
Kefu Chai
87a1384b1f
Merge pull request #41925 from tchaikov/wip-fmtlib
fmt: pickup fix of link failure with clang

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2021-06-19 16:49:31 +08:00
Patrick Donnelly
a7af60243f
Merge PR #41900 into master
* refs/pull/41900/head:
	qa: use centos latest for fs:upgrade

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-06-18 19:54:09 -07:00
Patrick Donnelly
71cca1e9c3
Merge PR #41899 into master
* refs/pull/41899/head:
	mon/MDSMonitor: check fscid exists for legacy case

Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-06-18 19:52:54 -07:00
Patrick Donnelly
af9b123bb7
Merge PR #41898 into master
* refs/pull/41898/head:
	mon/MDSMonitor: fix whitespace in debug message

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-06-18 19:52:24 -07:00
Patrick Donnelly
0fe649f237
Merge PR #41892 into master
* refs/pull/41892/head:
	client: remove unused include from barrier.cc

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-18 19:51:55 -07:00
Patrick Donnelly
a49db812d4
Merge PR #41833 into master
* refs/pull/41833/head:
	cephfs-mirror: silence warnings when connecting via mon host

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-18 19:51:05 -07:00
Patrick Donnelly
a8974febd2
Merge PR #41723 into master
* refs/pull/41723/head:
	mds: to print the unknow type value

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-06-18 19:50:22 -07:00
Patrick Donnelly
efea9ecac7
Merge PR #40997 into master
* refs/pull/40997/head:
	test: add test to verify adding an active peer back to source
	pybind/mirroring: disallow adding a active peer back to source
	pybind/cephfs: interface to fetch file system id

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-18 19:49:15 -07:00
Patrick Donnelly
c4de4b3df8
Merge PR #36823 into master
* refs/pull/36823/head:
	qa : add a test for the cmd, dump cache
	mds : add timeout to the command, dump cache, to prevent it from running too long and affecting the service

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-18 19:47:53 -07:00
Laura Flores
2ae0734dba doc: fixed a small typo in Perf Counters documentation
There is a small typo in the Perf Counters documentation. Gauge was spelled incorrectly.

Signed-off-by: Laura Flores <lflores@redhat.com>
2021-06-18 18:14:45 +00:00
Ernesto Puerta
05d7f883a0
Merge pull request #40506 from p-se/pse-update-grafana-deprecated-variables
mgr/dashboard: deprecated variable usage in Grafana dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: p-se <NOT@FOUND>
2021-06-18 20:08:11 +02:00
Ernesto Puerta
94a6c7120d
Merge pull request #41808 from rhcs-dashboard/51164-show-only-days-in-bucket-details
mgr/dashboard: bucket details: show lock retention period only in days

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-06-18 20:07:11 +02:00
Ernesto Puerta
6676352414
Merge pull request #41758 from rhcs-dashboard/support-multiple-crush-trees
mgr/dashboard: crushmap tree doesn't display crush type other than root

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-06-18 20:04:16 +02:00
Kefu Chai
8538644bc3
Merge pull request #35903 from agayev/fix-deployment-guide
doc: Add a missing instruction to manual deployment guide.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-18 16:26:09 +08:00
Abutalib Aghayev
413d344723 doc: Add a missing instruction to manual deployment guide.
Following the instructions as is results in the following error at step 15:

$ sudo -u ceph ceph-mon --mkfs -i node1 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
global_init: error reading config file.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
2021-06-18 14:32:54 +08:00
Kefu Chai
e745fc2184 ceph.spec.in: bump up the required version of fmt-devel to 6.2.1
6.2.1 is the version packaged by EPEL8, in other words, this is the
version we've been testing. so to be more consistent with the
known-to-be-good version, let's bump up the required version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 11:14:30 +08:00
Kefu Chai
8867d7cd82 debian/control: add libfmt-dev for "make check"
so, on debian derivatives, we can use the libfmt-dev package for
building Ceph. this change is created in hope to reduce the compile
time.

>= 6.1.2 is specified, as it is the version packaged by ubuntu focal,
which is used for running "make check" and intergration tests.

find_package(fmt 6.0.0 QUIET)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 11:13:45 +08:00
Kefu Chai
c0a43f960d fmt: pickup fix of link failure with clang
fmtlib v7.1.3 contains the fix of https://github.com/fmtlib/fmt/issues/1753

so let's bump up the submodule to the latest master HEAD of fmtlib
for more fixes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 11:04:12 +08:00
Kefu Chai
c64451a940
Merge pull request #41911 from tchaikov/wip-crimson-nbd-cleanup
crimson/tools/store_nbd: better cleanup

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-18 10:22:21 +08:00
Kefu Chai
e64d8b8ca1 crimson/tools/store_nbd: replace wait_pending with seastar::gate
the inc_pending + promise<> solution is pratically identical to
seastar::gate, so let's use the prepackaged solution instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Kefu Chai
e4288b68e6 crimson/tools/store_nbd: drop unnecessary seastar::now()
the body of handle_exception() is a synchronous operation, there is no
need to return seastar::now() here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Kefu Chai
a53fcc2eeb crimson/tools/store_nbd: better cleanup
* remove unix domain socket file when cleanup
  so we don't need to remove it manually after each run.
* shutdown input and output streams when cleanup
  so reactor does not watch them anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Kefu Chai
7f2e8c366e crimson/tools/store_nbd: s/socket/server_socket/
to prepare for the next commit, which will keep track of the
connected_socket as another member variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Amnon Hanuhov
ef8e545907
Merge pull request #41916 from AmnonHanuhov/wip-Refactor_test_messenger
test/crimson: Use crimson's make_message in test_messenger
2021-06-18 00:04:40 +03:00
Sage Weil
4f5b0b7532 ceph_test_librados_service: wait longer for servicemap to update
mon thrashing may make this take a long time

Fixes: https://tracker.ceph.com/issues/51234
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-17 17:01:09 -04:00