Commit Graph

135637 Commits

Author SHA1 Message Date
Pere Diaz Bou
8e07fbd2ea
Merge pull request #48843 from rhcs-dashboard/expose_slow_ops
mgr/prometheus: expose daemon health metrics

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-12-20 12:25:32 +01:00
Pedro Gonzalez Gomez
c2ae7cb4b8
Merge pull request #47276 from rhcs-dashboard/mirror-pool-health
mgr/dashboard: display real health in rbd mirroring pools


Reviewed-by: Pegonzal <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-12-20 10:49:31 +01:00
Pere Diaz Bou
5a2b7c25b6 mgr/prometheus: expose daemon health metrics
Until now daemon health metrics were stored without being used. One of
the most helpful metrics there is SLOW_OPS with respect to OSDs and MONs
which this commit tries to expose to bring fine grained metrics to find
troublesome OSDs instead of having a lone healthcheck of slow ops in the
whole cluster.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-12-20 09:44:49 +01:00
Yingxin
b69584cbbc
Merge pull request #49496 from cyx1231st/wip-seastore-fix-denc
crimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-12-20 13:27:07 +08:00
Kefu Chai
ead5a9a215
Merge pull request #49509 from rhcs-dashboard/fix-prometheus-module-blank-line
mgr/prometheus: Fix make check flake8 blank line missing

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2022-12-20 12:03:34 +08:00
colemitchell
a4d5157845
Merge pull request #49514 from zdover23/wip-doc-2022-12-20-glossary-cephx-config-ref-link
doc/glossary: link to CephX Config ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-19 19:41:17 -05:00
colemitchell
4a31ae3282
Merge pull request #49513 from zdover23/wip-doc-2022-12-20-rados-auth-config-ref-prompts
doc/rados: add prompts to auth-config-ref.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-19 13:18:59 -05:00
Zac Dover
36a8f27e96 doc/glossary: link to CephX Config ref
Link to the CephX Configuration Reference in the "CephX" entry in the
glossary.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-20 04:16:19 +10:00
Zac Dover
208bc2323b doc/rados: add prompts to auth-config-ref.rst
Add unselectable prompts to doc/rados/configuration/auth-config-ref.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-20 04:00:59 +10:00
colemitchell
bc971e5e90
Merge pull request #49510 from zdover23/wip-doc-2022-12-20-rados-config-msgr2-prompts
doc/rados: add prompts to msgr2.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-19 12:50:46 -05:00
Zac Dover
239e7c0e68 doc/rados: add prompts to msgr2.rst
Add unselectable prompts to doc/rados/configuration/msgr2.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-20 03:41:22 +10:00
Pedro Gonzalez Gomez
4630da8a93 mgr/prometheus: add blank like previously wrongly deleted that was causing issues on makecheck due to flake8
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2022-12-19 16:16:57 +01:00
Venky Shankar
ab8fbe4e59 Merge PR #48473 into main
* refs/pull/48473/head:
	qa: fix _run_mount_cmd() and _get_mount_cmd() args in vstart_runner

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-12-19 16:17:15 +05:30
Kefu Chai
4ffa58c889
Merge pull request #49490 from tchaikov/wip-pybind-mgr-tox-fixes
pybind/mgr: tox and test fixes

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2022-12-19 18:14:13 +08:00
Aashish Sharma
1c8cde77a1
Merge pull request #49049 from rhcs-dashboard/add-osd-blocklist-metric
mgr/dashboard: Add metric relative to osd blocklist
2022-12-19 13:37:12 +05:30
Aashish Sharma
1edef5576c
Merge pull request #49292 from rhcs-dashboard/rbd-mirror-improvements
mgr/dashboard: rbd-mirror improvements
2022-12-19 12:19:58 +05:30
Anthony D'Atri
d8986dd252
Merge pull request #49203 from ktdreyer/doc-simple-cephfs-create
doc/cephfs: simplify ceph fs volume create command
2022-12-19 01:46:06 -05:00
Yingxin Cheng
181702913f crimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t
* device_spec_t should not include denc and be packed at the same time
* the underlying type of enum:uint8_t should support denc_traits already

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2022-12-19 10:47:46 +08:00
Kefu Chai
97a5dd316e
Merge pull request #49489 from tchaikov/wip-cmake-with-fmt-version
cmake: set WITH_FMT_VERSION as a STRING option

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-12-19 08:47:29 +08:00
colemitchell
fe7b62d217
Merge pull request #49491 from zdover23/wip-doc-2022-12-18-rados-config-ceph-conf-prompts
doc/rados: add prompts to ceph-conf.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-18 13:09:54 -05:00
Zac Dover
874152535c doc/rados: add prompts to ceph-conf.rst
Add unselectable prompts to doc/rados/ceph-conf.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-19 01:29:33 +10:00
Kefu Chai
0cc649f029 pybind/mgr/tox.ini: add commas in "modules" variable
since tox v4.0.13, it parses the variables differently, so the newlines
in a variable are passed right to the command referencing it. so we now
have failure like:
```
flake8: commands[0] /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> flake8 --config=tox.ini alerts
flake8: commands[1] /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> balancer
flake8: exit 2 (0.00 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> balancer
flake8: FAIL ✖ in 3.33 seconds
```
so we have to add comma as line continuation separator to address
this problem.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 20:23:47 +08:00
Kefu Chai
5878b9486a pybind/mgr: s/setup(self)/setup_method(self)/
avoid pytest warnings like:

4: pg_autoscaler/tests/test_cal_final_pg_target.py::TestPgAutoscaler::test_even_pools_one_meta_three_bulk
4:   /home/kefu/dev/ceph/src/pybind/mgr/.tox/py3/lib/python3.10/site-packages/_pytest/fixtures.py:900: PytestRemovedIn8Warning: Support for nose tests is deprecated and will be removed in a future release.
4:   pg_autoscaler/tests/test_cal_final_pg_target.py::TestPgAutoscaler::test_even_pools_one_meta_three_bulk is using nose-specific method: `setup(self)`
4:   To remove this warning, rename it to `setup_method(self)`
4:   See docs: https://docs.pytest.org/en/stable/deprecations.html#support-for-tests-written-for-nose
4:     fixture_result = next(generator)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 20:23:47 +08:00
Kefu Chai
9f0ee97985 pybind/mgr/prometheus: avoid using distutils
to silence warnings like:

4: prometheus/module.py:35
4:   /var/ssd/ceph/src/pybind/mgr/prometheus/module.py:35: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
4:     v = StrictVersion(cherrypy.__version__)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 20:23:46 +08:00
Kefu Chai
0d41777cea cmake: set WITH_FMT_VERSION as a STRING option
option() defines a boolean, so its default value cannot be populated
to find_package() as expected. so we need to set it as a cached STRING
variable.

this addresses the regression introduced by 051ec194df

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 10:47:42 +08:00
Kefu Chai
a2bed04323
Merge pull request #49430 from tchaikov/wip-fmtlib-v9
cmake, script: run "make check" with fmtlib v9

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-17 15:08:29 +08:00
zdover23
e2d33ddd85
Merge pull request #49394 from zdover23/wip-doc-2022-12-10-rbd-rbd-snapshot-edit
doc/rbd: refine rbd-snapshot.rst

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2022-12-17 01:03:22 +10:00
Zac Dover
292c826acd doc/rbd: refine rbd-snapshot.rst
Refine the text in rbd-snapshot.rst

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-17 00:28:06 +10:00
Casey Bodley
7471527448
Merge pull request #49479 from tchaikov/wip-rgw-fmt-v8
rgw: be compatible with fmtlib v8

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-16 09:19:12 -05:00
Kefu Chai
97839b3428 osd/scrubber: mark fmt::formatter<..>::format() as const
as it should be const, and more importantly, it'd fail the build with
fmtlib v9, as its caller(s) is marked as `const`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 20:06:47 +08:00
Aashish Sharma
f2bb4897e2
Merge pull request #49173 from rhcs-dashboard/fix-encryption-config
mgr/dashboard: fix server side encryption config error
2022-12-16 17:05:13 +05:30
Kefu Chai
d012e47d86 rgw: be compatible with fmtlib v8
before fmtlib v9, fmtlib provides:

template<typename S, typename ...Args, typename Char = enable_if_t<detail::is_string<S>::value, char_t<S>>>
void fmt::print(std::basic_ostream<Char> &os, const S &format_str, Args&&... args)

but in fmtlib v9 and up, it provides:

template<typename ...T>
void fmt::print(std::ostream &os, format_string<T...> fmt, T&&... args)

so we need to use different function signatures for talking to different
fmtlib versions.

please see
https://fmt.dev/8.1.0/api.html#_CPPv4I0Dp0EN3fmt5printEvRNSt13basic_ostreamI4CharEERK1SDpRR4Args,
and
https://fmt.dev/9.0.0/api#_CPPv4IDpEN3fmt5printEvRNSt7ostreamE13format_stringIDp1TEDpRR1T

fore more details.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 19:18:05 +08:00
Aashish Sharma
a85ed9412f mgr/dashboard: Rbd Mirroring improvements
1.RBD images in dashboard shows default mirroring as journal
2.snapshot based mirroring schedule Interval got disabled to edit
3.unable to create snapshot of an image using dashboard
4.provide snapshot schedule info in a new column
5.dashboard doesn't allow importing peer bootstrap key to be imported for subsequent pools

Fixes: https://tracker.ceph.com/issues/58297
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 1a37c5e001e0bbc2720e6dfabf6ba04442acabb7)
2022-12-16 16:05:37 +05:30
Lucian Petrut
14c603a7c5 qa: add fs tests to test_rbd_wnbd.py
The Windows rbd-wnbd python test performs various IO operations
against raw disks.

However, it can be useful to test overlaying filesystems as well.
For this reason, we're adding the following tests:

* RbdFsTest
* RbdFsFioTest
* RbdFsStampFioTest

To simplify the implementation, those tests reuse the existing
ones along with a mixin class (RbdFsTestMixin).

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-12-16 12:20:02 +02:00
Lucian Petrut
d9decba6ba qa: test_rbd_wnbd.py improvements
We'll make the following improvements to the Windows rbd-wnbd
Python test:

* expose fio write validation, defaulting to crc32c
* change the default fio operation to "rw"
* enable the disk and clear the "rw" flag only if required by the
  test and if "--skip-enabling-disk" is not set (useful with custom
  SAN policies). This operation can take a significant amount of
  time under heavy load.
* print fio read and write results separately instead of
  aggregating them, useful when running rw tests

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-12-16 12:15:06 +02:00
Lucian Petrut
cceeb8be9f doc: document Windows SAN policy
The Windows SAN policy determines which disks will be mounted
automatically. We're going to document this as well as some
Windows limitations.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-12-16 12:14:54 +02:00
Kefu Chai
d9075c5fcf script/run-make: build with fmtlib 9.0.0
so we can identify changes which breaks the build with fmtlib v9.0
while running "make check" performed by jenkins.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 15:48:10 +08:00
Kefu Chai
051ec194df cmake: add an option named WITH_FMT_VERSION
so we can optionally force cmake to use the fmt submodule even if
fmtlib is available in system.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 15:48:10 +08:00
Kefu Chai
eb9771dbc0
Merge pull request #49452 from tchaikov/wip-crimson-test-fmt-v9
test/crimson: fixes related to fmtlib v9

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-16 15:46:59 +08:00
Aashish Sharma
7d2b83c57f mgr/dashboard: fix server side encryption config error
Fixes: https://tracker.ceph.com/issues/58296
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2022-12-16 10:57:36 +05:30
Kefu Chai
1fc102e6a3 test/crimson: include fmt/os.h
so we can format std::error_code

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
29b4c795d3 test/crimson: print conn instead of its address
now that crimson::net::Connection can be formatted using fmt::format(),
instead of printing the pointer, let's print its value.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
86be835fef test/crimson: trade operator<<() for fmt::format<>()
so we are able to format Payload using fmtlib v9. before fmtlib v9,
fmtlib is able to fall back to the operator<<() to format the the
types with the operator<<() defined. but after fmtlib v9, we need to
explicitly define them for accessing the specialized formatter.

now that we have specialized fmt::format<> for Payload, and the only
consumer of operator<<() is the fmtlib, we can safely drop the latter.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
5bb16452e9 test/crimson: specialize fmt::formater<> for types to be formatted
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<> is specialized for some types to be
formatted with fmtlib, so the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
19d69370c0 msg/msg_types.h: include fmt/format.h
for accessing the macro definition of FMT_VERSION, so we can have
definition of fmt::format<> if required.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
d887b97fe7 test/crimson: print pointer using fmt::ptr()
in fmtlib v9, it does not format typed pointers anymore, instead,
we need to print them using fmt::ptr().

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
3a1c5ac47d test/crimson: include system and 3rd-party headers first
see also https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
ethanwu
a4799689d5 mds: fix scan_stray_dir not reset next.frag on each run of stray inode
When we finish scanning one stray inode we should reset next.frag
so that next stray inode could start from its first dir fragment
rather than last stray inode's last frag.

Fixes: https://tracker.ceph.com/issues/58294
Signed-off-by: ethanwu <ethanwu@synology.com>
2022-12-16 09:44:38 +08:00
zdover23
5433784a3e
Merge pull request #49467 from zdover23/wip-doc-2022-12-16-rados-ec-default-simplest
doc/rados: clarify default EC pool from simplest

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-16 11:28:38 +10:00
Zac Dover
63c27ab8e7 doc/rados: clarify default EC pool from simplest
Distinguish between the default erasure-coded pool (k=2 m=2) and the
simplest erasure-coded pool (k=2 m=1).

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-16 11:07:51 +10:00