Commit Graph

135801 Commits

Author SHA1 Message Date
Venky Shankar
ab8fbe4e59 Merge PR #48473 into main
* refs/pull/48473/head:
	qa: fix _run_mount_cmd() and _get_mount_cmd() args in vstart_runner

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-12-19 16:17:15 +05:30
Kefu Chai
4ffa58c889
Merge pull request #49490 from tchaikov/wip-pybind-mgr-tox-fixes
pybind/mgr: tox and test fixes

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2022-12-19 18:14:13 +08:00
Aashish Sharma
1c8cde77a1
Merge pull request #49049 from rhcs-dashboard/add-osd-blocklist-metric
mgr/dashboard: Add metric relative to osd blocklist
2022-12-19 13:37:12 +05:30
Aashish Sharma
1edef5576c
Merge pull request #49292 from rhcs-dashboard/rbd-mirror-improvements
mgr/dashboard: rbd-mirror improvements
2022-12-19 12:19:58 +05:30
Anthony D'Atri
d8986dd252
Merge pull request #49203 from ktdreyer/doc-simple-cephfs-create
doc/cephfs: simplify ceph fs volume create command
2022-12-19 01:46:06 -05:00
Yingxin Cheng
181702913f crimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t
* device_spec_t should not include denc and be packed at the same time
* the underlying type of enum:uint8_t should support denc_traits already

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2022-12-19 10:47:46 +08:00
cao.leilc
3cd6f03b58 rgw: fix result when user's topics is null
Fixes: https://tracker.ceph.com/issues/53370
Signed-off-by: caolei <cao.leilc@inspur.com>
2022-12-19 10:31:05 +08:00
Kefu Chai
97a5dd316e
Merge pull request #49489 from tchaikov/wip-cmake-with-fmt-version
cmake: set WITH_FMT_VERSION as a STRING option

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-12-19 08:47:29 +08:00
colemitchell
fe7b62d217
Merge pull request #49491 from zdover23/wip-doc-2022-12-18-rados-config-ceph-conf-prompts
doc/rados: add prompts to ceph-conf.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-18 13:09:54 -05:00
Zac Dover
874152535c doc/rados: add prompts to ceph-conf.rst
Add unselectable prompts to doc/rados/ceph-conf.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-19 01:29:33 +10:00
Kefu Chai
0cc649f029 pybind/mgr/tox.ini: add commas in "modules" variable
since tox v4.0.13, it parses the variables differently, so the newlines
in a variable are passed right to the command referencing it. so we now
have failure like:
```
flake8: commands[0] /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> flake8 --config=tox.ini alerts
flake8: commands[1] /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> balancer
flake8: exit 2 (0.00 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/src/pybind/mgr> balancer
flake8: FAIL ✖ in 3.33 seconds
```
so we have to add comma as line continuation separator to address
this problem.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 20:23:47 +08:00
Kefu Chai
5878b9486a pybind/mgr: s/setup(self)/setup_method(self)/
avoid pytest warnings like:

4: pg_autoscaler/tests/test_cal_final_pg_target.py::TestPgAutoscaler::test_even_pools_one_meta_three_bulk
4:   /home/kefu/dev/ceph/src/pybind/mgr/.tox/py3/lib/python3.10/site-packages/_pytest/fixtures.py:900: PytestRemovedIn8Warning: Support for nose tests is deprecated and will be removed in a future release.
4:   pg_autoscaler/tests/test_cal_final_pg_target.py::TestPgAutoscaler::test_even_pools_one_meta_three_bulk is using nose-specific method: `setup(self)`
4:   To remove this warning, rename it to `setup_method(self)`
4:   See docs: https://docs.pytest.org/en/stable/deprecations.html#support-for-tests-written-for-nose
4:     fixture_result = next(generator)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 20:23:47 +08:00
Kefu Chai
9f0ee97985 pybind/mgr/prometheus: avoid using distutils
to silence warnings like:

4: prometheus/module.py:35
4:   /var/ssd/ceph/src/pybind/mgr/prometheus/module.py:35: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
4:     v = StrictVersion(cherrypy.__version__)

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 20:23:46 +08:00
Kefu Chai
0d41777cea cmake: set WITH_FMT_VERSION as a STRING option
option() defines a boolean, so its default value cannot be populated
to find_package() as expected. so we need to set it as a cached STRING
variable.

this addresses the regression introduced by 051ec194df

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-18 10:47:42 +08:00
Kefu Chai
a2bed04323
Merge pull request #49430 from tchaikov/wip-fmtlib-v9
cmake, script: run "make check" with fmtlib v9

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-17 15:08:29 +08:00
zdover23
e2d33ddd85
Merge pull request #49394 from zdover23/wip-doc-2022-12-10-rbd-rbd-snapshot-edit
doc/rbd: refine rbd-snapshot.rst

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2022-12-17 01:03:22 +10:00
Zac Dover
292c826acd doc/rbd: refine rbd-snapshot.rst
Refine the text in rbd-snapshot.rst

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-17 00:28:06 +10:00
Casey Bodley
7471527448
Merge pull request #49479 from tchaikov/wip-rgw-fmt-v8
rgw: be compatible with fmtlib v8

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-16 09:19:12 -05:00
Kefu Chai
97839b3428 osd/scrubber: mark fmt::formatter<..>::format() as const
as it should be const, and more importantly, it'd fail the build with
fmtlib v9, as its caller(s) is marked as `const`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 20:06:47 +08:00
Aashish Sharma
f2bb4897e2
Merge pull request #49173 from rhcs-dashboard/fix-encryption-config
mgr/dashboard: fix server side encryption config error
2022-12-16 17:05:13 +05:30
Kefu Chai
d012e47d86 rgw: be compatible with fmtlib v8
before fmtlib v9, fmtlib provides:

template<typename S, typename ...Args, typename Char = enable_if_t<detail::is_string<S>::value, char_t<S>>>
void fmt::print(std::basic_ostream<Char> &os, const S &format_str, Args&&... args)

but in fmtlib v9 and up, it provides:

template<typename ...T>
void fmt::print(std::ostream &os, format_string<T...> fmt, T&&... args)

so we need to use different function signatures for talking to different
fmtlib versions.

please see
https://fmt.dev/8.1.0/api.html#_CPPv4I0Dp0EN3fmt5printEvRNSt13basic_ostreamI4CharEERK1SDpRR4Args,
and
https://fmt.dev/9.0.0/api#_CPPv4IDpEN3fmt5printEvRNSt7ostreamE13format_stringIDp1TEDpRR1T

fore more details.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 19:18:05 +08:00
Aashish Sharma
a85ed9412f mgr/dashboard: Rbd Mirroring improvements
1.RBD images in dashboard shows default mirroring as journal
2.snapshot based mirroring schedule Interval got disabled to edit
3.unable to create snapshot of an image using dashboard
4.provide snapshot schedule info in a new column
5.dashboard doesn't allow importing peer bootstrap key to be imported for subsequent pools

Fixes: https://tracker.ceph.com/issues/58297
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 1a37c5e001e0bbc2720e6dfabf6ba04442acabb7)
2022-12-16 16:05:37 +05:30
Lucian Petrut
14c603a7c5 qa: add fs tests to test_rbd_wnbd.py
The Windows rbd-wnbd python test performs various IO operations
against raw disks.

However, it can be useful to test overlaying filesystems as well.
For this reason, we're adding the following tests:

* RbdFsTest
* RbdFsFioTest
* RbdFsStampFioTest

To simplify the implementation, those tests reuse the existing
ones along with a mixin class (RbdFsTestMixin).

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-12-16 12:20:02 +02:00
Lucian Petrut
d9decba6ba qa: test_rbd_wnbd.py improvements
We'll make the following improvements to the Windows rbd-wnbd
Python test:

* expose fio write validation, defaulting to crc32c
* change the default fio operation to "rw"
* enable the disk and clear the "rw" flag only if required by the
  test and if "--skip-enabling-disk" is not set (useful with custom
  SAN policies). This operation can take a significant amount of
  time under heavy load.
* print fio read and write results separately instead of
  aggregating them, useful when running rw tests

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-12-16 12:15:06 +02:00
Lucian Petrut
cceeb8be9f doc: document Windows SAN policy
The Windows SAN policy determines which disks will be mounted
automatically. We're going to document this as well as some
Windows limitations.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-12-16 12:14:54 +02:00
Kefu Chai
d9075c5fcf script/run-make: build with fmtlib 9.0.0
so we can identify changes which breaks the build with fmtlib v9.0
while running "make check" performed by jenkins.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 15:48:10 +08:00
Kefu Chai
051ec194df cmake: add an option named WITH_FMT_VERSION
so we can optionally force cmake to use the fmt submodule even if
fmtlib is available in system.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 15:48:10 +08:00
Kefu Chai
eb9771dbc0
Merge pull request #49452 from tchaikov/wip-crimson-test-fmt-v9
test/crimson: fixes related to fmtlib v9

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-16 15:46:59 +08:00
Aashish Sharma
7d2b83c57f mgr/dashboard: fix server side encryption config error
Fixes: https://tracker.ceph.com/issues/58296
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2022-12-16 10:57:36 +05:30
Kefu Chai
1fc102e6a3 test/crimson: include fmt/os.h
so we can format std::error_code

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
29b4c795d3 test/crimson: print conn instead of its address
now that crimson::net::Connection can be formatted using fmt::format(),
instead of printing the pointer, let's print its value.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
86be835fef test/crimson: trade operator<<() for fmt::format<>()
so we are able to format Payload using fmtlib v9. before fmtlib v9,
fmtlib is able to fall back to the operator<<() to format the the
types with the operator<<() defined. but after fmtlib v9, we need to
explicitly define them for accessing the specialized formatter.

now that we have specialized fmt::format<> for Payload, and the only
consumer of operator<<() is the fmtlib, we can safely drop the latter.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
5bb16452e9 test/crimson: specialize fmt::formater<> for types to be formatted
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<> is specialized for some types to be
formatted with fmtlib, so the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
19d69370c0 msg/msg_types.h: include fmt/format.h
for accessing the macro definition of FMT_VERSION, so we can have
definition of fmt::format<> if required.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
d887b97fe7 test/crimson: print pointer using fmt::ptr()
in fmtlib v9, it does not format typed pointers anymore, instead,
we need to print them using fmt::ptr().

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
Kefu Chai
3a1c5ac47d test/crimson: include system and 3rd-party headers first
see also https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-16 10:37:25 +08:00
ethanwu
a4799689d5 mds: fix scan_stray_dir not reset next.frag on each run of stray inode
When we finish scanning one stray inode we should reset next.frag
so that next stray inode could start from its first dir fragment
rather than last stray inode's last frag.

Fixes: https://tracker.ceph.com/issues/58294
Signed-off-by: ethanwu <ethanwu@synology.com>
2022-12-16 09:44:38 +08:00
zdover23
5433784a3e
Merge pull request #49467 from zdover23/wip-doc-2022-12-16-rados-ec-default-simplest
doc/rados: clarify default EC pool from simplest

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-16 11:28:38 +10:00
Zac Dover
63c27ab8e7 doc/rados: clarify default EC pool from simplest
Distinguish between the default erasure-coded pool (k=2 m=2) and the
simplest erasure-coded pool (k=2 m=1).

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-16 11:07:51 +10:00
Kamoltat
f4bda36a60 mon/Elector.cc: Compress peer >= rank_size sanity check into send_peer_ping
Problem:

Currently, https://github.com/ceph/ceph/pull/44993
failed to completely fix:

https://tracker.ceph.com/issues/50089

There are certain code paths such as

Elector::handle_ping → Elector::begin_peer_ping →
Elector::send_peer_ping.

that when a monitor is removed before shutdown in
Cephadm it can hit the assert failure.

Solution:

Therefore, we have to enforce sanity checks on
all code paths.

We do this by compressing the `peer >= rank_size`
sanity check into `send_peer_ping`. We also make
`send_peer_ping` return true/false

caller of `send_peer_ping` would drop itself if
recieves a `false`.

Fixes: https://tracker.ceph.com/issues/58155

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-12-15 22:11:15 +00:00
Kamoltat Sirivadhna
4aa8af29ae
Merge pull request #48991 from kamoltat/wip-ksirivad-fix-bz-2121452
mon/Elector: Change how we handle removed_ranks and notify_rank_removed()
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2022-12-15 17:07:38 -05:00
Mike Perez
b1e79a0011
Merge pull request #49436 from Thingee/add-alex-to-board
doc/foundation: Adding Alex Marangone to the board
2022-12-15 13:30:07 -08:00
Laura Flores
f8b969785c
Merge pull request #49416 from ljflores/wip-telemetry-label
.github: automatically add telemetry label to relevant files
2022-12-15 11:04:27 -06:00
Yuri Weinstein
a55987be80
Merge pull request #48641 from NitzanMordhai/wip-nitzan-valgrind-supp-adding-rocksdb-dl-init
src/valgrind.supp: Adding know leaks unrelated to ceph

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-15 08:14:03 -08:00
Yuri Weinstein
ad6eb019f4
Merge pull request #48416 from Rethan/wip-bluestore-discard-thread
blk/KernelDevice: don't start discard thread if device not support_di…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2022-12-15 08:13:33 -08:00
Yuri Weinstein
4c3212dc2b
Merge pull request #40066 from ideepika/wip-mempool-buffers
os/bluestore: improve cache_onode and cache_buffer mempool accounting

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-15 08:13:06 -08:00
Daniel Gryniewicz
0ac32cffb2 RGW - Make sure PostObj set bucket on s->object
s->bucket is guaranteed to exist at this point, since this is a bucket
OP (and we use it earlier in the function).

Fixes: https://tracker.ceph.com/issues/57911

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-15 11:01:29 -05:00
Daniel Gryniewicz
c466fd7bc3 RGW - Zipper - Chown rework
Chown was not unlinking, and was partially leaving the ownership wrong.
Fix this by reworking chown.  Bucket::chown now just changes ownership
of the bucket from it's previous owner to a new one.  An Object::chown
was added to change the ownership of an object.  The chown admin API was
modified to loop through all the objects in a bucket, changing the
ownership, so that a normal Bucket::chown doesn't need to do that
anymore.

Fixes https://tracker.ceph.com/issues/57936

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-15 10:24:22 -05:00
Ken Dreyer
05d5fda821 doc/cephfs: simplify ceph fs volume create command
Make it easy for new users to choose a sensible default "cephfs"
filesystem name.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-12-15 10:05:04 -05:00
Daniel Gryniewicz
dfc3709403
Merge pull request #49435 from dang/wip-dang-motr-build
RGW - Fix DAOS and MOTR stores to not use rgw_obj_key::to_str()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-15 09:57:51 -05:00