Commit Graph

135415 Commits

Author SHA1 Message Date
Laura Flores
acc8c7e2ef qa/workunits/rados: skip running envlibrados rocksdb tests on ubuntu
This test passes on centos and rhel, but fails on ubuntu from an
invalid pointer. Since the envlibrados rocksdb tests are experimental
and don't have any actual users, we can just run them on rhel and
centos.

At the moment, the actual bug is not fully understood, but it was
decided that fixing it is low priority, and removing the test from
problematic distros is okay for the time being. This commit
is considered a workaround to the actual issue.

Related tracker: https://tracker.ceph.com/issues/57632
Signed-off-by: Laura Flores <lflores@redhat.com>
2023-01-17 11:38:19 -06:00
Kefu Chai
d1ca83cf47
Merge pull request #49190 from tchaikov/wip-fmt
fmt: pickup v9.1.0

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-14 00:40:48 +08:00
Ronen Friedman
aa49dee4e6
Merge pull request #49410 from ronen-fr/wip-rf-cinode-tm
mds: remove unused declaration

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2022-12-13 16:02:24 +02:00
Ronen Friedman
8c46e4c3a3 mds: remove unused declaraion
as some Jenkins builds are set to treat warnings as errors, and
complain.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-13 14:24:08 +02:00
Matan
1aa2887c91
Merge pull request #49250 from Matan-B/wip-matanb-debug-subsys-doc
doc/rados/troubleshooting/log-and-debug: Update according to subsys.h

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-13 10:54:48 +02:00
Venky Shankar
2accc3a06a
Merge pull request #48449 from batrick/i57851
pybind/mgr: use memory temp_store

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2022-12-13 14:18:34 +05:30
Avan
fed4c25933
Merge pull request #49329 from rhcs-dashboard/fix-cors
mgr/dashboard: allow Origin url for CORS if present in config 

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2022-12-13 12:41:59 +05:30
colemitchell
f1acb2862a
Merge pull request #49396 from zdover23/wip-doc-2022-12-13-glossary-secrets
doc/glossary - add "secrets"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-12 23:47:41 -05:00
Zac Dover
a36409d3de doc/glossary - add "secrets"
Add "secrets" to doc/glossary.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-13 14:39:35 +10:00
Yuri Weinstein
b31e5551cb
Merge pull request #49391 from ceph/wip-yuri-crontab-main
qa/tests: added quincy-p2p tp the mix

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-12 13:35:26 -08:00
Yuri Weinstein
0583d631a3 qa/tests: added quincy-p2p tp the mix
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2022-12-12 11:31:52 -08:00
Yuri Weinstein
69d485a182
Merge pull request #49251 from shreyanshjain7174/cppool_space
rados: fix extra tabs on warning for pool copy

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-12-12 11:01:15 -08:00
Yuri Weinstein
ed8f60c6dc
Merge pull request #47596 from YiteGu/dump-slow-ops-count
osd: add dump slow ops count

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-12-12 11:00:19 -08:00
Casey Bodley
032aaa1e39
Merge pull request #48711 from cfsnyder/wip-cfsnyder-del-perf-iops
rgw: add 'inline_data' zone placement info option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-12 12:20:52 -05:00
Laura Flores
1ef4b49b0f
Merge pull request #49278 from Matan-B/wip-matanb-EIO-flag-fix
test/librados/aio_cxx: Fix Pool EIO flag tests
2022-12-12 11:11:44 -06:00
Laura Flores
64d8276797
Merge pull request #49180 from anthonyeleven/anthonyeleven/rework-pool-create-pg-error-message
src/mon: clarify message for pool creation failure due to max_pgs_per_osd
2022-12-12 11:11:02 -06:00
Venky Shankar
aa403a9532 Merge PR #45192 into main
* refs/pull/45192/head:
	mds: use the whole string as the snapshot long name

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-12-12 19:43:55 +05:30
Venky Shankar
e6684fa2b3 Merge PR #45312 into main
* refs/pull/45312/head:
	qa: new snapshot test for snapshot naming limits
	mds: limit the snapshot names to 240 characters

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2022-12-12 19:42:55 +05:30
Venky Shankar
8e6fae4f13 Merge PR #48450 into main
* refs/pull/48450/head:
	qa: add test_mdsdir_scrub_backtrace
	mds: avoid ~mdsdir's scrubbing and reporting damage health status

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-12 19:42:22 +05:30
Venky Shankar
f80d1ad238 Merge PR #48642 into main
* refs/pull/48642/head:
	qa: Fix test_subvolume_snapshot_info_if_orphan_clone

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-12 19:41:47 +05:30
Venky Shankar
756312cde0 Merge PR #48786 into main
* refs/pull/48786/head:
	mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-12-12 19:38:10 +05:30
Venky Shankar
6a87166923 Merge PR #48873 into main
* refs/pull/48873/head:
	mount: fix mount failure with old kernels

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-12-12 19:37:31 +05:30
Venky Shankar
e73e113b43 Merge PR #49117 into main
* refs/pull/49117/head:
	mds: remove redundant frag check in MDBalancer::maybe_fragment()

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-12-12 19:36:12 +05:30
Adam King
dbc4fab63a
Merge pull request #48258 from adk3798/upgrade-mgr-loop
mgr/cephadm: fix handling of mgr upgrades with 3 or more mgrs

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-12-12 08:44:34 -05:00
Adam King
d3737a9011
Merge pull request #48550 from rkachach/fix_issue_57894
mgr/cephadm: moving Prometheus spec check to service_spec module

Reviewed-by: Adam King <adking@redhat.com>
2022-12-12 08:43:25 -05:00
Lucian Petrut
f07b8a7984
Merge pull request #49277 from petrutlucian94/fix_log_win32
log: fix stderr handling on Windows
2022-12-12 12:44:48 +02:00
Cory Snyder
9052ca420f rgw: add 'inline_data' zone placement info option
Adds a new RGW zone placement info option to control whether
an object's first data chunk is placed in the head object.
This allows admins to make a tradeoff between optimizing for
PUT/GET performance vs. DELETE performance for some cluster
configurations.

Fixes: https://tracker.ceph.com/issues/57965
Signed-off-by: Cory Snyder <csnyder@iland.com>
2022-12-12 03:24:35 -05:00
Yingxin
388e45d345
Merge pull request #49325 from liu-chunmei/crimson-reactor_utilization
crimson/osd: dump each shard seastar metrics

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-12 15:43:12 +08:00
colemitchell
6ebdce58a4
Merge pull request #49383 from zdover23/wip-doc-2022-12-12-rados-user-management-prompts
doc/rados: add prompts to user-management.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-11 23:32:46 -05:00
Zac Dover
46f9506e4c doc/rados: add prompts to user-management.rst
Add unselectable prompts to doc/rados/operations/user-management.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-12 14:25:07 +10:00
Ronen Friedman
4d5c1bfc21
Merge pull request #48545 from ronen-fr/wip-rf-slow-replica
osd/scrub: report replicas slow to respond to scrub requests

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-11 19:05:34 +02:00
Ronen Friedman
7730baa6c5 osd/scrub: modifying some tests to ignore late replicas
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
Ronen Friedman
1295b3752c osd/scrub: earlier response to interval changes
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
Ronen Friedman
09ec8f544f osd/scrub: minor logging changes
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
Ronen Friedman
411ea10084 osd/scrub: report replicas slow to repond to scrub requests
Implemented timeouts:

1: Slow-Secondary Warning:

Once at least half of the replicas have accepted the reservation, we
start reporting any secondary that takes too long (more than <conf>
milliseconds after the previous response received) to respond to the reservation
request.
(Why? because we have encountered real-life situations where a specific
OSD was systematically very slow to respond (e.g. 5 seconds in one case) to
the reservation requests, slowing the scrub process to a crawl).

2: Reservation Process Timeout:

We now limit the total time the primary waits for the replicas to
respond to the reservation request. If we do not get all the responses
(either Grant or Reject) within <conf> milliseconds, we give up and release all the
reservations we have acquired so far.
(Why? because we have encountered instances where a reservation request
was lost - either due to a bug or due to a network issue.)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
colemitchell
0fb0add0a5
Merge pull request #49376 from zdover23/wip-doc-2022-12-11-install-clone-source-fix-git-command
doc/install: update clone-source.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-10 16:31:53 -05:00
Zac Dover
344ef1553e doc/install: update clone-source.rst
Beef up clone-source.rst. Repair semantics. Add internal references. Fix
a broken "git submodule foreach git clean -fdx" command.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-11 07:24:09 +10:00
Kefu Chai
c273757601
Merge pull request #49375 from adk3798/partial-revert-49359
pybind/mgr: partial reversion of #49359

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 23:57:01 +08:00
colemitchell
1f24b872a8
Merge pull request #49368 from zdover23/wip-doc-2022-12-10-rados-upmap-prompts
doc/rados: add prompts to upmap.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 23:42:48 -05:00
colemitchell
fe6033809a
Merge pull request #49367 from zdover23/wip-doc-2022-12-10-rados-stretch-mode-prompts
doc/rados: add prompts to stretch-mode.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 23:42:12 -05:00
Zac Dover
06d8ac91b2 doc/rados: add prompts to upmap.rst
Add unselectable prompts to doc/rados/operations/upmap.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-10 14:34:46 +10:00
Zac Dover
bafe76c204 doc/rados: add prompts to stretch-mode.rst
Add unselectable prompts to doc/rados/operations/stretch-mode.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-10 14:15:45 +10:00
Kefu Chai
1168be8b04 rgw: drop rgw_obj_key::to_str()
rgw_obj_key::to_str() is mainly used by operator<<(ostream&, ..), so
we can just implement it with the specialization of
fmt::formatter<rgw_obj_key>. and let operator<<(ostream&, ..) to
call into fmt::format(..):

1. for better readability and
2. for probably better performance -- we don't need to do deep copy
   for constructing a `std::string` from a `char[]`.
3. for better standard compliance -- we don't need to use variable-length
   array in C++ code. it is a part of C99 standard. but not a C++ standard.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 12:08:38 +08:00
Kefu Chai
a322cb6330 cmake: link rgw_common against fmt::fmt
as it includes rgw_data_sync.h which in turn uses fmtlib.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 11:42:29 +08:00
colemitchell
018d4351bd
Merge pull request #49364 from zdover23/wip-doc-2022-12-10-rbd-rados-rbd-retrieving-image-information-typo
doc/rbd: remove typo and ill-formed command

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 22:31:57 -05:00
Zac Dover
8df894b753 doc/rbd: remove typo and ill-formed command
Ingest Ilya Dryomov's comments here https://github.com/ceph/ceph/pull/49340/files#r1044365857

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-10 13:20:17 +10:00
Kefu Chai
a0e1591063 rgw: define FMT_HEADER_ONLY using CMakeLists.txt
less repeatings this way. and the macro definition can be populated
to the targets linking against `rgw_common`. this is more maintainable,
as rgw executables and libraries always link against `rgw_common`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 09:58:04 +08:00
Kefu Chai
ff7a7386c1 fmt: pickup v9.1.0
fmt 9.0.0 dropped automatic `std::ostream` insertion operator discovery
when `fmt/ostream.h` to prevent ODR violations. instead of defining
`FMT_DEPRECATED_OSTREAM`, we took efforts to specialize
`fmt::formatter<..>` to be compatible with the new fmtlib. to avoid
breaking the build with fmt v9 and up, let's bump up the fmt submodule.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 09:58:04 +08:00
Kefu Chai
525616e658 rgw: pass fmt::format_string to pretty_print()
otherwise we'd have following compiling failure:

In file included from /var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:13:
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.h:345:28: error: call to consteval function 'fmt::basic_format_string<char, std::basic_string<char> &, rgw_obj_key &, unsigned long &, std::basic_string<char> &>::basic_format_string<char[59], 0>' is not a constant expressio
n
    fmt::print(*env->ostr, std::forward<T>(t)...);
                           ^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:4136:8: note: in instantiation of function template specialization 'pretty_print<const char (&)[59], std::basic_string<char> &, rgw_obj_key &, unsigned long &, std::basic_string<char> &>' requested here
              pretty_print(sc->env, "Syncing object s3://{}/{} version {} in sync from zone {}\n",
              ^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:4070:3: note: in instantiation of member function 'RGWBucketSyncSingleEntryCR<rgw_obj_key, rgw_obj_key>::operate' requested here
  RGWBucketSyncSingleEntryCR(RGWDataSyncCtx *_sc,
  ^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.cc:4352:27: note: in instantiation of member function 'RGWBucketSyncSingleEntryCR<rgw_obj_key, rgw_obj_key>::RGWBucketSyncSingleEntryCR' requested here
          yield spawn(new SyncCR(sc, sync_pipe, entry->key,
                          ^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.h:345:44: note: function parameter 't' with unknown value cannot be used in a constant expression
    fmt::print(*env->ostr, std::forward<T>(t)...);
                                           ^
/var/ssd/ceph/src/rgw/store/rados/rgw_data_sync.h:343:53: note: declared here
void pretty_print(const RGWDataSyncEnv* env, T&& ...t) {
                                                    ^

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 09:58:04 +08:00
Kefu Chai
c7e3e95277 rgw: specialize fmt::formatter<> for rgw_obj_key
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<rgw_obj_key> is defined so the tree can
compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 09:58:01 +08:00