Commit Graph

133654 Commits

Author SHA1 Message Date
Daniel Gryniewicz
285326cd45 RGW - Zipper - Pass invaliate down from RadosStore
When we invalidate the state of a RadosObject, we need to also
invalidate it in the ctx for that object.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-31 11:16:25 -04:00
Ernesto Puerta
c25bde9c5e
Merge pull request #47641 from rhcs-dashboard/value-error-centos
install-deps: script exit on "/ValueError" in centos_stream8

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2022-08-31 15:59:03 +02:00
Daniel Gryniewicz
f8526840a0 RGW - Zipper - Remove a number of casts from rgw_admin
There are still a ton of casts to RadosStore in rgw_admin.  Remove the
easy ones.  Many of the rest represent actual operations that are
specific to RadosStore, and need to be split out.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-31 09:44:01 -04:00
Pedro Gonzalez Gomez
b99845174b
Merge pull request #47673 from rhcs-dashboard/test_rbd_list
mgr/dashboard: ensure rbd image limit 0 returns 0 images


Reviewed-by: Pegonzal <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-08-31 15:08:19 +02:00
Ionut Balutoiu
2d6305cb61 qa: update branch name for Windows job
Use `main` instead of `master` in the workunit scripts for the
Windows Teuthology job.

Signed-off-by: Ionut Balutoiu <ibalutoiu@cloudbasesolutions.com>
2022-08-31 12:06:14 +00:00
Redouane Kachach
1c8833feaf
mgr/cephadm: Fix how we check if a host belongs to public network
Fixes: https://tracker.ceph.com/issues/57060

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-08-31 13:49:37 +02:00
Kefu Chai
0daef7c82e
Merge pull request #47802 from NitzanMordhai/wip-nitzan-leak-some-memory-with-memory-optimization
common/ceph_context: leak some memory fail to show in valgrind

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2022-08-31 17:55:05 +08:00
Kefu Chai
2e9f5ae081
Merge pull request #47829 from tchaikov/wip-cmake-CMP0135
cmake: set CMP0135 policy 

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-31 17:51:44 +08:00
Redouane Kachach
1a722f35c1
mgr/cephadm: allow binding to loopback for rgw daemons
Fixes: https://tracker.ceph.com/issues/57304

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-08-31 11:40:23 +02:00
Ilya Dryomov
1cec9e83c0 test/cli-integration/rbd: iSCSI REST API responses aren't pretty-printed anymore
See https://github.com/ceph/ceph-iscsi/pull/263 and
https://github.com/pallets/flask/pull/2193.  Flask stopped
pretty-printing by default in 1.0:

  Change the default for JSONIFY_PRETTYPRINT_REGULAR to False.
  json.jsonify returns a compact format by default, and an indented
  format in debug mode.

Fixes: https://tracker.ceph.com/issues/57343
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-08-31 11:01:12 +02:00
J. Eric Ivancich
91c57c3fb1
Merge pull request #47772 from ivancich/wip-dout_subsys-header
rgw: remove dout_subsys defs from header files

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-08-30 18:11:00 -04:00
J. Eric Ivancich
bccf6c3074 rgw: remove dout_subsys defs from header files
Each compilation unit should be able to define its own dout_subsys
without generating a redefinition warning. When dout_subsys is defined
in header files, it complicates this matter. This commit removes
definitions and header files and makes sure definitions are added to
.cc files as needed.

Additionally, at Adam Emerson's suggestion, use "static constexpr"
rather than "#define" to set "dout_subsys" in a few places as a
reminder to ultimately do it more broadly.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2022-08-30 13:04:32 -04:00
Kefu Chai
976cd1a4e7
Merge pull request #47761 from tchaikov/wip-intarith-bit
include, os: s/ctz/std::countr_zero/

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-30 23:02:46 +08:00
Kefu Chai
f68dbeae37
Merge pull request #47830 from tchaikov/wip-fmtlib-v9
common, include, osd: compile with fmt v9

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2022-08-30 22:19:27 +08:00
Kefu Chai
63744eb5c7
Merge pull request #44993 from kamoltat/wip-ksirivad-fix-bug-50089
mon/Elector: Added sanity check when pinging a peer monitor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2022-08-30 21:19:01 +08:00
zdover23
7a6bd1f963
Merge pull request #47843 from zdover23/wip-doc-2022-08-29-mgr-dashboard-includes-prompt-update
doc/mgr: update prompts in dboard.rst includes

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-08-30 21:14:17 +10:00
Pere Diaz Bou
1c5a8ad21f
Merge pull request #46962 from rhcs-dashboard/auto-coloring-badges-from-color-pool
mgr/dashboard: auto-coloring-badges-component

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: nSedrickm <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-08-30 12:59:01 +02:00
Arthur Outhenin-Chalandre
f744a93ef1
Merge pull request #47707 from bosc0/fix_alert
Ceph-mixin: Fix CephNodeNetworkPacket alerts
2022-08-30 12:49:23 +02:00
Ilya Dryomov
ced071f0de rbd-mirror: skip setting error code on snapshot replayer shutdown
This is regarding failures in unregister_remote_update_watcher() and
unregister_local_update_watcher().  handle_replay_complete() can't be
called in these cases anymore as it would blindly attempt to unregister
watchers from scratch again.  Dropping handle_replay_complete() calls
there means that these failures would only be logged and would not be
surfaced by snapshot replayer.  But the only caller ignores them
anyway:

  void ImageReplayer<I>::shut_down(int r) {
    ...
    // close the replayer
    if (m_replayer != nullptr) {
      ctx = new LambdaContext([this, ctx](int r) {
        m_replayer->destroy();
        m_replayer = nullptr;
        ctx->complete(0);             <------
      });
      ctx = new LambdaContext([this, ctx](int r) {
        m_replayer->shut_down(ctx);
      });
    }

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-08-30 12:01:35 +02:00
Nizamudeen A
b73d7d22d4 install-deps: script exit on /ValueError: in centos_stream8
this is happening locally as well as in our ceph-dev runs too https://github.com/rhcs-dashboard/ceph-dev/runs/7850564011

Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-08-30 14:50:40 +05:30
Arthur Outhenin-Chalandre
4909e795c9
Merge pull request #47669 from MrFreezeex/jb-path
ceph-mixin: fix PATH issues with jsonnet-bundler
2022-08-30 08:35:04 +02:00
Adam King
5444c98697
Merge pull request #47808 from phlogistonjohn/jjm-remove-tox-env-lint
pybind/mgr: tox.ini remove redundant `tox` env

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-08-29 15:52:17 -04:00
Ilya Dryomov
fc4cc575bc rbd-mirror: resume pending shutdown on error in snapshot replayer
If a shutdown is requested, e.g. by update_pool_replayers() because
remote RADOS instance got blocklisted, and Replayer::shut_down() pends
it on completion of current snapshot sync, it gets stuck if replayer
encounters an error in the interim.  This is particularly likely in the
blocklist case: a higher layer may detect that client got blocklisted
and request a shutdown first, and then when replayer sees EBLOCKLISTED
in turn, it calls handle_replay_complete() -- which does not resume
a pending shutdown.  Because update_pool_replayers() blocks on shutdown
with Mirror::m_lock held, eventually the entire daemon hangs in
perpetuity.

Fixes: https://tracker.ceph.com/issues/56154
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-08-29 20:14:17 +02:00
Pedro Gonzalez Gomez
29f510a522 mgr/dashboard: auto-coloring-badges-component
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2022-08-29 18:03:11 +02:00
Kefu Chai
28d890ebf6
Merge pull request #47834 from tchaikov/wip-56850
mon/MgrMonitor: do not propose again for "mgr fail"

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-08-30 00:00:37 +08:00
Patrick Seidensal
79534e5dc3
mgr: Fix tox test issue due to PrettyTable update
Since the update of PrettyTable 3.3.0 to 3.4.0 on Aug 25, 2022, three
tests inside tox broke as the new version returns tables slightly
differently.

Alternatively, the tests could be adapted.

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2022-08-29 17:02:06 +02:00
Patrick Seidensal
3551d7f8b3
mgr/dashboard: enable addition custom Prometheus alerts
Fixes: https://tracker.ceph.com/issues/57294

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2022-08-29 17:01:21 +02:00
Casey Bodley
183a05643d
Merge pull request #47441 from mkogan1/wip-WITH_BOOST_VALGRIND
cmake: set WITH_BOOST_VALGRIND before building boost

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-08-29 09:51:21 -04:00
Arthur Outhenin-Chalandre
ff06790603
Merge pull request #47675 from bosc0/fix_multicluster
ceph-mixin: fix config inheritance
2022-08-29 14:55:43 +02:00
Zac Dover
fc70ccde75 doc/mgr: update prompts in dboard.rst includes
This PR adds unselectable prompts to three files that are
transcluded in the doc/mgr/dashboard.rst file. These three
files are:

 1. debug.inc.rst
 2. feature_toggles.inc.rst
 3. motd.inc.rst

The addition of unselectable prompts to these three files
completes the work begun in PR#47810 (d8064b4), which sought
to bring dashboard.rst into line with the unselectable prompt
standard introduced by Kefu Chai in 2020.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-08-29 10:39:51 +10:00
Ronen Friedman
07bfcac5ea
Merge pull request #47681 from ronen-fr/wip-rf-delog1
osd/scrub: improving scrub logs
Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
2022-08-28 14:10:10 +03:00
Yuval Lifshitz
30078cd292
Merge pull request #47728 from yuvalif/wip-yuval-cont-lease
rgw/multisite: check for late lease renewals

reviewed-by: cbodley
2022-08-28 10:44:07 +03:00
Ronen Friedman
95dd59ba87 osd/scrub: reduce the amount of log lines
Following call chains, and removing repeated data.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-08-28 08:47:46 +03:00
Ronen Friedman
93dfb7e88b common: improving fmtlib handling of ceph::utime_t
1. fixing the output to show local-time instead of UTC format, matching
   operator<<() handling (and all the rest of our logs)
2. adding a 'short' mode (as {:s}) for when, e.g. in most scrub logs,
   we only need 3 digits for the sub-second, and do not need the
   trailing TZ designation.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-08-28 08:47:46 +03:00
zdover23
2b7c0a1238
Merge pull request #47736 from ceph/release-docs
doc: Update release process doc to accurately reflect current process

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2022-08-28 07:13:39 +10:00
David Galloway
4829af891c doc: Update release process doc to accurately reflect current process
Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-08-28 06:40:49 +10:00
Kefu Chai
3d7a895d19 include/object_fmt: mark fmt::formatter<>::format() const
so these formatters can be used in methods with `const` specifier.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-27 23:59:24 +08:00
Kefu Chai
2901943618 common/LogEntry: specialize fmt::formatter<LogEntry>
so we can use the formatter defined for `LogEntry` in fmtlib v9.
in this new version of fmtlib, it is required to define a specialization
for the formatted type even when it comes to the types with an override of
operator<<(). since we already have an override for `LogEntry`, let's define
the specialization for `fmt::formatter<LogEntry>`.

this change should address the FTBFS when building with fmtlib v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-27 23:59:24 +08:00
Kefu Chai
6e223a6802 common/Journald: include msg/msg_fmt.h
so we can use the formatter defined for `entity_name_t`. in fmtlib v9,
it is required to define a specialization for the formatted type even
the type has an override of operator<<(). now that we already have a
formatter for `entity_name_t`, let's just use it.

this change should address the FTBFS when building with fmtlib v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-27 23:59:24 +08:00
Ilya Dryomov
de0ba80b37 librbd: use actual monitor addresses when creating a peer bootstrap token
Relying on mon_host config option is fragile, as the user may confuse
v1 and v2 addresses, group them incorrectly, etc.  Get mon_host value
only as a fallback.

Fixes: https://tracker.ceph.com/issues/57317
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-08-27 17:58:37 +02:00
Kefu Chai
5b1c6ad496 mon/MgrMonitor: do not propse again for "mgr fail"
in 23c3f76018, the change to fail the mgr
is proposed immediately. but `MgrMonitor::prepare_command()` method still
returns `true` in this case. its indirect caller of
`PaxosService::dispatch()` considers this as a sign that it needs to
propose the change with `propose_pending()`. but the pending change has
already been proposed by `MgrMonitor::prepare_command()`, and
`have_pending` is also cleared by this call. as we don't allow
consecutive paxos proposals, the second `propose_pending()` call is
delayed with a configured latency. but when the timer is fired, this
poseponed call would find itself trying to propose nothing. the change
to fail the mgr has been proposed. that's why we have
`ceph_assert(have_pending)` assertion failures.

in this change, the second proposal is not proposed anymore if the
proposal is proposed immediately. this should avoid the assertion
failure.

this change should address the regression introduced by
23c3f76018.

Fixes: https://tracker.ceph.com/issues/56850
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-27 23:57:40 +08:00
Duncan Bellamy
5120789f88 rgw: rgw_d3n_datacache.cc add check for musl
aio_init is glibc specific

Fixes: https://tracker.ceph.com/issues/54414
Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>
2022-08-27 10:28:35 +01:00
Lucian Petrut
c95b6b6c77 include: fix IS_ERR on Windows
The "long" type uses 32b on x64 Windows platforms, which means
it's not large enough to store a pointer. intptr_t or uintptr_t
should be used instead.

This change fixes include/err.h, using the right types. There was
a previous patch on this topic but unfortunately it didn't address
all the type casts.

This issue was brought up by the unittest_crush test, which recently
started to fail as the CrushWrapper methods use IS_ERR.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2022-08-27 09:16:50 +00:00
Kefu Chai
8103d11bd8 cmake: set CMP0135 policy
so the `DOWNLOAD_EXTRACT_TIMESTAMP` property of
`ExternalProject_Add()` command is set by default on CMake v3.24 and up.
it helps to set the a more accurate timestamp for the downloaded
content, hence the targets depending on the extracted content can be
rebuilt if the URL changes.

see also https://cmake.org/cmake/help/latest/policy/CMP0135.html

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-27 09:56:47 +08:00
Kefu Chai
591e0c6195 cmake: use a loop for setting new policies
we were using a for loop for this purpose, but the for loop was unrolled
when we bumped up the required cmake version.

this change paves the road to setting "CMP0135" to "NEW". this policy
is a new one introduced by CMake v3.24.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-27 09:49:27 +08:00
zdover23
75d4ce7169
Merge pull request #47810 from zdover23/wip-doc-2022-08-25-mgr-dashboard-prompt-correct
doc/mgr: add prompt directives to dashboard.rst

Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-08-27 05:45:28 +10:00
Laura Flores
8ccd4e2533
Merge pull request #47046 from rzarzynski/wip-dup-trimming-test2
osd, tools, kv: non-aggressive, on-line trimming of accumulated dups
2022-08-26 14:07:44 -05:00
Yehuda Sadeh
e28043f7af rgw: cloud sync: sign requests that read source objs
Fixes: https://tracker.ceph.com/issues/57307

Requests that were sent to fetch the objects from the source zone weren't
being signed. Changing the interface to use rRGWRESTStreamRWRequest::send()
that signs the requests.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2022-08-26 09:56:38 -04:00
Yehuda Sadeh
ae67726441 rgw: cloud sync: fix crash due to objs on cr stack
Fixes: https://tracker.ceph.com/issues/57306

When passing objects that are declared on the stack (as ref / ptr), called
cr can't access them (other than for initialization), as the yield destroys
them. We now keep these objects as class members.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2022-08-26 09:49:01 -04:00
Rishabh Dave
e7b6c9d8c2
Merge pull request #46994 from rishabh-d-dave/qa-caps-helper-upgrade
qa/cephfs: upgrade caps_helper.py

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-08-26 18:43:52 +05:30