Commit Graph

126736 Commits

Author SHA1 Message Date
chunmei-liu
75d95dcda3 crimson/store-nbd: fix store_nbd build error for futurized store mkfs
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-09-23 21:20:39 -07:00
Samuel Just
4a9f1d7909
Merge pull request #43261 from rzarzynski/wip-crimson-mkfs_ertr
crimson/{common,os,osd}: errorate the FuturizedStore::mkfs() paths

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-23 19:49:08 -07:00
Samuel Just
d8ee8f03b9
Merge pull request #43288 from rzarzynski/wip-crimson-skip-our-frames-in-bt
crimson/common: skip first 4 frames when dumping a backtrace.

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2021-09-23 19:48:04 -07:00
Sebastian Wagner
166cd9976c
Merge pull request #43275 from adk3798/maint-target
mgr/cephadm: base maintenance mode enter/exit success/failure on returned message

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-23 22:15:28 +02:00
Sebastian Wagner
d5d1f154bc
Merge pull request #43121 from mgfritch/cephadm-pull-error
cephadm: raise error during `pull` failure

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
2021-09-23 22:12:01 +02:00
Sebastian Wagner
44eb16dd13
Merge pull request #43062 from strenuous-life/wip-cephadm-zap-device
mgr/cephadm: osd should not be zap when it is running

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-23 22:11:30 +02:00
Kamoltat Sirivadhna
e3bbfcfd2e
Merge pull request #37544 from kamoltat/wip-mgr-progress-global-efficiency
mgr/progress: optimize global recovery module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-09-23 14:12:16 -04:00
Radoslaw Zarzynski
295268a11a tests/crimson: make the virtual methods of SeaStoreTestState final.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 17:27:39 +00:00
Radoslaw Zarzynski
c04f5a2acf crimson/os: workaround the segfaulting GCC 11 issue.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 17:27:39 +00:00
Radoslaw Zarzynski
e5f7ff249d crimson/osd, crimson/os: errorate the FuturizedStore::mkfs() paths.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 17:27:27 +00:00
Radoslaw Zarzynski
56208ed326 crimson/common: skip first 4 frames when dumping a backtrace.
It's all about these items:

```
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
```

They are part of our backtrace handling and typically developers
are not interested in them. Let's be consistent with the classical
OSD and hide them.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 16:44:50 +00:00
Samuel Just
2612268e0d
Merge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit
crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-22 22:59:57 -07:00
Adam King
393330bae6 mgr/cephadm: base maintenance enter/exit success off of return message
rather than on whether there was any stdout from the command

Signed-off-by: Adam King <adking@redhat.com>
2021-09-22 20:21:10 -04:00
Adam King
5350d58b9a mgr/cephadm: unit tests for maintenance enter/exit properly handling success/failure messages
Signed-off-by: Adam King <adking@redhat.com>
2021-09-22 20:21:10 -04:00
Adam King
73e6aa8d9c cephadm: unit tests for maitenance mode return values
Signed-off-by: Adam King <adking@redhat.com>
2021-09-22 20:21:02 -04:00
Adam King
8bdddfa02e cephadm: fix mypy complaints for ThreadedChildWatcher class
Signed-off-by: Adam King <adking@redhat.com>
2021-09-22 17:50:44 -04:00
Adam King
3a15f1dc81 cephadm: fix exiting maintenance when systemd target doesn't exist
If the systemd target doesn't exist we need to just bypass enabling
it and return success or the host will just be stuck in maitnenance
mode.

Signed-off-by: Adam King <adking@redhat.com>
2021-09-22 17:50:44 -04:00
Kamoltat
fa92db1b37 mgr/progress: optimize global recovery module
Instead of fetching `pg_stats` from the python
part of manager module, we filter out the pgs
that are in active + clean state in ActivePyModules.cc
then parse these pgs along with `reported_epoch` and
the `total_num_pgs` of the clusters to global recovery
module.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-09-22 19:18:31 +00:00
Radoslaw Zarzynski
9c3f51e99b crimson/os/alienstore: fix nullptr deref in OnCommit::finish().
`seastar::engine()` is available only for Seastar's threads;
it shouldn't be called outside of a reactor thread.
Unfortunately, this assumption is violated in `AlienStore`
where `OnCommit::finish()`, executed from a finisher thread
of `BlueStore`, calls `alien()` on `seastar::engine()`.
The net effect are crashes like the following one:

```
INFO  2021-09-22 14:26:33,214 [shard 0] osd - operator() writing superblock cluster_fsid 1d8f7908-2ebf-4a91-ae70-f445668c126b osd_fsid 4da9fe9a-1da5-4ea9-aa79-a1178165ede5         [381/1839]
Segmentation fault.
Backtrace:
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
 4# (anonymous namespace)::OnCommit::finish(int) at /home/rzarzynski/ceph1/build/../src/crimson/os/alienstore/alien_store.cc:53
 5# Context::complete(int) at /home/rzarzynski/ceph1/build/../src/include/Context.h:100
 6# Finisher::finisher_thread_entry() at /home/rzarzynski/ceph1/build/../src/common/Finisher.cc:65
 7# 0x00007F16BBA0915A in /lib64/libpthread.so.0
 8# clone in /lib64/libc.so.6
Dump of siginfo:
  ...
  si_addr: 0x10
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-22 17:04:12 +00:00
Sébastien Han
d855dbb4d8
Merge pull request #40728 from guits/doc_fix_pattern_rgw_nfs
doc/rgw-nfs: use same pattern for keyring name
2021-09-22 17:08:31 +02:00
Samuel Just
e89ed326b5
Merge pull request #43254 from cyx1231st/wip-seastore-fix-onode-order
crimson/onode-staged-tree: convert hash to the reversed version

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-22 07:05:01 -07:00
Kefu Chai
143fa9b3f6
Merge pull request #43249 from cyx1231st/wip-seastore-fix-omap-hint
crimson/os/seastore: add missing hints in omap tree

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-22 21:47:37 +08:00
Daniel Gryniewicz
254bf8e883
Merge pull request #43055 from soumyakoduri/wip-skoduri-lua
rgw/lua: Install the packages only for RadosStore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Daniel Gryniewicz <dang@redhat.com>
2021-09-22 08:12:23 -04:00
Daniel Gryniewicz
a5ffc44435
Merge pull request #43054 from soumyakoduri/wip-skoduri-dbstore-vstart
rgw: Add option to configure backend store

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-09-22 08:11:30 -04:00
Daniel Gryniewicz
2bdef857fc
Merge pull request #42911 from soumyakoduri/wip-skoduri-dbstore-object
rgw/dbstore object APIs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-09-22 08:10:52 -04:00
Sebastian Wagner
a2d9839f58
Merge pull request #43241 from sebastian-philipp/suites-orch-labelere
.github: fix path to cephadm suite

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-09-22 11:34:35 +02:00
Yingxin Cheng
0c1c972866 crimson/onode-staged-tree: consolidate laddr hint calculation
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-09-22 15:55:09 +08:00
Yingxin Cheng
179f826919 crimson/onode-staged-tree: convert hash to the reversed version
Store the reversed version of object hash to make sure that onodes in
the same PG are sorted together.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-09-22 15:55:09 +08:00
Yingxin Cheng
18bf246672 crimson/onode-staged-tree: print cursh-hash in hex mode
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-09-22 15:55:01 +08:00
Yingxin Cheng
30f22ecb02 crimson/os/seastore: add missing hints in omap tree
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-09-22 11:23:34 +08:00
Samuel Just
86b9f03094
Merge pull request #43247 from rzarzynski/wip-crimson-ertr-safe_then_unpack
crimson/common: add safe_then_unpack() to errorated futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-09-21 17:12:58 -07:00
Radoslaw Zarzynski
710b928f9d crimson/common: implement singleton_ec.
Unfortunately, GCC explodes when it sees that.

```
[rzarzynski@o06 build]$ ninja crimson-osd vstart
[1/3] Building CXX object src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o
FAILED: src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o
/usr/bin/ccache /opt/rh/gcc-toolset-9/root/usr/bin/c++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -DSEASTAR_API_LEVEL=6 -DWITH_SEASTAR=1 -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -Isrc/include -I../src -I../src/seastar/include -Isrc/seastar/gen/include -isystem boost/include -isystem include -isystem ../src/xxHash -isystem ../src/rapidjson/include -O2 -g -DNDEBUG -fPIC   -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -ftemplate-backtrace-limit=0 -Wno-non-virtual-dtor -std=gnu++17 -U_FORTIFY_SOURCE -Wno-maybe-uninitialized -DSEASTAR_SSTRING -Wno-error=unused-result -std=c++17 -MD -MT src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o -MF src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o.d -o src/crimson/os/cyanstore/CMakeFiles/crimson-cyanstore.dir/cyan_store.cc.o -c ../src/crimson/os/cyanstore/cyan_store.cc
during IPA pass: inline
../src/crimson/os/cyanstore/cyan_store.cc: In member function ‘std::string crimson::os::singleton_ec<MsgV>::this_error_category::message(int) const [with const char* MsgV = (& msg)]’:
../src/crimson/os/cyanstore/cyan_store.cc:97:17: internal compiler error: Segmentation fault
   97 |     std::string message([[maybe_unused]] const int ev) const final {
      |                 ^~~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
Preprocessed source stored into /tmp/ccFi4GGw.out file, please attach this to your bugreport.
ninja: build stopped: subcommand failed.
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-21 22:59:09 +00:00
Radoslaw Zarzynski
7f99a88ea9 crimson/common: add safe_then_unpack() to errorated futures.
It was a prerequisite for another commit I finally thrown
away. However, this little bit can be still be useful even
for the sake of compliance with the interruptible variant.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-21 21:34:00 +00:00
Radoslaw Zarzynski
06e19d817e crimson/common: assert_moveable() doesn't depend on 3rd party's always_false<>.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-21 21:28:02 +00:00
Samuel Just
2ec096b19c
Merge pull request #43243 from rzarzynski/wip-crimson-net-dangling-bindvec2
crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-21 11:56:12 -07:00
Samuel Just
ed9b233db0
Merge pull request #43209 from rzarzynski/wip-crimson-silent-check-bot
tests/crimson: don't be so verbose when run by the 'make check' bot.

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-21 11:55:30 -07:00
Patrick Donnelly
790b49a006 Merge PR #43219 into master
* refs/pull/43219/head:
	doc: clarify MDS failed state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-21 14:03:08 -04:00
Sage Weil
6bf32314f7 Merge PR #43156 into master
* refs/pull/43156/head:
	qa/suites/orch/rook/smoke: add flannel back in

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-21 11:37:50 -04:00
Radoslaw Zarzynski
71626b4729 crimson/net: fix dangling addrvec in bind(), the repeat_until_value() part.
This is a follow-up to commit 4802730827.

`seastar::do_with()` doesn't extend the life-time of the callable's

```cpp
auto
do_with_impl(T1&& rv1, T2&& rv2, More&&... more) {
    // ...
    auto&& just_values = cherry_pick_tuple(idx(), std::move(all));
    auto&& just_func = std::move(std::get<nr>(std::move(all)));
    // ...
    auto task = std::apply(
        [](auto&&... x) {
            return std::make_unique<internal::do_with_state<value_tuple, ret_type>>(std::forward<decltype(x)>(x)...);
        },
        std::move(just_values));
    auto fut = std::apply(just_func, task->data());
    // ...
}
```

closure. This means the `addrs` needs to be captured by value also
by the lambda passed to `repeat_until_value()`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-21 13:57:40 +00:00
Ilya Dryomov
be23d3cfbf
Merge pull request #43225 from guits/fix-iscsi-doc
doc/iscsi: fix some typos

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-21 14:35:59 +02:00
Sebastian Wagner
91a8b45fc8
.github: fix path to cepham suite
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-21 13:35:03 +02:00
Sebastian Wagner
cfae1cc08c
Merge pull request #43237 from javicacheiro/fix_upgrade_now_uses_quay.io
doc:  from v16.2.6 you have to point to the quay.io image

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-21 13:29:09 +02:00
Sebastian Wagner
7138738bfd
Merge pull request #43142 from sebastian-philipp/c-v-cephadm-osd-activate
doc/ceph-volume/activate: Users should call cephadm instead

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-21 13:12:07 +02:00
Sebastian Wagner
e4eaacbb89
Merge pull request #43162 from sebastian-philipp/cephadm-service-mgmt-reoder
doc/cephadm/service-management: Move "Updating Service Specifications" 

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-09-21 13:03:08 +02:00
Ernesto Puerta
fef5ab7b04
Merge pull request #43210 from rhcs-dashboard/config-e2e-failure
mgr/dashboard: Fix failing config dashboard e2e check

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-09-21 11:53:41 +02:00
Javier Cacheiro
961a4fcffa doc: from v16.2.6 you have to point to the quay.io image if using docker
New images are no longer uploaded to the Docker Hub registry, so Docker users should point the upgrade
command to the quay.io image.

Fixes: https://tracker.ceph.com/issues/52683

Signed-off-by: Javier Cacheiro <javier.cacheiro.lopez@cesga.es>
2021-09-21 11:50:04 +02:00
Brad Hubbard
8de72972d7
Merge pull request #43233 from badone/wip-install-xmlstarlet
Revert "rpm, debian: drop xmlstarlet from build deps"

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-21 15:29:28 +10:00
Brad Hubbard
9630b2d8df Revert "rpm, debian: drop xmlstarlet from build deps"
This reverts commit 707edc0b2d.

This commit was somewhat premature in removing the xmlstarlet
dependency.

Fixes: https://tracker.ceph.com/issues/52681

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-09-21 13:20:13 +10:00
Patrick Donnelly
541cc173c6 Merge PR #43179 into master
* refs/pull/43179/head:
	qa: lengthen grace for fs map showing dead MDS

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-09-20 20:48:00 -04:00
Patrick Donnelly
de40c30b90 Merge PR #43164 into master
* refs/pull/43164/head:
	test/libcephfs: put inodes after lookup

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-09-20 20:47:19 -04:00