Commit Graph

130541 Commits

Author SHA1 Message Date
Christoph Glaubitz
452e52a7e3
mgr/cephadm: support non-root ssh-user w permissions
Restructured code, so that in case of non-root, the resulting file will
be created with permissions set to the ssh-user. This allows the
subsequent scp to be able to write the file. The remaining code kept the
same, so that file permissions are restored to the expected ones, but
just runs after the scp.

Fixes: https://tracker.ceph.com/issues/54620
Signed-off-by: Christoph Glaubitz <c.glaubitz@syseleven.de>
2022-03-31 15:06:15 +02:00
Radoslaw Zarzynski
d04ea43cfd bdev: fix FTBFS on FreeBSD, keep the huge paged read buffers.
Special thanks to Willem Jan Withagen!

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-31 13:27:43 +02:00
Ilya Dryomov
d191b08bb6
Merge pull request #45660 from CongMinYin/fix-pwl-status
librbd/cache/pwl: correct cache state

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-31 12:30:03 +02:00
Nikhil Kshirsagar
1109e62f6a mon: Resolve unhandled exception in
__generate_command_map()

If improper json data is passed to rados, it can
end up crashing the mon. This fix will catch the
exception if thrown by __generate_command_map()
and avoid the mon getting terminated due to an
uncaught exception.

Fixes: https://tracker.ceph.com/issues/54558

Signed-off-by: Nikhil Kshirsagar <nkshirsagar@gmail.com>
2022-03-31 09:19:46 +05:30
Travis Nielsen
9cca95b16a
prometheus: spell check the alert descriptions
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-30 17:38:43 -06:00
Radosław Zarzyński
fc3d1b2f01 crimson/common: let errorator's handlers to conditionally forward errors
The use case is like the following one:

```diff
diff --git a/src/crimson/osd/replicated_recovery_backend.cc b/src/crimson/osd/replicated_recovery_backend.cc
index a96f1b8dd2b..055add2e407 100644
--- a/src/crimson/osd/replicated_recovery_backend.cc
+++ b/src/crimson/osd/replicated_recovery_backend.cc
@@ -516,10 +516,14 @@ ReplicatedRecoveryBackend::read_object_for_push_op(
       recovered_to = push_op->data_included.range_end();
     }
     return seastar::make_ready_future<uint64_t>(recovered_to);
-  }, PGBackend::read_errorator::all_same_way([](auto e) {
+  }, PGBackend::read_errorator::all_same_way([](auto e, auto&& e_raw) -> PGBackend::read_errorator::future<uint64_t> {
     logger().debug("build_push_op: read exception");
-    return seastar::make_exception_future<uint64_t>(e);
-  }));
+    if (42) {
+      return std::move(e_raw);
+    } else {
+      return seastar::make_exception_future<uint64_t>(e);
+    }
+  })).handle_error_interruptible(PGBackend::read_errorator::discard_all{});
 }

 RecoveryBackend::interruptible_future<>
```

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-03-31 01:17:16 +02:00
Adam King
6e4dd0e082
Merge pull request #45361 from adk3798/mds-upgrade-test
qa/suites/fs: stop looping in mds upgrade test if upgrade failed

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-03-30 18:56:45 -04:00
Guillaume Abrioux
a5fab15e44 ceph-volume/tests: reject loop devices in lvm.conf
The current task doesn't works (typo?).
Otherwise api/lvm.py can't work properly, functions such as
`get_single_lv()` and many other don't return the expected results.
Indeed, lvm is confused because of the nvme_loop setup.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Guillaume Abrioux
7f007e7fc7 ceph-volume: do not leave pv when zapping osds
when zapping a device and no vg/lv are left, the pv should be
removed too.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Guillaume Abrioux
8aa2f4745a orchestrator: support complex osd creation
This adds the support of complex OSD creation with command
`orch daemon add osd`.
Any argument supported by `DriveGroupSpec()` can be passed on the command line.

Usage:
```
ceph orch daemon add osd host:data_devices=device1,device2,db_devices=device3,osds_per_device=2,...
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Guillaume Abrioux
3a88547559 DriveSelection: skip unavailable devices
Cephadm shouldn't try to deploy a disk reported as unavailable by ceph-volume.
The idea here is to check the rejection reason so we can still use DB devices
in case of OSD replacement.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Guillaume Abrioux
9f4b830dcf ceph-volume: various fixes in arg_validators
if a device with an FS is passed, ceph-volume should abort
the OSD creation.

Fixes: https://tracker.ceph.com/issues/54535

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Guillaume Abrioux
d43189c17b doc/cephadm: fix a typo
s/osd_crush_choose_leaf_type/osd_crush_chooseleaf_type

```
[ceph: root@adm-1 /]# ceph config set global osd_crush_choose_leaf_type 0
Error EINVAL: unrecognized config option 'osd_crush_choose_leaf_type'
[ceph: root@adm-1 /]# ceph config set global osd_crush_chooseleaf_type 0
[ceph: root@adm-1 /]#
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Guillaume Abrioux
cd5eb7939e ceph-volume/tests: speed up tox tests
Let's use `--numprocesses=auto` in order to speed up the unit tests execution.

See the difference, without `--numprocesses=auto`:
```

... omitted output ...

real    1m22.884s
user    0m23.003s
sys     0m20.504s
```

with `--numprocesses=auto`:

```

... omitted output ...

real    0m18.767s
user    0m33.056s
sys     0m23.244s
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-30 23:49:01 +02:00
Adam King
7e41eb9f93
Merge pull request #45248 from adk3798/offline-thread
mgr/cephadm: offline host watcher

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-03-30 13:50:07 -04:00
Adam King
5a97bc15b7
Merge pull request #45344 from adk3798/raw-prepare-cmd
mgr/cephadm: generate one c-v raw prepare cmd per data device in raw mode

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-03-30 13:48:43 -04:00
Casey Bodley
7989a028c5
Merge pull request #45567 from cbodley/wip-55000
rgw multisite: data sync uses yield_spawn_window()

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-03-30 12:24:56 -04:00
Casey Bodley
9ce4431f26
Merge pull request #45515 from tchaikov/wip-cxx20-fixes-rgw
rgw: C++20 related fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-30 12:03:15 -04:00
Kefu Chai
fd5b4a5206
Merge pull request #45536 from tchaikov/wip-std-optional
compressor/QatAccel: s/boost::optional/std::optional/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-30 23:16:15 +08:00
Kefu Chai
4875c65998
Merge pull request #45648 from tchaikov/wip-kill-distutils
cmake: avoid using distutils

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-03-30 22:56:32 +08:00
Yuri Weinstein
daea072720
Merge pull request #45694 from amathuria/amathuri-53923-fix
osd/osd_types: Increasing decode version of scrub_duration in pg stats

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-30 07:16:44 -07:00
Kefu Chai
35f56a1850
Merge pull request #45142 from SoftIron/ceph-mgr-cephadm-python3-cherrypyTHREE
debian/control: fix python3-cherrypy*3* dependency

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-30 21:57:19 +08:00
Yin Congmin
6593e31fff librbd/cache/pwl: correct cache state
update cache state after dirty_entries or log_enties list updated.

Fixes: https://tracker.ceph.com/issues/50614

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2022-03-30 21:45:29 +08:00
Adam King
627a9df99d
Merge pull request #45217 from rkachach/fix_issue_47872
mgr/cephadm: check spec host when adding osd

Reviewed-by: Adam King <adking@redhat.com>
2022-03-30 09:01:40 -04:00
Adam King
5b49c10aaf
Merge pull request #45275 from rkachach/fix_issue_54342
mgr/cephadm: adding HostSpec validation

Reviewed-by: Adam King <adking@redhat.com>
2022-03-30 08:59:15 -04:00
Adam King
f1760bed61
Merge pull request #45534 from rkachach/fix_issue_50804
mgr/cephadm: Show warning when user provides --fsid option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-03-30 08:58:04 -04:00
Adam King
c985bd639c
Merge pull request #45692 from rkachach/fix_issue_55113
mgr/cephadm: fallback to normal sorted if cannot import natsorted

Reviewed-by: Adam King <adking@redhat,com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2022-03-30 08:48:33 -04:00
Kaleb S. KEITHLEY
38eeb1bc3e rgw: add missing virtual decls for dtors
As a side effect or benefit of zipper loadable module work have
found several classes that are missing virtual decl on dtors, or
are missing a dtor decl entirely.

In the zipper loadable module work these missing decls result in
undefined references the vtable for the various classes during the
link.

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2022-03-30 08:26:23 -04:00
Radosław Zarzyński
c976e67e3a crimson/common: avoid unnecessary throwing in stateful_error_t
This is an optimization for the flexible-but-slow `stateful_error_t`
error family of errorator. It's used infrequently.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-03-30 14:21:25 +02:00
Koen Kooi
b7b381fe91 debian/control: fix python3-cherrypy*3* dependency
The trailing '3' was missed in one instance, ceph-mgr-cephadm, leading to:

	Some packages could not be installed. This may mean that you have
	requested an impossible situation or if you are using the unstable
	distribution that some required packages have not yet been created
	or been moved out of Incoming.
	The following information may help to resolve the situation:

	The following packages have unmet dependencies:
	 ceph-mgr-cephadm : Depends: python3-cherrypy but it is not installable

Which makes the installation fail.

Fixes: 78983ad0d0cce422da32dc4876ac186f6d32c3f5
Signed-off-by: Koen Kooi <koen@softiron.com>
2022-03-30 05:18:51 -07:00
Ernesto Puerta
043f7953d8
Merge pull request #45335 from rhcs-dashboard/fix-54513-master
mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-03-30 14:05:38 +02:00
Ernesto Puerta
751d3df47b
Merge pull request #45559 from rhcs-dashboard/existing-host-test-fix
mgr/dashboard: fix 01-hosts.e2e test error

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-03-30 13:00:36 +02:00
Anthony D'Atri
39e8ec9389
Merge pull request #45701 from inspur-wyq/wip-doc-typos
doc/radosgw/sync-modules.rst: fix typos
2022-03-29 23:54:42 -07:00
Anthony D'Atri
2df627751b
Merge pull request #45702 from inspur-wyq/wip-doc-my-typos
doc/cephadm/operations.rst: fix typos
2022-03-29 23:43:46 -07:00
Tim Serong
94ad178bdc ceph.spec.in: remove build directory in %clean, not %install
Removing the build directory at the end of %install is too soon,
and means we get rid of a bunch of stuff needed to correctly
create debuginfo/debugsource packages, which happens automatically
right after %install.  So, let's put it where it really belongs, in
the %clean section.

Fixes: aa18cb12003e3526c8e8f23dc2335a483fbfa68e
Fixes: https://tracker.ceph.com/issues/55079
Signed-off-by: Tim Serong <tserong@suse.com>
2022-03-30 16:25:34 +11:00
Samuel Just
b73ed7bb39
Merge pull request #45630 from liu-chunmei/crimson-omap-cmp
crimson: add omap cmp

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-29 21:40:49 -07:00
wangyunqing
92eb799a95 doc/cephadm/operations.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2022-03-30 11:58:43 +08:00
wangyunqing
07584628d6 doc/radosgw/sync-modules.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2022-03-30 11:53:57 +08:00
Samuel Just
32ee47a864
Merge pull request #45526 from xxhdx1985126/wip-coordinate-segment-seq
crimson/os/seastore: coordinate segment seq of journal and ool segments

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-29 20:53:30 -07:00
Chunmei Liu
7244b80956 crimson: return 0 in osd_op.rval when enodata
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-29 17:44:54 -07:00
Chunmei Liu
019c4128cb crimson: add OP_OMAP_CLEAR support
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-29 17:44:40 -07:00
Chunmei Liu
3414b28278 crimson:add OMAP_CMP support
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-29 17:38:17 -07:00
Casey Bodley
90662cad56 cmake: WITH_SYSTEM_UTF8PROC defaults to OFF
change the default value of WITH_SYSTEM_UTF8PROC from ON to OFF, so that
centos/rhel users can build with the default cmake configuration. no other
WITH_SYSTEM_* variable in ceph defaults to ON, so this is consistent
with other bundled libraries like boost and rocksdb

unfortunately, this also means that users that do have system packages
must opt-in to using them with -DWITH_SYSTEM_UTF8PROC=ON

both deb and rpm builds dependended on the previous default value, so
their logic was negated to match the new default

Fixes: https://tracker.ceph.com/issues/55114

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-03-29 17:49:18 -04:00
Aishwarya Mathuria
3532b78901 osd/osd_types: Increasing decode version of scrub_duration in pg stats
All new fields added to pg stats after quincy RC need to have the decode field bumped up to avoid decoding errors during an upgrade from quincy RC to the quincy stable version

Fixes: https://tracker.ceph.com/issues/53923
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2022-03-29 23:40:54 +05:30
Redouane Kachach
19c07de820
mgr/cephadm: fallback to normal sorted if cannot import natsorted
Fixes: https://tracker.ceph.com/issues/55113

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-03-29 19:08:10 +02:00
Ernesto Puerta
64c6bb394b
Merge pull request #45563 from rhcs-dashboard/fix-54991-master
mgr/dashboard:  don't log 3xx as errors

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-03-29 19:03:49 +02:00
Yuval Lifshitz
44b5827050 multisite/test: running the multisite test script with multiple RGWs per zone
the RGW_PER_ZONE environment variables should be set to allow that
if not set, we assume 1 RGW per zone

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-03-29 16:21:46 +03:00
Boris Ranto
f1d17b04e4
Merge pull request #45295 from lbausch/tweak_mgr_alerts_mail
mgr/alerts: Add Message-Id and Date header to sent emails

Reviewed-by: Boris Ranto <branto@redhat.com>
2022-03-29 13:55:14 +02:00
Ernesto Puerta
406e54d8c4
mgr/dashboard: don't log 3xx as errors
Let's avoid printing these ugly/misleading/redundant messages:

```
0 [dashboard DEBUG controllers.home] frontend language from headers: ['en-us']
0 [dashboard DEBUG controllers.home] found directory for language 'en-us'
0 [dashboard DEBUG controllers.home] serving static content: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/src/pybind/mgr/dashboard/frontend/dist/en-US/styles.css
0 [dashboard ERROR exception] Internal Server Error
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/pybind/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/cherrypy/_cpdispatch.py", line 60, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/pybind/mgr/dashboard/controllers/home.py", line 134, in __call__
    return serve_file(full_path)
  File "/usr/lib/python3/dist-packages/cherrypy/lib/static.py", line 70, in serve_file
    cptools.validate_since()
  File "/usr/lib/python3/dist-packages/cherrypy/lib/cptools.py", line 117, in validate_since
    raise cherrypy.HTTPRedirect([], 304)
cherrypy._cperror.HTTPRedirect: ([], 304)
```

Fixes: https://tracker.ceph.com/issues/54991
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2022-03-29 11:52:18 +02:00
Kefu Chai
1651ce5b03
Merge pull request #45676 from rzarzynski/wip-crimson-toolset-10
ceph.spec.in: use gcc-toolset-10 for building crimson

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-29 17:03:14 +08:00