Commit Graph

115390 Commits

Author SHA1 Message Date
Lenz Grimmer
0c9e31cf6d
Merge pull request #37087 from tspmelo/wip-iscsi-logged-in
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2020-09-21 11:45:19 +02:00
Kefu Chai
983e28b46b
Merge pull request #37261 from tchaikov/wip-47552
common/BackTrace: do not use len for length of demangled symbol

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-09-21 17:43:17 +08:00
Kefu Chai
c2af45b659
Merge pull request #37185 from david-z/wip-fix-osdmaptool
tools/osdmaptool.cc: fix inaccurate pg map result when simulating osd out

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-21 17:38:56 +08:00
Kefu Chai
c2a94678a0
Merge pull request #37210 from changchengx/no_tune_message
messenger: remove unused variable

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-21 17:37:21 +08:00
Lenz Grimmer
f0fb21ada0
Merge pull request #37183 from rhcs-dashboard/fix-47434-master
mgr/dashboard: table detail rows overflow

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2020-09-21 10:42:19 +02:00
Kefu Chai
35613d091a
Merge pull request #37268 from anthonyeleven/anthonyeleven/doc-fixes
doc/man: Add optional reweight-by-utilization args

Reviewed-by: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-21 14:41:35 +08:00
Anthony D'Atri
815263b583 doc/man: Add optional reweight-by-utilization args
doc/mgr: Grammar and wording for Prometheus labels
doc/rados: Spelling and clarity

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2020-09-20 22:47:01 -07:00
Brad Hubbard
0a288f6690
Merge pull request #37176 from badone/wip-enable-mgr-client-debug
qa: Enable debug_client for mgr tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-21 11:34:58 +10:00
Changcheng Liu
4d6b678736 messenger: remove unused variable
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-09-21 09:22:07 +08:00
Kefu Chai
14b6d35b17 common/BackTrace: do not use len for length of demangled symbol
it turns out `len` is longer than the length of demangled symbol,
let's rely on the `\0` sentry in the returned char* string instead.

in this change, use `status` to tell if the demangle is successful or
not.

Fixes: https://tracker.ceph.com/issues/47552
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-20 11:34:53 +08:00
Ilya Dryomov
240c46a75a
Merge pull request #37072 from idryomov/wip-kcephfs-blacklisted-string
mds: add " (blacklisted)" to session reject error string

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-19 11:36:43 +02:00
Kefu Chai
47e7cd0f65
Merge pull request #37207 from tchaikov/wip-doc-dev-osx
doc/dev/macos.rst: disable features not supported on osx

Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-19 12:08:35 +08:00
Kefu Chai
266aa66aa9
Merge pull request #37252 from pponnuvel/spellcheck-docs
doc: Fixed a number of typos in documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-19 11:07:52 +08:00
Kefu Chai
7e15a95bab
Merge pull request #37216 from tchaikov/wip-doc-cephadmin-codeblock
doc/cephadm: use appropriate directive for formatting codeblocks

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-09-19 10:57:58 +08:00
Patrick Donnelly
7eceaf45de
Merge PR #37202 into master
* refs/pull/37202/head:
	mon: allow overriding the initial mon_host

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-18 18:54:57 -07:00
Kefu Chai
a251ea097d
Merge pull request #37224 from tchaikov/wip-cmake-boost-MPL-list-size
cmake: introduce Boost::MPL interface library for increasing BOOST_MPL_LIMIT_LIST_SIZE

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-19 09:33:31 +08:00
Patrick Donnelly
0275e1c7af
Merge PR #37214 into master
* refs/pull/37214/head:
	mgr/volumes/nfs: Check if orchestrator spec service_id is valid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-18 17:29:27 -07:00
Patrick Donnelly
22b3d48fa0
Merge PR #37190 into master
* refs/pull/37190/head:
	mon/MonCap: check profile_grants too while checking caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-18 17:27:09 -07:00
Patrick Donnelly
d1a0a6eecc
Merge PR #37148 into master
* refs/pull/37148/head:
	mds/FSMap: do not set legacy_client_fscid after filtering

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-18 17:24:38 -07:00
Patrick Donnelly
b64c3784f0
Merge PR #37037 into master
* refs/pull/37037/head:
	mds: fix purge_queue's _calculate_ops is inaccurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-18 17:23:32 -07:00
Patrick Donnelly
269667b3a2
Merge PR #37218 into master
* refs/pull/37218/head:
	qa: spawn MDS daemons before creating fs

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-18 16:25:59 -07:00
Neha Ojha
8ba0a61a51
Merge pull request #35906 from gregsfortytwo/wip-stretch-mode
Add a new stretch mode for 2-site Ceph clusters

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-18 14:31:45 -07:00
Patrick Donnelly
2aae719653
Merge PR #36957 into master
* refs/pull/36957/head:
	mds: convert stringstream to CachedStackStringStream

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2020-09-18 11:04:11 -07:00
Michael Fritch
a6c06a9ca7
Merge pull request #37245 from mgfritch/cephadm-extra-ceph-conf-test
mgr/cephadm: fixup expected extra ceph conf test result

Reviewed-by: Adam King <adking@redhat.com>
2020-09-18 11:50:35 -06:00
Ponnuvel Palaniyappan
d99d520493 doc: Fixed a number of typos in documentation
Signed-off-by: Ponnuvel Palaniyappan <pponnuvel@gmail.com>
2020-09-18 18:17:15 +01:00
Michael Fritch
b3205210a1
mgr/cephadm: fixup expected extra ceph conf test result
fix test failure introduced by:
ff7e76348e5457fa6acb23545fcef56d6640c50a

```
E           AssertionError: expected call not found.
E           Expected: _run_cephadm('test', 'mon.test', 'deploy', ['--name', 'mon.test', '--reconfig', '--config-json', '-'], stdin='{"config": "\\n\\n[mon]\\nk=v\\n", "keyring": ""}')
E           Actual: _run_cephadm('test', 'mon.test', 'deploy',
['--name', 'mon.test', '--reconfig', '--config-json', '-'],
stdin='{"config": "\\n\\n[mon]\\nk=v\\n", "keyring": ""}', image='')
```

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-09-18 08:54:57 -06:00
Lenz Grimmer
ec05d87432
Merge pull request #34545 from rhcs-dashboard/read_only
mgr/dashboard: Disabling the form inputs for the read_only modals

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-18 12:59:30 +02:00
Lenz Grimmer
0625509c7f
Merge pull request #37023 from p-se/grafana-many-to-many
mgr/dashboard: Fix many-to-many issue in host-details Grafana dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2020-09-18 11:58:47 +02:00
Joshua Schmid
4057f30187
Merge pull request #37059 from guits/guits-cephadm-shell-multiple-mounts
cephadm: support multiple mounts when running interactive shell
2020-09-18 10:57:52 +02:00
Joshua Schmid
1fb9082b09
Merge pull request #36890 from sebastian-philipp/cephadm-extend-ceph.conf
mgr/cephadm: Add extra-ceph-conf
2020-09-18 10:56:24 +02:00
Joshua Schmid
e87a408995
Merge pull request #37135 from sebastian-philipp/cephadm-race-add-host-vs-apply
mgr/cephadm: Fix race between host_add and _apply_all_specs
2020-09-18 10:55:17 +02:00
Joshua Schmid
a0065d4a95
Merge pull request #36969 from votdev/issue_46666_container_spec
cephadm: Introduce 'container' specification to deploy custom containers
2020-09-18 10:50:30 +02:00
Rishabh Dave
e7a721f0d2 mon/MonCap: check profile_grants too while checking caps
When checking if a certain fs subcommand can and should be executed in
FSCommands.cc, check permissions in "profile_grants" too when the caps
for that entity contains a cap profile.

Fixes: https://tracker.ceph.com/issues/47423
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-18 13:23:35 +05:30
Kefu Chai
989af0b7cf
Merge pull request #37158 from tchaikov/wip-no-more-assertDictContainsSubset
mgr/dashboard: replace assertDictContainsSubset() with assertLessEqual()

Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-18 15:18:31 +08:00
Kefu Chai
12457bab47
Merge pull request #37170 from yaarith/add-smartctl-nvme-dependencies
ceph.spec, debian: add smartmontools, nvme-cli dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-18 13:42:44 +08:00
Kefu Chai
0cf2e89948 cmake: introduce Boost::MPL interface library
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-18 12:26:09 +08:00
Kefu Chai
822b91e4d1 src: Revert "Fix to raise BOOST_MPL_LIMIT_LIST_SIZE from 20 to 30"
This reverts commit 3f4e9a4526b8f174888828078e610769b80e48ec.

will fix the FTBFS by introducing a interface library in CMake script

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-18 12:26:09 +08:00
Kefu Chai
709c73ef4a cmake: extract admin/CMakeLists.txt
for better modularity

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-18 12:26:09 +08:00
Yaarit Hatuka
d5528a7e8e ceph.spec.in, debian/control: add smartmontools and nvme-cli dependencies
These packages are needed in order to scrape device health metrics from
devices used by OSD and MON daemons.

smartmontools' smartctl is what we use in order to scrape devices' SMART
attributes and general health metrics.
In addition, we use nvme-cli tool on NVMe devices, which fetches
vendor specific NVMe related health metrics.

Ceph rely on these tools for proper functioning of the underlying layers
of devicehealth mgr module, and other mgr modules which use devicehealth
functionality (such as diskprediction_local, telemetry, dashboard).

Essentially, most of devicehealth commands rely on proper functioning of
smartctl, otherwise they lack the device health metrics.

For example, in case smartctl is missing, the commands:
    ceph device scrape-daemon-health-metrics <who>
    ceph device scrape-health-metrics [<devid>]
will not be able to scrape health metrics, and the command:
    ceph device predict-life-expectancy <devid>
will not provide any meaningful output (since there are no metrics).

In short, when we scrape a device by its daemon (be it an OSD or a MON):
  ceph device scrape-daemon-health-metrics <who>
The devicehealth module command eventually invokes a
block_device_get_metrics() call in either osd/OSD.cc or mon/Monitor.cc,
which wraps calls to both
    block_device_run_smartctl()       (spawns smartctl)
    block_device_run_vendor_nvme()    (spawns nvme)
in common/blkdev.cc.

Minimum version requirements:
'smartmontools' is the package name, which contains two utility
programs: 'smartd' and 'smartctl'. Ceph uses the latter.

Version 6.7 of smartctl first introduced the --json option (beta), which
allows to output the metrics in a JSON format. Since then a few
adjustments were made and the feature officially launched in smartctl
version 7.0.
Since we rely on the JSON format to process the metrics, we must have
smartmontools' smartctl version >= 7.

That said, we choose not to specify smartmontools version here on
purpose, since there might be a scenario where:
We specified smartmontools version to be >= 7.
smartmontools 7 is not available yet in rhel 8 / centos 8.
A user installs via rpm ceph-osd, for example.
smartmontools will not be installed (since version >= 7 is not available
in this repo yet).
Then the user upgrades to 8.3 (which should have smartmontools >= 7),
but smartmontools will not get upgraded (since it's not installed).
In the scenario where we do not specify a version, smartmontools 6.6
will be installed, but it will be upgraded to >= 7 when a user upgrades
(and if it's a fresh installation - version >= 7 would be installed
anyway).

nvme-cli does not have a minimum version.

We use 'Recommends' for both rpm and deb packages since we do not want
the installation to fail in case of conflicts. 'Recommends' weakens the
dependency to be installed in case possible, but ignores it in cases of
conflicts with other dependencies.

It's worth mentioning that smartmontools and nvme-cli dependencies exist
in ceph-container builds.
We add them here for the cases of bare metal installations.

In the future we will add a separate package (with smartmontools and
nvme-cli dependencies) that can be installed on any node (running
rbd-mirror, rgw, mds, mgr, etc.), in order to be able to collect the
health metrics of its devices and offer their life expectancy
prediction.

Fixes: https://tracker.ceph.com/issues/47479
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2020-09-18 03:25:56 +00:00
Patrick Donnelly
795cf4cecc
mds: convert stringstream to CachedStackStringStream
This is a simple performance refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-17 18:54:55 -07:00
Patrick Donnelly
ff16f5d67c
Merge PR #37163 into master
* refs/pull/37163/head:
	mds: silence warning ‘MDSRank::fs_name’ will be initialized after [-Wreorder]

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-17 18:30:11 -07:00
Patrick Donnelly
5126bfca8d
Merge PR #37147 into master
* refs/pull/37147/head:
	mds/FSMap: check parse_role return before filtering

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-17 18:28:13 -07:00
David Zafman
3a95735010
Merge pull request #36989 from AmnonHanuhov/wip-ObjectStore_EIO_Handling
osd: Got rid of global flag eio_errors_to_process

Reviewed-by: David Zafman <dzafman@redhat.com>
2020-09-17 17:45:42 -07:00
David Zafman
a97a1cd858
Merge pull request #36397 from dzafman/wip-39012
distinguish unfound + impossible to find, vs start some down OSDs to get

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-17 17:30:43 -07:00
Jason Dillaman
251b940f4f
Merge pull request #37132 from lixiaoy1/dirty_cache_feature
librbd: add DIRTY_CACHE in IMPLICIT_ENABLE

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-17 19:04:29 -04:00
Jason Dillaman
c4fb0c1dfa
Merge pull request #36586 from MahatiC/wip-ssd-integration
librbd/cache: SSD cache integration framework

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-17 17:52:30 -04:00
Kefu Chai
5df5d8f8ff
Merge pull request #37141 from sebastian-philipp/cephadm-fix-rm-util.load_from_store
mgr/cephadm: fix RemoveUtil.load_from_store()

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-09-18 00:32:22 +08:00
Patrick Donnelly
61db7a9c2e
qa: spawn MDS daemons before creating fs
This avoids unnecessary MDS_ALL_DOWN messages because the MDS daemons
have not yet been spawned.

Fixes: https://tracker.ceph.com/issues/47518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-17 09:26:38 -07:00
Kefu Chai
967104103c doc/cephadm: use appropriate directive for formatting codeblocks
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-17 23:42:27 +08:00
Patrick Donnelly
3428b020fd
Merge PR #37197 into master
* refs/pull/37197/head:
	doc: add "fs authorize" subcommand to ceph man page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-17 06:42:33 -07:00