Commit Graph

115247 Commits

Author SHA1 Message Date
Ponnuvel Palaniyappan
d99d520493 doc: Fixed a number of typos in documentation
Signed-off-by: Ponnuvel Palaniyappan <pponnuvel@gmail.com>
2020-09-18 18:17:15 +01:00
Lenz Grimmer
ec05d87432
Merge pull request #34545 from rhcs-dashboard/read_only
mgr/dashboard: Disabling the form inputs for the read_only modals

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-18 12:59:30 +02:00
Lenz Grimmer
0625509c7f
Merge pull request #37023 from p-se/grafana-many-to-many
mgr/dashboard: Fix many-to-many issue in host-details Grafana dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2020-09-18 11:58:47 +02:00
Joshua Schmid
4057f30187
Merge pull request #37059 from guits/guits-cephadm-shell-multiple-mounts
cephadm: support multiple mounts when running interactive shell
2020-09-18 10:57:52 +02:00
Joshua Schmid
1fb9082b09
Merge pull request #36890 from sebastian-philipp/cephadm-extend-ceph.conf
mgr/cephadm: Add extra-ceph-conf
2020-09-18 10:56:24 +02:00
Joshua Schmid
e87a408995
Merge pull request #37135 from sebastian-philipp/cephadm-race-add-host-vs-apply
mgr/cephadm: Fix race between host_add and _apply_all_specs
2020-09-18 10:55:17 +02:00
Joshua Schmid
a0065d4a95
Merge pull request #36969 from votdev/issue_46666_container_spec
cephadm: Introduce 'container' specification to deploy custom containers
2020-09-18 10:50:30 +02:00
Kefu Chai
989af0b7cf
Merge pull request #37158 from tchaikov/wip-no-more-assertDictContainsSubset
mgr/dashboard: replace assertDictContainsSubset() with assertLessEqual()

Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-18 15:18:31 +08:00
Kefu Chai
12457bab47
Merge pull request #37170 from yaarith/add-smartctl-nvme-dependencies
ceph.spec, debian: add smartmontools, nvme-cli dependencies

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-18 13:42:44 +08:00
Yaarit Hatuka
d5528a7e8e ceph.spec.in, debian/control: add smartmontools and nvme-cli dependencies
These packages are needed in order to scrape device health metrics from
devices used by OSD and MON daemons.

smartmontools' smartctl is what we use in order to scrape devices' SMART
attributes and general health metrics.
In addition, we use nvme-cli tool on NVMe devices, which fetches
vendor specific NVMe related health metrics.

Ceph rely on these tools for proper functioning of the underlying layers
of devicehealth mgr module, and other mgr modules which use devicehealth
functionality (such as diskprediction_local, telemetry, dashboard).

Essentially, most of devicehealth commands rely on proper functioning of
smartctl, otherwise they lack the device health metrics.

For example, in case smartctl is missing, the commands:
    ceph device scrape-daemon-health-metrics <who>
    ceph device scrape-health-metrics [<devid>]
will not be able to scrape health metrics, and the command:
    ceph device predict-life-expectancy <devid>
will not provide any meaningful output (since there are no metrics).

In short, when we scrape a device by its daemon (be it an OSD or a MON):
  ceph device scrape-daemon-health-metrics <who>
The devicehealth module command eventually invokes a
block_device_get_metrics() call in either osd/OSD.cc or mon/Monitor.cc,
which wraps calls to both
    block_device_run_smartctl()       (spawns smartctl)
    block_device_run_vendor_nvme()    (spawns nvme)
in common/blkdev.cc.

Minimum version requirements:
'smartmontools' is the package name, which contains two utility
programs: 'smartd' and 'smartctl'. Ceph uses the latter.

Version 6.7 of smartctl first introduced the --json option (beta), which
allows to output the metrics in a JSON format. Since then a few
adjustments were made and the feature officially launched in smartctl
version 7.0.
Since we rely on the JSON format to process the metrics, we must have
smartmontools' smartctl version >= 7.

That said, we choose not to specify smartmontools version here on
purpose, since there might be a scenario where:
We specified smartmontools version to be >= 7.
smartmontools 7 is not available yet in rhel 8 / centos 8.
A user installs via rpm ceph-osd, for example.
smartmontools will not be installed (since version >= 7 is not available
in this repo yet).
Then the user upgrades to 8.3 (which should have smartmontools >= 7),
but smartmontools will not get upgraded (since it's not installed).
In the scenario where we do not specify a version, smartmontools 6.6
will be installed, but it will be upgraded to >= 7 when a user upgrades
(and if it's a fresh installation - version >= 7 would be installed
anyway).

nvme-cli does not have a minimum version.

We use 'Recommends' for both rpm and deb packages since we do not want
the installation to fail in case of conflicts. 'Recommends' weakens the
dependency to be installed in case possible, but ignores it in cases of
conflicts with other dependencies.

It's worth mentioning that smartmontools and nvme-cli dependencies exist
in ceph-container builds.
We add them here for the cases of bare metal installations.

In the future we will add a separate package (with smartmontools and
nvme-cli dependencies) that can be installed on any node (running
rbd-mirror, rgw, mds, mgr, etc.), in order to be able to collect the
health metrics of its devices and offer their life expectancy
prediction.

Fixes: https://tracker.ceph.com/issues/47479
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2020-09-18 03:25:56 +00:00
Patrick Donnelly
ff16f5d67c
Merge PR #37163 into master
* refs/pull/37163/head:
	mds: silence warning ‘MDSRank::fs_name’ will be initialized after [-Wreorder]

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-17 18:30:11 -07:00
Patrick Donnelly
5126bfca8d
Merge PR #37147 into master
* refs/pull/37147/head:
	mds/FSMap: check parse_role return before filtering

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-17 18:28:13 -07:00
David Zafman
3a95735010
Merge pull request #36989 from AmnonHanuhov/wip-ObjectStore_EIO_Handling
osd: Got rid of global flag eio_errors_to_process

Reviewed-by: David Zafman <dzafman@redhat.com>
2020-09-17 17:45:42 -07:00
David Zafman
a97a1cd858
Merge pull request #36397 from dzafman/wip-39012
distinguish unfound + impossible to find, vs start some down OSDs to get

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-17 17:30:43 -07:00
Jason Dillaman
251b940f4f
Merge pull request #37132 from lixiaoy1/dirty_cache_feature
librbd: add DIRTY_CACHE in IMPLICIT_ENABLE

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-17 19:04:29 -04:00
Jason Dillaman
c4fb0c1dfa
Merge pull request #36586 from MahatiC/wip-ssd-integration
librbd/cache: SSD cache integration framework

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-17 17:52:30 -04:00
Kefu Chai
5df5d8f8ff
Merge pull request #37141 from sebastian-philipp/cephadm-fix-rm-util.load_from_store
mgr/cephadm: fix RemoveUtil.load_from_store()

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-09-18 00:32:22 +08:00
Patrick Donnelly
3428b020fd
Merge PR #37197 into master
* refs/pull/37197/head:
	doc: add "fs authorize" subcommand to ceph man page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-17 06:42:33 -07:00
Kefu Chai
f09fb275a7
Merge pull request #37120 from tchaikov/wip-rados-type-hintings
pybind/rados: add more type hintings

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-17 18:40:06 +08:00
Kefu Chai
d814c5533b
Merge pull request #37143 from dvanders/dvanders_flush
ceph.in: ignore failures to flush stdout

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-17 18:39:09 +08:00
Kefu Chai
8abfc5b7e2
Merge pull request #37100 from rhcs-dashboard/fix-47400-master
ceph: ignore BrokenPipeError when printing help

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-17 18:38:15 +08:00
Kefu Chai
499d0c7f2b
Merge pull request #37045 from tchaikov/wip-crimson-bt
common/BackTrace: extract demangle() out

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-09-17 18:16:06 +08:00
Volker Theile
7219377cfc cephadm: Introduce 'container' specification to deploy custom containers
Fixes: https://tracker.ceph.com/issues/46666

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-09-17 10:04:11 +02:00
Kefu Chai
141f23480e common/BackTrace: let abi::__cxa_demangle() do the malloc
also use the returned length for constructing the string_view to be
appended.

we could reuse the buffer across multiple demangle() call for saving the
calls to malloc()/free(). but the upside of this change is that it's
simpler.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-17 15:24:37 +08:00
Kefu Chai
ea6abcf34c common/BackTrace: extract demangle() out
so it can be reused by crimson

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-17 15:24:37 +08:00
Rishabh Dave
e461a763c8 doc: add "fs authorize" subcommand to ceph man page
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-17 10:06:21 +05:30
Kefu Chai
36d67cf59e
Merge pull request #37187 from kamoltat/patch-1
doc/dev/macos: remove nonexistence flag `--with-toolchain`

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-17 12:04:08 +08:00
Patrick Donnelly
23f9da70a7
Merge PR #37199 into master
* refs/pull/37199/head:
	qa/cephfs: add a note to test_nfs.py

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-16 16:44:38 -07:00
David Zafman
bb5058074d doc: Add doc and PendingRealeaseNotes
Add help about new list_unfound fields in troubleshooting-pg.rst
Add info in PendingReleaseNotes

Signed-off-by: David Zafman <dzafman@redhat.com>
2020-09-16 16:40:10 -07:00
Patrick Donnelly
08c9cb85f9
Merge PR #37159 into master
* refs/pull/37159/head:
	mon/MDSMonitor: return zero when mds is absent for "mds fail"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-09-16 12:20:13 -07:00
Rishabh Dave
aa08a38b61 qa/cephfs: add a note to test_nfs.py
A note that tells that test_nfs.py is not yet compatible with
vstart_runner.py.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-16 22:10:42 +05:30
Neha Ojha
ca43c6e8ee
Merge pull request #37167 from neha-ojha/wip-47239
qa/tasks/ceph_manager.py: remove redundant check in raw_cluster_cmd_result

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-16 09:30:11 -07:00
Jos Collin
2c2bcf68f1
mds: silence warning ‘MDSRank::fs_name’ will be initialized after [-Wreorder]
Fixes the warning:
ceph/src/mds/MDSRank.h:436:17: warning: ‘MDSRank::fs_name’ will be initialized after [-Wreorder]

Signed-off-by: Jos Collin <jcollin@redhat.com>
2020-09-16 21:35:43 +05:30
Kamoltat Sirivadhna
825df66bb8 doc/dev/macos: remove nonexistence flag --with-toolchain
` --with-toolchain` is an old option from llvm 6

Signed-off-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2020-09-16 22:39:57 +07:00
Abhishek L
0f61e6ea28
Merge pull request #37109 from theanalyst/doc/releases/v15.2.5
doc: releases: notes for v15.2.5 octopus

Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>
2020-09-16 16:51:22 +02:00
Kefu Chai
cdca6c3992
Merge pull request #37166 from adamemerson/wip-undetailed-time
common/ceph_time: Don't define public things in time_detail

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-16 20:59:35 +08:00
Rishabh Dave
a246a56776 mon/MDSMonitor: return zero when mds is absent for "mds fail"
... instead of EINVAL.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-16 18:28:36 +05:30
Jan Fajerski
3ca25daa1d
Merge pull request #37093 from guits/guits-bz1877672
ceph-volume: fix simple activate when legacy osd

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2020-09-16 14:25:24 +02:00
Lenz Grimmer
b17ca08037
Merge pull request #35956 from ishanrai05/hover
mgr/dashboard: Display description on mouse hover

Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-16 13:49:54 +02:00
Matt Benjamin
6b31e3747b
Merge pull request #35924 from pritha-srivastava/wip-rgw-sts-error-messages
rgw/sts: adding error log messages to STS code.
2020-09-16 07:20:51 -04:00
Matt Benjamin
a9e40accb8
Merge pull request #36255 from pritha-srivastava/wip-rgw-sts-user-as-owner
rgw/sts: adding code for federated user as owner in case of STS.
2020-09-16 07:09:14 -04:00
Guillaume Abrioux
a3e9e215bd ceph-volume: fix simple activate when legacy osd
`ceph-volume simple activate --all` relies on the presence of json files
in `/etc/ceph/osd` that was created with `ceph-volume simple scan`
command.

In a cluster lifecycle, it is very likely an OSD which was deployed with
ceph-disk at some point gets removed or replaced. It means the corresponding
json file in `/etc/ceph/osd` becomes unrelevant. It makes `ceph-volume
simple activate --all` fails because it tries to mount non existing
partitions.
The idea here is to simply warn the user that the osd described in the
json file doesn't exist anymore and exit properly instead of throwing an
error.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1877672
Closes: https://tracker.ceph.com/issues/47493

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-16 11:13:11 +02:00
Joshua Schmid
25e5b8726d
Merge pull request #36898 from sebastian-philipp/doc-fix-orch-apply-mgr
doc/mgr: fix outdated `orch apply` docs.
2020-09-16 10:04:28 +02:00
Ali Maredia
6607cd8538
Merge pull request #35986 from TRYTOBE8TME/wip-rgw-sts-new
STS Testing Files
2020-09-16 01:38:50 -04:00
Kefu Chai
8016ba885c
Merge pull request #37172 from tchaikov/wip-crimson-object-context-lock
crimson/common/tri_mutex: update the class comment

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-16 12:42:39 +08:00
Kefu Chai
0a5fb96b10 crimson/osd/object_context: s/rwlock/lock/
so it's less confusing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-16 10:28:58 +08:00
Kefu Chai
441360dbb0 crimson/common/tri_mutex: update the class comment
to explain the reason why we have tri_mutex, and how it is related to
pipelined read / write. and the mutual exclusion between read, write
and rmw operations.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-16 09:10:14 +08:00
Kefu Chai
03d5b5d9dc
Merge pull request #37131 from tchaikov/wip-47311
crimson/object_context: make obc interruptible

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-16 09:01:32 +08:00
Ernesto Puerta
39768459f6
ceph: ignore BrokenPipeError when printing help
ignore BrokenPipeError which is thrown when piping the output of ceph
CLI to a tool which might close its stdin before ceph CLI sends the
whole help message.

Follow approach suggested by Kefu: 7b0ed43af5
This doesn't manage the clean-up/exit logic, as that's deferred to the
last part of the __main__ code.

Fixes: https://tracker.ceph.com/issues/47400
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2020-09-15 20:20:43 +02:00
Adam C. Emerson
01f706ca0f common/ceph_time: Don't define public things in time_detail
Defining things in a detail section and then using them outside turned
out to not be the best idea.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-09-15 13:58:30 -04:00