Commit Graph

125508 Commits

Author SHA1 Message Date
Varsha Rao
bdb2241ca5 doc/cephadm: update about disabling logging to journald for quincy
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-08-03 17:50:57 +05:30
Varsha Rao
9691f76d46 doc/cephadm: remove repeated section on disabling logging to stderr
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-08-03 16:42:14 +05:30
Ernesto Puerta
77871e27e6
Merge pull request #42328 from aaryanporwal/visual-tests
mgr/dashboard: Visual regression tests for ceph dashboard

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-08-03 12:55:45 +02:00
Yingxin
03692ba7f1
Merge pull request #42514 from liu-chunmei/seastore-onode-interruptible-future
crimson/onode-staged-tree: integrate interruptible future

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-08-03 16:55:04 +08:00
Yingxin Cheng
03cd4d1518 crimson/onode-staged-tree: misc fixes to integrate interruptive-future
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-03 11:53:53 +08:00
Ilya Dryomov
76294e57f9
Merge pull request #42471 from Dorthu/fix-pybind-rbd-mirror-image-get-status
pybind/rbd: fix mirror_image_get_status

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-02 22:04:04 +02:00
Ilya Dryomov
8248fbe2e4
Merge pull request #42570 from tchaikov/wip-libcryptsetup
librbd/crypto/luks: require libcryptsetup v2.0.5

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-02 21:35:16 +02:00
Sebastian Wagner
d6960d0e09
Merge pull request #42551 from adk3798/maint-message
mgr/cephadm: make return message for entering maintenance mode more explicit

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-02 18:00:48 +02:00
Harish Munjulur
a62e0dd755
Merge pull request #42382 from xrmeng8756/master
rgw: avoid occuring radosgw daemon crash when access a conditionally …
2021-08-02 08:28:02 -07:00
Harish Munjulur
71ed958936
Merge pull request #42387 from wzbxqt327/master
rgw:add lock to copy object
2021-08-02 08:27:53 -07:00
Harish Munjulur
00dc0b91b3
Merge pull request #42404 from ivancich/wip-broken-list-plain-entries
rgw: bucket index list can produce I/O errors
2021-08-02 08:27:37 -07:00
Kefu Chai
4231bfda04
Merge pull request #42582 from tchaikov/wip-doc-health-report
doc/dev: add health-reports.rst

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-08-02 21:40:48 +08:00
Kefu Chai
c50d8ebdef
Merge pull request #42577 from tchaikov/wip-rtd
.readthedocs.yml: use python3.8 and native ditaa

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-08-02 20:15:04 +08:00
Kefu Chai
ab3c95d5ed doc/dev: add health-reports.rst
to explain the data flow of health metrics.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-02 20:06:07 +08:00
Ernesto Puerta
edf694eebc
Merge pull request #42263 from rhcs-dashboard/51612-cephadm-e2e-improvements
mgr/dashboard: cephadm-e2e job script: improvements
2021-08-02 13:09:32 +02:00
Kefu Chai
48a91b92e1
Merge pull request #42562 from tchaikov/wip-bluestore-cleanups
os/bluestore: use scope_guard do to cleanups

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-08-02 19:04:23 +08:00
Ernesto Puerta
cc6b18a92c
Merge pull request #41880 from david-caro/fix_cluster_grafana_dashboard
monitoring/grafana/cluster: use per-unit max and limit values

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
2021-08-02 13:03:46 +02:00
Kefu Chai
1a36bf82fc admin, doc: introduce sphinxcontrib.seqdiag
for rendering sequence-diagram. unlike ditaa, seqdiag allows us to
create sequence-diagram without worrying about the layout. and the
syntax is quite like that of dot.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-02 19:02:56 +08:00
Kefu Chai
bc4aacbbbd
Merge pull request #42572 from tchaikov/wip-cmake-mgr-cleanup
cmake: initialize dpdk_LIBRARIES with empty list

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-08-02 11:09:47 +08:00
Kefu Chai
02e7acc376 doc/conf.py: run ditaa with java
just in case, otherwise we could have

  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/42577/lib/python3.8/site-packages/sphinxcontrib/ditaa.py", line 200, in html_visit_ditaa
    render_ditaa_html(self, node, node['code'], node['options'])
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/42577/lib/python3.8/site-packages/sphinxcontrib/ditaa.py", line 177, in render_ditaa_html
    fname, outfn = render_ditaa(self, code, options, prefix)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/42577/lib/python3.8/site-packages/sphinxcontrib/ditaa.py", line 141, in render_ditaa
    p = Popen(ditaa_args, stdout=PIPE, stdin=PIPE, stderr=PIPE)
  File "/home/docs/.pyenv/versions/3.8.6/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/docs/.pyenv/versions/3.8.6/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/bin/ditaa'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-02 01:42:08 +08:00
Kefu Chai
57f57c9419 .readthedocs.yml: use ditaa instead of plantweb
use ditaa to reader ditaa images instead of relying plantweb service.
more stable this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-02 01:00:35 +08:00
Kefu Chai
0bde0f3a38 .readthedocs.yml: use python3.8
to prepare the python3.8 migration

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-02 00:41:34 +08:00
Kefu Chai
a35aeeb53a
Merge pull request #42129 from tchaikov/wip-cmake-build-type
cmake: set CMAKE_BUILD_TYPE only if .git exists

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-08-01 12:24:07 +08:00
Kefu Chai
ff459c0f6f
Merge pull request #42573 from tchaikov/wip-crimson-header
cmake: let crimson-admin depend on legacy-option-headers

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-08-01 09:58:56 +08:00
Kefu Chai
8f1629e788
Merge pull request #42445 from LiumxNL/ups-eliminate-rollfwd
osd/PGLog: set acceptable rollback_info_trimmed_to for pg of replicated pool

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-01 09:58:15 +08:00
Patrick Donnelly
775f92dd6e Merge PR #40511 into master
* refs/pull/40511/head:
	qa: update mds_pre_upgrade to no longer stop standbys
	qa: update mds_pre_upgrade to disable standby-replay
	qa: add tests for compat manipulation and upgrade
	doc: remove deprecated compat commands
	doc: update MDS upgrade procedure
	mon,mds: use per-MDS compat to inform replacement
	mon: do not update inline incompat except via mds
	mds: add MDSMap method for creating null MDSMap
	mds: only update beacon epoch if newer
	mds: harden standby_mds lookup
	mon/FSCommands: accept generic ostream rather than stringstream
	include: add less verbose CompatSet dump
	include: add dump operator for Feature
	include: add const qualifier to appropriate CompatSet methods

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-07-31 13:34:56 -07:00
Kefu Chai
d93880880d cmake: let crimson-admin depend on legacy-option-headers
legacy-option-headers provides global_legacy_options.h, which is in turn
required for building crimson-admin.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-31 17:03:00 +08:00
Kefu Chai
84425c8de6
Merge pull request #42400 from adk3798/offline-daemon-removal
mgr/cephadm: don't remove daemons from offline hosts

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
2021-07-31 16:35:37 +08:00
Kefu Chai
edacb6eab4 cmake: prefer static library when finding DPDK
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-31 16:29:28 +08:00
Kefu Chai
ee378a3405 cmake: initialize dpdk_LIBRARIES with empty list
set(dpdk_LIBRARIES) does not reset this variable, it leaves it
unchanged.

if pkg-config manages to find DPDK libraries, dpdk_LIBRARIES would be
set with a string like "rte_node;rte_graph;..." by
pkg_check_modules(dpdk QUIET libdpdk).

but we would want to set this variable to the import paths of the
required libraries. so reset it before appending them to this variable.

this change helps to address the build failure when building Ceph with
DPDK installed into system along with its .pc file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-31 15:53:03 +08:00
Kefu Chai
3adad9309a mgr/ServiceMap: do not include unused headers
<experimental/iterator> was included for std::experimental::make_ostream_joiner
in a968f65d78. but the code using
std::experimental::make_ostream_joiner was later rewritten in
ab0d8f2ae9, in which
std::experimental::make_ostream_joiner is not used anymore.

so let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-31 15:48:18 +08:00
Kefu Chai
4e207a0419
Merge pull request #42553 from cfsnyder/wip_51961
mgr/cephadm: fix exceptions causing stuck progress indicators

Reviewed-by: Adam King <adking@redhat.com>
2021-07-31 13:10:06 +08:00
Kefu Chai
3301ef623b
Merge pull request #42506 from p-se/mgr-prom-counter-type
mgr/prometheus: Fix metric types from gauge to counter

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-31 13:08:01 +08:00
Kefu Chai
8de47b212b librbd/crypto/luks: require libcryptsetup v2.0.5
- ubuntu focal ships libcryptsetup-dev (2:2.2.2),
- centos 8 app stream comes with cryptsetup-devel-2.3.3.
- openSUSE Leap 15.3 packages libcryptsetup-devel-2.3.4
- openSUSE Leap 15.2 packages libcryptsetup-devel-2.0.5

so we can drop the support for libcryptsetup < 2.0.5

see also ea3c1bfb9e

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-31 11:24:58 +08:00
Patrick Donnelly
2cd3494771 qa: update mds_pre_upgrade to no longer stop standbys
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
8e0b9bcad6 qa: update mds_pre_upgrade to disable standby-replay
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
295971b9c6 qa: add tests for compat manipulation and upgrade
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
54b649af9b doc: remove deprecated compat commands
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
efb70f2b33 doc: update MDS upgrade procedure
Now that CompatSet changes to the FSMap no longer cause old MDS to
suicide.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
58eaa237b0 mon,mds: use per-MDS compat to inform replacement
This diff makes the following changes:

- FSMap::compat is now just a "default compat" of currently unknown
  utility. It is used when constructing a new file system but does
  not really have any effect or current use.

- The `mds compat *` CLI commands are deprecated. They manipulate
  the default compat which has no useful effect.

- Each MDS sends its compat to the mons in its beacon. This is from
  MDSMap::get_compat_set_all() at MDS boot. This CompatSet does not
  change for the duration of the MDS lifetime.

- Mons record each MDS compat in the FSMap to inform standby failover.
  An MDS is only promoted if it is compatible with the file system
  compat.

- Mons upgrade (merge) the file system compat when (a) the number of
  *in* MDS is 1 (effected by max_mds=1) and (b) the mons are promoting a
  standby with a new compat. A file system is never upgraded when there
  is more than 1 rank to prevent two MDS with incompatible compat.

- A suite of `fs compat` commands exist to manipulate the file system
  compat. These exist mostly for testing.

The consequence of these changes is that the upgrade procedure for MDS
can be updated to no longer require turning off all MDS but rank 0
before performing any upgrades. A CompatSet change would cause all MDS
receiving the new MDSMap to suicide due to incompatibility (if so).
Instead, the monitors will no longer assign an incompatible MDS to a
file system and enforce an upgrade procedure if incompatibilities exist.

Fixes: https://tracker.ceph.com/issues/49720
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:54 -07:00
Patrick Donnelly
8cdc36c89d mon: do not update inline incompat except via mds
The MDS_FEATURE_INCOMPAT_INLINE feature indicates that an MDS knows how
to read/write inline data and that the file system may have it. The
separate setting for inline_data protects this file system feature.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
4a10b6016f mds: add MDSMap method for creating null MDSMap
It's not necessary to distribute a CompatSet with the null mdsmap. We
only need to communicate that the MDS is not part of any map.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
0256ae010f mds: only update beacon epoch if newer
This is a defensive programming change. We don't want the beacon epoch
to ever go backwards.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
b8ad8a8c82 mds: harden standby_mds lookup
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
31c8edd603 mon/FSCommands: accept generic ostream rather than stringstream
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
56b36e69fa include: add less verbose CompatSet dump
For printing in `fs dump`.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
5da02036ee include: add dump operator for Feature
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
d179fa2bc0 include: add const qualifier to appropriate CompatSet methods
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 16:28:53 -07:00
Patrick Donnelly
5ae7b9202b Merge PR #42513 into master
* refs/pull/42513/head:
	qa: multifs already enabled as default

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 14:03:36 -07:00
Patrick Donnelly
54fea240de Merge PR #42499 into master
* refs/pull/42499/head:
	client:make sure only to update dir dist from auth mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-30 14:02:33 -07:00