Commit Graph

110013 Commits

Author SHA1 Message Date
Venky Shankar
3c531197a6 qa: whitelist cluster warning message for removed MDS
Fixes: http://tracker.ceph.com/issues/44677
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-03-31 02:41:00 -04:00
Venky Shankar
bccbf1fa03 mgr: force purge normal ceph entities from service map
Normal ceph services can send task status updates to manager.
Task status is tracked in service map implying that normal
ceph services have entries in service map and daemon tracking
index (daemon state). But the manager prunes entries from daemon
state when it receives an updated map (fs, mon, etc...). This
causes periodic pruning of service map entries to fail for normal
ceph services (those which send task status updates) since it
expects a corresponding entry in daemon state.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-03-31 02:41:00 -04:00
Kefu Chai
f2d162d54b test/crimson: increase variance of stdev to 0.20
per Mark Nelson,

> yeah, 5% variation is way too low
> Sometimes we can stay within 5%, but especially if we are pushing the
> system hard we can see closer to 10-20% sometimes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-31 12:24:06 +08:00
Yan, Zheng
7aa22fac60 ceph-fuse: don't get mount options from /etc/fstab when doing remount
If there happen to be an kcephfs entry in /etc/fstab for ceph-fuse's
mount point. 'mount -o remount' may get options from that entry. fuse
may not understand some options (E.g name option).

Fixes: https://tracker.ceph.com/issues/44771
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-31 11:47:51 +08:00
Yuan Lu
b98a869d2c librbd: add flush test case
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-03-31 11:07:10 +08:00
Yuan Lu
d860e909fb librbd: add internal flush
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-03-31 11:06:51 +08:00
Yuan Lu
555e101110 librbd: flush dirty entries to osd
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-03-31 11:06:42 +08:00
Yuan Lu
a1c3fdf77b librbd: flush sync point into cache device
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-03-31 11:06:33 +08:00
Yuan Lu
07fb05a658 librbd: add FlushRequest to handle flush
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-03-31 11:06:24 +08:00
Jason Dillaman
42e60584d4
Merge pull request #34112 from trociny/wip-44610
pybind/rbd: ensure image is open before permitting operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-03-30 20:50:44 -04:00
Jason Dillaman
d45de978d1
Merge pull request #34019 from changchengx/blockguard
librbd: refine BlockGuard debug output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-03-30 20:49:48 -04:00
Sage Weil
5a516db38f Merge PR #34150 into master
* refs/pull/34150/head:
	cephadm: rm-cluster clean up /etc/ceph

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-30 15:22:32 -05:00
Daniel-Pivonka
bb4554a3a6 mgr/cephadm: add useful error if python3 is not on remote host
show 'Error ENOENT: New host example (example) failed check: ["Can't communicate with
remote host, possibly because python3 is not installed there"]' instead of traceback
with OSError: cannot send(already closed?) when adding host if python3 is not on host

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2020-03-30 14:51:53 -04:00
Sebastian Wagner
874ba5049a python-common: raise on emtpy drive groups
adds a test for https://tracker.ceph.com/issues/44758

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-30 19:51:31 +02:00
Sebastian Wagner
8194d4972f cephadm: check hostnames case insensitive
Salt converts minion names to lowercase.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-30 18:16:57 +02:00
Jason Dillaman
d1a67c3bda librbd: re-add support for nautilus clients talking to jewel clusters
We want to support N - 3 client backward compatibility (special case
to support Jewel since it was a LTS release). The "get_snapshot_timestamp"
cls method does not exist in Jewel clusters so librbd should fallback
to excluding the op if it fails.

Note that this N - 3 also needs to apply for downstream releases as well,
which implies we still need Jewel for the time being.

Fixes: http://tracker.ceph.com/issues/39450
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c644121820b83c97e68f9896393a45cd34787672)

Conflicts:
	src/test/librbd/image/test_mock_RefreshRequest.cc: tweaked to support pool configs
2020-03-30 11:37:12 -04:00
Jason Dillaman
ab720d4c00 librbd: ignore -EOPNOTSUPP errors when retrieving image group membership
The Luminous release did not support adding images to a group (it only
included the bare-minimum support for creating groups). Commit f76df32666
incorrectly dropped support for ignoring this possible failure. This
prevents Nautilus-release clients from opening images contained within
a Luminous-release cluster.

Fixes: http://tracker.ceph.com/issues/38834
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6f29dc69a0)
2020-03-30 11:20:21 -04:00
Casey Bodley
c52cbe212f
Merge pull request #34253 from cbodley/wip-qa-rgw-too-many-shards
qa/rgw: remove 1999-shard tests from rgw/verify

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2020-03-30 10:06:11 -04:00
Casey Bodley
308b6c2c0f
Merge pull request #33557 from yuvalif/add_2pc_cls_queue
cls/queue: add 2-phase-commit queue implementation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-03-30 09:49:19 -04:00
Casey Bodley
804dc5d064
Merge pull request #33933 from cbodley/wip-rgw-doc-reshard-v2
doc/rgw: update multisite reshard design for cross-datalog-shard coordination

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-03-30 09:38:40 -04:00
Sage Weil
07d9231ffb Merge PR #34260 into master
* refs/pull/34260/head:
	Revert "Merge PR #34091 into master"

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-30 08:36:44 -05:00
Sage Weil
803f05f254 Merge PR #34248 into master
* refs/pull/34248/head:
	qa/tasks/cephadm: no need to explicitly reconfig
	qa/tasks/cephadm: fetch final ceph.conf
	qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-30 08:24:59 -05:00
Sage Weil
4f4efde17a Merge PR #34266 into master
* refs/pull/34266/head:
	mgr/DaemonServer: fetch metadata for new daemons (e.g., mons)

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-30 08:24:47 -05:00
Tiago Melo
c6829da46c mgr/dashboard: Reduce size of component styles
Fixes: https://tracker.ceph.com/issues/44811

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-03-30 11:21:23 +00:00
Shilpa Jagannath
0a15f54803 rgw: replace RGWBucketInfo's sharding-related fields with BucketLayout
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-03-30 16:31:24 +05:30
Casey Bodley
60af77262a rgw: add BucketLayout to RGWBucketInfo
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-30 16:31:24 +05:30
Casey Bodley
d6686816b7 rgw: use qualified encode/decode in putobj processor
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-30 16:31:24 +05:30
Laura Paduano
8e64da5334
Merge pull request #34279 from votdev/issue_44805
mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-30 11:58:26 +02:00
Volker Theile
1efe1a76da mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar
Fixes: https://tracker.ceph.com/issues/44805

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-03-30 11:08:13 +02:00
Brad Hubbard
c1f3f05011
Merge pull request #34276 from badone/wip-fix-argument-ordering-docker-test
tests: Reinstate "opts" as final argument

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-30 13:23:36 +10:00
Yan, Zheng
107b91d528 qa/cephfs: test case for auto reconnect after blacklisted
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/42085
2020-03-30 10:24:47 +08:00
Yan, Zheng
30dd9d2834 client: add debug output for ll_read result
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
aa9db2ef73 client: invalidate all write mode filp after reconnect
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
27da39ae5d client: auto reconnect using new entity addrs after blacklisted
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
173bb632e8 client: -EIO on read/write if file hander has lost filelocks
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
b87f3cafe0 client: -EIO on filelock operations if inode has lost filelocks
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
0f74020b44 client: purge dirty data when session gets evicted
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
2cf6184e41 client: check sync write error
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
c310d09e21 client: use session state to track if it's rejected by mds
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:47 +08:00
Yan, Zheng
a716bc7f9b msg: add Messenger::client_reset()
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-03-30 10:24:46 +08:00
Brad Hubbard
c951132aee tests: Reinstate "opts" as final argument
a1e8f61cb72 broke the "opts" argument.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-03-30 12:20:24 +10:00
Kefu Chai
011e7b3736
Merge pull request #34252 from JackSlateur/doc-os-recommendations2
doc/start/os-recommendations.rst: recommend bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-30 00:26:59 +08:00
Kefu Chai
54185b8042
Merge pull request #34267 from JackSlateur/doc-man-monmaptool
doc/man/8/monmaptool.rst: update synopsis etc with the latest options

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-30 00:23:47 +08:00
Alexandre Bruyelles
400ace85db
doc/man/8/monmaptool.rst: update the page with the latest options
Signed-off-by: Alexandre Bruyelles <jack@jack.fr.eu.org>
2020-03-29 18:00:02 +02:00
Kefu Chai
bd0e885a8a
Merge pull request #34224 from markhpc/wip-bluefs-disable-buffered-io
common/options: Disable bluefs_buffered_io by default again.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-29 22:21:05 +08:00
Kefu Chai
aececeab8b
Merge pull request #33052 from ifed01/wip-ifed-fix-unused2
os/bluestore: fix 'unused' calculation

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-29 22:19:23 +08:00
Kefu Chai
44842fcd6a
Merge pull request #34203 from majianpeng/bluefs-remove-unused-func
os/bluestore: remove unused func.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-29 22:18:02 +08:00
Sage Weil
48b6cc6182 Merge PR #34061 into master
* refs/pull/34061/head:
	mgr/orch: Add `ceph orch ls --export`
	mgr/dashboard: adapt to new `ServiceDescription.to_json()`
	python-common: add `service_name` to `ServiceSpec.to_json`
	python-common: make ServiceSpec and ServiceDescription compatible
	src/ceph.in: add yaml to known formats
	mgr/orch: add yaml to `orch ls`
	mgr/orch: remove `orch spec dump`
	python-common: reorder RGWSpec arguments
	python-common: prevent ServiceSpec of wrong type
	pybind/mgr: tox.ini: omit cov report
	mgr/cephadm: test describe_service
	mgr/orch: ServiceDescription: change json representation
	mgr/orch: ServiceDescription: Make spec a requirement

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-29 07:17:37 -05:00
Sage Weil
932d152f35 Merge PR #34215 into master
* refs/pull/34215/head:
	cephadm: Fix trucated output of `mgr dump`

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-29 07:16:59 -05:00
Kefu Chai
260b1a922e
Merge pull request #34268 from poyea/docs-options-1
doc: Fix inconsistency in logging settings

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-29 14:24:09 +08:00