Commit Graph

103548 Commits

Author SHA1 Message Date
Ilya Dryomov
5011cc926c qa/suites/krbd: run unmap subsuite with msgr1 only
pre-single-major.yaml kernel doesn't have any of the monitor client
fixes that came in 4.6.  If the connection is closed, it closes the
session and retries only after 10 seconds.  On top of that, there is
nothing to prevent it from picking the same monitor when reconnecting.
This means that when given both v1 and v2 ports (which look like two
different monitors), it is susceptible to mount_timeout (60 seconds):

  $ sudo rbd map img
  rbd: sysfs write failed
  In some cases useful info is found in syslog - try "dmesg | tail".
  rbd: map failed: (5) Input/output error

  [  822.242313] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)
  [  832.265494] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)
  [  842.296175] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)
  [  852.326924] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)
  [  862.357611] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)
  [  872.388373] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)
  [  882.676136] libceph: mon0 172.21.15.132:3300 socket closed (con state CONNECTING)

Unlike newer kernels that return ETIMEDOUT, it returns EIO.

Newer kernels are much more aggressive about retries and will pick
a different monitor when reconnecting, hence they are always able to
establish the session in time.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-10-30 19:51:55 +01:00
Ilya Dryomov
9c17ca0aa7
Merge pull request #31023 from idryomov/wip-krbd-udev-enumerate-retry
krbd: retry on transient errors from udev_enumerate_scan_devices()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2019-10-29 11:40:45 +01:00
Jan Fajerski
aa34a05aa2
Merge pull request #31072 from sebastian-philipp/ceph-volume-inventory-device-id
ceph-volume: add Ceph's device id to inventory
2019-10-29 08:16:14 +01:00
Patrick Donnelly
eb00dcd660
Merge PR #31063 into master
* refs/pull/31063/head:
	qa: disable too few PG warning during Mimic deploy

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-10-28 20:37:31 -07:00
Kefu Chai
6e4bc90863
Merge pull request #31178 from tchaikov/wip/rpm/pyyaml
ceph.spec.in: add missing python-yaml dependency for mgr-k8sevents

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2019-10-29 07:57:36 +08:00
Sage Weil
d927374bb4 Merge PR #31168 into master
* refs/pull/31168/head:
	ceph-daemon: try py2 import before py3
	qa/suites/rados/singleton-nomsgr/ceph-daemon: make sure python3 is installed
	qa/standalone/test_ceph_damon.sh: test with python2 and python3
	mgr/ssh: python, not python3
	ceph-daemon: python, not python3
	ceph-daemon: os.makedirs
	ceph-daemon: configparser is ConfigParser on py2
	ceph-daemon: avoid py3-isms

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-10-28 14:59:43 -05:00
Sage Weil
85c750be26 ceph-daemon: try py2 import before py3
See f499a89b15

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:17:07 -05:00
Sage Weil
9fe9653c8c qa/suites/rados/singleton-nomsgr/ceph-daemon: make sure python3 is installed
Centos7 doesn't have it by default.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Sage Weil
debde146d2 qa/standalone/test_ceph_damon.sh: test with python2 and python3
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Sage Weil
cfe1898b91 mgr/ssh: python, not python3
python 2.x is okay too.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Sage Weil
79a70558da ceph-daemon: python, not python3
python2 is okay too.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Sage Weil
4330502768 ceph-daemon: os.makedirs
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Sage Weil
97d9664927 ceph-daemon: configparser is ConfigParser on py2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Sage Weil
beb09ff548 ceph-daemon: avoid py3-isms
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 12:15:47 -05:00
Kefu Chai
1365b45c26
Merge pull request #31180 from liewegas/revert-bluestore-pinned-list
os/bluestore: revert cache pinned list

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2019-10-28 23:18:02 +08:00
Sebastian Wagner
e70d6041c1 ceph-volume: add Ceph's device id to inventory
This will benefit the orchestrator and dashboard to show a unified view of devices with SMART data

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-10-28 15:55:18 +01:00
Sebastian Wagner
002f563c7e
Merge pull request #31071 from sebastian-philipp/python-common-mypy
python-common: Add mypy testing

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2019-10-28 15:07:08 +01:00
Jan Fajerski
56c16b1628
Merge pull request #30995 from wjwithagen/wjw-wip-volume-zfs-inventory
ceph-volume-zfs: add the inventory command
2019-10-28 14:57:37 +01:00
Kefu Chai
965fd1acfa
Merge pull request #31014 from tchaikov/wip-38378
mgr/dashboard: accept expected exception when SSL handshaking

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-10-28 20:53:07 +08:00
Kefu Chai
42928b3c0d
Merge pull request #31092 from liewegas/wip-revert-42026-test
Revert "test: librados startup/shutdown racer test"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-10-28 20:36:13 +08:00
Kefu Chai
7048420c59
Merge pull request #31153 from ceph/wip-restful-node-items
restful: Query nodes_by_id for items

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-28 20:29:20 +08:00
Kefu Chai
9619168525
Merge pull request #31176 from tchaikov/wip/mgr/diskprediction_cloud/cleanup
mgr/diskprediction_cloud: refactor timeout() decorator

Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-10-28 20:27:22 +08:00
Kefu Chai
9766f328c2
Merge pull request #31175 from tchaikov/wip/mon/OSDMonitor/cleanup
mon/OSDMonitor: use initializer_list<> for {si,iec}_options

Reviewed-by: Sage Weil <sage@redhat.com>
2019-10-28 20:24:48 +08:00
Kefu Chai
b224fb73f0
Merge pull request #30279 from runsisi/wip-fix-strerror_r
common: fix compat of strerror_r

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-28 20:11:14 +08:00
Kefu Chai
801dbc1bf7
Merge pull request #29836 from majianpeng/fix-async-msgr-perfcounter-bug
msg/async: Don't dec(msgr_active_connections) if conn still in accept…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-10-28 20:09:58 +08:00
Kefu Chai
70d27ce352
Merge pull request #31156 from cyx1231st/wip-seastar-msgr-testpeer
test/crimson: implement a remote async TestPeer for crimson msgr tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-28 12:27:11 +08:00
Yingxin Cheng
9888a4f8f4 test/crimson: add nonstop option in async TestPeer
If set, TestPeer won't shutdown if all tests are done, and will wait for
another round of tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-10-28 10:44:42 +08:00
Yingxin Cheng
43f05b7c1c test/crimson: add delay to markdown_peer()
Let test sleep awhile for peer markdown propagated.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-10-28 10:43:08 +08:00
Yingxin Cheng
41bfffc8b8 test/crimson: async msgr TestPeer for v2 failover tests
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-10-28 10:42:18 +08:00
Yingxin Cheng
24bfdc08f7 test/crimson: allow v2 failover tests to connect to remote TestPeer
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-10-28 10:30:18 +08:00
Yingxin Cheng
f7c37dfa6b test/crimson: extract msgr test commands to header
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-10-28 10:30:18 +08:00
J. Eric Ivancich
10253251e2
Merge pull request #30610 from ivancich/wip-no-schedule-reshard-during-reshard
rgw: prevent bucket reshard scheduling if bucket is resharding

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-10-27 14:47:39 -04:00
J. Eric Ivancich
1d61b25dbf
Merge pull request #30795 from ivancich/wip-dynamic-reshard-adjustments
rgw: adjust allowable bucket index shard counts for dynamic resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-10-27 14:46:01 -04:00
J. Eric Ivancich
2a1720d5c2
Merge pull request #30845 from croit/rgw-bucket-stats-add-num-shards
rgw: add num_shards to radosgw-admin bucket stats

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-10-27 14:44:47 -04:00
J. Eric Ivancich
a2745ce1a4
Merge pull request #30956 from joke-lee/rgw-storage-class-post-obj
rgw: set correct storage class for post object upload

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-10-27 14:42:27 -04:00
J. Eric Ivancich
2d5bac020e
Merge pull request #30970 from theanalyst/rgw/list-common-prefix
rgw: url encode common prefixes for List Objects response 

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-10-27 14:39:46 -04:00
Kefu Chai
20ea3df0a0
Merge pull request #31179 from liewegas/cleanup-exec-bit
os/bluestore,comon,erasure-code: chmod -x source files

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-27 23:58:59 +08:00
Kefu Chai
5047b3222c
Merge pull request #31171 from liewegas/bug-42496
qa/tasks/cbt: run stop-all.sh while shutting down

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-27 23:57:57 +08:00
Sage Weil
f03395e5a6 Revert "Merge pull request #30964 from markhpc/wip-bs-cache-trim-pinned"
This reverts commit 304c37f521, reversing
changes made to 7b4f9a083f.

This causes some bluestore test failures due to an ENOENT right after a
new object is created.  The simplest reproducer is the
ObjectStore/StoreTest.FiemapEmpty/2 test.

Fixes: https://tracker.ceph.com/issues/42495
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-27 09:59:10 -05:00
Sage Weil
032c880240 os/bluestore,comon,erasure-code: chmod -x source files
Fixes

*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/common/obj_bencher.cc is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/erasure-code/shec/determinant.c is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/os/bluestore/AvlAllocator.cc is executable but has empty or no shebang, removing executable bit
*** WARNING: ./usr/src/debug/ceph-15.0.0-6512.g62bd825.el8.x86_64/src/os/bluestore/AvlAllocator.h is executable but has empty or no shebang, removing executable bit

in rpm build.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-27 09:26:39 -05:00
Sage Weil
6653f73bbd Merge PR #31170 into master
* refs/pull/31170/head:
	src/telemetry: remove, now lives in ceph-telemetry.git

Reviewed-by: Sage Weil <sage@redhat.com>
2019-10-27 09:21:56 -05:00
Sage Weil
44d45ff693 Merge PR #31131 into master
* refs/pull/31131/head:
	crush: remove invalid upmap items

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-10-27 09:20:51 -05:00
Sage Weil
b4974b42ff Merge PR #31154 into master
* refs/pull/31154/head:
	auth/cephx: always initialize local variables

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-27 09:19:56 -05:00
Sage Weil
e771ece974 Merge PR #31167 into master
* refs/pull/31167/head:
	os/bluestore: do not mark per_pool_omap updated unless we fixed it

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-10-27 09:19:40 -05:00
Kefu Chai
482561bd55 mgr/dashboard: accept socket error 0
see
9d73226ab2

Fixes: https://tracker.ceph.com/issues/38378
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 14:34:48 +08:00
Kefu Chai
8a743a8a4e mgr/dashboard: accept exceptions from builtin SSL
see also https://github.com/cherrypy/cheroot/pull/4

so we don't panic when client is trying to talk with us with an
unsupported protocol, the exception should be accepted, and the
client can fallback to supported protocol.

Fixes: https://tracker.ceph.com/issues/38378
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 14:34:48 +08:00
Kefu Chai
01cb398019 mgr/dashboard: extract monkey patches for cherrypy out
so they are less distracting to the dashboard developers. and probably
less scaring.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 14:34:48 +08:00
Kefu Chai
8adbb86fb8 ceph.spec.in: add missing python-yaml dependency for mgr-k8sevents
otherwise we might have:

```
ceph/src/pybind/mgr/k8sevents/__init__.py", line 1, in <module>
    from .module import Module
  File "/home/kchai/ceph/src/pybind/mgr/k8sevents/module.py", line 28, in <module>
    import yaml
ImportError: No module named yaml
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 12:46:15 +08:00
Kefu Chai
3f56b1f3ef mgr/diskprediction_cloud: refactor timeout() decorator
* timeout() is never passed any parameter when being called, so let's
  remove the parameters list of "seconds" and "error_message"
* use `getattr()` instead of `hasattr()` for retrieving the
  member variable of `self`
* pass `self` to wrapper function explicitly.
* return `func()` right away.
* hardwire the error message of `TimeoutError` to "Timer expired",
  because
  - as neither errno.ETIME nor errno.ETIMEOUT is portable
  - the only caller of `TimeoutError` is `timeout()`, so there is
    no need to have the flexibility to pass a different error message
* use `wraps()` as a decorator, simpler this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 10:15:09 +08:00
Kefu Chai
48bf1c9bf3 mon/OSDMonitor: use initializer_list<> for {si,iec}_options
* use initializer_list<> for {si,iec}_options, no need to uset set<>
* remove the comments, the variable names are self-documented.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 09:43:43 +08:00