Commit Graph

100549 Commits

Author SHA1 Message Date
Kefu Chai
2094c2f2fd pybind/mgr: install setuptools >= 12
otherwise we have
```
  Running setup.py egg_info for package apache-libcloud

    libcloud/utils/py3.py:76: UserWarning: Missing backports.ssl_match_hostname package
      warnings.warn("Missing backports.ssl_match_hostname package")
    Traceback (most recent call last):
...
      File "build/bdist.linux-x86_64/egg/setuptools_scm/version.py", line 66, in _warn_if_setuptools_outdated

...
setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)
...
Command python setup.py egg_info failed with error code 1 in /tmp/tmp.i95Fg82Ea4/venv/build/apache-libcloud
```

and more important thing is, setuptools >= 12 allows us to install the
python modules in venv, without uninstalling existing modules if they
are also installed in by system packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-31 16:26:29 +08:00
Sebastian Wagner
ee6a415cbb mgr/rook: monkey-patch for CoreV1Api.list_node
`kubernetes.client.apis.core_v1_api.CoreV1Api#list_node` is
broken and needs a monkey patch. Might be related to
an incompatible k8s cluster version.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 10:15:38 +02:00
Sebastian Wagner
76b66d904d mgr/orchestrator: better format service ls
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 10:15:31 +02:00
Sebastian Wagner
676bde1741 mgr/rook: Implement fallback for CentOS 7
`kubernetes-client` requires `urllib3` > 1.15, but CentOS 7 ships 1.10

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 10:15:15 +02:00
Sebastian Wagner
29aeebb40f mgr/rook: Move KubernetesResource to rook_cluser.py
Use `KubernetesResource` as cache for pods, inventory maps and nodes.
This is required for the dashboard.

Also:
  properly return K8s nodes
  fix minor issues

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:59:36 +02:00
Paul Cuzner
00cfc5c402 mgr/rook: Add feature to gather k8s storageclass information
Provide ability to fetch or watch k8s resources,
and hold the resulting objects in a dict. A method
has been added to the rook module to allow other
mgr modules to fetch the state of specific k8s
resources.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2019-07-31 09:57:33 +02:00
Sebastian Wagner
cf7ce17795 mgr/orchestrator_clli: Always return str
Also: Don't return `"None"`
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:49:07 +02:00
Sebastian Wagner
1d72af6258 mgr/ansible: Moved RGW defaults to ansible
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:49:03 +02:00
Sebastian Wagner
c09cd1ccd8 mgr/rook: Fix RGW creation
* Moved RGWSpec defaults to a new method

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:48:50 +02:00
Sebastian Wagner
dc54e77bd6 mgr/orchestrator: Split *_stateless_service and add get_feature_set
Split *_stateless_service, because every orchestrator
did some method dispatching internally anyway.

Also: Added test

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:44:57 +02:00
Jan Fajerski
8fcdb4d628 ceph-volume: batch ensure device lists are disjoint
Fixes: https://tracker.ceph.com/issues/41018

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-07-31 09:41:14 +02:00
Kefu Chai
5ca4881dbb test/python: remove test for ceph cli
we have abundant test for exercising `ceph` cli, like

qa/standalone/mon/misc.sh.

and `test_ceph.py` does not test any functionality of this python script
other than importing it, which basically runs it through python's syntax
check.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-31 14:29:27 +08:00
Kefu Chai
1d445110db test/python: remove test for brag/client
ceph-brag is now take replaced by telemetry. and it was removed in
cf97b5c1 .

so no need to keep its test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-31 14:25:48 +08:00
Sage Weil
9cb6108eb7 Merge PR #29363 into master
* refs/pull/29363/head:
	qa/suites/multimds/basic/tasks/ceph_test_snapshots: disable RECENT_CRASH
	qa/suites/kcephfs/recovery/failover.yaml: disable RECENT_CRASH
	qa/suites/fs/multifs/tasks/failover.yaml: disable RECENT_CRASH

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-30 22:46:55 -05:00
Sage Weil
6d1dd6181b qa/tasks/mgr/dashboard/test_mgr_module: adjust expected schema
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-30 22:40:56 -05:00
Konstantin Shalygin
3e72a58698
doc: mds-config-ref: update 'mds_log_max_segments' value.
Also drop options that not exists on code:

* mds_log_max_expiring
* mds_log_eopen_size

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>
2019-07-31 10:35:08 +07:00
Sage Weil
4e69c42872 Merge PR #29185 into master
* refs/pull/29185/head:
	os/bluestore: add tail length when caculating reserved buffer size
	os/bluestore: record next_offs when offset is not sequential.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-07-30 21:55:04 -05:00
Sage Weil
34a27a58ee Merge PR #29356 into master
* refs/pull/29356/head:
	pybind/mgr/restful: use dict.items() for py3 compatibility

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-07-30 21:54:36 -05:00
Josh Durgin
bf1e860b84
Merge pull request #29098 from liewegas/wip-unhexdump-c
src/script/unhexdump-C: script to reverse a 'hexdump -C' style hexdump

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-07-30 18:27:27 -07:00
Neha Ojha
7c03f088f1 common/options.cc: change default value of bluestore_fsck_on_mount_deep to false
Fixes: https://tracker.ceph.com/issues/41017
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-07-30 17:10:12 -07:00
Casey Bodley
99966b3e1e
Merge pull request #29394 from mkogan1/wip-rgw-bi-list-shard-id
rgw: add option to specify shard-id for "bi list" admin command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-07-30 15:16:56 -04:00
Patrick Donnelly
ad574171ca
Merge PR #29362 into master
* refs/pull/29362/head:
	Update nautilus release for 14.2.2
	Updated schedule with latest Nautilus release

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-30 11:40:56 -07:00
Patrick Donnelly
0f31e1e1fb
Merge PR #28239 into master
* refs/pull/28239/head:
	cephfs-shell: Add tox for testing with flake8

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-30 11:35:13 -07:00
Andrew Schoen
eb48959d2e
Merge pull request #29386 from alfredodeza/wip-rm40987
ceph-volume  simple: better detection when 'type' file is not present

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-07-30 13:33:40 -05:00
Varsha Rao
e90d58208d
cephfs-shell: Add tox for testing with flake8
Fixes: https://tracker.ceph.com/issues/39947
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-07-30 10:34:24 -07:00
Patrick Donnelly
6015ff4cf6
Merge PR #29331 into master
* refs/pull/29331/head:
	common: add comment about pod memory requests/limits

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2019-07-30 10:14:57 -07:00
Kefu Chai
32595268a3
Merge pull request #29161 from valentinbajrami/improve-grep-regex
src/script/check_commands.sh: fix grep regex class range

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-30 22:09:32 +08:00
Willem Jan Withagen
fdc0e2baeb cmake: update FindBoost.cmake
FreeBSD is moving to Boost 1.70
we should allow user to use boost 1.70.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-07-30 16:00:04 +02:00
Jan Fajerski
f42287c341 ceph-volume: refactor ceph-disk_member unittests
This adds a parameterized fixture as input to the tests. Should make
writing new tests easier since only fixture parameters need to be added.
In the future we should be able to reuse the fixtures to build other
fixtures and reuse the parametrization.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-07-30 15:27:20 +02:00
Mark Kogan
b0f6946a85 rgw: add option to specify shard-id for "bi list" admin command
related to: https://tracker.ceph.com/issues/40700
Signed-off-by: Mark Kogan <mkogan@redhat.com>
2019-07-30 08:55:07 -04:00
Valentin Bajrami
02ebe3c344
src/script/check_commands.sh: fix grep regex class range
Correct the faulty regex class.

Signed-off-by: Valentin Bajrami <valentin.bajrami@gmail.com>
2019-07-30 11:14:20 +02:00
Sebastian Wagner
475159ec73
Merge pull request #28915 from sebastian-philipp/python-common-package-2
python-common:  Python common package

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2019-07-30 09:31:34 +02:00
Sebastian Wagner
3999238081
Merge pull request #27336 from SUSE/wip-orchestrator-deepsea-iscsi-ganesha
mgr/deepsea: return ganesha and iscsi endpoint URLs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-30 09:13:25 +02:00
Kai Wagner
3660e17e4b doc: pg_num should always be a power of two
Signed-off-by: Kai Wagner <kwagner@suse.com>
2019-07-30 08:35:16 +02:00
Jan Fajerski
60fe6eac74
Merge pull request #29387 from ceph/wip-ceph-volume-tests-no-dashboard
ceph-volume: when testing disable the dashboard
2019-07-30 08:23:59 +02:00
Kefu Chai
3169da7a1a
Merge pull request #29347 from smithfarm/wip-40974
rpm: improve ceph-mgr plugin package summaries

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-30 12:46:37 +08:00
Kefu Chai
bcd4ed6724
Merge pull request #29346 from smithfarm/wip-40973
rpm: fdupes in SUSE builds to conform with packaging guidelines

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-30 12:44:32 +08:00
Kefu Chai
227d1e9486
Merge pull request #29367 from tchaikov/wip-doc-crush-rule
doc/rados/operations: crush_rule is a name

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-07-30 09:28:08 +08:00
Kefu Chai
d59090afe0
Merge pull request #29368 from tchaikov/wip-40618
qa/tasks/mgr/test_progress.py: s/ev/new_event/

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-07-30 09:27:29 +08:00
Sage Weil
f1eac8ba4b mgr/telemetry: specify license when opting in
Choosing not to include this in the docs so that the user is more likely
to see this interactively.  (That is...probably good?)

Choose sharing-1-0.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-29 19:37:25 -05:00
Sage Weil
90f04775fc qa/suites/multimds/basic/tasks/ceph_test_snapshots: disable RECENT_CRASH
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-29 18:17:15 -05:00
Patrick Donnelly
3f3189b4d4
Merge PR #28804 into master
* refs/pull/28804/head:
	mds: delay exporting directory whose pin value exceeds max rank id

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-29 14:53:28 -07:00
Patrick Donnelly
ecf49c0820
Merge PR #29339 into master
* refs/pull/29339/head:
	client: don't report any vxattrs to listxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
2019-07-29 14:49:29 -07:00
Patrick Donnelly
7cec28542e
Merge PR #29304 into master
* refs/pull/29304/head:
	cephfs: avoid map been inserted by mistake

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-29 14:47:03 -07:00
Patrick Donnelly
7b59c02c87
Merge PR #29321 into master
* refs/pull/29321/head:
	Client: unlink dentry for inode with llref=0

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-07-29 14:45:44 -07:00
Patrick Donnelly
602e5f6eb4
qa: sleep briefly after resetting kclient
Continuation of 6b83f43ba0.

Fixes: https://tracker.ceph.com/issues/40999
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-29 14:08:17 -07:00
Andrew Schoen
7b7ff0d530 ceph-volume: when testing disable the dashboard
ceph-ansible has started enabling the dashboard by
default as of https://github.com/ceph/ceph-ansible/pull/4268

Disabling the dashboard gets us around needing to run
the grafana_server role which is required for the dashboard
and not needed for ceph-volume functional tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2019-07-29 15:25:19 -05:00
Patrick Donnelly
618c3576c0
Merge PR #29338 into master
* refs/pull/29338/head:
	qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-07-29 13:10:02 -07:00
Patrick Donnelly
2a82081f77
Merge PR #28825 into master
* refs/pull/28825/head:
	qa: wait for kernel client death
	qa: use hard_reset to reboot kclient

Reviewed-by: David Galloway <dgallowa@redhat.com>
2019-07-29 13:08:29 -07:00
Alfredo Deza
e48ad3ab26 ceph-volume tests: ensure that better heuristics exist for objectstore detection
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-29 15:50:14 -04:00