Commit Graph

100372 Commits

Author SHA1 Message Date
J. Eric Ivancich
a9a3e59abc
Merge pull request #29178 from ivancich/wip-small-rgw-efficiency
rgw: add a small efficiency

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-01 18:36:30 -04:00
J. Eric Ivancich
092ef6085c
Merge pull request #29179 from ivancich/wip-bucket-list-excessive-max-entries
rgw: mitigate bucket list with max-entries excessively high

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-01 18:34:35 -04:00
J. Eric Ivancich
0ef6bee2d5
Merge pull request #29240 from mdw-at-linuxbox/wip-master-rgw-swift-metadata-versioning
rgw: Fix bucket versioning vs. swift metadata bug.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-01 18:28:19 -04:00
Matt Benjamin
57df918a1a
Merge pull request #28331 from linuxbox2/wip-rgw-uinfo-by-acc
rgw: permit rgw-admin to populate user info by access-key
2019-08-01 16:07:34 -04:00
Kefu Chai
d835e9a926
Merge pull request #29413 from tchaikov/wip-remove-test/python
test/python: remove stale tests

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-08-02 00:38:28 +08:00
Jason Dillaman
8c69bf7cb1
Merge pull request #29445 from ricardoasmarques/fix-doc-rbd-namespace
doc: Fix rbd namespace documentation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-08-01 12:03:25 -04:00
Ricardo Marques
029dade640 doc: Fix rbd namespace documentation
Pool and namespace should be separated by a slash.

Fixes: https://tracker.ceph.com/issues/41046

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-08-01 15:45:24 +01:00
Kefu Chai
b6da01e75f
Merge pull request #29446 from tchaikov/wip-cmake-tox-test
cmake: add add_tox_test()

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-08-01 22:04:30 +08:00
Kefu Chai
821050f3b1
Merge pull request #29423 from rhcs-dashboard/wip-41023-master
doc: Show Jenkins commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-01 21:43:15 +08:00
Kefu Chai
1792ab404b
Merge pull request #29297 from iotcg/starlingx
ceph-objectstore-tool: set log date format

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-01 21:41:06 +08:00
Kefu Chai
b9c9f74f06 cmake: add add_tox_test()
to consolidate the tox tests

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-01 21:33:30 +08:00
Abhishek L
067db76a4c
Merge pull request #29364 from capri1989/doc-pg-num
doc: pg_num should always be a power of two

Reviewed-By: Jan Fajerski <jfajerski@suse.com>
Reviewed-By: Sage Weil <sage@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-01 14:18:53 +02:00
Sebastian Wagner
1a3a1d1310
Merge pull request #29063 from sebastian-philipp/orchestrator-get_feature_set
mgr/orchestrator: Split *_stateless_service and add get_feature_set
2019-08-01 09:37:16 +02:00
Sebastian Wagner
a74fc09b16
Merge pull request #29131 from sebastian-philipp/rook-caching
mgr/rook: Add caching for the Dashboard

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2019-08-01 09:35:21 +02:00
Sage Weil
9edfc9578f Merge PR #29334 into master
* refs/pull/29334/head:
	qa/tasks/mgr/dashboard/test_mgr_module: adjust expected schema
	mgr/telemetry: separate out cluster config vs running daemons
	mgr/telemetry: include any config options that are customized

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
2019-07-31 20:27:27 -05:00
Sage Weil
76885c1a13 Merge PR #29340 into master
* refs/pull/29340/head:
	mgr/telemetry: specify license when opting in

Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
2019-07-31 20:27:06 -05:00
Jason Dillaman
838430e1f1
Merge pull request #29419 from vshankar/wip-fix-rbd-support
pybind/rbd: fix call to unregister_osd_perf_queries

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-07-31 17:30:54 -04:00
Yuri Weinstein
a319023d09
Merge pull request #29406 from yuriw/wip-yuriw-upgrade-rwg-master_2
qa/tests: added rgw into upgrade sequence to improve coverage
2019-07-31 12:32:49 -07:00
Yuri Weinstein
803b8fb752
Merge pull request #29428 from yuriw/wip-yuriw-crontab-master
qa/tests: added mimic-x on master run
2019-07-31 12:32:24 -07:00
Patrick Donnelly
9e106f1422
Merge PR #29104 into master
* refs/pull/29104/head:
	qa/cephfs: test case for timeout config of individual session
	mds: add command that config individual client session

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-31 12:06:39 -07:00
Casey Bodley
ff13529790
Merge pull request #29430 from uweigand/fix-no-boost-context-build
rgw: tests: Fix building with -DWITH_BOOST_CONTEXT=OFF

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-31 15:00:17 -04:00
Patrick Donnelly
fc2bb44fbd
Merge PR #29412 into master
* refs/pull/29412/head:
	doc: mds-config-ref: update 'mds_log_max_segments' value.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-31 11:59:22 -07:00
Patrick Donnelly
a6066f4214
Merge PR #29355 into master
* refs/pull/29355/head:
	mgr/volumes: set uid/gid of FS client's mount as 0/0

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-31 11:33:36 -07:00
Ricardo Dias
a8840f6d97
Merge pull request #29133 from ricardoasmarques/prevent-iscsi-target-delete-with-sessions
mgr/dashboard: Prevent deletion of iSCSI IQNs with open sessions

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-07-31 18:59:33 +01:00
Patrick Donnelly
837836107c
Merge PR #29388 into master
* refs/pull/29388/head:
	qa: sleep briefly after resetting kclient

Reviewed-by: David Galloway <dgallowa@redhat.com>
2019-07-31 10:33:18 -07:00
Patrick Donnelly
3d68e6f0a5
Merge PR #29377 into master
* refs/pull/29377/head:
	mds: add perf counter for finisher of MDSRank

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-31 10:32:50 -07:00
J. Eric Ivancich
300429c9e9 rgw: mitigate bucket list with max-entries excessively high
When listing a bucket with radosgw-admin, the user can specify the
maximum number of entries. That number can be unreasonably large, and
can affect the performance and memory availability. For example:

    radosgw-admin bucket list --bucket mybucket1 --max-entries=10000000

This has the potential for creating large data structures at multiple
levels in the the call stack of the radosgw(-admin) process,
potentially causing the process to run out of memory. This change
limits the maximum number of entries requested in all but the high
level code to help mitigate this issue.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-07-31 13:08:20 -04:00
Yuri Weinstein
f34f96dc4a qa/tests: added rgw into upgrade sequence to improve coverage
added rgw into upgrade sequence to improve coverage - splits
See https://github.com/ceph/ceph/pull/29234 https://github.com/ceph/ceph/pull/29282

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-31 08:48:03 -07:00
Ulrich Weigand
82d66424b7 rgw: tests: Fix building with -DWITH_BOOST_CONTEXT=OFF
Attempting to build with -DWITH_BOOST_CONTEXT=OFF will result in
a "conflicting declaration" error when building
test/rgw/test_rgw_dmclock_scheduler.cc

Fixed by avoiding the boost/asio/spawn.hpp include in that case.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-07-31 17:38:23 +02:00
Yuri Weinstein
467b81f2e7 qa/tests: added mimic-x on master run
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-31 07:56:25 -07:00
Lars Marowsky-Bree
3a7e171df6 doc: Address further comments on choosing pg_num
Signed-off-by: Lars Marowsky-Bree <lmb@suse.com>
2019-07-31 15:14:04 +02:00
Lars Marowsky-Bree
bd20d692bb doc: adjust examples to use 2^n pg_num
The examples used pg_num set to 300 or 1000. This misled some users.

Signed-off-by: Lars Marowsky-Bree <lmb@suse.com>
2019-07-31 15:03:48 +02:00
Kefu Chai
fc0744bf27
Merge pull request #29405 from wjwithagen/wjw-fix-env-vs-args.t
test/cli: Make the ceph-conf test more liberal

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-31 20:42:06 +08:00
Kefu Chai
b7e6e5fda0
Merge pull request #29414 from tchaikov/wip-mgr-api-test
pybind/mgr: install setuptools >= 12

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-07-31 19:51:31 +08:00
Jan Fajerski
7ea50c1729
Merge pull request #27754 from jan--f/c-v-validate-device-lists
ceph-volume: batch ensure device lists are disjoint
2019-07-31 13:16:11 +02:00
Ernesto Puerta
4963daf5cb
doc: Show Jenkins commands
Fixes: https://tracker.ceph.com/issues/41023
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2019-07-31 12:58:37 +02:00
Willem Jan Withagen
1a4a8b3bc6 test/cli: Make the ceph-conf test more liberal
The tests look for the exact error text.
But they are OS dependant, and thus on FreeBSD the output is
slightly different:
2019-07-30T19:04:16.581+0000 80bc48000 -1 parse_file: filesystem error: in file_size: No such file or directory [from-env]

This stems from the fact that different OSes use different
standard libraries: libc++ <> libstd++

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-07-31 12:22:39 +02:00
Venky Shankar
37a9b5375f pybind/rbd: fix call to unregister_osd_perf_queries
.. should be invoked as self.unregister_osd_perf_queries()
rather than self.module.unregister_osd_perf_queries().

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2019-07-31 05:28:49 -04:00
Jan Fajerski
74d18394bf
Merge pull request #29318 from tspmelo/wip-backport-component
script/ceph-backport.sh: Allow to set component label
2019-07-31 10:33:23 +02:00
Kefu Chai
2094c2f2fd pybind/mgr: install setuptools >= 12
otherwise we have
```
  Running setup.py egg_info for package apache-libcloud

    libcloud/utils/py3.py:76: UserWarning: Missing backports.ssl_match_hostname package
      warnings.warn("Missing backports.ssl_match_hostname package")
    Traceback (most recent call last):
...
      File "build/bdist.linux-x86_64/egg/setuptools_scm/version.py", line 66, in _warn_if_setuptools_outdated

...
setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)
...
Command python setup.py egg_info failed with error code 1 in /tmp/tmp.i95Fg82Ea4/venv/build/apache-libcloud
```

and more important thing is, setuptools >= 12 allows us to install the
python modules in venv, without uninstalling existing modules if they
are also installed in by system packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-31 16:26:29 +08:00
Sebastian Wagner
ee6a415cbb mgr/rook: monkey-patch for CoreV1Api.list_node
`kubernetes.client.apis.core_v1_api.CoreV1Api#list_node` is
broken and needs a monkey patch. Might be related to
an incompatible k8s cluster version.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 10:15:38 +02:00
Sebastian Wagner
76b66d904d mgr/orchestrator: better format service ls
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 10:15:31 +02:00
Sebastian Wagner
676bde1741 mgr/rook: Implement fallback for CentOS 7
`kubernetes-client` requires `urllib3` > 1.15, but CentOS 7 ships 1.10

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 10:15:15 +02:00
Sebastian Wagner
29aeebb40f mgr/rook: Move KubernetesResource to rook_cluser.py
Use `KubernetesResource` as cache for pods, inventory maps and nodes.
This is required for the dashboard.

Also:
  properly return K8s nodes
  fix minor issues

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:59:36 +02:00
Paul Cuzner
00cfc5c402 mgr/rook: Add feature to gather k8s storageclass information
Provide ability to fetch or watch k8s resources,
and hold the resulting objects in a dict. A method
has been added to the rook module to allow other
mgr modules to fetch the state of specific k8s
resources.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2019-07-31 09:57:33 +02:00
Sebastian Wagner
cf7ce17795 mgr/orchestrator_clli: Always return str
Also: Don't return `"None"`
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:49:07 +02:00
Sebastian Wagner
1d72af6258 mgr/ansible: Moved RGW defaults to ansible
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:49:03 +02:00
Sebastian Wagner
c09cd1ccd8 mgr/rook: Fix RGW creation
* Moved RGWSpec defaults to a new method

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:48:50 +02:00
Sebastian Wagner
dc54e77bd6 mgr/orchestrator: Split *_stateless_service and add get_feature_set
Split *_stateless_service, because every orchestrator
did some method dispatching internally anyway.

Also: Added test

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-31 09:44:57 +02:00
Jan Fajerski
8fcdb4d628 ceph-volume: batch ensure device lists are disjoint
Fixes: https://tracker.ceph.com/issues/41018

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-07-31 09:41:14 +02:00