Commit Graph

95828 Commits

Author SHA1 Message Date
Nathan Cutler
5a0846631c rpm: put numpy and scipy deps in distro conditionals
Without this patch, attempts to install the ceph-mgr-diskprediction-local RPM
fail on SUSE platforms with the following error:

can't install ceph-mgr-diskprediction-local-14.1.0.402+ga396e8bf3b-3742.1.noarch:
  nothing provides numpy needed by ceph-mgr-diskprediction-local-14.1.0.402+ga396e8bf3b-3742.1.noarch
  nothing provides scipy needed by ceph-mgr-diskprediction-local-14.1.0.402+ga396e8bf3b-3742.1.noarch

Also take into account package naming differences between Fedora and
RHEL/CentOS.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-03-06 15:13:23 +01:00
Jeff Layton
a256735d4c mgr/orchestrator: allow scaling the NFS server count up and down
Add a new 'ceph orchestrator nfs update' command that will take the
NFS clustername and a new count as arguments. That will get translated
to a StatelessServiceSpec and passed to update_stateless_service.

Also, add the necessary stubs to the test_orchestrator and the CLI
QA test.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-03-06 07:15:14 -05:00
Jeff Layton
32161b7699 mgr/rook: allow scaling nfs count
Allow rook to handle scaling the NFS server count up and down in an NFS
cluster. We just manifest these changes as change to the
spec.server.active field in the CRD.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-03-06 07:14:46 -05:00
Jeff Layton
296d4a3e32 orchestrator: drop id_ argument from update_stateless_service
...we have a "name" field in StatelessServiceSpec.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-03-06 07:14:46 -05:00
Jeff Layton
c792c5a353 mgr/orchestrator: just keep a single count value in StatelessServiceSpec
We currently have a min_size/max_size values in here, but we don't have
any orchestrators that can take advantage of two values. Let's just keep
a simple count for now, until we do.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-03-06 07:14:46 -05:00
Ricardo Marques
ea5541b9d0
Merge pull request #26388 from tspmelo/wip-iscsi-no-auth-ui
mgr/dashboard: Add UI for disabling ACL authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-03-06 12:11:05 +00:00
Volker Theile
8b797b7393
Merge pull request #26382 from ricardoasmarques/no-auth
mgr/dashboard: iSCSI - Add support for disabling ACL authentication (backend)

Reviewed-by: Volker Theile <vtheile@suse.com>
2019-03-06 12:45:33 +01:00
Kefu Chai
9d59e17404
Merge pull request #26783 from cyx1231st/wip-fix-async-echo
test/crimson: add dummy_auth to test_async_echo

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-06 18:15:09 +08:00
Yingxin Cheng
bb42dada61 test/crimson: add dummy_auth to test_async_echo
fix echo between async-msgrs

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-03-06 23:16:07 +08:00
Yan, Zheng
3ae56ab82b mds: avoid calling clientreplay_done() prematurely
maybe_clientreplay_done() does not correctly handle the case that
replayed request is in the finished_queue (hasn't been dispatched)

Fixes: https://tracker.ceph.com/issues/38597
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-03-06 16:15:26 +08:00
xie xingguo
ba210653b5 osd/PrimaryLogPG: fix last_peering_reset checking on manifest flushing
```handle_manifest_flush``` is obviously using the wrong
**last_peering_reset** to check whether a new peering procedure
has been re-initialized by then.

Fix by using a different alias of the local copy of the
pg-wide **last_peering_reset** variable, which is less confusing and
error-prone.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-06 14:28:55 +08:00
Sage Weil
76ea722880 Merge PR #26749 into master
* refs/pull/26749/head:
	common/util: report CONTAINER_IMAGE in daemon metadata

Reviewed-by: Sébastien Han <seb@redhat.com>
2019-03-05 21:12:19 -06:00
xie xingguo
af02d1031d mgr: 'osd df' by specified class or (crush) name
For large clusters, we use device classes to isolate storage pools.
The existing 'osd df' output turns out to be too nosiy, say, if
you care about only single storage pool with osds possibly spanning over
all hosts.

With this change you are now being able to do 'osd df' by class (or by pool,
if you simply use classes to separate different pools), or by a specified
crush bucket name you are currently interested in, which is much more
convenient.

Some examples:
```
$ bin/ceph osd df tree
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60 1.00   -        root default
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
-5       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
 1   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  61     up         osd.1
 2   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  51     up         osd.2
                    TOTAL 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree class aaa
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60 1.00   -        root default
-3       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
-5       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
                    TOTAL 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree name ceph11
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
                    TOTAL 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-06 11:10:56 +08:00
Kefu Chai
e6d741d7f1
Merge pull request #26774 from liu-chunmei/ceph_seastar_osd_preboot
crimson/osd: correct the order of parameters passed to OSD::_preboot()

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-06 10:42:55 +08:00
chunmei Liu
1ebbddca1c crimson/osd: correct the order of parameters passed to OSD::_preboot()
Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
2019-03-05 17:34:28 -08:00
Ilya Dryomov
7ab3153902 qa/suites/krbd/wac: bluestore snippet is placed incorrectly
Instead of generating three tests, each with bluestore-bitmap.yaml, it
generates four tests: one consisting of just bluestore-bitmap.yaml and
the other three without any trace of bluestore.  This was introduced in
commit 711df71790fa ("qa: objectstore snippets for krbd").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-05 23:07:27 +01:00
Patrick Donnelly
d23e1555df
mds: refactor subtree accessors
Mostly avoiding inefficient use of std::set.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-05 13:40:17 -08:00
Sage Weil
e030ba465d Merge PR #26740 into master
* refs/pull/26740/head:
	rpm: advance spec file copyright date

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2019-03-05 14:05:54 -06:00
Sage Weil
bf23c64eeb Merge PR #26767 into master
* refs/pull/26767/head:
	Revert "msg/async/Protocol*: send keep alive if existing wins"

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-05 11:08:03 -06:00
Sage Weil
11f986830e Merge PR #26719 into master
* refs/pull/26719/head:
	rpm: re-add pyOpenSSL as ceph-mgr runtime dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-05 09:12:43 -06:00
Sage Weil
bf615abbce Merge PR #25540 into master
* refs/pull/25540/head:
	mgr/ServiceMap: show service's id on summary

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-05 09:12:26 -06:00
Sage Weil
a603609a00 Merge PR #26567 into master
* refs/pull/26567/head:
	os/bluestore: fix bluefs inability to reclaim space

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-03-05 09:11:53 -06:00
Sage Weil
af9e76e29c Merge PR #26752 into master
* refs/pull/26752/head:
	msg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-05 09:10:33 -06:00
Sage Weil
9ad8946492 Merge PR #26742 into master
* refs/pull/26742/head:
	osd/PG: do not touch this->cct after PG is destroyed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-05 09:09:29 -06:00
Patrick Donnelly
8a85bd2731
Merge PR #26460 into master
* refs/pull/26460/head:
	client: parameter "cap" is not used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-03-05 06:25:19 -08:00
Xie Xingguo
ad8e7d33b1
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-03-05 20:16:18 +08:00
xie xingguo
fd520a17db Revert "msg/async/Protocol*: send keep alive if existing wins"
This reverts commit 61b9432ef9a3847eceb96f8d5a854567c49bbf61.

If we are in the middle of replacing, we can not queue any further
write events into the old center because we may end up replacing
existing connection's center with a new one, and hence executing
the newly queued write events in the old thread.

See **transfer_existing** for a detailed description.

Also the patch does not make a lot of sense for the original issue
it tried to resolve, because **send_keepalive** is a pure noop if the
underlying connection is not ready, which is obviously true for the
case demonstrated in http://tracker.ceph.com/issues/38493..

Fixes: http://tracker.ceph.com/issues/38569
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 20:02:23 +08:00
huanwen ren
1b64815f13 client: parameter "cap" is not used
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2019-03-05 19:52:34 +08:00
xie xingguo
4ff990fa25 mon/OSDMonitor: add 'osd crush get-device-class' command
which should work as a good complementation of
the existing **set-device-class** and "rm-device-class"
command family.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 17:21:08 +08:00
Sebastian Wagner
e280c1a0df
Merge pull request #26715 from sebastian-philipp/script-pypy
src/script: add run_mypy to run static type checking on Python code

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-03-05 10:18:18 +01:00
xie xingguo
733fff22bb mon/OSDMonitor: add boundary check for pool recovery_priority
See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 08:15:39 +08:00
Sage Weil
bd52f35fd0 Merge PR #26444 into master
* refs/pull/26444/head:
	ceph_features: deprecate MON_SINGLE_PAXOS
	osdmap: map versions to releases
	mon: retire some `osd set/unset` flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-04 17:26:12 -06:00
Sage Weil
e6d6716ea0 Merge PR #26754 into master
* refs/pull/26754/head:
	osdc: remove lderr() hexdump from handle_osd_op_reply

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 17:15:41 -06:00
Sage Weil
db6201acaa Merge PR #26744 into master
* refs/pull/26744/head:
	mds: fix encoded version of mds_info_t

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 17:13:30 -06:00
Sage Weil
5c6d9d556f Merge PR #26694 into master
* refs/pull/26694/head:
	rpm: drop use of $FIRST_ARG

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2019-03-04 16:25:18 -06:00
Alfredo Deza
e5c1a5b423
Merge pull request #26753 from ceph/ceph-volume-osd-ports
ceph-volume: set number of osd ports in the tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 16:58:36 -05:00
Casey Bodley
19eafda6a7 osdc: remove lderr() hexdump from handle_osd_op_reply
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-03-04 16:29:51 -05:00
Sage Weil
d233010f9a Merge PR #26631 into master
* refs/pull/26631/head:
	msg/async, v2: fix unused variable warning in ::discard_out_queue().

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 15:00:09 -06:00
Andrew Schoen
1132cb3c25 ceph-volume: set number of osd ports in the tests
The number of ports the OSDs listen on depend on the version of ceph
being used, so we need to test for that number accordingly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2019-03-04 14:43:37 -06:00
Sage Weil
ceb66a4996 ceph_test_rados_[a]io_pp: add test verifying read-into-bufferlist behavior
libradosstriper relies on this, and other callers may too.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-04 14:24:05 -06:00
Radoslaw Zarzynski
b498c5b503 msg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-04 20:58:47 +01:00
Sage Weil
94b825448f Merge PR #26745 into master
* refs/pull/26745/head:
	ceph_release: we're in rc phase

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 13:45:23 -06:00
Sage Weil
42ae29a129 common/util: report CONTAINER_IMAGE in daemon metadata
Rook will set this to the registry + image string.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-04 13:04:22 -06:00
Casey Bodley
97dbbf0cfd
Merge pull request #26741 from cbodley/wip-test-rgw-reshard-wait2
test/rgw: fix race in test_rgw_reshard_wait

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-04 12:30:48 -05:00
Sage Weil
8d26102b2a ceph_release: we're in rc phase
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-04 11:17:20 -06:00
Patrick Donnelly
fb594328e9
mds: fix encoded version of mds_info_t
Introduced-by: 1f5892f820
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-04 09:06:16 -08:00
Sebastian Wagner
881a14cb20
Merge pull request #26635 from noahdesu/cherrypy-ipv6
mgr/dashboard: fix for using '::' on hosts without ipv6

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-03-04 17:51:34 +01:00
Andrew Schoen
04c577429f
Merge pull request #26739 from alfredodeza/wip-rm38568
ceph-volume: replace testinfra command with py.test

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-03-04 10:29:33 -06:00
Volker Theile
3c7ee45666 mgr/dashboard: Cleanup Python code
Signed-off-by: Volker Theile <vtheile@suse.com>
2019-03-04 17:21:35 +01:00
Patrick Donnelly
ff1c7adfd3
Merge PR #26734 into master
* refs/pull/26734/head:
	doc: correct vstart.sh file location

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-04 07:39:08 -08:00