Commit Graph

83547 Commits

Author SHA1 Message Date
xie xingguo
1e3498ad91 pybind/mgr/balancer: sanity check against empty roots
To avoid crashes as below:
```
File "/usr/lib/ceph/mgr/balancer/module.py", line 181, in calc_stats
    score += target[k] * (math.erf(((adjusted - avg)/avg) / math.sqrt(2.0)))
ZeroDivisionError: float division by zero
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-08 09:05:58 +08:00
xie xingguo
193c3af4fb pybind/mgr/balancer: s/iteritems/iterkeys/
We don't care about the values here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-08 09:05:43 +08:00
xie xingguo
201fc98c35 pybind/mgr/balancer: skip out/crush-out osds
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-07 11:14:12 +08:00
Alfredo Deza
cf65a91acf
Merge pull request #20673 from ceph/wip-rm23148
docs fix ceph-volume missing sub-commands

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-03-02 09:49:30 -05:00
Xie Xingguo
01fc18c447
Merge pull request #20684 from liewegas/wip-bluestore-cache-debug
os/bluestore: no trim debug noise if there is no trimming to be done

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-02 10:57:57 +08:00
Sage Weil
4cdadddffb os/bluestore: no trim debug noise if there is no trimming to be done
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-01 20:33:24 -06:00
Kefu Chai
7c1ab086da
Merge pull request #20637 from wjwithagen/wjw-fix-MDSmonitor.h
mon/MDSMonitor: fix clang build failure

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-03-02 10:20:54 +08:00
Kefu Chai
b60659564f
Merge pull request #20671 from batrick/rados-doc-flags
doc: init flags to 0 in rados example

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-02 09:19:54 +08:00
Kefu Chai
1b5c937509
Merge pull request #20653 from xiexingguo/wip-fix-upmap
mon, osd: fix potential collided *Up Set* after PG remapping

Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-02 09:01:06 +08:00
Kefu Chai
759e5f5a9b
Merge pull request #20655 from xiexingguo/wip-fix-calc-upmap
osd/OSDMap: skip out/crush-out osds

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-02 08:59:49 +08:00
Kefu Chai
ca2fd5abb8
Merge pull request #20506 from jan--f/prometheus-rm-outdated-daemon-metrics
pybind/mgr/prometheus: don't export metrics for dead daemon; new metrics

Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-02 08:57:42 +08:00
Kefu Chai
8e272b2549
Merge pull request #20667 from tchaikov/wip-qa-standalone
qa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-03-02 08:56:11 +08:00
Jason Dillaman
aed3a1db9d
Merge pull request #20648 from trociny/wip-snap_set_diff-empty_snapset
librados/snap_set_diff: don't assert on empty snapset

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-01 15:35:40 -05:00
Jason Dillaman
f91753d709
Merge pull request #20613 from Songweibin/wip-23038
rbd: import with option --export-format 2 fails to protect snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-01 15:35:23 -05:00
Jason Dillaman
6eaa1c4bc3
Merge pull request #20608 from Songweibin/wip-rbd-snap-purge
rbd: allow remove all unprotected snapshots

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-01 15:35:02 -05:00
Jason Dillaman
f9130d4a3d
Merge pull request #20578 from trociny/wip-pybind-group
librbd: small cleanup for recently merged code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-01 15:34:26 -05:00
Casey Bodley
21cf09f8c6
Merge pull request #20621 from cbodley/wip-rgw-data_lock
rgw: don't hold data_lock over frontend io

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-03-01 15:05:50 -05:00
Casey Bodley
e05b74cb05
Merge pull request #20449 from cbodley/wip-rgw-beast-coroutine
rgw: switch beast frontend back to stackful coroutine

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-03-01 15:04:59 -05:00
Alfredo Deza
c66300db83 doc/install add missing lvm subcommand for ceph-volume
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-03-01 14:22:01 -05:00
Alfredo Deza
1c846f3389 doc/rados add missing lvm subcommand for ceph-volume
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-03-01 14:21:05 -05:00
Casey Bodley
855512aa28
Merge pull request #20546 from yehudasa/wip-rgw-gc-aio
rgw: gc use aio

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2018-03-01 13:54:11 -05:00
Patrick Donnelly
f52782694f
Merge PR #20643 into master
* refs/pull/20643/head:
	ceph-volume-client: fix command argument order

Reviewed-by: Ramana Raja <rraja@redhat.com>
2018-03-01 08:48:30 -08:00
Andrew Schoen
d14e4e6452
Merge pull request #20650 from ceph/wip-rm22841
ceph-volume be resilient to $PATH issues

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-03-01 16:42:38 +01:00
Patrick Donnelly
a276c7097e
doc: init flags to 0 in rados example
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-03-01 07:02:00 -08:00
Jason Dillaman
ecadb2c85b
Merge pull request #20656 from yaozongyou/rbd-fix-return
librbd: fix missing return in NotifyMessage::get_notify_op

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-01 08:45:03 -05:00
songweibin
7deeef9164 rbd: allow remove all unprotected snapshots
allow remove all unprotected snapshots when exiting
protected snapshots in the same image.
Fixes: http://tracker.ceph.com/issues/23126

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-03-01 21:01:24 +08:00
songweibin
8f43eff6da rbd: import with option --export-format fails to protect snapshot
Fixes: http://tracker.ceph.com/issues/23038

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-03-01 20:48:38 +08:00
xie xingguo
5fa467e06b mon, osd: fix potential collided *Up Set* after PG remapping
The mgr balancer module are basically doing optimizations based on
the snapshots of OSDMap at certain moments, which turns out to be
the culprit of data loss since it can produce bad PG mapping results
sometimes while in upmap mode.
I.e.:
1) original cluster topology:

-5       2.00000     host host-a
 0   ssd 1.00000         osd.0       up  1.00000 1.00000
 1   ssd 1.00000         osd.1       up  1.00000 1.00000
-7       2.00000     host host-b
 2   ssd 1.00000         osd.2       up  1.00000 1.00000
 3   ssd 1.00000         osd.3       up  1.00000 1.00000
-9       2.00000     host host-c
 4   ssd 1.00000         osd.4       up  1.00000 1.00000
 5   ssd 1.00000         osd.5       up  1.00000 1.00000

2) mgr balancer applies optimization for PG 3.f:

            pg-upmap-items[3.f : 1->4]
3.f [1 3] + -------------------------> [4 3]

3) osd.3 is out/reweighted etc., original crush mapping of 3.f changed
   (while pg-upmap-items did not):

            pg-upmap-items[3.f : 1->4]
3.f [1 5] + -------------------------> [4 5]

4) we are now mapping PG 3.f to two OSDs(osd.4 & osd.5) on the same host
   (host-c).

Fix the above problem by putting a guard procedure before we can
finally encode these *unsafe* upmap remappings into OSDMap.
If any of them turns out to be inappropriate, we can simply cancel it
since balancer can still re-calculate and re-generate later if necessary.

Fixes: http://tracker.ceph.com/issues/23118
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-01 19:35:49 +08:00
Kefu Chai
e1c23cde27
Merge pull request #20640 from tchaikov/wip-async-dpdk-temp-str
msg/async: avoid referencing the temporary string

Reviewed-by: Haomai Wang <haomai@xsky.com>
2018-03-01 19:19:01 +08:00
Kefu Chai
8157dbb31f
Merge pull request #20531 from xiexingguo/wip-ec-scrub-sleep
osd/ECBackend: inject sleep during deep scrub

Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-01 19:17:26 +08:00
Kefu Chai
fc43ae1724 qa/standalone: s/delete_erasure_pool/delete_erasure_coded_pool/
it's a regression introduced by ac56a202

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-03-01 19:09:31 +08:00
Willem Jan Withagen
7301a4f994 mon: fix clang error
The specific specialisations in both .h and .cc does not really fly
with Clang

/home/jenkins/workspace/ceph-master/src/mon/MDSMonitor.cc:79:22: error: template parameter redefines default argument
template <int dblV = 7>
                     ^
/home/jenkins/workspace/ceph-master/src/mon/MDSMonitor.h:76:23: note: previous default template argument defined here
  template<int dblV = 7>
                      ^
1 error generated.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-03-01 10:11:30 +01:00
Jan Fajerski
d03c79676b pybind/mgr/prometheus: extend daemon metadata with hostname and ceph version
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-03-01 08:44:27 +01:00
Jan Fajerski
0095e6a1ba pybind/mgr/prometheus: add fs and mds metadata metrics
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-03-01 08:43:09 +01:00
Jan Fajerski
71a6408b1a pybind/mgr/prometheus: new mon metrics
Remove mon_quorum_count and replace it with per-MON quorum status
(mon_quorum_status). Also add mon_metadata metrics.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-03-01 08:43:09 +01:00
Jan Fajerski
f88877ab0b pybind/mgr/prometheus: add Metrics class to manage Metric instances
The central change of this commit is that per-daemon metrics are now
managed by first appending the metric (using Metrics.append) to a
staging area. Then the metrics for specific paths (metric names) are
overwritten by the staged metrics (by calling Metrics.reset). This gets
rid of metrics from daemon that are no longer in the cluster. I.e. when
ceph no longer reports metrics for one OSD daemon (because it was
removed from the cluster) the prometheus module will no longer export
metrics for that daemon.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-03-01 08:43:09 +01:00
Jos Collin
59da11b195
Merge pull request #20630 from joscollin/wip-doc-monitoring
doc: Update monitoring.rst

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>
Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
2018-03-01 11:46:34 +05:30
Jos Collin
5542338eb9 doc: Update monitoring.rst
Updated monitoring.rst with
	* Latest command output
	* misc doc fixes

Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-03-01 10:15:18 +05:30
Yao Zongyou
2fa18ef9e0 rbd: fix return is missing in NotifyMessage::get_notify_op
Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
2018-03-01 11:15:44 +08:00
xie xingguo
1f78ca221d osd/OSDMap: skip out/crush-out osds
We might do 'ceph osd out <osd.x>' or 'ceph osd crush reweight <osd.x> 0'
for various reasons, and hence can produce 0-weighted OSDs.
Skip those OSDs when trying to calculdate PG upmaps so we won't be able to
hit the *assert* below:

/build/ceph-13.0.1-2232-g64665c7/src/osd/OSDMap.cc: 4179: FAILED assert(target > 0)

See also:

http://pulpito.ceph.com/xxg-2018-02-28_09:02:53-rados-wip-fix-upmap-distro-basic-smithi/2235497/

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-01 10:52:08 +08:00
Alfredo Deza
161f331382 ceph-volume process always try to expand the executable
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-02-28 15:06:25 -05:00
Alfredo Deza
d26215a49b ceph-volume tests.util ensure behavior of new which utility
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-02-28 12:10:48 -05:00
Alfredo Deza
c36afed9a2 ceph-volume util.system add a helper to find system executables
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-02-28 12:10:47 -05:00
Kefu Chai
1932cd1a83
Merge pull request #20646 from tchaikov/wip-qa/run-standalone.sh/python-path
qa/run-standalone.sh: set PYTHONPATH for FreeBSD also

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-03-01 00:03:28 +08:00
Kefu Chai
33aa2e5ee3
Merge pull request #20634 from tchaikov/wip-delete-pool-in-standalone-test
qa/standalone: extract delete_pool()

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-02-28 23:47:39 +08:00
Abhishek L
16fa9820a7
Merge pull request #20619 from theanalyst/doc/release-notes/12.2.4
doc: 12.2.4 release notes

Reviewed-By: Alfredo Deza <adeza@redhat.com>
Reviewed-By: Lenz Grimmer <lgrimmer@suse.com>
2018-02-28 16:42:12 +01:00
Mykola Golub
2be4840afd librados/snap_set_diff: don't assert on empty snapset
Instead treat the diff as a full-object delta.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-02-28 17:33:50 +02:00
Kefu Chai
aaffaee38b qa/run-standalone.sh: set PYTHONPATH for FreeBSD also
do not set PYTHONPATH=/usr/lib/python2.7/dist-packages anymore, it will
be overridden later on anyway.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-28 22:30:32 +08:00
Yuri Weinstein
57ffaaa389
Merge pull request #19015 from hansbogert/master
mon: Add `ceph osd get-require-min-compat-client` command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
2018-02-28 06:25:31 -08:00
Kefu Chai
8c4f10c9bb
Merge pull request #20607 from shangfufei/test_stack
msg/async:fix the incoming parameter type of EventCenter::process_events()

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-02-28 22:02:02 +08:00