Commit Graph

76585 Commits

Author SHA1 Message Date
Sage Weil
cb807b0006 crush: fix preservation of shadow bucket ids
1- a decompiled and recompiled was parsing the class bucket ids but it
wasn't actually using them.
2- rebuild_roots_with_classes() was throwing out the old ids and assigning
new ids when the tree was rebuilt.

Fix by passing in a (potentially partial) class_bucket map into
populate_classes().  Take care to allocate new bucket ids that don't
collide with previously used ids.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 22:32:47 -04:00
Patrick Donnelly
d89af4a3e8
Merge PR #16802 into master
* refs/remotes/upstream/pull/16802/head:
	qa: update wait_for_health for new health json syntax

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 16:20:20 -07:00
John Spray
0613d411aa qa: update wait_for_health for new health json syntax
Fixes: http://tracker.ceph.com/issues/20890
Signed-off-by: John Spray <john.spray@redhat.com>
2017-08-03 23:46:41 +01:00
Patrick Donnelly
051dd86898
Merge PR #16766 into master
* refs/remotes/upstream/pull/16766/head:
	cephfs: Clarify health warning
	cephfs: Always print filesystem name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 15:05:53 -07:00
Douglas Fuller
e697636ebb cephfs: Clarify health warning
Add text to MDS_INSUFFICIENT STANDBY to clarify it refers
to MDS daemons.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-08-03 17:45:48 -04:00
Douglas Fuller
53bf6df4c9 cephfs: Always print filesystem name
Previously, the FSMap summary omitted the filesystem
name in clusters with only one filesystem. Print the
file system name no matter what.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-08-03 17:44:44 -04:00
Kefu Chai
007095b7ae qa/workunits/mon/crush_ops.sh: remove existing dev class before setting it
we cannot overwrite existing dev class, and "osd_class_update_on_start"
is true by default (see 0c885d6). so we should remove all device classes before
setting them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-08-03 17:26:26 -04:00
xie xingguo
d4390486a0 mon/OSDMonitor: kill unused variable 'ts'
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-03 17:26:26 -04:00
xie xingguo
734b5f2c60 test/osd-fast-mark-down: enable 'osd-class-update-on-start' by default
116cf759c8
will now hide all shadow trees(roots), so this is not applicable anymore
(actually it is misleading).

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-03 17:26:26 -04:00
xie xingguo
f134a14eff crush: make 'remove_root' idempotent
We might use 'crush link' to link same host into
different roots, which as a result can cause different
shadow trees reference same hosts too.

This means we may need to destory the same buckets(hosts, racks, etc.)
multiple times during rebuilding all shadow trees and hence 'remove_root'
shall be idempotent.

Fixes: http://tracker.ceph.com/issues/20845
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-03 17:26:26 -04:00
Sage Weil
b17287ef14 Merge pull request #16740 from liewegas/wip-crush-tree
mon: show class in 'osd crush tree' output; sort output

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-03 16:16:17 -05:00
Patrick Donnelly
b8a474d9a0
Merge PR #16772 into master
* refs/remotes/upstream/pull/16772/head:
	qa: add health whitelist for all fs sub-suites

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 14:07:58 -07:00
Patrick Donnelly
9d348ad8c9
qa: add health whitelist for all fs sub-suites
Fixes: http://tracker.ceph.com/issues/20892

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 14:01:28 -07:00
Patrick Donnelly
60fa9714d4
Merge PR #16768 into master
* refs/remotes/upstream/pull/16768/head:
	qa: fix log whitelist string

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-03 13:55:42 -07:00
Patrick Donnelly
f2a1a10da5
Merge PR #16743 into master
* refs/remotes/upstream/pull/16743/head:
	mds: add inter-mds operations to PerfCounter
2017-08-03 13:55:39 -07:00
Patrick Donnelly
fda4188772
Merge PR #16654 into master
* refs/remotes/upstream/pull/16654/head:
	mds: fix race in "open inode by inode number"
	mds: cleanup {do_open,_do_find}_ino_peer()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 13:55:36 -07:00
Patrick Donnelly
3755649a4f
Merge PR #16631 into master
* refs/remotes/upstream/pull/16631/head:
	cephfs: Resend discover when joining from starting state
	cephfs: Wait for MDCache to be ready before starting log segment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 13:55:33 -07:00
Patrick Donnelly
8fa8c241bd
Merge PR #16363 into master
* refs/remotes/upstream/pull/16363/head:
	mds: fix health checks for multiple filesystems
	mon: fix log message on mds rank stop
	mon: uniform way of referring to MDS daemons
	mds: revise health checks for _FAILED/_DEGRADED
	mon/MDSMonitor: log messages on MDS rank changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 13:55:29 -07:00
Patrick Donnelly
66756c4f65
Merge PR #16292 into master
* refs/remotes/upstream/pull/16292/head:
	qa: use new hex rep of inode
	qa: fix whitelist error message
	mds: refine "Scrub error" cluster log message
	mds: polish clog messages
	doc: developer logging guidance

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 13:55:21 -07:00
Sage Weil
342607f4d5 Merge pull request #16749 from tchaikov/wip-restful-delete-key
mgr: handle "module.set_config(.., None)" correctly 

Reviewed-by: John Spray <john.spray@redhat.com>
2017-08-03 15:53:27 -05:00
Sage Weil
af145dd592 Merge pull request #16756 from liewegas/wip-20113
ceph: wait for maps before doing 'ceph tell ... help'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-03 15:52:56 -05:00
Yuri Weinstein
09fd18d031 Merge pull request #16760 from cbodley/wip-rgw-disable-lifecycle-s3tests
qa/rgw: disable lifecycle tests because of expiration failures

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2017-08-03 13:25:39 -07:00
Yuri Weinstein
0f481d069d Merge pull request #16491 from cbodley/wip-20308
rgw: match wildcards in StringLike policy conditions

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2017-08-03 13:24:59 -07:00
Loic Dachary
ed22159fdd ceph-disk: dmcrypt cluster must default to ceph
If ceph_fsid is not found, which is the case for legacy dmcrypted OSD,
the cluster must default to ceph, as it was before support for non
standard cluster names was introduced.

Fixes: http://tracker.ceph.com/issues/20893

Signed-off-by: Loic Dachary <loic@dachary.org>
2017-08-03 21:40:43 +02:00
Josh Durgin
6e38a470bc osd/PGLog: avoid infinite loop if missing version is corrupted
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-08-03 14:24:25 -04:00
Sage Weil
133e712f35 Merge pull request #16738 from liewegas/wip-20810
os/bluestore: fix dirty_range on _do_clone_range

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-08-03 13:22:46 -05:00
Josh Durgin
b172642124 Merge pull request #16789 from liewegas/wip-ec-m-2
qa: avoid map-gap tests for k=2 m=1

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-08-03 11:20:13 -07:00
Sage Weil
b8627f897a Merge pull request #16795 from liewegas/wip-mgr-whitelist
qa/suites/rados/mgr/tasks/failover: whitelist

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-08-03 13:14:20 -05:00
Sage Weil
ef21c9d7df qa/suites/rados/thrash-erasure-code: do not test map gap with m=1
We test EC profiles with m=1 here, and mapgap can lead to incomplete pgs
because it takes an osd down and waits for healthy.

Fixes: http://tracker.ceph.com/issues/20844
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 14:13:02 -04:00
Sage Weil
f74d71f708 qa/suites/rados/thrash-erasure-coe-big/clsuter: 12 osds on 3 nodes not 4
smithi have 4 nvme partitions available, not 3.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 14:11:43 -04:00
Josh Durgin
ae48c75065 Merge pull request #16797 from jdurgin/wip-upgrade-jewel-x
qa: timeout when waiting for mgr to be available in healthy()

Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-03 11:11:41 -07:00
Sage Weil
63221e21f5 qa/suites/rados/thrash-erasure-code-big: add k=4 m=2
Get better coverage for larger codes.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 14:10:36 -04:00
Sage Weil
e994b03335 qa/suites/rados/monthrash/worklaods/rados_api_tests: whitelist SMALLER_PGP_NUM
The rados/test.sh fiddles with pg_num.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 13:31:39 -04:00
Sage Weil
7c350180b1 qa/suites/rados/mgr/tasks/failover: whitelist
remote/smithi025/log/ceph.log.gz:2017-08-03 07:02:15.049074 mon.b mon.0 172.21.15.25:6789/0 197 : cluster [INF] Manager daemon x is unresponsive, replacing it with standby daemon y
remote/smithi025/log/ceph.log.gz:2017-08-03 07:03:10.078032 mon.b mon.0 172.21.15.25:6789/0 226 : cluster [WRN] Manager daemon x is unresponsive.  No standby daemons available.

x and y may be swapped, so whitelist the rest of the string.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 12:40:01 -04:00
Sage Weil
0291118183 doc/rados/configuration/bluestore-config-ref: fix indentation
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 12:24:33 -04:00
Sage Weil
196dc8b399 doc/release-notes: fix bluestore links
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 09:33:55 -04:00
Sage Weil
24c5dea378 Merge pull request #16764 from liewegas/wip-release-notes
doc/release-notes: sort release note changes into the right section
2017-08-03 08:28:20 -05:00
Sage Weil
ea96265ed1 Merge pull request #16765 from liewegas/wip-bluestore-docs
doc/rados/configuration: document bluestore
2017-08-03 08:27:41 -05:00
Sage Weil
f2bcd0250b doc/rados/configuration/bluestore-config-ref: devices, checksumming, cache
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-03 09:21:18 -04:00
Sage Weil
029314954e Merge pull request #16773 from dmick/wip-make-check-cleanup
Clean up make check for persistent test nodes (like arm64)

Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-03 08:19:48 -05:00
Kefu Chai
088f98148e Merge pull request #16683 from Liuchang0812/clear-result-in-getchildren
os/bluestore: clear result in BlueRocksEnv::getChildren

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-03 21:17:06 +08:00
Kefu Chai
d6b9f36c48 Merge pull request #16717 from gmayyyha/osd-objectstore-20865
ceph-osd: fix auto detect which objectstore is currently running

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2017-08-03 21:14:34 +08:00
Kefu Chai
8c29f314c4 Merge pull request #16777 from SUSE/wip-20894
doc: document "rados cleanup" in rados manpage

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-03 20:22:03 +08:00
Abhishek L
ddab8e0dc2 Merge pull request #16775 from ovh/bp-fr-pendingnotes-update
PendingReleaseNotes: mention forced recovery

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-08-03 11:30:01 +02:00
Nathan Cutler
c02d41dcdb doc: document "rados cleanup" in rados manpage
Fixes: http://tracker.ceph.com/issues/20894
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-08-03 11:08:48 +02:00
Piotr Dałek
b57fb1eef1 PendingReleaseNotes: mention forced recovery
Mention forced recovery in release notes.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-08-03 09:45:28 +02:00
Patrick Donnelly
8d33cbbf5c
qa: use new hex rep of inode
Resolves a failure from QA:

    2017-08-02T19:23:27.489 INFO:tasks.cephfs_test_runner:======================================================================
    2017-08-02T19:23:27.489 INFO:tasks.cephfs_test_runner:FAIL: test_oversize (tasks.cephfs.test_fragment.TestFragmentation)
    2017-08-02T19:23:27.489 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20170802/qa/tasks/cephfs/test_fragment.py", line 71, in test_oversize
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:    self.assertEqual(frags[0]['dirfrag'], "10000000000.0*")
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:AssertionError: u'0x10000000000.0*' != '10000000000.0*'
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-02 21:39:48 -07:00
Dan Mick
8a53120037 test/encoding/readable.sh: clean up all tmpfiles/dirs
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-02 21:12:13 -07:00
Dan Mick
9f3b1cf5ea run-make-check.sh: set ulimit -n (because no one else does)
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-02 21:12:13 -07:00
Gregory Farnum
740daa3ca1 Merge pull request #16678 from mychoxin/yuanxin_make_print_informative
print more information when run ceph-osd cmd with 'check options'

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-08-02 19:59:50 -07:00