Commit Graph

78359 Commits

Author SHA1 Message Date
Dan Mick
eee54243a0 Merge pull request #17713 from tchaikov/wip-ceph.in-python
ceph.in: execv using the same python
2017-09-14 19:07:11 -07:00
Josh Durgin
520a5a218c Merge pull request #17583 from neha-ojha/wip-cbt-teuthology-integration
qa: add cbt task for performance testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-09-14 16:36:26 -07:00
vasukulkarni
fc196abd37 Merge pull request #17680 from ceph/wip-smoke-whitelist
qa/tests: Various whitelists for smoke suite
2017-09-14 16:13:37 -07:00
Vasu Kulkarni
1f5aba674c Various whitelists for tests to pass
See tracker http://tracker.ceph.com/issues/21376 for details

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-14 13:32:25 -07:00
Sage Weil
8f8f0b2505 mon/OSDMonitor: improve crush map validation
- move into OSDMap method
- ensure that rules exist for each pool
- ensure pool type matches rule type
- ensure rule mask min/max size cover the pool size

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:31:43 -04:00
John Spray
71d4b2bed5 mon: more forceful renumbering of legacy ruleset IDs
Previously, the rules were only modified in the trivial case,
so we continued to potentially have CRUSH maps with the
legacy ruleset functionality in use.

In order to ultimately remove rulesets entirely, we need
to do this more aggressively, renumbering all the rules
and then updating any pools as needed.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-14 12:30:17 -04:00
John Spray
675483ecca osd: s/crush_ruleset_in_use/crush_rule_in_use/
Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-14 12:30:17 -04:00
Sage Weil
9bd98b42fe ceph-bluestore-tool: better default logging; --log-file and --log-level options
- keep derr going to stderr (so we see 'fsck error' messages)
- hide the rest
- add friendly --log-file option

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:38 -04:00
Sage Weil
7d381af7ae os/bluestore: fsck: fix error prefixes
__func__ is now _fsck instead of fsck

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:38 -04:00
Sage Weil
aa4baad529 ceph-bluestore-tool: add 'bluefs-bdev-expand' to expand wal or db usage
If you are using the wal or db devices, this will expand bluefs's usage to
include the entire block device.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:01 -04:00
Sage Weil
9e492f2a22 ceph-bluestore-tool: add 'bluefs-bdev-sizes' command
Show bdev sizes vs owned extents.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:01 -04:00
Sage Weil
86db2d7b09 ceph-bluestore-tool: factor out bluefs mount
Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:01 -04:00
Sage Weil
659fcd279a os/bluestore: allow repair invocation
Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:01 -04:00
Sage Weil
c7b7a1f04f os/ObjectStore: add repair interface
Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-14 12:15:01 -04:00
Kefu Chai
d3dd77880a Merge pull request #17733 from yaozongyou/fix-doc-boolean-type-zero-to-false
doc: change boolean option default value from zero to false.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-14 22:53:47 +08:00
Sage Weil
1587f87b6b Merge pull request #17610 from liewegas/wip-bluestore-fm-mas
os/bluestore: set bitmap freelist resolution to min_alloc_size

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Varada Kari <varada.kari@gmail.com>
2017-09-14 09:18:41 -05:00
Kefu Chai
eb8bfd1955 Merge pull request #17617 from liewegas/wip-drop-disk-log
mon/PGMap: drop DISK LOG column

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-14 21:42:56 +08:00
Yao Zongyou
69129b73a5 doc: change boolean option default value from zero to false.
Signed-off-by: Yao Zongyou yaozongyou@vip.qq.com
2017-09-14 21:37:04 +08:00
Sage Weil
c1bb53aaba Merge pull request #17628 from majianpeng/bluestore-narrow-deferred-lock
os/bluestore/BlueStore: narrow deferred_lock in _deferred_submit_unlock.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-14 08:30:17 -05:00
songweibin
9ccb710aa4 doc,os,osdc: drop and modify some comments
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2017-09-14 21:27:38 +08:00
Xie Xingguo
9b71d98ff6 Merge pull request #17627 from majianpeng/bluefs-exce-debug-code
os/bluestore/BlueFS: Don't call debug related code under any condition.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-14 18:27:02 +08:00
Xie Xingguo
b4bb7ce2da Merge pull request #17371 from xiexingguo/wip-per-pool-full-control
mon, osd: per pool space-full flag support

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-14 18:26:12 +08:00
Xie Xingguo
78acfcf05b Merge pull request #17368 from xiexingguo/wip-nice-data
mon/PGMap: nice numbers for 'data' section of 'ceph df' command

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-14 18:25:24 +08:00
Kefu Chai
3bf2382376 pybind: remove unused get_ceph_version()
get_ceph_version() was get_version() before 7b04e7ff. and after that
change, the version number of the python extension is hardwired to
`__version__`. because pybind is a wrapper around the public APIs
exposed by librados, librbd and libcephfs, which are versioned
separately from ceph daemons and tools, it's reasonable not to use the
version number in ceph_ver.h. thus, it's safe to drop this function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-14 18:02:54 +08:00
kungf
c1dba46084 osd: get loadavg per cpu for scrub load threshold check
Signed-off-by: kungf <yang.wang@easystack.cn>
2017-09-14 17:00:23 +08:00
Kefu Chai
d803576002 ceph.in: execv using the same python
otherwise we could be using another python when respawing the ceph cli.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-14 11:33:48 +08:00
Patrick Donnelly
d929dae49b
Merge PR #17657 into master
* refs/remotes/upstream/pull/17657/head:
	mds: optimize MDCache::rejoin_scour_survivor_replicas()
	mds: fix MDSCacheObject::clear_replica_map
	mds: support limiting cache by memory
	common: refactor of lru
	mds: resolve unsigned coercion compiler warning
	common: use safer uint64_t for list size
	common: add bytes2str pretty print function
	mds: check if waiting is allocated before use
	mds: go back to compact_map for replicas
	mds: use mempool for cache objects
	mds: cleanup replica_map access
	common: add alloc_ptr smart pointer
	common: add warning on base class use of mempool
	common: use atomic uin64_t for counter

Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-09-13 20:08:51 -07:00
Patrick Donnelly
f818905e0f
Merge PR #17689 into master
* refs/remotes/upstream/pull/17689/head:
	ceph: fix "stopping mds can't export subtree"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-13 18:29:20 -07:00
Patrick Donnelly
d89653dc56
Merge PR #17545 into master
* refs/remotes/upstream/pull/17545/head:
	client: fix request reference leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2017-09-13 18:29:18 -07:00
vasukulkarni
f1ffcaed6d Merge pull request #17700 from cbodley/wip-qa-smoke-s3tests
qa/smoke: add rgw crypto config for s3tests
2017-09-13 17:44:24 -07:00
J. Eric Ivancich
360c8954aa osd: fix tests of mclock implementations to track opimizations for op wq
wip-osd-deq changes client identifiers from entity_inst_t to
uint64_t. Updates tests to do this as well.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-09-13 18:17:33 -04:00
Xuehan Xu
5531b6bb51 ceph_test_rados_api_tier: add ListSnap test
Signed-off-by: Xuehan Xu <xuxuehan@360.cn>
2017-09-13 17:49:54 -04:00
David Zafman
8805ef5342 osd: Only scan for omap corruption once
Before
 	state 2: Can have complete tables (some may be bad)
	state 3: Never had complete tables
After
 	state 2: Can have complete tables (some may be bad)
	state 3 with legacy: Can have complete tables (bad ones are cleared)
	state 3: Never had complete tables

Once OSDs boot with this change you can't downgrade to a previous release.
If someone does downgrade they could have unstable OSDs that hit assert(state.v < 3).
The following command run after shutting down the cluster but before downgrading
ceph packages would be a way to fix this.

ceph-osdomap-tool --omap-path ... --command resetv2

Fixes: http://tracker.ceph.com/issues/21328

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:25 -07:00
David Zafman
3e4d68640c tools: Add the ability to reset state to v2
Available for testing and user downgrade.

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:25 -07:00
David Zafman
3214882a95 tools: Show DB state information
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:25 -07:00
David Zafman
de43493990 tools: Add --backend option to ceph-osdomap-tool default to rocksdb
Fix hard-coded "leveldb" backend.  The command is broken in Luminous
now that "rocksdb" is the default.

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:24 -07:00
Patrick Donnelly
a73ad7676a
Merge PR #17608 into master
* refs/remotes/upstream/pull/17608/head:
	doc/cephfs/posix: put posix notes in perspective

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-13 13:37:57 -07:00
Sage Weil
67d3e510c2 osd: use deque instead of list for op wq to_process
Fewer allocations.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-13 16:29:22 -04:00
Sage Weil
d9a3bf043b osd: use global id instead of entity_inst_t as client identifier
entity_inst_t is huge; use uint64_t instead.

Note that this will currently confuse mds.0 vs osd.0, but we plan
to switch the mds to use mds.$gid soon anyway so that will go
away.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-13 16:28:10 -04:00
David Zafman
143111a112 osd: CLEANUP: Change getattr_maybe_cache() to reflect how it is being used
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 13:16:12 -07:00
David Zafman
50e08b0a5d test: Add a removal test for erasure code read
Test feature: http://tracker.ceph.com/issues/14513

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 13:15:52 -07:00
Yuri Weinstein
796ca1235d Merge pull request #17641 from amitkumar50/cov-messages-12
messages: Initialization of variable beat

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-09-13 12:57:45 -07:00
J. Eric Ivancich
d72b10fdfc Merge commit '5ff127736bc10d657ccdee9aabf179bbfa81d5c0' into wip-pull-updated-dmclock
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-09-13 15:52:13 -04:00
J. Eric Ivancich
5ff127736b Squashed 'src/dmclock/' changes from 3408cb8f3c..a9e777f08f
a9e777f08f Merge pull request #35 from joscollin/wip-warning-dmclock
45c63a5b10 Merge pull request #36 from ceph/wip-fix-functional
40a9fe55a6 With upgrade to gcc 7.1.1 need to include <functional> in additional header files.
a2dd155f08 src/dmclock_server.h: silence warning from -Wmaybe-uninitialized
0d6c7c6544 Merge pull request #24 from bspark8/wip_online_client_info_f
a5e8cdea31 Add unit test for update_client_info, dynamic_cli_info_f.
1faf3524c6 Modify each client's QoS parameter applied time from client_rec creation time to each request's tagging creation time.

git-subtree-dir: src/dmclock
git-subtree-split: a9e777f08f288fe6834599afdeefcd9c7d31e6c1
2017-09-13 15:48:19 -04:00
Douglas Fuller
b059cb6290 qa/cephfs: Fix error in test_filtered_df
ceph df accounts for pool size, so there is no need to do it in the test.

Fixes: http://tracker.ceph.com/issues/21381
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-09-13 14:02:24 -04:00
Yuri Weinstein
1985345c9a Merge pull request #10460 from weiqiaomiao/wqm-wip-rgw-dellc
rgw: add tenant to shard_id in RGWDeleteLC::execute()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2017-09-13 08:30:03 -07:00
Yuri Weinstein
06b8556f17 Merge pull request #17061 from shashalu/fix-rgw-lc
rgw: fix lc process only schdule the first item of lc objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: fang.yuxiang <fang.yuxiang@eisoo.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2017-09-13 08:28:24 -07:00
Casey Bodley
72b08e8f96 qa/smoke: add rgw crypto config for s3tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 11:22:28 -04:00
Yuri Weinstein
b7e9473e74 Merge pull request #17631 from joke-lee/RGW-CORS-Access-Control-Allow-Origin-HEADER
rgw: return 'Access-Control-Allow-Origin' header when the set bucket versioning through XMLHttpRequest

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 08:20:38 -07:00
Yuri Weinstein
e680ed47fc Merge pull request #17632 from joke-lee/RGW-CORS-Access-Control-Allow-Origin-HEADER-WEBSITE
rgw: return 'Access-Control-Allow-Origin' header when the set and delete bucket website through XMLHttpRequest

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 08:20:17 -07:00