Commit Graph

78339 Commits

Author SHA1 Message Date
Sage Weil
1587f87b6b Merge pull request #17610 from liewegas/wip-bluestore-fm-mas
os/bluestore: set bitmap freelist resolution to min_alloc_size

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Varada Kari <varada.kari@gmail.com>
2017-09-14 09:18:41 -05:00
Kefu Chai
eb8bfd1955 Merge pull request #17617 from liewegas/wip-drop-disk-log
mon/PGMap: drop DISK LOG column

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-14 21:42:56 +08:00
Yao Zongyou
69129b73a5 doc: change boolean option default value from zero to false.
Signed-off-by: Yao Zongyou yaozongyou@vip.qq.com
2017-09-14 21:37:04 +08:00
Sage Weil
c1bb53aaba Merge pull request #17628 from majianpeng/bluestore-narrow-deferred-lock
os/bluestore/BlueStore: narrow deferred_lock in _deferred_submit_unlock.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-14 08:30:17 -05:00
songweibin
9ccb710aa4 doc,os,osdc: drop and modify some comments
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2017-09-14 21:27:38 +08:00
Xie Xingguo
9b71d98ff6 Merge pull request #17627 from majianpeng/bluefs-exce-debug-code
os/bluestore/BlueFS: Don't call debug related code under any condition.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-14 18:27:02 +08:00
Xie Xingguo
b4bb7ce2da Merge pull request #17371 from xiexingguo/wip-per-pool-full-control
mon, osd: per pool space-full flag support

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-14 18:26:12 +08:00
Xie Xingguo
78acfcf05b Merge pull request #17368 from xiexingguo/wip-nice-data
mon/PGMap: nice numbers for 'data' section of 'ceph df' command

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-14 18:25:24 +08:00
Kefu Chai
3bf2382376 pybind: remove unused get_ceph_version()
get_ceph_version() was get_version() before 7b04e7ff. and after that
change, the version number of the python extension is hardwired to
`__version__`. because pybind is a wrapper around the public APIs
exposed by librados, librbd and libcephfs, which are versioned
separately from ceph daemons and tools, it's reasonable not to use the
version number in ceph_ver.h. thus, it's safe to drop this function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-14 18:02:54 +08:00
kungf
c1dba46084 osd: get loadavg per cpu for scrub load threshold check
Signed-off-by: kungf <yang.wang@easystack.cn>
2017-09-14 17:00:23 +08:00
Kefu Chai
d803576002 ceph.in: execv using the same python
otherwise we could be using another python when respawing the ceph cli.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-14 11:33:48 +08:00
Patrick Donnelly
d929dae49b
Merge PR #17657 into master
* refs/remotes/upstream/pull/17657/head:
	mds: optimize MDCache::rejoin_scour_survivor_replicas()
	mds: fix MDSCacheObject::clear_replica_map
	mds: support limiting cache by memory
	common: refactor of lru
	mds: resolve unsigned coercion compiler warning
	common: use safer uint64_t for list size
	common: add bytes2str pretty print function
	mds: check if waiting is allocated before use
	mds: go back to compact_map for replicas
	mds: use mempool for cache objects
	mds: cleanup replica_map access
	common: add alloc_ptr smart pointer
	common: add warning on base class use of mempool
	common: use atomic uin64_t for counter

Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-09-13 20:08:51 -07:00
Patrick Donnelly
f818905e0f
Merge PR #17689 into master
* refs/remotes/upstream/pull/17689/head:
	ceph: fix "stopping mds can't export subtree"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-13 18:29:20 -07:00
Patrick Donnelly
d89653dc56
Merge PR #17545 into master
* refs/remotes/upstream/pull/17545/head:
	client: fix request reference leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2017-09-13 18:29:18 -07:00
vasukulkarni
f1ffcaed6d Merge pull request #17700 from cbodley/wip-qa-smoke-s3tests
qa/smoke: add rgw crypto config for s3tests
2017-09-13 17:44:24 -07:00
J. Eric Ivancich
360c8954aa osd: fix tests of mclock implementations to track opimizations for op wq
wip-osd-deq changes client identifiers from entity_inst_t to
uint64_t. Updates tests to do this as well.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-09-13 18:17:33 -04:00
Xuehan Xu
5531b6bb51 ceph_test_rados_api_tier: add ListSnap test
Signed-off-by: Xuehan Xu <xuxuehan@360.cn>
2017-09-13 17:49:54 -04:00
David Zafman
8805ef5342 osd: Only scan for omap corruption once
Before
 	state 2: Can have complete tables (some may be bad)
	state 3: Never had complete tables
After
 	state 2: Can have complete tables (some may be bad)
	state 3 with legacy: Can have complete tables (bad ones are cleared)
	state 3: Never had complete tables

Once OSDs boot with this change you can't downgrade to a previous release.
If someone does downgrade they could have unstable OSDs that hit assert(state.v < 3).
The following command run after shutting down the cluster but before downgrading
ceph packages would be a way to fix this.

ceph-osdomap-tool --omap-path ... --command resetv2

Fixes: http://tracker.ceph.com/issues/21328

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:25 -07:00
David Zafman
3e4d68640c tools: Add the ability to reset state to v2
Available for testing and user downgrade.

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:25 -07:00
David Zafman
3214882a95 tools: Show DB state information
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:25 -07:00
David Zafman
de43493990 tools: Add --backend option to ceph-osdomap-tool default to rocksdb
Fix hard-coded "leveldb" backend.  The command is broken in Luminous
now that "rocksdb" is the default.

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 14:06:24 -07:00
Patrick Donnelly
a73ad7676a
Merge PR #17608 into master
* refs/remotes/upstream/pull/17608/head:
	doc/cephfs/posix: put posix notes in perspective

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-13 13:37:57 -07:00
Sage Weil
67d3e510c2 osd: use deque instead of list for op wq to_process
Fewer allocations.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-13 16:29:22 -04:00
Sage Weil
d9a3bf043b osd: use global id instead of entity_inst_t as client identifier
entity_inst_t is huge; use uint64_t instead.

Note that this will currently confuse mds.0 vs osd.0, but we plan
to switch the mds to use mds.$gid soon anyway so that will go
away.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-13 16:28:10 -04:00
David Zafman
143111a112 osd: CLEANUP: Change getattr_maybe_cache() to reflect how it is being used
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 13:16:12 -07:00
David Zafman
50e08b0a5d test: Add a removal test for erasure code read
Test feature: http://tracker.ceph.com/issues/14513

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-13 13:15:52 -07:00
Yuri Weinstein
796ca1235d Merge pull request #17641 from amitkumar50/cov-messages-12
messages: Initialization of variable beat

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-09-13 12:57:45 -07:00
J. Eric Ivancich
d72b10fdfc Merge commit '5ff127736bc10d657ccdee9aabf179bbfa81d5c0' into wip-pull-updated-dmclock
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-09-13 15:52:13 -04:00
J. Eric Ivancich
5ff127736b Squashed 'src/dmclock/' changes from 3408cb8f3c..a9e777f08f
a9e777f08f Merge pull request #35 from joscollin/wip-warning-dmclock
45c63a5b10 Merge pull request #36 from ceph/wip-fix-functional
40a9fe55a6 With upgrade to gcc 7.1.1 need to include <functional> in additional header files.
a2dd155f08 src/dmclock_server.h: silence warning from -Wmaybe-uninitialized
0d6c7c6544 Merge pull request #24 from bspark8/wip_online_client_info_f
a5e8cdea31 Add unit test for update_client_info, dynamic_cli_info_f.
1faf3524c6 Modify each client's QoS parameter applied time from client_rec creation time to each request's tagging creation time.

git-subtree-dir: src/dmclock
git-subtree-split: a9e777f08f288fe6834599afdeefcd9c7d31e6c1
2017-09-13 15:48:19 -04:00
Douglas Fuller
b059cb6290 qa/cephfs: Fix error in test_filtered_df
ceph df accounts for pool size, so there is no need to do it in the test.

Fixes: http://tracker.ceph.com/issues/21381
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-09-13 14:02:24 -04:00
Yuri Weinstein
1985345c9a Merge pull request #10460 from weiqiaomiao/wqm-wip-rgw-dellc
rgw: add tenant to shard_id in RGWDeleteLC::execute()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2017-09-13 08:30:03 -07:00
Yuri Weinstein
06b8556f17 Merge pull request #17061 from shashalu/fix-rgw-lc
rgw: fix lc process only schdule the first item of lc objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: fang.yuxiang <fang.yuxiang@eisoo.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2017-09-13 08:28:24 -07:00
Casey Bodley
72b08e8f96 qa/smoke: add rgw crypto config for s3tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 11:22:28 -04:00
Yuri Weinstein
b7e9473e74 Merge pull request #17631 from joke-lee/RGW-CORS-Access-Control-Allow-Origin-HEADER
rgw: return 'Access-Control-Allow-Origin' header when the set bucket versioning through XMLHttpRequest

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 08:20:38 -07:00
Yuri Weinstein
e680ed47fc Merge pull request #17632 from joke-lee/RGW-CORS-Access-Control-Allow-Origin-HEADER-WEBSITE
rgw: return 'Access-Control-Allow-Origin' header when the set and delete bucket website through XMLHttpRequest

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 08:20:17 -07:00
Yuri Weinstein
ef188a6a15 Merge pull request #17663 from shashalu/fix-rgw-LCFilter
rgw: Fix closing tag for Prefix

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-09-13 08:19:33 -07:00
Sage Weil
e200f35849 os/bluestore: allocate entire write in one go
On the first pass through the writes, compress data and calculate a final
amount of space we need to allocate.  On the second pass, assign the
extents to blobs and queue the writes.

This allows us to do a single allocation for all blobs, which will lead
to less fragmentation and a much better write pattern.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-13 09:49:50 -05:00
Sage Weil
2496470df3 Merge pull request #17543 from iliul/fix-dup-global-section
vstart.sh: Remove duplicate global section

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-13 09:38:19 -05:00
iliul
549c910715 vstart.sh: Remove duplicate global section
Signed-off-by: iliul <liul124@chinaunicom.cn>
2017-09-13 14:04:17 +00:00
Yan, Zheng
98d86a0752 qa/cephfs: kill mount if it gets evicted by mds
otherwise, teardown() hange at umount

Fixes: http://tracker.ceph.com/issues/21275
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-09-13 21:30:51 +08:00
Sage Weil
28c1ce209d Merge pull request #17544 from joelin/wip-delete-forgot-mark
msg: remove the ),it's redundant.

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2017-09-13 08:19:26 -05:00
Xie Xingguo
0e604b112e Merge pull request #17515 from xiexingguo/wip-data-digest
osd/PrimaryLogPG: do not set data digest for bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-13 18:31:10 +08:00
Yan Jun
ccaba817dc test/config: regulate parameter order of ASSERT_EQ
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2017-09-13 18:15:33 +08:00
Yan Jun
3f075372d1 common: fix daemon abnormal exit at parsing invalid arguments
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2017-09-13 17:58:40 +08:00
Yan, Zheng
8433ced847 qa/cephfs: fix test_evict_client
executing mount_a.kill() twice, then executing mount_b.kill_cleanup()
twice do not make sense.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-09-13 16:17:42 +08:00
Yan, Zheng
d32a237604 mds: optimize MDCache::rejoin_scour_survivor_replicas()
avoid iterating dentries if dirfrag is non-auth

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-09-13 15:55:46 +08:00
Yan, Zheng
fd447405be mds: fix MDSCacheObject::clear_replica_map
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-09-13 15:54:45 +08:00
Nathan Cutler
07159477ee Merge pull request #17366 from smithfarm/wip-21176
build/ops: python-numpy-devel build dependency for SUSE

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Ewaznazari <Nazari@keyinternational.academy>
2017-09-13 09:05:01 +02:00
Jos Collin
c383f03b3e Merge pull request #17685 from ygtzf/HashIndex-clerical-error
os/filestore/HashIndex.h: fixed a typo in comment

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-09-13 05:29:35 +00:00
Haomai Wang
9bf9a4189a Merge pull request #17688 from TsaiJin/wip-fix-assert-bug
msg/asyc/rmda: fix the bug of assert when Infiniband::recv_msg receives disconnect message

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-09-12 22:25:26 -07:00