Commit Graph

78806 Commits

Author SHA1 Message Date
Patrick Donnelly
534c30aca4
Merge PR #18041 into master
* refs/remotes/upstream/pull/18041/head:
	qa: relax cap expected value check
2017-09-30 17:43:56 -07:00
Patrick Donnelly
08384c0ad5
Merge PR #17990 into master
* refs/remotes/upstream/pull/17990/head:
	mds: open purge queue when transitioning out of standby replay
	mds: always re-probe mds log when standby replay done

Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-30 17:43:54 -07:00
Patrick Donnelly
e48683e35d
Merge PR #17935 into master
* refs/remotes/upstream/pull/17935/head:
	ceph_volume_client: fix setting caps for IDs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-30 17:43:52 -07:00
Marcus Watts
61f292ec3c Merge pull request #17958 from gaosibei/branch1
RGW: fix a bug about inconsistent unit of comparison
2017-09-30 18:30:02 -04:00
Xie Xingguo
cd6b9830d1 Merge pull request #15199 from xiexingguo/wip-object-logic-size
osd: fine-grained statistics of logical object space usage

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-30 14:50:32 +08:00
Kefu Chai
259b3c1ead Merge pull request #16884 from liewegas/wip-20919
osd/PrimaryLogPG: send requests to primary on cache miss

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-09-30 12:14:14 +08:00
Kefu Chai
3dfe209499 Merge pull request #17955 from asomers/bin_bash2
test: fix bash path in shebangs (part 2)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-30 12:13:35 +08:00
Kefu Chai
709c77b2b5 Merge pull request #17985 from dzafman/wip-21327
ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-30 12:12:25 +08:00
Kefu Chai
d877b0b07d Merge pull request #18005 from jcsp/wip-21577
tools: update monstore tool for fsmap, mgrmap

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-30 12:11:21 +08:00
Kefu Chai
583f62bd98 Merge pull request #18015 from tchaikov/wip-kill-warnings
osd,os/bluestore: kill clang analyzer warnings

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-30 12:10:49 +08:00
Kefu Chai
c76742b748 Merge pull request #18018 from tchaikov/wip-ceph-disk-cleanup
ceph-disk: more precise error message when a disk is specified

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-30 12:10:06 +08:00
Kefu Chai
48582cb00e Merge pull request #18034 from tchaikov/wip-options
common/options: pass by reference and use user-literals for size

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2017-09-30 12:07:15 +08:00
Jason Dillaman
ae1530bbfb Merge pull request #17971 from idryomov/wip-krbd-exclude-shared-298
qa/suites/krbd: exclude shared/298

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-09-29 16:07:59 -04:00
David Zafman
2f466f8b26 Merge pull request #17920 from dzafman/wip-21382
Erasure code recovery should send additional reads if necessary

Fixes: http://tracker.ceph.com/issues/21382

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-29 09:04:43 -07:00
Patrick Donnelly
b37c7f7db7
qa: relax cap expected value check
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-29 08:48:14 -07:00
Haomai Wang
fd704ab8b3 Merge pull request #18036 from ownedu/wip-fix-asyncrdma-coredump
msg/async/rdma: fix a potential coredump when handling tx_buffers under heavy RDMA

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-09-29 10:33:14 -05:00
xie xingguo
1e4263fa5d osd/PrimaryLogPG: allow trimmed read for OP_CHECKSUM
Normal reads support trimmed read length, and so shall checksums!

This fixes occasionally failure of rados/thrash test scripts, e.g.:
(1) create object using WriteOp with random generated length
(2) normal writes might accompany with TruncOp of randomized chosen truncate_size
(3) for ReadOp, pick a random 'length' to read, and do checksum simultaneously
    for the same range ([0, 'length']) to read too.

Since the 'length' for reading is randomized chosen, it might
exceed the current object size, and hence causing an EOVERFLOW error.

Related issues:
http://qa-proxy.ceph.com/teuthology/xxg-2017-09-22_01:52:47-rados-wip-object-logic-size-distro-basic-smithi/1657337
http://qa-proxy.ceph.com/teuthology/xxg-2017-09-22_14:14:19-rados-wip-object-logic-size-distro-basic-smithi/1658015

Fix the above problems by keeping pace with normal reads.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-29 21:16:19 +08:00
xie xingguo
421aee3aea osd: fine-grained statistics of logical object space usage
To test this change, we create an image of 5GB and do rbd bench write of 1GB:
./bin/rbd create bar -s 5120 && ./bin/rbd bench --io-type write --io-size 32K --io-total 100M --io-pattern rand  rbd/bar

Below is the test result.

Was:

GLOBAL:
    SIZE       AVAIL      RAW USED     %RAW USED
    30911M     27052M        3859M         12.49
POOLS:
    NAME                  ID     USED      %USED     MAX AVAIL     OBJECTS
    rbd                   0      3191M     26.36         8914M        1174
    cephfs_data_a         1          0         0         8914M           0
    cephfs_metadata_a     2       2246         0         8914M          21

Now:

GLOBAL:
    SIZE       AVAIL      RAW USED     %RAW USED
    30911M     27050M        3861M         12.49
POOLS:
    NAME                  ID     USED        %USED     MAX AVAIL     OBJECTS
    rbd                   0      101216k      1.10         8913M        1178
    cephfs_data_a         1            0         0         8913M           0
    cephfs_metadata_a     2          892         0         8913M          21

E.g., this change can make "osd pool set-quota max_bytes" work nicely.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-29 21:16:19 +08:00
Mykola Golub
7dc2f78215 Merge pull request #17979 from dillaman/wip-21559
rbd-mirror: forced promotion can result in incorrect status

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-09-29 15:20:12 +03:00
Sage Weil
faefd681b5 Merge pull request #18035 from tchaikov/wip-bit-cleanup
script/build-integration-branch: python3 compatible and pep8 clean
2017-09-29 07:03:16 -05:00
Ramana Raja
baf3b88800 ceph_volume_client: fix setting caps for IDs
... that have empty OSD and MDS caps. Don't add a ',' at the
start of OSD and MDS caps.

Fixes: http://tracker.ceph.com/issues/21501
Signed-off-by: Ramana Raja <rraja@redhat.com>
2017-09-29 17:06:05 +05:30
Joao Eduardo Luis
fd04ff81eb Merge pull request #17846 from jecluis/wip-21300
mon/MgrMonitor: read cmd descs if empty on update_from_paxos()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:25:56 +01:00
David Zafman
390d12f71a osd: For recovery get all possible shards to read on errors
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-28 23:31:25 -07:00
David Zafman
1235810c2a osd: Allow recovery to send additional reads
For now it doesn't include non-acting OSDs
Added test for this case

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-28 23:31:18 -07:00
David Zafman
f92aa6c824 test: Allow modified options to existing setup functions
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-28 23:31:18 -07:00
David Zafman
43e3206de2 test: Use feature to get last array element
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-28 23:31:18 -07:00
ownedu
f54a522ce1 Merge branch 'wip-fix-asyncrdma-coredump' of https://github.com/ownedu/ceph into wip-fix-asyncrdma-coredump 2017-09-29 14:00:29 +08:00
ownedu
24280897be msg/async/rdma: fix a potential coredump when handling tx_buffers under
heavy RDMA traffic, there are chances to access a current_chunk which can
be beyond the range of pre-allocated Tx buffer pool thus causes a coredump

Signed-off-by: Yan Lei <yongyou.yl@alibaba-inc.com>
2017-09-29 13:58:37 +08:00
ownedu
59c54caec0 Fix a potential coredump when handling tx_buffers under heacy RDMA
traffic, there are chances to access a current_chunk which can be beyond the
range of pre-allocated Tx buffer pool thus causes a coredump.

Signed-off-by: Yan Lei <yongyou.yl@alibaba-inc.com>
2017-09-29 13:50:23 +08:00
xie xingguo
f90bd4b957 common/interval_set: override subset_of for given range
E.g.:
subset_of([5~10,20~5], 0, 100)  -> [5~10,20~5]
subset_of([5~10,20~5], 5, 25)   -> [5~10,20~5]
subset_of([5~10,20~5], 1, 10)   -> [5~5]
subset_of([5~10,20~5], 8, 24)   -> [8~7, 20~4]

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-29 12:30:46 +08:00
Kefu Chai
4e8e311694 script/build-integration-branch: python3 compatible and pep8 clean
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 12:05:49 +08:00
Kefu Chai
6788ea0e9a common/options: use user-defined literals for sizes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:14:25 +08:00
Kefu Chai
d2364bde9b common/options: pass by reference
for better performance. and for better consistency, pass by
std::initializer_list.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:09:35 +08:00
Kefu Chai
61a61f9968 tools/rados: do not assign never read variable
actually "r" is always 0 in that branch. so it's a no-op.

this silences the clang analyzer warning of

	Value stored to 'r' is never read

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:01:53 +08:00
Kefu Chai
bd4db59197 os/bluestore: do not assign never read variable
shrink the lexical scope of "csum_order" and do not set it if it is
never read.

this silences the clang analyzer warning of:

	Value stored to 'csum_order' is never read

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:01:53 +08:00
Kefu Chai
1db7bf2662 os/bluestore: do not assign never read variable
"git" is not read after being reset.

this silences clang analyzer warning of:

	Value stored to 'gift' is never read

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:01:53 +08:00
Kefu Chai
a9c5e9dd0e os/filestore: do not set never read variable
do not set "omap_fsid_fd" to -1 after ::close() it.

also move it's definition to where it is use for the first time for
better readability.

this silences the clang analyzer warning of:

	Value stored to 'omap_fsid_fd' is never read

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:01:53 +08:00
Kefu Chai
1087350aac osd: fail if unable to update device class in crush
device-class is supported since luminous, so we should fail the init()
if device-class is not updated in crush.

this reverts 244db740: if the device class changes then the user should
either follow the instruction to "ceph osd crush rm-device-class" or fix
CrushWrapper::update_device_class().

this silences clang analyzer warning of:

	Value stored to 'r' is never read

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-29 11:01:53 +08:00
Patrick Donnelly
1aef50a1ed
Merge PR #17697 into master
* refs/remotes/upstream/pull/17697/head:
	pybind/ceph_volume_client: add get, put, and delete object interfaces
	pybind/ceph_volume_client: remove 'compat_version'
	pybind/ceph_volume_client: set the version

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-28 18:26:06 -07:00
Patrick Donnelly
cce07d7865
Merge PR #17604 into master
* refs/remotes/upstream/pull/17604/head:
	client:fix a bug in lseek

Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-09-28 18:26:04 -07:00
gaosibei
cb39f065d6 RGW: fix a bug about inconsistent unit of comparison
Fixes:http://tracker.ceph.com/issues/21590

Signed-off-by: gaosibei <gaosb@inspur.com>
2017-09-29 08:44:25 +08:00
Patrick Donnelly
1da4a5090a
Merge PR #16036 into HEAD
* refs/remotes/upstream/pull/16036/head:
	mds: improve cap min/max ratio descriptions
	mds: fix whitespace
	mds: cap client recall to min caps per client
	mds: fix conf types
	mds: fix whitespace
	doc/cephfs: add client min cache and max cache ratio describe
	mds: adding tunable features for caps_per_client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-09-28 17:00:39 -07:00
Patrick Donnelly
dcf97d1735
mds: improve cap min/max ratio descriptions
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-28 16:01:00 -07:00
Patrick Donnelly
ab69d68293
mds: fix whitespace
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-28 15:56:26 -07:00
Patrick Donnelly
538834171f
mds: cap client recall to min caps per client
Fixes: http://tracker.ceph.com/issues/21575

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-28 15:55:57 -07:00
Yuri Weinstein
a2d72791ef Merge pull request #17774 from amitkumar50/cov-misc-4
include,messages,rbd: Initialize counter,group_pool

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-09-28 15:53:57 -07:00
Yuri Weinstein
488c6e4e3f Merge pull request #17820 from zmedico/PGPool-update-optimize-with-subset-of
osd/PGPool::update: optimize with subset_of

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-28 15:53:29 -07:00
Yuri Weinstein
fc7ad29927 Merge pull request #17884 from kungf/stats_invalid_scrub
osd: make scrub right now when pg stats_invalid is true

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Amit Kumar amitkuma@redhat.com
2017-09-28 15:51:47 -07:00
Yuri Weinstein
b362626e99 Merge pull request #17984 from adamemerson/wip-lists-of-pointers-are-bad-in-options
options: Do not use linked lists of pointers!

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-28 15:50:05 -07:00
Yuri Weinstein
67828fcfbc Merge pull request #17987 from xiexingguo/wip-kick-recovery-for-degraded-obj
osd/PrimaryLogPG: kick off recovery on backoffing a degraded object

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-28 15:48:51 -07:00