Commit Graph

80128 Commits

Author SHA1 Message Date
Patrick Donnelly
58d7d91516
Merge PR #18537 into master
* refs/pull/18537/head:
	mds: set PRIO_USEFUL on num_sessions counter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-26 19:39:04 -07:00
lu.shasha
f61a371438 common: strict_strtol INT_MAX and INT_MIN is valid
Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
2017-10-27 10:14:14 +08:00
Neha Ojha
9cf94276cf qa/suites/rados/perf: add workloads
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-10-26 18:10:49 -07:00
Matt Benjamin
f9d5528a19 rgw_log, rgw_file: account for new required envvars
rgw_log_op() expects more server and request variables from the
request environment.  Fix rgw_lib to set more that it can
usefully, but also make rgw_log_op() check for the existence
of all keys it will use.

Fixes: http://tracker.ceph.com/issues/21942

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-10-26 20:38:35 -04:00
Matt Benjamin
b1f528d35a rgw_file: set s->obj_size from bytes_written
Required to store correct final accounted_size of objects in
RGWWriteWriteRequest::write_finish.

Fixes: http://tracker.ceph.com/issues/21940

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-10-26 19:23:09 -04:00
Sage Weil
3f6e0b6e80 Merge pull request #18524 from liewegas/wip-backfill-rops
osd/PG: fix recovery op leak

Reviewed-by: David Zafman <dzafman@redhat.com>
2017-10-26 17:48:25 -05:00
Sage Weil
7740af808c Merge pull request #18552 from liewegas/wip-mds-cluster-down
qa/workunits/rest/test.py: stop trying to test obsolte cluster_up/down

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-26 17:47:51 -05:00
Yehuda Sadeh
a7efb9d84f Merge pull request #18489 from cbodley/wip-21895
rgw: fix extra_data_len handling in PutObj filters

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-10-26 14:16:46 -07:00
Alfredo Deza
634b5caaed ceph-volume lvm.activate remove links on activate before priming
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-26 15:09:18 -04:00
Sage Weil
de8dc42d42 ceph-bluestore-tool: prime-osd-dir: update symlinks instead of bailing
If the symlink points to the right location, do nothing.  If it doesn't,
replace it.  If it's not a symlink, bail with EEXIST.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-26 13:51:40 -05:00
vasukulkarni
b2a1fd1972 Merge pull request #18451 from zmc/wip-cephmetrics
Add cephmetrics suite
2017-10-26 11:31:19 -07:00
Matt Benjamin
664ad1a16c Merge pull request #18198 from cbodley/wip-21735
rgw: fix error handling in ListBucketIndexesCR
2017-10-26 13:41:19 -04:00
Ilya Dryomov
48784f3f32 qa/tasks/rbd: run all xfstests runs to completion
rbd.xfstests task allows spawning xfstests runs on multiple nodes.
Don't unwind task contexts if one of the runs fails -- let the other
runs finish.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2017-10-26 19:30:09 +02:00
Zack Cerza
41f55068cd Add cephmetrics suite
Signed-off-by: Zack Cerza <zack@redhat.com>
2017-10-26 10:55:30 -06:00
Jason Dillaman
b003ff3200 test/librbd: added update_features RPC message to test_notify
Fixes: http://tracker.ceph.com/issues/21936
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-26 12:50:43 -04:00
Sage Weil
810a138133 qa/workunits/rest/test.py: stop trying to test obsolete 'mds *' commands
These were recently removed.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-26 11:22:20 -05:00
John Spray
1fbd3b7b9f Merge pull request #18511 from smithfarm/wip-rpm-recommend-influxdb
build/ops: rpm: recommend python-influxdb with ceph-mgr

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2017-10-26 17:23:34 +02:00
John Spray
15e8b1904c Merge pull request #18182 from nrdmn/dashboard
mgr/dashboard: improve error handling

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-26 17:17:37 +02:00
Mykola Golub
0cb485ecb3 librbd: don't read metadata twice on image open
After adding get_metadata to RefreshRequest it has become redundant
in OpenRequest.

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-26 17:55:11 +03:00
Sage Weil
c337b8739d Merge pull request #18533 from tchaikov/wip-qa-cephtool
qa: do not "ceph fs get cephfs" w/o a cephfs fs

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2017-10-26 09:19:11 -05:00
Sage Weil
8afd4ec647 osd/PG: handle remote backfill recovation while waiting for other targets
If we have multiple targets, we may still be waiting on them when we get
a revocation.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-26 08:42:34 -05:00
Luo Kexue
c5041726f3 os/bluestore/BlueFS: Reduce unnecessary operations in collect_metadata
Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>
2017-10-26 20:48:42 +08:00
Alfredo Deza
fb36087bc3 ceph-volume lvm.activate ceph-bluestore-tool needs --cluster too
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-26 07:48:24 -04:00
Mykola Golub
fb48989eeb Merge pull request #18545 from batrick/pr18393-revert
Revert "ceph_context: re-expand admin_socket metavariables in child process"

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-26 10:28:58 +03:00
Sage Weil
028e15d62f Merge pull request #18496 from liewegas/wip-pgmap
build: remove PGMap.cc from libcommon

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-25 21:53:07 -05:00
Sage Weil
efd1a77147 osd/PG: make scan recovery op cancellation match up reliably
Previously, there was only one time we would end up in this region of code:
when the backfill was rejected by the peer.  Previously that was apparently
reliably when we had an outstanding SCAN request, because we would
unconditionally cancle the MAX recovery op and clear waiting_on_backfill.

See 624aaf2a4ea9950153a89ff921e2adce683a6f51 for when this code appeared.

Now we have several similar paths, and we don't always have an outstanding
scan call (I don't think!).  Regardless, move most these three cases into
a common helper and make the finish_recovery_op completion conditional
on whether there is an outstanding SCAN.  This fixes a leak of a recovery
op when we defer while a scan is outstanding (this bug was recently
introduced by e708410542b0a52fbb29e14b76f49c94adbc0a59 and then
duplicated by 2463c6463d1ed38a2e15a0960ed1530a47851489).

Note that there is still one other time we register MAX ops: when we are
finishing backfill.  There, we start one per target.  But we will always
get back our reply and process it in the normal way (that old commit
did not change the timing for these).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-25 21:52:12 -05:00
Sage Weil
ebb4093c2c osd/PG: make recovering_oids a multiset
For multiple backfill targets, we start MAX multiple times.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-25 21:50:12 -05:00
Sage Weil
f1afb1684c Merge pull request #18527 from xiexingguo/wip-hunt-scrub-mismatch
osd/PrimaryLogPG: several oi.extents fixes

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-25 21:49:45 -05:00
Sage Weil
b3bb61ede6 Merge pull request #18531 from majianpeng/msg-async
msg/async misc cleanup 

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-10-25 21:49:27 -05:00
Jianpeng Ma
e564518350 msg/async: record correctly l_msgr_send_bytes if "rc == 0".
If rc == 0, it mean all data of outcoming_bl whether old or new data.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-26 18:08:54 +08:00
Sage Weil
68f38a0544 osd/osd_types: fix object_stat_sum_t decode
Broken by 71bf04775bef90c9291bd825da626bc9de6f9ec1

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-25 21:39:23 -05:00
Patrick Donnelly
2bba5d8e0f
Merge PR #18192 into master
* refs/pull/18192/head:
	qa/cephfs: test ec data pool
	qa/suites/fs/basic_functional/clusters: more osds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-25 17:05:38 -07:00
Alfredo Deza
2b7ff497fe ceph-volume lvm.activate consume cluster_name from lvm metadata
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 18:57:28 -04:00
Alfredo Deza
6d38ad987e ceph-volume lvm.prepare persist cluster_name on lvm metadata
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 18:57:03 -04:00
Patrick Donnelly
1e571befcf
Revert "ceph_context: re-expand admin_socket metavariables in child process"
This commit introduces a regression where the admin socket specified in
ceph.conf overrides command line arguments or environment variables.

This reverts commit 1630f4bbef3a3ff6efb56acbaa9b8786b945b1e6.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-25 13:29:42 -07:00
Sage Weil
92141dc122 Merge pull request #18485 from liewegas/wip-remote-res-preemption
osd: fix recovery reservation bugs, and implement remote reservation preemption

Reviewed-by: David Zafman <dzafman@redhat.com>
2017-10-25 14:47:44 -05:00
Alfredo Deza
fd616acd6a ceph-volume util.prepare bluestore db and wal paths are to devices, not osd dir paths
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 15:30:43 -04:00
Alfredo Deza
80698d2309 ceph-volume lvm.prepare bluestore wal and db links get created with mkfs
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 15:30:06 -04:00
Casey Bodley
ac7ffe6ef7 rgw: fix extra_data_len handling in PutObj filters
the RGWPutObj_Compress filter relies on a starting offset of 0 to ensure
that we only compress entire objects

for RGWRados::fetch_remote_obj(), we also fetch object metadata. the
replies come back with a 'Rgwx-Embedded-Metadata-Len' header, which
specifies how many bytes of object metadata are at the front of the
request body. when this is present, the offsets passed from
RGWRadosPutObj down to the RGWPutObjDataProcessor filters are offsets
into the http response body, rather than logical offsets into the object
data itself

this commit adds a transformation to RGWRadosPutObj so that only that
logical offset is visible to the RGWPutObjDataProcessor

Fixes: http://tracker.ceph.com/issues/21895

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-25 14:25:09 -04:00
Neha Ojha
973e21694f qa: add cbt repo parameter
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-10-25 10:22:43 -07:00
Jianpeng Ma
1a6817a507 msg/async: updating l_msgr_running_send_time if write_message failed.
Also for the most time write_message return zero. So avoid checking
"r < 0 or r > 0" every time, it should firstly check "r == 0".

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-26 00:50:29 +08:00
David Zafman
e31075fd1d Merge pull request #18528 from tchaikov/wip-osd-misplaced
osd/PG: fix calc of misplaced objects

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-25 09:36:04 -07:00
Jianpeng Ma
3350b35045 msg/async: only system can accept more data, continue send.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-26 00:34:45 +08:00
Kefu Chai
718496bf72 Merge pull request #18523 from tchaikov/wip-vstart-quoted
vstart.sh: should quote the parameters to get them quoted

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-26 00:18:40 +08:00
Jianpeng Ma
2371b149e0 msg/async: remove redundant condition.
From the code, if out_q[r].is_empty, it already remove.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-26 00:16:45 +08:00
Danny Al-Gaaf
fc2f9bdb1c librbd/test_librbd.cc: fix format qualifiers
Fix for:

[src/test/librbd/test_librbd.cc:2610]: (warning) %lld in format string
 (no. 1) requires 'long long' but the argument type is 'unsigned long long'.
[src/test/librbd/test_librbd.cc:2610]: (warning) %lld in format string
 (no. 2) requires 'long long' but the argument type is 'unsigned long long'.
[src/test/librbd/test_librbd.cc:2610]: (warning) %lld in format string
 (no. 3) requires 'long long' but the argument type is 'unsigned long long'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2017-10-25 18:14:05 +02:00
Danny Al-Gaaf
f250b9c32e librbd/test_internal.cc: fix format qualifiers to %lu
Fix for:

[src/test/librbd/test_internal.cc:1054]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1054]: (warning) %ld in format string
 (no. 2) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1063]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1073]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1114]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1121]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1129]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1196]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1196]: (warning) %ld in format string
 (no. 2) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1205]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.
[src/test/librbd/test_internal.cc:1215]: (warning) %ld in format string
 (no. 1) requires 'long' but the argument type is 'unsigned long'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2017-10-25 18:14:05 +02:00
Danny Al-Gaaf
e045962cc8 rgw_rados.cc: use compare() instead of find()
Fix for:

[src/rgw/rgw_rados.cc:3752]: (performance) Inefficient usage of
 string::find() in condition; string::compare() would be faster.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2017-10-25 18:14:05 +02:00
Danny Al-Gaaf
e3f9fa9cc5 rgw_rados.cc: fix format qualifiers
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2017-10-25 18:14:05 +02:00
Danny Al-Gaaf
ce0fa3564a osd/osd_types.cc: fix sscanf format qualifiers to %u
Fix for:

[src/osd/osd_types.cc:439]: (warning) %d in format string (no. 1) requires
 'int *' but the argument type is 'unsigned int *'.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2017-10-25 18:14:05 +02:00