Commit Graph

79932 Commits

Author SHA1 Message Date
Jason Dillaman
8be1d7d776 Merge pull request #18478 from trociny/wip-valgrid-trim-unittest
test/librbd: memory leak in recently added test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 06:51:58 -04:00
Kefu Chai
8622fdddf2 Merge pull request #15811 from x-ion-de/wip-20309
doc: Update manual deployment

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 18:42:46 +08:00
Mykola Golub
e68fb7409c test/librbd: memory leak in recently added test
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-23 12:34:19 +03:00
Haomai Wang
0ce27b9f6b compressor: kill AsyncCompressor which is broken
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 16:55:23 +08:00
Haomai Wang
bfdf438790 os/ObjectStore: fix get_data_alignment return -1 causing problem in msg header
sizeof(data_off) in ceph_msg_header is uint16_t, so -1 will be 65535. When peer
side receive MOSDRepOp message, it will allocate extra 1 byte buffer to store
nonexisted buffer.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 16:49:22 +08:00
Haomai Wang
7ba4d5324e msg/simple/Pipe: remove legacy feature case handle
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 14:33:46 +08:00
Zhi Zhang
1630f4bbef ceph_context: re-expand admin_socket metavariables in child process
Reset admin_socket raw value if it is defined in conffile.
Just in case it used metavarirables (e.g, $pid) which could be expanded
again to the correct value in child process.

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
2017-10-23 14:20:22 +08:00
Haomai Wang
d105ce6708 osd/OpRequest: don't mark_event when disable tracker
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:54:07 +08:00
Haomai Wang
a37dd14ec8 iosdc/Objecter: using coarse_mono instead
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:50:11 +08:00
Haomai Wang
7f5eff7b73 osd/ReplicatedPG: add omap write bytes to delta_stats
under rgw case, it will consume lots of omap read/write but we can't see the
statistics.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:29:15 +08:00
Haomai Wang
17a192ba5c common/TrackedOp: make tracking_enabled atomic without lock protect
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:19:06 +08:00
Haomai Wang
83f7e392a1 msg/async/AsyncConnection: remove legacy feature case handle
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:08:44 +08:00
Haomai Wang
f2eb981377 msg/async/AsyncConnection: state will be NONE if replacing by another one
Fixes: http://tracker.ceph.com/issues/21883
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 12:38:56 +08:00
Sage Weil
09c50e2fe4 osd/PG: on recovery done, requeue for backfill
We were keeping our existing recovery reservation slot (with a high
priority) and going straight to waiting for backfill reservations on
the peers.  This is a problem because the reserver thinks we're doing
high priority work when we're actually doing lower-priority backfill.

Fix by closing out our recovery reservation and going to the
WaitLocalBackfillReserved state, where we'll re-request backfill at the
appropriate priority.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 22:46:00 -05:00
Sage Weil
dde007749b osd/PG: move local_reserver recovery cancel to Recovering state transition
This is easier to follow than canceling the reservation in the next state.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 22:43:18 -05:00
Sage Weil
2463c6463d osd/PG: allow preemption of remote backfill reservations
If we have granted a remote backfill reservation, and a higher priority
request comes in, send a REVOKE message back to the primary and drop the
reservation (allowing the higher-priority reservation to be GRANTed).

We can only do this if the primary is running new code because it must
understand the REVOKE message.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 22:37:36 -05:00
Patrick Donnelly
2f7ae0e160
Merge PR #18372 into master
* refs/pull/18372/head:
	ptl-tool: Fix more readme examples

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-22 20:26:16 -07:00
Sage Weil
0e9dac1ae1 osd/PG: explicit TOOFULL verb for backfill cancellation
We were sending REJECT if the replica filled up, and the primary would set
the BACKFILL_TOOFULL state as a result.  Make it an explicit verb for
clarity.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 21:38:14 -05:00
Jason Dillaman
251658471e librbd: default to sparse-reads for any IO operation over 64K
Testing BlueStore against both HDDs and OSDs with fully allocated
and sparse-allocated objects shows a performance improvement with
sparse-read between 32K and 64K.

Fixes: http://tracker.ceph.com/issues/21849
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:57:26 -04:00
Jason Dillaman
dd54520040 librbd: templatize io::CopyupRequest and io::ObjectRequest
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:57:26 -04:00
Jason Dillaman
a878eca9d8 librbd: sync flush should re-use existing async flush logic
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:38:39 -04:00
Sage Weil
d4e4922958 messages/MBackfillReserve: rename CANCEL -> RELEASE
This way me match the terminology used by MRecoveryReserve.  It is also
a bit more suggestive of primary->replica action, whereas "cancel" could
mean replica canceling its grant.

Document the meaning in the headers to clarify meaning.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 14:19:06 -05:00
Sage Weil
4078102bdc osd/PG: respect primary's priority for remote recovery reservation
This now mirrors the backfill approach (e.g., RequestBackfillPrio).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 14:09:53 -05:00
Sage Weil
a78dd16770 messages/MRecoveryReserve: pass priority to replica
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 14:07:58 -05:00
Kefu Chai
6a35e37eb9 Merge pull request #18368 from kungf/scrub_week_day_bound
osd: add scrub week day constraint

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-22 03:33:14 +08:00
Sage Weil
edc411ee4c crush/CrushWrapper: add bucket_set_alg()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-21 12:59:16 -05:00
Sage Weil
7c387fd04a Merge pull request #18400 from dillaman/wip-21844
osdc/Objecter: skip sparse-read result decode if bufferlist is empty

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 12:46:57 -05:00
Sage Weil
b834df8823 Merge pull request #18418 from dillaman/wip-21860
common/common_init: disable ms subsystem log gathering for clients

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 12:46:14 -05:00
Sage Weil
953d785eb5 Merge pull request #18427 from dillaman/wip-21845
osdc/Objecter: delay initialization of hobject_t in _send_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-10-21 12:45:36 -05:00
Sage Weil
40f0bbc1c5 Merge pull request #18450 from liewegas/wip-osd-removing
osd: add numpg_removing metric

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 12:44:29 -05:00
Sage Weil
1e5fce39a4 Merge pull request #18452 from liewegas/wip-21880
ceph_test_objectstore: disable filestore_fiemap for tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 12:44:00 -05:00
Sage Weil
90e13e47a4 Merge pull request #18454 from liewegas/wip-21882
messages/MOSDMap: do compat reencode of crush map, too

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 12:43:20 -05:00
Sage Weil
95f5bdb752 Merge pull request #18457 from liewegas/wip-restful-health
qa/suites/rados/rest/mgr-restful: whitelist more health
2017-10-21 12:41:58 -05:00
xie xingguo
29fc326861 osd/PG: caluculate acting by comparing with oldest log entry of auth
In GetLog, we will request logs back to the min last_update over our
acting_backfill set, which will result in our log being extended
as far backwards as necessary to pick up any peers which can
be log recovered by auth_log_shard's log.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 14:20:52 +08:00
xie xingguo
5fc46a619a osd/PG: drop want_primary from calc_replicated[ec]_acting
This is generally not necessary because the first valid
osd in want set will be chosen as primary!

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 14:20:52 +08:00
Sage Weil
ddf3e9162d qa/suites/rados/rest/mgr-restful: whitelist more health
The test is fiddling with OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 23:14:36 -05:00
Sage Weil
9eaafc6627 messages/MOSDMap: do compat reencode of crush map, too
If we are reencoding an incremental, and it embeds a crush map, we need
to reencode that in a compatible way too.  This is especially true now
because we have the compat crush weight-sets.  Otherwise, a client may
learn the crush map through an incremental but not understand choose_args,
and not see/understand the alternate weight set.  It will send requests
to the wrong OSDs where they will just get dropped.

Fixes: http://tracker.ceph.com/issues/21882
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 22:36:57 -05:00
Kefu Chai
e6668b5646 Merge pull request #18406 from adamemerson/wip-insecticide
test,osd,kvstore_tool: silence warnings and prepare test buffer in the right way

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 10:42:04 +08:00
Kefu Chai
b40d3eb709 Merge pull request #18379 from liewegas/wip-kill-replay-interval
osd/osd_types: pg_pool_t: remove crash_replay_interval member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-10-21 10:39:40 +08:00
Kefu Chai
17e7e835b6 Merge pull request #18407 from adamemerson/wip-upgrade-zstd
zstd: Upgrade to v1.3.2

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 10:39:18 +08:00
Kefu Chai
4ec719bf9e Merge pull request #18357 from asomers/have_libaio
Build: create a proper WITH_BLUESTORE option

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-10-21 10:38:37 +08:00
Sage Weil
1cb521e08c Merge pull request #18354 from kungf/no_scurb_deadline
osd: make scrub no deadline when max interval is zero

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-20 17:22:43 -05:00
Yuri Weinstein
5f763bf137 Merge pull request #18396 from xiexingguo/wip-scrub-mismatch
osd/osd_types.cc: copy extents map too while making clone

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-20 14:47:14 -07:00
Sage Weil
b8ba9334d9 ceph_test_objectstore: disable filestore_fiemap for tests
This option was enabled in 87f33376d9 but
causes ObjectStore/StoreTest.Synthetic/1 (filestore) to fail.  Revert that
bit for now until we fix fiemap properly.

See http://tracker.ceph.com/issues/21880

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 16:32:59 -05:00
Yuri Weinstein
9b18093892 Merge pull request #16958 from mikulely/fix-full-uri
rgw: fix opslog uri as per Amazon s3

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-10-20 13:49:29 -07:00
Yuri Weinstein
7ccd491953 Merge pull request #17546 from mikulely/rgw-cleanup-min
rgw: cleanup MIN macro with std::min

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2017-10-20 13:47:34 -07:00
Yuri Weinstein
8f2f0f0fc0 Merge pull request #17934 from joke-lee/rgw_list_objecst_wrong_result_with_marker_when_enable_versioning
rgw:fix list objects with marker wrong result  when bucket is enable versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2017-10-20 13:46:09 -07:00
Ali Maredia
2e477f3c05 Merge pull request #17673 from mogeb/blkin
blkin: link against lttng-ust-fork
2017-10-20 15:52:55 -04:00
Alfredo Deza
1b671a8602 ceph-volume util.system use strings for comparison in py3
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 15:18:20 -04:00
Alfredo Deza
3ddcd3c2f1 f ceph-volume tests update changes from the help menu in lvm
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 15:10:08 -04:00