Commit Graph

79920 Commits

Author SHA1 Message Date
Haomai Wang
f2eb981377 msg/async/AsyncConnection: state will be NONE if replacing by another one
Fixes: http://tracker.ceph.com/issues/21883
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 12:38:56 +08:00
Sage Weil
09c50e2fe4 osd/PG: on recovery done, requeue for backfill
We were keeping our existing recovery reservation slot (with a high
priority) and going straight to waiting for backfill reservations on
the peers.  This is a problem because the reserver thinks we're doing
high priority work when we're actually doing lower-priority backfill.

Fix by closing out our recovery reservation and going to the
WaitLocalBackfillReserved state, where we'll re-request backfill at the
appropriate priority.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 22:46:00 -05:00
Sage Weil
dde007749b osd/PG: move local_reserver recovery cancel to Recovering state transition
This is easier to follow than canceling the reservation in the next state.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 22:43:18 -05:00
Sage Weil
2463c6463d osd/PG: allow preemption of remote backfill reservations
If we have granted a remote backfill reservation, and a higher priority
request comes in, send a REVOKE message back to the primary and drop the
reservation (allowing the higher-priority reservation to be GRANTed).

We can only do this if the primary is running new code because it must
understand the REVOKE message.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 22:37:36 -05:00
Patrick Donnelly
2f7ae0e160
Merge PR #18372 into master
* refs/pull/18372/head:
	ptl-tool: Fix more readme examples

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-22 20:26:16 -07:00
Sage Weil
0e9dac1ae1 osd/PG: explicit TOOFULL verb for backfill cancellation
We were sending REJECT if the replica filled up, and the primary would set
the BACKFILL_TOOFULL state as a result.  Make it an explicit verb for
clarity.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 21:38:14 -05:00
Jason Dillaman
251658471e librbd: default to sparse-reads for any IO operation over 64K
Testing BlueStore against both HDDs and OSDs with fully allocated
and sparse-allocated objects shows a performance improvement with
sparse-read between 32K and 64K.

Fixes: http://tracker.ceph.com/issues/21849
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:57:26 -04:00
Jason Dillaman
dd54520040 librbd: templatize io::CopyupRequest and io::ObjectRequest
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:57:26 -04:00
Jason Dillaman
a878eca9d8 librbd: sync flush should re-use existing async flush logic
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:38:39 -04:00
Sage Weil
d4e4922958 messages/MBackfillReserve: rename CANCEL -> RELEASE
This way me match the terminology used by MRecoveryReserve.  It is also
a bit more suggestive of primary->replica action, whereas "cancel" could
mean replica canceling its grant.

Document the meaning in the headers to clarify meaning.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 14:19:06 -05:00
Sage Weil
4078102bdc osd/PG: respect primary's priority for remote recovery reservation
This now mirrors the backfill approach (e.g., RequestBackfillPrio).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 14:09:53 -05:00
Sage Weil
a78dd16770 messages/MRecoveryReserve: pass priority to replica
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-22 14:07:58 -05:00
Kefu Chai
6a35e37eb9 Merge pull request #18368 from kungf/scrub_week_day_bound
osd: add scrub week day constraint

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-22 03:33:14 +08:00
Sage Weil
edc411ee4c crush/CrushWrapper: add bucket_set_alg()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-21 12:59:16 -05:00
Sage Weil
7c387fd04a Merge pull request #18400 from dillaman/wip-21844
osdc/Objecter: skip sparse-read result decode if bufferlist is empty

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 12:46:57 -05:00
Sage Weil
b834df8823 Merge pull request #18418 from dillaman/wip-21860
common/common_init: disable ms subsystem log gathering for clients

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 12:46:14 -05:00
Sage Weil
953d785eb5 Merge pull request #18427 from dillaman/wip-21845
osdc/Objecter: delay initialization of hobject_t in _send_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-10-21 12:45:36 -05:00
Sage Weil
40f0bbc1c5 Merge pull request #18450 from liewegas/wip-osd-removing
osd: add numpg_removing metric

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 12:44:29 -05:00
Sage Weil
1e5fce39a4 Merge pull request #18452 from liewegas/wip-21880
ceph_test_objectstore: disable filestore_fiemap for tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 12:44:00 -05:00
Sage Weil
90e13e47a4 Merge pull request #18454 from liewegas/wip-21882
messages/MOSDMap: do compat reencode of crush map, too

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 12:43:20 -05:00
Sage Weil
95f5bdb752 Merge pull request #18457 from liewegas/wip-restful-health
qa/suites/rados/rest/mgr-restful: whitelist more health
2017-10-21 12:41:58 -05:00
xie xingguo
29fc326861 osd/PG: caluculate acting by comparing with oldest log entry of auth
In GetLog, we will request logs back to the min last_update over our
acting_backfill set, which will result in our log being extended
as far backwards as necessary to pick up any peers which can
be log recovered by auth_log_shard's log.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 14:20:52 +08:00
xie xingguo
5fc46a619a osd/PG: drop want_primary from calc_replicated[ec]_acting
This is generally not necessary because the first valid
osd in want set will be chosen as primary!

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 14:20:52 +08:00
Sage Weil
ddf3e9162d qa/suites/rados/rest/mgr-restful: whitelist more health
The test is fiddling with OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 23:14:36 -05:00
Sage Weil
9eaafc6627 messages/MOSDMap: do compat reencode of crush map, too
If we are reencoding an incremental, and it embeds a crush map, we need
to reencode that in a compatible way too.  This is especially true now
because we have the compat crush weight-sets.  Otherwise, a client may
learn the crush map through an incremental but not understand choose_args,
and not see/understand the alternate weight set.  It will send requests
to the wrong OSDs where they will just get dropped.

Fixes: http://tracker.ceph.com/issues/21882
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 22:36:57 -05:00
Kefu Chai
e6668b5646 Merge pull request #18406 from adamemerson/wip-insecticide
test,osd,kvstore_tool: silence warnings and prepare test buffer in the right way

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 10:42:04 +08:00
Kefu Chai
b40d3eb709 Merge pull request #18379 from liewegas/wip-kill-replay-interval
osd/osd_types: pg_pool_t: remove crash_replay_interval member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-10-21 10:39:40 +08:00
Kefu Chai
17e7e835b6 Merge pull request #18407 from adamemerson/wip-upgrade-zstd
zstd: Upgrade to v1.3.2

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 10:39:18 +08:00
Kefu Chai
4ec719bf9e Merge pull request #18357 from asomers/have_libaio
Build: create a proper WITH_BLUESTORE option

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-10-21 10:38:37 +08:00
Sage Weil
1cb521e08c Merge pull request #18354 from kungf/no_scurb_deadline
osd: make scrub no deadline when max interval is zero

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-20 17:22:43 -05:00
Yuri Weinstein
5f763bf137 Merge pull request #18396 from xiexingguo/wip-scrub-mismatch
osd/osd_types.cc: copy extents map too while making clone

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-20 14:47:14 -07:00
Sage Weil
b8ba9334d9 ceph_test_objectstore: disable filestore_fiemap for tests
This option was enabled in 87f33376d977962ab7438c46873ea9b6292390d1 but
causes ObjectStore/StoreTest.Synthetic/1 (filestore) to fail.  Revert that
bit for now until we fix fiemap properly.

See http://tracker.ceph.com/issues/21880

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 16:32:59 -05:00
Yuri Weinstein
9b18093892 Merge pull request #16958 from mikulely/fix-full-uri
rgw: fix opslog uri as per Amazon s3

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-10-20 13:49:29 -07:00
Yuri Weinstein
7ccd491953 Merge pull request #17546 from mikulely/rgw-cleanup-min
rgw: cleanup MIN macro with std::min

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2017-10-20 13:47:34 -07:00
Yuri Weinstein
8f2f0f0fc0 Merge pull request #17934 from joke-lee/rgw_list_objecst_wrong_result_with_marker_when_enable_versioning
rgw:fix list objects with marker wrong result  when bucket is enable versioning

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2017-10-20 13:46:09 -07:00
Ali Maredia
2e477f3c05 Merge pull request #17673 from mogeb/blkin
blkin: link against lttng-ust-fork
2017-10-20 15:52:55 -04:00
Alfredo Deza
1b671a8602 ceph-volume util.system use strings for comparison in py3
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 15:18:20 -04:00
Alfredo Deza
3ddcd3c2f1 f ceph-volume tests update changes from the help menu in lvm
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 15:10:08 -04:00
Kefu Chai
9970342d58 Merge pull request #18414 from majianpeng/pmem-work
libmpem: Revert "submodule: make libmpem as a submodule."

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 02:58:01 +08:00
Sage Weil
b50ee3e795 osd: add numpg_removing metric
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 13:34:20 -05:00
Ashish Singh
2a5755e076 doc: Added CHAP cofiguration instructions for iSCSI
Added details to specify the CHAP username and password while
discovering/login the iSCSI target.

Signed-off-by: Ashish Singh <assingh@redhat.com>
2017-10-20 23:52:14 +05:30
Alfredo Deza
a8282e9a58 ceph-volume lvm.common --journal-size doesn't accept a volume group
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 14:12:02 -04:00
Alfredo Deza
3653f10592 ceph-volume lvm.prepare filestore does not require a volume group
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 14:04:31 -04:00
Alfredo Deza
526c0aa81a ceph-volume lvm.common volume groups are no longer required
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 14:04:05 -04:00
Alfredo Deza
8788fe3721 doc/ceph-volume activate workflow mention tmpfs in osd dirs
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:11:40 -04:00
Alfredo Deza
288a312479 doc/ceph-volume update activation to include bluestore support
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:11:40 -04:00
Alfredo Deza
61ce8b4b84 doc/ceph-volume update create to indicate bluestore support
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:11:39 -04:00
Alfredo Deza
9f1a562739 doc/ceph-volume update prepare with bluestore workflow
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:08 -04:00
Alfredo Deza
c93603e576 doc/ceph-volume update filestore to indicate it is not the only/default objectstore
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:08 -04:00
Alfredo Deza
1dc24d1a70 doc/ceph-volume elaborate on env vars for interval and tries in activation
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:08 -04:00