Commit Graph

79770 Commits

Author SHA1 Message Date
Mykola Golub
1100a00d2d Merge pull request #18403 from dillaman/wip-librbd-flush
librbd: sync flush should re-use existing async flush logic

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-23 21:12:20 +03:00
Jason Dillaman
918df91213 rbd: bench should flush prior to starting a test
For the new read-based bench tests, flushing prior to the start of the test
will result in the exclusive lock being acquired and the object map being
utilized.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 13:22:50 -04:00
Sage Weil
19efccef24 /etc/sysconfig/ceph: remove jemalloc option
This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 12:05:39 -05:00
Sage Weil
d0213e3d37 osd/PG: _update_calc_stats: backfill targets do not always affect degraded
Our progress on a backfill target should only count toward degraded if
the target is needed in order for us to reach the target pool size.  If
we have remapped other complete copies, we can backfill and not be degraded
at all.  And we may be some combination of the two if there are multiple
backfill targets; use the target(s) with the most objects.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 10:05:08 -07:00
Sage Weil
a95d325bfa Merge pull request #18486 from liewegas/wip-20557
etc/default/ceph: remove jemalloc option

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-10-23 11:52:43 -05:00
Sage Weil
fba2055b4e etc/default/ceph: remove jemalloc option
This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 11:16:26 -05:00
Sage Weil
efff14c417 Merge pull request #18408 from liewegas/wip-buffer-mempool-abi
buffer: avoid changing bufferlist ABI by removing new _mempool field

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 10:52:42 -05:00
Kefu Chai
6c6db0271f Merge pull request #18356 from wido/docs-zabbix-mgr
doc: Updated documentation for Zabbix Mgr module

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 23:35:51 +08:00
Kefu Chai
ae054510c3 Merge pull request #18474 from tchaikov/wip-debian-fix-upgrade
debian: fix package relationships after 40caf6a6

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 23:30:22 +08:00
Alfredo Deza
9bb6cfd867 ceph-volume lvm.activate only prime-osd-dir when directory is empty
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-23 09:44:27 -04:00
J. Eric Ivancich
729cb600ac osd: replace mclock subop opclass w/ rep_op opclass; combine duplicated code
The OSD subops were replaced with replication ops (i.e., MSG_OSD_REPOP
and MSG_OSD_REPOPREPLY) in jewel. mClock incorrectly used them as an
op class rather than the newer replication ops. This fixes that and
places eight ops into the repop class -- MSG_OSD_REPOP,
MSG_OSD_REPOPREPLY, MSG_OSD_PG_UPDATE_LOG_MISSING,
MSG_OSD_PG_UPDATE_LOG_MISSING_REPLY, MSG_OSD_EC_WRITE,
MSG_OSD_EC_WRITE_REPLY, MSG_OSD_EC_READ, and MSG_OSD_EC_READ_REPLY.

This alo removes dupliated code related to op classes in
mClockOpClassQueue and mClockClientQueue and combines into
mClockOpClassSupport.

The three configuration options
osd_op_queue_mclock_osd_subop_{res,wgt,lim} have also been replaced
with osd_op_queue_mclock_osd_rep_op_{res,wgt,lim}.

This also removes OpQueueItem::OpQueueable::op_type_t::osd_subop since
it's not a category core ceph recognizes and is instead it's part of
an mclock view of ops.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-10-23 09:25:32 -04:00
Alfredo Deza
c9b5f352a8 ceph-volume lvm.activate safeguard against auto detect objectstore flag not being set
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-23 09:20:26 -04:00
Jason Dillaman
d8dacf8261 Merge pull request #18423 from ashishkumsingh/wip-doc-fix-iscsi-chap-config
doc: Added CHAP configuration instructions for iSCSI

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 08:37:43 -04:00
Sage Weil
4dcef73369 mon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'
This is a common and recommended step that users should take once they
have clients that support straw2 (hammer!).  Let's make it easy.

This may result in a small amount of data movement, but usually it is
very little.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 07:34:20 -05:00
Kefu Chai
ed988fc660 debian: fix package relationships after d3ac8d18
d3ac8d18 moves ceph-client-debug from ceph-test to ceph-base without
updating the package relationships between the two involved packages.
which results in:

dpkg: error processing archive /var/cache/apt/archives/ceph-test_12.2.1-241-g43e027b-1trusty_amd64.deb (--unpack):
 trying to overwrite '/usr/bin/ceph-client-debug', which is also in package ceph-base 10.2.10-14-gcbaddae-1trusty
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-23 20:18:15 +08:00
Kefu Chai
50dad805d9 debian: fix package relationships after 40caf6a6
we have issues when running upgrade tests:

dpkg: error processing archive /var/cache/apt/archives/ceph-osd_13.0.0-2201-g6cc0b41-1trusty_amd64.deb (--unpack):
trying to overwrite '/usr/bin/ceph-osdomap-tool', which is also in package ceph-test 10.2.10-14-gcbaddae-1trusty

in 40caf6a6, we moves some tools from ceph-test out into ceph-osd,
ceph-mon and ceph-base respectively. but didn't update the relationships
between these packages accordingly. this causes the upgrade failure.

see https://www.debian.org/doc/debian-policy/#document-ch-relationships
for more details on "Breaks" and "Conflicts".

the reason why the package version to be replaced/conflicted is 12.2.2
is that: i assume that this change will be backported to luminous, and
the next release of it will be 12.2.2 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-23 20:18:14 +08:00
Liao Weizhong
7d0360a42b
osd/OSD.cc: use option 'osd_scrub_cost' instead
Signed-off-by: Liao Weizhong <liaoweizhong@unionpay.com>
2017-10-23 19:43:20 +08:00
Jianpeng Ma
164c9a762d msg/async: batch handle msg_iovlen.
Also remove redundan code "msg.msg_iovlen = 0".

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-23 19:24:08 +08:00
Kefu Chai
e21cd99855 doc: update manual-deployment with latest code
rbd pool is not created by default any more.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-23 19:18:39 +08:00
Jason Dillaman
8be1d7d776 Merge pull request #18478 from trociny/wip-valgrid-trim-unittest
test/librbd: memory leak in recently added test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 06:51:58 -04:00
Kefu Chai
8622fdddf2 Merge pull request #15811 from x-ion-de/wip-20309
doc: Update manual deployment

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 18:42:46 +08:00
Mykola Golub
e68fb7409c test/librbd: memory leak in recently added test
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-23 12:34:19 +03:00
Haomai Wang
0ce27b9f6b compressor: kill AsyncCompressor which is broken
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 16:55:23 +08:00
Haomai Wang
bfdf438790 os/ObjectStore: fix get_data_alignment return -1 causing problem in msg header
sizeof(data_off) in ceph_msg_header is uint16_t, so -1 will be 65535. When peer
side receive MOSDRepOp message, it will allocate extra 1 byte buffer to store
nonexisted buffer.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 16:49:22 +08:00
Haomai Wang
7ba4d5324e msg/simple/Pipe: remove legacy feature case handle
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 14:33:46 +08:00
Zhi Zhang
1630f4bbef ceph_context: re-expand admin_socket metavariables in child process
Reset admin_socket raw value if it is defined in conffile.
Just in case it used metavarirables (e.g, $pid) which could be expanded
again to the correct value in child process.

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
2017-10-23 14:20:22 +08:00
Haomai Wang
d105ce6708 osd/OpRequest: don't mark_event when disable tracker
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:54:07 +08:00
Haomai Wang
17a192ba5c common/TrackedOp: make tracking_enabled atomic without lock protect
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:19:06 +08:00
Haomai Wang
83f7e392a1 msg/async/AsyncConnection: remove legacy feature case handle
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 13:08:44 +08:00
Haomai Wang
f2eb981377 msg/async/AsyncConnection: state will be NONE if replacing by another one
Fixes: http://tracker.ceph.com/issues/21883
Signed-off-by: Haomai Wang <haomai@xsky.com>
2017-10-23 12:38:56 +08:00
Patrick Donnelly
2f7ae0e160
Merge PR #18372 into master
* refs/pull/18372/head:
	ptl-tool: Fix more readme examples

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-22 20:26:16 -07:00
Jason Dillaman
251658471e librbd: default to sparse-reads for any IO operation over 64K
Testing BlueStore against both HDDs and OSDs with fully allocated
and sparse-allocated objects shows a performance improvement with
sparse-read between 32K and 64K.

Fixes: http://tracker.ceph.com/issues/21849
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:57:26 -04:00
Jason Dillaman
dd54520040 librbd: templatize io::CopyupRequest and io::ObjectRequest
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:57:26 -04:00
Jason Dillaman
a878eca9d8 librbd: sync flush should re-use existing async flush logic
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-22 20:38:39 -04:00
Kefu Chai
6a35e37eb9 Merge pull request #18368 from kungf/scrub_week_day_bound
osd: add scrub week day constraint

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-22 03:33:14 +08:00
Sage Weil
edc411ee4c crush/CrushWrapper: add bucket_set_alg()
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-21 12:59:16 -05:00
Sage Weil
7c387fd04a Merge pull request #18400 from dillaman/wip-21844
osdc/Objecter: skip sparse-read result decode if bufferlist is empty

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 12:46:57 -05:00
Sage Weil
b834df8823 Merge pull request #18418 from dillaman/wip-21860
common/common_init: disable ms subsystem log gathering for clients

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 12:46:14 -05:00
Sage Weil
953d785eb5 Merge pull request #18427 from dillaman/wip-21845
osdc/Objecter: delay initialization of hobject_t in _send_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-10-21 12:45:36 -05:00
Sage Weil
40f0bbc1c5 Merge pull request #18450 from liewegas/wip-osd-removing
osd: add numpg_removing metric

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-21 12:44:29 -05:00
Sage Weil
1e5fce39a4 Merge pull request #18452 from liewegas/wip-21880
ceph_test_objectstore: disable filestore_fiemap for tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 12:44:00 -05:00
Sage Weil
90e13e47a4 Merge pull request #18454 from liewegas/wip-21882
messages/MOSDMap: do compat reencode of crush map, too

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 12:43:20 -05:00
Sage Weil
95f5bdb752 Merge pull request #18457 from liewegas/wip-restful-health
qa/suites/rados/rest/mgr-restful: whitelist more health
2017-10-21 12:41:58 -05:00
Sage Weil
ddf3e9162d qa/suites/rados/rest/mgr-restful: whitelist more health
The test is fiddling with OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 23:14:36 -05:00
Sage Weil
9eaafc6627 messages/MOSDMap: do compat reencode of crush map, too
If we are reencoding an incremental, and it embeds a crush map, we need
to reencode that in a compatible way too.  This is especially true now
because we have the compat crush weight-sets.  Otherwise, a client may
learn the crush map through an incremental but not understand choose_args,
and not see/understand the alternate weight set.  It will send requests
to the wrong OSDs where they will just get dropped.

Fixes: http://tracker.ceph.com/issues/21882
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 22:36:57 -05:00
Kefu Chai
e6668b5646 Merge pull request #18406 from adamemerson/wip-insecticide
test,osd,kvstore_tool: silence warnings and prepare test buffer in the right way

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-21 10:42:04 +08:00
Kefu Chai
b40d3eb709 Merge pull request #18379 from liewegas/wip-kill-replay-interval
osd/osd_types: pg_pool_t: remove crash_replay_interval member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-10-21 10:39:40 +08:00
Kefu Chai
17e7e835b6 Merge pull request #18407 from adamemerson/wip-upgrade-zstd
zstd: Upgrade to v1.3.2

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-21 10:39:18 +08:00
Kefu Chai
4ec719bf9e Merge pull request #18357 from asomers/have_libaio
Build: create a proper WITH_BLUESTORE option

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-10-21 10:38:37 +08:00
Sage Weil
1cb521e08c Merge pull request #18354 from kungf/no_scurb_deadline
osd: make scrub no deadline when max interval is zero

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-20 17:22:43 -05:00