Commit Graph

79932 Commits

Author SHA1 Message Date
Brad Hubbard
63ef97b3db osd: Move creation of 'master_set' to scrub_compare_maps
Moving it to scrub_compare_maps allows us to pass it to multiple
subsequently called functions.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-24 17:27:51 +10:00
Kefu Chai
fdc0183871 mon/OSDMonitor: more options in new-style
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-24 15:14:09 +08:00
Kefu Chai
ca1d9566d0 common/options: set max_background_jobs instead of max_background_compactions
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-24 15:14:09 +08:00
Ziye Yang
e5f3af028a bluestore/NVMEDevice: convert the legacy config opt
Change all the legacy config opt related with SPDK in NVMEDevice

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
2017-10-24 14:56:10 +08:00
Haomai Wang
9f5b36879c Merge pull request #18409 from liu-chunmei/fix_dpdk_assert
Fix two dpdk assert happened in dpdk library.

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-10-24 01:06:53 -05:00
Luo Kexue
351874587e doc: complete and update the subsystem logging level info table
Signed-off-by: Luo Kexue <luo.kexue@zte.com.cn>
2017-10-24 14:04:59 +08:00
chenliuzhong
36326dc710 ceph.spec.in,debian/rules: change aio-max-nr to 1048576
when osd is more than 14 in one host,it report error that aio is not enough.
As the default aio-max-nr is 65536, one OSD needs 4096 aios and other programs may use aios.
This patch change aio-max-nr to 1048576 when install ceph-osd rpm package and debian package

Signed-off-by: chenliuzhong <liuzhong.chen@easystack.cn>
2017-10-24 13:16:36 +08:00
Brad Hubbard
2e3af697a8 unittest_crypto: Don't exceed limit for getentropy
CryptoRandom::get_bytes calls getentropy directly if available and it
enforces a limit of 256 bytes.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-24 14:34:26 +10:00
Kefu Chai
bc139f02d2 Merge pull request #18460 from liewegas/wip-straw2-command
mon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-24 12:27:15 +08:00
xie xingguo
f82228c4af osd/osd_type.cc: dump extents map object_info_t
which is good for bug hunting and diagnosing.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-24 11:46:23 +08:00
xie xingguo
9da3f6bac1 osd/PrimaryLogPG: fix incorrect extents update on write_full
Two problems:
(1) write_full may cause object size to shrink, but
    current code logical won't be able to handle this properly.
(2) update stats in a simpler and consistent way.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-24 11:46:23 +08:00
Sage Weil
fed1938bbb Merge pull request #18428 from liewegas/wip-21878
os/bluestore/BlueFS: fix race with log flush during async log compaction

Reviewed-by: Varada Kari <varada.kari@gmail.com>
2017-10-23 22:30:05 -05:00
Sage Weil
a91064af10 Merge pull request #18470 from yuyuyu101/wip-tracker-lock
osd/OpRequest: reduce overhead when disabling tracking

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 22:29:41 -05:00
Sage Weil
557088e07d Merge pull request #18472 from yuyuyu101/wip-kill-asynccompressor
compressor: kill AsyncCompressor which is broken

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 22:29:19 -05:00
David Zafman
2b792c5077 osd: Handle objects on non-actingbackfill nodes during recovery
With this fix we can see more misplaced and no degraded when
doing recovery to a new up/acting set with node(s) containing
objects that are not longer part of up/acting.

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-10-23 20:20:44 -07:00
Sage Weil
d0732fc96f qa/cephfs: test ec data pool
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 21:11:24 -05:00
Ziye Yang
15113037ba bluestore/NVMEDevice: Minor fixes.
1 After SPDK initialization, release DPDK thread core affinity binding
2 When there are errors for polling, throw error. And if there
are no completed ones, sleep 5 us.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
2017-10-24 09:51:49 +08:00
xie xingguo
46bfe362ed osd/osd_type: kill log.approx_size()
We used approx_size() because std::list::size() used to be O(n) instead of O(1),
but it is constant now in C++11.

Also counting the precise number of current log entries here is much more
meaningful.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-24 08:37:17 +08:00
xie xingguo
94c7e97c6d osd/osd_type.h: kill pg_missing_set.have_old
which has no consumers, and removing the unused code should
always be the preferred option.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-24 08:37:17 +08:00
xie xingguo
348b319ad1 os/PG: choose replicated acting by last_update
Candidate which has a newer last_update should take precedence,
since it can be technically brought back to normal sooner than others.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-24 08:37:17 +08:00
David Zafman
3818e8ed2c osd: Adjust misplaced calculation
When there are more backfill ndoes then misplaced nodes the
object calculation would cause misplace to drop too quickly.
Assuming that all backfill nodes are at approximate the same
place, we compute backfill per node and only adjust by
the number of misplaced nodes.

Fixes: http://tracker.ceph.com/issues/21898

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-10-23 17:09:24 -07:00
Patrick Donnelly
a9baa8b33a
Merge PR #18255 into master
* refs/pull/18255/head:
	os/bluestore: handle compressed extents in blob unsharing checks

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-23 17:07:37 -07:00
Sage Weil
3a6255d206 build: remove PGMap.cc from libcommon
Not needed.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 17:01:11 -05:00
Matt Benjamin
6bef8e3bcc Merge pull request #17722 from amitkumar50/cov-rgw-6
rgw: Initialization of epoch,len
2017-10-23 17:00:37 -04:00
Yuri Weinstein
33191d23ad Merge pull request #17305 from theanalyst/rgw-lc-tagging
RGW: support for tagging in lifecycle policies

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
2017-10-23 13:48:40 -07:00
Yuri Weinstein
c3d84fa3bb Merge pull request #18291 from ZVampirEM77/wip-copy-errormsg
rgw: copy object add response error messages

Reviewed-by: Amit Kumar amitkuma@redhat.com
Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-10-23 13:46:01 -07:00
Sage Weil
fc0b547ba5 Merge pull request #18194 from ivancich/wip-mclock-repops-bitmap
osd: replace mclock subop opclass w/ rep_op opclass; combine duplicated code

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 14:33:57 -05:00
Sage Weil
3c3a53b538 Merge pull request #18487 from liewegas/wip-20557
/etc/sysconfig/ceph: remove jemalloc option

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-10-23 14:09:53 -05:00
Jason Dillaman
70dc22e03a rbd-mirror: strip environment/CLI overrides for remote cluster
Fixes: http://tracker.ceph.com/issues/21894
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 15:02:26 -04:00
Andrew Schoen
6e49b7dce6 Merge pull request #18448 from ceph/wip-bz1499840
ceph-volume lvm bluestore support

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2017-10-23 13:55:55 -05:00
Sage Weil
d7524e9c61 doc/release-notes: add jewel->kraken notes
Forgot to include this!

Also add a bit more warning around the 'crush location' option change.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 13:18:39 -05:00
Mykola Golub
1100a00d2d Merge pull request #18403 from dillaman/wip-librbd-flush
librbd: sync flush should re-use existing async flush logic

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-23 21:12:20 +03:00
Jason Dillaman
918df91213 rbd: bench should flush prior to starting a test
For the new read-based bench tests, flushing prior to the start of the test
will result in the exclusive lock being acquired and the object map being
utilized.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 13:22:50 -04:00
Sage Weil
19efccef24 /etc/sysconfig/ceph: remove jemalloc option
This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 12:05:39 -05:00
Sage Weil
d0213e3d37 osd/PG: _update_calc_stats: backfill targets do not always affect degraded
Our progress on a backfill target should only count toward degraded if
the target is needed in order for us to reach the target pool size.  If
we have remapped other complete copies, we can backfill and not be degraded
at all.  And we may be some combination of the two if there are multiple
backfill targets; use the target(s) with the most objects.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 10:05:08 -07:00
Sage Weil
a95d325bfa Merge pull request #18486 from liewegas/wip-20557
etc/default/ceph: remove jemalloc option

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-10-23 11:52:43 -05:00
Sage Weil
fba2055b4e etc/default/ceph: remove jemalloc option
This breaks when used with rocksdb, which is now the default.

See http://tracker.ceph.com/issues/20557

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 11:16:26 -05:00
Sage Weil
efff14c417 Merge pull request #18408 from liewegas/wip-buffer-mempool-abi
buffer: avoid changing bufferlist ABI by removing new _mempool field

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 10:52:42 -05:00
Kefu Chai
6c6db0271f Merge pull request #18356 from wido/docs-zabbix-mgr
doc: Updated documentation for Zabbix Mgr module

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-23 23:35:51 +08:00
Kefu Chai
ae054510c3 Merge pull request #18474 from tchaikov/wip-debian-fix-upgrade
debian: fix package relationships after 40caf6a6

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 23:30:22 +08:00
Alfredo Deza
9bb6cfd867 ceph-volume lvm.activate only prime-osd-dir when directory is empty
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-23 09:44:27 -04:00
J. Eric Ivancich
729cb600ac osd: replace mclock subop opclass w/ rep_op opclass; combine duplicated code
The OSD subops were replaced with replication ops (i.e., MSG_OSD_REPOP
and MSG_OSD_REPOPREPLY) in jewel. mClock incorrectly used them as an
op class rather than the newer replication ops. This fixes that and
places eight ops into the repop class -- MSG_OSD_REPOP,
MSG_OSD_REPOPREPLY, MSG_OSD_PG_UPDATE_LOG_MISSING,
MSG_OSD_PG_UPDATE_LOG_MISSING_REPLY, MSG_OSD_EC_WRITE,
MSG_OSD_EC_WRITE_REPLY, MSG_OSD_EC_READ, and MSG_OSD_EC_READ_REPLY.

This alo removes dupliated code related to op classes in
mClockOpClassQueue and mClockClientQueue and combines into
mClockOpClassSupport.

The three configuration options
osd_op_queue_mclock_osd_subop_{res,wgt,lim} have also been replaced
with osd_op_queue_mclock_osd_rep_op_{res,wgt,lim}.

This also removes OpQueueItem::OpQueueable::op_type_t::osd_subop since
it's not a category core ceph recognizes and is instead it's part of
an mclock view of ops.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-10-23 09:25:32 -04:00
Alfredo Deza
c9b5f352a8 ceph-volume lvm.activate safeguard against auto detect objectstore flag not being set
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-23 09:20:26 -04:00
Jason Dillaman
d8dacf8261 Merge pull request #18423 from ashishkumsingh/wip-doc-fix-iscsi-chap-config
doc: Added CHAP configuration instructions for iSCSI

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-10-23 08:37:43 -04:00
Sage Weil
4dcef73369 mon/OSDMonitor: add 'osd crush set-all-straw-buckets-to-straw2'
This is a common and recommended step that users should take once they
have clients that support straw2 (hammer!).  Let's make it easy.

This may result in a small amount of data movement, but usually it is
very little.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 07:34:20 -05:00
Kefu Chai
ed988fc660 debian: fix package relationships after d3ac8d18
d3ac8d18 moves ceph-client-debug from ceph-test to ceph-base without
updating the package relationships between the two involved packages.
which results in:

dpkg: error processing archive /var/cache/apt/archives/ceph-test_12.2.1-241-g43e027b-1trusty_amd64.deb (--unpack):
 trying to overwrite '/usr/bin/ceph-client-debug', which is also in package ceph-base 10.2.10-14-gcbaddae-1trusty
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-23 20:18:15 +08:00
Kefu Chai
50dad805d9 debian: fix package relationships after 40caf6a6
we have issues when running upgrade tests:

dpkg: error processing archive /var/cache/apt/archives/ceph-osd_13.0.0-2201-g6cc0b41-1trusty_amd64.deb (--unpack):
trying to overwrite '/usr/bin/ceph-osdomap-tool', which is also in package ceph-test 10.2.10-14-gcbaddae-1trusty

in 40caf6a6, we moves some tools from ceph-test out into ceph-osd,
ceph-mon and ceph-base respectively. but didn't update the relationships
between these packages accordingly. this causes the upgrade failure.

see https://www.debian.org/doc/debian-policy/#document-ch-relationships
for more details on "Breaks" and "Conflicts".

the reason why the package version to be replaced/conflicted is 12.2.2
is that: i assume that this change will be backported to luminous, and
the next release of it will be 12.2.2 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-23 20:18:14 +08:00
Liao Weizhong
7d0360a42b
osd/OSD.cc: use option 'osd_scrub_cost' instead
Signed-off-by: Liao Weizhong <liaoweizhong@unionpay.com>
2017-10-23 19:43:20 +08:00
Jianpeng Ma
164c9a762d msg/async: batch handle msg_iovlen.
Also remove redundan code "msg.msg_iovlen = 0".

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-23 19:24:08 +08:00
Kefu Chai
e21cd99855 doc: update manual-deployment with latest code
rbd pool is not created by default any more.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-23 19:18:39 +08:00