Commit Graph

71263 Commits

Author SHA1 Message Date
Kefu Chai
b3075c485c qa/suites: Revert "qa/suites: add mon-reweight-min-pgs-per-osd = 4"
This reverts commit ef7583d717.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 14:18:21 +08:00
Kefu Chai
e6a436bb27 qa/tasks/ceph_manager: be able to store options with service type
so we are able to change options for services other than mon while
thrashing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 14:18:21 +08:00
optimistyzy
057fe4b7c5 NVMEDevice: remove unnessary dpdk header file
Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-04-20 13:56:11 +08:00
Yu Shengzuo
e97f6eb081 doc: correct the package name in ceph/admin/build-doc
In document, it is called 'libxml2-devel', one of the dependent
package of CentOS/RHEL7. But the 'build-doc' check 'libxml-devel'.
And it id also called 'libxml2-devel' in software of yum repo .
So correct the checked package name in ceph/admin/build-doc.

Signed-off-by: Yu Shengzuo <yu.shengzuo@99cloud.net>
2017-04-20 05:14:29 +00:00
Yu Shengzuo
41c902e532 doc: update dependent packages links and remove a 'important' tip
Some links in document has been deprecated, such as:
http://rpmfind.net/linux/centos/7.0.1406/, So update it;
The 'fc21, fc24, fc26(newest)' rpm 'ditaa' can be used in
CentOS/RHEL7 to build doc. So remove the '...important' tip.

Signed-off-by: Yu Shengzuo <yu.shengzuo@99cloud.net>
2017-04-20 03:39:29 +00:00
Kefu Chai
ee653ba87c Merge pull request #14608 from tchaikov/wip-19594
qa/tasks: assert on pg status with a timeout

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-20 10:49:12 +08:00
Zhiqiang Wang
aab8562eae osd: support dumping historic slow ops
ceph daemon osd.x dump_historic_slow_ops

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
2017-04-20 10:26:07 +08:00
Zhiqiang Wang
47248f3554 common: support tracking slow op
Add slow op in op tracker history.

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
2017-04-20 10:22:42 +08:00
Kefu Chai
cc61bb878f mon/OSDMonitor: use inc.modified as the ctime for MOSDPGCreate
so the last_scrub_time of the newly created pg is consistent

Fixes: http://tracker.ceph.com/issues/19594
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 09:35:05 +08:00
Kefu Chai
960032e513 qa/tasks: update tests with helper to wait for pg-stats
and remove unused helpers

Fixes: http://tracker.ceph.com/issues/19594
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 09:35:05 +08:00
Kefu Chai
1207caf3a2 qa/tasks/ceph_manager: add a "wait_for_pg_stats()" decorator
and accompany it with two helpers to access the pg stats in a more
natural way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 09:35:04 +08:00
Greg Farnum
3a4d5ec81b osd: pglog: with config, don't assert in the presence of stale divergent_priors
Fixes: http://tracker.ceph.com/issues/17916

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-04-19 17:57:52 -07:00
Josh Durgin
535b2cff57 Merge pull request #14496 from jdurgin/wip-ec-overwrites-cleanup
osd: make ec overwrites ready to use

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 17:56:28 -07:00
Josh Durgin
cab2968e8f qa: use bluestore for (k)rbd ec overwrites tests
Only bluestore supports ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:49:11 -07:00
Josh Durgin
f52b9d19f2 qa: use bluestore for cephfs cephtool tests
This lets the tests enable ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:44 -07:00
Josh Durgin
4a7c058183 qa: use 4k stripe_width again for test_rados_tool.sh
The stripe_unit change altered the default stripe_width to depend on k.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:44 -07:00
Josh Durgin
a3e5cba3f1 qa: move ec overwrites tests to a bluestore-only subsuite
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
bad606605a doc: update ec docs for overwrite support
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
25e8e1a08e PendingReleaseNotes: note about ec overwrites
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
4315221179 osd,OSDMonitor: try to protect against ec overwrites with filestore
This isn't perfect, but it's better than nothing. Prevent enabling the
allow_ec_overwrites flag if any of a sample of pgs in the pool map to
osds using filestore. This mainly protects filestore-only clusters
from enabling it.

If a filestore osd is started later, warn in the cluster log when it
gets a pg with ec overwrites enabled.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
3ca750d41d test/osd/osd-scrub-repair.sh: add ec overwrites test cases
Move pool and profile creation into a single function, and
add a 'allow_overwrites' parameter for it so each ec test
can be paramterized by it.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
a219319137 qa/tasks/rados: test sparse reads with ec overwrites
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
6fba80c1fa osd, OSDMonitor, qa: mark ec overwrites non-experimental
Keep the pool flag around so we can distinguish between a pool that
should maintain hashes for each chunk, and a missing one is a bug, vs
an overwrites pool where we rely on bluestore checksums for detecting
corruption.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
930eae214c OSDMonitor: get stripe_width via stripe_unit in ec profile
With bluestore, making the smallest write match min_alloc_size avoids
write amplification. With EC pools this is the stripe unit, or
stripe_width / num_data_chunks. Rather than requiring people to divide
by k to get the smallest ec write, allow it to be specified directly
via stripe_unit. Store it in the ec profile so changing a monitor
config option isn't necessary to set it.

This is particularly important for ec overwrites since they allow random i/o
which should match bluestore's checksum granularity (aka min_alloc_size).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Sage Weil
5080affedc Merge pull request #14256 from dillaman/wip-19297
osd: new op for calculating an extent checksum

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 17:08:35 -05:00
Sage Weil
abd52cb941 Merge pull request #14414 from Adirl/is_ready
msg/async: Postpone bind if network stack is not ready

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-04-19 17:08:11 -05:00
Sage Weil
4e8dd17ad2 Merge pull request #14486 from dachary/wip-multiweight
crush: weight_set and id remapping

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 17:07:19 -05:00
Sage Weil
ee1bb01a54 Merge pull request #14556 from liewegas/wip-pgupmap
osd: pg-remap -> pg-upmap

Reviewed-by: David Zafman <dzafman@redhat.com>
2017-04-19 17:07:01 -05:00
Sage Weil
3ef3e399b8 mon/MgrMonitor: only propose if we updated
Otherwise we generate a new map on every beacon, even
though there is no change.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-04-19 17:04:53 -04:00
John Spray
b4bd579fdd Merge pull request #14554 from ukernel/wip-19630
mds: fix stray creation/removal notification

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-19 19:24:01 +01:00
Yehuda Sadeh
205a39c8d9 Merge pull request #14433 from cbodley/wip-rgw-multi-py
test/rgw: refactor test_multi.py for use in qa suite

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-04-19 10:22:40 -07:00
Sage Weil
9d463ed9d7 Merge pull request #14623 from liewegas/wip-bluestore-compression
qa/objectstore: test bluestore with aggressive compression
2017-04-19 12:01:00 -05:00
Matt Benjamin
1d73a5764b Merge pull request #14642 from linuxbox2/wip-rgw-poolcreate-erange
rgw: error more verbosely in RGWRados::create_pool
2017-04-19 11:52:52 -04:00
Matt Benjamin
8ea698425a rgw: error more verbosely in RGWRados::create_pool
Adds detail when pool creation fails with ERANGE, which can be
caused by pool settings.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-19 10:58:16 -04:00
Sage Weil
4d18f07b80 Merge pull request #14637 from ovh/bp-test-osd-dup-warn-on-low-limit
test/osd/osd-dup.sh: warn on low open file limit

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 09:46:28 -05:00
Jason Dillaman
0dcba41cba test: rbd master/slave notify test should test active features
Fixes: http://tracker.ceph.com/issues/19692
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-04-19 09:26:31 -04:00
Jason Dillaman
7ab93b7779 qa/suites: client-upgrade/jewel-client-x should test multiple features
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-04-19 09:12:04 -04:00
Kefu Chai
b5dd112ce2 mon/OSDMonitor: use lec in OSDBeacon to trim osdmap
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-19 20:01:41 +08:00
Jason Dillaman
ca70fe440a Merge pull request #14631 from idryomov/wip-krbd-data-pool-fix
qa: krbd_data_pool.sh: account for rbd_info metadata object

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-04-19 07:30:27 -04:00
Abhishek L
0eae8d37d0 Merge pull request #14385 from smithfarm/wip-release-script-strip-periods
tools: ceph-release-notes: strip trailing punctuation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-04-19 13:08:47 +02:00
Piotr Dałek
2dd7718409 test/osd/osd-dup.sh: warn on low open file limit
This test fails badly when open file limit is low. Increasing to around
1536 seems to be doing the trick, so warn the user with appropriate
message and try to proceed anyway.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-04-19 12:57:38 +02:00
Loic Dachary
0011127604 Merge pull request #14599 from smithfarm/wip-crushtool-mappings
doc: mention --show-mappings in crushtool manpage

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-04-19 11:42:37 +02:00
Tim Serong
4fe7a87076 yasm-wrapper: strip -E (stops ccache trashing source files)
When building with ccache (`./do_cmake.sh -DWITH_CCACHE=ON`), it seems
ccache invokes `yast-wrapper` more than once per source file, the first
time with '-E' immediately before the source file in the argument list.
'-E' is an option for gcc which tells it to stop after the preprocessing
stage.  With yasm however, -E means "redirect error messages to file",
which makes yasm thinks there's no input files, and writes "yasm: No input
files specified" *over* the source file.  Then, the second run of
`yast-wrapper` (attempting to actually compile) fails with:

~/src/common/crc32c_intel_fast_asm.S:1: error: instruction expected after label

Curiously, this doesn't seem to affect all asm files; for me it
trashes crc32c_intel_fast_asm.S, but various other asm files build fine
without being clobbered.  I'm uncertain whether this is an issue
peculiar to openSUSE Tumbleweed, to ccache 3.3.3, or something else,
but in any case, having `yast-wrapper` strip '-E' fixes it.

Signed-off-by: Tim Serong <tserong@suse.com>
2017-04-19 19:05:32 +10:00
Kefu Chai
f6768064f6 osd: piggyback lec in OSDBeacon message
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-19 15:05:12 +08:00
Kefu Chai
5f95c972a4 messages/MOSDBeacon: add last_epoch_clean support
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-19 15:05:12 +08:00
Kefu Chai
6b75ec3c92 osd/OSDMonitor: trim last_osd_report if the osd is marked up/down
and clear it at restart()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-19 15:05:03 +08:00
Jos Collin
b985d1e6a2 osd: renamed the new vector name in OSDMap::build_simple_crush_map_from_conf
Renamed the new vector name in OSDMap::build_simple_crush_map_from_conf, to avoid confusing it
with the sections vector outside the for loop and which is iterating.

Resolved merge conflicts.

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-04-19 08:23:10 +05:30
Kefu Chai
9d5fa6129d Merge pull request #14483 from wjwithagen/wip-wjw-freebsd-ceph-disk-list
src/ceph-disk/ceph_disk/main.py: Make 'ceph-disk list' work on FreeBSD

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-19 09:23:32 +08:00
Jos Collin
81c794a0ed osd: renamed the new vector name in OSDMap::build_simple_crush_map_from_conf
Renamed the new vector name in OSDMap::build_simple_crush_map_from_conf, to avoid confusing it with the sections vector outside the for loop and which is iterating.

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-04-19 06:45:39 +05:30
Casey Bodley
55f276ba53 test/rgw: rewrite test_multi.py in terms of rgw_multi
logging changes inspired by Abhishek Lekshmanan <abhishek@suse.com>

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-18 19:13:40 -04:00