Commit Graph

71078 Commits

Author SHA1 Message Date
Sage Weil
0486c63354 Merge pull request #14423 from tchaikov/wip-cmake-crc32-cleanup
cmake: should not compile crc32c_ppc.c on intel arch.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-04-20 15:13:40 -05:00
Sage Weil
adada74873 Merge pull request #14434 from ifed01/wip-bluestore-params
os/bluestore: make bluestore_max_blob_size parameter hdd/ssd case dependant

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-20 15:13:18 -05:00
Sage Weil
9ddd888382 Merge pull request #14448 from majianpeng/bluestore-kv-queued-lat
os/bluestore: Record l_bluestore_state_kv_queued_lat for sync_submit_…

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-20 15:12:58 -05:00
Sage Weil
d710e06a1b Merge pull request #14549 from tchaikov/wip-kill-clang-warnings
misc: kill clang warnings

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-04-20 15:12:36 -05:00
Sage Weil
a0a1e66474 Merge pull request #14557 from liewegas/wip-bluefs-balance-interval
os/bluestore: do not balance bluefs on every kv_sync_thread iteration

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-04-20 15:12:09 -05:00
Matt Benjamin
dcbc28fc46 Merge pull request #11796 from linuxbox2/wip-bucket-size-ck
rgw: add bucket size limit check to radosgw-admin
2017-04-20 13:42:03 -04:00
Sage Weil
ab9e8419ec Merge pull request #14690 from ceph/luminous
build/ops: add 12.0.1 release tag in master
2017-04-20 11:55:00 -05:00
Matt Benjamin
7bc144ce36 rgw: add bucket size limit check to radosgw-admin
The change adds a new list of all buckets x all users, with
fields for bucket name, tenant name, current num_objects,
current num_shards, current objects per shard, and the
corresponding fill_status--the latter consisting of 'OK',
'WARN <n>%', or 'OVER <n>%.'

The warning check is relative to two new tunables.  The threshold
max objects per shard is set as rgw_bucket_safe_max_objects_per_shard,
which defaults to 100K.  The value rgw_bucket_warning_threshold is
a percent of the current safe max at which to warn (defaults to
90% of full).

From review:

* fix indentation (rgw_admin)
* if user a user_id is provided, check only buckets for that user
* update shard warn pct to be pct-of-fill (not 100 - pct-of-fill)
* print only buckets near or over per-shard limit, if --warnings-only
* s/bucket limitcheck/bucket limit check */
* sanity shard limit should be 90, not 10 (because that changed)
* fixes for memleaks and other points found by cbodley

Fixes: http://tracker.ceph.com/issues/17925

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-20 11:22:50 -04:00
John Spray
f854e3769e Merge pull request #14447 from ukernel/wip-19450
mds: drop partial entry and adjust write_pos when opening PurgeQueue

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-20 15:55:43 +01:00
Nathan Cutler
210249a8d7 Merge pull request #14638 from dillaman/wip-19692
test: rbd master/slave notify test should test active features

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2017-04-20 14:12:24 +02:00
John Spray
9264e1c85d Merge pull request #14197 from jcsp/wip-19395-counter
mds: use same inode count in health check as in trim

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-04-20 09:40:37 +01:00
Kefu Chai
f6db83ab2f Merge pull request #14649 from hikycool/master
doc: update packages mentioned by build-doc and related doc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-20 14:45:29 +08:00
Haomai Wang
69904747ce Merge pull request #14650 from optimistyzy/420_1
NVMEDevice: remove unnessary dpdk header file

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-04-20 14:29:10 +08:00
optimistyzy
057fe4b7c5 NVMEDevice: remove unnessary dpdk header file
Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-04-20 13:56:11 +08:00
Yu Shengzuo
e97f6eb081 doc: correct the package name in ceph/admin/build-doc
In document, it is called 'libxml2-devel', one of the dependent
package of CentOS/RHEL7. But the 'build-doc' check 'libxml-devel'.
And it id also called 'libxml2-devel' in software of yum repo .
So correct the checked package name in ceph/admin/build-doc.

Signed-off-by: Yu Shengzuo <yu.shengzuo@99cloud.net>
2017-04-20 05:14:29 +00:00
Yu Shengzuo
41c902e532 doc: update dependent packages links and remove a 'important' tip
Some links in document has been deprecated, such as:
http://rpmfind.net/linux/centos/7.0.1406/, So update it;
The 'fc21, fc24, fc26(newest)' rpm 'ditaa' can be used in
CentOS/RHEL7 to build doc. So remove the '...important' tip.

Signed-off-by: Yu Shengzuo <yu.shengzuo@99cloud.net>
2017-04-20 03:39:29 +00:00
Kefu Chai
ee653ba87c Merge pull request #14608 from tchaikov/wip-19594
qa/tasks: assert on pg status with a timeout

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-20 10:49:12 +08:00
Kefu Chai
cc61bb878f mon/OSDMonitor: use inc.modified as the ctime for MOSDPGCreate
so the last_scrub_time of the newly created pg is consistent

Fixes: http://tracker.ceph.com/issues/19594
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 09:35:05 +08:00
Kefu Chai
960032e513 qa/tasks: update tests with helper to wait for pg-stats
and remove unused helpers

Fixes: http://tracker.ceph.com/issues/19594
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 09:35:05 +08:00
Kefu Chai
1207caf3a2 qa/tasks/ceph_manager: add a "wait_for_pg_stats()" decorator
and accompany it with two helpers to access the pg stats in a more
natural way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-20 09:35:04 +08:00
Josh Durgin
535b2cff57 Merge pull request #14496 from jdurgin/wip-ec-overwrites-cleanup
osd: make ec overwrites ready to use

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 17:56:28 -07:00
Josh Durgin
cab2968e8f qa: use bluestore for (k)rbd ec overwrites tests
Only bluestore supports ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:49:11 -07:00
Josh Durgin
f52b9d19f2 qa: use bluestore for cephfs cephtool tests
This lets the tests enable ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:44 -07:00
Josh Durgin
4a7c058183 qa: use 4k stripe_width again for test_rados_tool.sh
The stripe_unit change altered the default stripe_width to depend on k.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:44 -07:00
Josh Durgin
a3e5cba3f1 qa: move ec overwrites tests to a bluestore-only subsuite
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
bad606605a doc: update ec docs for overwrite support
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
25e8e1a08e PendingReleaseNotes: note about ec overwrites
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
4315221179 osd,OSDMonitor: try to protect against ec overwrites with filestore
This isn't perfect, but it's better than nothing. Prevent enabling the
allow_ec_overwrites flag if any of a sample of pgs in the pool map to
osds using filestore. This mainly protects filestore-only clusters
from enabling it.

If a filestore osd is started later, warn in the cluster log when it
gets a pg with ec overwrites enabled.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
3ca750d41d test/osd/osd-scrub-repair.sh: add ec overwrites test cases
Move pool and profile creation into a single function, and
add a 'allow_overwrites' parameter for it so each ec test
can be paramterized by it.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
a219319137 qa/tasks/rados: test sparse reads with ec overwrites
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
6fba80c1fa osd, OSDMonitor, qa: mark ec overwrites non-experimental
Keep the pool flag around so we can distinguish between a pool that
should maintain hashes for each chunk, and a missing one is a bug, vs
an overwrites pool where we rely on bluestore checksums for detecting
corruption.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Josh Durgin
930eae214c OSDMonitor: get stripe_width via stripe_unit in ec profile
With bluestore, making the smallest write match min_alloc_size avoids
write amplification. With EC pools this is the stripe unit, or
stripe_width / num_data_chunks. Rather than requiring people to divide
by k to get the smallest ec write, allow it to be specified directly
via stripe_unit. Store it in the ec profile so changing a monitor
config option isn't necessary to set it.

This is particularly important for ec overwrites since they allow random i/o
which should match bluestore's checksum granularity (aka min_alloc_size).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:43 -07:00
Sage Weil
5080affedc Merge pull request #14256 from dillaman/wip-19297
osd: new op for calculating an extent checksum

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 17:08:35 -05:00
Sage Weil
abd52cb941 Merge pull request #14414 from Adirl/is_ready
msg/async: Postpone bind if network stack is not ready

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-04-19 17:08:11 -05:00
Sage Weil
4e8dd17ad2 Merge pull request #14486 from dachary/wip-multiweight
crush: weight_set and id remapping

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 17:07:19 -05:00
Sage Weil
ee1bb01a54 Merge pull request #14556 from liewegas/wip-pgupmap
osd: pg-remap -> pg-upmap

Reviewed-by: David Zafman <dzafman@redhat.com>
2017-04-19 17:07:01 -05:00
John Spray
b4bd579fdd Merge pull request #14554 from ukernel/wip-19630
mds: fix stray creation/removal notification

Reviewed-by: John Spray <john.spray@redhat.com>
2017-04-19 19:24:01 +01:00
Yehuda Sadeh
205a39c8d9 Merge pull request #14433 from cbodley/wip-rgw-multi-py
test/rgw: refactor test_multi.py for use in qa suite

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-04-19 10:22:40 -07:00
Sage Weil
9d463ed9d7 Merge pull request #14623 from liewegas/wip-bluestore-compression
qa/objectstore: test bluestore with aggressive compression
2017-04-19 12:01:00 -05:00
Matt Benjamin
1d73a5764b Merge pull request #14642 from linuxbox2/wip-rgw-poolcreate-erange
rgw: error more verbosely in RGWRados::create_pool
2017-04-19 11:52:52 -04:00
Matt Benjamin
8ea698425a rgw: error more verbosely in RGWRados::create_pool
Adds detail when pool creation fails with ERANGE, which can be
caused by pool settings.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-04-19 10:58:16 -04:00
Sage Weil
4d18f07b80 Merge pull request #14637 from ovh/bp-test-osd-dup-warn-on-low-limit
test/osd/osd-dup.sh: warn on low open file limit

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-19 09:46:28 -05:00
Jason Dillaman
0dcba41cba test: rbd master/slave notify test should test active features
Fixes: http://tracker.ceph.com/issues/19692
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-04-19 09:26:31 -04:00
Jason Dillaman
7ab93b7779 qa/suites: client-upgrade/jewel-client-x should test multiple features
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-04-19 09:12:04 -04:00
Jason Dillaman
ca70fe440a Merge pull request #14631 from idryomov/wip-krbd-data-pool-fix
qa: krbd_data_pool.sh: account for rbd_info metadata object

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-04-19 07:30:27 -04:00
Abhishek L
0eae8d37d0 Merge pull request #14385 from smithfarm/wip-release-script-strip-periods
tools: ceph-release-notes: strip trailing punctuation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-04-19 13:08:47 +02:00
Piotr Dałek
2dd7718409 test/osd/osd-dup.sh: warn on low open file limit
This test fails badly when open file limit is low. Increasing to around
1536 seems to be doing the trick, so warn the user with appropriate
message and try to proceed anyway.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-04-19 12:57:38 +02:00
Loic Dachary
0011127604 Merge pull request #14599 from smithfarm/wip-crushtool-mappings
doc: mention --show-mappings in crushtool manpage

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-04-19 11:42:37 +02:00
Kefu Chai
9d5fa6129d Merge pull request #14483 from wjwithagen/wip-wjw-freebsd-ceph-disk-list
src/ceph-disk/ceph_disk/main.py: Make 'ceph-disk list' work on FreeBSD

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-19 09:23:32 +08:00
Casey Bodley
55f276ba53 test/rgw: rewrite test_multi.py in terms of rgw_multi
logging changes inspired by Abhishek Lekshmanan <abhishek@suse.com>

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-04-18 19:13:40 -04:00