Commit Graph

69873 Commits

Author SHA1 Message Date
wangzhengyong
02add5825c doc: add some undocumented options to rbd-nbd
Signed-off-by: wangzhengyong@cmss.chinamobile.com
2017-03-25 22:37:58 +08:00
Kefu Chai
6a771d1ef5 Merge pull request #14103 from tchaikov/wip-https-github
script: ceph-release-notes: use https instead of http

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-03-24 14:43:48 +08:00
Sage Weil
bb8f633f5d Merge pull request #14085 from wjwithagen/wip-wjw-bluestore-fixture
test/objectstore/store_test_fixture.cc: Exclude bluestore code if required.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 20:47:45 -05:00
Sage Weil
117a63277c Merge pull request #13931 from wangzhengyong/extent
os/bluestore: fix bug for calc extent_avg in reshard function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-03-23 20:47:12 -05:00
Sage Weil
8275fda6de Merge pull request #14073 from liewegas/wip-bluestore-nullptr
os/bluestore: avoid nullptr in bluestore_extent_ref_map_t::bound_encode

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 20:44:59 -05:00
Sage Weil
c820749a04 Merge pull request #13577 from yonghengdexin735/wip-zzz-openalloc
os/bluestore: fix bug in _open_alloc()

Reviewed-by: Varada Kari <varada.kari@sandisk.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 20:44:35 -05:00
Loic Dachary
709dc377c7 Merge pull request #14110 from dachary/wip-crush-cleanup
crush: builder: clean the arguments of crush_reweight* methods

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-03-23 21:48:00 +01:00
Sahid Orentino Ferdjaoui
2c31903355 crush: builder: clean the arguments of crush_reweight* methods
This commit is just a cleanup to make the arguments of the method
around crush_reweight all coherent.

Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>
2017-03-23 17:54:52 +01:00
Yuri Weinstein
18ca002958 Merge pull request #14050 from ovh/bp-dump-ops-by-duration
common/TrackedOp: allow dumping historic ops sorted by duration

Reviewed-by: Sage Weil <sage@redhat.com>
2017-03-23 08:47:55 -07:00
Yuri Weinstein
0620e81581 Merge pull request #14060 from LiumxNL/wip-170321
osd: combine unstable stats with info.stats when publish stats to osd

Reviewed-by: Sage Weil <sage@redhat.com>
2017-03-23 08:46:36 -07:00
Yuri Weinstein
c54eb54a2a Merge pull request #13293 from Liuchang0812/cleanup-coverity
test, osd: fix some coverity issues

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 08:45:58 -07:00
Casey Bodley
e7093c1e96 Merge pull request #14014 from Liuchang0812/wip-fix-seg-fault
rgw: fix memory leak in RGWGetObjLayout

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-03-23 09:54:47 -04:00
Sage Weil
1ed62076a3 os/bluestore: avoid nullptr in bluestore_extent_ref_map_t::bound_encode
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-23 08:21:39 -05:00
Haomai Wang
93de4c60e6 Merge pull request #14094 from optimistyzy/322
bluestore, NVMeDevice: use task' own lock for (random) read

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-03-23 19:23:34 +08:00
Kefu Chai
b9783f1047 script: ceph-release-notes: use https instead of http
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-03-23 19:13:41 +08:00
Kefu Chai
72097c73a3 Merge pull request #14004 from liewegas/wip-osd-full-failsafe
osd: fall back to failsafe threshold if osdmap doesn't set [near]full

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 16:09:34 +08:00
Kefu Chai
1a9eae1b23 Merge pull request #13903 from wjwithagen/wip-wjw-run-classes-sed
test: sed on FreeBSD requires "-i extension", so use gsed

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 16:08:22 +08:00
Kefu Chai
136c28f9dc Merge pull request #9940 from aclamk/common-recursive-mutex-fix
common: fix lockdep vs recursive mutexes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-23 16:04:52 +08:00
Ziye Yang
e359189d72 bluestore, NVMeDevice: use task' own lock for (random) read
The reason is that ioc may be reaped in _aio_thread function
with  the following statements:
for (auto &&it : registered_devices)
          it->reap_ioc();

So if we still use ioc's lock for (random) read, it will cause
core dump.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-03-23 10:31:23 +08:00
Loic Dachary
950abe1b40 Merge pull request #14080 from ceph/evelu-ceph-disk
ceph-disk: Reporting /sys directory in get_partition_dev()

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-03-22 19:43:37 +01:00
Kefu Chai
3ffa869af2 Merge pull request #13942 from xiexingguo/wip-cleanup-proc-repinfo
osd/PG: conditionally retry on receiving pg-notify when Primary is Incomplete

Reviewed-by: Sage Weil <sage@redhat.com>
2017-03-22 23:57:13 +08:00
Kefu Chai
5fe34baae1 Merge pull request #14061 from tchaikov/wip-19312
tests: ceph_test_rados_api_watch_notify: test timeout using rados_wat…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-03-22 23:56:27 +08:00
Casey Bodley
fac3d91584 Merge pull request #12449 from cbodley/wip-rgw-test-multi-vers-acl
test/rgw: add bucket acl and versioning tests to test_multi.py

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-03-22 11:46:33 -04:00
Kefu Chai
55a4cb8a20 Merge pull request #14059 from vumrao/wip-vumrao-19318
common/config_opts.h: Remove deprecated osd_compact_leveldb_on_mount option

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-03-22 22:43:41 +08:00
Mark Nelson
f2c312f0ad Merge pull request #14076 from liewegas/wip-bluestore-min-alloc-size
os/bluestore: default 16KB min_alloc_size on ssd
2017-03-22 09:12:41 -05:00
Willem Jan Withagen
8053afec2c test/objectstore/store_test_fixture.cc: Exclude bluestore code if required.
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-03-22 15:03:32 +01:00
Haomai Wang
6962b0dfc0 Merge pull request #14068 from optimistyzy/321_new
Bluestore, NVMEDevice: add the spdk core mask check

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-03-22 21:16:48 +08:00
Piotr Dałek
8b415e6aed TrackedOp: allow dumping historic ops sorted by duration
Currently dump_historic_ops dumps ops sorted by their initiation time,
which may not have any relation to how long it took, and sorting output
of that command by op duration is neither fast nor convenient.
New asok command ("dump_historic_ops_by_duration") outputs the same
op list, but ordered by their duration time (longest first).

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-03-22 13:32:04 +01:00
optimistyzy
ef8fceff21 Bluestore, NVMEDevice: add the spdk core mask check
This patch adds the spdk core mask check and also
set the master core for starting DPDK.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-03-22 20:18:19 +08:00
liuchang0812
384496dbd1 rgw/rgw_op: fix memory leak in RGWGetObjLayout
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2017-03-22 17:27:20 +08:00
Erwan Velu
413c9fcfbe ceph-disk: Reporting /sys directory in get_partition_dev()
When get_partition_dev() fails, it reports the following message :
    ceph_disk.main.Error: Error: partition 2 for /dev/sdb does not appear to exist
The code search for a directory inside the /sys/block/get_dev_name(os.path.realpath(dev)).

The issue here is the error message doesn't report that path when failing while it might be involved in.

This patch is about reporting where the code was looking at when trying to estimate if the partition was available.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2017-03-22 10:11:44 +01:00
Sage Weil
7412b19d11 os/bluestore: default 16KB min_alloc_size on ssd
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 21:27:23 -05:00
Orit Wasserman
68bc509413 Merge pull request #13963 from cbodley/wip-18725
rgw-admin: remove deprecated regionmap commands
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2017-03-21 23:44:22 +02:00
Sage Weil
66b42be685 Merge pull request #13888 from liewegas/wip-bluestore-dw
os/bluestore: fix deferred writes; improve flush

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-03-21 15:05:56 -05:00
Casey Bodley
5444c9d092 Merge pull request #13902 from Wilhelmshaven/rm_redundant_code
rgw: remove redundant codes in rgw_cache.h

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-03-21 15:43:48 -04:00
Sage Weil
def17606fc os/bluestore: handle zombie OpSequencers
It's possible for the Sequencer to go away while the OpSequencer still has
txcs in flight.  We were handling the case where the osr was on the
deferred_queue, but it may be off the deferred_queue but waiting for the
commit to happen, and we still need to wait for that.

Fix this by introducing a 'zombie' state for the osr, in which we keep the
osr in the osr_set.

Clean up the OpSequencer methods and a few other method names.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:31 -05:00
Sage Weil
d8fa788ca8 os/bluestore: clean up flush_all()
Add assertions if we fail to flush everything.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:31 -05:00
Sage Weil
9732b6c8e9 os/bluestore: move cached items around on collection split
We've been avoiding doing this for a while and it has finally caught up
with us: the SharedBlob may outlive the split due to deferred IO, and
a read on the child collection may load a competing Blob and SharedBlob
and read from the on-disk blocks that haven't been written yet.

Fix by preserving the one-SharedBlob-instance invariant by moving cache
items to the new Collection and cache shard like we should have from the
beginning.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
e4d547ede7 os/bluestore: simplify flush() wake-up condition
Clearer, and fewer wakeups.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
52c93f5b71 ceph_test_objectstore: set bluestore cache shards to 5
Better test coverage!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
d93d6d0968 unittest_bluestore_types: fix Collection using tests
We can't use a bare Collection since we get/put refs, the last put will
delete it, and the dtor asserts nref == 0 (no faking a ref and deliberately
leaking!).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
4de29d0f14 os/bluestore/KernelDevice: drop unused flush_lock
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
ed9f54bae7 os/bluestore: better debugging around collections
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
3ad789cef3 os/bluestore: nicer Onode dout prefix
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
e83fc55491 os/bluestore: flush_cache on umount, fsck finish, etc.
Otherwise cache items survive beyond umount into the next mount cycle!

Also, ensure that we flush_cache *before* clearing coll_map, as some cache
items have references back to the Collection.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
01ef844421 os/bluestore: take Collection ref from SharedBlob
These can survive as long as the txc, which can be longer than the
Collection.  Make sure we have a valid ref as both finish_write and
~SharedBlob use coll for the SharedBlobSet (and coll->store->cct for
debug).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
81e8682be1 os/bluestore: fix perfcounters for deferred io
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:30 -05:00
Sage Weil
f4d4c9c68a os/bluestore: remove dead _do_deferred_op code
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:29 -05:00
Sage Weil
3a3d9ad097 os/bluestore: make throttles tunable online
Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:29 -05:00
Sage Weil
3f9c216145 os/bluestore: prevent throttle deadlock due to deferred writes
Kick off deferred IOs if we pass the throttle midpoint or if we would
block during submission.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-03-21 13:56:29 -05:00