1
0
mirror of https://github.com/ceph/ceph synced 2025-01-20 01:51:34 +00:00
Commit Graph

59418 Commits

Author SHA1 Message Date
Mykola Golub
5577ada030 librbd: proper check for get_data_pool compatibility
Fixes: http://tracker.ceph.com/issues/17791
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-11-03 15:27:14 +02:00
Sage Weil
fac6335a1e Merge pull request from liewegas/wip-mempool
mempool: mempool infrastructure, bluestore changes to use it

Reviewed-by: Allen Samuels <allen.samuels@sandisk.com>
2016-11-02 12:50:23 -05:00
Sage Weil
9cecff8d18 mempool: use function static for pool table
The compiler/linker guarantee this is initialized before any invocation
of this function... even if it is by a ctor in another compilation unit
that is initialized by the mempool.cc compilation unit.

Suggested by Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>

Signed-off-by: Allen Samuels <allen.samuels@sandisk.com>
2016-11-02 13:48:49 -04:00
Sage Weil
ebf052762e mempool: do not dump unused pools
Notably, unittest_[12]. :)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
a54d21749d mempool: put pool namespaces within mempool::
e.g., mempool::bluestore_meta_other::list<...> ...

This avoids colliding with other names and types when the pool
name is something common (like "osd").

Signed-off-by: Sage Weil <sage@redhat.com>

# Conflicts:
#	src/include/mempool.h
2016-11-02 13:48:49 -04:00
Sage Weil
b15a377a7b mempool: simplify debug settings
There's no need for a per-pool debug flag; just use the global
debug_mode bool!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
62fa72c0e1 mempool: force per-type factories to register with the pool
We want these to get their debug flag adjusted when it is turned
on and off.

(In contrast, we accept that STL containers will only be debugged
if mempool debug was on when they are constructed.)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
82251cc51d include/encoding: parameterize stl containers encoders with allocator
This will let us encode/decode containers with custom allocators
(i.e., in mempools).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
bfeaed6412 mempool: simplify initialization
Assume that the array will be zeroed when the process starts, and
avoid the possibility that it will be zeroed *again* when the
ctors run for this link module (potentially clobbering values
that have been filled in by other ctors who ran earlier than
ours).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
79ee6eec6c include/types: parameterize operator<< helpers on comparator, allocator
This allows these templates to apply to containers with
non-default comparators or allocators.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
ca93284120 include/mempool: fix comment
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
049797641b mempool: dynamic index by type in pool_t
This avoids having to use statics for the pool_allocators and
guessing what intenral types the containers are going to need.
It'll be a bit slower in debug on on construction and destruction,
but who cares!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
55d42d0830 include/mempool: define additional pair factory for unordered_map
This fixes build on debian jessie, libstdc++-4.9-dev:amd64 4.9.2-10.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
a6eec64c95 mempool: pick shard on every allocation; move types to pool_t
If we have a static choice of shard for every *type*
(pool_allocator_base_t) then we will hammer that shard from all
threads and play cacheline ping-pong.  Instead, move the types
list to pool_t (there aren't that many anyway) and pick a shard
on every allocate/deallocate call.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:49 -04:00
Sage Weil
b174694493 mempool: make num_shards a power of 2
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
735be3662f include/mempool: rename containers -> types
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
1b51c05068 include/mempool: note about runtime complexity
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
34f434be90 os/bluestore/bluestore_types: disable ref_map internal _check
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
bcf20a1ca1 os/bluestore: restructure cache trimming in terms of mempool
Trim cache based on overall memory utilization by cache objects,
as tracked by the bluestore_meta_* mempools.  This lets you
configure the bluestore cache size in terms of bytes of memory.

Note that we do not account for other memory utilization by the
OSD.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:48:48 -04:00
Sage Weil
030bc063e4 os/bluestore: move most cache types into mempools
Keep onodes separate so we can use onodes as a sentinal for
overall memory usage (this is what we trim in terms of anyway).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
81295c61c4 global: introduce mempool_debug config option, asok command
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
9cbacc81cc include/mempool: introduce mempool infrastructure
This is heavily based on Allen Samuels' prototype.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 13:47:44 -04:00
Sage Weil
d033f15b2d Merge pull request from liewegas/wip-bluestore-fsck-omap
os/bluestore: fsck: fix omap_head check

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 08:49:11 -05:00
Sage Weil
886d201eea os/bluestore: fsck: fix omap_head check
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-02 09:48:37 -04:00
Sage Weil
bcbcc5a4ce Merge pull request from xiexingguo/xxg-wip-bluestore-1102
os/bluestore: less code redundancy

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-02 08:47:53 -05:00
Sage Weil
a126f8d59e Merge pull request from liewegas/wip-bluestore-shard-span
os/bluestore: fix extent shard span check

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2016-11-02 08:34:50 -05:00
Kefu Chai
39cbb557f3 Merge pull request from ceph/wip-cmake
cmake: do not link erasure tests again libosd

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-11-02 18:44:26 +08:00
Kefu Chai
9613de3002 Merge pull request from wjwithagen/patch-14
test_subman.sh: Don't use --tmpdir

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-02 18:39:28 +08:00
Willem Jan Withagen
0894865ed8 test_subman.sh: Don't use --tmpdir
test_subman.sh: Don't use --tmpdir since on most Linux-is it is consideren default
when creating a directory:
`If TEMPLATE is not specified, use tmp.XXXXXXXXXX, and --tmpdir is implied.`

And on FreeBSD --tmpdir is not available anyways.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-11-02 11:03:34 +01:00
xie xingguo
ec071376e7 os/bluestore: use string for most cases in _open_db()
For a better readability.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 17:41:49 +08:00
Kefu Chai
9bdc5d1328 cmake: do not link erasure tests again libosd
they are not referencing any symbols from it. and also the tests testing
plugins are not linked against the ec plugins, they *depend* on them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-11-02 14:56:05 +08:00
Kefu Chai
3d83dc8454 Merge pull request from ceph/wip-17654
librados-dev: install inline_memory.h

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-02 10:53:24 +08:00
xie xingguo
857263fe3e os/bluestore: drop unused parameter "deep" from _fsck_check_extents()
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 09:56:37 +08:00
xie xingguo
69088dc893 tools/ceph_objectstore_tool: handle fsck and fsck-deep op in unified path
181e28ba99
introduces too much code redundancy, this patch simplifies it a little.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-02 09:19:15 +08:00
Josh Durgin
ac1c1f6f0e librados-dev: install inline_memory.h
This is needed to compile against the c++ api since
c38869232c3c852d98fa1826632db360c5a6afd4

Fixes: http://tracker.ceph.com/issues/17654
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-11-01 15:15:38 -07:00
Yehuda Sadeh
f890a71501 Merge pull request from weiqiaomiao/wqm-wip-rgw-bilist
rgw: fix osd crashes when execute "radosgw-admin bi list --max-entries=1" command

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-01 13:57:07 -07:00
Sage Weil
352078deaa Merge pull request from tanghaodong25/cache-logger-wip
os/bluestore: update cache logger after 'trim_cache' operation

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-01 13:25:06 -05:00
Yehuda Sadeh
d36cc6d0fd Merge pull request from cbodley/wip-17569
rgw: clear data_sync_cr if RGWDataSyncControlCR fails

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-01 11:16:09 -07:00
Matt Benjamin
2aecf6b73c Merge pull request from yehudasa/wip-17755
rgw: json encode/decode index_type, allow modification
2016-11-01 13:51:54 -04:00
Sage Weil
27cee5ab9e os/bluestore: rename variable in set_lextent
Defensive naming

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 13:46:06 -04:00
Sage Weil
7c96b674f5 os/bluestore: fix shard span check in set_lextent
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 13:46:06 -04:00
Sage Weil
be22364959 os/bluestore: fsck: check for extents spanning shards
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 13:46:06 -04:00
haodong
6c7ed8f050 os/bluestore: udpate cache logger after trim_cache
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2016-11-02 01:44:23 +08:00
Sage Weil
888c8e635b Merge pull request from nitinkamble/master
install-deps.sh: allow building on SLES systems
2016-11-01 12:30:52 -05:00
Matt Benjamin
2c8d25b6fd Merge pull request from rzarzynski/wip-rgw-frontend-rework
rgw: frontend subsystem rework

all requested changes have been made by the author
tested by-hand w/sigv2 and sigv4, and already passed teuthology rgw suite
2016-11-01 13:11:07 -04:00
Sage Weil
2847e17769 Merge pull request from liewegas/wip-bluestore-fsck
os/bluestore: deep fsck

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2016-11-01 11:21:16 -05:00
Sage Weil
6fc48bc7b1 os/bluestore: verify we can read objects on deep fsck
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 12:20:51 -04:00
Sage Weil
181e28ba99 os: add fsck deep/shallow option
deep==false checks just metadata, and deep==true will (eventually) check
all data (and checksums) too.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 12:20:50 -04:00
Sage Weil
a9c3e9b916 ceph_test_objectstore: fix synthetic fsck
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-01 12:20:12 -04:00
Sage Weil
0b7577f8f5 Merge pull request from liewegas/wip-bluestore-parallel
os/bluestore: refactor bluestore_sync_submit_transaction

Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-11-01 10:59:30 -05:00