RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-03-25 11:48:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
59,405 Commits 226 Branches 396 Tags 1.7 GiB
a6eec64c95
Commit Graph

59405 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sage Weil
a6eec64c95 mempool: pick shard on every allocation; move types to pool_t 
If we have a static choice of shard for every *type*
(pool_allocator_base_t) then we will hammer that shard from all
threads and play cacheline ping-pong.  Instead, move the types
list to pool_t (there aren't that many anyway) and pick a shard
on every allocate/deallocate call.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:48:49 -04:00
Sage Weil
b174694493 mempool: make num_shards a power of 2 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:48:48 -04:00
Sage Weil
735be3662f include/mempool: rename containers -> types 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:48:48 -04:00
Sage Weil
1b51c05068 include/mempool: note about runtime complexity 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:48:48 -04:00
Sage Weil
34f434be90 os/bluestore/bluestore_types: disable ref_map internal _check 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:48:48 -04:00
Sage Weil
bcf20a1ca1 os/bluestore: restructure cache trimming in terms of mempool 
Trim cache based on overall memory utilization by cache objects,
as tracked by the bluestore_meta_* mempools.  This lets you
configure the bluestore cache size in terms of bytes of memory.

Note that we do not account for other memory utilization by the
OSD.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:48:48 -04:00
Sage Weil
030bc063e4 os/bluestore: move most cache types into mempools 
Keep onodes separate so we can use onodes as a sentinal for
overall memory usage (this is what we trim in terms of anyway).

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:47:44 -04:00
Sage Weil
81295c61c4 global: introduce mempool_debug config option, asok command 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:47:44 -04:00
Sage Weil
9cbacc81cc include/mempool: introduce mempool infrastructure 
This is heavily based on Allen Samuels' prototype.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 13:47:44 -04:00
Sage Weil
d033f15b2d Merge pull request #11726 from liewegas/wip-bluestore-fsck-omap 
os/bluestore: fsck: fix omap_head check

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-11-02 08:49:11 -05:00
Sage Weil
886d201eea os/bluestore: fsck: fix omap_head check 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-02 09:48:37 -04:00
Sage Weil
bcbcc5a4ce Merge pull request #11740 from xiexingguo/xxg-wip-bluestore-1102 
os/bluestore: less code redundancy

Reviewed-by: Sage Weil <sage@redhat.com>
 2016-11-02 08:47:53 -05:00
Sage Weil
a126f8d59e Merge pull request #11725 from liewegas/wip-bluestore-shard-span 
os/bluestore: fix extent shard span check

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
 2016-11-02 08:34:50 -05:00
Kefu Chai
39cbb557f3 Merge pull request #11738 from ceph/wip-cmake 
cmake: do not link erasure tests again libosd

Reviewed-by: Loic Dachary <ldachary@redhat.com>
 2016-11-02 18:44:26 +08:00
Kefu Chai
9613de3002 Merge pull request #11384 from wjwithagen/patch-14 
test_subman.sh: Don't use --tmpdir

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
 2016-11-02 18:39:28 +08:00
Willem Jan Withagen
0894865ed8 test_subman.sh: Don't use --tmpdir 
test_subman.sh: Don't use --tmpdir since on most Linux-is it is consideren default
when creating a directory:
`If TEMPLATE is not specified, use tmp.XXXXXXXXXX, and --tmpdir is implied.`

And on FreeBSD --tmpdir is not available anyways.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
 2016-11-02 11:03:34 +01:00
xie xingguo
ec071376e7 os/bluestore: use string for most cases in _open_db() 
For a better readability.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-11-02 17:41:49 +08:00
Kefu Chai
9bdc5d1328 cmake: do not link erasure tests again libosd 
they are not referencing any symbols from it. and also the tests testing
plugins are not linked against the ec plugins, they *depend* on them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-11-02 14:56:05 +08:00
Kefu Chai
3d83dc8454 Merge pull request #11730 from ceph/wip-17654 
librados-dev: install inline_memory.h

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2016-11-02 10:53:24 +08:00
xie xingguo
857263fe3e os/bluestore: drop unused parameter "deep" from _fsck_check_extents() 
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-11-02 09:56:37 +08:00
xie xingguo
69088dc893 tools/ceph_objectstore_tool: handle fsck and fsck-deep op in unified path 
181e28ba99
introduces too much code redundancy, this patch simplifies it a little.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-11-02 09:19:15 +08:00
Josh Durgin
ac1c1f6f0e librados-dev: install inline_memory.h 
This is needed to compile against the c++ api since
c38869232c

Fixes: http://tracker.ceph.com/issues/17654
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
 2016-11-01 15:15:38 -07:00
Yehuda Sadeh
f890a71501 Merge pull request #11697 from weiqiaomiao/wqm-wip-rgw-bilist 
rgw: fix osd crashes when execute "radosgw-admin bi list --max-entries=1" command

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
 2016-11-01 13:57:07 -07:00
Sage Weil
352078deaa Merge pull request #11695 from tanghaodong25/cache-logger-wip 
os/bluestore: update cache logger after 'trim_cache' operation

Reviewed-by: Sage Weil <sage@redhat.com>
 2016-11-01 13:25:06 -05:00
Yehuda Sadeh
d36cc6d0fd Merge pull request #11506 from cbodley/wip-17569 
rgw: clear data_sync_cr if RGWDataSyncControlCR fails

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
 2016-11-01 11:16:09 -07:00
Matt Benjamin
2aecf6b73c Merge pull request #11707 from yehudasa/wip-17755 
rgw: json encode/decode index_type, allow modification
 2016-11-01 13:51:54 -04:00
Sage Weil
27cee5ab9e os/bluestore: rename variable in set_lextent 
Defensive naming

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 13:46:06 -04:00
Sage Weil
7c96b674f5 os/bluestore: fix shard span check in set_lextent 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 13:46:06 -04:00
Sage Weil
be22364959 os/bluestore: fsck: check for extents spanning shards 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 13:46:06 -04:00
haodong
6c7ed8f050 os/bluestore: udpate cache logger after trim_cache 
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
 2016-11-02 01:44:23 +08:00
Sage Weil
888c8e635b Merge pull request #11708 from nitinkamble/master 
install-deps.sh: allow building on SLES systems
 2016-11-01 12:30:52 -05:00
Matt Benjamin
2c8d25b6fd Merge pull request #10767 from rzarzynski/wip-rgw-frontend-rework 
rgw: frontend subsystem rework

all requested changes have been made by the author
tested by-hand w/sigv2 and sigv4, and already passed teuthology rgw suite
 2016-11-01 13:11:07 -04:00
Sage Weil
2847e17769 Merge pull request #11724 from liewegas/wip-bluestore-fsck 
os/bluestore: deep fsck

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
 2016-11-01 11:21:16 -05:00
Sage Weil
6fc48bc7b1 os/bluestore: verify we can read objects on deep fsck 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 12:20:51 -04:00
Sage Weil
181e28ba99 os: add fsck deep/shallow option 
deep==false checks just metadata, and deep==true will (eventually) check
all data (and checksums) too.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 12:20:50 -04:00
Sage Weil
a9c3e9b916 ceph_test_objectstore: fix synthetic fsck 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 12:20:12 -04:00
Sage Weil
0b7577f8f5 Merge pull request #11537 from liewegas/wip-bluestore-parallel 
os/bluestore: refactor bluestore_sync_submit_transaction

Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
 2016-11-01 10:59:30 -05:00
Sage Weil
2bb9f9bcb3 Merge pull request #11661 from ceph/wip-git-ver 
cmake: fix git version string, cleanup
 2016-11-01 10:04:10 -05:00
Sage Weil
516b0db87e os/bluestore: default to bluestore_sync_submit_transaction=true 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:31:27 -04:00
Sage Weil
24eb39055d os/bluestore: randomly inject serialize transaction submission 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:31:27 -04:00
Sage Weil
4c13014a2a ceph_test_objectstore: move main() to bottom 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:31:05 -04:00
Sage Weil
27257b4bfa os/bluestore: fix {nid,blobid}_max 
- single case to cover increases ahead of schedule or just in time
- update global max only after txn commits, eliminating race
- drop unneeded id_lock
- improve t naming

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:30:03 -04:00
Sage Weil
722bb20306 os/bluestore: prevent sync transaction submit with extent freelist 
It doesn't support unordered transactions.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:30:03 -04:00
Sage Weil
e2ba42bd75 os/bluestore: submit txn via kv_sync_thread if ids exceed max 
This ensures the txn will not commit before an update to the
global max.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:30:03 -04:00
Sage Weil
e013293b7f os/bluestore: refactor bluestore_sync_submit_transaction 
And drop bluestore_sync_transaction, at least for now.

The key change here is to make a per-txc flag indicating
whether the txn was already submitted.  This allows us
to make a choice between sync and not-sync on a per-txn
basis.

Signed-off-by: Sage Weil <sage@redhat.com>
 2016-11-01 10:30:03 -04:00
Kefu Chai
0d93b608b8 Merge pull request #11721 from dachary/wip-17561-log 
tests: TestErasureCodePluginJerasure must stop the log thread

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2016-11-01 21:53:52 +08:00
Sage Weil
c0c0f4664e Merge pull request #11718 from xiexingguo/xxg-wip-bluestore-1101 
os/bluestore: add counter to trace blob splitting

Reviewed-by: Sage Weil <sage@redhat.com>
 2016-11-01 08:17:43 -05:00
Loic Dachary
17fe16d461 tests: TestErasureCodePluginJerasure must stop the log thread 
When a log entry is created in dout.h, it holds a pointer to the

   static size_t _log_exp_length

variable. When OnExitManager::~OnExitManager from Log.cc is called
during global destruction, it may happen after the static variables
referenced by some entries have been deallocated. The flush() function
will try to access these deallocated variables via the hint_size()
method and core dump.

To prevent this race, the Log::stop() method must be called explicitly
to ensure no entries are left when the global destructors are called.

Fixes: http://tracker.ceph.com/issues/17561

Signed-off-by: Loic Dachary <ldachary@redhat.com>
 2016-11-01 13:33:08 +01:00
Alfredo Deza
6d01da687c Merge pull request #10248 from ktdreyer/wip-doc-no-rpm-md 
doc: rm "type=rpm-md" from yum repositories

Reviewed-by: Alfredo Deza <adeza@redhat.com>
 2016-11-01 08:24:17 -04:00
xiexingguo
bafec40191 Merge pull request #11690 from yonghengdexin735/wip-zzz-fixword 
src/common/perf_counters.h: fix wrong word

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-11-01 20:03:53 +08:00