Commit Graph

48828 Commits

Author SHA1 Message Date
Sage Weil
6a8cafe198 Merge pull request #5931 from ceph/wip-cxx11concurrency
librados: move to c++11 concurrency types

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-02-08 09:23:15 -05:00
Piotr Dałek
7181a6f1b0 msg/simple: significantly reduce minimal memory usage of Pipes
Pipe class instance weights over 17 KB in size, and most of it is used by
msgvec structure, which weights 16 KB. Reduce it to 4KB by reducing
number of iovecs in this array down to 256 (= 1024/4). This won't cause
noticeable perf decrease, as even under extreme loads, we exceed 256 used
iovecs in only less than 1% of all write calls. For high-density nodes and
large clusters, savings will exceed hundreds of megabytes per node.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-02-08 15:18:39 +01:00
Piotr Dałek
69d98b6563 msg/async: significantly reduce minimal memory usage of AsyncConnections
AsyncConnection class instance weights over 18 KB in size, and most of it
is used by msgvec structure, which weights 16 KB. Reduce it to 4KB by reducing
number of iovecs in this array down to 256 (= 1024/4). This won't cause
noticeable perf decrease, as even under extreme loads, we exceed 256 used
iovecs in only less than 1% of all write calls. For high-density nodes and
large clusters, savings will exceed hundreds of megabytes per node.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-02-08 15:16:35 +01:00
Sage Weil
7e44299618 Merge branch 'print-blocked-op' of git://github.com/majianpeng/ceph
Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-08 09:14:21 -05:00
Sage Weil
f17dbc65df Merge pull request #6608 from esnyder/wip-6634
osd: remove unused SnapContext member snapc from MOSDSubOp message

Reviewed-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-08 09:17:49 -05:00
Sage Weil
658780730a Merge branch 'master' of git://github.com/joconcepts/ceph
Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-08 09:12:12 -05:00
Sage Weil
a981c1ecc2 Merge pull request #7181 from songbaisen/song5
crush: reply quickly from get_immediate_parent

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-02-08 09:16:14 -05:00
Sage Weil
20228cb0a2 Merge pull request #7437 from Ved-vampir/wip_zlib_plugin
common: add zlib compression plugin

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-08 09:15:04 -05:00
Sage Weil
34d33bcc5b Merge pull request #7527 from liewegas/wip-no-rgw-prestart
systemd/ceph-radosgw-prestart.sh: remove

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2016-02-08 09:13:38 -05:00
Igor Fedotov
12c083ed43 OSD/ReplicatedPG: Fixes unneccessary object promotion when deleting from the cache that lacks hit_set configured.
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-02-08 15:39:09 +03:00
Haomai Wang
90561b466d Merge pull request #7547 from yuyuyu101/wip-14679
AsyncConnection: avoid debug log in cleanup_handler
2016-02-08 14:15:37 +08:00
Kefu Chai
3395fb9e06 crush/CrushTester: workaround a bug in boost::icl
the boost::icl in boost v1.55 has a bug when working with libc++,

> Assertion failed: (this->_map.find(inter_val) == this->_map.end()),
> function gap_insert, file
> /usr/local/include/boost/icl/interval_base_map.hpp, line 555.

see https://svn.boost.org/trac/boost/ticket/9501 and
    https://svn.boost.org/trac/boost/ticket/9987

the bug was fixed in boost v1.56. as a workaround we use the the
`set` from `boost::container` instead of the `std::set` from libc++.
we should drop this workaround once we bump the supported boost
version from 1.42 to 1.56.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-08 13:37:45 +08:00
Kefu Chai
d21724a145 Merge pull request #7554 from tchaikov/wip-ceph-disk-test
tests: ceph-disk.sh should compare with the resolved path

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-08 01:52:16 +08:00
Xinze Chi
990b437f4e OSDMap: reset osd_primary_affinity shared_ptr when deepish_copy_from
Base on current logic, OSDMonitor may call create_pending and
encode_pending twice for the some epoch.

In encode_pending:

tmp.deepish_copy_from(osdmap);
tmp.apply_incremental(pending_inc);

This Op would change the tmp osd_primary_affinity, but the osd_primary_affinity
is declared as ceph::shared_ptr, so this would change the osdmap too. When this
round encode_pending is proposed fail. We may call encode_pending again, but the
osdmap is changed last round, so the pending_inc would be wrong.

Fixes: #14686
Signed-off-by: Xinze Chi <xinze@xsky.com>
2016-02-07 22:19:35 +08:00
Sage Weil
e604d681a4 Merge pull request #7551 from JevonQ/update-libcephfs-version
libcephfs: update LIBCEPHFS_VERSION to indicate the interface was changed

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-07 09:01:04 -05:00
Kefu Chai
d267aaf8f3 tests: ceph-disk.sh should compare with the resolved path
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 04:39:24 -08:00
Jevon Qiao
4d3830225c libcephfs: Update LIBCEPHFS_VERSION to indicate the interface was changed
Signed-off-by: Jevon Qiao <scaleqiao@gmail.com>
2016-02-07 14:00:53 +08:00
Kefu Chai
c21de4d08a AsyncMessenger: remove unused variables
* remove AsyncMessenger::listen_sd
* remove `ee` in KqueueDriver::del_event()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 02:37:46 +08:00
Kefu Chai
c45854a806 crypto: remove unused Digest::sec_type
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 01:59:47 +08:00
Kefu Chai
93ac31c7d0 journal: remove unused ObjectPlayer::m_fetch_interval
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 01:59:46 +08:00
Kefu Chai
2a53737054 librbd: remove unused C_DiffObject::m_request_num
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 01:59:46 +08:00
Kefu Chai
9c3f821936 librbd: replace atomic_t with std::atomic<int>
it fails to compile with clang, and turns out it should be
`ceph::atomic_t`, but I take this chance to replace it
with std::atomic<> to phase out atomic_t.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 01:59:46 +08:00
Kefu Chai
64361c3986 ECUtil: remove unused stripe_info_t::stripe_size
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 01:59:46 +08:00
Kefu Chai
7d08bbf595 osd: init HeartbeatDispatcher using OSD->cct
its base class 'Dispatcher' is uninitialized when used here to access
'Dispatcher::cct', so we'd better use OSD::cct to initialize it instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-07 01:59:44 +08:00
Kefu Chai
7f854cc180 Merge pull request #7549 from tchaikov/wip-fix-ceph-disk-tests
ceph-disk/test: fix test_prepare.py::TestPrepare tests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-06 23:43:31 +08:00
Kefu Chai
08364c2ff7 ceph-disk/test: fix test_prepare.py::TestPrepare tests
actually we are verifying if the variable is an instance of
specified class. for example, the `prepare.data` should be
a `PrepareFilestoreData` if `--bluestore` is not specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-06 06:27:30 -08:00
Haomai Wang
8569d38f98 AsyncConnection: avoid debug log in cleanup_handler
local connection will be stop and call cleanup_handler after messenger
is down
introduced in
commit(9da2fffd31)

Fix #14679

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-02-06 15:40:25 +08:00
Josh Durgin
0c6a724821 Merge pull request #7540 from dillaman/wip-librados-test-stub-unwatch
librados_test_stub: protect against notify/unwatch race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-05 17:41:03 -08:00
Josh Durgin
2b2ce9b909 Merge pull request #7529 from dillaman/wip-14412
librbd: journal framework for tracking exclusive lock transitions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-05 17:39:03 -08:00
Boris Ranto
a48c081d20 rgw/rgw_orphan: check the return value of save_state
It was discovered by covscan that we do not check the return value of
save_state in this one case.

Signed-off-by: Boris Ranto <branto@redhat.com>
2016-02-06 02:09:00 +01:00
Sage Weil
2e4fa4b3bd Merge pull request #7377 from liewegas/wip-datadir-search
config: add $data_dir/config to config search path
2016-02-05 20:03:50 -05:00
Sage Weil
f3a0157587 Merge pull request #7466 from adamemerson/wip-bsd-time-second-try
common: time: have skewing-now call non-skewing now
2016-02-05 20:03:35 -05:00
Sage Weil
220a473094 Merge pull request #7463 from dachary/wip-13422-pid-file
global: do not start two daemons with a single pid-file (part 2)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-05 20:03:00 -05:00
Sage Weil
c6ff94c21a Merge pull request #7460 from xiexingguo/xxg-wip-block
osd: blockdevice: avoid implicit cast and add guard

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-05 20:02:34 -05:00
Sage Weil
5f99361070 Merge pull request #7530 from xiexingguo/xxg-wip-nid
osd: bluestore: assign nid for newly created onode

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-05 20:02:07 -05:00
Sage Weil
3ac4d711d2 Merge pull request #7521 from liewegas/wip-memstore
osd: memstore: set Collection::cid on create
2016-02-05 20:01:47 -05:00
Sage Weil
86978957cb Merge pull request #7486 from ceph/wip-bluestore-oref
osd: bluestore: fix two wal bugs; reuse OnodeRefs when applying txc

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-02-05 20:01:32 -05:00
Sage Weil
4dbd727f27 Merge pull request #7531 from jjhuo/master
osd: bluestore: remove unused intrusive member hook in enode

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-05 20:01:05 -05:00
Sage Weil
caab2e1d15 Merge pull request #7535 from liewegas/wip-bluefs-mkfs
osd: bluestore/BlueFS: initialize super block_size earlier in mkfs

Reviewed-by: Dan Mick <dmick@redhat.com>
2016-02-05 20:00:31 -05:00
Sage Weil
7813776afd Merge pull request #7536 from liewegas/wip-bluestore-block
osd: bluestore: fix block device file creation
2016-02-05 20:00:07 -05:00
Ali Maredia
0594614781 Merge pull request #7541 from linuxbox2/cmake-feb5
Cmake feb5
2016-02-05 19:53:08 -05:00
Adam C. Emerson
f4b083a5e3 osdc: Update to use C++11 concurrency
The only externally visible change is that Objecter::get_osdmap_read
and Objedcter::put_osdmap_read have been removed in favor of
Objecter::with_osdmap.

It can be used like:

objecter->with_osdmap([&](const OSDMap& o) { o.do_stuff(); };
int x = objecter->with_osdmap([&](const OSDMap& o) {
    return get_thing(id); });
objecter->with_osdmap(std::mem_fn(&OSDMap::do_stuff));
int x = objecter->with_osdmap(std::mem_fn(&OSDMap::get_thing),
                              id);

The choice between the style of the first two examples or the second two
is arbitrary and depends on what you prefer.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2016-02-05 17:07:28 -05:00
Matt Benjamin
21438a6773 cmake: add libboost_system to EXTRALIBS
This concisely fixes several unittest builds, and reflects the
fact that this library dependency has moved into several areas
of the codebase (libcephfs, librbd, librgw).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-05 16:46:21 -05:00
Matt Benjamin
2669e0c280 cmake: don't try to install src/ceph-disk directory
The proper cmake boilerplate to call ceph-disk/ceph-disk's
python installer is not present, FYI.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-05 16:46:16 -05:00
Matt Benjamin
33bc2ba55e cmake: don't add_test cephtool_test_osd twice
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-05 16:46:13 -05:00
Jason Dillaman
38b9be2a99 librados_test_stub: protect against notify/unwatch race
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-05 16:14:39 -05:00
Adam C. Emerson
c1b3138f2e concurrency: Add shunique_lock
Locker class tracking shared/unique/unlocked state of SharedMutex
concept.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2016-02-05 15:57:03 -05:00
Jason Dillaman
9418f055bd librbd: partial implementation of journal client / tag allocation
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-05 15:21:28 -05:00
Jason Dillaman
8443e50710 librbd: initial version of journal client and tag metadata
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-05 15:21:28 -05:00
Jason Dillaman
608947cb3f journal: added tag support methods
librbd, for example, will allocate a new tag after acquiring
the exclusive lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-05 15:21:28 -05:00