Commit Graph

56009 Commits

Author SHA1 Message Date
Kefu Chai
cf6dcf0ae7 cmake: add "WITH_PROFILER" option
this option matches '--with-profiler' option in autoconf. and it is off
by default. we should not link against libprofiler unless asked to do
so. this change fixes this problem.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
8582aa8608 cmake: add_subdirectory(include)
fixes the packaging failure introduced by 741f990

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Nathan Cutler
0a44aaa74c Merge pull request #10353 from rjfd/wip-fix-ceph-mount
rpm: Fix creation of mount.ceph symbolic link for SUSE distros

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2016-07-20 04:22:21 +02:00
Kefu Chai
f0426fbf54 Merge pull request #10351 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-20 10:06:29 +08:00
Mark Nelson
472a5d8d6c Merge pull request #10295 from xiexingguo/xxg-wip-bluestore-2016-07-14
kv/MemDB: misc fixes and cleanups

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:16:29 -05:00
Mark Nelson
dd4de0f786 Merge pull request #10294 from xiexingguo/xxg-wip-bluestore-2016-07-13
os/bluestore: use BE for gifting and reclaiming from bluefs

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:15:25 -05:00
Mark Nelson
81c10210af Merge pull request #10277 from xiexingguo/xxg-wip-fix-posix-fallocate
os/bluestore: fix error handling of posix_fallocate()

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:12:13 -05:00
Mykola Golub
ee75375f3c Merge pull request #10249 from dillaman/wip-16622
rbd-mirror: snap rename does not properly replicate to peers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-07-19 20:16:04 +03:00
Kefu Chai
6199d2ef05 Merge pull request #10313 from ceph/objclass-perm-feedback
Objclass perm feedback

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:43:29 +08:00
Ricardo Dias
976c9d4f31
rpm: Fix creation of mount.ceph symbolic link for SUSE distros
Signed-off-by: Ricardo Dias <rdias@suse.com>
2016-07-19 15:40:36 +01:00
Kefu Chai
eb71afdbe3 cmake: disable VTA for osdcap.cc
this silences the warning of:
```
In file included from
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:58:0,
                 from
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:20:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:
In member function ‘virtual void
OSDCap_AllowClassMulti_Test::TestBody()’:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:766:6:
note: variable tracking size limit exceeded with
-fvar-tracking-assignments, retrying without
 TEST(OSDCap, AllowClassMulti) {
      ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/internal/gtest-internal.h:1211:3:
note: in definition of macro ‘GTEST_TEST_CLASS_NAME_’
   test_case_name##_##test_name##_Test
   ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2181:3:
note: in expansion of macro ‘GTEST_TEST_’
   GTEST_TEST_(test_case_name, test_name, \
   ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2187:42:
note: in expansion of macro ‘GTEST_TEST’
 # define TEST(test_case_name, test_name) GTEST_TEST(test_case_name,
 # test_name)
                                          ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:766:1:
note: in expansion of macro ‘TEST’
 TEST(OSDCap, AllowClassMulti) {
 ^
```

see also b668051

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:25:45 +08:00
Kefu Chai
3173d2ea12 cmake: remove util.cc from lib{rados,cephfs},ceph-objectstore-tool
util.cc is included by both librados and libcephfs, the `lvm` static
variable in `lsb_release_parse()` will be free twice by them. this
could lead to double free issue. and util.cc is not used by client at all, so
remove it from them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:25:36 +08:00
Kefu Chai
741f99073c cmake: restructure src/CMakeLists.txt in a more hierarchical way
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:23:12 +08:00
Jason Dillaman
4df913d10b rbd-mirror: preprocess journal events prior to applying
Fixes: http://tracker.ceph.com/issues/16622
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:28:36 -04:00
Jason Dillaman
fdfca55737 rbd-mirror: event preprocessor to handle snap rename operations
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:28:36 -04:00
Jason Dillaman
270cb74bc2 librbd: improve journaling debug log messages
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:28:00 -04:00
Jason Dillaman
57cd75e805 librbd: separate journal event decoding and processing
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:28:00 -04:00
Jason Dillaman
f70b90c48d librbd: record original snap name in snap rename journal event
Remote peers need a key to map snapshot ids between clusters.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:28:00 -04:00
Jason Dillaman
77699bfe74 librbd: simple duplicate op checks for all maintenance operations
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:28:00 -04:00
Jason Dillaman
2f4cb26d8b qa/workunits/rbd: exercise snapshot renames within rbd-mirror test
Snapshot rename operations utilize the (cluster) unique snapshot
sequence to prevent attempts at replays. When mirroring to a
different cluster, these sequences will not align.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-19 09:27:32 -04:00
John Spray
865780075c Merge pull request #10250 from gregsfortytwo/wip-sessionmap-committing
mds: Return "committing" rather than "committed" member in get_committing

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-19 10:51:26 +01:00
Kefu Chai
6f3ce3aed9 Merge pull request #10346 from tchaikov/wip-make-check
test: ceph-disk.sh do not kill all daemons

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2016-07-19 17:50:21 +08:00
Kefu Chai
1727fdd593 Revert "run-make-check.sh: run tests in two steps"
This reverts commit cd0dfc305d.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
3db9ed50e7 Revert "cmake: label some tests with "Racing""
This reverts commit 3a21d0040a.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
700db8e80a test: ceph-disk.sh: do not kill all daemons
this causes mysterious "racing" issues when running ceph-helper tests in
parallel.

Fixes: http://tracker.ceph.com/issues/16729
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
d1f681a274 Merge pull request #10317 from wjwithagen/wip-wjw-fix-dl-lib
Cmake: fix using CMAKE_DL_LIBS instead of dl

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-19 10:36:42 +08:00
Nathan Cutler
12c068365c Merge pull request #10147 from SUSE/wip-16598
rpm: move mount.ceph from ceph-base to ceph-common and add symlink in /sbin for SUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com
2016-07-19 02:28:15 +02:00
Yuri Weinstein
88e6244e30 Merge pull request #9411 from ceph/wip-rocksdb-perf
os/RocksDBStore: use effective Get API instead of iterator api

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-18 16:28:21 -07:00
Yuri Weinstein
cbd8acbb3b Merge pull request #10130 from yonghengdexin735/wip-zzz-add-
src/common/buffer.cc fix judgment for lseek

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 16:23:59 -07:00
Yuri Weinstein
4f3ba2262b Merge pull request #10197 from songbaisen/b0
mon: write fsid use the right return value

Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-18 16:22:40 -07:00
Jason Dillaman
a3438bac71 Merge pull request #9291 from trociny/wip-15715
rbd-nbd does not properly handle resize notifications

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-07-18 16:44:58 -04:00
Jason Dillaman
ada1abb60c Merge pull request #9878 from neurodrone/rbd_api_shrink_check_on_resize
librbd: add explicit shrink check while resizing images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-07-18 16:44:15 -04:00
John Spray
0080b6bc92 Merge pull request #9955 from ukernel/wip-mds-snap-failover
mds: snap failover fixes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-07-18 21:01:35 +01:00
Ali Maredia
132b6ff9a2 Merge pull request #10283 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-18 13:59:14 -04:00
Nathan Cutler
b2675ff824 build/ops: build mount.ceph and mount.fuse.ceph as client binaries
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-18 19:21:37 +02:00
xie xingguo
e3bd10336b os/bluestore: use p2 macros to simplify bit-allocator block alignment
Mark's comments:

This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Closes #10253

os/bluestore: require block_size to be power of 2 aligned

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: use ISP2 macro for zone/span size checking

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 10:47:41 -05:00
xie xingguo
7f564d9e2c os/bluestore: fix potential uninitialized nid of onode
The _zero() process may implicitly create a new onode,
thus we shall call _assign_nid() to initialize the nid
properly. And if the onode already has one, _assign_nid()
does nothing.

So it is proper to call _assign_nid() here under any case.

Mark's comments:

This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Closes #10236

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: check against we don't overflow

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: try to reap as many collections as we can

So if there is one collection getting contiguously stucking,
we don't abort at the same point each time.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: make device size of BitFreelistManager is block-size aligned

Otherwise if we try to set past-eof blocks as allocated durint create(),
the call to _xor() will trigger the firing of the following assert:

   assert((length & block_mask) == length);

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 10:45:23 -05:00
xie xingguo
61769636ff os/bluestore: end scope of std::hex properly; convert csum error to EIO
Mark's comments:

This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Closes #10225

os/bluestore: end scope of std::hex properly

To avoid side-effects by accident.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: convert csum error to EIO

The verify_csum() method either returns -1 or -EOPNOTSUPP, which
is not very proper and difficult for user understanding.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: assert lextent is shared

Otherwise we are risking of accessing violation.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: drop duplicated assignment of result code

These two methods never fail actually.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: improve _do_read() a little

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: assert decoding of shard of key to be successful

Otherwise we are risking of acessing null pointer.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 10:42:33 -05:00
Nathan Cutler
b8c24bf2f8 rpm: move mount.ceph from ceph-base to ceph-common
Ceph clients use mount.ceph to mount CephFS filesystems, and
ceph-base is not expected to be installed on client systems.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-18 16:16:11 +02:00
Nathan Cutler
b090e9da32 rpm: create mount.ceph symlink in /sbin (SUSE only)
Fixes: http://tracker.ceph.com/issues/16598
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-18 15:36:29 +02:00
Mark Nelson
cab254e924 os/bluestore: revert preferred csum behavior
This passes "ceph_test_objectstore --gtest_filter=*/2".
This restores 4K random read performance to previous levels when objects
are were previously written out using large IOs (4MB in this case):

pre-patch: 26MB/s
post-pated: 610MB/s

Closes #10320

Signed-off-by: Mark Nelson <mnelson@redhat.com>
2016-07-17 21:33:59 -05:00
Somnath Roy
bf70bcb6c5 Bluestore: Fixed a Bluestore crash
A bluestore race condition is been fixed by protecting txc structures
within _txc_state_poc with collection lock.

Mark's comments:

This fixes segfaults during random write tests with bluestore.
This passes "ceph_test_objectstore --gtest_filter=*/2".
This may introduce a small performance regresion, though there is enough
noise in the results to make it inconclusive.

Closes #10220

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2016-07-17 21:26:31 -05:00
Willem Jan Withagen
aab972c0d5 Cmake: fix using CMAKE_DL_LIBS instead of dl
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-16 14:08:45 +02:00
Kefu Chai
c30c5223b5 cmake: only allow up to 1 hour for a ceph test
quote from
https://cmake.org/cmake/help/v3.0/prop_test/TIMEOUT.html?highlight=timeout

> If it exceeds that the test process will be killed and ctest will move
> to the next test.

this helps us to identify test hang.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:47 +08:00
Kefu Chai
5a695974cc cmake: restructure src/CMakeLists.txt in a more hierarchical way
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:47 +08:00
Kefu Chai
45176611b3 cmake: do not pass --disable-pip-version-check if not supported
on older versions of pip, this option is not supported, and
--disable-pip-version-check is implied with --no-index. so no need to
use them when --no-index is passed to pip.

this partially reverts 395f2c5

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
941605d4f3 cmake: remove duplicated src from ceph_rgw_jsonparser
they are included by rgw_a as well. and ceph_rgw_jsonparser is linked
against rgw_a.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
64a8bfbb4a cmake: link libcommon against $CRYPTO_LIBS
as ceph_crypt.cc is using the symbols in it, and libcommon contains
ceph_crypt.cc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
cd0dfc305d run-make-check.sh: run tests in two steps
this is a workaround of the timeout found in jenkins. currently three
tests are found timeout, and they are labeld with "Racing" and
"LongRunning". so, to workaround this issue, we run the tests in two
phases:

1. run the racing tests with -j1
2. run the non-racing tests with -jN

if we all all tests with -j1, the total test time is 2683.57 sec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
3a21d0040a cmake: label some tests with "Racing"
two tests timesout for unknown reasons, so label them with
"Racing" and "LongRunning" labels.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:45 +08:00