Commit Graph

55979 Commits

Author SHA1 Message Date
John Spray
c76c31d312 tools: remove legacy cephfs tool
This has been deprecated for quite some time, in favour
of using proper vxattr and libcephfs interfaces.

Fixes: http://tracker.ceph.com/issues/16035
Signed-off-by: John Spray <john.spray@redhat.com>
2016-07-19 14:00:30 +01:00
John Spray
c49d87b87d mds: return error on empty command message
Previously this returned 0 and an empty
response, which in turn upsets the python
code that calls commands.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-07-19 13:59:34 +01:00
John Spray
a2682edb76 mds: remove TMAP2OMAP check and refactor
Removing this check allows us to finally move
the Objecter instance down into MDSRank where
it belongs.

Fixes: http://tracker.ceph.com/issues/15923
Signed-off-by: John Spray <john.spray@redhat.com>
2016-07-19 13:59:33 +01:00
John Spray
865780075c Merge pull request #10250 from gregsfortytwo/wip-sessionmap-committing
mds: Return "committing" rather than "committed" member in get_committing

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-19 10:51:26 +01:00
Kefu Chai
6f3ce3aed9 Merge pull request #10346 from tchaikov/wip-make-check
test: ceph-disk.sh do not kill all daemons

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2016-07-19 17:50:21 +08:00
Kefu Chai
1727fdd593 Revert "run-make-check.sh: run tests in two steps"
This reverts commit cd0dfc305d.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
3db9ed50e7 Revert "cmake: label some tests with "Racing""
This reverts commit 3a21d0040a.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
700db8e80a test: ceph-disk.sh: do not kill all daemons
this causes mysterious "racing" issues when running ceph-helper tests in
parallel.

Fixes: http://tracker.ceph.com/issues/16729
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
d1f681a274 Merge pull request #10317 from wjwithagen/wip-wjw-fix-dl-lib
Cmake: fix using CMAKE_DL_LIBS instead of dl

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-19 10:36:42 +08:00
Nathan Cutler
12c068365c Merge pull request #10147 from SUSE/wip-16598
rpm: move mount.ceph from ceph-base to ceph-common and add symlink in /sbin for SUSE

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com
2016-07-19 02:28:15 +02:00
Yuri Weinstein
88e6244e30 Merge pull request #9411 from ceph/wip-rocksdb-perf
os/RocksDBStore: use effective Get API instead of iterator api

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-18 16:28:21 -07:00
Yuri Weinstein
cbd8acbb3b Merge pull request #10130 from yonghengdexin735/wip-zzz-add-
src/common/buffer.cc fix judgment for lseek

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 16:23:59 -07:00
Yuri Weinstein
4f3ba2262b Merge pull request #10197 from songbaisen/b0
mon: write fsid use the right return value

Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-18 16:22:40 -07:00
Jason Dillaman
a3438bac71 Merge pull request #9291 from trociny/wip-15715
rbd-nbd does not properly handle resize notifications

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-07-18 16:44:58 -04:00
Jason Dillaman
ada1abb60c Merge pull request #9878 from neurodrone/rbd_api_shrink_check_on_resize
librbd: add explicit shrink check while resizing images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-07-18 16:44:15 -04:00
John Spray
0080b6bc92 Merge pull request #9955 from ukernel/wip-mds-snap-failover
mds: snap failover fixes

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-07-18 21:01:35 +01:00
Ali Maredia
132b6ff9a2 Merge pull request #10283 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-18 13:59:14 -04:00
Nathan Cutler
b2675ff824 build/ops: build mount.ceph and mount.fuse.ceph as client binaries
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-18 19:21:37 +02:00
xie xingguo
e3bd10336b os/bluestore: use p2 macros to simplify bit-allocator block alignment
Mark's comments:

This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Closes #10253

os/bluestore: require block_size to be power of 2 aligned

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: use ISP2 macro for zone/span size checking

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 10:47:41 -05:00
xie xingguo
7f564d9e2c os/bluestore: fix potential uninitialized nid of onode
The _zero() process may implicitly create a new onode,
thus we shall call _assign_nid() to initialize the nid
properly. And if the onode already has one, _assign_nid()
does nothing.

So it is proper to call _assign_nid() here under any case.

Mark's comments:

This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Closes #10236

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: check against we don't overflow

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: try to reap as many collections as we can

So if there is one collection getting contiguously stucking,
we don't abort at the same point each time.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: make device size of BitFreelistManager is block-size aligned

Otherwise if we try to set past-eof blocks as allocated durint create(),
the call to _xor() will trigger the firing of the following assert:

   assert((length & block_mask) == length);

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 10:45:23 -05:00
xie xingguo
61769636ff os/bluestore: end scope of std::hex properly; convert csum error to EIO
Mark's comments:

This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Closes #10225

os/bluestore: end scope of std::hex properly

To avoid side-effects by accident.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: convert csum error to EIO

The verify_csum() method either returns -1 or -EOPNOTSUPP, which
is not very proper and difficult for user understanding.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: assert lextent is shared

Otherwise we are risking of accessing violation.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: drop duplicated assignment of result code

These two methods never fail actually.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: improve _do_read() a little

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: assert decoding of shard of key to be successful

Otherwise we are risking of acessing null pointer.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-18 10:42:33 -05:00
Nathan Cutler
b8c24bf2f8 rpm: move mount.ceph from ceph-base to ceph-common
Ceph clients use mount.ceph to mount CephFS filesystems, and
ceph-base is not expected to be installed on client systems.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-18 16:16:11 +02:00
Nathan Cutler
b090e9da32 rpm: create mount.ceph symlink in /sbin (SUSE only)
Fixes: http://tracker.ceph.com/issues/16598
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-07-18 15:36:29 +02:00
Mark Nelson
cab254e924 os/bluestore: revert preferred csum behavior
This passes "ceph_test_objectstore --gtest_filter=*/2".
This restores 4K random read performance to previous levels when objects
are were previously written out using large IOs (4MB in this case):

pre-patch: 26MB/s
post-pated: 610MB/s

Closes #10320

Signed-off-by: Mark Nelson <mnelson@redhat.com>
2016-07-17 21:33:59 -05:00
Somnath Roy
bf70bcb6c5 Bluestore: Fixed a Bluestore crash
A bluestore race condition is been fixed by protecting txc structures
within _txc_state_poc with collection lock.

Mark's comments:

This fixes segfaults during random write tests with bluestore.
This passes "ceph_test_objectstore --gtest_filter=*/2".
This may introduce a small performance regresion, though there is enough
noise in the results to make it inconclusive.

Closes #10220

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
2016-07-17 21:26:31 -05:00
Willem Jan Withagen
aab972c0d5 Cmake: fix using CMAKE_DL_LIBS instead of dl
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-16 14:08:45 +02:00
Kefu Chai
c30c5223b5 cmake: only allow up to 1 hour for a ceph test
quote from
https://cmake.org/cmake/help/v3.0/prop_test/TIMEOUT.html?highlight=timeout

> If it exceeds that the test process will be killed and ctest will move
> to the next test.

this helps us to identify test hang.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:47 +08:00
Kefu Chai
5a695974cc cmake: restructure src/CMakeLists.txt in a more hierarchical way
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:47 +08:00
Kefu Chai
45176611b3 cmake: do not pass --disable-pip-version-check if not supported
on older versions of pip, this option is not supported, and
--disable-pip-version-check is implied with --no-index. so no need to
use them when --no-index is passed to pip.

this partially reverts 395f2c5

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
941605d4f3 cmake: remove duplicated src from ceph_rgw_jsonparser
they are included by rgw_a as well. and ceph_rgw_jsonparser is linked
against rgw_a.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
64a8bfbb4a cmake: link libcommon against $CRYPTO_LIBS
as ceph_crypt.cc is using the symbols in it, and libcommon contains
ceph_crypt.cc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
cd0dfc305d run-make-check.sh: run tests in two steps
this is a workaround of the timeout found in jenkins. currently three
tests are found timeout, and they are labeld with "Racing" and
"LongRunning". so, to workaround this issue, we run the tests in two
phases:

1. run the racing tests with -j1
2. run the non-racing tests with -jN

if we all all tests with -j1, the total test time is 2683.57 sec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
3a21d0040a cmake: label some tests with "Racing"
two tests timesout for unknown reasons, so label them with
"Racing" and "LongRunning" labels.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:45 +08:00
Kefu Chai
5b14e28392 cmake: no need to depend run-tox-ceph-disk on test
run-tox-ceph-disk and run-tox-ceph-detect-init are already added as
test, so "ctest" and "make {test,check}" will run them without extra
settings.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 10:57:14 +08:00
Kefu Chai
ce5724effb cmake: add a "tests" target to build tests
please note "make test" is used by cmake to run tests, so we cannot just
repurpose it to *build* them.

* AddCephTest.cmake: depends on "tests"
* CMakeLists.txt: let "check" depend on "tests"
* src/CMakeLists.txt: update the run-tox tests
* run-make-check.sh: use "make tests" and "ctest" instead of "make check"
* ceph-detect-init/CMakeLists.txt: let "tests" depend on
    "ceph-detect-init"
* ceph-disk/CMakeLists.txt: let "tests" depend on "ceph-disk"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 10:57:12 +08:00
Yehuda Sadeh
3e5775475b Merge pull request #10301 from cbodley/wip-rgw-meta-stack-wakeup
rgw: RGWMetaSyncCR holds refs to stacks instead of crs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2016-07-15 19:19:57 -07:00
Casey Bodley
e4bc16044e rgw: RGWMetaSyncCR holds refs to stacks for wakeup
because RGWCoroutine::wakeup() calls RGWCoroutinesStack::wakeup(), the
stack must also stay alive

Fixes: http://tracker.ceph.com/issues/16666

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-15 11:20:13 -04:00
Kefu Chai
4025a2eb80 Merge pull request #10306 from tchaikov/wip-no-mktemp-p
qa/workunits/cephtool/test.sh: s/TMPDIR/TEMP_DIR/

Reviewed-by: Haomai Wang <haomai@xsky.com>
2016-07-15 21:34:02 +08:00
Kefu Chai
bf58aeb6d5 Merge pull request #9972 from ceph/objclass-perm
osd: object class loading and execution permissions

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2016-07-15 17:25:44 +08:00
Kefu Chai
316c15bc9d Merge pull request #9980 from gaowanlong/split_out_handle_pg_scrub
osd: small cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 17:09:14 +08:00
Kefu Chai
9103958688 Merge pull request #10126 from dx9/wip-fcntl-warns
test/libcephfs: fix gcc sys/fcntl.h warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 17:05:15 +08:00
Kefu Chai
56df5dcda4 Merge pull request #10166 from wjwithagen/wip-wjw-cmake-test_rados_tool.sh
test_rados_tool.sh: Make script work under ctest

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 17:03:59 +08:00
Kefu Chai
e0a5e67401 Merge pull request #9782 from Ved-vampir/zlib_cleanup
compressor: zlib compressor plugin  cleanup

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 17:03:38 +08:00
Kefu Chai
83e64fad87 test: run_seed_to_range.sh: check existance of a directory using [ -d "$dir" ]
sadly, sh evalutes `[ -d ]` to true. as it takes "-d" as a non-empty
string as true.

this fixes following failure
```
2016-07-12T23:22:02.839 INFO:teuthology.orchestra.run.mira084.stderr:cp:
missing destination file operand after ‘.’
2016-07-12T23:22:02.839 INFO:teuthology.orchestra.run.mira084.stderr:Try
'cp --help' for more information.
```
see
http://pulpito.ceph.com/kchai-2016-07-12_23:09:35-rados-wip-kefu-testing2---basic-mira/311334/

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-15 16:48:22 +08:00
Kefu Chai
7c1b456022 qa/workunits/cephtool/test.sh: s/TMPDIR/TEMP_DIR/
this fixes the test failure of
```
2016-07-12T23:29:40.935
INFO:tasks.workunit.client.0.mira101.stderr:/home/ubuntu/cephtest/workunit.client.0/cephtool/test.sh:
line 153: /CEPH_WATCH_9445: Permission denied
```
see
http://pulpito.ceph.com/kchai-2016-07-12_23:09:35-rados-wip-kefu-testing2---basic-mira/311333/

it's a regression introduced by e5c262b

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-15 16:20:37 +08:00
Kefu Chai
e5c262bfe6 qa/workunits/cephtool/test.sh: use mktemp $TEMP_DIR/XXX instead
mktemp -p is not supported on FreeBSD

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-15 14:21:52 +08:00
Kefu Chai
e0b94ff38d Merge pull request #10131 from badone/wip-peering-doc-fixes
doc: peering.rst, fix typo

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 12:15:55 +08:00
Kefu Chai
b342e4f06b Merge pull request #10292 from badone/wip-perf-counters-doc-fixes
doc: perf_counters.rst fix trivial typo

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 12:13:19 +08:00
Kefu Chai
3be6e1c76b test: ceph-detect-init/run-tox.sh: run it from any path
this follows the pattern in ceph-disk. this enables us to run
ceph-detect-init/run-tox.sh from the ${CMAKE_BINARY_DIRECTORY}
as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-15 09:55:51 +08:00
Yan, Zheng
265f96bda7 client: fix MetaRequest::set_other_inode()
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2016-07-15 09:15:55 +08:00