Commit Graph

60425 Commits

Author SHA1 Message Date
Loic Dachary
5e625674a8 tests: fix ceph-helpers.sh wait_for_clean delays
The TENTH_TIMEOUT was not delcared as an int and failed to be set with
the correct number. The test of the function did not catch this.

Implement computing of the increasingly large sleep delays in a separate
function so that it can be tested more easily. Give up on sub-second
sleep because a the function will not sleep at all if the cluster is
already clean. And if it is not already clean, it is very unlikely to
become clean within less than a second. The downside of having very
short sleep time is that it needlessly stress the machine and also
possibly spam the logs.

Refs: http://tracker.ceph.com/issues/17830

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-11-21 11:42:42 +01:00
Haomai Wang
199aebe835 msg/async/rdma: avoid use global NetHandler which refer to cephcontext
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-21 18:39:08 +08:00
Ramana Raja
b0fa3a4033 ceph_volume_client: set an existing auth ID's default mon caps
... as 'allow r' (the minimum mon caps required to access a share)
when:

* authorizing the auth ID to access a volume.

* deauthorizing the auth ID to access a volume, but the auth ID is
  authorized to access other volumes.

In both the above cases, the ceph_volume_client previously tried to
set the mon caps of the auth ID to an invalid value, None.

Fixes: http://tracker.ceph.com/issues/17800

Signed-off-by: Ramana Raja <rraja@redhat.com>
2016-11-21 15:35:21 +05:30
Kefu Chai
af3e7143d7 yasm-wrapper: translate "-isystem $1" to "-i $1"
this silences the warning of:

yasm: warning: can open only one input file, only the last file will be
processed

as yasm does not understand -isystem, but gcc does.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-11-21 12:00:25 +08:00
Venky Shankar
3a91d7bd2e qa / workunits: make rbd import/export work with image data pool
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
3f6ef037ce tests / rbd: fix rbd python binding test
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
e98b8ceb86 test / rbd-mirror: image data pool test support
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
3d3d0fb31e test / librbd: image data pool test support
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
1c1f4d2a34 test / librbd: use data pool IoCtx while listing objects
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
0206e15b74 librbd / ObjectMap: use data_ctx/md_ctx pool context as required
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
75def529ee librbd / CopyupRequest: use data pool ctx during copyup
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
cce335fccf librados: drop io_ctx_impl on ioctx_create/create2
close() was never called for the passed in IoCtx which
could probably result in an IoCtx leak if the original
IoCtx was a valid pool context allocated earlier.

Its kind of better to do it here rather than to leave
the destruction on the caller for better (or cleaner)
common case handling.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Kefu Chai
49bc71bacf Merge pull request #12080 from wjwithagen/wip-wjw-freebsd-install-deps
install-deps.sh: JQ is needed in one script

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-20 22:17:58 +08:00
Orit Wasserman
2c82b4318b Merge pull request #12076 from cbodley/wip-rgw-region-root
rgw: region conversion respects pre-existing rgw_region_root_pool
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-11-19 22:36:43 +01:00
Willem Jan Withagen
5268a3e3e0 install-deps.sh: JQ is needed in one script
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-11-19 20:10:51 +01:00
Dongsheng Yang
9024e16437 librbd: cleanup: fix a typo in CreateRequest.
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-11-19 08:04:53 -05:00
xie xingguo
44eaa02772 os/bluestore: refactor ctor of SharedBlob
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-19 13:52:27 +08:00
Patrick Donnelly
7b4561091e
mds: use projected path construction for access
A new CDentry will not have a parent until its projected parent is
flushed to journal. During path construction a given dentry may have no
parent yet which will cause fallbacks to be used (the inode number).
This can cause access checks to fail:

    2016-11-17 19:50:43.830207 7eff9977a700 20 Session check_access path #10000000002/3

compare to an earlier check:

    2016-11-17 19:50:43.824223 7eff9977a700 20 Session check_access path /test/1/2

This commit refactors path construction to optionally use projected
parents for the entire chain of directories. Existing use of real stable
parents is unchanged. For example, this line is the same before and
after the patch:

    2016-11-18 23:17:15.611680 7f153f97a700 12 mds.0.cache.dir(10000000002) add_null_dentry [dentry #10000000002/3 [2,head] auth NULL (dversion lock) pv=0 v=1 inode=0 0x55e0f771f5f0]

Here inode "#10000000002" has no stable parent yet. So the path is
constructed as "#10000000002/3".

One notable change in this commit is the removal of
make_path_string_projected which was only used in debugging code. Here's
an example difference:

    2016-11-17 19:50:43.827915 7eff9977a700 10 mds.0.server traverse_to_auth_dir [dir 10000000003 {#10000000003 #10000000002/3}/ [2,head] auth v=1 cv=0/0 state=1073741826|complete f() n() hs=0+0,ss=0+0 0x55f5d35e2ee0]

to:

    2016-11-18 23:17:15.617757 7f153f97a700 10 mds.0.server traverse_to_auth_dir [dir 10000000003 /test/1/2/3/ [2,head] auth v=1 cv=0/0 state=1073741826|complete f() n() hs=0+0,ss=0+0 0x55e0f7706ee0]

Fixes: http://tracker.ceph.com/issues/17858

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-11-18 23:40:22 -05:00
Casey Bodley
b81d880084 rgw: look for region_map in rgw_region_root_pool
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-18 17:04:57 -05:00
Casey Bodley
8b70dc62f4 rgw: region conversion respects pre-existing rgw_region_root_pool
Fixes: http://tracker.ceph.com/issues/17963

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-18 16:29:06 -05:00
Jeff Layton
28988655b0 libcephfs: vet the CEPH_REQ_* flags in the C wrappers
Ensure that we can add new flags in the future by not allowing anyone
to set flags that we don't currently recognize.

Fixes: http://tracker.ceph.com/issues/17911
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2016-11-18 13:59:49 -05:00
Jeff Layton
eb518d7577 libcephfs: move ceph_mount_perms near the other UserPerm functions
...in the header. Also, add a doc comment.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2016-11-18 13:59:48 -05:00
Radoslaw Zarzynski
8bd2407e71 rgw: compilation of the ASIO front-end is enabled by default.
We're changing the default value because the previous one was
a makeshift solution to not fail Ceph compilation due to
the Beast's dependency on Boost >= 1.54 that wasn't available
on CentoOS 7. As we got the in-tree Boost we can compile
the ASIO front-end by default.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-11-18 19:23:46 +01:00
Sage Weil
64bcf92e87 Merge pull request #12057 from liewegas/wip-debug-hitset
ceph_test_rados_api_tier: dump hitset that we fail to decode

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com.>
2016-11-18 12:06:44 -06:00
Sage Weil
662cda684a os/bluestore: prevent sync_submit if there is unstable io for this osr
If this txc or any txc that precedes it has unstable IO we cannot queue
our kv transaction without a bdev sync.  Currently such a sync is only
triggered from the kv_sync_thread, so we need to do this kv submission
async.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-18 08:58:16 -06:00
Sage Weil
503538885f os/bluestore: count txcs with unstable io per osr
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-18 08:58:16 -06:00
Sage Weil
b02d0043cd Merge pull request #12040 from liupan1111/wip-bluestore-lat-report
os/bluestore: fix latency calculation

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-18 08:18:18 -06:00
Pan Liu
01dfc1ba07 OSD: remove 'has_inst', which has the same function as 'is_up'
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 20:49:15 +08:00
Pan Liu
e95026f6be OSD: heartbeat peers need to be updated when a new OSD added into an already existed cluster
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 20:01:42 +08:00
Jianpeng Ma
298ade8ace kv/RocksDBStore: Don't update rocksdb perf_context if rocksdb_perf disable.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-11-18 19:41:43 +08:00
Haomai Wang
eb25965b74 msg/async/rdma: ensure CephContext existed
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-18 19:20:10 +08:00
Kefu Chai
8218ff8b25 Merge pull request #11978 from tchaikov/wip-cmake-jemalloc
cmake: should link against ${ALLOC_LIBS}

Reviewed-by: Varada Raja Kumar Kari <kariraja@yahoo.com>
2016-11-18 17:53:56 +08:00
Kefu Chai
2c16786f86 Merge pull request #12048 from dzafman/wip-dz-memstore
tests: Minor clean-ups

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-18 17:47:14 +08:00
Loic Dachary
2a302ecdf4 Merge pull request #12066 from dachary/wip-16014-cot
tests: save 9 characters for asok paths

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-18 10:37:02 +01:00
Pan Liu
c7f55c40ab os/bluestore: fix a bug: when using bluestore, the output of
"ceph osd perf" is always 0ms.

Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 16:26:19 +08:00
Pan Liu
3ea4760da8 OSD: change the prefix from fs_* to os_*, because commit_latency and
apply_latency will be used not only for filestore.

Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 16:25:31 +08:00
Loic Dachary
cd72ff9f74 tests: save 9 characters for asok paths
For vstart.sh powered tests, save 9 characters in the path name
by replacing testdir/test- with td/t-

60 characters imposed by jenkins
9 characters for src/test
5 characters for td/t-

33 left (instead of 24) for the test to create asok such as out/client.admin.25327.asok

Moving these files outside of the build directory is a bad idea because
tests should only create/use files within the builddir and not write
outside of this directory. Doing so would make things more complicated
for cleanup in case the test fail and create other problems as a
consequence (filling out disk space, conflicting directories between
runs etc.).

For ceph-helpers.sh tests replace testdir with td, saving 5 characters.
This is not strictly necessary but keeps the directory names consistent:
if the developer wants to get rid of all the test leftovers, it is
enough to remove the a single directory: td.

Fixes: http://tracker.ceph.com/issues/16014

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-11-18 09:19:18 +01:00
xie xingguo
8037900c22 os/bluestore: add counter to trace extents have been removed due to compression
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-18 16:18:39 +08:00
xie xingguo
5013f9a3f5 os/bluestore: rename Extent::end() to Extent::logical_end()
and use the new method to simplify code.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-18 14:11:54 +08:00
David Zafman
db1a9434f1 Merge pull request #12058 from dachary/wip-17830-eio
test: disable osd-scrub-repair and test-erasure-eio
2016-11-17 20:44:22 -08:00
Sage Weil
99536f351c PendingReleaseNotes: note on new omap limits
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 22:28:21 -06:00
xie xingguo
8170b52e6b os/bluestore: kill dead gc-related counters
As the gc logic is deprecated by https://github.com/ceph/ceph/pull/12042

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-18 11:30:42 +08:00
Patrick Donnelly
b8b110971b
client: improve failure messages/debugging
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-11-17 21:03:02 -05:00
Sage Weil
a3747de953 os/bluestore: kill kv_submitted bool; use a new state
We have state definitions; use them!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 19:33:12 -06:00
Sage Weil
af12f842a4 os/bluestore: remove unused KV_COMMITTING state
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 19:33:12 -06:00
Sage Weil
315e777c02 ceph_test_objectstore: test w/ and w/o sync_submit_transaction
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 19:33:12 -06:00
Sage Weil
652d18f359 os/bluestore: fix alloc release timing on sync submits
If we submit the txc synchronously, we can't immediately release our
freed space to the allocator; that still needs to be done between
commit_start() and commit_finish() from the kv_sync_thread, protected
by the bdev barriers.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 19:33:12 -06:00
Sage Weil
cc37efa47c Merge pull request #11709 from iain-buclaw-sociomantic/librados_aioexec
librados: Add rados_aio_exec to the C API

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 18:50:04 -06:00
Sage Weil
060307cc49 Merge pull request #11921 from adamemerson/wip-clangtastic
build: The Light Clangtastic

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 18:49:02 -06:00
Willem Jan Withagen
ce47832cf7 common/strtol.cc: Get error testing also to work on FreeBSD
- change order of testing
 - But report the same error types.
 - Changed to report for the last error since the value is there but
   not allowed characters follow.

Error found by: run-cli-tests, because the wrong string was returned.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-11-18 00:37:37 +01:00