Commit Graph

60343 Commits

Author SHA1 Message Date
Venky Shankar
1c1f4d2a34 test / librbd: use data pool IoCtx while listing objects
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
0206e15b74 librbd / ObjectMap: use data_ctx/md_ctx pool context as required
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
75def529ee librbd / CopyupRequest: use data pool ctx during copyup
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Venky Shankar
cce335fccf librados: drop io_ctx_impl on ioctx_create/create2
close() was never called for the passed in IoCtx which
could probably result in an IoCtx leak if the original
IoCtx was a valid pool context allocated earlier.

Its kind of better to do it here rather than to leave
the destruction on the caller for better (or cleaner)
common case handling.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2016-11-20 21:21:50 +05:30
Kefu Chai
49bc71bacf Merge pull request #12080 from wjwithagen/wip-wjw-freebsd-install-deps
install-deps.sh: JQ is needed in one script

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-20 22:17:58 +08:00
Orit Wasserman
2c82b4318b Merge pull request #12076 from cbodley/wip-rgw-region-root
rgw: region conversion respects pre-existing rgw_region_root_pool
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-11-19 22:36:43 +01:00
Willem Jan Withagen
5268a3e3e0 install-deps.sh: JQ is needed in one script
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-11-19 20:10:51 +01:00
Dongsheng Yang
9024e16437 librbd: cleanup: fix a typo in CreateRequest.
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2016-11-19 08:04:53 -05:00
xie xingguo
44eaa02772 os/bluestore: refactor ctor of SharedBlob
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-19 13:52:27 +08:00
Patrick Donnelly
7b4561091e
mds: use projected path construction for access
A new CDentry will not have a parent until its projected parent is
flushed to journal. During path construction a given dentry may have no
parent yet which will cause fallbacks to be used (the inode number).
This can cause access checks to fail:

    2016-11-17 19:50:43.830207 7eff9977a700 20 Session check_access path #10000000002/3

compare to an earlier check:

    2016-11-17 19:50:43.824223 7eff9977a700 20 Session check_access path /test/1/2

This commit refactors path construction to optionally use projected
parents for the entire chain of directories. Existing use of real stable
parents is unchanged. For example, this line is the same before and
after the patch:

    2016-11-18 23:17:15.611680 7f153f97a700 12 mds.0.cache.dir(10000000002) add_null_dentry [dentry #10000000002/3 [2,head] auth NULL (dversion lock) pv=0 v=1 inode=0 0x55e0f771f5f0]

Here inode "#10000000002" has no stable parent yet. So the path is
constructed as "#10000000002/3".

One notable change in this commit is the removal of
make_path_string_projected which was only used in debugging code. Here's
an example difference:

    2016-11-17 19:50:43.827915 7eff9977a700 10 mds.0.server traverse_to_auth_dir [dir 10000000003 {#10000000003 #10000000002/3}/ [2,head] auth v=1 cv=0/0 state=1073741826|complete f() n() hs=0+0,ss=0+0 0x55f5d35e2ee0]

to:

    2016-11-18 23:17:15.617757 7f153f97a700 10 mds.0.server traverse_to_auth_dir [dir 10000000003 /test/1/2/3/ [2,head] auth v=1 cv=0/0 state=1073741826|complete f() n() hs=0+0,ss=0+0 0x55e0f7706ee0]

Fixes: http://tracker.ceph.com/issues/17858

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-11-18 23:40:22 -05:00
Casey Bodley
b81d880084 rgw: look for region_map in rgw_region_root_pool
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-18 17:04:57 -05:00
Casey Bodley
8b70dc62f4 rgw: region conversion respects pre-existing rgw_region_root_pool
Fixes: http://tracker.ceph.com/issues/17963

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-18 16:29:06 -05:00
Radoslaw Zarzynski
8bd2407e71 rgw: compilation of the ASIO front-end is enabled by default.
We're changing the default value because the previous one was
a makeshift solution to not fail Ceph compilation due to
the Beast's dependency on Boost >= 1.54 that wasn't available
on CentoOS 7. As we got the in-tree Boost we can compile
the ASIO front-end by default.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-11-18 19:23:46 +01:00
Sage Weil
64bcf92e87 Merge pull request #12057 from liewegas/wip-debug-hitset
ceph_test_rados_api_tier: dump hitset that we fail to decode

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com.>
2016-11-18 12:06:44 -06:00
Sage Weil
b02d0043cd Merge pull request #12040 from liupan1111/wip-bluestore-lat-report
os/bluestore: fix latency calculation

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-18 08:18:18 -06:00
Pan Liu
01dfc1ba07 OSD: remove 'has_inst', which has the same function as 'is_up'
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 20:49:15 +08:00
Pan Liu
e95026f6be OSD: heartbeat peers need to be updated when a new OSD added into an already existed cluster
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 20:01:42 +08:00
Jianpeng Ma
298ade8ace kv/RocksDBStore: Don't update rocksdb perf_context if rocksdb_perf disable.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-11-18 19:41:43 +08:00
Haomai Wang
eb25965b74 msg/async/rdma: ensure CephContext existed
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-18 19:20:10 +08:00
Kefu Chai
8218ff8b25 Merge pull request #11978 from tchaikov/wip-cmake-jemalloc
cmake: should link against ${ALLOC_LIBS}

Reviewed-by: Varada Raja Kumar Kari <kariraja@yahoo.com>
2016-11-18 17:53:56 +08:00
Kefu Chai
2c16786f86 Merge pull request #12048 from dzafman/wip-dz-memstore
tests: Minor clean-ups

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-18 17:47:14 +08:00
Loic Dachary
2a302ecdf4 Merge pull request #12066 from dachary/wip-16014-cot
tests: save 9 characters for asok paths

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-18 10:37:02 +01:00
Pan Liu
c7f55c40ab os/bluestore: fix a bug: when using bluestore, the output of
"ceph osd perf" is always 0ms.

Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 16:26:19 +08:00
Pan Liu
3ea4760da8 OSD: change the prefix from fs_* to os_*, because commit_latency and
apply_latency will be used not only for filestore.

Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-11-18 16:25:31 +08:00
Loic Dachary
cd72ff9f74 tests: save 9 characters for asok paths
For vstart.sh powered tests, save 9 characters in the path name
by replacing testdir/test- with td/t-

60 characters imposed by jenkins
9 characters for src/test
5 characters for td/t-

33 left (instead of 24) for the test to create asok such as out/client.admin.25327.asok

Moving these files outside of the build directory is a bad idea because
tests should only create/use files within the builddir and not write
outside of this directory. Doing so would make things more complicated
for cleanup in case the test fail and create other problems as a
consequence (filling out disk space, conflicting directories between
runs etc.).

For ceph-helpers.sh tests replace testdir with td, saving 5 characters.
This is not strictly necessary but keeps the directory names consistent:
if the developer wants to get rid of all the test leftovers, it is
enough to remove the a single directory: td.

Fixes: http://tracker.ceph.com/issues/16014

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-11-18 09:19:18 +01:00
xie xingguo
8037900c22 os/bluestore: add counter to trace extents have been removed due to compression
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-18 16:18:39 +08:00
xie xingguo
5013f9a3f5 os/bluestore: rename Extent::end() to Extent::logical_end()
and use the new method to simplify code.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-18 14:11:54 +08:00
David Zafman
db1a9434f1 Merge pull request #12058 from dachary/wip-17830-eio
test: disable osd-scrub-repair and test-erasure-eio
2016-11-17 20:44:22 -08:00
Sage Weil
99536f351c PendingReleaseNotes: note on new omap limits
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 22:28:21 -06:00
xie xingguo
8170b52e6b os/bluestore: kill dead gc-related counters
As the gc logic is deprecated by https://github.com/ceph/ceph/pull/12042

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-18 11:30:42 +08:00
Patrick Donnelly
b8b110971b
client: improve failure messages/debugging
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-11-17 21:03:02 -05:00
Sage Weil
cc37efa47c Merge pull request #11709 from iain-buclaw-sociomantic/librados_aioexec
librados: Add rados_aio_exec to the C API

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 18:50:04 -06:00
Sage Weil
060307cc49 Merge pull request #11921 from adamemerson/wip-clangtastic
build: The Light Clangtastic

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 18:49:02 -06:00
Willem Jan Withagen
ce47832cf7 common/strtol.cc: Get error testing also to work on FreeBSD
- change order of testing
 - But report the same error types.
 - Changed to report for the last error since the value is there but
   not allowed characters follow.

Error found by: run-cli-tests, because the wrong string was returned.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-11-18 00:37:37 +01:00
Sage Weil
41d46e4921 osd/ReplicatedPG: limit omap request by bytes
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 17:00:37 -06:00
Sage Weil
91bb5fbb87 osd/ReplicatedPG: osd configured limit on max omap keys during read
This doesn't apply to the ops that explicitly name keys to read; those
aren't as risky.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 17:00:36 -06:00
Dan Mick
ee08d38cb9 test: disable osd-scrub-repair and test-erasure-eio
While it is being worked on, because it frequently fails.

Refs: http://tracker.ceph.com/issues/17830

Signed-off-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-11-17 23:50:17 +01:00
Loic Dachary
8249503688 Merge pull request #10585 from zhjwpku/patch-2
doc/start/hardware-recommentdations: cosmetic

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-11-17 23:41:11 +01:00
Sage Weil
b52d602510 ceph_test_rados_api_tier: dump hitset that we fail to decode
See http://tracker.ceph.com/issues/17945

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 16:18:47 -06:00
Matt Benjamin
3f161671f7 Merge pull request #12008 from linuxbox2/civet-copy-target
cmake: produce civetweb.h, again

ok, w/upstream acks, merging--jenkins build did succeed (this is a build-only change)
2016-11-17 16:16:44 -05:00
Sage Weil
f61b7ce6a9 Merge pull request #12054 from ddiss/doc_osd_pool_restriction
doc: clarify file deletion from OSD restricted pool behaviour

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 14:57:24 -06:00
Sage Weil
a863ae1c0f osdc/Objecter: handle race between calc_target and handle_osd_map
If we fail to get an existing session and have to take the exclusive lock,
we may race with an OSDMap update and end up with a stale target.  Check
for an epoch change and, if it happens, recalculate the mapping.

Fixes: http://tracker.ceph.com/issues/17942
Reported-by: wangdongxu <wangdongxu@cmss.chinamobile.com>
Reported-by: menglingkun <menglingkun@cmss.chinamobile.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-17 14:05:26 -06:00
David Disseldorp
a138178fbe doc/cephfs/standby: fix minor typos
Signed-off-by: David Disseldorp <ddiss@suse.de>
2016-11-17 20:46:17 +01:00
David Disseldorp
f00546fee0 doc/cephfs: add note about deletion from OSD restricted pool
As described in http://tracker.ceph.com/issues/17937, a client with
restricted pool access can still delete files unless a corresponding
MDS path restriction is also in place.

Signed-off-by: David Disseldorp <ddiss@suse.de>
2016-11-17 20:45:57 +01:00
Samuel Just
64cc94c522 Merge pull request #11701 from athanatos/wip-ec-partial-overwrites
osd: EC Overwrites

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 10:54:19 -08:00
Matt Benjamin
53f6462a01 cmake: produce civetweb.h, again
The recent change to do this logic with file copy (and in src/rgw)
resolved the build problem, but now updates to the civetweb
submodule were not reflected in the build.

Move the copy into a custom target which will always source the
current submodule version at build time.

Avoid using the BYPRODUCTS option, as it is not supported in many
older cmake versions (e.g., Centos 7).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-17 13:49:14 -05:00
Samuel Just
0cf383da07 ReplicatedPG: clamp SPARSE_READ to object size for ec pool
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:41:35 -08:00
Samuel Just
0e7860b1e4 osd/: add some debugging to copyfrom
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:41:34 -08:00
Samuel Just
d8e0965cc6 ReplicatedPG: we might actually recover an object past crt on repair
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:41:34 -08:00
Samuel Just
ccbc90dd73 osd/: add projected_log to do client dup detection on not yet committed log entries
Log entries don't get added to the log for ECBackend until reads are
done, yet we still want any other requests with the same id to wait.

ReplicatedPG::update_range should consider the projected log as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:41:34 -08:00