Commit Graph

59723 Commits

Author SHA1 Message Date
David Zafman
eb73dd4736 doc, test: Add schemas for list-inconsistent-* rados command output
If jsonschema cmd is available use it to test output against schema

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 09:38:15 -08:00
David Zafman
b7bacd219b test: Update testing for new list-inconsistent-obj output
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 09:38:15 -08:00
David Zafman
df3ff6dafe rados, osd: Improve attrs output of list-inconsistent-obj
Persist the user_version and shard id of scrubbed obj
Rados command dump inconsistent obj's version and shard-id
    so they can be passed to repair command
Rados list-inconsistent-obj output of attrs
    Make attrs an array since there are more than one
    Use base64 encode for values with non-printable chars
    Add indication if base64 encoding used
Add checking for ss_attr_missing and ss_attr_corrupted
    Rename attr errors to attr_key_mismatch and attr_value_mismatch
Add missing size_mismatch_oi scrub checking
For erasure coded pools add ec_size_error and ec_hash_error not just read_error
Use oi_attr_missing and oi_attr_corrupted just like list-inconsistent-snap does
Pick an object info based on version and use that to find specific shards in error
    Check for object info inconsistency which should be rare
Make all errors based on comparing shards to each other object errors
    We don't want give the impression that we've picked the correct one

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 09:38:15 -08:00
David Zafman
b39001ec6f osd: Fix logging to help with diagnostics
These messages were not outputing the intended information

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
David Zafman
d0503a52d0 test: Fix use of wait_for_clean()
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
David Zafman
92e982c719 common: Change cleanbin() to use base64 encoding, update ceph-objectstore-tool
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
David Zafman
5c79074ffa common: Move cleanbin() function to common/util.cc
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
David Zafman
907e79e2b7 test: Add test support for deep-scrub
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
David Zafman
a74418a29e common: Fix indentation
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
David Zafman
30f8b0d659 osd: Handle corrupt attributes in get_object_context()
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-08 15:16:52 -08:00
Gregory Farnum
b2fe54bfaf Merge pull request #11681 from stiopaa1/mds_mdstableclient_addConsttoMemFun
mds/MDSTableClient.h: add const to member funct

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-11-08 14:49:34 -08:00
Sage Weil
267cd5a6a2 Merge pull request #11818 from liewegas/wip-bluestore-aio-stalls3
os/bluestore: optional debug mode to identify aio stalls
2016-11-08 15:22:06 -06:00
Sage Weil
5bf2ad74c3 Merge pull request #11826 from ovh/wip-doc-update-xattr-config-documentation
doc: Update filestore xattr config documentation.

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-08 14:46:27 -06:00
Matt Benjamin
3420cae593 Merge pull request #11831 from ceph/wip-dang-rgw-pybind
RGW Python bindings - use explicit array
2016-11-08 14:22:05 -05:00
Daniel Gryniewicz
b655c31838 RGW Python bindings - use explicit array
Older Cython doesn't to implicit simple arrays, so use an explicit array
for stat output.

Revert change removing stat

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2016-11-08 13:50:03 -05:00
Yehuda Sadeh
98c2f39a7b Merge pull request #11816 from cbodley/wip-17812
rgw: delete entries_index in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-08 09:31:26 -08:00
Sage Weil
a38d53e1d2 Merge pull request #11817 from linuxbox2/wip-boost-3
boost: embedded
2016-11-08 10:13:06 -06:00
Kefu Chai
e4feeb507a Merge pull request #11825 from ceph/wip-11820
pybind: fix build failure of rgwfile binding

Reviewed-by: Haomai Wang <haomai@xsky.com>
2016-11-08 22:15:22 +08:00
Bartłomiej Święcki
253df6b2d7 doc: Update filestore xattr config documentation.
Signed-off-by: Bartłomiej Święcki <bartlomiej.swiecki@corp.ovh.com>
2016-11-08 13:41:24 +01:00
Kefu Chai
a439e9abac pybind: fix build failure of rgwfile binding
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-11-08 19:39:33 +08:00
Sage Weil
9197475b23 Merge pull request #9624 from gaurav36/coverity-1355603-dead-code
coverity fix: fixing few coverity issue
2016-11-07 20:59:30 -06:00
Sage Weil
37b5c6aeab Merge pull request #11798 from stiopaa1/mds_fsmap_preventcopyinginloop
mds/FSMap.cc: prevent unneeded copy of map entry

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-07 20:44:08 -06:00
Sage Weil
c9f7a5ae4f Merge pull request #11802 from stiopaa1/mds_fsmap_constReference
mds/FSMap.h: add const and reference

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-07 20:43:49 -06:00
Sage Weil
f1cc1904ba Merge pull request #11819 from stiopaa1/mds_migrator_addConstTomemFun
mds/Migrator.h: add const to member functions

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-07 20:39:34 -06:00
Sage Weil
4bc6bad6de Merge pull request #11807 from xiexingguo/xxg-wip-bs-1107
os/bluestore: speed up omap-key generation for same onode

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-07 20:38:20 -06:00
xie xingguo
db769dca88 os/bluestore: speed up omap-key generation for same onode
This is useful if we have to batch-process omap kvs.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-08 09:00:34 +08:00
Sage Weil
27efa84c0e Merge pull request #11754 from athanatos/wip-17789
osd/ReplicatedPG: do_update_log_missing: take the pg lock in the callback

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-07 17:07:08 -06:00
Sage Weil
3b1b5a9c7c os/bluestore/KernelDevice: debug support to detect stalled aios
This is additional bookkeeping to track the oldest aio in flight, so we
make it optional, and disabled by default.  It is useful for identifying
buggy kernel aio or bad devices.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-07 17:57:01 -05:00
Sage Weil
3bbfcf153f os/fs: add aio_list_t type
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-07 17:51:17 -05:00
Sage Weil
9cb879ad2b os/fs: uninline aio methods
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-07 17:51:01 -05:00
Michal Jarzabek
be26aaa794 mds/Migrator.h: add const to member functions
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-11-07 22:49:49 +00:00
Matt Benjamin
9dd5b70abc boost: pull up boost to enable boost_python build
Pullup to just the merges on Sep 10 fixes an obvious compile
problem, but there are other relevant changes.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-07 16:01:29 -05:00
Matt Benjamin
132469c4e9 Adapt for ceph-manager
The ceph manager change added a separate Boost dependency
check, add it to the combined section.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-07 15:57:02 -05:00
Sage Weil
feefd35853 do_cmake.sh: set BOOST_J to $(nproc)
This lets us build boost with all processors.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-07 15:56:53 -05:00
Matt Benjamin
81e1fcc4f0 boost: update boost to 8/29
Pull-up Boost to the commit which merged the upstream fix for
upstream Boost/iostreams bug #11860 (template arity warnings),
per discussion in core.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-07 15:56:43 -05:00
Matt Benjamin
5f869450ea cmake: find and build bundled boost
Build Boost using the src/boost submodule, unless overridden
by -DWITH_SYSTEM_BOOST.

If -DBOOST_J=<n> is provided, builds Boost with <n> jobs.

Boost builds in the configured Build directory.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-07 15:56:29 -05:00
Matt Benjamin
d8dff81c3e boost: add submodule at tag boost-1.61.0
This implements option #4 for external boost, based on upstream
discussion.  In option #4:

1. boost is added as a submodule
2. builds default to using the attached boost module
3. building against a system-provided boost is supported, but
   must be configured explicitly

Because all of the boost components are attached as nested
submodules in the upstream boost repository, neither the nested
submodules nor the root boost submodule have been cloned into
modules in github.com/ceph (acked by Sage).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-11-07 15:56:17 -05:00
John Spray
f80c7a8f20 Merge pull request #11647 from ceph/wip-jlayton-cephfs
libcephfs client API overhaul and update

Reviewed-by: John Spray <john.spray@redhat.com>
2016-11-07 21:07:40 +01:00
Yehuda Sadeh
351911a20e Merge pull request #11425 from weiqiaomiao/wqm-rgw-oplog
rgw: fix the field 'total_time'  of log entry in log show opt

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-11-07 10:45:10 -08:00
Yehuda Sadeh
ace03b97ab Merge pull request #8712 from diwakargoel/master
doc: Correcting the sample python tempurl generation script.

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-07 10:41:04 -08:00
Yehuda Sadeh
82740e9be3 Merge pull request #11648 from cbodley/wip-17698
rgw: fix for bucket delete racing with mdlog sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2016-11-07 10:36:29 -08:00
Yehuda Sadeh
996e59f05c Merge pull request #11598 from cbodley/wip-rgw-bucket-sync
rgw multisite: move lease up to RunBucketSync instead of child crs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-11-07 10:35:26 -08:00
Casey Bodley
9c731ed6b6 rgw: delete entries_index in RGWFetchAllMetaCR
use a unique_ptr to ensure the entries_index is freed on destruction

Fixes: http://tracker.ceph.com/issues/17812

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-11-07 13:08:11 -05:00
Loic Dachary
b51aec70fb Merge pull request #11715 from dzafman/wip-test-ec-eio
Fix racey test by setting noout flag (tracker 17757)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-11-07 17:26:53 +01:00
Daniel Gryniewicz
28a338a381 Merge pull request #11624 from yuyuyu101/wip-rgwfile-pybind
Wip rgwfile pybind
2016-11-07 11:08:35 -05:00
Haomai Wang
b4fc122754 pybind/rgwfile: change description
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-08 00:04:07 +08:00
Haomai Wang
334d125f78 pybind/rgwfile: fixes hardcode way to find include dir
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-08 00:00:57 +08:00
Haomai Wang
aef68c458e pybind/rgwfile: raise Exception when failing to umount
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-07 23:58:59 +08:00
Haomai Wang
7a4ecd8323 pybind/rgwfile: change FileHandler to FileHandle
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-07 23:56:05 +08:00
Casey Bodley
a50ede03f6 Merge pull request #11794 from rzarzynski/wip-rgw-17803
rgw: fix uncompressed object size deduction in RGWRados::copy_obj_data.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-11-07 10:13:45 -05:00