Commit Graph

78755 Commits

Author SHA1 Message Date
Kefu Chai
73d4afbf8c Merge pull request #17747 from tchaikov/wip-qa
qa/standalone: respect $TEMPDIR

Reviewed-by: David Zafman <dzafman@redhat.com>
2017-09-20 23:08:47 +08:00
Josh Durgin
5cb4a6da3d Merge pull request #17828 from jdurgin/wip-21428
osd/PrimaryLogPG: request osdmap update in the right block

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-20 08:07:56 -07:00
Ramana Raja
f4fc172259 ceph_volume_client: perform snapshot operations in
... client configured snap directory name, instead of in hard-coded
'.snap' directory.

Fixes: http://tracker.ceph.com/issues/21476

Signed-off-by: Ramana Raja <rraja@redhat.com>
2017-09-20 20:31:47 +05:30
Mohamad Gebai
2e5aed5fc6 doc: update link to placing-different-pools
Signed-off-by: Mohamad Gebai <mgebai@suse.com>
2017-09-20 13:19:55 +02:00
Yan Jun
f8b5e4cbe8 osd/PrimaryLogPG: drop unused variables
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2017-09-20 17:05:23 +08:00
Yan Jun
bebea1c5d6 osd/PrimaryLogPG: avoid accessing value of map.end()
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2017-09-20 17:04:56 +08:00
YunfeiGuan
ffa1f9998d librados: Fix a potential risk of buffer::list::claim_prepend(list& bl, unsigned int flags)
Last_p should point to the _buffers.begin() when push front a ptr to _buffers,
which make the bufferlist can be full amount copied.

Fixes: http://tracker.ceph.com/issues/21338
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
2017-09-20 15:39:27 +08:00
Josh Durgin
afc6624f76 osd/PrimaryLogPG: request osdmap update in the right block
Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-09-20 01:18:35 -04:00
Kefu Chai
f27251432a Merge pull request #17785 from dzafman/wip-add-repair
test: Fix ceph-objectstore-tool usage check

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-20 12:35:16 +08:00
xie xingguo
05e92c0bfc osd/osd_type: get_clone_bytes - inline size() for overlapping size
This is faster and simpler. A similar issue can be found at:
https://github.com/ceph/ceph/pull/17807

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 11:31:35 +08:00
Kefu Chai
b49dd813a9 Merge pull request #17138 from Liuchang0812/wip-updata-osd-hostname-in-metaupdater
mgr: preventing blank hostname in DaemonState

Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-20 11:22:49 +08:00
Sage Weil
8399833f3d osd/OSDMap: ignore xinfo if features == 0
Some old bug (e.g., http://tracker.ceph.com/issues/20751) could
result in an UP+EXISTS osd having features==0.  If that happens,
we shouldn't crash the mon, which (reasonably) does

   if (osdmap.get_epoch()) {
     if (osdmap.get_num_up_osds() > 0) {
       assert(osdmap.get_up_osd_features() & CEPH_FEATURE_MON_STATEFUL_SUB);
       check_subs();
     }
   }

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-20 13:11:10 +10:00
Kefu Chai
497c845c66 Merge pull request #17738 from jtlayton/wip-jlayton-20988
common: lockdep fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-20 11:11:02 +08:00
Kefu Chai
c2ef12d42a Merge pull request #17753 from tchaikov/wip-osd-cleanup
osd: cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 11:06:45 +08:00
Kefu Chai
176c01f15e Merge pull request #17787 from tchaikov/wip-os-release-wo-version_id
common/util: do not print error if VERSION_ID is missing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-09-20 11:02:00 +08:00
Kefu Chai
fa85dd0565 Merge pull request #17788 from tchaikov/wip-cmake-rocksdb-tcmalloc
cmake: identify the possible incompatibility of rocksdb and tcmalloc

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-09-20 10:30:37 +08:00
Kefu Chai
ff8d6a730b cmake: error out if rocksdb is incompatible w/ tcmalloc
the commit d406f228 in gperf implements a c11 feature used by a
recent change in rocksdb: 16e03882, which uses aligned_alloc().
and 16e03882 in rocksdb was merged after v5.7 was tagged, while
16e03882 in gperf was merged after v2.6.1 was tagged.

because aligned_alloc() is not implemented by tcmalloc until the
not-yet-released 2.6.2, if we call aligned_alloc() in an application
linked against tcmalloc, what gets called will be the glibc's
aligned_alloc(). but if we free() the memory chunk allocated by
aligned_alloc(), the tcmalloc's implementation kicks in, then
InvalidFree() is called, because the memory chunk being freed was
allocated by tcmalloc. in short, "mixing allocators", quote from
Dan Mick.

in rocksdb, aligned_alloc() is used if _ISOC11_SOURCE is defined, this
makes sense, because aligned_alloc() is a C11 function. we could avoid
using it by not defining _ISOC11_SOURCE. but as long as _GNU_SOURCE is
defined, glibc defines _ISOC11_SOURCE. and libstdc++ requires
_GNU_SOURCE, because it uses a fair amount of GNU extensions.

Fixes: http://tracker.ceph.com/issues/21422
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-20 10:23:16 +08:00
xie xingguo
b60d3498e2 osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC
We now set full flag if a pool is currently running out of space and
set both full and full_no_quota flags if it is running out of quota.
Therefore the full_no_quota flag should be instead used to uniquely
identify whether we are running out of quota or not.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 08:49:42 +08:00
Vasu Kulkarni
e11a1f773d qa/tests: skip test that use public bucket landsat-pds from AWS
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-19 17:12:15 -07:00
Vasu Kulkarni
184d5e6bad qa/tests: update required ceph-ansbile vars
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-19 17:12:09 -07:00
Vasu Kulkarni
077904a0b5 qa/tests: Fix ceph-ansible upstream vars
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-19 17:11:56 -07:00
Josh Durgin
311373a12a Merge pull request #17795 from jdurgin/wip-21428
osd: request new map from PG when needed

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-19 16:43:24 -07:00
Neha Ojha
11d8dfe591 qa/suites/rados/perf: create pool with lower pg_num
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-09-19 16:40:45 -07:00
Josh Durgin
dd33360e5a osd: request new map from PG when needed
The fast dispatch refactor in 3cc48278bf
eliminated the osdmap subscription in the ms_fast_dispatch path, which
meant ops could reach a PG without having the latest map. In a cluster
with few osdmap updates, where the monitor fails to send a new map to
an osd (it tries one random osd), this can result in indefinitely
blocked requests.

Fix this by adding an OSDService mechanism for scheduling a new osdmap
subscription request.

Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-09-19 14:27:29 -04:00
Sage Weil
25e2fe3dd5 Merge pull request #17607 from liewegas/wip-post-luminous-more-mon
mon: post-luminous cleanup (part 3 of ?)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-09-19 13:19:06 -05:00
Sage Weil
1722d458dd Merge pull request #16913 from liewegas/wip-20944
os/filestore: fix device/partition metadata detection

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-09-19 13:00:33 -05:00
Sage Weil
6767f841e5 Merge pull request #17427 from liewegas/wip-pg-num-limits
mon/OSDMonitor: implement cluster pg limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-09-19 12:57:10 -05:00
Sage Weil
5a32ef77e1 Merge pull request #17760 from liewegas/wip-21410-b
Revert "osd/OSDMap: allow bidirectional swap of pg-upmap-items"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-19 12:31:44 -05:00
Matt Benjamin
1456dbfeab Merge pull request #15369 from rzarzynski/wip-rgw-custom-errmsgs-2
rgw: improve handling of Swift's error messages and limits
2017-09-19 12:09:38 -04:00
Matt Benjamin
032f8cc804 Merge pull request #12704 from rzarzynski/wip-rgw-per-storage-policy-stats
rgw: add support for Swift's per storage policy statistics
2017-09-19 12:04:28 -04:00
huangjun
fa40add7f0 qa/ceph_manager: check pg state again before timedout
Pg state maybe all in active+clean when no recovering going on,
so check it again before timedout.

Fixes: http://tracker.ceph.com/issues/21294

Signed-off-by: huangjun <huangjun@xsky.com>
2017-09-20 00:04:04 +08:00
Josh Durgin
401ccd5966 Merge pull request #17786 from neha-ojha/wip-cbt-teuthology-integration-cntd
qa/suites/rados/perf: add optimized settings

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-09-19 08:35:10 -07:00
amitkuma
73099dd7ee librbd,librados,tools: Initialize aio_comp,m_ variables
** 1414517 Uninitialized pointer field
CID 1414517 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member aio_comp is not initialized
in this constructor nor in any functions that it calls.

** 1274324 Uninitialized scalar field
2. uninit_member: Non-static class member m_pool_id is not initialized
in this constructor nor in any functions that it calls.
CID 1274324 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member m_snap_seq is not initialized
in this constructor nor in any functions that it calls.

** 1356906 Uninitialized scalar field
CID 1356906 (#1-2 of 2): Uninitialized scalar field (UNINIT_CTOR)
5. uninit_member: Non-static class member m_snap_ret is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>
2017-09-19 19:59:44 +05:30
amitkuma
98ff21a1f4 librbd: Initialize on_finish,locker,force,snap_id
** 396154 Uninitialized pointer field
CID 1396154 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member on_finish is not initialized
in this constructor nor in any functions that it calls.

** 1396158 Uninitialized pointer field
2. uninit_member: Non-static class member snap_id is not initialized
in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member force is not initialized
in this constructor nor in any functions that it calls.
CID 1396158 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
6. uninit_member: Non-static class member on_finish is not initialized
in this constructor nor in any functions that it calls.

** 1399593 Uninitialized pointer field
2. uninit_member: Non-static class member locker is not initialized
in this constructor nor in any functions that it calls.
CID 1399593 (#2 of 2): Uninitialized pointer field (UNINIT_CTOR)
4. uninit_member: Non-static class member on_finish is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>
2017-09-19 19:45:48 +05:30
Radoslaw Zarzynski
9b06985cae rgw: rename the configurables for metadata limits to start with rgw_.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2017-09-19 14:47:16 +02:00
Sage Weil
096aab7a1a common/options: document objecter, filer, and journal options
Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-19 07:31:59 -05:00
Radoslaw Zarzynski
5e5c0677ba rgw: calculate and print Swift's X-Account-Storage-Policy-* headers.
Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-09-19 13:51:17 +02:00
Radoslaw Zarzynski
de7a8679af rgw: bucket linking stores also the info about a placement rule.
Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-09-19 13:51:16 +02:00
Radoslaw Zarzynski
5eca89f08d rgw: convey placement rule in RGWBucketEnt and cls_user_bucket_entry.
Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-09-19 13:51:16 +02:00
Radoslaw Zarzynski
545dabe31a rgw: clean-up around and implement the move semantics in RGWBucketEnt.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2017-09-19 13:51:07 +02:00
xie xingguo
6d437c74e3 osd/PrimaryLogPG: kill add_interval_usage
Use inline size() member of interval_set instead, which is faster and simpler.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-19 18:29:44 +08:00
Kefu Chai
f59dc0acfc Merge pull request #17508 from liewegas/wip-crushrule-convert
mon: more aggressively convert crush rulesets -> distinct rules

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-19 18:18:55 +08:00
Kefu Chai
42bb43b206 Merge pull request #17802 from ygtzf/option-default-value-error
doc: update default value of option mon_sync_timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
2017-09-19 17:18:08 +08:00
Yao Guotao
f22c502553 doc: update default value of option mon_sync_timeout
Signed-off-by: Yao Guotao <yaoguot@gmail.com>
2017-09-19 04:47:00 -04:00
huangjun
4d30d02e52 qa/suites/rest/basic/tasks/rest_test: more whitelisting
Fixes: http://tracker.ceph.com/issues/21425

Signed-off-by: huangjun <huangjun@xsky.com>
2017-09-19 16:13:34 +08:00
Yan, Zheng
69adaabfac client: set client_try_dentry_invalidate to false by default
By default, ceph-fuse uses side effect of 'dentry invalidation' to
trim kernel dcache if it runs on kernel < 3.18. The implemention of
kernel function d_invalidate() changed in 3.18 kernel, the method no
longer works for upstream kernel >= 3.18.

RHEL 3.10 kernel includes backport of patches that change implemention
of d_invalidate(). So checking kernel version to decide if 'dentry
invalidation' method works is unreliable.

Fixes: http://tracker.ceph.com/issues/21423
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-09-19 14:58:49 +08:00
Jos Collin
6854af9ab0 test: silence warning from -Wsign-compare
Fixed warnings:
ceph/src/test/test_xlist.cc:46:3:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1392:11: warning: comparison between
signed and unsigned integer expressions [-Wsign-compare]

ceph/src/test/librados/tier.cc:1360:5:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1526:28: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

ceph/src/test/librados/tier.cc:1365:2:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1526:28: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-09-19 12:27:37 +05:30
Kefu Chai
c999abd7a8 cmake: requires rockdb version 5.8
we've updated the rockdb wrapper on ceph side to be compatible with
the latest version of rocksdb upstream. so ceph is not compatible with
older version of rocksdb.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
987c1b3ec2 cmake: detect rocksdb's version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
58179f098b cmake: detect gperf-tools' version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00