Commit Graph

78690 Commits

Author SHA1 Message Date
Kefu Chai
b49dd813a9 Merge pull request #17138 from Liuchang0812/wip-updata-osd-hostname-in-metaupdater
mgr: preventing blank hostname in DaemonState

Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-20 11:22:49 +08:00
Sage Weil
8399833f3d osd/OSDMap: ignore xinfo if features == 0
Some old bug (e.g., http://tracker.ceph.com/issues/20751) could
result in an UP+EXISTS osd having features==0.  If that happens,
we shouldn't crash the mon, which (reasonably) does

   if (osdmap.get_epoch()) {
     if (osdmap.get_num_up_osds() > 0) {
       assert(osdmap.get_up_osd_features() & CEPH_FEATURE_MON_STATEFUL_SUB);
       check_subs();
     }
   }

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-20 13:11:10 +10:00
Kefu Chai
497c845c66 Merge pull request #17738 from jtlayton/wip-jlayton-20988
common: lockdep fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-20 11:11:02 +08:00
Kefu Chai
c2ef12d42a Merge pull request #17753 from tchaikov/wip-osd-cleanup
osd: cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 11:06:45 +08:00
Kefu Chai
176c01f15e Merge pull request #17787 from tchaikov/wip-os-release-wo-version_id
common/util: do not print error if VERSION_ID is missing

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-09-20 11:02:00 +08:00
Kefu Chai
fa85dd0565 Merge pull request #17788 from tchaikov/wip-cmake-rocksdb-tcmalloc
cmake: identify the possible incompatibility of rocksdb and tcmalloc

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-09-20 10:30:37 +08:00
Kefu Chai
ff8d6a730b cmake: error out if rocksdb is incompatible w/ tcmalloc
the commit d406f228 in gperf implements a c11 feature used by a
recent change in rocksdb: 16e03882, which uses aligned_alloc().
and 16e03882 in rocksdb was merged after v5.7 was tagged, while
16e03882 in gperf was merged after v2.6.1 was tagged.

because aligned_alloc() is not implemented by tcmalloc until the
not-yet-released 2.6.2, if we call aligned_alloc() in an application
linked against tcmalloc, what gets called will be the glibc's
aligned_alloc(). but if we free() the memory chunk allocated by
aligned_alloc(), the tcmalloc's implementation kicks in, then
InvalidFree() is called, because the memory chunk being freed was
allocated by tcmalloc. in short, "mixing allocators", quote from
Dan Mick.

in rocksdb, aligned_alloc() is used if _ISOC11_SOURCE is defined, this
makes sense, because aligned_alloc() is a C11 function. we could avoid
using it by not defining _ISOC11_SOURCE. but as long as _GNU_SOURCE is
defined, glibc defines _ISOC11_SOURCE. and libstdc++ requires
_GNU_SOURCE, because it uses a fair amount of GNU extensions.

Fixes: http://tracker.ceph.com/issues/21422
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-20 10:23:16 +08:00
xie xingguo
b60d3498e2 osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC
We now set full flag if a pool is currently running out of space and
set both full and full_no_quota flags if it is running out of quota.
Therefore the full_no_quota flag should be instead used to uniquely
identify whether we are running out of quota or not.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 08:49:42 +08:00
Vasu Kulkarni
e11a1f773d qa/tests: skip test that use public bucket landsat-pds from AWS
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-19 17:12:15 -07:00
Vasu Kulkarni
184d5e6bad qa/tests: update required ceph-ansbile vars
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-19 17:12:09 -07:00
Vasu Kulkarni
077904a0b5 qa/tests: Fix ceph-ansible upstream vars
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-19 17:11:56 -07:00
Josh Durgin
311373a12a Merge pull request #17795 from jdurgin/wip-21428
osd: request new map from PG when needed

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-19 16:43:24 -07:00
Neha Ojha
11d8dfe591 qa/suites/rados/perf: create pool with lower pg_num
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-09-19 16:40:45 -07:00
Josh Durgin
dd33360e5a osd: request new map from PG when needed
The fast dispatch refactor in 3cc48278bf
eliminated the osdmap subscription in the ms_fast_dispatch path, which
meant ops could reach a PG without having the latest map. In a cluster
with few osdmap updates, where the monitor fails to send a new map to
an osd (it tries one random osd), this can result in indefinitely
blocked requests.

Fix this by adding an OSDService mechanism for scheduling a new osdmap
subscription request.

Fixes: http://tracker.ceph.com/issues/21428
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-09-19 14:27:29 -04:00
Sage Weil
25e2fe3dd5 Merge pull request #17607 from liewegas/wip-post-luminous-more-mon
mon: post-luminous cleanup (part 3 of ?)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-09-19 13:19:06 -05:00
Sage Weil
1722d458dd Merge pull request #16913 from liewegas/wip-20944
os/filestore: fix device/partition metadata detection

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-09-19 13:00:33 -05:00
Sage Weil
6767f841e5 Merge pull request #17427 from liewegas/wip-pg-num-limits
mon/OSDMonitor: implement cluster pg limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-09-19 12:57:10 -05:00
Sage Weil
5a32ef77e1 Merge pull request #17760 from liewegas/wip-21410-b
Revert "osd/OSDMap: allow bidirectional swap of pg-upmap-items"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-19 12:31:44 -05:00
Matt Benjamin
1456dbfeab Merge pull request #15369 from rzarzynski/wip-rgw-custom-errmsgs-2
rgw: improve handling of Swift's error messages and limits
2017-09-19 12:09:38 -04:00
Matt Benjamin
032f8cc804 Merge pull request #12704 from rzarzynski/wip-rgw-per-storage-policy-stats
rgw: add support for Swift's per storage policy statistics
2017-09-19 12:04:28 -04:00
huangjun
fa40add7f0 qa/ceph_manager: check pg state again before timedout
Pg state maybe all in active+clean when no recovering going on,
so check it again before timedout.

Fixes: http://tracker.ceph.com/issues/21294

Signed-off-by: huangjun <huangjun@xsky.com>
2017-09-20 00:04:04 +08:00
Josh Durgin
401ccd5966 Merge pull request #17786 from neha-ojha/wip-cbt-teuthology-integration-cntd
qa/suites/rados/perf: add optimized settings

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-09-19 08:35:10 -07:00
amitkuma
73099dd7ee librbd,librados,tools: Initialize aio_comp,m_ variables
** 1414517 Uninitialized pointer field
CID 1414517 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member aio_comp is not initialized
in this constructor nor in any functions that it calls.

** 1274324 Uninitialized scalar field
2. uninit_member: Non-static class member m_pool_id is not initialized
in this constructor nor in any functions that it calls.
CID 1274324 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
4. uninit_member: Non-static class member m_snap_seq is not initialized
in this constructor nor in any functions that it calls.

** 1356906 Uninitialized scalar field
CID 1356906 (#1-2 of 2): Uninitialized scalar field (UNINIT_CTOR)
5. uninit_member: Non-static class member m_snap_ret is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>
2017-09-19 19:59:44 +05:30
amitkuma
98ff21a1f4 librbd: Initialize on_finish,locker,force,snap_id
** 396154 Uninitialized pointer field
CID 1396154 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
2. uninit_member: Non-static class member on_finish is not initialized
in this constructor nor in any functions that it calls.

** 1396158 Uninitialized pointer field
2. uninit_member: Non-static class member snap_id is not initialized
in this constructor nor in any functions that it calls.
4. uninit_member: Non-static class member force is not initialized
in this constructor nor in any functions that it calls.
CID 1396158 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
6. uninit_member: Non-static class member on_finish is not initialized
in this constructor nor in any functions that it calls.

** 1399593 Uninitialized pointer field
2. uninit_member: Non-static class member locker is not initialized
in this constructor nor in any functions that it calls.
CID 1399593 (#2 of 2): Uninitialized pointer field (UNINIT_CTOR)
4. uninit_member: Non-static class member on_finish is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>
2017-09-19 19:45:48 +05:30
Radoslaw Zarzynski
9b06985cae rgw: rename the configurables for metadata limits to start with rgw_.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2017-09-19 14:47:16 +02:00
Sage Weil
096aab7a1a common/options: document objecter, filer, and journal options
Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-19 07:31:59 -05:00
Radoslaw Zarzynski
5e5c0677ba rgw: calculate and print Swift's X-Account-Storage-Policy-* headers.
Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-09-19 13:51:17 +02:00
Radoslaw Zarzynski
de7a8679af rgw: bucket linking stores also the info about a placement rule.
Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-09-19 13:51:16 +02:00
Radoslaw Zarzynski
5eca89f08d rgw: convey placement rule in RGWBucketEnt and cls_user_bucket_entry.
Fixes: http://tracker.ceph.com/issues/17932
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-09-19 13:51:16 +02:00
Radoslaw Zarzynski
545dabe31a rgw: clean-up around and implement the move semantics in RGWBucketEnt.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2017-09-19 13:51:07 +02:00
xie xingguo
6d437c74e3 osd/PrimaryLogPG: kill add_interval_usage
Use inline size() member of interval_set instead, which is faster and simpler.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-19 18:29:44 +08:00
Kefu Chai
f59dc0acfc Merge pull request #17508 from liewegas/wip-crushrule-convert
mon: more aggressively convert crush rulesets -> distinct rules

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-19 18:18:55 +08:00
Kefu Chai
42bb43b206 Merge pull request #17802 from ygtzf/option-default-value-error
doc: update default value of option mon_sync_timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
2017-09-19 17:18:08 +08:00
Yao Guotao
f22c502553 doc: update default value of option mon_sync_timeout
Signed-off-by: Yao Guotao <yaoguot@gmail.com>
2017-09-19 04:47:00 -04:00
huangjun
4d30d02e52 qa/suites/rest/basic/tasks/rest_test: more whitelisting
Fixes: http://tracker.ceph.com/issues/21425

Signed-off-by: huangjun <huangjun@xsky.com>
2017-09-19 16:13:34 +08:00
Yan, Zheng
69adaabfac client: set client_try_dentry_invalidate to false by default
By default, ceph-fuse uses side effect of 'dentry invalidation' to
trim kernel dcache if it runs on kernel < 3.18. The implemention of
kernel function d_invalidate() changed in 3.18 kernel, the method no
longer works for upstream kernel >= 3.18.

RHEL 3.10 kernel includes backport of patches that change implemention
of d_invalidate(). So checking kernel version to decide if 'dentry
invalidation' method works is unreliable.

Fixes: http://tracker.ceph.com/issues/21423
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-09-19 14:58:49 +08:00
Jos Collin
6854af9ab0 test: silence warning from -Wsign-compare
Fixed warnings:
ceph/src/test/test_xlist.cc:46:3:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1392:11: warning: comparison between
signed and unsigned integer expressions [-Wsign-compare]

ceph/src/test/librados/tier.cc:1360:5:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1526:28: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

ceph/src/test/librados/tier.cc:1365:2:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1526:28: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-09-19 12:27:37 +05:30
Kefu Chai
c999abd7a8 cmake: requires rockdb version 5.8
we've updated the rockdb wrapper on ceph side to be compatible with
the latest version of rocksdb upstream. so ceph is not compatible with
older version of rocksdb.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
987c1b3ec2 cmake: detect rocksdb's version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
58179f098b cmake: detect gperf-tools' version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
aa603621bd cmake: extract BuildRocksDB into BuildRocksDB.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
da8ce07a21 cmake: enable find_package(perftools) to check non-default dir
so it checks $ENV{GPERF_ROOT} first.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Neha Ojha
2635e7a591 qa/suites/rados/perf: add optimized settings
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-09-18 15:53:28 -07:00
David Zafman
0364ae104a test: Fix ceph-objectstore-tool usage check
Caused by: c7b7a1f04f

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-18 15:29:22 -07:00
John Spray
15ad33f99b vstart: create MDSs before filesystems
This avoids spurious complaints about not
enough MDS daemons in the logs.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-18 17:30:54 -04:00
John Spray
511439b515 mon/MDSMonitor: fix clog msgs on MDS rank add/rm
Because these were generated before the actual
change, the get_num_in values did not make sense.

Fixes: http://tracker.ceph.com/issues/21421
Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-18 17:30:28 -04:00
Matt Benjamin
4639b96cae Merge pull request #15273 from rzarzynski/wip-rgw-18977
rgw: list_objects() honors end_marker regardless of namespace.
2017-09-18 16:10:08 -04:00
Matt Benjamin
6994e1dc5f Merge pull request #17320 from rzarzynski/wip-rgw-21148
rgw: add support for Swift's reversed account listings
2017-09-18 15:51:43 -04:00
Patrick Donnelly
bc3dd60e99
client: move Fh init to ctor
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-18 12:48:12 -07:00
Yuri Weinstein
f1f960e443 Merge pull request #17777 from cbodley/wip-qa-hadoop
qa/rgw: hadoop-s3a suite targets centos_latest

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2017-09-18 12:32:14 -07:00