Commit Graph

73305 Commits

Author SHA1 Message Date
John Spray
7e5799b03b mon: don't prefix mgr summary with epoch number
Missed this one when culling the various epochs
from the human readable ceph status output.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-06-06 13:17:00 +01:00
Sage Weil
3134fbbea5 Merge pull request #14657 from chardan/jfw-wip-halflife_atomic_t-mothra
messenger,client,compressor: migrate atomic_t to std::atomic

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-05 13:35:27 -05:00
Sage Weil
92d4866b6e Merge pull request #15380 from xiexingguo/wip-cache-trim
os/bluestore: move cache_trim into MempoolThread

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-06-05 13:35:01 -05:00
Sage Weil
52a983080c Merge pull request #15409 from Liuchang0812/wip-support-mon-target-in-pybind
pybind: support mon target in pybind

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-05 13:34:33 -05:00
Sage Weil
765b9d25e4 Merge pull request #15469 from wjwithagen/wip-wjw-revert-wvla
build: revert -Wvla from #15342

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-05 13:33:41 -05:00
Sage Weil
06f097c4ac Merge pull request #15470 from xiexingguo/wip-rm-false-assert
os/bluestore: fix false asserts in Cache::trim_all()

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-05 13:33:16 -05:00
Sage Weil
ac7c2c1174 Merge pull request #15478 from xiexingguo/wip-perf-avg-time
common/perf_counters: add average time for PERFCOUNTER_TIME

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-05 13:32:53 -05:00
xie xingguo
767502e26a common/perf_counters: add average time for PERFCOUNTER_TIME
Otherwise we'll have to calculate this manually, which is annoying.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-06-05 18:25:57 +08:00
xie xingguo
34b1ff73aa os/bluestore: fix false asserts in Cache::trim_all()
These asserts are true if we are going to shutdown the BlueStore instance.
But the caller can also be something like "ceph daemon out/osd.1.asok flush_store_cache",
which can fire these asserts as a result.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-06-05 18:08:00 +08:00
Nathan Cutler
b4560572b2 Merge pull request #15231 from smithfarm/wip-20052
build/ops: rpm: make librbd1 %post scriptlet depend on coreutils

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-06-05 11:38:40 +02:00
liuchang0812
583cbbbdf7 tool/ceph: pep8 clean up
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2017-06-05 13:44:31 +08:00
liuchang0812
f71e64bee3 pybind: support mon target, and clean up tool
Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2017-06-05 13:44:31 +08:00
Kefu Chai
4722abe95c Merge pull request #15165 from badone/wip-cls-optimize-header-file-dependency
cls: optimize header file dependency

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-05 10:15:56 +08:00
Willem Jan Withagen
bdf524e05b build: revert -Wvla from #15342
- VLAs are in GCC and Clang, and are there to stay forever,
   if only to be compatible with all the software that is already
   out there.
   - Theoretical debates about VLA being hard to implement are
     long superceded by th actual implentations
 - Before setting this flag is would be required to first start
   work on fixing all the fallout/warnings that will arise from
   setting -Wvla
 - Allocating large variable/stuctures on the stack could be asking
   for trouble, but changes that ceph tools are going to be running
   on small embedded devices are rather slim.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-06-04 21:25:08 +02:00
Sage Weil
554cf8394a Merge pull request #15073 from liewegas/wip-mgr-stats
mon,mgr: extricate PGmap from monitor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2017-06-04 13:36:01 -05:00
Kefu Chai
8185bc059d qa/suites/upgrade/hammer-jewel-x: don't initially start mgr daemons
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-04 14:35:15 -04:00
Kefu Chai
69bb53b06f qa/workunits/rados/test_health_warning: misc fixes
* do not let osd shutdown itself by enlarge osd_max_markdown_count and
  shorten osd_max_markdown_period
* do not shutdown all osds in the last test. if all osds are shutdown at
  the same time. none of them will get updated osdmap after noup is
  unset. we should leave at least one of them, so the gossip protocol
  can kick in, and populate the news to all osds.

Fixes: http://tracker.ceph.com/issues/20174
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-04 14:35:15 -04:00
Kefu Chai
c24a08d6f0 mon/OSDMonitor: add a space after __func__ in log message
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-04 01:37:05 +08:00
Jason Dillaman
4cb62c2f5c Merge pull request #15443 from trociny/wip-rbd_image_options_t
librbd: remove unused rbd_image_options_t ostream operator

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-06-03 09:43:11 -04:00
Nathan Cutler
28a5098796 build/ops: rpm: make librbd1 %post scriptlet depend on coreutils
Fixes: http://tracker.ceph.com/issues/20052
Signed-off-by: Giacomo Comes <comes@naic.edu>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-06-03 15:01:43 +02:00
Kefu Chai
97d6e150e5 Merge pull request #15347 from tchaikov/wip-cmake
cmake: rgw: do not link against boost in a wholesale

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2017-06-03 20:44:24 +08:00
Jos Collin
2b56d24f52 Merge pull request #15353 from wjwithagen/wip-wjw-compare-object_catcher
test/osdc: fix comparison error and silence warning from -Wunused-value

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-06-03 17:24:40 +05:30
Kefu Chai
ba1ca1ce17 cmake: rgw: do not link against boost in a wholesale
With the new Beast frontend, RGW now has a small Boost dependency [1] which was
being addressed by statically (and unconditionally) linking *all* the Boost
libraries. This patch ensures that only the necessary Boost components are
linked.

We use the target_link_libraries(<target> <item>...) [2] syntax to ensure that the
library dependencies are transitive: i.e. "when this target is linked into
another target then the libraries linked to this target will appear on the link
line for the other target too."

[1] The boost/asio/spawn.hpp header used by rgw_asio_frontend.cc depends on
    boost::coroutine/boost::context

[2] https://cmake.org/cmake/help/v3.3/command/target_link_libraries.html#libraries-for-both-a-target-and-its-dependents

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-03 17:25:20 +08:00
Mykola Golub
c8717ca88c librbd: remove unused rbd_image_options_t ostream operator
This fixes librbd crashes currently observed on master, when
debug is on, because `rbd_image_options_t` is typedef-ed `void *`
and it's operator is used when attempting to print out an
address (`void *`) of any object.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2017-06-03 10:43:01 +02:00
xie xingguo
cb7f43cfac os/bluestore: move cache_trim into MempoolThread
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-06-03 09:24:21 +08:00
Sage Weil
cef5c1e213 mon: clean up PGMapDigest encoding
Remove compat cruft due to intitial testing on bigbang.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 15:12:18 -04:00
Casey Bodley
bf72bd8dcb Merge pull request #15119 from ceph/wip-rgw-config-docs
doc: mention certain conf vars should be in global

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-06-02 14:09:33 -04:00
Kefu Chai
8f6a526f9a selinux: clip the ceph context to ceph-mgr also
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:50 -04:00
Kefu Chai
e8b23d6852 qa/tasks: add a blacklist for flush_pg_stats()
so we don't wait for marked out osds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:50 -04:00
Kefu Chai
befe07f197 mon/OSDMonitor: filter the added creating_pgs added from pgmap
the creating_pgs added from pgmap might contains pgs whose containing
pools have been deleted. this is fine with the PGMonitor, as it has the
updated pg mapping which is consistent with itself. but it does not work
with OSDMonitor's creating_pgs, whose pg mapping is calculated by
itself. so we need to filter the pgmap's creating_pgs when adding them to
OSDMonitor's creating_pgs with the latest osdmap.get_pools().

Fixes: http://tracker.ceph.com/issues/20067
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:50 -04:00
Sage Weil
5281a0ca96 mon/MonmapMonitor: initialize new cluster monmap with default persistent features
This is similar to what we do for OSDMonitor::create_initial().

Avoid setting these initial features just for teh mon test that verifies
persistent features get set on a full quorum.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:06:49 -04:00
Greg Farnum
27c5f6b6e9 mon: mgr: remove osd_stats map from PGMapDigest
We use this information only for dumps. Stop dumping per-OSD stats as they're
not needed. In order to maintain pool "fullness" information, calculate
the OSDMap-based rule availibility ratios on the monitor and include those
values in the PGMapDigest. Also do it whenever we call dump_pool_stats_full()
on the manager.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:49 -04:00
Greg Farnum
ce38a6c17a mon: pgstat: remove unused get_osd_sum from post-luminous state
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:49 -04:00
Greg Farnum
8b164903be mon: mgr: pgstats: do available-space calculations on mgr instead of mon
This means we don't need the osd_stat map on the monitor at all. We're
about to remove it entirely!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:49 -04:00
Greg Farnum
ad630462e7 mon: pgstat: remove unused get_pg_sum interface
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:48 -04:00
Greg Farnum
9fd76f57fd mon: pgstat: remove get_num_pg_by_osd() from post-luminous state
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:48 -04:00
Greg Farnum
459ec61901 mon: pgstat: remove get_osd_stat() from post-luminous state
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:48 -04:00
Greg Farnum
fc070255e8 osdmonitor: use a new get_full_osd_counts to generate health without osdstat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-06-02 13:06:48 -04:00
Kefu Chai
d1eca5f669 mon/OSDMonitor: print pgid before looking it up in mapping
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:48 -04:00
Sage Weil
f42182c5e0 mon: handle MGetPoolStats using PGStatService
otherwise ceph_test_rados_api_stat: LibRadosStat.PoolStat will always
timeout once the cluster is switched to luminous

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:47 -04:00
Sage Weil
2de0e07c40 mon: handle MStatfs using PGStatService
otherwise ceph_test_rados_api_stat: LibRadosStat.ClusterStat will always
timeout once the cluster is switched to luminous

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:47 -04:00
Sage Weil
6a68877f59 mon/PGMap: strip out PGMapDigest compat cruft
This was needed for bigbang testing, but not for the final version.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:06:47 -04:00
Kefu Chai
aad1afcb96 mgr: reset pending_inc after applying it
we cannot apply pending_inc twice and expect the result is the same. in
other words, pg_map.apply_incremental(pending_inc) is not an idempotent
operation.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:47 -04:00
Sage Weil
af2994dcae osd: do_shutdown takes precedence over fetching more maps
This is making my osd-markdown.sh test fail reliably.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:06:47 -04:00
Sage Weil
ebc496d7f4 osd/OSDMap: more efficient PGMapTemp
Use a flat_map with pointers into a buffer with the actual data.  For a
decoded mapping, we have just two allocations (one for flat_map and one
for the encoded buffer).

This can get slow if you make lots of incremental changes after the fact
since flat_map is not efficient for modifications at large sizes.  :/

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:06:46 -04:00
Kefu Chai
60a30ff9d9 mon: print log for the creating_pgs changes
print more log messages when updating creating_pgs.

see-also: http://tracker.ceph.com/issues/20067
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-02 13:06:46 -04:00
Sage Weil
9332bb9be4 mgr: mgr_tick_period = 2
5 seconds is driving me nuts.  We cap the health message size so the
digest is now small and lightweight.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:06:46 -04:00
Sage Weil
2f881dbf2c mon/PGMap: count 'unknown' pgs
Also, count "not active" (inactive) pgs instead of active so that we
list "bad" things consistently, and so that 'inactive' is a separate
bucket of pgs than the 'unknown' ones.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:06:46 -04:00
Sage Weil
15f823b43e mgr/MgrStandby: reset subscriptions when we become non-active
This is a goofy workaround that we're also doing in Mgr::init().  Someday
we should come up with a more elegant solution.  In the meantime, this
works just fine!

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:02:54 -04:00
Sage Weil
6afca3beb2 mgr/ClusterState: make pg stat filtering less fragile
We want to drop updates for pgs for pools that don't exist.  Keep an
updated set of those pools instead of relying on the previous PGMap
having them instantiated.  (The previous map may drift due to bugs.)

Signed-off-by: Sage Weil <sage@redhat.com>
2017-06-02 13:02:54 -04:00