Commit Graph

78690 Commits

Author SHA1 Message Date
John Spray
125294ab9d mgr/influx: enable self-test without dependencies
The idea of self-test commands is that they're self
contained and just exercise the module's calls
to the Ceph-side.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:21 -04:00
John Spray
59b48e7660 mgr/influx: revise perf counter handling
- Use new get_all_perf_counters path
- Consequently get counters for all daemons, not just OSD
- Tag stats with ceph_daemon rather than osd_id, as some
  stats appear from more than one daemon type
- Remove summing of perf counters, external TSDB and/or queries
  can do this.
- Remove mgr_id tag: this would change depending on which
  mgr was active, which is certainly not desirable.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:21 -04:00
John Spray
8d1277fa5c mgr: omit module list in beacon logging
This is useful in itself, but awkward when dealing
with logs generally, because it means that when you
grep on the name of a module, you get mostly beacon
messages rather than the log messages from the
module.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:21 -04:00
John Spray
39ab28ed47 mgr: define perf counter constants in mgr_module
So that modules can consume perf counter data
intelligently without having to hunt around
in C land for these constants and redefine them.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:20 -04:00
John Spray
30a74ce343 ceph.in: use PRIO_INTERESTING as daemonperf threshold
Using PRIO_USEFUL as the threshold for what goes into
time series databases.  I'm claiming that we have
more "useful" counters than fit on the screen,
so daemonperf's "a screen's worth" threshold
should be at the "interesting" level.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:20 -04:00
John Spray
29a71c35c3 mon: set some priorities on perf counters
Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:20 -04:00
John Spray
eb524c272c mgr/prometheus: tag stats by daemon name
Using osd=0 or similar tags was problematic because
daemons of different types have some same-named
counters (e.g. MDS and OSD both have objecter
perf counters).

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:20 -04:00
John Spray
11137aa269 mgr/prometheus: use new get_all_perf_counters interface
Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:20 -04:00
John Spray
ba08fc1008 common: used fixed size int for perf counter prio
...to avoid any ambiguity in allowed range and
make clear how to encode it down the wire.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:20 -04:00
John Spray
f304f84cfb mgr: transmit perf counter prio to the mgr
Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:20:16 -04:00
John Spray
e631f1a727 common: always include priority in perf counter dump
JSON output with inconsistent sets of members is
annoying to use on the receiving side.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:18:01 -04:00
John Spray
9a42d4255d mgr: add get_all_perf_counters to MgrModule interface
This is for use by modules that dump counters
in bulk, e.g. to a TSDB.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-27 14:18:01 -04:00
Jan Fajerski
13b1236b96 pybind/mgr/prometheus: export cluster-wide pg stats, not per osd
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:01 -04:00
Jan Fajerski
e7704fa9cc pybind/mgr/prometheus: add more osd metadata
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:01 -04:00
Jan Fajerski
d4ba07d044 pybind/mgr/prometheus: don't get perf counters that are not in schema
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:01 -04:00
Jan Fajerski
fa25d31263 pybind/mgr/prometheus: add mon and osd perf counters to export
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:00 -04:00
Jan Fajerski
d99a506ed3 pybind/mgr/prometheus: add index page, export metrics under metrics/
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:00 -04:00
Jan Fajerski
f6e2e36ba7 pybind/mgr/prometheus: export selected perf_counters
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:00 -04:00
Jan Fajerski
2bea381469 pybind/mgr/prometheus: export osd and pool metadata
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:00 -04:00
Jan Fajerski
c288624eed pybind/mgr/prometheus: actually emit reported pg counts
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:00 -04:00
Jan Fajerski
ead0973d7d pybind/mgr/prometheus: no need to wait for notify event
If stats or perf counters are not available they won't be emitted.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:18:00 -04:00
Jan Fajerski
5e4b4b5ea2 pybind/mgr/prometheus: no need to convert perf_schema to ordered_dict
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:17:59 -04:00
Jan Fajerski
76d1918724 pybind/mgr/prometheus: add device_class label to osd metrics
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:17:59 -04:00
Jan Fajerski
49b3ff83cd pybind/mgr/prometheus: add cluster wide metrics; no perf counters for now
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:17:59 -04:00
Jan Fajerski
177afcc788 pybind/mgr/prometheus: prefix metrics with 'ceph'; replace :: with _
Both follow prometheus best practices. While : is a legal metric
character, "Exposed metrics should not contain colons, these are for
users to use when aggregating."

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-09-27 14:17:59 -04:00
mhdo2
e345fe3c57 doc/mgr: add influx plugin docs
Signed-off-by: My Do <mhdo@umich.edu>
2017-09-27 14:17:59 -04:00
mhdo2
68ae26c014 mgr/influx: added influx plugin
Signed-off-by: My Do <mhdo@umich.edu>
2017-09-27 14:17:59 -04:00
Sage Weil
2237624846 Merge pull request #17978 from alram/rbdmap-fix
rbdmap: fix umount when multiple mounts use the same RBD

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-27 12:57:33 -05:00
Casey Bodley
f3e0e0171d Merge pull request #17962 from joscollin/wip-test-signed-unsigned-warning
test: silence warnings from -Wsign-compare

Reviewed-by: Amit Kumar <amitkuma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-09-27 10:26:49 -04:00
Jos Collin
4483bc7c61 Merge pull request #17988 from wumingqiao/wip-rados-update
include/rados: fix typo in librados.h

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-09-27 09:07:29 +00:00
wumingqiao
99d79afe6e include/rados: fix typo in librados.h
opeprate -> operate

Signed-off-by: wumingqiao <wumingqiao@inspur.com>
2017-09-27 16:31:14 +08:00
Alexandre Marangone
40825daece rbdmap: fix umount when multiple mounts use the same RBD
When a Kubernetes Pod consumes a RBD it is mounted two
times on the same host. When the host shutdown umount will
fail leading to a hung system

Signed-off-by: Alexandre Marangone <amarango@redhat.com>
2017-09-26 16:19:25 -07:00
Yuri Weinstein
85f3227885 Merge pull request #17063 from bspark8/wip-dynamic-cli-info-f
osd: use dmclock library client_info_f function dynamically

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2017-09-26 16:02:19 -07:00
Yuri Weinstein
cf1bcf5b3b Merge pull request #17335 from Carudy/cxw
crush: safe check for 'ceph osd crush swap-bucket'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-26 16:01:14 -07:00
Yuri Weinstein
4d50fb56b8 Merge pull request #17410 from zmedico/PGPool-update-optimize-removed_snaps-comparison-when-possible
PGPool::update: optimize removed_snaps comparison when possible

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2017-09-26 15:59:59 -07:00
Sage Weil
36c178feb0 Merge pull request #17980 from liewegas/wip-boost-url
make-dist,cmake: move boost tarball location to download.ceph.com

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2017-09-26 17:07:03 -05:00
Sage Weil
19987549e0 make-dist,cmake: move boost tarball location to download.ceph.com
Sourceforge is down.  Also, we can hammer our servers instead of
theirs.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-26 17:22:36 -04:00
Matt Benjamin
bf99f4d86b Merge pull request #17589 from myENA/feature/20883-adminapi_sync_stats_master
rgw: admin api - add ability to sync user stats from admin api
2017-09-26 12:45:24 -04:00
Matt Benjamin
fcc014cdb0 Merge pull request #17433 from C2python/check_action
rgw: Return Error if Bucket Policy Contians Undefined Action
2017-09-26 12:40:08 -04:00
Jason Dillaman
2360bfca75 Merge pull request #17969 from ceph/rbd-mirror-reboot
systemd: rbd-mirror does not start on reboot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-09-26 12:10:16 -04:00
Sage Weil
62bbfc1b4d Merge pull request #17929 from renhwztetecs/renhw-wip-mgrmon-command_descs_prefix
mon/mgr: sync "mgr_command_descs","osd_metadata" and "mgr_metadata" prefixes to new mons

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-26 08:38:14 -05:00
Sage Weil
0227c91649 Merge pull request #17932 from jcsp/wip-21197
mgr: store declared_types in MgrSession

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-26 08:36:24 -05:00
Sage Weil
03137272eb Merge pull request #17933 from jcsp/wip-17737
mgr: fix crashable DaemonStateIndex::get calls

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-26 08:35:52 -05:00
Sage Weil
e2aef59dd7 Merge pull request #17944 from thmour/tmour_upmap
osd/OSDMap: upmap should respect the osd reweights

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-26 08:35:07 -05:00
Matt Benjamin
637eead62f Merge pull request #17880 from amitkumar50/cov-rgw-9
rgw: Error check on return of read_line()
2017-09-26 08:29:09 -04:00
Sébastien Han
e6cd9570ba rbd-mirorr: does not start on reboot
The current systemd unit file misses 'PartOf=ceph-rbd-mirror.target',
which results in the unit not starting after reboot.
If you have ceph-rbd-mirror@rbd-mirror.ceph-rbd-mirror0, it won't start
after reboot even if enabled.
Adding 'PartOf=ceph-rbd-mirror.target' will enable
ceph-rbd-mirror.target when ceph-rbd-mirror@rbd-mirror.ceph-rbd-mirror0
gets enabled.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-26 14:05:37 +02:00
John Spray
99b15792a3 Merge pull request #17961 from tchaikov/wip-doc-local-pool-in-toc
doc/mgr: add "local pool" plugin to toc

Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-26 11:31:09 +01:00
Kefu Chai
dca1306347 Merge pull request #17949 from asomers/nproc_freebsd
do_{cmake,freebsd}: Don't invoke nproc(1) on FreeBSD

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-26 17:45:56 +08:00
Kefu Chai
d4da5e2063 Merge pull request #17940 from jcsp/wip-21534
mon: update get_store_prefixes implementations

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-26 17:09:54 +08:00
Jos Collin
fdbbe8a32c test: silence warnings from -Wsign-compare
Fixed the warnings:

ceph/src/test/bufferlist.cc:2150:3:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1392:11: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (lhs == rhs) {

ceph/src/test/bufferlist.cc:2151:3:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1392:11: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   if (lhs == rhs) {

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-09-26 10:24:15 +05:30