Commit Graph

80138 Commits

Author SHA1 Message Date
Sage Weil
df2e008d33 mgr/balancer: crush-compat: throttle changes based on max_misplaced
Take smaller steps if we overshoot max_misplaced.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 07:28:49 -05:00
Sage Weil
020be37fd0 mgr/balancer: remove dead code
Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 07:28:49 -05:00
Sage Weil
f6f2d253c3 mgr/balancer: include pg up mapping in MappingState
Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 07:28:49 -05:00
Sage Weil
99fcd161bd mgr/balancer: normalize weight-set weights to sum to target weight
Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 07:28:49 -05:00
Sage Weil
26a7ec0c8a mgr/balancer: note root id in Eval
Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 07:28:49 -05:00
Sage Weil
d9a31595ba mgr/balancer: make crush-compat mode work!
- it does multiple iterations, like the upmap optimizer.
- it decreases the step size if it isn't improving, in the hope
that it is overshooting the minimum
- debug output is cleaned up a bit (the info level should be
genuinely useful)

Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 07:28:49 -05:00
Sage Weil
2912364ff8
Merge pull request #16651 from jcsp/wip-mgr-module-interface
mgr: python interface rework + enable modules to run in standby mode

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-01 07:27:20 -05:00
John Spray
5861c4c022 mon: include disabled modules in mgr module ls
Otherwise, when someone wants to see what's possible
to do with `mgr module enable` they have to trawl
through the whole mgr map dump.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:43 -04:00
John Spray
d220e1adc3 doc: describe using mgr module ... commands
...including the new "mgr services" command.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:43 -04:00
John Spray
ceb2a91190 doc: describe how to implement standby modules
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:43 -04:00
John Spray
9988ebed95 qa: fix mgr caps
This was still using Kraken era settings

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:43 -04:00
John Spray
05e648be6a qa: expand mgr testing
Some extra coverage of the dashboard, including its standby
redirect mode and the publishing of URIs.

Also invoking the command_spam mode of the selftest module.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:42 -04:00
John Spray
a382c3f1ca mgr/selftest: extend test and add background spam mode
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:42 -04:00
John Spray
27ee148e04 mgr: drop GIL around set_uri, set_health_checks
These didn't need to keep the GIL to go and do their
pure C++ parts, and by keeping it they could deadlock
while trying to take ActiveMgrModules::lock.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:42 -04:00
John Spray
58dfa97ba8 mgr: fix ~MonCommandCompletion
This was doing a Py_DECREF outside of the Gil.

Fixes: http://tracker.ceph.com/issues/21593
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:42 -04:00
John Spray
29193a47e6 mgr: update for SafeThreadState
A bunch of the previous commits were done
before this class existed, so updating in
one go instead of trying to edit history
in fine detail.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:42 -04:00
John Spray
7e61f79f5d mgr: refactor PyOSDMap etc implementation
Implement real python classes from the C side,
rather than exposing only module methods.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:21:42 -04:00
Sage Weil
eacc902145 mgr/PyOSDMap: add CRUSH get_item_weight
Signed-off-by: Sage Weil <sage@redhat.com>
2017-11-01 08:20:22 -04:00
John Spray
0d5b1d70e6 mgr: fix py_module_registry shutdown
Was calling way too early, which did a
Py_Finalize before the modules had been
joined.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:22 -04:00
John Spray
bb4e71ed2e mgr: fix thread naming
Was passing a reference to a local stringstream into
Thread::create, not realising that it was taking a char*
reference instead of a copy.  Result was garbage (or usually,
all threads having the name of the last one created)

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:22 -04:00
John Spray
df8797320b mgr: cut down duplication between active+standby
...by using PyModuleRunner class from ActivePyModule too.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
e2442c1e20 mgr: fix os._exit overrides
These would throw an exception when passed
a status code.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
7b629ae465 mon/MgrMonitor: reset services map on drop_active
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
4f7007d1b0 mgr/dashboard: implement standby mode
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
3048e85cd7 pybind/mgr: add MgrStandbyModule
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
c1471c7501 mgr: standby modules come up and run now
...they still don't have access to any config though.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
25566d1edc mgr: enable running modules in standby mode
Modules can implement a second, separate class
that has access to very little state about the
system and can't implement commands.

They have just enough information to redirect
or forward incoming requests/traffic to the
active instance of the module on the active mgr.

This enables module authors to create modules
that end users can access via any (running) mgr node
at any time, rather than having to first work out
which mgr node is active.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:21 -04:00
John Spray
70d45a6b93 mgr: clean up python source file naming
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:20 -04:00
John Spray
9718896c8b mgr: refactor python module management
Separate out the *loading* of modules from
the *running* of modules.

This is a precursor to enabling modules to run
in standby mode.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:20 -04:00
John Spray
089e105dd7 pybind/mgr: use set_uri hook from dashboard+restful modules
No more guessing the URL!

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:20 -04:00
John Spray
a0183a63fa mgr: enable python modules to advertise their service URI
Fixes: http://tracker.ceph.com/issues/17460
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:20 -04:00
John Spray
c3c3e4e90b mon/MgrMonitor: store services in map and expose with command
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:20 -04:00
John Spray
236841b3b6 messages: services in MMgrBeacon
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:19 -04:00
John Spray
3f703bd91f mon/MgrMap: store list of services
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:19 -04:00
John Spray
e938bf9b9d mgr: carry PyModules ref in MonCommandCompletion
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:19 -04:00
John Spray
62cb512e47 pybind: update MgrModule for ceph_state->ceph_module
& tidy up the places where ceph_state was getting
used outside of MgrModule.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:19 -04:00
John Spray
563878ba21 mgr: refactor python interface
Expose a python class instead of a module,
so that we have a place to carry our reference
to our MgrPyModule* and to PyModules*, rather than
passing a handle for the former and using
a global pointer for the latter.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:19 -04:00
John Spray
99fa1fdf4e mgr/dashboard: remove blue highlight on scrubbing pg states
This was kind of unnecessary, highlighting a completely normal
and healthy situation in a different colour.  The blue was
also really hard to read against a grey background.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:19 -04:00
John Spray
5e64787c0a mgr/dashboard: clean up fs standby list when empty
Signed-off-by: John Spray <john.spray@redhat.com>
2017-11-01 08:20:18 -04:00
John Spray
83b88f93fd
Merge pull request #18243 from jan--f/mgr-prom-osd-status
pybind/mgr/prometheus: add osd_in/out metric; make osd_weight a metric

Reviewed-by: John Spray <john.spray@redhat.com>
2017-11-01 12:42:15 +01:00
Kefu Chai
5b3224aeff
Merge pull request #18651 from aermolaev/patch-1
doc: fix typo in radosgw/dynamicresharding.rst

Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-11-01 18:01:14 +08:00
Alexander Ermolaev
231b9ab610 doc: fix typo in dynamicresharding.rst: admon => admin
Signed-off-by: Alexander Ermolaev <ave@integros.com>
2017-11-01 12:52:14 +03:00
Haomai Wang
ffb129ef2b
Merge pull request #18660 from ownedu/wip-fix-async-rdma-typo
msg/async/rdma: a tiny typo fix.

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-11-01 01:14:53 -05:00
Kefu Chai
f225a32c83
Merge pull request #18268 from jcsp/wip-20629
mon: handle monitor lag when killing mgrs

Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-11-01 14:10:03 +08:00
ownedu
4b4e499b4e msg/async/rdma: a tiny typo fix.
Signed-off-by: Yan Lei <yongyou.yl@alibaba-inc.com>
2017-11-01 11:38:27 +08:00
Kefu Chai
db89b2c1b0
Merge pull request #18329 from ifed01/wip-fix-mempool-dump
mempool: fix lack of pool names in mempool:dump output for JSON format

Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-11-01 10:54:26 +08:00
Kefu Chai
d07ba9016b
Merge pull request #18575 from majianpeng/msg-cleanup-2
msg/async:  misc cleanup.

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-11-01 10:53:43 +08:00
Kefu Chai
cc484b2606
Merge pull request #18581 from wzliao/wip-drop-unused-parameter
osd/PrimaryLogPG: drop unused parameters

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-11-01 10:53:09 +08:00
Kefu Chai
c5fdce5422
Merge pull request #18588 from mogeb/rados-bench-clock
tools/rados: use the monotonic clock in rados bench

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-01 10:52:40 +08:00
Kefu Chai
737ef1b275
Merge pull request #18595 from batrick/mon-clock
mon: a few conversions to monotonic clock

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-01 10:50:52 +08:00