Commit Graph

58181 Commits

Author SHA1 Message Date
John Spray
88977572f9 packaging: add boost-python dependency
Although ceph-mgr isn't packaged yet, builders
use these files to cue installation of dependencies.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
567b9fb692 doc: add initial ceph-mgr docs
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
ba6484786d pybind/mgr: add fsstatus module
A demonstration of creating a simple
ASCII-art status display by fusing
the metadata, perf counters, and maps that
are available to python modules in ceph-mgr.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
fc3090fcdd mgr: skip past broken modules and load the rest
...instead of stopping at the first python module
that fails to load.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
8db0137dc3 mgrc: less vebose logging of messages
...no need to tell us about all the things you're *not* handling.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
7b8179bd5d mon: implement MgrMonitor.preprocess_beacon
So that we're not throwing out new mgrmap epochs
every beacon.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
7122fac0a5 mgr: expose osd stats from pgmap
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
0812669cf2 mgr: dispose of connections after handle_command
...to avoid leaking them and eventually running out of fds.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
c38d4915a9 mgrc: mark down connections with dead sessions
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:03 +01:00
John Spray
e6de53aa60 pybind/mgr: fix logging
Previously only the errors were getting through, because
other stuff was getting held up in the python logging
framework.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:03 +01:00
John Spray
ea46778e36 mgr: expose "df" to python modules
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:03 +01:00
John Spray
519a01d973 mon: move "df" dump code from PGMonitor to PGMap
...so that ceph-mgr can use it too.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
ab24038049 common/TextTable: add include guards
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
f77276a5c6 pybind/mgr: update rest module for pg_summary
...which is now generated properly from PGMap
instead of reconstituted from some JSON.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
fa147e3a59 mgr: handle PGStats with a PGMap
No longer need the mon to send us
the pg_summary json hack.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
30dec8e281 osd: send PGStats to mgr as well as mon
This is a rough implementation that sends all the stats,
without the efficiency of the queue+acks that we use
when talking to the mon.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:01 +01:00
John Spray
803b66a3b4 mgrc: enable sending pgstats
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:01 +01:00
John Spray
80281ed245 pybind/mgr: expose get_counter in MgrModule
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:01 +01:00
John Spray
e3104862b8 mgrc: fix report sending after failover
The send_report loop doesn't get kicked off
on configure messages unless the period is
set to its initial value (0).

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
John Spray
9501bfdd71 mgr: store some counter history
...and expose it to python modules.

Also fix some of the code around how we updated
DaemonState.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
John Spray
5366980c38 pybind/mgr: move rest into dir
Let's make each module its own directory.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
John Spray
7eb4572944 pybind/mgr: remove unused gevent code
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
Tim Serong
054708a6f0 mgr: Use signed int log level in PyModules::log()
Using an unsigned causes a build failure when expanding the dout macro.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:26:59 +01:00
John Spray
96e10e9328 mds: embed a MgrClient
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:59 +01:00
John Spray
fd537bc696 mgr: enable active daemon to return to standby
Rather than respawn a whole process, just fall back
to being a standby.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:59 +01:00
John Spray
1643a7fc60 mgrc: reset send_report timer on session change
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:59 +01:00
John Spray
7845f8d757 mgr: flesh out standby/HA
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
0bff7af039 pybind/mgr: implement shutdown() in rest.py
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
c81e542b32 mgr: enable multiple python modules
serve() each one in a separate thread, include a shutdown()
hook so that we can tear down cleanly.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
d9dfb436ea mgr: s/DaemonMetadata/DaemonState/g
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
cc0b8fb7c1 pybind/mgr: fix django request logging
This was going to a text file, hook it in
to the proper logging.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:57 +01:00
John Spray
a53dc64655 pybind/mgr: remove unused CalamariConfig
No longer need to set CALAMARI_CONFIG when running.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:57 +01:00
John Spray
b7c9561acc mgr: fix locking in DaemonMetadata
Locks are great but it helps if you actually
bother acquiring them...

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:57 +01:00
Tim Serong
eafe2e9446 mgr: api_docs.py: Update for use outside Calamari
The ceph_state module is implemented in C++, and isn't available at
build time, which causes import failures when api_docs.py tries to
import calamari_rest in order to introspect it (MgrModule is eventually
imported, which in turn tries to import ceph_state, which fails unless
we stub it out).

Additional changes:

* Search calamari_rest.urls (not calamari_web)

calamari_web isn't available - it largely held static content in
Calamari, and in turn included calamari_rest.urls.  Here in ceph-mgr we
only have calamari_rest.

* note that --list-urls does nothing, apparently

* Pass actions to old as_view method

django-rest-framework 3.x raises TypeError if the actions argument isn't
passed to as_view().

* Use view().get_view_name() instead of metadata

Attempting to access view().metadata(None)['name'] results in
"AttributeError: 'super' object has no attribute 'metadata'".

* Use somewhat unfriendly field class names

django-rest-framework 3.x seems to have done away with type_label for
fields, so instead use the field's class name, which is somewhat
unfriendly, but arguably better than nothing.

* import global_instance to fix ImportError

This is necessary to avoid what seems to be weird import loops (if this
isn't present, we later get "ImportError: cannot import name
UserRequest" or similar).

* Make api_examples.json optional

This way we at least get bare docs with no examples, rather than no docs
at all.

* Explain how to generate API docs

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:26:56 +01:00
John Spray
b82533fcb7 pybind/mgr: add the rest module
This is derived from what used to be Calamari.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:56 +01:00
John Spray
44c95af285 pybind: create mgr python module folder
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:56 +01:00
John Spray
b3189d6824 cmake: update for ceph-mgr
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:56 +01:00
John Spray
3f4d80c7e3 pybind: expose mgr commands
This is "tell mgr" at the moment but it should
be a slicker syntax later (ceph.in is awkward
to refactor just now)

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:55 +01:00
John Spray
157f986d91 librados: expose mgr_command
This involves giving RadosClient an embedded
MgrClient.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:55 +01:00
John Spray
10da6d23cd osd: embed a MgrClient
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:55 +01:00
John Spray
88442d2391 mgrc: create MgrClient
The ceph-mgr equivalent to monclient

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:54 +01:00
John Spray
3283b1e409 common: accessors for list of perf counters
...and store the list by a string path, for
consumption by the world outside of integer
perf counter/subsystem IDs.

This is for consumption by MgrClient.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:54 +01:00
John Spray
ac30e6cee2 mgr: create ceph-mgr service
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:54 +01:00
John Spray
bbc66310ef auth: add mgr service type
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:54 +01:00
John Spray
e32f2a90b8 vstart: set up and run ceph-mgr
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:53 +01:00
John Spray
84508408d9 mon: add MgrMonitor
This is responsible for learning the locations of
mgrs, picking an active one, and sharing that
info with MgrClient consumers who subscribe to 'mgrmap'

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:53 +01:00
John Spray
eaada77fea msg: introduce mgr entity_type_t
Mgr will probably ultimately have different auth caps,
so let's make it a separate entity up front instead
of e.g. pretending to be a mon.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:53 +01:00
John Spray
e3b2225f0b messages: add ceph-mgr messages
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:53 +01:00
John Spray
7e6b4f1fcb client: refactor command handling
Create a CommandTable structure for places
in Client, MgrClient, Objecter where we do
basically the same kind of thing for sending
and tracking MCommands.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:52 +01:00
John Spray
a11ff90fea pybind: fix handling bad cmd json
The 'save_exception' local would otherwise
be referenced before assignment during
exception handling.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:52 +01:00