Commit Graph

58138 Commits

Author SHA1 Message Date
John Spray
592bb9c2a7 vstart: include cython mods in PYTHONPATH
This is useful for ceph-mgr because it means that
any in-mgr modules that want to use e.g. rbd
can find it when running in a vstart cluster.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:17 +01:00
John Spray
f8514ef581 mgr: remove hardcoded site-packages
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:17 +01:00
John Spray
042b6b4454 packaging: add debian packaging for ceph-mgr
...at least, try to!  This is a copy-paste of the ceph-mds
packaging with a search and replace mds to mgr.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:17 +01:00
Tim Serong
082199f69d systemd: autogenerate ceph-mgr key during daemon startup
This is a hack to inject a key for the mgr daemon, using whatever
key already exists on the mon on this node to gain sufficient
permissions to create the mgr key.  Failure is ignored at every
step (the '-' prefix) in case someone has already used some other
trick to set everything up manually.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
61d779345e systemd: encourage ceph-mgr to start in sync with ceph-mon
This change introduces the following behaviour:

- When ceph-mon starts, it will try to start ceph-mgr with the same
  instance id (Wants=), but will *not* fail to start if ceph-mgr
  doesn't start (i.e. the mon still works as it always did).
- ceph-mgr will start After= ceph-mon, and will stop and start when
  ceph-mon stops and starts, because it's PartOf= ceph-mon.

If you don't want ceph-mgr to run on the mons, you need to mask the
service, i.e. `systemctl mask ceph-mgr@INSTANCE`.  Hostnames are
typically instance names, so `systemctl mask ceph-mgr@$(hostname)`
should suffice if you wish to disable ceph-mgr on the mons.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
2e9e21eb42 ceph.spec.in: add ceph-mgr package
This adds ceph-mgr following the same pattern used for ceph-mon,
ceph-mds, ceph-osd.  I've removed the boost-python BuildRequire
for SUSE distros, because we don't ship that (the relevant bits
are already included in boost-devel).

NOTE: This gives an installable ceph-mgr, but not necessarily a
runnable ceph-mgr.  This is because ceph-mgr needs the following
python packages:

- CherryPy
- Django
- gevent
- python-dateutil
- djangorestframework

Not all of these are available as distro packages.  For example,
on openSUSE Tumbleweed, we don't have djangorestframework available
in the base distro, and our Django version is 1.9, which seems to
be incompatible with ceph-mgr.  I'm not sure of the situation on
other distros.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
d8ded57a87 systemd: add ceph-mgr service and target files
Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:08 +01:00
Tim Serong
c6972b983b cmake: install ceph-mgr and its python modules
Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:27:07 +01:00
John Spray
da76c59efc pybind/mgr: clean up unused code
...and change the "if False" django-rest-framework
conditionals to actual version checks, with a view
to genuinely supporting 2.x in future if needed.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:07 +01:00
John Spray
1bb9edbaf2 mgr: fix OSDMap notifications
This broken when MgrStandby was separated out,
because it now eats the original OSDMap message
without calling maybe_request_map.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:07 +01:00
John Spray
45b33934e8 mgr: fixup ClusterState with_*map fns
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:06 +01:00
John Spray
78b6eebf71 mon: remove need for friending OSDMap to PGMapUpdater
This used to be the friend relationship to PGMonitor.  It
was never really needed anyway, it was only being used
to skip the getter on `pools`

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:06 +01:00
John Spray
afa7078763 mon: refactor PGMap updating code for reuse in mgr
...and remove the copypasta from mgr.

mgr still doesn't do all the same logic (it doesn't
have osdmap history handy) but it is now much easier
to see which bits are used one place vs. the other.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:06 +01:00
John Spray
58dd3db0be mgr: update for Dispatcher::ms_handle_refused
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:06 +01:00
John Spray
2210772ed8 common: refactor CommandTable
Avoid handling out raw pointers to the ops,
which are in fact owned by the table.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
88977572f9 packaging: add boost-python dependency
Although ceph-mgr isn't packaged yet, builders
use these files to cue installation of dependencies.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
567b9fb692 doc: add initial ceph-mgr docs
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
ba6484786d pybind/mgr: add fsstatus module
A demonstration of creating a simple
ASCII-art status display by fusing
the metadata, perf counters, and maps that
are available to python modules in ceph-mgr.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
fc3090fcdd mgr: skip past broken modules and load the rest
...instead of stopping at the first python module
that fails to load.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:05 +01:00
John Spray
8db0137dc3 mgrc: less vebose logging of messages
...no need to tell us about all the things you're *not* handling.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
7b8179bd5d mon: implement MgrMonitor.preprocess_beacon
So that we're not throwing out new mgrmap epochs
every beacon.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
7122fac0a5 mgr: expose osd stats from pgmap
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
0812669cf2 mgr: dispose of connections after handle_command
...to avoid leaking them and eventually running out of fds.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:04 +01:00
John Spray
c38d4915a9 mgrc: mark down connections with dead sessions
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:03 +01:00
John Spray
e6de53aa60 pybind/mgr: fix logging
Previously only the errors were getting through, because
other stuff was getting held up in the python logging
framework.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:03 +01:00
John Spray
ea46778e36 mgr: expose "df" to python modules
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:03 +01:00
John Spray
519a01d973 mon: move "df" dump code from PGMonitor to PGMap
...so that ceph-mgr can use it too.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
ab24038049 common/TextTable: add include guards
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
f77276a5c6 pybind/mgr: update rest module for pg_summary
...which is now generated properly from PGMap
instead of reconstituted from some JSON.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
fa147e3a59 mgr: handle PGStats with a PGMap
No longer need the mon to send us
the pg_summary json hack.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:02 +01:00
John Spray
30dec8e281 osd: send PGStats to mgr as well as mon
This is a rough implementation that sends all the stats,
without the efficiency of the queue+acks that we use
when talking to the mon.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:01 +01:00
John Spray
803b66a3b4 mgrc: enable sending pgstats
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:01 +01:00
John Spray
80281ed245 pybind/mgr: expose get_counter in MgrModule
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:01 +01:00
John Spray
e3104862b8 mgrc: fix report sending after failover
The send_report loop doesn't get kicked off
on configure messages unless the period is
set to its initial value (0).

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
John Spray
9501bfdd71 mgr: store some counter history
...and expose it to python modules.

Also fix some of the code around how we updated
DaemonState.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
John Spray
5366980c38 pybind/mgr: move rest into dir
Let's make each module its own directory.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
John Spray
7eb4572944 pybind/mgr: remove unused gevent code
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:27:00 +01:00
Tim Serong
054708a6f0 mgr: Use signed int log level in PyModules::log()
Using an unsigned causes a build failure when expanding the dout macro.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:26:59 +01:00
John Spray
96e10e9328 mds: embed a MgrClient
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:59 +01:00
John Spray
fd537bc696 mgr: enable active daemon to return to standby
Rather than respawn a whole process, just fall back
to being a standby.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:59 +01:00
John Spray
1643a7fc60 mgrc: reset send_report timer on session change
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:59 +01:00
John Spray
7845f8d757 mgr: flesh out standby/HA
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
0bff7af039 pybind/mgr: implement shutdown() in rest.py
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
c81e542b32 mgr: enable multiple python modules
serve() each one in a separate thread, include a shutdown()
hook so that we can tear down cleanly.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
d9dfb436ea mgr: s/DaemonMetadata/DaemonState/g
Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:58 +01:00
John Spray
cc0b8fb7c1 pybind/mgr: fix django request logging
This was going to a text file, hook it in
to the proper logging.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:57 +01:00
John Spray
a53dc64655 pybind/mgr: remove unused CalamariConfig
No longer need to set CALAMARI_CONFIG when running.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:57 +01:00
John Spray
b7c9561acc mgr: fix locking in DaemonMetadata
Locks are great but it helps if you actually
bother acquiring them...

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:57 +01:00
Tim Serong
eafe2e9446 mgr: api_docs.py: Update for use outside Calamari
The ceph_state module is implemented in C++, and isn't available at
build time, which causes import failures when api_docs.py tries to
import calamari_rest in order to introspect it (MgrModule is eventually
imported, which in turn tries to import ceph_state, which fails unless
we stub it out).

Additional changes:

* Search calamari_rest.urls (not calamari_web)

calamari_web isn't available - it largely held static content in
Calamari, and in turn included calamari_rest.urls.  Here in ceph-mgr we
only have calamari_rest.

* note that --list-urls does nothing, apparently

* Pass actions to old as_view method

django-rest-framework 3.x raises TypeError if the actions argument isn't
passed to as_view().

* Use view().get_view_name() instead of metadata

Attempting to access view().metadata(None)['name'] results in
"AttributeError: 'super' object has no attribute 'metadata'".

* Use somewhat unfriendly field class names

django-rest-framework 3.x seems to have done away with type_label for
fields, so instead use the field's class name, which is somewhat
unfriendly, but arguably better than nothing.

* import global_instance to fix ImportError

This is necessary to avoid what seems to be weird import loops (if this
isn't present, we later get "ImportError: cannot import name
UserRequest" or similar).

* Make api_examples.json optional

This way we at least get bare docs with no examples, rather than no docs
at all.

* Explain how to generate API docs

Signed-off-by: Tim Serong <tserong@suse.com>
2016-09-29 17:26:56 +01:00
John Spray
b82533fcb7 pybind/mgr: add the rest module
This is derived from what used to be Calamari.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-09-29 17:26:56 +01:00