Commit Graph

36 Commits

Author SHA1 Message Date
Sage Weil
a12efa204e Merge pull request #6276 from david-z/wip-systemd-finegrain-ceph-service
systemd: start/stop/restart ceph services by daemon type

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Sage Weil  <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2015-11-28 08:25:40 -05:00
suckowbiz
5972a44106 doc: fix message typos in systemd
Signed-off-by: Tobias Suckow <tobias@suckow.biz>
2015-11-23 16:50:07 +01:00
Boris Ranto
9224ac2ad2 rbdmap: systemd support
Fixes: #13374
Signed-off-by: Boris Ranto <branto@redhat.com>
2015-11-06 10:26:22 +01:00
Zhi Zhang
cfa2d0a08a fine-grained control systemd to start/stop/restart ceph services at once
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
2015-10-26 15:13:19 +08:00
Sage Weil
fb5f058a92 Merge remote-tracking branch 'gh/infernalis' 2015-09-22 14:04:44 -04:00
Loic Dachary
f0a47578c7 ceph-disk: systemd must not kill a running ceph-disk
When activating a device, ceph-disk trigger restarts the ceph-disk
systemd service. Two consecutive udev add on the same device will
restart the ceph-disk systemd service and the second one may kill the
first one, leaving the device half activated.

The ceph-disk systemd service is instructed to not kill an existing
process when restarting. The second run waits (via flock) for the second
one to complete before running so that they do not overlap.

http://tracker.ceph.com/issues/13160 Fixes: #13160

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:56 +02:00
Sage Weil
ea977611c4 systemd: increase nproc ulimit
We were observed to be hitting the limit on centos7
(triggering pthread_create failures) on a ~2000 OSD cluster.

Increasing this resolves it!

Reported-by: Dan van der Ster <daniel.vanderster@cern.ch>
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-21 14:35:15 -04:00
Sage Weil
8e13d89f0f systemd: eliminate ceph-rgw tmpfiles.d file
This is for storing the rgw socket files for fastcgi, which we do not
want to enable by default.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-14 14:00:26 -04:00
Sage Weil
367c794cb1 systemd: no need to preprocess ceph-osd@service
This used to be necessary but now is not.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-14 14:00:26 -04:00
Sage Weil
8453a89cb2 systemd: set nofile limit in unit files
Make it big so hopefully nobody has to change it.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-14 14:00:26 -04:00
Sage Weil
ea91c4ef85 systemd: tmpfiles.d in /run, not /var/run
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-11 11:38:47 -04:00
Sage Weil
3aa38bc07f make /var/run/ceph 770 ceph:ceph
This allows members of the ceph group to make librados clients (like the
ceph cli and qemu) create sockets in the default /var/run/ceph/* location.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-11 11:26:59 -04:00
Sage Weil
f1b80e99b0 systemd: consolidate into a single ceph-disk@.service
This simple service will 'ceph-disk trigger DEV --sync'.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-01 11:22:25 -04:00
Sage Weil
8f3185bade systemd: use --setuser and --setgroup for all daemons
Allow all daemons drop privilege themselves, instead of letting
systemd do it.

Among other things, this means that admins can conditionally not
drop prives by setting

  setuser match path = /var/lib/ceph/$type/$cluster-$id

in their ceph.conf to ease the pain of upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
2015-08-26 20:34:15 -04:00
Sage Weil
c7ee798a0f set nofile ulimit in /etc/security/limits.d/ceph only
Specify the nofile ulimit in one standard place, where everyone expects it
to be.  Drop it from the ceph-osd unit file.

Leave upstart and sysvinit untouched for the time being to avoid compat
issues.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-26 20:34:15 -04:00
Sage Weil
7c9fdf44f2 systemd: make ceph-osd setuid/gid to ceph:ceph
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-26 20:34:15 -04:00
Sage Weil
b8893f6b8a systemd: chown ceph:ceph /var/run/ceph
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-26 20:34:14 -04:00
Sage Weil
ec1ee5e901 systemd: run mon and mds as ceph:ceph
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-26 20:34:14 -04:00
Sage Weil
4d10dc134b systemd: fix ceph-radosgw@ service
There's no prestart.  Fix the instance name.  Cleanup.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-01 09:58:34 -04:00
David Disseldorp
85a894697e systemd: activate disks via systemd service instead of udev
The udev(7) man page states:
  RUN
  ...
  This can only be used for very short-running foreground tasks. Running
  an event process for a long period of time may block all further
  events for this or a dependent device.

  Starting daemons or other long-running processes is not appropriate
  for udev; the forked processes, detached or not, will be
  unconditionally killed after the event handling has finished.

ceph-disk activate is far from a short-running task:
- check whether path is a block dev, for dirs call through to
  activate_dir()
- call blkid to obtain the filesystem type for the block dev
- pull mount options from hard-coded ceph.conf file
- mount the OSD dev at a temporary path
- check the ceph magic for mounted filesystem
- read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf
  path
- read or generate (if missing) the OSD uuid
- create a file indicating init system usage (systemd)
- mount the device at a second (final) location
- umount (lazy) the temporary mount path
- enable the systemd ceph-osd@{osd_id} service
- start the systemd ceph-osd@{osd_id} service

This logic is therefore best left in a systemd service for execution. As
it is less limited in terms of execution time, and also allows for
improved event handling in future (fsck, dmcrypt mapping etc.).

This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or
ceph-disk activate-journal via new ceph-disk-activate-journal@.service,
ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service
systemd service files.

ceph-disk-dmcrypt-activate@.service makes use of the newly added
--dmcrypt parameter for ceph-disk activate.

Signed-off-by: David Disseldorp <ddiss@suse.de>
2015-08-01 09:58:34 -04:00
Sage Weil
6842a20915 systemd: add ceph-create-keys@ service
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-01 09:58:33 -04:00
Sage Weil
43d6b12149 system: include all unit files
Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-01 09:58:33 -04:00
Owen Synge
1ba4694d7e Autotools to install systemd unit files.
To simplify the spec file we should install as much using autotools
and as little as possible in the spec file.

Signed-off-by: Owen Synge <osynge@suse.com>
2015-08-01 09:58:33 -04:00
Owen Synge
be4f878aaf radosgw systemd prestart moved to systemd
prestart scripts are going to be added to systemd to simplify the switching behavior.

Signed-off-by: Owen Synge <osynge@suse.com>
2015-08-01 09:58:33 -04:00
Owen Synge
fbc85e3ce1 radosgw systemd support
Added a radosgw systemd support and associated prestart script.
  - With improved checking over first revison.
  - ceph-radosgw-prestart.sh now installed in /usr/lib/ceph-radosgw

Signed-off-by: Owen Synge <osynge@suse.com>
2015-08-01 09:58:33 -04:00
Owen Synge
61240190c7 Added tmpfiles.d for rgw: templated user and group.
tmpfiles.d are part of system.d and define how temporary directories are setup.
rgw needs a socket directory. To do this we template tmpfiles.d user and group
for rgw and fill in the values using autotools.

Note1: Added to spec file.
Note2: Name changed to rgw from radosgw as is preferred name by Sage.
Note3: Adds configure options
    --with-rgw-user=UserName
    --with-rgw-group=GroupName
Note4: Defaults set for debian
Note5: spec file overrides defaults for redhat and suse

Signed-off-by: Owen Synge <osynge@suse.com>
2015-08-01 09:58:33 -04:00
Nathan Cutler
05424a803b logrotate.conf: fixes for systemd
Before this patch, the command 'logrotate -f /etc/logrotate.d/ceph'
was generating an error "Failed to reload ceph.target: Job type reload is not
applicable for unit ceph.target".

Before we issue systemctl reload, check that there is at least
one active ceph-* service. (The hyphen is significant.)

Since we use grep, make the grep package a dependency.

http://tracker.ceph.com/issues/12173 Fixes: #12173

Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Lars Marowsky-Bree <lmb@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2015-06-26 19:43:44 +02:00
Owen Synge
dfda3ff874 Bug fix to ceph systemV compatability script.
Was failing with more than one OSD / MON deamon on a single node.
Fixes suse bugzilla #927862

Signed-off-by: Owen Synge <osynge@suse.com>
2015-06-18 17:15:22 +02:00
Owen Synge
bfa0c4a626 Fixes to rcceph script
- only start OSDs if mon daemons are also present
 - adds support for mask and unmask
 - removes support for cluster with non default cluster name,
   as this was very limited and inconsistent
 - Reapplied from a patch as could not cherry-pick
   66cb46c411 from Mon Jan 12
   as this produced issues with src/gmock

Signed-off-by: Owen Synge <osynge@suse.com>
2015-06-18 14:16:03 +02:00
Owen Synge
ac347dc340 Template systemd/ceph-osd@.service with autotools,
The libexec path is different for different distributions.
systemd. This path is defined by a new variable on the
configure path.

This variable can be set with enviroment SYSTEMD_LIBEXEC_DIR.
The parameter --with-systemd-libexec-dir overrides the enviroment
variable.

Appropriate conditionals are set for SUSE and RHEL derivatives.

This is then used to template out systemd/ceph-osd@.service

Signed-off-by: Owen Synge <osynge@suse.com>
2015-05-26 19:04:22 +02:00
Owen Synge
ebda4ba1c6 Increase max files open limit for OSD daemon.
Under heavy load the number of file descriptors opened
by the OSD can go beyond the 64K file limit. This patch
increases the default to 128K.

Signed-off-by: Owen Synge <osynge@suse.com>
2015-04-21 11:31:24 +02:00
Ken Dreyer
71a5090bca ceph.spec.in: fix handling of /var/run/ceph
Prior to this commit, we didn't install /var/run/ceph as a normal
directory. We used the %ghost directive and created the directory with
a "mkdir" command in %post.

This was lacking in several ways:

  1) Simplicy: there is no need to use %ghost; other packages (eg.
     mariadb) simply use a normal %dir for their socket directory.

  2) RPM does not have control over the permissions of the /var/run/ceph
     directory. This does not interact well with "rpm -V". Moreover,
     once Ceph itself gets unprivileged user support, RPM itself won't
     be able to set the permissions of the directory for a (future)
     unprivileged UID.

  3) On distributions that use systemd as an init system, /var/run is a
     symlink to /run, which is tmpfs. This means that /var/run/ceph does
     not persist across reboots on those systems.

Remove the %ghost directive; it makes more sense for RPM to simply
install this directory like the rest of the %files.

Add a "_with_systemd" conditional so we know which distros use systemd
as their init system.  Add the /etc/tmpfiles.d/ceph.conf file on those
distros. See
http://www.freedesktop.org/software/systemd/man/tmpfiles.d.html

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-03-17 12:28:28 -06:00
Owen Synge
929fcd67aa better systemd support
Added ceph.target
Made ceph-mds, ceph-mon, ceph-osd services part of ceph.target
Made ceph-mds, ceph-mon, ceph-osd services require partitions to be available.
Added ceph init script with sysV like behaviour.

Provided by Tim Serong tserong@suse.com and Owen Synge osynge@suse.com

Signed-off-by: Owen Synge <osynge@suse.com>
2014-12-01 14:45:31 +01:00
Sage Weil
f25bca3136 systemd: ceph-mds and ceph-mon need networking too
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-27 09:22:59 -07:00
Marco Garcês
1835778282 needs network or it will not start if enabled
example error message: "unable to find any IP address in networks: 10.2.27.0/25", so the service wont start
2014-08-22 12:30:58 +02:00
Federico Simoncelli
c26477448e init: add systemd service files
This patch adds systemd service files. It is possible to start and
enable multiple instances (per monid, osdid, mds name), e.g.

 # systemctl start ceph-mon@node01
 # systemctl enable ceph-mon@node01

 # systemctl start ceph-osd@0
 # systemctl enable ceph-osd@0

The ceph cluster can be set in the system config file:

 /etc/sysconfig/ceph

adding or editing the CLUSTER environment variable.

Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
2014-07-22 11:30:55 -04:00