This is a hack to inject a key for the mgr daemon, using whatever
key already exists on the mon on this node to gain sufficient
permissions to create the mgr key. Failure is ignored at every
step (the '-' prefix) in case someone has already used some other
trick to set everything up manually.
Signed-off-by: Tim Serong <tserong@suse.com>
This change introduces the following behaviour:
- When ceph-mon starts, it will try to start ceph-mgr with the same
instance id (Wants=), but will *not* fail to start if ceph-mgr
doesn't start (i.e. the mon still works as it always did).
- ceph-mgr will start After= ceph-mon, and will stop and start when
ceph-mon stops and starts, because it's PartOf= ceph-mon.
If you don't want ceph-mgr to run on the mons, you need to mask the
service, i.e. `systemctl mask ceph-mgr@INSTANCE`. Hostnames are
typically instance names, so `systemctl mask ceph-mgr@$(hostname)`
should suffice if you wish to disable ceph-mgr on the mons.
Signed-off-by: Tim Serong <tserong@suse.com>
When ceph-disk runs from udev or init script, it is in the background
and should it block for any reason, it may keep a lock forever. All
calls to ceph-disk in these context are changed to timeout.
The TimeoutStartSec= and TimeoutStopSec= which are both set via
TimeoutSec= do not apply to Type=oneshot services.
https://www.freedesktop.org/software/systemd/man/systemd.service.html
Fixes: http://tracker.ceph.com/issues/16580
Signed-off-by: Loic Dachary <loic@dachary.org>
Some distros, like Fedora and openSUSE, have a policy that all services are
disabled by default.
This patch changes that default for the ceph.target and
ceph-{mds,mon,osd,radosgw}.target services.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Boris Ranto <branto@redhat.com>
Currently, the systemd daemons are not restarted on failure. This patch
adds this functionality and sets the defaults to those defined in
upstart. This resolves to 3 fails per 30 minutes for osd, mon and mds
and 5 fails per 30 seconds for radosgw.
Signed-off-by: Boris Ranto <branto@redhat.com>
If systemd has task accounting enabled, a default of 512 tasks
will be applied to all systemd units.
For ceph, this is way to low even for a modest cluster, so stop
this restriction being applied and allow administrators to apply
limits using sysctl.
Signed-off-by: James Page <james.page@ubuntu.com>
These are not supported by /usr/lib/ceph/ceph-osd-prestart.sh,
resulting in warnings:
ceph-osd-prestart.sh[23367]: getopt: unrecognized option '--setuser'
ceph-osd-prestart.sh[23367]: getopt: unrecognized option '--setgroup'
--setuser and --setgroup are only needed for the ceph-osd process.
Signed-off-by: James Page <james.page@ubuntu.com>
First, it makes sense for both ceph_common.sh and ceph-osd-prestart.sh to
reside in the same directory: make it so.
Second, /usr/lib exists on both RHEL/Fedora and SLE/openSUSE, whereas
the later lacks /usr/libexec. To make this less painful, package
ceph_common.sh and ceph-osd-prestart.sh in /usr/lib/ceph.
Third, allow e.g. FreeBSD to do its own thing by using the $(libexecdir)
Autoconf variable (but set it to /usr/lib in the spec file).
http://tracker.ceph.com/issues/14687Fixes: #14687
Signed-off-by: Nathan Cutler <ncutler@suse.com>
This change makes it so the mon/osd/mds/radosgw daemons:
o Cannot write to /usr, /etc, and /boot.
o Cannot access /home, /root, or /run/user.
o Each daemon gets its own private /tmp and /var/tmp.
o All daemons get a private /dev without physical devices (exception: osd)
I'm not sure if the osd daemon needs access to a full /dev so I left
ProtectDevices out for ceph-osd@.service.
Signed-off-by: Patrick Donnelly <batrick@batbytes.com>
The flock command may be installed elsewhere, depending on the
system. Let the PATH search figure that out.
http://tracker.ceph.com/issues/13975Fixes: #13975
Signed-off-by: Loic Dachary <loic@dachary.org>
systemd: start/stop/restart ceph services by daemon type
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
When activating a device, ceph-disk trigger restarts the ceph-disk
systemd service. Two consecutive udev add on the same device will
restart the ceph-disk systemd service and the second one may kill the
first one, leaving the device half activated.
The ceph-disk systemd service is instructed to not kill an existing
process when restarting. The second run waits (via flock) for the second
one to complete before running so that they do not overlap.
http://tracker.ceph.com/issues/13160Fixes: #13160
Signed-off-by: Loic Dachary <ldachary@redhat.com>
We were observed to be hitting the limit on centos7
(triggering pthread_create failures) on a ~2000 OSD cluster.
Increasing this resolves it!
Reported-by: Dan van der Ster <daniel.vanderster@cern.ch>
Signed-off-by: Sage Weil <sage@redhat.com>
This allows members of the ceph group to make librados clients (like the
ceph cli and qemu) create sockets in the default /var/run/ceph/* location.
Signed-off-by: Sage Weil <sage@redhat.com>
Allow all daemons drop privilege themselves, instead of letting
systemd do it.
Among other things, this means that admins can conditionally not
drop prives by setting
setuser match path = /var/lib/ceph/$type/$cluster-$id
in their ceph.conf to ease the pain of upgrade.
Signed-off-by: Sage Weil <sage@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Specify the nofile ulimit in one standard place, where everyone expects it
to be. Drop it from the ceph-osd unit file.
Leave upstart and sysvinit untouched for the time being to avoid compat
issues.
Signed-off-by: Sage Weil <sage@redhat.com>
The udev(7) man page states:
RUN
...
This can only be used for very short-running foreground tasks. Running
an event process for a long period of time may block all further
events for this or a dependent device.
Starting daemons or other long-running processes is not appropriate
for udev; the forked processes, detached or not, will be
unconditionally killed after the event handling has finished.
ceph-disk activate is far from a short-running task:
- check whether path is a block dev, for dirs call through to
activate_dir()
- call blkid to obtain the filesystem type for the block dev
- pull mount options from hard-coded ceph.conf file
- mount the OSD dev at a temporary path
- check the ceph magic for mounted filesystem
- read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf
path
- read or generate (if missing) the OSD uuid
- create a file indicating init system usage (systemd)
- mount the device at a second (final) location
- umount (lazy) the temporary mount path
- enable the systemd ceph-osd@{osd_id} service
- start the systemd ceph-osd@{osd_id} service
This logic is therefore best left in a systemd service for execution. As
it is less limited in terms of execution time, and also allows for
improved event handling in future (fsck, dmcrypt mapping etc.).
This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or
ceph-disk activate-journal via new ceph-disk-activate-journal@.service,
ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service
systemd service files.
ceph-disk-dmcrypt-activate@.service makes use of the newly added
--dmcrypt parameter for ceph-disk activate.
Signed-off-by: David Disseldorp <ddiss@suse.de>
To simplify the spec file we should install as much using autotools
and as little as possible in the spec file.
Signed-off-by: Owen Synge <osynge@suse.com>
Added a radosgw systemd support and associated prestart script.
- With improved checking over first revison.
- ceph-radosgw-prestart.sh now installed in /usr/lib/ceph-radosgw
Signed-off-by: Owen Synge <osynge@suse.com>
tmpfiles.d are part of system.d and define how temporary directories are setup.
rgw needs a socket directory. To do this we template tmpfiles.d user and group
for rgw and fill in the values using autotools.
Note1: Added to spec file.
Note2: Name changed to rgw from radosgw as is preferred name by Sage.
Note3: Adds configure options
--with-rgw-user=UserName
--with-rgw-group=GroupName
Note4: Defaults set for debian
Note5: spec file overrides defaults for redhat and suse
Signed-off-by: Owen Synge <osynge@suse.com>
