ceph/systemd/ceph-disk@.service

[Unit]
Description=Ceph disk activation: %f
After=local-fs.target
Wants=local-fs.target

[Service]
Type=oneshot
KillMode=none
ExecStart=/bin/sh -c 'timeout 120 flock /var/lock/ceph-disk-$(basename %f) /usr/sbin/ceph-disk --verbose --log-stdout trigger --sync %f'
TimeoutSec=0
systemd: activate disks via systemd service instead of udev The udev(7) man page states: RUN ... This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Starting daemons or other long-running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished. ceph-disk activate is far from a short-running task: - check whether path is a block dev, for dirs call through to activate_dir() - call blkid to obtain the filesystem type for the block dev - pull mount options from hard-coded ceph.conf file - mount the OSD dev at a temporary path - check the ceph magic for mounted filesystem - read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf path - read or generate (if missing) the OSD uuid - create a file indicating init system usage (systemd) - mount the device at a second (final) location - umount (lazy) the temporary mount path - enable the systemd ceph-osd@{osd_id} service - start the systemd ceph-osd@{osd_id} service This logic is therefore best left in a systemd service for execution. As it is less limited in terms of execution time, and also allows for improved event handling in future (fsck, dmcrypt mapping etc.). This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or ceph-disk activate-journal via new ceph-disk-activate-journal@.service, ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service systemd service files. ceph-disk-dmcrypt-activate@.service makes use of the newly added --dmcrypt parameter for ceph-disk activate. Signed-off-by: David Disseldorp <ddiss@suse.de> 2015-05-11 23:45:34 +00:00			`[Unit]`
			`Description=Ceph disk activation: %f`
ceph-disk: systemd unit must run after local-fs.target A ceph udev action may be triggered before the local file systems are mounted because there is no ordering in udev. The ceph udev action delegates asynchronously to systemd via ceph-disk@.service which will fail if (for instance) the LVM partition required to mount /var/lib/ceph is not available yet. The systemd unit will retry a few times but will eventually fail permanently. The sysadmin can systemctl reset-fail at a later time and it will succeed. Add a dependency to ceph-disk@.service so that it waits until the local file systems are mounted: After=local-fs.target Since local-fs.target depends on lvm, it will wait until the lvm partition (as well as any dm devices) is ready and mounted before attempting to activate the OSD. It may still fail because the corresponding journal/data partition is not ready yet (which is expected) but it will no longer fail because the lvm/filesystems/dm are not ready. Fixes: http://tracker.ceph.com/issues/17889 Signed-off-by: Loic Dachary <loic@dachary.org> 2016-11-22 13:45:45 +00:00			`After=local-fs.target`
			`Wants=local-fs.target`
systemd: activate disks via systemd service instead of udev The udev(7) man page states: RUN ... This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Starting daemons or other long-running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished. ceph-disk activate is far from a short-running task: - check whether path is a block dev, for dirs call through to activate_dir() - call blkid to obtain the filesystem type for the block dev - pull mount options from hard-coded ceph.conf file - mount the OSD dev at a temporary path - check the ceph magic for mounted filesystem - read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf path - read or generate (if missing) the OSD uuid - create a file indicating init system usage (systemd) - mount the device at a second (final) location - umount (lazy) the temporary mount path - enable the systemd ceph-osd@{osd_id} service - start the systemd ceph-osd@{osd_id} service This logic is therefore best left in a systemd service for execution. As it is less limited in terms of execution time, and also allows for improved event handling in future (fsck, dmcrypt mapping etc.). This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or ceph-disk activate-journal via new ceph-disk-activate-journal@.service, ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service systemd service files. ceph-disk-dmcrypt-activate@.service makes use of the newly added --dmcrypt parameter for ceph-disk activate. Signed-off-by: David Disseldorp <ddiss@suse.de> 2015-05-11 23:45:34 +00:00
			`[Service]`
			`Type=oneshot`
ceph-disk: systemd must not kill a running ceph-disk When activating a device, ceph-disk trigger restarts the ceph-disk systemd service. Two consecutive udev add on the same device will restart the ceph-disk systemd service and the second one may kill the first one, leaving the device half activated. The ceph-disk systemd service is instructed to not kill an existing process when restarting. The second run waits (via flock) for the second one to complete before running so that they do not overlap. http://tracker.ceph.com/issues/13160 Fixes: #13160 Signed-off-by: Loic Dachary <ldachary@redhat.com> 2015-09-20 11:54:05 +00:00			`KillMode=none`
systemd/ceph-disk: reduce ceph-disk flock contention "ceph-disk trigger" invocation is currently performed in a mutually exclusive fashion, with each call first taking an flock on the path /var/lock/ceph-disk. On systems with a lot of osds, this leads to a large amount of lock contention during boot-up, and can cause some service instances to trip the 120 second timeout. Take an flock on a device specific path instead of /var/lock/ceph-disk, so that concurrent "ceph-disk trigger" invocations are permitted for independent osds. This greatly reduces lock contention and consequently the chance of service timeout. Per-device concurrency restrictions required for http://tracker.ceph.com/issues/13160 are maintained. Fixes: http://tracker.ceph.com/issues/18049 Signed-off-by: David Disseldorp <ddiss@suse.de> 2016-11-28 13:16:22 +00:00			`ExecStart=/bin/sh -c 'timeout 120 flock /var/lock/ceph-disk-$(basename %f) /usr/sbin/ceph-disk --verbose --log-stdout trigger --sync %f'`
systemd: activate disks via systemd service instead of udev The udev(7) man page states: RUN ... This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Starting daemons or other long-running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished. ceph-disk activate is far from a short-running task: - check whether path is a block dev, for dirs call through to activate_dir() - call blkid to obtain the filesystem type for the block dev - pull mount options from hard-coded ceph.conf file - mount the OSD dev at a temporary path - check the ceph magic for mounted filesystem - read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf path - read or generate (if missing) the OSD uuid - create a file indicating init system usage (systemd) - mount the device at a second (final) location - umount (lazy) the temporary mount path - enable the systemd ceph-osd@{osd_id} service - start the systemd ceph-osd@{osd_id} service This logic is therefore best left in a systemd service for execution. As it is less limited in terms of execution time, and also allows for improved event handling in future (fsck, dmcrypt mapping etc.). This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or ceph-disk activate-journal via new ceph-disk-activate-journal@.service, ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service systemd service files. ceph-disk-dmcrypt-activate@.service makes use of the newly added --dmcrypt parameter for ceph-disk activate. Signed-off-by: David Disseldorp <ddiss@suse.de> 2015-05-11 23:45:34 +00:00			`TimeoutSec=0`