ceph/systemd/ceph-disk@.service

[Unit]
Description=Ceph disk activation: %f
After=local-fs.target
Wants=local-fs.target

[Service]
Type=oneshot
KillMode=none
Environment=CEPH_DISK_TIMEOUT=10000
ExecStart=/bin/sh -c 'timeout $CEPH_DISK_TIMEOUT flock /var/lock/ceph-disk-$(basename %f) /usr/sbin/ceph-disk --verbose --log-stdout trigger --sync %f'
TimeoutSec=0
systemd: activate disks via systemd service instead of udev The udev(7) man page states: RUN ... This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Starting daemons or other long-running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished. ceph-disk activate is far from a short-running task: - check whether path is a block dev, for dirs call through to activate_dir() - call blkid to obtain the filesystem type for the block dev - pull mount options from hard-coded ceph.conf file - mount the OSD dev at a temporary path - check the ceph magic for mounted filesystem - read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf path - read or generate (if missing) the OSD uuid - create a file indicating init system usage (systemd) - mount the device at a second (final) location - umount (lazy) the temporary mount path - enable the systemd ceph-osd@{osd_id} service - start the systemd ceph-osd@{osd_id} service This logic is therefore best left in a systemd service for execution. As it is less limited in terms of execution time, and also allows for improved event handling in future (fsck, dmcrypt mapping etc.). This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or ceph-disk activate-journal via new ceph-disk-activate-journal@.service, ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service systemd service files. ceph-disk-dmcrypt-activate@.service makes use of the newly added --dmcrypt parameter for ceph-disk activate. Signed-off-by: David Disseldorp <ddiss@suse.de> 2015-05-11 23:45:34 +00:00			`[Unit]`
			`Description=Ceph disk activation: %f`
ceph-disk: systemd unit must run after local-fs.target A ceph udev action may be triggered before the local file systems are mounted because there is no ordering in udev. The ceph udev action delegates asynchronously to systemd via ceph-disk@.service which will fail if (for instance) the LVM partition required to mount /var/lib/ceph is not available yet. The systemd unit will retry a few times but will eventually fail permanently. The sysadmin can systemctl reset-fail at a later time and it will succeed. Add a dependency to ceph-disk@.service so that it waits until the local file systems are mounted: After=local-fs.target Since local-fs.target depends on lvm, it will wait until the lvm partition (as well as any dm devices) is ready and mounted before attempting to activate the OSD. It may still fail because the corresponding journal/data partition is not ready yet (which is expected) but it will no longer fail because the lvm/filesystems/dm are not ready. Fixes: http://tracker.ceph.com/issues/17889 Signed-off-by: Loic Dachary <loic@dachary.org> 2016-11-22 13:45:45 +00:00			`After=local-fs.target`
			`Wants=local-fs.target`
systemd: activate disks via systemd service instead of udev The udev(7) man page states: RUN ... This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Starting daemons or other long-running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished. ceph-disk activate is far from a short-running task: - check whether path is a block dev, for dirs call through to activate_dir() - call blkid to obtain the filesystem type for the block dev - pull mount options from hard-coded ceph.conf file - mount the OSD dev at a temporary path - check the ceph magic for mounted filesystem - read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf path - read or generate (if missing) the OSD uuid - create a file indicating init system usage (systemd) - mount the device at a second (final) location - umount (lazy) the temporary mount path - enable the systemd ceph-osd@{osd_id} service - start the systemd ceph-osd@{osd_id} service This logic is therefore best left in a systemd service for execution. As it is less limited in terms of execution time, and also allows for improved event handling in future (fsck, dmcrypt mapping etc.). This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or ceph-disk activate-journal via new ceph-disk-activate-journal@.service, ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service systemd service files. ceph-disk-dmcrypt-activate@.service makes use of the newly added --dmcrypt parameter for ceph-disk activate. Signed-off-by: David Disseldorp <ddiss@suse.de> 2015-05-11 23:45:34 +00:00
			`[Service]`
			`Type=oneshot`
ceph-disk: systemd must not kill a running ceph-disk When activating a device, ceph-disk trigger restarts the ceph-disk systemd service. Two consecutive udev add on the same device will restart the ceph-disk systemd service and the second one may kill the first one, leaving the device half activated. The ceph-disk systemd service is instructed to not kill an existing process when restarting. The second run waits (via flock) for the second one to complete before running so that they do not overlap. http://tracker.ceph.com/issues/13160 Fixes: #13160 Signed-off-by: Loic Dachary <ldachary@redhat.com> 2015-09-20 11:54:05 +00:00			`KillMode=none`
ceph-disk: set the default systemd unit timeout to 3h There needs to be a timeout to prevent ceph-disk from hanging forever. But there is no good reason to set it to a value that is less than a few hours. Each OSD activation needs to happen in sequence and not in parallel, reason why there is a global activation lock. It would be possible, when an OSD is using a device that is not otherwise used by another OSD (i.e. they do not share an SSD journal device etc.), to run all activations in parallel. It would however require a more extensive modification of ceph-disk to avoid any chances of races. Fixes: http://tracker.ceph.com/issues/20229 Signed-off-by: Loic Dachary <loic@dachary.org> 2017-06-08 20:29:48 +00:00			`Environment=CEPH_DISK_TIMEOUT=10000`
systemd/ceph-disk: make it possible to customize timeout When booting a server with 20+ HDDs udev has to process a lot of events (especially if dm-crypt is used), and 2 minutes might be not enough for that. Make it possible to override the timeout (via systemd drop-in files), and use a longer timeout (5 minutes) by default. Fixes: http://tracker.ceph.com/issues/18740 Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com> 2017-01-31 09:54:38 +00:00			`ExecStart=/bin/sh -c 'timeout $CEPH_DISK_TIMEOUT flock /var/lock/ceph-disk-$(basename %f) /usr/sbin/ceph-disk --verbose --log-stdout trigger --sync %f'`
systemd: activate disks via systemd service instead of udev The udev(7) man page states: RUN ... This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device. Starting daemons or other long-running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished. ceph-disk activate is far from a short-running task: - check whether path is a block dev, for dirs call through to activate_dir() - call blkid to obtain the filesystem type for the block dev - pull mount options from hard-coded ceph.conf file - mount the OSD dev at a temporary path - check the ceph magic for mounted filesystem - read cluster uuid and locate corresponding /etc/ceph/{cluster}.conf path - read or generate (if missing) the OSD uuid - create a file indicating init system usage (systemd) - mount the device at a second (final) location - umount (lazy) the temporary mount path - enable the systemd ceph-osd@{osd_id} service - start the systemd ceph-osd@{osd_id} service This logic is therefore best left in a systemd service for execution. As it is less limited in terms of execution time, and also allows for improved event handling in future (fsck, dmcrypt mapping etc.). This change sees 95-ceph-osd.rules.systemd trigger ceph-disk activate or ceph-disk activate-journal via new ceph-disk-activate-journal@.service, ceph-disk-activate@.service and ceph-disk-dmcrypt-activate@.service systemd service files. ceph-disk-dmcrypt-activate@.service makes use of the newly added --dmcrypt parameter for ceph-disk activate. Signed-off-by: David Disseldorp <ddiss@suse.de> 2015-05-11 23:45:34 +00:00			`TimeoutSec=0`