Instead of storing the dmcrypt keys in the /etc/ceph/dmcrypt-keys
directory, they are stored in the monitor. If a machine with
OSDs created with ceph-disk prepare --dmcrypt is lost, it does
not contain the key that would allow to decrypt their content.
The dmcrypt key is retrieved from the monitor using a different keyring
for each OSD. It is stored in a small partition called the lockbox. At
boot time the lockbox is mounted
/var/lib/ceph/osd-lockbox/$uuid
and used when the $uuid partition is detected by udev to map it with
cryptsetup.
The OSDs that were prepared prior to the lockbox implementation are
supported by looking up the key found in /etc/ceph/dmcrypt-keys before
looking in /var/lib/ceph/osd-lockbox/$uuid.
http://tracker.ceph.com/issues/14669Fixes: #14669
Signed-off-by: Loic Dachary <loic@dachary.org>
Copy paste the journal code and s/journal/block/
More work will be needed to support multiple auxiliary
devices (block.wal etc). But the goal is to minimize the change because
this commit is part of a series of commits focusing on refactoring
prepare, not the entire ceph-disk codebase.
Signed-off-by: Loic Dachary <loic@dachary.org>
On udev change the owner of the device switch back to the default. If
that happens on a journal while an OSD is being activated, it will fail
with permission denied.
Make sure all ceph device types are chown to ceph on udev change.
http://tracker.ceph.com/issues/13000Fixes: #13000
Signed-off-by: Loic Dachary <ldachary@redhat.com>
A multipath device is detected because there is a
/sys/dev/block/M:m/dm/uuid file with the mpath- prefix (or part\w+-mpath
prefix).
When ceph-disk prepares data or journal devices on a multipath device,
it sets the partition typecode to MPATH_JOURNAL_UUID, MPATH_OSD_UUID and
MPATH_TOBE_UUID to
a) help the udev rules distinguish them from other devices in
devicemapper
b) allow ceph-disk to fail if an attempt is made to activate a device
with this type without accessing it via a multipath device
The 95-ceph-osd.rules call ceph-disk activate on partitions of type
MPATH_JOURNAL_UUID, MPATH_OSD_UUID. It relies on ceph-disk to do nothing
if the device is not accessed via multipath.
http://tracker.ceph.com/issues/11881Fixes: #11881
Signed-off-by: Loic Dachary <ldachary@redhat.com>
LUKS allows for validation of the key at mount time (rather than
simply mounting a random partition), specification of the encryption
parameters in the header and key rollover of the slot key (the one
that needs to be stored).
New parameters 'osd cryptsetup parameters' and 'osd dmcrypt key size' are
added. These allow these important policy choices to be overridden or
kept consistent per-site.
The previous default plain mode (rather than using LUKS) remains, select
LUKS by setting 'osd dmcrypt type = luks'
Signed-off-by: Andrew Bartlett <abartlet@catalyst.net.nz>
Activate an osd via its journal device. udev populates its symlinks and
triggers events in an order that is not related to whether the device is
an osd data partition or a journal. That means that triggering
'ceph-disk activate' can happen before the journal (or journal symlink)
is present and then fail.
Similarly, it may be that they are on different disks that are hotplugged
with the journal second.
This can be wired up to the journal partition type to ensure that osds are
started when the journal appears second.
Include the udev rules to trigger this.
Signed-off-by: Sage Weil <sage@inktank.com>
Automatically map encrypted journal partitions.
For encrypted OSD partitions, map them, wait for the mapped device to
appear, and then ceph-disk-activate.
This is much simpler than doing the work in ceph-disk-activate.
Signed-off-by: Sage Weil <sage@inktank.com>
Zhi Zhang
Loic Dachary
Sage Weil
Andrew Bartlett
Sage Weil
Alexandre Marangone