diff --git a/doc/dev/osd-class-path.rst b/doc/dev/osd-class-path.rst index d0e54acac3b..66796eedd4a 100644 --- a/doc/dev/osd-class-path.rst +++ b/doc/dev/osd-class-path.rst @@ -4,13 +4,25 @@ :: - 2011-12-05 17:41:00.994075 7ffe8b5c3760 librbd: failed to assign a block name for image - create error: error 5: Input/output error + $ rbd create rbd/test --size 100M + 2021-03-16 01:26:59.012 7fe41426f080 -1 librbd::PoolMetadata: list: failed listing metadata: (95) Operation not supported + 2021-03-16 01:26:59.012 7fe41426f080 -1 librbd::Config: apply_pool_overrides: failed to read pool config overrides: (95) Operation not supported + 2021-03-16 01:26:59.012 7fe400ff9640 -1 librbd::image::CreateRequest: 0x55d62341bb30 handle_add_image_to_directory: error adding image to directory: (95) Operation not supported + rbd: create error: (95) Operation not supported -This usually happens because your OSDs can't find ``cls_rbd.so``. They -search for it in ``osd_class_dir``, which may not be set correctly by -default (http://tracker.ceph.com/issues/1722). +After adding ``--debug-ms=1``, you can see which OSD is contacted. +In the contacted OSD's log, you'll find the reason why this happens, +and usually it is because the "rados classes" can't be loaded at runtime with ``dlopen``: -Most likely it's looking in ``/usr/lib/rados-classes`` instead of -``/usr/lib64/rados-classes`` - change ``osd_class_dir`` in your -``ceph.conf`` and restart the OSDs to fix it. +:: + + 2021-03-16 01:26:59.013 7f6c6dff3640 10 _load_class rbd from /usr/local/lib64/rados-classes/libcls_rbd.so + 2021-03-16 01:26:59.013 7f6c6dff3640 0 _load_class could not stat class /usr/local/lib64/rados-classes/libcls_rbd.so: (2) No such file or directory + 2021-03-16 01:26:59.013 7f6c6dff3640 -1 osd.3 112 class rbd open got (2) No such file or directory + 2021-03-16 01:26:59.013 7f6c6dff3640 1 -- [...] --> [...] -- osd_op_reply(5 rbd_directory [call rbd.dir_add_image] v0'0 uv0 ondisk = -95 ((95) Operation not supported)) v8 -- 0x7f6c6800fed0 con 0x7f6cb80100c0 + +This means the OSD could not find ``libcls_rbd.so``. +You can customize the load path of these modules in ``ceph.conf`` with ``osd_class_dir``. +By default, this is ``$libdir/rados-classes``, so when developing, you likely have to adjust the path. + +These class libraries are used for extending RADOS, see :ref:`rados-objclass-api-sdk`. diff --git a/doc/rados/api/objclass-sdk.rst b/doc/rados/api/objclass-sdk.rst index 6b1162fd494..90b8eb01892 100644 --- a/doc/rados/api/objclass-sdk.rst +++ b/doc/rados/api/objclass-sdk.rst @@ -1,3 +1,5 @@ +.. _`rados-objclass-api-sdk`: + =========================== SDK for Ceph Object Classes ===========================