From 0befc9c9dfd495ad3f64b154383a114f69cb51ac Mon Sep 17 00:00:00 2001 From: Willem Jan Withagen Date: Mon, 21 Nov 2016 11:39:44 +0100 Subject: [PATCH] README.FreeBSD: update current status Signed-off-by: Willem Jan Withagen --- README.FreeBSD | 54 +++++++++++++++++++++++--------------------------- 1 file changed, 25 insertions(+), 29 deletions(-) diff --git a/README.FreeBSD b/README.FreeBSD index a4a32c1f5db..edca6b3c200 100644 --- a/README.FreeBSD +++ b/README.FreeBSD @@ -1,48 +1,57 @@ + +Last updated: 2016-11-21 + The FreeBSD build will build most of the tools in Ceph. -Note that the RBD dependant items will not work since FreeBSD does not +Note that the (kernel) RBD dependant items will not work since FreeBSD does not have RBD (yet) I started looking into Ceph, because the HAST solution with CARP and ggate did not really do what I was looking for. But I'm aiming for -running a Ceph storage cluster of storage nodes that are running ZFS. -End station would be running bhyve on RBD disk that are stored in Ceph. +running a Ceph storage cluster on storage nodes that are running ZFS. +In the end the cluster would be running bhyve on RBD disk that are stored in +Ceph. Progress from last report: ========================== Most important change: - - All test (but one) run to completion for the current selection of + - All test run to completion for the current selection of tools. This is only the case for "My Fork" repository. Some of the commits need to be pulled into the HEAD - - - Cmake is now the only way of building Ceph + - As of now Cmake is the only way of building Ceph + - And testing would be best done thru ctest. - Reworked threading/polling code for the simple socket code. Now uses a selfpipe, instead of using an odd shutdown() signaling Linux feature. - Modified the EventKqueue code to work around the "feature" that - starting threads destroys the the kqueue handles. Just finshed the - code so it is not yet submitted to the main repository - + starting threads destroys the the kqueue handles. + - ceph-disk should now be able to support FileStore on a ZFS disk. + The main reason that it needs to be ZFS is for xattribute: Size and number. + Build Prerequisites =================== - Compiling and building Ceph is tested on 12-CURRENT, but I guess + Compiling and building Ceph is tested on 12-CURRENT, but I guess/expect 11-RELEASE will also work. And Clang is at 3.8.0. - It uses the CLANG toolset that is available, 3.7 is not tested, but was - working when that was with 11-CURRENT. + It uses the CLANG toolset that is available, 3.7 is no longer tested, + but was working when that was with 11-CURRENT. Clang 3.4 (on 10.2-STABLE) does not have all required capabilites to compile everything The following setup will get things running for FreeBSD: + + This all require root privilidges. - Install bash and link it in /bin - # requires root privileges sudo pkg install bash sudo ln -s /usr/local/bin/bash /bin/bash - Need to add one compatability line to /usr/include/errno.h #define ENODATA 87 /* Attribute not found */ (Otherwise some cython compiles will fail.) + - getopt is used by several testscripts but it requires more than what + the native getopt(1) delivers. So best is to install getopt from ports + and remove/replace the getopt in /usr/bin. Getting the FreeBSD work on Ceph: ================================= @@ -57,8 +66,6 @@ Building Ceph - Go and start building ./do_freebsd.sh -Note that the old way of building using automake is no longer used. - Parts not (yet) included: ========================= @@ -82,18 +89,9 @@ from the testset Tests not (yet) include: ======================= - - run-tox-ceph-disk - This is the main test that tests the basic Ceph disk install tool. - One of the next things to complete porting for. - - ceph-detect-init/run-tox.sh - Because the current implementation does not know anything about - FreeBSD rc-init. - - Tests that make use of nosetests. - Calling these does not really work since nosetests is not in - /usr/bin, and /usr/bin/env nosetests does not work on FreeBSD. - test/pybind/test_ceph_argparse.py - test/pybind/test_ceph_daemon.py - + - None, although some test can fail if running tests in parallel and there is + not enough swap. Then tests will start to fail in strange ways. + Task to do: =========== - Build an automated test platform that will build ceph/master on @@ -108,8 +106,6 @@ Task to do: - Run integration tests to see if the FreeBSD daemons will work with a Linux Ceph platform. - - Get the python tests that are currently excluded to work, and test OKE - - Compile and test the user space RBD (Rados Block Device). - Investigate and see if an in-kernel RBD device could be developed a la