Commit Graph

398 Commits

Author SHA1 Message Date
haodong
f54e42f3a3 msg/async/rdma: cmake find_package for librdmacm
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2018-06-20 14:08:26 +08:00
haodong tang
a240aef143 msg/async/rdma: add RDMA iWARP protocol support
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2018-06-20 14:08:20 +08:00
Kefu Chai
265b6d179d cmake: disable -Werror-stringop-truncation for rocksdb
this option was introduced in GCC-8.1, and is enabled by default.
otherwise we will have:

src/rocksdb/util/status.cc:28:15: error: ‘char* strncpy(char*, const
char*, size_t)’ output truncated before terminating nul copying as many
bytes from a string as its length [-Werror=string
op-truncation]
   std::strncpy(result, state, cch - 1);
   ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
/var/ssd/ceph/src/rocksdb/util/status.cc:19:18: note: length computed
here
       std::strlen(state) + 1; // +1 for the null terminator
       ~~~~~~~~~~~^~~~~~~
cc1plus: all warnings being treated as errors

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-18 14:18:47 +08:00
Kefu Chai
4c0a2b9373 cmake: add WITH_GTEST_PARALLEL option
and remove src/test/gtest-parallel submodule, because gtest-parallel is
only useful for running tests. and not all end-users are interested in
running test not to mention running them in parallel. so, to avoid
including gtest-parallel scripts in the dist tarball. it'd be better to
make it optional, and an external project.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-15 15:55:15 +02:00
Erwan Velu
13bc625b5f src/test: Using gtest-parallel to speedup unittests
Unittests are run sequentially and could take a long while to run.

This commit is about using gtest-parallel on some of them which are
known to be very slow due to this sequentiality.

To enable the parallel features, the 'parallel' argument just have to be
added to the add_ceph_unittest() call like in :
    -add_ceph_unittest(unittest_throttle)
    +add_ceph_unittest(unittest_throttle parallel)

This commit impact the following tests :

Test name                          Before   After (in seconds)
unittest_erasure_code_shec_all:       212      43
unittest_throttle                      15       5
unittest_crush                          9       6
unittest_rbd_mirror                    79      21

Total                                 315      75

This commit saves 240 seconds (4 minutes) per build.

Note it exist several other long tests but can't be parallelized since
there is explicit dependencies in the order to run the subtests.
Those stay sequential.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-06-15 15:53:30 +02:00
Kefu Chai
c2026b7473 cmake: update BuildSPDK for spdk-18.05
in spdk v18.05, libuuid is linked by libspdk_util.a, in which,
it is used by lib/util/uuid.c. and libspdk_vol.a uses the wrapper
function exposed by libspdk_util.a, so update the CMakefile script to
reflect the change.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-14 09:34:01 +08:00
Willem Jan Withagen
0c63f42219 cmake: cmake: oath lives in liboath
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-06-10 16:21:41 +02:00
Kefu Chai
89ed08569b
Merge pull request #22439 from shaba/devel/fix-rocksdb
cmake: fix find system rockdb

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-06-08 21:22:45 +08:00
Alexey Shabalin
2ac26bd0e0 cmake: fix find system rockdb
- fix path for version.h
- fix typo ROCKDB -> ROCKSDB

Signed-off-by: Alexey Shabalin <shaba@altlinux.org>
2018-06-07 14:19:41 +03:00
Kefu Chai
ca78ab2254 cmake: find liboath using the correct name
we should reference liboath by the $name in Find${name}.cmake, also the
$name should be consistent when calling find_package_handle_standard_args().
in this change
* rename Findliboath.cmake to FindOATH.cmake to be consistent with other
  find_package() moduless.
* use "OATH" in find_package_handle_standard_args() instead of "oath"
* set the interface properties for OATH::OATH, so the target linking
  against it can reference its header directories and libraries automatically.
* remove the stale comment for find_package_handle_standard_args()
* set OATH_INCLUDE_DIRS and OATH_LIBRARIES to follow the convention of
  find_package(), even they are not used directly in this project.

Reported-by: Erwan Velu <erwan@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-06 22:49:35 +08:00
Igor Fedotov
83841bf3de build/cmake: enable RTTI for both debug and release RocksDB builds.
Overwise ceph build in Release mode is failing.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-05-28 23:00:42 +03:00
Kefu Chai
2d351c2893
Merge pull request #22019 from majianpeng/cmake-fio
cmake: update fio version from 3.5 to 540e235dcd276e63c57.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-05-27 19:37:59 +08:00
Kefu Chai
77b35faa36
Merge pull request #22204 from tchaikov/wip-cmake-silence-bjam
cmake: always turn off bjam debugging output

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-05-25 22:25:22 +08:00
Kefu Chai
3766cdff1f cmake: always turn off bjam debugging output
we have 'DH_VERBOSE=1' in debian/rules, which instructs debhelper to
pass -DCMAKE_VERBOSE_MAKEFILE=ON to cmake. but the verbose output for
building boost does not really help if something goes wrong while
building the deb packages. if we do want to enable the verbose output,
we can always enable it by passing `-d <N>' to it. see
https://boostorg.github.io/build/manual/develop/index.html#bbv2.overview.invocation.options
.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-24 16:06:09 +08:00
Kefu Chai
71462ddf05 cmake/modules/BuildRocksDB.cmake: enable compressions for rocksdb
we should enable them if they are found.

currently, we don't have bzip2 compressor plugin, so it's not detected
in the cmake script. we can always enable it for rocksdb in future.

Fixes: http://tracker.ceph.com/issues/24025
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-23 23:59:41 +08:00
Kefu Chai
4d646ef579 cmake: do not check for aligned_alloc() anymore
this reverts 16f0a67d

Fixes: http://tracker.ceph.com/issues/23653
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-17 10:33:04 +08:00
Jianpeng Ma
56ce1b793c cmake: update fio version from 3.5 to 540e235dcd276e63c57.
Fio commit(d04f1d5b342e7733) make external ioengine work.
This need when build libfio_ceph_objectstore.so.

Meanwhile fix a comiler error:
/home/ceph/src/test/fio/fio_ceph_objectstore.cc: In constructor ‘{anonymous}::ceph_ioengine::ceph_ioengine()’:
/home/ceph/src/test/fio/fio_ceph_objectstore.cc:733:19: error: invalid conversion from ‘int (*)(thread_data*, io_u*)’ to ‘fio_q_status (*)(thread_data*, io_u*)’ [-fpermissive]
     queue       = fio_ceph_os_queue;
                   ^~~~~~~~~~~~~~~~~
src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/build.make:62: recipe for target 'src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/fio_ceph_objectstore.cc.o' failed

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-05-16 17:56:20 +08:00
Dan Mick
39b2626c7a Remove embedded 'cephd' code
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2018-05-11 13:10:26 -07:00
Kefu Chai
19230378c8 cmake: specify missing settings for dpdk
* after upgrading to gcc-8, seems dpdk requires more settings to compile.
  in which, CONFIG_RTE_MAX_VFIO_GROUPS=64 is copied from config/common_base.
* librte_bus_pci.a depends on librte_pci.a, so reorder the libraries
  in DPDK_LIBRARIES.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-06 15:26:14 +08:00
Kefu Chai
50a052d79e
Merge pull request #19714 from qwren/qatzip-compressor
compressor: add QAT support

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-05-04 09:17:17 +08:00
Kefu Chai
791af5459e cmake: enable CMP0051
the new behavior of get_target_property(<var> <target> SOURCES) will be
enforced in future versions of cmake, so let ready for this change now
by removing the generator expressions from the returned source file
list.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-26 16:47:54 +08:00
Kefu Chai
4aea3bed7f cmake: enable CMP0046
so cmake will error on "non-existent dependency in add_dependencies", we
add dependency on CONFIGURE_FILE() output, which is wrong, as its output
is not a target, and is generated when cmake runs. so remove them from
dependencies.

regarding to ceph_ver.h, its path is also wrong. it is created under
${CMAKE_BINARY_DIR}/src/include. so this is another reason to remove it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-26 16:44:09 +08:00
Kefu Chai
53a1500a8f cmake,make-dist: bump up boost version to 1.67
* to pick up the fix of https://svn.boost.org/trac10/ticket/11622
* also the boost::python's library name now includes the version suffix
  of python version, so update BuildBoost.cmake accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 09:02:28 +08:00
Kefu Chai
112f79f8de cmake: s/boost_256/boost_sha256/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:33:24 +08:00
Kefu Chai
3178bb9638 cmake: disable gflags support in rocksdb
to silence the warnings like

CMake Warning at CMakeLists.txt:73 (find_package):
  By not providing "Findgflags.cmake" in CMAKE_MODULE_PATH this project
has
  asked CMake to find a package configuration file provided by "gflags",
but
  CMake did not find one.

  Could not find a package configuration file provided by "gflags" with
any
  of the following names:

    gflagsConfig.cmake
    gflags-config.cmake

  Add the installation prefix of "gflags" to CMAKE_PREFIX_PATH or set
  "gflags_DIR" to a directory containing one of the above files.  If
"gflags"
  provides a separate development package or SDK, be sure it has been
  installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:17:37 +08:00
Kefu Chai
19aee456c0 cmake,make-dist: use sha256 instead of md5 for checksum
the boost's download page offers the SHA256 hash, so it'd be easier to
verify the hash this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:17:35 +08:00
Qiaowei Ren
9f3965aef3 compressor: add QAT support
This patch adds new QATzip plugin to support QAT for compression.

QATZip is a user space library which builds on top of the Intel
QAT (QuickAssist Technology) user space library, to provide extended
accelerated compression and decompression services by offloading the
actual compression and decompression request(s) to the hardware
QAT accelerators, which are more efficient in terms of cost and power
than general purpose CPUs for those specific compute-intensive
workloads.

Based on QAT accelerators, QATZip can support several compression
algorithm, including deflate, snappy, lz4, etc..

Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com>
2018-04-18 12:05:44 +08:00
Kefu Chai
faa84ab975 cmake: expose Boost_VERSION
to appease the consumers of FindBoost.cmake, they are expecting this
variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-13 12:19:36 +08:00
Kefu Chai
a15d3d39a7 cmake: rename the boost header target to Boost::boost
because FindBoost.cmake defines Boost::boost as the target for header
only dependencies.

due to the limit of cmake < 3.1, this only applies to cmake > 3.1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-13 12:19:36 +08:00
Yehuda Sadeh
c3a6f7588a rgw: mfa - initial work
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Igor Fedotov
939805165e test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-03-05 20:35:08 +03:00
Kefu Chai
9535165149 cmake: add implicit dependencies between boost libs
so the dependencies can be included when linking against the one which
depends on them. for example, libboost_filesystem depends on
libboost_system, if we don't link against the latter, linker will bail
out when linking an executable using Boost::filesystem.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-27 21:26:42 +08:00
Kefu Chai
854ca24b19 cmake: add an alias for unit_test_framework
should pass --with-test for building Boost::unit_test_framework

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-27 21:26:42 +08:00
Casey Bodley
a467e5b1cf cmake: skip boost dependency on ALIAS targets
the boost override for add_library() does not work for ALIAS targets:

CMake Error at cmake/modules/BuildBoost.cmake:227 (add_dependencies):
  Cannot add target-level dependencies to alias target "Seastar::seastar".

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-02-27 21:26:42 +08:00
Kefu Chai
d051bf0441 cmake: disable DOWNLOAD_NO_PROGRESS if cmake ver is lower than 3.1
see https://cmake.org/cmake/help/v3.1/module/ExternalProject.html and
https://cmake.org/cmake/help/v3.0/module/ExternalProject.html. the
former has DOWNLOAD_NO_PROGRESS, while the latter does not.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-20 06:40:15 +08:00
Kefu Chai
2277a10279 cmake: fix the include dir for building boost::python
the "includes" directories are separated by spaces not semicolons.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-06 00:36:26 +08:00
Kefu Chai
6f725e76d5 cmake: build Boost.python with specified version of python
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-25 22:26:10 +08:00
Casey Bodley
ba85740955 cmake: remove cryptopp option
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-19 10:25:41 -05:00
Sage Weil
755ea1c054
Merge pull request #19914 from tchaikov/wip-cmake-check-boost-in-src
cmake: check version of boost in src/boost

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2018-01-13 09:56:08 -06:00
Kefu Chai
fd5a522928 cmake: check version of boost in src/boost
after bumping up the required boost version, old copy of boost in
src/boost will not work. so we need to error out early.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-11 13:19:15 +08:00
Kefu Chai
9345a28a8a cmake: BuildBoost.cmake: use specified compiler for building boost
* user-config.jam is the suggested way for adding toolset for Boost.Build.
* always specify toolset when running b2, so we can build boost with
specified compiler instead of the default one guessed by boost building system.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-10 23:13:39 +08:00
Casey Bodley
0e47a339df cmake: update minimum boost version to 1.66
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-05 13:52:47 -05:00
Kefu Chai
6559a85051 cmake: disable FAIL_ON_WARNINGS for rocksdb
otherwise -Werror=implicit-fallthrough will fail the build with GCC-7

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-11 14:09:45 +08:00
Kefu Chai
99c0306324
Merge pull request #19208 from tchaikov/wip-spdk-17.10
spdk: update SPDK to v17.10

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2017-11-30 00:39:18 +08:00
Kefu Chai
b3ceafe7d8 cmake,rpm,deb: update to accommodate SPDK v17.10
* cmake/modules/BuildSPDK.cmake: add lvol
* cmake/modules/BuildDPDK.cmake: add pci and bus_pci
* ceph.spec.in, cmake/modules/BuildSPDK.cmake, debian/control:
  re-introduce libuuid dependency, as 17.07 added lvol. and the latter
  depends on uuid.
* cmake/modules/BuildSPDK.cmake: avoid introducing local variable of
  `iface_libs`.
* cmake/modules/patch-dpdk-conf.sh: disable
  CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES, this option introduces the
  balanced allocation of memory. but it also requires libnuma-dev.
  let's disable it for now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-29 18:03:23 +08:00
Kefu Chai
12dc5733c9
Merge pull request #18938 from tchaikov/wip-cmake
cmake: check gcc version not release date for libstdc++ saneness

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-29 15:38:16 +08:00
Kefu Chai
e3b703eb12 cmake: check bootstrap.sh instead before downloading boost
config.hpp does not exist if boost is not configured. so a boost source
directory unpacked from dist tarball does not have this file. the
bootstrap.sh instead, it is used to create the necessary config files
for jam.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-24 11:13:16 +08:00
Kefu Chai
9b3f9cd253 cmake: BuildDPDK: use ${CMAKE_MAKE_PROGRAM} when appropriate
DPDK requires GNU Make. see
spdk/dpdk/doc/guides/freebsd_gsg/build_dpdk.rst. so, we should use the
configured make program to do the job.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-21 17:47:17 +08:00
Kefu Chai
7c3221fda1 cmake: BuildDPDK: use POSIX compliant syntax
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-21 17:06:50 +08:00
Kefu Chai
a11ab928f7 cmake: check libaio on linux before building SPDK
SPDK requires libaio on linux.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 11:18:02 +08:00
Kefu Chai
e8add075ec cmake: add rte_mempool_{ring,stack} to Finddpdk.cmake
* also refactor it a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:44:56 +08:00
Kefu Chai
9964ac9251 cmake: BuildDPDK: do not build kernel modules
see also http://pkgs.fedoraproject.org/cgit/rpms/dpdk.git/tree/dpdk.spec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:44:56 +08:00
Kefu Chai
01a9f17825 cmake: build spdk/dpdk in cmake modules
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: wanjun.lp <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
2017-11-17 15:37:53 +08:00
Kefu Chai
571a786afc cmake: pass LDSHARED env var to distutils
otherwise, the default gcc will be used, and the $CMAKE_C_COMPILER
passed to the outer CMakeLists.txt won't kick in. moreover, if the
building script (ceph.spec for instance) could set the $PATH, and
expect that the CMakeLists.txt will use the toolchain executables
in the $PATH to build Ceph, distutils will continue using the default
$CC for linking the python bindings, on UNIX it will be gcc in the
new shell's $PATH, because we are using `install(CODE "... execute_process(
...))` for installing the python bindings. apparently, this is not
expected. because the new shell's $PATH is very likely different
from the one changed by the building script. to address this, we
should always specify the `$LDSHARED` env var explicitly.

also, pass env vars using `ENV{}` instead of the `env` command to
workaround the issue of https://cmake.org/pipermail/cmake/2015-December/062216.html,
because it's not straightforward to set environment variables with
spaces in the them using cmake. and because one cannot use add_custom_target()
in the script mode of cmake. this leave me only limited options to
fix this issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-15 15:22:07 +08:00
Kefu Chai
70b5b4bfbf cmake: should check the availability of aligned_alloc() by running the test
this mimics the failure of rocksdb without a working
aligned_alloc()/free().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-31 17:39:09 +08:00
Kefu Chai
599a9ef3ca cmake: should use the value of GPERFTOOLS_LIBRARIES as REQUIRED_VARS
otherwise it's always true, even if the libaries are not found. also
the cmake output will be wrong without this change:

without this change, it is:

-- Found gperftools: GPERFTOOLS_TCMALLOC_LIBRARY;GPERFTOOLS_TCMALLOC_MINIMAL_LIBRARY;GPERFTOOLS_PROFILER_LIBRARY (found version "2.5.93")

with this change, it would be:

-- Found gperftools: /tmp/gperf/lib/libtcmalloc.so (found version "2.5.93")

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-31 16:58:04 +08:00
Kefu Chai
16f0a67d83 cmake: check for aligned_alloc()
in case developer is using patched tcmalloc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-27 11:13:15 +08:00
Jianpeng Ma
b4cc9a9aa8 cmake: Build libpmem from source code.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-20 00:44:09 +08:00
Kefu Chai
caae3c4757 cmake: fix gperftools version detection
there is chance that its "patch" version is empty. in that case, do not
include it in its version string. otherwise, we will have something
like:

Found gperftools:
GPERFTOOLS_TCMALLOC_LIBRARY;GPERFTOOLS_TCMALLOC_MINIMAL_LIBRARY;GPERFTOOLS_PROFILER_LIBRARY
(found version "2.4.#define TC_VERSION_PATCH  """)

when running cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-13 15:11:09 +08:00
Sage Weil
96c831167b Merge pull request #18092 from tchaikov/wip-cmake-cleanup
cmake: do not link against unused or duplicated libraries

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-05 11:18:27 -05:00
Brad Hubbard
598556f98b make-dist,cmake: Try multiple URLs to download boost before failing
Remove SPOF during boost download for make-dist and later cmake > 3.7

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-04 11:13:34 +10:00
Kefu Chai
2f4cc32741 cmake: always use CMAKE_RUNTIME_OUTPUT_DIRECTORY as the unittests' path
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-03 23:44:28 +08:00
Sage Weil
19987549e0 make-dist,cmake: move boost tarball location to download.ceph.com
Sourceforge is down.  Also, we can hammer our servers instead of
theirs.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-26 17:22:36 -04:00
Kefu Chai
04697cdbdc cmake: update the error message for gperftools bug
it's a follow-up of #17788

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-22 10:40:29 +08:00
Kefu Chai
ff8d6a730b cmake: error out if rocksdb is incompatible w/ tcmalloc
the commit d406f228 in gperf implements a c11 feature used by a
recent change in rocksdb: 16e03882, which uses aligned_alloc().
and 16e03882 in rocksdb was merged after v5.7 was tagged, while
16e03882 in gperf was merged after v2.6.1 was tagged.

because aligned_alloc() is not implemented by tcmalloc until the
not-yet-released 2.6.2, if we call aligned_alloc() in an application
linked against tcmalloc, what gets called will be the glibc's
aligned_alloc(). but if we free() the memory chunk allocated by
aligned_alloc(), the tcmalloc's implementation kicks in, then
InvalidFree() is called, because the memory chunk being freed was
allocated by tcmalloc. in short, "mixing allocators", quote from
Dan Mick.

in rocksdb, aligned_alloc() is used if _ISOC11_SOURCE is defined, this
makes sense, because aligned_alloc() is a C11 function. we could avoid
using it by not defining _ISOC11_SOURCE. but as long as _GNU_SOURCE is
defined, glibc defines _ISOC11_SOURCE. and libstdc++ requires
_GNU_SOURCE, because it uses a fair amount of GNU extensions.

Fixes: http://tracker.ceph.com/issues/21422
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-20 10:23:16 +08:00
Kefu Chai
987c1b3ec2 cmake: detect rocksdb's version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
58179f098b cmake: detect gperf-tools' version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
aa603621bd cmake: extract BuildRocksDB into BuildRocksDB.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
da8ce07a21 cmake: enable find_package(perftools) to check non-default dir
so it checks $ENV{GPERF_ROOT} first.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
b567261208 cmake: require lz4 v1.7 for building lz4 compressor
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-08-17 16:01:55 +08:00
Boris Ranto
70473011a0 Merge pull request #16753 from ceph/wip-fix-ppc64
Fix ppc64 support for ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-14 15:41:28 +02:00
Dan Mick
de06f8e504 cmake/Distutils.cmake: make pkg files optional, add bin
Skip nonexistent 'expected' Python packaging files, and add bin/ as
a possible file for symlinking during build (for Python projects with
slightly different forms, currently ceph-volume)

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-04 10:25:57 -04:00
Boris Ranto
2172dfb594 Fix ppc64 support for ceph
The ppc64 support requires a couple of changes:
- adding the ppc64 support to cmake
- changing optimized crc32 code to compile on ppc64le only
- moving ifdef condition before crc32_align to avoid defined but not
used warning

Signed-off-by: Boris Ranto <branto@redhat.com>
2017-08-04 10:56:03 +02:00
Ilsoo Byun
8134620ad4 os/bluestore: fix build errors when spdk is on
- remove pthread library link from dpdk cmake
- fix the bugs of NVMEDevice

Signed-off-by: Ilsoo Byun <ilsoo.byun@sk.com>
2017-07-05 01:42:10 -04:00
Kefu Chai
f8d5f74663 cmake: compile libzfs backend conditionally
* do not REQUIRE libzfs if it is enabled, this follows the way how we
  handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
  little bit.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-27 23:28:03 +08:00
Kefu Chai
96b2016489 cmake: do not add dependencies to INTERFACE library on cmake < 3.3
otherwise we will have

add_dependencies Cannot add target-level dependencies to INTERFACE
library

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-21 23:34:51 +08:00
Kefu Chai
83432b4461 cmake: build boost as an external project
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-14 23:53:50 +08:00
Sage Weil
c2e346cf12 Merge pull request #15562 from tchaikov/wip-no-assert.h
common,osdc: remove atomic_t completely

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-08 20:38:23 -05:00
Sage Weil
b0e9f3ed51 Merge pull request #15102 from majianpeng/blockdevice-pmem
os/bluestore/BlockDevice: support pmem device as bluestore backend

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-08 13:15:52 -05:00
Kefu Chai
a87fa37e46 cmake,debian,rpm: remove atomic_t completely
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-08 12:21:00 +08:00
Jianpeng Ma
e25accff7d os/bluestore/BlockDevice:support pmem device as bluestore backend.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-06-07 23:22:52 +08:00
James Page
5d92c79b8e build: misc fixes for cmake build on i386
Scope SIMD CPU flag detection base on target architecture,
resolving build failures in gf-complete on i386; this is
inline with the previous autotools based feature detection.

Only compile crypto/isa-l if yasm 64 bit complier is found,
effectively limiting scope to x86_64.

Signed-off-by: James Page <james.page@ubuntu.com>
2017-06-06 16:34:50 +01:00
Nathan Cutler
4acd2cef35 cmake: align cmake names of library packages
The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"

Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.

Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-05-04 11:14:43 +02:00
Kefu Chai
4847a88556 cmake: do not try to add submodule to exclude list if .git is not around
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-13 13:49:54 +08:00
Kefu Chai
e1991238e7 cmake: support optional argument for overriding default ctag excludes
so instead of excluding all submodules, one can just exclude boost and
rocksdb using:

  cmake -DCTAG_EXCLUDES="boost;rocksdb" ..

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-07 12:06:52 +08:00
Dan Mick
3681683df7 cmake: Add simple recursive ctags target for Ceph source only
Excludes submodules and random JavaScript.

Signed-off-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-06 13:27:12 +08:00
Kefu Chai
195425c9ca cmake: extract ppc64le detection into SIMDExt.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-03 01:44:17 +08:00
Dan Mick
ac4815f74b cmake/modules/SIMDExt.cmake: armv8 crypto intrinsics
Test not only for -march support, but also the actual
presence of the intrinsic routines.  Not sure why, but gcc
4.8.5 passes the first but not the second.

Fixes: http://tracker.ceph.com/issues/19386
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-03-27 09:35:22 -07:00
Dan Mick
ef8980bad2 cmake/modules/SIMDExt.cmake: add whitespace
readability, man, readability

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-03-24 20:10:15 -07:00
Kefu Chai
0603794d53 Merge pull request #12977 from weixiaowilliam/crc32c_opt
crc32c: optimize aarch64 crc32c implementation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-02-18 17:13:50 +08:00
Adir Lev
74fc3f6b83 msg/async/rdma: check if exp verbs avail
issue: 975125

Change-Id: I9daa168ca0299887a7238a688508c773b98abde9
Signed-off-by: Adir Lev <adirl@mellanox.com>
Signed-off-by: Oren Duer <oren@mellanox.com>
2017-02-13 13:28:21 +00:00
wei xiao
b0ae5ae2ad crc32c: optimize aarch64 crc32c implementation
ARMv8 defines PMULL crypto instruction.
This patch optimizes crc32c calculate with the instruction when
available rather than original linear crc instructions.

ceph crc32c performance unit test shows that the optimization get
~ x3.90 speedups on ThunderX ARM Core@2.0GHz (Cavium)
~ x1.45 speedups on ARM Cortex-A57@2.1GHz (Huaiwei)
~ x1.16 speedups on ARM Cortex-A57@2.0GHz (Softiron)

Jira: ENTLLT-358
Change-Id: I657422cd20c9ca78237cd060210a5383f4122575
Signed-off-by: wei xiao <wei.xiao@linaro.org>
2017-02-04 06:20:50 +00:00
Bassam Tabbara
615b4b2600 cmake: support for external rocksdb
add support for building with an external rocksdb.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2017-01-16 09:33:32 -08:00
John Lin
fbe78efd8a cmake: search for nspr include files for both suffixes: nspr4 and nspr
Fixes: http://tracker.ceph.com/issues/18535
Signed-off-by: John Lin <johnlinp@gmail.com>
2017-01-16 16:35:41 +08:00
Haomai Wang
836c5686b5 Merge pull request #12776 from optimistyzy/201714
cmake: remove Findpciaccess.cmake

Reviewed-by: Kefu Chai <kefu@redhat.com>
Reviewed-by: Pan Liu <pan.liu@istuary.com>
2017-01-06 11:20:27 +08:00
Pascal Bach
c79bf93d02 cmake: search for Keyutils in default paths and cleanup
This makes it more portable

Remove /usr/include as it is searched by find_path anyway.
Remove unnecessary include that weren't used.


Signed-off-by: Pascal Bach <pascal.bach@nextrem.ch>
2017-01-04 12:47:17 +01:00
optimistyzy
52494f15a3 cmake,modules: remove Findpciaccess.cmake
Now SPDK uses the DPDK's library for PCI device
enumeration, we can remove this dependency

Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-01-04 12:32:30 +08:00
Pan Liu
2c2cae16cf dpdk: make compilation successful when use dpdk v16.11
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2017-01-02 16:41:08 +08:00
Pan Liu
1ed6a37611 bluestore: fixed compilation error when enable spdk
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-12-27 12:42:44 +08:00
Bassam Tabbara
bdf4db8c4c cmake: simplify find_package jemalloc
simplified the package to find jemalloc. this one matches
the one in src/rocksdb/cmake/modules. The existing one
was struggling to find jemalloc on some hosts.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2016-12-13 11:39:35 -05:00
Bassam Tabbara
c6067f48ec embedded: Add a skeleton libcephd library
libcephd is a library that contains ceph daemon code
that can be statically linked in other applications.

Added MergeStaticLibraries.cmake that can merge static libraries
to form a bigger one. This approach avoids the need to mess with
STATIC libraries all over the code base.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2016-11-28 23:48:02 -08:00
Adam C. Emerson
4731433eac cmake: Suppress warnings on Cython output
They are automatically generated code and we don't really control it.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2016-11-16 19:54:31 -05:00
Haomai Wang
eae7d5a645 msg/async: add dpdk stack plugin
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-09 11:39:20 +08:00
Haomai Wang
cf23acef71 msg/async/rdma: add build ops
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-11-01 15:34:04 +08:00
Alexander Graf
e70ab48b7f AArch64: Detect crc32 extension support from assembler
The used compiler may or may not be recent enough to recognize the
crc32 extended cpu type. However, it does not really have to know about
them either, since all we do is pass inline assembly instructions to
the assembler.

This patch moves the crc cpu extension detection from compiler based
to assembler based, so that we can build optimized code even when the
compiler does not know about the cpu type yet.

Fixes: http://tracker.ceph.com/issues/17516
Signed-off-by: Alexander Graf <agraf@suse.de>
2016-10-10 14:51:44 +02:00
Bassam Tabbara
e7e0b1bc1e erasure-code: Backward compatibility with legacy EC plugins
Resurrected jerasure_generic, jerasure_sse3, jerasure_sse4, jerasure_neon,
shec_generic, shec_sse3, shec_sse4 and shec_neon. These all are exact
copies of the new jerasure and shec plugins that support SIMD detection.

Moved EC preload code in ceph-mon and ceph-osd to a central location, added
warning when preloading legacy plugins.

OSMonitor::get_erasure_code and OSDMonitor:normalize_profile will now check
if legacy EC plugins are used and log a warning.

Added tests to check that warnings make it to the log.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2016-09-29 10:34:34 -07:00
Bassam Tabbara
10d06eb115 cmake: normalize use of SIMD flags
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2016-09-29 10:34:33 -07:00
Bassam Tabbara
cc62c11a76 erasure-code: Remove SIMD flavors for jerasure and shec
By switching to a new gf-complete with SIMD runtime detection, we can now remove all the different flavors of jerasure and shec. This simplifies deployment and configuration of erasure coding, enables hetergenous OSDs, and enables us to take advantage of new performance improvements in jerasure without config/build changes.

This commit removes flavors from cmake, removes ErasureCodePluginSelect___, and fixes unit tests. There is now a single plugin for jerasure and a single plugin for shec.

SIMDExt.cmake was changed so that its a little more generic, and is not polluted with gf-complete specific CFLAG defines. The #define for SIMD instruction were based on gf-complete.

I also added a small init helper for jerasure that has code that was common between jerasure and shec.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2016-09-29 10:34:33 -07:00
Casey Bodley
1a76324981 fio: generalize for other ObjectStores
* renamed target to fio_ceph_objectstore
* moved into src/test/fio subdirectory
* added to cmake build
* added support for DDIR_READ
* added required fio option 'conf' to load a ceph configuration file
* added multiple collections per job to simulate parallelism from pgs
* creates objects of the appropriate size on setup
* added support for multiple jobs that share an ObjectStore instance

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-09-13 11:02:17 -04:00
Dan Mick
3fdccc9e2b Set ARM_CRC_FLAGS from the CRC test rather than ARM_NEON_FLAGS
Fixes: http://tracker.ceph.com/issues/17250
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2016-09-08 20:28:55 -07:00
Kefu Chai
367755e7f7 cmake: pass -DINTEL* to gf-complete cflags
* in addition to the -march=foobar, we also need to pass -DFOOBAR to
  gf-complete sources' cflags, so that the SIMD instructions can be
  generated as expected.
* also extract the compiler support for instruction extensions detections
  into SIMDExt.cmake.
* and only check the compiler support if CMAKE_SYSTEM_PROCESSOR matches
  with the instruction-set/arch to be checked.

Signed-off-by: tone.zhang <tone.zhang@linaro.org>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-09-02 16:23:54 +08:00
Kefu Chai
2ed44f993f cmake: replace Findlttng-ust.cmake with FindLTTngUST.cmake
find_package(lttng-ust REQUIRED) fails to find the lttng library without
this change. because find_path(LTTNG_LIBRARY_DIR ...) does not search
in the default library paths. and the second mode of
FIND_PACKAGE_HANDLE_STANDARD_ARGS() does not stop the cmake with a fatal
message, even some of the required vars are missing. so use the
implemetantion from cmake upstream.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-30 15:51:51 +08:00
Josh Durgin
aaac836380 Merge pull request #10208 from BlaXpirit/cmake-py3
Include Python 3 bindings into the cmake build and make packages for them

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-08-12 16:30:35 -07:00
Oleh Prypin
5683a8259b cmake: Also build Python bindings for Python 3
Signed-off-by: Oleh Prypin <oleh@pryp.in>
2016-08-11 16:36:57 +03:00
Kefu Chai
5d9debd57d cmake: cleanup Findxio.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-11 14:53:03 +08:00
Kefu Chai
96fdb6914a cmake: cleanup Findleveldb.cmake
* and reference leveldb library and include dir with fullpath

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-11 14:53:02 +08:00
Kefu Chai
6c12edbf9d cmake: cleanup Findgperftools.cmake
* remove Findtcmalloc.cmake, use Findgperftools.cmake instead.
* tcmalloc is packaged in gperftools-dev, so we can check them in a single
  place.
* fix the check of heap-profiler.h, we should enable the check as long
  as tcmalloc is enabled or profiler is enabled.
* only check gperftools headers in "include/gperftools", and do not
  "include/google" anymore. as we only support the distros which
  shipping recent gperftools-dev package with "include/gperftools".
  and "google/*.h" are deprecated.
* set ALLOC_LIBS with GPERFTOOLS_TCMALLOC_LIBRARY, so we can link
  against tcmalloc with full path.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-11 14:06:39 +08:00
Kefu Chai
5444721b1d cmake: install platlib into a subdir of build-base dir
distutils' install command installs all stuff under $build-base, which
includes the temporary files. and rpm packaging does not like::
  Installed (but unpackaged) file(s)
and to support both py2 and py3, we should encode the python version
into the path, otherwise the built files will be overwritten.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-11 00:14:17 +08:00
Kefu Chai
300aedd8a6 cmake: fix wrong path introduced by bb163e9
s/CYTHON_MODULE_LIB/CYTHON_MODULE_DIR/

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-10 15:59:30 +08:00
Kefu Chai
bb163e98c0 cmake: specify distutils build path explicitly
so we don't need to replicate the logic in distutils/command/build.py
at different places.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-10 09:58:22 +08:00
Kefu Chai
333877023e doc,pybind: bypass sanity check if building doc
the empty .so file does not compile at all, so we need to bypass the
sanity check, if we are building docs. and what sphinx needs is just the
docstrings.

Fixes: http://tracker.ceph.com/issues/16940
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-08-09 17:08:45 +08:00
Kefu Chai
6c04c94d03 Merge pull request #10098 from dzafman/wip-16568
tools: New "removeall" used to remove head with snapshots

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-08-02 00:26:54 +08:00
Kefu Chai
8bf5cd1606 cmake: cleanup Findfuse.cmake
* remove unused stuff, we do not pass/use FUSE_VERSION in cmake.
  the FUSE_VERSION we are using is defined by "fuse_common.h".
* use FindPackageHandleStandardArgs for handling find_package()
  arguments.
* also detect osxfuse on MacOS.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-29 23:12:33 +08:00
David Zafman
b0be797ae6 cmake: Support extra arguments to tests using add_ceph_test
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-07-27 09:58:59 -07:00
Kefu Chai
07206f3173 cmake: cleanup Findgperftools.cmake
remove unused bits

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 16:17:42 +08:00
Kefu Chai
98c3df5a0d cmake: cleanup Findudev.cmake
remove the crufts for handling the REQUIRED argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:40:30 +08:00
Kefu Chai
342d8279d4 cmake: cleanup Findblkid.cmake
* remove unused bits
* fix the wrong mark_as_advanced() call.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:40:30 +08:00
Kefu Chai
94db627c31 cmake: fix libaio detection
* we used to check the header file of "aio.h" which is part of glibc,
  the glibc async i/o, and the corresponding library is librt. but what
  we are using is libaio.
* reference libaio using AIO_LIBRARIES for consistency.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:40:30 +08:00
Kefu Chai
bfc25cbc93 cmake: cleanup Findbabeltrace.cmake
* we don't need the BABELTRACE_PATH_HINT setting, as packaging script
  requires the the babeltrace pacakges explicitly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:39:02 +08:00
Kefu Chai
03a860ed11 cmake: add missing FindBacktrace.cmake
FindBacktrace.cmake is missing in older versions of cmake. so put a copy
in our tree.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:39:02 +08:00
Kefu Chai
e21d98aed4 cmake: cleanup FindOpenLdap.cmake
* use find_package_handle_standard_args() and remove unused bits.
* rename OPENLDAP_LIBS to OPENLDAP_LIBRARIES, to be consistent with
  find_package() modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:39:01 +08:00
Kefu Chai
3f6fd585f8 cmake: cleanup Findtcmalloc.cmake
use find_package_handle_standard_args() and remove unused bits.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:39:01 +08:00
Kefu Chai
3170f16e86 cmake: do not check libuuid anymore
libuuid dependency is not found in ceph.spec.in or debian/control.
and we are not using libuuid since 62bfc7a.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-25 15:39:01 +08:00
Kefu Chai
af1598a3b3 Merge pull request #10352 from wjwithagen/wip-wjw-freebsd-cmake-2
cmake: work to get inital FreeBSD stuff

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-24 18:18:45 +08:00
Willem Jan Withagen
b4b00f43b1 FindNSS.cmake: find includes in /usr/local/includes/nss/nss
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-20 15:57:42 +02:00
Kefu Chai
e8a16b7fda cmake: remove unused Findfio.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:22:29 +08:00
Kefu Chai
b5607f88e8 cmake: Findfcgi.cmake cleanups
FindPackageHandleStandardArgs() takes care of the find_package()
boilderplate stuff. so no need to repeat them. and remove the checkings
in env variables.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:21:06 +08:00
Kefu Chai
636522b669 cmake: use FindBacktrace instead of Findexecinfo
always better to use upstream cmake modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
c30c5223b5 cmake: only allow up to 1 hour for a ceph test
quote from
https://cmake.org/cmake/help/v3.0/prop_test/TIMEOUT.html?highlight=timeout

> If it exceeds that the test process will be killed and ctest will move
> to the next test.

this helps us to identify test hang.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:47 +08:00
Kefu Chai
ce5724effb cmake: add a "tests" target to build tests
please note "make test" is used by cmake to run tests, so we cannot just
repurpose it to *build* them.

* AddCephTest.cmake: depends on "tests"
* CMakeLists.txt: let "check" depend on "tests"
* src/CMakeLists.txt: update the run-tox tests
* run-make-check.sh: use "make tests" and "ctest" instead of "make check"
* ceph-detect-init/CMakeLists.txt: let "tests" depend on
    "ceph-detect-init"
* ceph-disk/CMakeLists.txt: let "tests" depend on "ceph-disk"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 10:57:12 +08:00
Kefu Chai
c57dbd3451 cmake: pass necessary cflags to build_ext
this fixes:
```
creating
/home/kefu/ceph/build/src/pybind/rados/var/ceph/ceph/build/src/pybind/rados
creating
/home/kefu/ceph/build/src/pybind/rados/var/ceph/ceph/build/src/pybind/rados/pyrex
/usr/bin/gcc-6 -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-fno-strict-aliasing -iquote /var/ceph/ceph/src/include -fPIC
-I/usr/include/python2.7 -I/usr/include/x86_64-linux-
gnu/python2.7 -I/usr/include/python2.7 -c
/home/kefu/ceph/build/src/pybind/rados/pyrex/rados.c -o
/home/kefu/ceph/build/src/pybind/rados/var/ceph/ceph/build/src/pybind/rados/pyrex/ra
dos.o -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g
-fstack-protector-strong -Wformat -Werror=format-security -DNDEBUG -g
-fwrapv -O2 -Wall -Wstrict-prototypes -lpython2.
7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1
-Wl,-Bsymbolic-functions
/home/kefu/ceph/build/src/pybind/rados/pyrex/rados.c:239:28: fatal error:
rados/librados.h: No such file or directory
 #include "rados/librados.h"
                            ^
compilation terminated.
```

seems we can not pass the CFLAGS with space in it to setup.py using env variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-11 16:23:20 +08:00
Kefu Chai
6a99eec653 cmake: put cythonized .c files into ${CMAKE_CURRENT_BINARY_DIR}
it was put into the directory where .pyx is located. it's not expected
in an out-of-tree build.
* {rados,rbd,cephfs}/setup.py: use build_ext from cython if possible, fallback to
  the one from setuptools or distutils
* Distutils.cmake: pass --cython-c-in-temp --build-temp and
  --cython-include-dirs to "build_ext" command

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-11 16:23:20 +08:00
Kefu Chai
edd7c9d709 cmake: creage egg_info files into build directory
otherwise they will go to the source tree, and "git ls-files" will list
them as untracked files, which annoy gitbuilder-ceph-tarball*-cmake
gitbuilders. like

+ echo 'error: Added files:'
error: Added files:
+ cat .git/added-files
src/pybind/rados/rados.egg-info/PKG-INFO
...

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-08 15:02:59 +08:00
Kefu Chai
b5bb77d566 cmake: use the stock FindCURL.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 23:26:02 +08:00
Kefu Chai
3646fa5c6b cmake: use the stock FindEXPAT.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 23:26:02 +08:00
Josh Durgin
1c7ad3523d Merge pull request #9833 from onyb/wip-rados-pypi
pybind: make rados ready for PyPI

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-06 16:01:50 -07:00
Kefu Chai
0c5fb822dd cmake: refactor distutils_install_module to follow the pattern of distutils_install_cython_module
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 11:49:26 +08:00
Kefu Chai
8c9ff9c5cb cmake: install python packages into $CMAKE_INSTALL_PREFIX
* Distutils.cmake:
  set --prefix=${CMAKE_INSTALL_PREFIX} for python packages installed using
  setuptools. it was set to --prefix=/user only when $DESTDIR is set. so
  if user installs ceph using -DCMAKE_INSTALL_PREFIX, these python
  packages still go to /usr, which is unexpected.
* ceph-disk/CMakeLists.txt:
  install script into ${CMAKE_INSTALL_SBINDIR} instead of /usr/sbin

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 11:49:26 +08:00
Anirudha Bose
9a1d64c93d pybind: Add CMake support in rados
Signed-off-by: Anirudha Bose <ani07nov@gmail.com>
2016-06-19 20:20:56 +05:30
Kefu Chai
a21071d1ba Merge pull request #9119 from runsisi/wip-fix-cmake-desc
cmake: update description for cmake module Findkeyutils.cmake

Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-06-18 23:51:41 +08:00
Daniel Gryniewicz
15b11ad5b6 Update git rev parsing in CMake
Git now can have a history for a short branch in the logs, breaking the
rev parsing.  Update to a newer version that fixes this.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2016-06-10 14:22:31 -04:00
Kefu Chai
c4a8784330 cmake: install ceph-disk to /usr/sbin
* add keyword "INSTALL_SCRIPT" to distutils_install_module(), so we can
  override the install path of ceph-disk script.
* refactor the Distutils.cmake module a little bit, the ${option}
  variable out lives the CODE snipplet. so we need to reset it
  at the beginning.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-04 09:37:33 +08:00
Kefu Chai
bbf51e1562 cmake: install cython modules
* fix CYTHON_ADD_MODULE() macro. because python_add_module() offered by
  FindPythonLibs.cmake creates a target with name of ${name}, which conflicts
  with existing targets like "rbd" or "rados". so we can not reuse the
  name in ${name}.pyx. and instead, we should specify the target name
  explicitly.
* add distutils_install_cython_module() function to build and install
  cython modules.
* we can split build and install of cython module, but the install phase
  always tries to build the module. so keep it this way. will look at it
  later on.
* move the variables initializations into the Distutils.cmake module.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:50:05 +08:00
Kefu Chai
b02aaebc45 cmake: install ceph-{detect-init,disk}
add a cmake module named Distutils.cmake for setting up python modules
using setup.py.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-06-01 23:49:46 +08:00
Kefu Chai
a2706c7e16 cmake: revamp on lttng support
* Findlttng-ust.cmake: detect lttng-gen-tp, and make it a required var
* Findlttng-ust.cmake: fix the detection of lttng
* src/tracing/CMakeLists.txt:
  - do not put the generated header files into ${CMAKE_SOURCE_DIR}/src/tracing,
    instead we should put the generated files into ${PROJECT_BINARY_DIR}/include.
  - do not compile the tracing library using the .c files in the repo,
    instead, we should generate them at compile time using lttng-gen-tp,
    and compile the genererated .c files.
* CMakeLists.txt: make the lttng-use package REQUIRED if WITH_LTTNG=ON
* src/CMakeLists.txt: let librados depend on librados-tp if LTTNG is
  enabled: librados includes tracing/librados.h. the same applies to
  librbd, libos and libosd. cmake could deduce the dependencies if we
  add the lttng header files under include/tracing into the
  corresponding target, as the lttng header files are already listed
  as OUTPUT and GENERATED files, but this requires us to put all the
  involved targets in the same CMakeLists.txt. in this case, three
  CMakeLists.txts are impacted. so we create a new target for each
  lttng header, and let the impacted target depend on it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-05-27 19:54:42 +08:00
Ali Maredia
0ef09f32d4 cmake: Added #defines to config-h.in.cmake
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-05-26 17:26:48 -04:00
runsisi
533dbe04a7 cmake: update description for cmake module Findkeyutils.cmake
Signed-off-by: runsisi <runsisi@zte.com.cn>
2016-05-13 22:02:34 +08:00
Kefu Chai
25030b0436 cmake: pass CEPH_BUILD_VIRTUALENV to test as an env variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-05-10 21:50:17 +08:00
Matt Benjamin
4df918ae8c cmake: <UPPERCASED_NAME>_FOUND in Findxio.cmake
For uniformity, upcase all these variables.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-05-03 17:36:59 -04:00
Ali Maredia
65963739cd cmake: ceph_objectstore_tool.py test passing
Replaced relative paths in shell scripts in
test/ceph_objectstore_tool.py and init-ceph.in to
work with CEPH_FOO environment variables set in cmake.
Also added CEPH_BUILD_DIR environment variable set
to CMAKE_BINARY_DIR. It is used in init-ceph and
ceph_objectstore_tool.py.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-04-14 20:48:21 -04:00
Ali Maredia
a7bd5e4bdc cmake: Reorganized test dir, added cmake functions
make check targets now use add_ceph_test, or
add_ceph_unittest to be added to CTest. Renamed and
reordered targets in tests/CMakeLists.txt

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-04-14 20:48:19 -04:00
Matt Benjamin
1539d9029c rgw-ldap: conditional build
Permit building without LDAP support--support is enabled by default.
Tested with CMake and autotools standalone builds.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-03-16 16:31:20 -04:00
Matt Benjamin
bcddb5672b cmake: avoid false-positive LDAP header detect
Pre-setting a value in OPENLDAP_INCLUDE_DIR broke the subsequent
test.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-03-14 14:11:45 -04:00
Matt Benjamin
38fd3f190a rgw: LDAP pass-through authentication
Implement a new external authenticator based on LDAP and the
new external token format.

External LDAP auth now works, at least with openldap/X.500 style
naming and ldaps:// (SSL).

The latter is AD-friendly, but since AD uses dnattr=cn (IIRC)
everywhere, AD will need testing.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-03-12 09:57:12 -08:00
Kefu Chai
8c2bb7e5c6 cmake: add Find{dpdk,pciaccess}.cmake
to ready the SPDK support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-02-04 14:03:21 +08:00
John Coyle
6d860ba46c cmake: musl libc does not implement backtrace. Added feature check and fallback to libexecinfo.
Signed-off-by: John Coyle <dx9err@gmail.com>
2016-01-29 08:50:46 -05:00
Kefu Chai
fd8ab0bd07 cmake: remove the copyright of FindSnappy.cmake
as we've basically removed all the HyperTable code.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-01-28 20:50:51 +08:00
Kefu Chai
5efc5dce73 cmake: check for libsnappy in default path also
* some distros install libsnappy into /usr/lib/x86_64-linux-gnu
  which is completely resonable. but this directory is not listed
  in the Findsnappy.cmake.
* also add SNAPPY_ROOT_DIR, in case user install snappy in some
  other directory.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-01-26 23:53:31 +08:00
Kefu Chai
f5da159457 makefile: remove dependency on libedit
turns out we are not using it anymore

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-01-20 00:35:50 +08:00
Kefu Chai
3626d86de6 cmake: add check for liblz4
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-01-07 03:59:36 -08:00
Ali Maredia
cc30175480 cmake: udev linking and Findudev module changes
Linked udev into librbd, restored UDEV_I to
UDEV_INCLUDE_DIR in Findudev.cmake

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-01-05 15:42:06 -05:00
Ali Maredia
7151a47240 cmake: Findxio.cmake update Xio OFF by default
Option WITH_XIO now OFF but required by default,
Findxio.cmake now using
FIND_PACKAGE_HANDLE_STANDARD_ARGS.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-01-05 15:42:06 -05:00
Ali Maredia
8ef0dfb30e cmake: Check for required Cython package
Added a FindCython.cmake, Cython is required.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-01-05 15:42:06 -05:00
Casey Bodley
048f3bd5a3 cmake: add detection for HAVE_CURL_MULTI_WAIT
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-01-05 15:42:06 -05:00
Ali Maredia
330777051e cmake: Babeltrace detection supported
Babeltrace defaults to ON, Findbabeltrace.cmake &
BABELTRACE variables in config-h.in.cmake added.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-01-05 15:42:06 -05:00
Ali Maredia
1aa7bc2ec4 cmake: LTTng functionality added
Set LTTNG option to ON by default, added
Findlttng.cmake, added CMakeLists.txt in
src/tracing that generates .h files using
lttng-gen-tp, added WITH_LTTNG to
config-h.in.cmake.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-01-05 15:42:06 -05:00
Ali Maredia
a614c3b9c1 cmake: Updated allocator checking
Added a FindJeMalloc.cmake file, changed
allocator checking logic (tcmalloc is default),
replaced TCMALLOC_LIBS to ALLOC_LIBS, added
HAVE_LIB(ALLOCATOR) to config-h.in.cmake.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-01-05 15:42:06 -05:00
Samuel Just
40522828c1 cmake: add nss as a suffix for pk11pub.h
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-11-12 09:12:17 -08:00
Daniel Gryniewicz
b1401e5124 CMake - fix check for UDEV
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:55 -04:00
Daniel Gryniewicz
a866a8947d CMake - add check for libedit
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
4d02f8746c CMake - fix check for Snappy
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
3570dd5625 CMake - add check for XFS
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
571215ef20 CMake - fix check for AIO
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
6adf1cbb33 CMake - fix check for blkid
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
1a4262c31a CMake - add check for libcurl
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
a6c8b46101 CMake - fix check for fcgi
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
88f7df7d1a CMake - add check for libuuid
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
a18b649c1e CMake - fix check for leveldb
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
cb7fe45823 CMake - fix check for expat
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
a7ce8f5d68 CMake - Add check for keyutils
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
dcf647ec04 CMake - fix check for NSS
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Daniel Gryniewicz
b02e0f9bfe CMake - fix libatomic_ops and gperftools checks
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2015-09-23 10:55:54 -04:00
Sage Weil
030b3edeac Merge pull request #5744 from ceph/wip-12909
cmake: update FUSE_INCLUDE_DIRS to match autoconf
2015-09-03 15:09:49 -04:00
Ali Maredia
283e81c215 cmake: Removed traces of CDS, minor cmake fixes
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2015-09-03 11:37:31 -04:00
Casey Bodley
7c00bf05db cmake: update FUSE_INCLUDE_DIRS to match autoconf
client/fuse_ll.cc is now including <fuse.h> and <fuse_lowlevel.h>
instead of <fuse/fuse.h> and <fuse/fuse_lowlevel.h>, so we need to add
the fuse directory to the FUSE_INCLUDE_DIRS variable

using find_path() with just fuse.h was finding a /usr/include/fuse.h
instead of the one in /usr/include/fuse/. looking for fuse_common.h and
fuse_lowlevel.h first causes it to generate the correct
FUSE_INCLUDE_DIRS=/usr/include/fuse

Fixes: #12909

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2015-09-01 14:11:03 -04:00
Danny Al-Gaaf
ff4993310f CMake: add FindNSPR.cmake
The nss lib needs the nspr library to build. Add file to
get the nspr lib/includes from the pc file.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2015-05-07 13:19:50 +02:00
Ali Maredia
0f6b9f2816 Combined CMake Build for Hammer
CMake Ceph Build System (Firefly)
CMake.  Add tests.
Respace src/CMakeLists.txt.
CMake.  Spacing cleanups.
CMake for Firefly is Triumphant
CMake for Giant
Adapt to Giant.
Fix installation for scripts and man pages
Fix CEPH_LIBDIR and CEPH_PKGLIBDIR defines
Add erasure-code libraries
	uses try_compile() to detect support for -msse flags
Fix rados object classes
Propagate Casey's cls library change to src/test.
Fix CMake build for Hammer.
Try-add rados and common to librbd link.
Fix name and linkage of libec_lrc.
Rename arch/neon.c arm.c
Fix libcommon.a dependencies (some unit tests).

Authors:
	Ali Maredia <ali@cohortfs.com>
	Casey Bodley <casey@cohortfs.com>
	Adam Emerson <aemerson@cohortfs.com>
	Marcus Watts <mdw@cohortfs.com>
	Matt Benjamin <matt@cohortfs.com>

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
2015-01-14 16:40:05 -05:00