Commit Graph

155 Commits

Author SHA1 Message Date
Kefu Chai
6559a85051 cmake: disable FAIL_ON_WARNINGS for rocksdb
otherwise -Werror=implicit-fallthrough will fail the build with GCC-7

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-11 14:09:45 +08:00
Kefu Chai
99c0306324
Merge pull request #19208 from tchaikov/wip-spdk-17.10
spdk: update SPDK to v17.10

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2017-11-30 00:39:18 +08:00
Kefu Chai
b3ceafe7d8 cmake,rpm,deb: update to accommodate SPDK v17.10
* cmake/modules/BuildSPDK.cmake: add lvol
* cmake/modules/BuildDPDK.cmake: add pci and bus_pci
* ceph.spec.in, cmake/modules/BuildSPDK.cmake, debian/control:
  re-introduce libuuid dependency, as 17.07 added lvol. and the latter
  depends on uuid.
* cmake/modules/BuildSPDK.cmake: avoid introducing local variable of
  `iface_libs`.
* cmake/modules/patch-dpdk-conf.sh: disable
  CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES, this option introduces the
  balanced allocation of memory. but it also requires libnuma-dev.
  let's disable it for now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-29 18:03:23 +08:00
Kefu Chai
12dc5733c9
Merge pull request #18938 from tchaikov/wip-cmake
cmake: check gcc version not release date for libstdc++ saneness

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-29 15:38:16 +08:00
Kefu Chai
e3b703eb12 cmake: check bootstrap.sh instead before downloading boost
config.hpp does not exist if boost is not configured. so a boost source
directory unpacked from dist tarball does not have this file. the
bootstrap.sh instead, it is used to create the necessary config files
for jam.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-24 11:13:16 +08:00
Kefu Chai
9b3f9cd253 cmake: BuildDPDK: use ${CMAKE_MAKE_PROGRAM} when appropriate
DPDK requires GNU Make. see
spdk/dpdk/doc/guides/freebsd_gsg/build_dpdk.rst. so, we should use the
configured make program to do the job.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-21 17:47:17 +08:00
Kefu Chai
7c3221fda1 cmake: BuildDPDK: use POSIX compliant syntax
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-21 17:06:50 +08:00
Kefu Chai
a11ab928f7 cmake: check libaio on linux before building SPDK
SPDK requires libaio on linux.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 11:18:02 +08:00
Kefu Chai
e8add075ec cmake: add rte_mempool_{ring,stack} to Finddpdk.cmake
* also refactor it a little bit

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:44:56 +08:00
Kefu Chai
9964ac9251 cmake: BuildDPDK: do not build kernel modules
see also http://pkgs.fedoraproject.org/cgit/rpms/dpdk.git/tree/dpdk.spec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:44:56 +08:00
Kefu Chai
01a9f17825 cmake: build spdk/dpdk in cmake modules
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: wanjun.lp <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
2017-11-17 15:37:53 +08:00
Kefu Chai
571a786afc cmake: pass LDSHARED env var to distutils
otherwise, the default gcc will be used, and the $CMAKE_C_COMPILER
passed to the outer CMakeLists.txt won't kick in. moreover, if the
building script (ceph.spec for instance) could set the $PATH, and
expect that the CMakeLists.txt will use the toolchain executables
in the $PATH to build Ceph, distutils will continue using the default
$CC for linking the python bindings, on UNIX it will be gcc in the
new shell's $PATH, because we are using `install(CODE "... execute_process(
...))` for installing the python bindings. apparently, this is not
expected. because the new shell's $PATH is very likely different
from the one changed by the building script. to address this, we
should always specify the `$LDSHARED` env var explicitly.

also, pass env vars using `ENV{}` instead of the `env` command to
workaround the issue of https://cmake.org/pipermail/cmake/2015-December/062216.html,
because it's not straightforward to set environment variables with
spaces in the them using cmake. and because one cannot use add_custom_target()
in the script mode of cmake. this leave me only limited options to
fix this issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-15 15:22:07 +08:00
Kefu Chai
70b5b4bfbf cmake: should check the availability of aligned_alloc() by running the test
this mimics the failure of rocksdb without a working
aligned_alloc()/free().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-31 17:39:09 +08:00
Kefu Chai
599a9ef3ca cmake: should use the value of GPERFTOOLS_LIBRARIES as REQUIRED_VARS
otherwise it's always true, even if the libaries are not found. also
the cmake output will be wrong without this change:

without this change, it is:

-- Found gperftools: GPERFTOOLS_TCMALLOC_LIBRARY;GPERFTOOLS_TCMALLOC_MINIMAL_LIBRARY;GPERFTOOLS_PROFILER_LIBRARY (found version "2.5.93")

with this change, it would be:

-- Found gperftools: /tmp/gperf/lib/libtcmalloc.so (found version "2.5.93")

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-31 16:58:04 +08:00
Kefu Chai
16f0a67d83 cmake: check for aligned_alloc()
in case developer is using patched tcmalloc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-27 11:13:15 +08:00
Jianpeng Ma
b4cc9a9aa8 cmake: Build libpmem from source code.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-20 00:44:09 +08:00
Kefu Chai
caae3c4757 cmake: fix gperftools version detection
there is chance that its "patch" version is empty. in that case, do not
include it in its version string. otherwise, we will have something
like:

Found gperftools:
GPERFTOOLS_TCMALLOC_LIBRARY;GPERFTOOLS_TCMALLOC_MINIMAL_LIBRARY;GPERFTOOLS_PROFILER_LIBRARY
(found version "2.4.#define TC_VERSION_PATCH  """)

when running cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-13 15:11:09 +08:00
Sage Weil
96c831167b Merge pull request #18092 from tchaikov/wip-cmake-cleanup
cmake: do not link against unused or duplicated libraries

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-05 11:18:27 -05:00
Brad Hubbard
598556f98b make-dist,cmake: Try multiple URLs to download boost before failing
Remove SPOF during boost download for make-dist and later cmake > 3.7

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-04 11:13:34 +10:00
Kefu Chai
2f4cc32741 cmake: always use CMAKE_RUNTIME_OUTPUT_DIRECTORY as the unittests' path
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-03 23:44:28 +08:00
Sage Weil
19987549e0 make-dist,cmake: move boost tarball location to download.ceph.com
Sourceforge is down.  Also, we can hammer our servers instead of
theirs.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-26 17:22:36 -04:00
Kefu Chai
04697cdbdc cmake: update the error message for gperftools bug
it's a follow-up of #17788

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-22 10:40:29 +08:00
Kefu Chai
ff8d6a730b cmake: error out if rocksdb is incompatible w/ tcmalloc
the commit d406f228 in gperf implements a c11 feature used by a
recent change in rocksdb: 16e03882, which uses aligned_alloc().
and 16e03882 in rocksdb was merged after v5.7 was tagged, while
16e03882 in gperf was merged after v2.6.1 was tagged.

because aligned_alloc() is not implemented by tcmalloc until the
not-yet-released 2.6.2, if we call aligned_alloc() in an application
linked against tcmalloc, what gets called will be the glibc's
aligned_alloc(). but if we free() the memory chunk allocated by
aligned_alloc(), the tcmalloc's implementation kicks in, then
InvalidFree() is called, because the memory chunk being freed was
allocated by tcmalloc. in short, "mixing allocators", quote from
Dan Mick.

in rocksdb, aligned_alloc() is used if _ISOC11_SOURCE is defined, this
makes sense, because aligned_alloc() is a C11 function. we could avoid
using it by not defining _ISOC11_SOURCE. but as long as _GNU_SOURCE is
defined, glibc defines _ISOC11_SOURCE. and libstdc++ requires
_GNU_SOURCE, because it uses a fair amount of GNU extensions.

Fixes: http://tracker.ceph.com/issues/21422
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-20 10:23:16 +08:00
Kefu Chai
987c1b3ec2 cmake: detect rocksdb's version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
58179f098b cmake: detect gperf-tools' version
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
aa603621bd cmake: extract BuildRocksDB into BuildRocksDB.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
da8ce07a21 cmake: enable find_package(perftools) to check non-default dir
so it checks $ENV{GPERF_ROOT} first.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-09-19 13:46:50 +08:00
Kefu Chai
b567261208 cmake: require lz4 v1.7 for building lz4 compressor
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-08-17 16:01:55 +08:00
Boris Ranto
70473011a0 Merge pull request #16753 from ceph/wip-fix-ppc64
Fix ppc64 support for ceph

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-14 15:41:28 +02:00
Dan Mick
de06f8e504 cmake/Distutils.cmake: make pkg files optional, add bin
Skip nonexistent 'expected' Python packaging files, and add bin/ as
a possible file for symlinking during build (for Python projects with
slightly different forms, currently ceph-volume)

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-04 10:25:57 -04:00
Boris Ranto
2172dfb594 Fix ppc64 support for ceph
The ppc64 support requires a couple of changes:
- adding the ppc64 support to cmake
- changing optimized crc32 code to compile on ppc64le only
- moving ifdef condition before crc32_align to avoid defined but not
used warning

Signed-off-by: Boris Ranto <branto@redhat.com>
2017-08-04 10:56:03 +02:00
Ilsoo Byun
8134620ad4 os/bluestore: fix build errors when spdk is on
- remove pthread library link from dpdk cmake
- fix the bugs of NVMEDevice

Signed-off-by: Ilsoo Byun <ilsoo.byun@sk.com>
2017-07-05 01:42:10 -04:00
Kefu Chai
f8d5f74663 cmake: compile libzfs backend conditionally
* do not REQUIRE libzfs if it is enabled, this follows the way how we
  handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
  little bit.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-27 23:28:03 +08:00
Kefu Chai
96b2016489 cmake: do not add dependencies to INTERFACE library on cmake < 3.3
otherwise we will have

add_dependencies Cannot add target-level dependencies to INTERFACE
library

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-21 23:34:51 +08:00
Kefu Chai
83432b4461 cmake: build boost as an external project
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-14 23:53:50 +08:00
Sage Weil
c2e346cf12 Merge pull request #15562 from tchaikov/wip-no-assert.h
common,osdc: remove atomic_t completely

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-08 20:38:23 -05:00
Sage Weil
b0e9f3ed51 Merge pull request #15102 from majianpeng/blockdevice-pmem
os/bluestore/BlockDevice: support pmem device as bluestore backend

Reviewed-by: Sage Weil <sage@redhat.com>
2017-06-08 13:15:52 -05:00
Kefu Chai
a87fa37e46 cmake,debian,rpm: remove atomic_t completely
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-08 12:21:00 +08:00
Jianpeng Ma
e25accff7d os/bluestore/BlockDevice:support pmem device as bluestore backend.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-06-07 23:22:52 +08:00
James Page
5d92c79b8e build: misc fixes for cmake build on i386
Scope SIMD CPU flag detection base on target architecture,
resolving build failures in gf-complete on i386; this is
inline with the previous autotools based feature detection.

Only compile crypto/isa-l if yasm 64 bit complier is found,
effectively limiting scope to x86_64.

Signed-off-by: James Page <james.page@ubuntu.com>
2017-06-06 16:34:50 +01:00
Nathan Cutler
4acd2cef35 cmake: align cmake names of library packages
The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"

Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.

Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-05-04 11:14:43 +02:00
Kefu Chai
4847a88556 cmake: do not try to add submodule to exclude list if .git is not around
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-13 13:49:54 +08:00
Kefu Chai
e1991238e7 cmake: support optional argument for overriding default ctag excludes
so instead of excluding all submodules, one can just exclude boost and
rocksdb using:

  cmake -DCTAG_EXCLUDES="boost;rocksdb" ..

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-07 12:06:52 +08:00
Dan Mick
3681683df7 cmake: Add simple recursive ctags target for Ceph source only
Excludes submodules and random JavaScript.

Signed-off-by: Dan Mick <dan.mick@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-06 13:27:12 +08:00
Kefu Chai
195425c9ca cmake: extract ppc64le detection into SIMDExt.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-03 01:44:17 +08:00
Dan Mick
ac4815f74b cmake/modules/SIMDExt.cmake: armv8 crypto intrinsics
Test not only for -march support, but also the actual
presence of the intrinsic routines.  Not sure why, but gcc
4.8.5 passes the first but not the second.

Fixes: http://tracker.ceph.com/issues/19386
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-03-27 09:35:22 -07:00
Dan Mick
ef8980bad2 cmake/modules/SIMDExt.cmake: add whitespace
readability, man, readability

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-03-24 20:10:15 -07:00
Kefu Chai
0603794d53 Merge pull request #12977 from weixiaowilliam/crc32c_opt
crc32c: optimize aarch64 crc32c implementation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-02-18 17:13:50 +08:00
Adir Lev
74fc3f6b83 msg/async/rdma: check if exp verbs avail
issue: 975125

Change-Id: I9daa168ca0299887a7238a688508c773b98abde9
Signed-off-by: Adir Lev <adirl@mellanox.com>
Signed-off-by: Oren Duer <oren@mellanox.com>
2017-02-13 13:28:21 +00:00
wei xiao
b0ae5ae2ad crc32c: optimize aarch64 crc32c implementation
ARMv8 defines PMULL crypto instruction.
This patch optimizes crc32c calculate with the instruction when
available rather than original linear crc instructions.

ceph crc32c performance unit test shows that the optimization get
~ x3.90 speedups on ThunderX ARM Core@2.0GHz (Cavium)
~ x1.45 speedups on ARM Cortex-A57@2.1GHz (Huaiwei)
~ x1.16 speedups on ARM Cortex-A57@2.0GHz (Softiron)

Jira: ENTLLT-358
Change-Id: I657422cd20c9ca78237cd060210a5383f4122575
Signed-off-by: wei xiao <wei.xiao@linaro.org>
2017-02-04 06:20:50 +00:00