Commit Graph

495 Commits

Author SHA1 Message Date
Xiubo Li
7a3cc61de0 libfuse: check the libfuse version from the pkconfig/fuse{3}.pc file
Since libfuse 3.2 to 3.8 the minor version for FUSE library has
stopped updating together with the releases. So we cannot check
version by using the FUSE_VERSION macro in the fuse_common.h header
file directly.

This will check the major/minor version from the fuse{3}.pc pkconfig
file.

Fixes: https://tracker.ceph.com/issues/45396
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-05-24 21:39:47 -04:00
Abutalib Aghayev
ab865ca002 os/bluestore: Enable writing data to HM-SMR hard drives.
Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
2020-05-22 12:37:24 -04:00
Kefu Chai
47ce69aa57 cmake: remove cppcheck and iwyu targets
these targets are not used by any other targets, and are not integrated with CI.
so let's drop them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-13 20:34:27 +08:00
Kefu Chai
b557467df0
Merge pull request #34535 from changchengx/refine_build_liburing
cmake: refactor liburing support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-18 11:39:32 +08:00
Changcheng Liu
36918d7f78 cmake: use cmake module to integrate liburing
1. WITH_LIBURING is used to set HAVE_LIBURING to decide
   use liburing in KernelDevice or not.
2. WITH_SYSTEM_LIBURING is to choose use system installed
   liburing or build the liburing from source code.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-04-17 23:22:56 +08:00
Kefu Chai
c9b38a5210 cmake: detect fuse3 as well
Fixes: https://tracker.ceph.com/issues/44891
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-17 14:31:17 +08:00
Kefu Chai
220fa91d80 cmake: check for FUSE version
and refactor FindFUSE.cmake a little bit to conditionalize OSX support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-17 14:10:55 +08:00
Kefu Chai
b823952d24 cmake: link against FUSE::FUSE
instead of specifying the include directory and linkage separately

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-17 14:10:55 +08:00
Kefu Chai
2b3adb5c5e cmake: rename Findfuse.cmake to FindFUSE.cmake
so its name is consistent with upstream name, and with the variables
exposed by FindFUSE.cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-17 14:10:55 +08:00
Kefu Chai
a05d7179e2 cmake: check $top_srcdir/.git directly
in 0437adc33a, we stop right before
reaching $top_srcdir, but we should stop at its parent directory.

in this change, instead of trying to be smart and to walk all the way
up to the root directory or $top_srcdir, we just check $top_srcdir/.git
directly, as we just know it's there or it does not exist at all.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-08 18:03:37 +08:00
Kefu Chai
ce84ee7856
Merge pull request #34369 from smithfarm/wip-43895
cmake: really stop at top of source code tree

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-08 09:57:04 +08:00
Kefu Chai
b439835cd6 cmake: specify CC when compiling pmem
make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
6fca5e7d37 cmake: do not use $(MAKE) in Buildpmem.cmake
we cannot assume that we are using `make` as the cmake generatator,
for instance, if ninja is used, `$(MAKE)` won't be substituted by ninja.
so we need to check if Make is used as generator, if that's the case, we
can just use `$(MAKE)` so we can benefit from the job control of `make`,
otherwise, `make` is used, because currently, PMDK uses Makefile to
build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
8f7fae7a78 cmake: add find_make() function
it's a shorthand for finding "make" or "gmake" (for FreeBSD), and set
the path to the executable and the command to use in the generated
"Makefile" or whatever build script generated by cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Nathan Cutler
0437adc33a cmake: really stop at top of source code tree
Previously, we were recursing right up to /

Patch submitted by Tony Davies via https://tracker.ceph.com

Fixes: https://tracker.ceph.com/issues/43895
Signed-off-by: Tony Davies <anthony.t.davies@gmail.com>
2020-04-03 06:19:09 +02:00
Adam C. Emerson
647819c632 cmake: Don't enable BOOST_USE_VALGRIND when not requested
We were adding the define without support in the library if
WITH_BOOST_VALGRIND was turned off.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-20 20:40:39 -04:00
Adam C. Emerson
7f573bb334 build: Add WITH_BOOST_VALGRIND option
To build Boost.Context (and other libraries) with support to allow
them to be valground usefully, and to include the define to link
against them.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-20 00:51:49 -04:00
Kefu Chai
aff9557a70
Merge pull request #33256 from tchaikov/wip-cmake-cares
cmake: should expose ${C-ARES_BINARY_DIR} from c-ares

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-02-21 22:04:23 +08:00
Lucian Petrut
dbdc91db6c cmake: [win32] Fix rocksdb cross compiling
In order to be able to cross compile rocksdb using mingw, we'll
have to do the following:
* pass on specified toolchain files
* pass the CMAKE_PREFIX_PATH. We'll have to replace the ";" separator
  with something else (we went with "!"), otherwise CMake won't be
  able to properly handle the command arguments
* pass on the received LZ4 lib location since it cannot be properly
  detected when cross compiling.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-02-19 08:24:21 +00:00
Lucian Petrut
d15481f1a5 cmake: [win32] Update cmake files
We'll update the cmake files in order to be able to build ceph
components for Windows targets.

Cross compiling using MINGW is the easiest approach for now.
Subsequently, we'll add support for Clang and MSVC.

This patch provides the following changes:
* include winsock2.h, which provides ntohl on Windows
* avoid unsupported compiler flags when using msvc
* add a custom toolchain file for mingw
* update install command for ceph-common in order to work with mingw
* avoid running test sample when cross compiling
* link against the ws_32 lib
* set argeted Windows version
* skip yasm checks when targeting Windows
* allow multiple redefinitions when using mingw, picking the first one.
  this is a workaround for a mingw TLS bug:
  https://sourceforge.net/p/mingw-w64/bugs/816/

Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-02-19 08:24:21 +00:00
Kefu Chai
12e738f5a1
Merge pull request #32693 from CongMinYin/update-pmdk-version
cmake: Update pmdk version to 1.7

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-13 19:24:05 +08:00
Yin Congmin
6a102677e7 cmake: Update pmdk version to 1.7
nvml is renamed to pmdk and upgrading to 1.7 to fix gcc9 build issue

Signed-off-by: Yin, Congmin <congmin.yin@intel.com>
2020-02-14 01:50:07 +08:00
Kefu Chai
01129acbf0 cmake: should expose ${C-ARES_BINARY_DIR} from c-ares
as it's public header `ares.h` includes `ares_build.h`, which is
generated in the build directory.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-13 17:51:45 +08:00
Ulrich Weigand
c58fc258a2 cmake: Improve test for 16-byte atomic support on IBM Z
Commit d1b9d14324 added a check for
16-byte atomics without library support on IBM Z.  Unfortunately
it turns out this test only works correctly when the test case
is built without optimization.  (This is normally true, but it
may not be the case when passing explicit CXXFLAGS to cmake.)

The underlying reason is that GCC may choose to use either an
inline implementation of the 16-byte atomics or library calls,
depending on whether or not it is able to prove the atomic
variable is properly aligned.  At -O0 it is never able to prove
that, but at higher optimization levels it depends on the complexity
of the expression (in particular, whether GCC can track down the
definition of the underlying object).

As the test case uses a very simple expression, it may happen that
this test can be built without requiring library support, but some
of the "real" uses of atomics in Ceph code cannot.   This defeats
the whole purpose of the test at configure time.

Fixed by making the access pattern in the test more complex, so
that the test fails even at high optimization levels.

Fixes: https://tracker.ceph.com/issues/43747

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2020-01-23 13:13:58 +01:00
Daniel Gryniewicz
0f4cb207bb CMake - Workaround set Boost_component_FOUND
BuildBoost.cmake (used when we're building the submodule) doesn't
provide parity with FindBoost.cmake (used with system Boost).
Specifically, it doesn't set the _FOUND variables for the various
components, making it hard to depend on finding those features.

Set Boost_<component>_FOUND for all the components we're building in
BuildBoost.cmake to make using these variables possible.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2020-01-08 11:31:04 -05:00
Kefu Chai
e461bda6c9 cmake: build boost v1.72 instead of v1.67
so we can drop two patches which are already included by v1.72

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Kefu Chai
be192fba4a cmake: update FindBoost.cmake for 1.72
adapted from
https://github.com/Kitware/CMake/blob/master/Modules/FindBoost.cmake
commit e51232b739b3eeae642be8a7e89079f7c2e8c1b2

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-22 23:56:52 +08:00
Kotresh HR
886b18a9a2 cmake/Boost: Fix python3 version
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2019-12-19 14:40:11 +05:30
Kefu Chai
46358e04d2 cmake: drop Find{Python,Python2}.cmake
since we are python3 only, there is no need to check Python2 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:56 +08:00
Kefu Chai
79584862f6 src,qa: install python bindings into lib/cython_modules/lib.3
instead of checking for the WITH_PYTHON* options, just hardwire to lib.3

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:54 +08:00
Kefu Chai
5fc657b40d cmake: drop WITH_PYTHON2 option
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
  python version for python binding and ceph-mgr embedded python
  interpreter anymore. as before switching to python3-only build,
  we can build python3 and python2 python bindings, and ceph-mgr
  can use either of them. but after switching to python3-only
  build, ceph-mgr has to use whatever python version used to
  build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
  and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
  parameter anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 10:58:58 +08:00
Kefu Chai
47d0e1e16d cmake: CMAKE_ROLE defaults to "PROJECT"
so we can use exported targets like "Python3::Python", which is defined
only if CMAKE_ROLE is "PROJECT". but this global property was introduced
by CMake 3.14, but the minimum required version of CMake is 3.10.2 in
Ceph project at this moment. let's set it if it's not defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 10:58:58 +08:00
Kefu Chai
1f8e3bd5e7 cmake: check version of librdkafka
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-15 22:16:08 +08:00
Kefu Chai
1c0bc5dad4 cmake: silence messages when cppcheck/IWYU is not found
* remove messages spit when these tools are not found
  they are not necessary for building Ceph. and the messages in the cmake
  output are distracting.
* fix a typo

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-10 11:08:02 +08:00
Kefu Chai
f5b2ec5d8f cmake: do not use CMP0093 unless it is supported
CMake CMP0093 was introduced by CMake 3.15, but we only require CMake
v3.10.2. so use the old `Boost_VERSION_MACRO` variable by default,
unless specified otherwise.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-03 20:00:59 +08:00
Kefu Chai
dd0cdb3eb6 cmake: do not use CMP0074 unless it is supported
CMake CMP0074 was introduced by CMake 3.12, but we only require CMake
v3.10.2. so default to check `Boost_ROOT` unless specified otherwise.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-03 09:51:17 +08:00
Kefu Chai
c93b0881e6
Merge pull request #31317 from wjwithagen/wjw-update-FindBoost.cmake
cmake: update FindBoost.cmake for 1.71

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-02 16:46:00 +08:00
Kefu Chai
a56efc1610
Merge pull request #31841 from rosinL/wip-support-dpdk-numa
cmake: Add dpdk numa support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-02 16:45:11 +08:00
luo rixin
32f967b420 cmake: Add shared library as dpdk dependency
librte_eal.a refers to libpthread.so and libdl.so, and libceph-common.so
refers to librte_eal.a. Many test application link libceph-common.so
without linking pthread, it will cause link error.
Such as unittest_interval_set, target_link_libraries (unittest_interval_set
ceph-common)

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2019-11-28 20:18:09 +08:00
Chunsong Feng
4f3606f87b cmake: Add dpdk numa support
Find and link the numa library for DPDK.

Fixes: https://tracker.ceph.com/issues/42275

Signed-off-by: Hu Ye <yehu5@huawei.com>
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2019-11-28 20:15:39 +08:00
Sage Weil
cf676c73ab Merge PR #31781 into master
* refs/pull/31781/head:
	cmake/modules: look for GNU make first instead of BSD make
	cmake: support parallel build for rocksdb

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-25 21:49:31 -06:00
Sage Weil
0e0a3853c9 Merge PR #31579 into master
* refs/pull/31579/head:
	cmake: add cppcheck and iwyu static analysis targets

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-11-25 21:49:18 -06:00
Deepika Upadhyay
abf3dc0cf9 cmake/modules: look for GNU make first instead of BSD make
* BuildDPDK.cmake, BuildSPDK.cmake:
      * give priority to build with gmake
      * throw error if make not found

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2019-11-25 13:16:04 +05:30
Deepika Upadhyay
a930706aa6 cmake: support parallel build for rocksdb
* cmake/modules/BuildRocksDB.cmake: if "make" detected
use concurrent processing, by passing the jobserver to
sub-make using `$(MAKE)`

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2019-11-25 13:14:22 +05:30
Casey Bodley
8aa638b0fa
Merge pull request #30960 from yuvalif/wip-yuval-add-kafka-notif-endpoint
rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-11-21 14:42:23 -05:00
Willem Jan Withagen
9c0a948900 cmake: update FindBoost.cmake for 1.71
This comes from cmake v 3.15.3

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-11-20 17:13:13 +01:00
Yuval Lifshitz
c876a0d817 cmake: add cppcheck and iwyu static analysis targets
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-18 19:43:14 +02:00
Kefu Chai
d6998a3b95 cmake: detect librt for POSIX time functions
per clock_gettime(3),

> On POSIX systems on which these functions are available, the symbol
> _POSIX_TIMERS is defined in <unistd.h> to a value greater than 0.

and

> Most systems require the program be linked with the librt library to
> use these functions.

so, we should detect this symbol and link against librt for using these
functions.

in this change, librt is linked for checking the existence of
clock_gettime(), if it exists. RT_LIBRARY is defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-14 21:26:57 +08:00
Kefu Chai
4c42b4d49d
Merge pull request #31544 from ifed01/wip-ifed-gettid2
test/fio: bump to fio-3.15

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-13 18:58:36 +08:00
Yingxin Cheng
bf958abc0f crimson: build seastar dpdk from src/seastar/dpdk
src/spdk/dpdk and src/seastar/dpdk are both at their private branches
with project-specific modifications, so select proper dpdk source
directory according to flags WITH_SPDK and Seastar_DPDK.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-11-13 15:53:43 +08:00
Yingxin Cheng
ab84158fc0 crimson: link required dpdk components for seastar
see https://github.com/scylladb/seastar/blob/
809b3ef875e7edb7c56643812dc8347ff9a82cee/cmake/Finddpdk.cmake#L60-L85

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-11-13 15:53:31 +08:00
Igor Fedotov
fcfb0c9442 cmake: detect gettid() presense
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-11-12 12:42:14 +03:00
Igor Fedotov
710e5da85b test/fio: bump to fio 3.15
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-11-12 02:24:45 +03:00
Yuval Lifshitz
354d775e04 rgw/pubsub: add kafka notification endpoint
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-05 19:25:03 +02:00
Kefu Chai
a123c85256 cmake: enable Finddpdk to find DPDK to be built
find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 17:54:05 +08:00
Kefu Chai
aa25f6202f cmake: define dpdk_LIBRARIES properly
dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Kefu Chai
d82ea5d738 cmake: refactor BuildDPDK
* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
  as the name of this target should be hidden from the caller of this
  function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Sage Weil
da0473e251 Merge PR #30772 into master
* refs/pull/30772/head:
	cmake/modules/GetGitRevisionDescription: update to work with git-worktree

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-10-22 15:57:02 -05:00
Kefu Chai
a826efc52f cmake: detect linker support
check the linker support of `--version-script` and `--exclude-libs`
flags. and use them only if they are supported.

llvm-ld does not support `--version-script` at this moment. and it's
reported that old versions of llvm-ld do not support `--exclude-libs`
as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-08 21:53:03 +08:00
Sage Weil
165f09d7e7 cmake/modules/GetGitRevisionDescription: update to work with git-worktree
Found this here: https://phabricator.kde.org/D9172

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-07 14:26:07 -05:00
Patrick Donnelly
bbe3a33775
Merge PR #30552 into master
* refs/pull/30552/head:
	cmake: fix libtsan detection
	test: librados startup/shutdown racer test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-09-30 20:46:14 -07:00
Jeff Layton
09bd258a9f cmake: fix libtsan detection
This variable is case-sensitive.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-30 14:49:12 -04:00
Ulrich Weigand
d1b9d14324 cmake: Test for 16-byte atomic support on IBM Z
On IBM Z the Boost tagged pointer implementation cannot use
"pointer compression" as there are no unused bits in an address;
the whole 64-bit address space is available to user space code.

Instead, Boost uses 16-byte atomics.  This is always supported
on IBM Z, but depending on the particular compiler (version)
it may require linking against libatomic.  The existing checks
in CheckCxxAtomic.cmake do not catch this, however, as they only
test for (up to) 8-byte atomic support.

Fixed by adding a test for 16-byte atomic support on IBM Z.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-29 16:22:52 +02:00
Ulrich Weigand
0065f76d53 headers: Remove __le16/__le32/__le64
These types should no longer be used; use ceph_le16/32/64 instead.

Also removes the cmake check whether those types are provided by
kernel headers (likewise for the __be types, which already were
not used anywhere).

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-18 13:35:21 +02:00
Kefu Chai
01ffda8fff
Merge pull request #30362 from uweigand/fix-unaligned-check
cmake: Fix unaligned check on big-endian systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-14 09:46:41 +08:00
Kefu Chai
8e9400db9c
Merge pull request #29761 from wjwithagen/wjw-fix-venv-python-version
test/venv: do not hardwire to py2.7 for tox tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-13 16:59:27 +08:00
Ulrich Weigand
5a189bdd41 cmake: Fix unaligned check on big-endian systems
On big-endian systems, ntohl is a no-op, so "good" never does
any conversion.  Fix this by keeping the test constants in
big-endian (network) order and using ntohl to convert them
to native (host) byte order.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-12 15:46:06 +02:00
Kefu Chai
34b33d447f cmake: check for unaligned access
C and C++ allow us to cast a pointer with an alignment requirement to
one with a different alignment requirement. but if the destination
type's alignment is greater/stricter than the source type, the behavior
is undefined. on amd64, it's allowed. and on aarch64, GCC offers an
option named `-munaligned-access` to enable this behavior. so we cannot
take this as guaranteed.

in denc.h and messenger implementations, we just cast the pointers to
the type we want to access without memcpy(), which is, strictly
speaking, not quite standard compliant.

so, to be on the safe side, we detect the support of unaligned access at
compile time.

please note, this would fail the cross-compile. as the target platform
might not support unaligned access, and the produced executable does not
run on the build platform. so this test is skipped if the building
host's architecture is different than that of target host.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-11 12:47:26 +08:00
Kefu Chai
67bf182aa5 cmake: extract checks into CephChecks.cmake
also use case chosen in the macro definition as per suggested by cmake
document. see
https://cmake.org/cmake/help/latest/command/macro.html#invocation

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-11 12:47:26 +08:00
Willem Jan Withagen
c70e7af4a6 test/venv: Fix the python version of the test environment
Otherwise it will result in:
[ 59%] preparing venv for mgr-dashboard
New python executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python2.7
Also creating executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /usr/local/bin/python2.7

Even though the Cmake options contained:
  '-DWITH_PYTHON2=OFF' '-DWITH_PYTHON3=3' '-DMGR_PYTHON_VERSION=3'

Mostly due to the fact that
  src/tools/setup-virtualenv.sh
default to:
  PYTHON_BINARY="python2.7"

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-08-27 15:10:21 +02:00
Ernesto Puerta
1e07237d92
mgr/dashboard: clean-up tox and unit tests
Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.

Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)

Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst

Doc:
- Update HACKING.rst

Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.

Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-16 19:17:27 +02:00
Kefu Chai
793308f82d cmake: pass tox envs to run_tox.sh explicitly
* add two optional keyword parameters to `add_tox_test()`
* use the default `TOX_PATH` parameter instead passing it explicitly,
  as it is always `${CMAKE_CURRENT_SOURCE_DIR}`
* drop the code to guess the tox envs in `run_tox.sh`, and always pass
  them explicitly in CMake script using the `--tox-envs` argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-06 00:06:29 +08:00
Kefu Chai
d68a84b0fd cmake: do not add gtest-parallel_ext if already added
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-05 19:11:51 +08:00
Kefu Chai
d359227ec0 cmake/modules/AddCephTest.cmake: make ${name}-venv a target
so we can prepare the venv for tox with `make tests`. and it's optional,
as run_tox.sh will check for it and setup a venv if it's not around.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-05 18:47:08 +08:00
Kefu Chai
bbb69fe793 cmake: require CMake v3.10.2
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-02 22:09:12 +08:00
Kefu Chai
95fd792be1
Merge pull request #29396 from wjwithagen/wjw-fix-boost-1.70
cmake: update FindBoost.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:49:34 +08:00
Kefu Chai
b9c9f74f06 cmake: add add_tox_test()
to consolidate the tox tests

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-01 21:33:30 +08:00
Willem Jan Withagen
fdc0e2baeb cmake: update FindBoost.cmake
FreeBSD is moving to Boost 1.70
we should allow user to use boost 1.70.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-07-30 16:00:04 +02:00
Kefu Chai
de9a04027f cmake/modules/Distutils: do not add ${name}-clone if already added
if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:

add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-26 12:11:25 +02:00
Kefu Chai
492efbd694 cmake: use python2 by default
and s/PYTHON_EXECUTABLE/Python_EXECUTABLE/

it's a regression introduced by 5e2bd7fc4d

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-22 16:07:56 +08:00
Kefu Chai
606b9c1822 cmake: pass PYTHON_VERSION explicitly
allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-21 00:47:39 +08:00
Kefu Chai
5e2bd7fc4d cmake: update FindPython* modules
use the ones shiped from the latest cmake. which

* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
  simpler this way and less error prone.

and to accomodate this change:

* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
  `Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
  `find_package(Python3...)` to find python2 or python3 instead of using
  `find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
  defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
  variables. because we find the python interpreter and development
  files using `find_pacakge(Python...)` for ceph-mgr.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-20 22:35:07 +08:00
Kefu Chai
7910bd6f02
Merge pull request #28802 from lixiaoy1/cmake_pmdk
cmake: pmem/pmdk changes to cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-12 14:55:34 +08:00
Kefu Chai
fcfc33b412
Merge pull request #28984 from tchaikov/wip-cmake-rebuild-cleanup
cmake: use BUILD_ALWAYS for rebuilding external project

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-12 14:54:30 +08:00
Kefu Chai
6958d2dcb1 cmake/modules/BuildRocksDB.cmake: use BUILD_ALWAYS
this option was introduced by cmake 3.1, and we require cmake 3.5.1, so
we can use it without using the homebrew solution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-11 19:30:28 +08:00
lixiaoy1
4959835c07 cmake: Add the support to use system PMDK library
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2019-07-10 11:43:27 -04:00
lixiaoy1
4528aa2d0f cmake: move pmdk build script to Buildpmem.cmake
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2019-07-10 11:43:27 -04:00
Kefu Chai
6453200265
Merge pull request #28920 from tchaikov/wip-cmake-pybind
cmake: avoid rebuilding extensions, and using python-config

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-09 16:05:44 +08:00
Kefu Chai
dadc66a58e
Merge pull request #28843 from tchaikov/wip-armhf-build
cmake, pybind: fix build on armhf

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-09 16:03:35 +08:00
Kefu Chai
1468c8fc35 cmake/modules/Distutils.cmake: add setup as a dependency
so the extension will be rebuilt if setup.py is updated

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:30:14 +08:00
Kefu Chai
f17861cdfa cmake/modules/Distutils.cmake: avoid rebuilding cython ext
use the combination of add_custom_command() and add_custom_target() to
build cython extensions to avoid rebuilding the cython extension
everytime we run "make"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:30:14 +08:00
Kefu Chai
83ffb1e2ca cmake/modules/Distutils.cmake: add byproduct
so add_custom_target() has better idea what is generated. if any target
will use it, it can be used as a dependency.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:29:07 +08:00
Kefu Chai
855685f0fc cmake: add an arg for name of cython extension
so we can deduce the path of cython extension

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 19:32:51 +08:00
Kefu Chai
a4ddc4bd85
Merge pull request #27834 from rzarzynski/wip-nss-drop-cms
rgw, common, build: drop NSS support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-05 11:03:33 +08:00
Kefu Chai
2fa4013818 cmake/modules/BuildBoost.cmake: fix build for armhf
see also
https://www.boost.org/doc/libs/1_70_0/libs/context/doc/html/context/architectures.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-02 23:40:21 +08:00
Jason Dillaman
b595568218 cmake: boost valgrind fixes for boost::lockfree::queue
The issue has been fixed upstream under lockfree commit
7e23dac52d08ed1a099de9a6fb8bcdefbb06d2da but is not yet available in
a boost release.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-28 10:56:03 -04:00
Radoslaw Zarzynski
79edb93d06 common, crypto, build: get rid of NSS entirely.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-06-26 22:10:17 +02:00
Kefu Chai
c028de2f93 cmake: rewrite Findgenl to support components argument
* rename genl to nl:
  "genl" is not very specific without more context. and libnl is actually
  a library suite. so it would be better if we can rename the library to
  be found to "libnl", and by following the naming convention of CMake,
  in this change, Findgenl.cmake is renamed to Findnl.cmake
* Findnl.cmake: support the component argument, in our specific case,
  what we want is "libnl-genl" library, which is one of the libraries
  offered by libnl. so let's just make it a component.
* Findnl.cmake: should pass the names of the required variables instead of
  their values to `find_package_handle_standard_args()`. before this
  change, we pass `GENL_LIBRARIES` to this function. it was correct at the
  first glance. but it is not able to handle the case where case where
  libnl-genl is not installed. so the fix is to pass all the names of
  required library paths to this function. in this change, their name
  are concatenated to a single variable -- `nl_LIBRARIES`, and the
  value of this variable is passed to
  `find_package_handle_standard_args()`. and the error message would
  be more specific this way if libnl-genl is not found:
    Could NOT find nl (missing: nl_genl_LIBRARY)
* Findnl.cmake: add nl::<component> as imported library, it helps the
  consumer of these libraries to import them -- no need to
  specify the `target_include_directories()` separately anymore.
* move the find_package() code to where it is used. it helps to improve
  the readability.
* remove `HAVE_GENL` variable: it's not used anywhere.
* drop the messages of "Not using foobar", they do not help.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-19 21:21:00 +08:00
Kefu Chai
764de84088 cmake: silence -Waddress-of-packed-member warnings from SPDK
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
b3cac157f5 cmake: silence GCC-9 warnings from rocksdb
GCC-9 complains:

rocksdb/db/version_edit.h:86:8: warning: implicitly-declared ‘constexpr
rocksdb::FileDescriptor::FileDescriptor(const rocksdb::FileDescriptor&)’
is deprecated [-Wdeprecated-copy]
   86 | struct FileMetaData {
      |        ^~~~~~~~~~~~
/var/ssd/ceph/src/rocksdb/db/version_edit.h:55:19: note: because
‘rocksdb::FileDescriptor’ has user-provided ‘rocksdb::FileDescriptor&
rocksdb::FileDescriptor::operator=(const rocksdb::FileDescriptor&)’
   55 |   FileDescriptor& operator=(const FileDescriptor& fd) {
      |                   ^~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
485017a3ef cmake: set ccache for rocksdb
should use launcher to set ccache

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
423c28fb8d cmake: update built libs of SPDK and DPDK
we need to link against these new libs for fulfilling the dependencies
required by new SPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
b2bdd3f9ee cmake: disable isal and vhost support of SPDK
* isal needs nasm, which is a new dependency. so let's disable it
  at this moment.
* vhost requires DPDK compiled with vhost enabled. but it's disabled
  when we compile DPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:23 +08:00
Kefu Chai
111d10df52 cmake: move boost patch to cmake/modules
as it's used by cmake only

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 16:32:10 +08:00
Kefu Chai
b848bce504 cmake/modules/BuildDPDK.cmake: update to adapt upstream changes
* update execenv to remove "app", the targets ending with "app" changed
  to the ones without "app" now.
* pass CC by make variables not env variables, otherwise the default
  `cc` is used

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 15:43:05 +08:00
Mike Christie
8815671553 cmake and build: add netlink lib requirement detection
The next patch adds netlink support to rbd-nbd. This patch just adds the
build related changes to bring in the netlink lib and test and build
against it.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Kefu Chai
5bcaeb3269
Merge pull request #28252 from tchaikov/wip-cmake-cleanup
cmake: cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-05 00:50:02 +08:00
Kefu Chai
ed2ce0efad
Merge pull request #28339 from tchaikov/wip-cmake-asan
cmake/modules: replace ";" with " " in compile flags

Reviewed-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-06-03 12:28:32 +08:00
Kefu Chai
714fbf64ae cmake/modules: replace ";" with " " in compile flags
as the flags should be a string, not a list

tested using

cmake -DCMAKE_LINKER=gold -DWITH_ASAN=ON -DWITH_ASAN_LEAK=ON \
-DCMAKE_BUILD_TYPE=Debug

on ubuntu xenial

without this change, cmake will fail with following warning:

CMake Error at src/CMakeLists.txt:112 (message):
  The compiler /usr/bin/c++ has no C++17 support.

-- Configuring incomplete, errors occurred!

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-31 16:19:08 +08:00
Daniel Glaser
c89a6998f4 cmake: boost fixes for ARM 32 bit
see https://github.com/boostorg/context/pull/94

we should drop this change once ceph uses boost 1.70 or up

Signed-off-by: Daniel Glaser <me@the78mole.de>
2019-05-29 18:30:59 +08:00
Kefu Chai
63242a0191 cmake: use BUILD_BYPRODUCTS to fix build w/ ninja
ninja scan the build dependencies differently than GNU make, so we need
to specify the product of `ExternalProject_Add()`. with this change,
we can build `vstart-base` without building the ExternalProject
separately.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-27 19:57:18 +08:00
dudengke
7b6e05a030 cmake/FindRocksDB: fix IMPORTED_LOCATION for ROCKSDB_LIBRARIES
Signed-off-by: dudengke <pinganddu90@gmail.com>
2019-03-27 10:18:26 +08:00
Sage Weil
be1187575b Merge PR #27021 into master
* refs/pull/27021/head:
	msg: remove XioMessenger
	qa/suites/rados/thrash-old-clients: add nautilus
	qa/suites/rados/thrash-old-clients: add mimic v1 variant
	qa/suites/rados/thrash-old-clients: add mimic
	qa/suites/rados/thrash-old-clients: collapse msgr and client choice
	qa: remove simplemessenger tests
	ceph_test_msgr: remove simple
	msg: remove SimpleMessenger

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-22 04:42:30 -05:00
Kefu Chai
96775c6fed
Merge pull request #27067 from SUSE/wip-fix-38788
cmake: remove cython 0.29's subinterpreter check during install

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-22 15:31:53 +08:00
Kefu Chai
085fdf57fd
Merge pull request #27089 from tchaikov/wip-cmake-with-ninja
cmake: do not assume ${CMAKE_GENERATOR} == make

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-21 22:05:54 +08:00
Kefu Chai
77553040fc cmake: workaround of false alarm from ubsan
to workaround a GCC bug, see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88684

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 18:57:27 +08:00
Kefu Chai
a0c6fb9d40 cmake: do not assume ${CMAKE_GENERATOR} == make
* modules/BuildDPDK.cmake: always use "make"
* modules/BuildSPDK.cmake: always use "make"
  DPDK only support "make". if user wants to use other generator than
  "make", we will fail to build DPDK support. so we should always use
  "make" here. and before using "make", we need to detect it.
* cmake/modules/BuildRocksDB.cmake: use cmake for build
* src/compressor/zstd/CMakeLists.txt: use cmake for build
  just use whatever generator use specifies, and use ${CMAKE_COMMAND}
  for building the specified target, no need to repeat
  ${CMAKE_GENERATOR} for building the target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 15:45:07 +08:00
Sage Weil
cc9a9142fd msg: remove XioMessenger
There is a lot of good stuff going on here, but nobody is investing in xio
and it is not expected to be the path forward for RDMA.  If that ever
changes, we can resurrect the code.  Until then, let's clean up the tree
and reduce friction for changes going forward.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-21 00:57:33 -05:00
Tim Serong
108462e3fb cmake: remove cython 0.29's subinterpreter check during install
Commit 3bde34af8a removed cython 0.29's subinterpreter check when
building the various python modules during `make`, but unforunately
they're *rebuilt* during `make install`, with the rebuild overwriting
the original build.  The original fix was of course missing from the
install stage...

Fixes: https://tracker.ceph.com/issues/38788
Signed-off-by: Tim Serong <tserong@suse.com>
2019-03-20 18:56:43 +11:00
Kefu Chai
95401d8c2f cmake: find_package(RabbitMQ) instead of just checking its header
rabbitmq-c is not a header-only library. so we need to ensure that the
library is also around before linking against it. in this change:

* move the detection down to where librabbitmq is used
* add FindRabbitMQ.cmake for find_package(RabbitMQ)
* s/rabbitmq/RabbitMQ::RabbitMQ/ as the latter is an alias library
target, which has more information attached to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-21 19:00:24 +08:00
Kefu Chai
d8b83cc283
Merge pull request #26316 from wjwithagen/wjw-fix-FindStdFilesystem
cmake: Make the tests for finding Filesystem with more serious functions

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-15 01:06:08 +08:00
Willem Jan Withagen
1d58db8c9b cmake: Make the tests for finding Filesystem with more serious functions
Otherwise it is linked agains lc++ on FreeBSD,a dn that will satisfy both
tested libraries.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-02-14 13:18:50 +01:00
Kefu Chai
1910e09459 cmake: update FindBoost.cmake
we should allow user to use boost 1.69, and fc30 is moving to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-07 15:35:16 +08:00
Kefu Chai
e57649dd9c
Merge pull request #22800 from wjwithagen/wjw-ctest-timeout
cmake: allow setting of the CTest timeout during building.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-21 23:40:27 +08:00
Tim Serong
3bde34af8a cmake: remove cython 0.29's subinterpreter check
cython 0.29 introduced a check which prevents multiple python
subinterpreters from loading the same module:

https://github.com/cython/cython/commit/7e27c7c

Unfortunately, this completely breaks ceph-mgr.  Until we can
figure out a better long term solution, this commit removes
cython's subinterpreter check, via some careful abuse of the
C preprocessor.

This works because when cython is invoked, it first generates
some C code, then compiles it.  We know it's going to generate
C code including:

  int __Pyx_check_single_interpreter(void) { ... }

and:

  if (__Pyx_check_single_interpreter())
      return NULL;

So, we can do the following:

  #define void0 dead_function(void)
  #define __Pyx_check_single_interpreter(ARG)=ARG ## 0

This replaces the call to __Pyx_check_single_interpreter()
with a literal 0, removing the subinterpreter check.

The void0 dead_function(void) thing is necessary because
the __Pyx_check_single_interpreter() macro also clobbers
that function definition, so we need to make sure it's
replaced with something that works as a function definition.

Fixes: https://tracker.ceph.com/issues/37472
Signed-off-by: Tim Serong <tserong@suse.com>
2018-12-17 23:27:06 +11:00
Sage Weil
59d0844c12 Merge PR #23550 into master
* refs/pull/23550/head:
	auth: Kerberos authentication
2018-12-05 13:42:17 -06:00
oliveiradan
67784065ce auth: Kerberos authentication
Signed-off-by: Daniel Oliveira <doliveira@suse.com> (github: oliveiradan)
2018-12-03 18:55:46 -07:00
Kefu Chai
ebfa2df2d5 cmake: do not pass unnecessary param to setup.py
--single-version-externally-managed is used for packaging python eggs
to deb. so the metadata can be stored in the egg, and can be accessed
by user if necessary, see [1]. but this does not apply to users who
just want to "make install" without packaging Ceph.

the problem is, if we pass --single-version-externally-managed to
setup.py, we need to pass --record or --root also. otherwise setuptools
complains:

error: You must specify --record or --root when building system packages

so in this change, we choose not to pass
"--single-version-externally-managed", unless $DESTDIR is set.

both "ceph.spec.in" and "debian/rules" set $DESTDIR when running "make
install". so we can always check $DESTDIR to tell if current "make
install" is launched by the packaging script or not. this might not be
accurate, but good enough. assuming enduser does not use DESTDIR when
running plain "make install".

--
[1] https://wiki.debian.org/Python/FAQ#How_should_we_package_Python_eggs.3F

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-20 22:20:41 +08:00
Willem Jan Withagen
fd71812f87 cmake: allow setting of the CTest timeout during building.
It is rather had to overide a once set timeout during CTesting.
This is the best next compromise. Disadvantage is that a rebuild
is required.
But it helps in my Jenkins testing timely catching run-away programs.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-10-29 23:16:33 +01:00
Kefu Chai
ef305c178a cmake: update dpdk drivers/modules to accomodate dpdk submodule
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-26 21:49:03 +08:00
Kefu Chai
7a3bf10907
Merge pull request #23962 from aclamk/fio-fix-compilation
objectstore/test/fio: Fixed fio compilation when tcmalloc is used.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-25 10:24:28 +08:00
Kefu Chai
fd58e5d4ad cmake,ceph.in: preload libasan if WITH_ASAN
we need to preload libasan.so as the python exectuable is not likely to
be compiled with ASan enabled.
see:
https://github.com/google/sanitizers/wiki/AddressSanitizerAsDso#asan-and-ld_preload

just to ease the use of ASan, for fine-tuned behaviour, use
`ASAN_OPTIONS`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
38e7686a37 cmake: pass cflags to disutils using CC instead of CFLAGS
in python's distutils.ccompiler, linker_exe is composed using CC instead
of LDFLAGS. the latter only effects how it builds (shared) library.

and put CMAKE_C_FLAGS into the cflags for the compiler for building
python C extensions, it's more consistent this way. more importantly,
if we build with ASan enabled, the canary program, a.k.a. rados_dummy.c,
won't link without proper CFLAGS.

without this change, rados.so fails to build with errors like:

/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_stack_free_10'
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_report_exp_store8'
...
...

clang: error: linker command failed with exit code 1 (use -v to see
invocation)

Link Error: RADOS library not found
make[3]: ***
[src/pybind/rados/CMakeFiles/cython_rados.dir/build.make:57:
src/pybind/rados/CMakeFiles/cython_rados] Error 1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
ccb0489895
Merge pull request #24605 from tchaikov/wip-use-ccache-the-old-way
cmake: fix compilation with distcc and other compiler wrappers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-19 20:57:04 +08:00
Ganesh Maharaj Mahalingam
bf5f83decc crypto: QAT Wrapper for Ceph Encryption
- Crypto plugin created and currently handles only AES_256_CBC
- Contents
-- Poller thread for QAT HW
-- Contiguous memory allocator using QAT's usdm driver
-- Do in-place crypto operation reducing mem usage.
-- Enable multi-QAT instance to allow parallel operations.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2018-10-18 21:48:18 -07:00
Alexey Sheplyakov
2eeb48dd33 fix compilation with distcc and other compiler wrappers
When building with ccache, distcc, and other compiler wrappers (such
as STLFilt):

CC='ccache gcc' CXX='ccache g++' cmake /path/to/ceph
make

python modules fail to compile since distutils try to execute the
wrapper itself without specifying the actual compiler.
Although cmake has a special magic switch for compiling with ccache
(cmake -DWITH_CCACHE=ON) other tools (distcc) are not supported, and
specifying the compiler as

CC=/whatever/compiler/is

used to work for decades, and it's a good idea to keep it working

Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-16 11:33:29 +08:00
Brad Hubbard
2b188a2aed cmake/modules/BuildDPDK.cmake: Build required DPDK libraries
Make sure the mbuf and ethdev libraries are created as they export
required symbols.

Fixes: http://tracker.ceph.com/issues/36341

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-10-10 14:29:10 +10:00
Josh Durgin
8f826e146c
Merge pull request #24168 from tchaikov/wip-17516
cmake: detect armv8 crc and crypto feature using CHECK_C_COMPILER_FLAG

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-04 16:10:18 -07:00
Kefu Chai
3229c7acd9 cmake: fix typos
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-21 12:41:42 +08:00
Kefu Chai
010edc0f1d cmake: detect armv8 crc and crypto feature using CHECK_C_COMPILER_FLAG
we are using GCC7 and up for C++17 support since mimic, and per
https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/AArch64-Options.html ,
GCC 4.9 and up should be able to support crc and crypto features if
these archs are enabled at GCC's configure-time. so we should always use
the -march for detecting the compiler's support instead of using the
inline assembly now.

GCC 4.8 is an ancient compiler, and per
https://gcc.gnu.org/gcc-4.8/changes.html, it was the the first GCC
release which offers AArch64 support. so we don't need to cater for
this GCC version. and we can trust GCC-7 and up.

Fixes: http://tracker.ceph.com/issues/17516
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-20 22:15:56 +08:00
Kefu Chai
07a3e8d7a6
Merge pull request #23650 from tchaikov/wip-dpdk-cmake
cmake: fix the build WITH_DPDK=ON

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-09-14 23:44:01 +08:00
Kefu Chai
b61afd73e4
Merge pull request #23995 from tchaikov/wip-add-ceph-libboost-repo
install-deps,run-make-check: use ceph-libboost repo

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Erwan Velu <erwan@redhat.com>
2018-09-13 19:40:26 +08:00
Adam Kupczyk
a0025dbaf4 objectstore/test/fio: Fixed fio compilation when tcmalloc is used.
Updated fio README.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2018-09-13 09:43:48 +02:00
Kefu Chai
027164e156 cmake/modules: add FindBoost.cmake
the FindBoost.cmake shipped by cmake 3.5.1 is not new enough to identify
the dependencies of boost 1.63 and up. so we need to copy the
FindBoost.cmake from cmake upstream. it supports up to boost 1.68.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-11 22:22:47 +08:00
Mohamad Gebai
872d909d22 cmake: fix version matching for Findfmt
Signed-off-by: Mohamad Gebai <mgebai@suse.com>
2018-09-10 10:22:24 -04:00
Kefu Chai
2c823a18cb cmake: pass DPDK_DIR explicitly
so we don't rely on build_dpdk() to be a macro to set variables in
its parent scope.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-05 17:33:59 +08:00
Kefu Chai
1a6877dc04 cmake/modules/BuildDPDK.cmake: link whole-archive
we build dpdk as static library, linking against them without specifying
--whole-archive will get the constructor symbols droppped by the linker
as they are not referenced directly by the excecutable. so we need to do
so to make sure we call all the constructors.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-05 15:40:30 +08:00
Kefu Chai
c976f580e7 cmake: fix the build WITH_DPDK=ON
* add Findcryptopp.cmake back
  cryptopp support was dropped in #20015, but it's required by
  src/msg/async/dpdk/TCP.h, which `#include <cryptopp/md5.h>`
  so, to fix the FTBFS of WITH_DPDK=ON, we need to bring
  Findcryptopp.cmake back. it was also removed in #20015.
* pass "-march=core2" when building sources which include dpdk headers.
  i was wrong that the headers shipped by distro are generic.
  the headers use the sse instructions for speedup memcpy, see
  /usr/include/x86_64-linux-gnu/dpdk/rte_memcpy.h .
* also, we need to include the arch specific include directory
  for building with dpdk.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-05 13:53:57 +08:00
Kefu Chai
dda006a007 cmake: should use lowercase for component name in FindSanitizers.cmake
and s/undefined-behavior/undefined_behavior/ to be compatible with
seastar

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-21 20:05:49 +08:00
Kefu Chai
1d1833ece0
Merge pull request #23628 from tchaikov/wip-cmake-gtest-cleanup
cmake: link against gtest in a better way

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-08-19 22:58:47 +08:00