Commit Graph

317 Commits

Author SHA1 Message Date
Kefu Chai
de9a04027f cmake/modules/Distutils: do not add ${name}-clone if already added
if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:

add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-26 12:11:25 +02:00
Kefu Chai
492efbd694 cmake: use python2 by default
and s/PYTHON_EXECUTABLE/Python_EXECUTABLE/

it's a regression introduced by 5e2bd7fc4d

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-22 16:07:56 +08:00
Kefu Chai
606b9c1822 cmake: pass PYTHON_VERSION explicitly
allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-21 00:47:39 +08:00
Kefu Chai
5e2bd7fc4d cmake: update FindPython* modules
use the ones shiped from the latest cmake. which

* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
  simpler this way and less error prone.

and to accomodate this change:

* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
  `Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
  `find_package(Python3...)` to find python2 or python3 instead of using
  `find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
  defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
  variables. because we find the python interpreter and development
  files using `find_pacakge(Python...)` for ceph-mgr.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-20 22:35:07 +08:00
Kefu Chai
7910bd6f02
Merge pull request #28802 from lixiaoy1/cmake_pmdk
cmake: pmem/pmdk changes to cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-12 14:55:34 +08:00
Kefu Chai
fcfc33b412
Merge pull request #28984 from tchaikov/wip-cmake-rebuild-cleanup
cmake: use BUILD_ALWAYS for rebuilding external project

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-12 14:54:30 +08:00
Kefu Chai
6958d2dcb1 cmake/modules/BuildRocksDB.cmake: use BUILD_ALWAYS
this option was introduced by cmake 3.1, and we require cmake 3.5.1, so
we can use it without using the homebrew solution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-11 19:30:28 +08:00
lixiaoy1
4959835c07 cmake: Add the support to use system PMDK library
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2019-07-10 11:43:27 -04:00
lixiaoy1
4528aa2d0f cmake: move pmdk build script to Buildpmem.cmake
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2019-07-10 11:43:27 -04:00
Kefu Chai
6453200265
Merge pull request #28920 from tchaikov/wip-cmake-pybind
cmake: avoid rebuilding extensions, and using python-config

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-09 16:05:44 +08:00
Kefu Chai
dadc66a58e
Merge pull request #28843 from tchaikov/wip-armhf-build
cmake, pybind: fix build on armhf

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-09 16:03:35 +08:00
Kefu Chai
1468c8fc35 cmake/modules/Distutils.cmake: add setup as a dependency
so the extension will be rebuilt if setup.py is updated

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:30:14 +08:00
Kefu Chai
f17861cdfa cmake/modules/Distutils.cmake: avoid rebuilding cython ext
use the combination of add_custom_command() and add_custom_target() to
build cython extensions to avoid rebuilding the cython extension
everytime we run "make"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:30:14 +08:00
Kefu Chai
83ffb1e2ca cmake/modules/Distutils.cmake: add byproduct
so add_custom_target() has better idea what is generated. if any target
will use it, it can be used as a dependency.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:29:07 +08:00
Kefu Chai
855685f0fc cmake: add an arg for name of cython extension
so we can deduce the path of cython extension

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 19:32:51 +08:00
Kefu Chai
a4ddc4bd85
Merge pull request #27834 from rzarzynski/wip-nss-drop-cms
rgw, common, build: drop NSS support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-05 11:03:33 +08:00
Kefu Chai
2fa4013818 cmake/modules/BuildBoost.cmake: fix build for armhf
see also
https://www.boost.org/doc/libs/1_70_0/libs/context/doc/html/context/architectures.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-02 23:40:21 +08:00
Jason Dillaman
b595568218 cmake: boost valgrind fixes for boost::lockfree::queue
The issue has been fixed upstream under lockfree commit
7e23dac52d08ed1a099de9a6fb8bcdefbb06d2da but is not yet available in
a boost release.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-28 10:56:03 -04:00
Radoslaw Zarzynski
79edb93d06 common, crypto, build: get rid of NSS entirely.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-06-26 22:10:17 +02:00
Kefu Chai
c028de2f93 cmake: rewrite Findgenl to support components argument
* rename genl to nl:
  "genl" is not very specific without more context. and libnl is actually
  a library suite. so it would be better if we can rename the library to
  be found to "libnl", and by following the naming convention of CMake,
  in this change, Findgenl.cmake is renamed to Findnl.cmake
* Findnl.cmake: support the component argument, in our specific case,
  what we want is "libnl-genl" library, which is one of the libraries
  offered by libnl. so let's just make it a component.
* Findnl.cmake: should pass the names of the required variables instead of
  their values to `find_package_handle_standard_args()`. before this
  change, we pass `GENL_LIBRARIES` to this function. it was correct at the
  first glance. but it is not able to handle the case where case where
  libnl-genl is not installed. so the fix is to pass all the names of
  required library paths to this function. in this change, their name
  are concatenated to a single variable -- `nl_LIBRARIES`, and the
  value of this variable is passed to
  `find_package_handle_standard_args()`. and the error message would
  be more specific this way if libnl-genl is not found:
    Could NOT find nl (missing: nl_genl_LIBRARY)
* Findnl.cmake: add nl::<component> as imported library, it helps the
  consumer of these libraries to import them -- no need to
  specify the `target_include_directories()` separately anymore.
* move the find_package() code to where it is used. it helps to improve
  the readability.
* remove `HAVE_GENL` variable: it's not used anywhere.
* drop the messages of "Not using foobar", they do not help.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-19 21:21:00 +08:00
Kefu Chai
764de84088 cmake: silence -Waddress-of-packed-member warnings from SPDK
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
b3cac157f5 cmake: silence GCC-9 warnings from rocksdb
GCC-9 complains:

rocksdb/db/version_edit.h:86:8: warning: implicitly-declared ‘constexpr
rocksdb::FileDescriptor::FileDescriptor(const rocksdb::FileDescriptor&)’
is deprecated [-Wdeprecated-copy]
   86 | struct FileMetaData {
      |        ^~~~~~~~~~~~
/var/ssd/ceph/src/rocksdb/db/version_edit.h:55:19: note: because
‘rocksdb::FileDescriptor’ has user-provided ‘rocksdb::FileDescriptor&
rocksdb::FileDescriptor::operator=(const rocksdb::FileDescriptor&)’
   55 |   FileDescriptor& operator=(const FileDescriptor& fd) {
      |                   ^~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
485017a3ef cmake: set ccache for rocksdb
should use launcher to set ccache

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
423c28fb8d cmake: update built libs of SPDK and DPDK
we need to link against these new libs for fulfilling the dependencies
required by new SPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:26 +08:00
Kefu Chai
b2bdd3f9ee cmake: disable isal and vhost support of SPDK
* isal needs nasm, which is a new dependency. so let's disable it
  at this moment.
* vhost requires DPDK compiled with vhost enabled. but it's disabled
  when we compile DPDK.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-13 19:35:23 +08:00
Kefu Chai
111d10df52 cmake: move boost patch to cmake/modules
as it's used by cmake only

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 16:32:10 +08:00
Kefu Chai
b848bce504 cmake/modules/BuildDPDK.cmake: update to adapt upstream changes
* update execenv to remove "app", the targets ending with "app" changed
  to the ones without "app" now.
* pass CC by make variables not env variables, otherwise the default
  `cc` is used

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 15:43:05 +08:00
Mike Christie
8815671553 cmake and build: add netlink lib requirement detection
The next patch adds netlink support to rbd-nbd. This patch just adds the
build related changes to bring in the netlink lib and test and build
against it.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Kefu Chai
5bcaeb3269
Merge pull request #28252 from tchaikov/wip-cmake-cleanup
cmake: cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-05 00:50:02 +08:00
Kefu Chai
ed2ce0efad
Merge pull request #28339 from tchaikov/wip-cmake-asan
cmake/modules: replace ";" with " " in compile flags

Reviewed-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-06-03 12:28:32 +08:00
Kefu Chai
714fbf64ae cmake/modules: replace ";" with " " in compile flags
as the flags should be a string, not a list

tested using

cmake -DCMAKE_LINKER=gold -DWITH_ASAN=ON -DWITH_ASAN_LEAK=ON \
-DCMAKE_BUILD_TYPE=Debug

on ubuntu xenial

without this change, cmake will fail with following warning:

CMake Error at src/CMakeLists.txt:112 (message):
  The compiler /usr/bin/c++ has no C++17 support.

-- Configuring incomplete, errors occurred!

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-31 16:19:08 +08:00
Daniel Glaser
c89a6998f4 cmake: boost fixes for ARM 32 bit
see https://github.com/boostorg/context/pull/94

we should drop this change once ceph uses boost 1.70 or up

Signed-off-by: Daniel Glaser <me@the78mole.de>
2019-05-29 18:30:59 +08:00
Kefu Chai
63242a0191 cmake: use BUILD_BYPRODUCTS to fix build w/ ninja
ninja scan the build dependencies differently than GNU make, so we need
to specify the product of `ExternalProject_Add()`. with this change,
we can build `vstart-base` without building the ExternalProject
separately.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-27 19:57:18 +08:00
dudengke
7b6e05a030 cmake/FindRocksDB: fix IMPORTED_LOCATION for ROCKSDB_LIBRARIES
Signed-off-by: dudengke <pinganddu90@gmail.com>
2019-03-27 10:18:26 +08:00
Sage Weil
be1187575b Merge PR #27021 into master
* refs/pull/27021/head:
	msg: remove XioMessenger
	qa/suites/rados/thrash-old-clients: add nautilus
	qa/suites/rados/thrash-old-clients: add mimic v1 variant
	qa/suites/rados/thrash-old-clients: add mimic
	qa/suites/rados/thrash-old-clients: collapse msgr and client choice
	qa: remove simplemessenger tests
	ceph_test_msgr: remove simple
	msg: remove SimpleMessenger

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-22 04:42:30 -05:00
Kefu Chai
96775c6fed
Merge pull request #27067 from SUSE/wip-fix-38788
cmake: remove cython 0.29's subinterpreter check during install

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-22 15:31:53 +08:00
Kefu Chai
085fdf57fd
Merge pull request #27089 from tchaikov/wip-cmake-with-ninja
cmake: do not assume ${CMAKE_GENERATOR} == make

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-21 22:05:54 +08:00
Kefu Chai
77553040fc cmake: workaround of false alarm from ubsan
to workaround a GCC bug, see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88684

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 18:57:27 +08:00
Kefu Chai
a0c6fb9d40 cmake: do not assume ${CMAKE_GENERATOR} == make
* modules/BuildDPDK.cmake: always use "make"
* modules/BuildSPDK.cmake: always use "make"
  DPDK only support "make". if user wants to use other generator than
  "make", we will fail to build DPDK support. so we should always use
  "make" here. and before using "make", we need to detect it.
* cmake/modules/BuildRocksDB.cmake: use cmake for build
* src/compressor/zstd/CMakeLists.txt: use cmake for build
  just use whatever generator use specifies, and use ${CMAKE_COMMAND}
  for building the specified target, no need to repeat
  ${CMAKE_GENERATOR} for building the target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 15:45:07 +08:00
Sage Weil
cc9a9142fd msg: remove XioMessenger
There is a lot of good stuff going on here, but nobody is investing in xio
and it is not expected to be the path forward for RDMA.  If that ever
changes, we can resurrect the code.  Until then, let's clean up the tree
and reduce friction for changes going forward.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-21 00:57:33 -05:00
Tim Serong
108462e3fb cmake: remove cython 0.29's subinterpreter check during install
Commit 3bde34af8a removed cython 0.29's subinterpreter check when
building the various python modules during `make`, but unforunately
they're *rebuilt* during `make install`, with the rebuild overwriting
the original build.  The original fix was of course missing from the
install stage...

Fixes: https://tracker.ceph.com/issues/38788
Signed-off-by: Tim Serong <tserong@suse.com>
2019-03-20 18:56:43 +11:00
Kefu Chai
95401d8c2f cmake: find_package(RabbitMQ) instead of just checking its header
rabbitmq-c is not a header-only library. so we need to ensure that the
library is also around before linking against it. in this change:

* move the detection down to where librabbitmq is used
* add FindRabbitMQ.cmake for find_package(RabbitMQ)
* s/rabbitmq/RabbitMQ::RabbitMQ/ as the latter is an alias library
target, which has more information attached to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-21 19:00:24 +08:00
Kefu Chai
d8b83cc283
Merge pull request #26316 from wjwithagen/wjw-fix-FindStdFilesystem
cmake: Make the tests for finding Filesystem with more serious functions

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-15 01:06:08 +08:00
Willem Jan Withagen
1d58db8c9b cmake: Make the tests for finding Filesystem with more serious functions
Otherwise it is linked agains lc++ on FreeBSD,a dn that will satisfy both
tested libraries.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-02-14 13:18:50 +01:00
Kefu Chai
1910e09459 cmake: update FindBoost.cmake
we should allow user to use boost 1.69, and fc30 is moving to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-07 15:35:16 +08:00
Kefu Chai
e57649dd9c
Merge pull request #22800 from wjwithagen/wjw-ctest-timeout
cmake: allow setting of the CTest timeout during building.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-21 23:40:27 +08:00
Tim Serong
3bde34af8a cmake: remove cython 0.29's subinterpreter check
cython 0.29 introduced a check which prevents multiple python
subinterpreters from loading the same module:

https://github.com/cython/cython/commit/7e27c7c

Unfortunately, this completely breaks ceph-mgr.  Until we can
figure out a better long term solution, this commit removes
cython's subinterpreter check, via some careful abuse of the
C preprocessor.

This works because when cython is invoked, it first generates
some C code, then compiles it.  We know it's going to generate
C code including:

  int __Pyx_check_single_interpreter(void) { ... }

and:

  if (__Pyx_check_single_interpreter())
      return NULL;

So, we can do the following:

  #define void0 dead_function(void)
  #define __Pyx_check_single_interpreter(ARG)=ARG ## 0

This replaces the call to __Pyx_check_single_interpreter()
with a literal 0, removing the subinterpreter check.

The void0 dead_function(void) thing is necessary because
the __Pyx_check_single_interpreter() macro also clobbers
that function definition, so we need to make sure it's
replaced with something that works as a function definition.

Fixes: https://tracker.ceph.com/issues/37472
Signed-off-by: Tim Serong <tserong@suse.com>
2018-12-17 23:27:06 +11:00
Sage Weil
59d0844c12 Merge PR #23550 into master
* refs/pull/23550/head:
	auth: Kerberos authentication
2018-12-05 13:42:17 -06:00
oliveiradan
67784065ce auth: Kerberos authentication
Signed-off-by: Daniel Oliveira <doliveira@suse.com> (github: oliveiradan)
2018-12-03 18:55:46 -07:00
Kefu Chai
ebfa2df2d5 cmake: do not pass unnecessary param to setup.py
--single-version-externally-managed is used for packaging python eggs
to deb. so the metadata can be stored in the egg, and can be accessed
by user if necessary, see [1]. but this does not apply to users who
just want to "make install" without packaging Ceph.

the problem is, if we pass --single-version-externally-managed to
setup.py, we need to pass --record or --root also. otherwise setuptools
complains:

error: You must specify --record or --root when building system packages

so in this change, we choose not to pass
"--single-version-externally-managed", unless $DESTDIR is set.

both "ceph.spec.in" and "debian/rules" set $DESTDIR when running "make
install". so we can always check $DESTDIR to tell if current "make
install" is launched by the packaging script or not. this might not be
accurate, but good enough. assuming enduser does not use DESTDIR when
running plain "make install".

--
[1] https://wiki.debian.org/Python/FAQ#How_should_we_package_Python_eggs.3F

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-20 22:20:41 +08:00