1
0
mirror of https://github.com/ceph/ceph synced 2024-12-26 13:33:57 +00:00
Commit Graph

354 Commits

Author SHA1 Message Date
Sage Weil
cf676c73ab Merge PR into master
* refs/pull/31781/head:
	cmake/modules: look for GNU make first instead of BSD make
	cmake: support parallel build for rocksdb

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-25 21:49:31 -06:00
Sage Weil
0e0a3853c9 Merge PR into master
* refs/pull/31579/head:
	cmake: add cppcheck and iwyu static analysis targets

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-11-25 21:49:18 -06:00
Deepika Upadhyay
abf3dc0cf9 cmake/modules: look for GNU make first instead of BSD make
* BuildDPDK.cmake, BuildSPDK.cmake:
      * give priority to build with gmake
      * throw error if make not found

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2019-11-25 13:16:04 +05:30
Deepika Upadhyay
a930706aa6 cmake: support parallel build for rocksdb
* cmake/modules/BuildRocksDB.cmake: if "make" detected
use concurrent processing, by passing the jobserver to
sub-make using `$(MAKE)`

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2019-11-25 13:14:22 +05:30
Casey Bodley
8aa638b0fa
Merge pull request from yuvalif/wip-yuval-add-kafka-notif-endpoint
rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-11-21 14:42:23 -05:00
Yuval Lifshitz
c876a0d817 cmake: add cppcheck and iwyu static analysis targets
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-18 19:43:14 +02:00
Kefu Chai
d6998a3b95 cmake: detect librt for POSIX time functions
per clock_gettime(3),

> On POSIX systems on which these functions are available, the symbol
> _POSIX_TIMERS is defined in <unistd.h> to a value greater than 0.

and

> Most systems require the program be linked with the librt library to
> use these functions.

so, we should detect this symbol and link against librt for using these
functions.

in this change, librt is linked for checking the existence of
clock_gettime(), if it exists. RT_LIBRARY is defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-14 21:26:57 +08:00
Kefu Chai
4c42b4d49d
Merge pull request from ifed01/wip-ifed-gettid2
test/fio: bump to fio-3.15

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-13 18:58:36 +08:00
Yingxin Cheng
bf958abc0f crimson: build seastar dpdk from src/seastar/dpdk
src/spdk/dpdk and src/seastar/dpdk are both at their private branches
with project-specific modifications, so select proper dpdk source
directory according to flags WITH_SPDK and Seastar_DPDK.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-11-13 15:53:43 +08:00
Yingxin Cheng
ab84158fc0 crimson: link required dpdk components for seastar
see https://github.com/scylladb/seastar/blob/
809b3ef875e7edb7c56643812dc8347ff9a82cee/cmake/Finddpdk.cmake#L60-L85

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-11-13 15:53:31 +08:00
Igor Fedotov
fcfb0c9442 cmake: detect gettid() presense
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-11-12 12:42:14 +03:00
Igor Fedotov
710e5da85b test/fio: bump to fio 3.15
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-11-12 02:24:45 +03:00
Yuval Lifshitz
354d775e04 rgw/pubsub: add kafka notification endpoint
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-05 19:25:03 +02:00
Kefu Chai
a123c85256 cmake: enable Finddpdk to find DPDK to be built
find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 17:54:05 +08:00
Kefu Chai
aa25f6202f cmake: define dpdk_LIBRARIES properly
dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Kefu Chai
d82ea5d738 cmake: refactor BuildDPDK
* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
  as the name of this target should be hidden from the caller of this
  function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Sage Weil
da0473e251 Merge PR into master
* refs/pull/30772/head:
	cmake/modules/GetGitRevisionDescription: update to work with git-worktree

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-10-22 15:57:02 -05:00
Kefu Chai
a826efc52f cmake: detect linker support
check the linker support of `--version-script` and `--exclude-libs`
flags. and use them only if they are supported.

llvm-ld does not support `--version-script` at this moment. and it's
reported that old versions of llvm-ld do not support `--exclude-libs`
as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-08 21:53:03 +08:00
Sage Weil
165f09d7e7 cmake/modules/GetGitRevisionDescription: update to work with git-worktree
Found this here: https://phabricator.kde.org/D9172

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-07 14:26:07 -05:00
Patrick Donnelly
bbe3a33775
Merge PR into master
* refs/pull/30552/head:
	cmake: fix libtsan detection
	test: librados startup/shutdown racer test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-09-30 20:46:14 -07:00
Jeff Layton
09bd258a9f cmake: fix libtsan detection
This variable is case-sensitive.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-30 14:49:12 -04:00
Ulrich Weigand
d1b9d14324 cmake: Test for 16-byte atomic support on IBM Z
On IBM Z the Boost tagged pointer implementation cannot use
"pointer compression" as there are no unused bits in an address;
the whole 64-bit address space is available to user space code.

Instead, Boost uses 16-byte atomics.  This is always supported
on IBM Z, but depending on the particular compiler (version)
it may require linking against libatomic.  The existing checks
in CheckCxxAtomic.cmake do not catch this, however, as they only
test for (up to) 8-byte atomic support.

Fixed by adding a test for 16-byte atomic support on IBM Z.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-29 16:22:52 +02:00
Ulrich Weigand
0065f76d53 headers: Remove __le16/__le32/__le64
These types should no longer be used; use ceph_le16/32/64 instead.

Also removes the cmake check whether those types are provided by
kernel headers (likewise for the __be types, which already were
not used anywhere).

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-18 13:35:21 +02:00
Kefu Chai
01ffda8fff
Merge pull request from uweigand/fix-unaligned-check
cmake: Fix unaligned check on big-endian systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-14 09:46:41 +08:00
Kefu Chai
8e9400db9c
Merge pull request from wjwithagen/wjw-fix-venv-python-version
test/venv: do not hardwire to py2.7 for tox tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-13 16:59:27 +08:00
Ulrich Weigand
5a189bdd41 cmake: Fix unaligned check on big-endian systems
On big-endian systems, ntohl is a no-op, so "good" never does
any conversion.  Fix this by keeping the test constants in
big-endian (network) order and using ntohl to convert them
to native (host) byte order.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-12 15:46:06 +02:00
Kefu Chai
34b33d447f cmake: check for unaligned access
C and C++ allow us to cast a pointer with an alignment requirement to
one with a different alignment requirement. but if the destination
type's alignment is greater/stricter than the source type, the behavior
is undefined. on amd64, it's allowed. and on aarch64, GCC offers an
option named `-munaligned-access` to enable this behavior. so we cannot
take this as guaranteed.

in denc.h and messenger implementations, we just cast the pointers to
the type we want to access without memcpy(), which is, strictly
speaking, not quite standard compliant.

so, to be on the safe side, we detect the support of unaligned access at
compile time.

please note, this would fail the cross-compile. as the target platform
might not support unaligned access, and the produced executable does not
run on the build platform. so this test is skipped if the building
host's architecture is different than that of target host.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-11 12:47:26 +08:00
Kefu Chai
67bf182aa5 cmake: extract checks into CephChecks.cmake
also use case chosen in the macro definition as per suggested by cmake
document. see
https://cmake.org/cmake/help/latest/command/macro.html#invocation

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-11 12:47:26 +08:00
Willem Jan Withagen
c70e7af4a6 test/venv: Fix the python version of the test environment
Otherwise it will result in:
[ 59%] preparing venv for mgr-dashboard
New python executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python2.7
Also creating executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /usr/local/bin/python2.7

Even though the Cmake options contained:
  '-DWITH_PYTHON2=OFF' '-DWITH_PYTHON3=3' '-DMGR_PYTHON_VERSION=3'

Mostly due to the fact that
  src/tools/setup-virtualenv.sh
default to:
  PYTHON_BINARY="python2.7"

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-08-27 15:10:21 +02:00
Ernesto Puerta
1e07237d92
mgr/dashboard: clean-up tox and unit tests
Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.

Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)

Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst

Doc:
- Update HACKING.rst

Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.

Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-16 19:17:27 +02:00
Kefu Chai
793308f82d cmake: pass tox envs to run_tox.sh explicitly
* add two optional keyword parameters to `add_tox_test()`
* use the default `TOX_PATH` parameter instead passing it explicitly,
  as it is always `${CMAKE_CURRENT_SOURCE_DIR}`
* drop the code to guess the tox envs in `run_tox.sh`, and always pass
  them explicitly in CMake script using the `--tox-envs` argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-06 00:06:29 +08:00
Kefu Chai
d68a84b0fd cmake: do not add gtest-parallel_ext if already added
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-05 19:11:51 +08:00
Kefu Chai
d359227ec0 cmake/modules/AddCephTest.cmake: make ${name}-venv a target
so we can prepare the venv for tox with `make tests`. and it's optional,
as run_tox.sh will check for it and setup a venv if it's not around.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-05 18:47:08 +08:00
Kefu Chai
bbb69fe793 cmake: require CMake v3.10.2
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-02 22:09:12 +08:00
Kefu Chai
95fd792be1
Merge pull request from wjwithagen/wjw-fix-boost-1.70
cmake: update FindBoost.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:49:34 +08:00
Kefu Chai
b9c9f74f06 cmake: add add_tox_test()
to consolidate the tox tests

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-01 21:33:30 +08:00
Willem Jan Withagen
fdc0e2baeb cmake: update FindBoost.cmake
FreeBSD is moving to Boost 1.70
we should allow user to use boost 1.70.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-07-30 16:00:04 +02:00
Kefu Chai
de9a04027f cmake/modules/Distutils: do not add ${name}-clone if already added
if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:

add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-26 12:11:25 +02:00
Kefu Chai
492efbd694 cmake: use python2 by default
and s/PYTHON_EXECUTABLE/Python_EXECUTABLE/

it's a regression introduced by 5e2bd7fc4d

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-22 16:07:56 +08:00
Kefu Chai
606b9c1822 cmake: pass PYTHON_VERSION explicitly
allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-21 00:47:39 +08:00
Kefu Chai
5e2bd7fc4d cmake: update FindPython* modules
use the ones shiped from the latest cmake. which

* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
  simpler this way and less error prone.

and to accomodate this change:

* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
  `Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
  `find_package(Python3...)` to find python2 or python3 instead of using
  `find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
  defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
  variables. because we find the python interpreter and development
  files using `find_pacakge(Python...)` for ceph-mgr.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-20 22:35:07 +08:00
Kefu Chai
7910bd6f02
Merge pull request from lixiaoy1/cmake_pmdk
cmake: pmem/pmdk changes to cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-12 14:55:34 +08:00
Kefu Chai
fcfc33b412
Merge pull request from tchaikov/wip-cmake-rebuild-cleanup
cmake: use BUILD_ALWAYS for rebuilding external project

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-12 14:54:30 +08:00
Kefu Chai
6958d2dcb1 cmake/modules/BuildRocksDB.cmake: use BUILD_ALWAYS
this option was introduced by cmake 3.1, and we require cmake 3.5.1, so
we can use it without using the homebrew solution.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-11 19:30:28 +08:00
lixiaoy1
4959835c07 cmake: Add the support to use system PMDK library
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2019-07-10 11:43:27 -04:00
lixiaoy1
4528aa2d0f cmake: move pmdk build script to Buildpmem.cmake
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
2019-07-10 11:43:27 -04:00
Kefu Chai
6453200265
Merge pull request from tchaikov/wip-cmake-pybind
cmake: avoid rebuilding extensions, and using python-config

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-09 16:05:44 +08:00
Kefu Chai
dadc66a58e
Merge pull request from tchaikov/wip-armhf-build
cmake, pybind: fix build on armhf

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-09 16:03:35 +08:00
Kefu Chai
1468c8fc35 cmake/modules/Distutils.cmake: add setup as a dependency
so the extension will be rebuilt if setup.py is updated

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:30:14 +08:00
Kefu Chai
f17861cdfa cmake/modules/Distutils.cmake: avoid rebuilding cython ext
use the combination of add_custom_command() and add_custom_target() to
build cython extensions to avoid rebuilding the cython extension
everytime we run "make"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-08 20:30:14 +08:00