Commit Graph

363 Commits

Author SHA1 Message Date
Kefu Chai
1f8e3bd5e7 cmake: check version of librdkafka
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-15 22:16:08 +08:00
Kefu Chai
1c0bc5dad4 cmake: silence messages when cppcheck/IWYU is not found
* remove messages spit when these tools are not found
  they are not necessary for building Ceph. and the messages in the cmake
  output are distracting.
* fix a typo

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-10 11:08:02 +08:00
Kefu Chai
f5b2ec5d8f cmake: do not use CMP0093 unless it is supported
CMake CMP0093 was introduced by CMake 3.15, but we only require CMake
v3.10.2. so use the old `Boost_VERSION_MACRO` variable by default,
unless specified otherwise.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-03 20:00:59 +08:00
Kefu Chai
dd0cdb3eb6 cmake: do not use CMP0074 unless it is supported
CMake CMP0074 was introduced by CMake 3.12, but we only require CMake
v3.10.2. so default to check `Boost_ROOT` unless specified otherwise.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-03 09:51:17 +08:00
Kefu Chai
c93b0881e6
Merge pull request #31317 from wjwithagen/wjw-update-FindBoost.cmake
cmake: update FindBoost.cmake for 1.71

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-02 16:46:00 +08:00
Kefu Chai
a56efc1610
Merge pull request #31841 from rosinL/wip-support-dpdk-numa
cmake: Add dpdk numa support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-02 16:45:11 +08:00
luo rixin
32f967b420 cmake: Add shared library as dpdk dependency
librte_eal.a refers to libpthread.so and libdl.so, and libceph-common.so
refers to librte_eal.a. Many test application link libceph-common.so
without linking pthread, it will cause link error.
Such as unittest_interval_set, target_link_libraries (unittest_interval_set
ceph-common)

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2019-11-28 20:18:09 +08:00
Chunsong Feng
4f3606f87b cmake: Add dpdk numa support
Find and link the numa library for DPDK.

Fixes: https://tracker.ceph.com/issues/42275

Signed-off-by: Hu Ye <yehu5@huawei.com>
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2019-11-28 20:15:39 +08:00
Sage Weil
cf676c73ab Merge PR #31781 into master
* refs/pull/31781/head:
	cmake/modules: look for GNU make first instead of BSD make
	cmake: support parallel build for rocksdb

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-25 21:49:31 -06:00
Sage Weil
0e0a3853c9 Merge PR #31579 into master
* refs/pull/31579/head:
	cmake: add cppcheck and iwyu static analysis targets

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-11-25 21:49:18 -06:00
Deepika Upadhyay
abf3dc0cf9 cmake/modules: look for GNU make first instead of BSD make
* BuildDPDK.cmake, BuildSPDK.cmake:
      * give priority to build with gmake
      * throw error if make not found

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2019-11-25 13:16:04 +05:30
Deepika Upadhyay
a930706aa6 cmake: support parallel build for rocksdb
* cmake/modules/BuildRocksDB.cmake: if "make" detected
use concurrent processing, by passing the jobserver to
sub-make using `$(MAKE)`

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2019-11-25 13:14:22 +05:30
Casey Bodley
8aa638b0fa
Merge pull request #30960 from yuvalif/wip-yuval-add-kafka-notif-endpoint
rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-11-21 14:42:23 -05:00
Willem Jan Withagen
9c0a948900 cmake: update FindBoost.cmake for 1.71
This comes from cmake v 3.15.3

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-11-20 17:13:13 +01:00
Yuval Lifshitz
c876a0d817 cmake: add cppcheck and iwyu static analysis targets
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-18 19:43:14 +02:00
Kefu Chai
d6998a3b95 cmake: detect librt for POSIX time functions
per clock_gettime(3),

> On POSIX systems on which these functions are available, the symbol
> _POSIX_TIMERS is defined in <unistd.h> to a value greater than 0.

and

> Most systems require the program be linked with the librt library to
> use these functions.

so, we should detect this symbol and link against librt for using these
functions.

in this change, librt is linked for checking the existence of
clock_gettime(), if it exists. RT_LIBRARY is defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-14 21:26:57 +08:00
Kefu Chai
4c42b4d49d
Merge pull request #31544 from ifed01/wip-ifed-gettid2
test/fio: bump to fio-3.15

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-13 18:58:36 +08:00
Yingxin Cheng
bf958abc0f crimson: build seastar dpdk from src/seastar/dpdk
src/spdk/dpdk and src/seastar/dpdk are both at their private branches
with project-specific modifications, so select proper dpdk source
directory according to flags WITH_SPDK and Seastar_DPDK.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-11-13 15:53:43 +08:00
Yingxin Cheng
ab84158fc0 crimson: link required dpdk components for seastar
see https://github.com/scylladb/seastar/blob/
809b3ef875e7edb7c56643812dc8347ff9a82cee/cmake/Finddpdk.cmake#L60-L85

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-11-13 15:53:31 +08:00
Igor Fedotov
fcfb0c9442 cmake: detect gettid() presense
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-11-12 12:42:14 +03:00
Igor Fedotov
710e5da85b test/fio: bump to fio 3.15
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-11-12 02:24:45 +03:00
Yuval Lifshitz
354d775e04 rgw/pubsub: add kafka notification endpoint
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-05 19:25:03 +02:00
Kefu Chai
a123c85256 cmake: enable Finddpdk to find DPDK to be built
find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 17:54:05 +08:00
Kefu Chai
aa25f6202f cmake: define dpdk_LIBRARIES properly
dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Kefu Chai
d82ea5d738 cmake: refactor BuildDPDK
* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
  as the name of this target should be hidden from the caller of this
  function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Sage Weil
da0473e251 Merge PR #30772 into master
* refs/pull/30772/head:
	cmake/modules/GetGitRevisionDescription: update to work with git-worktree

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-10-22 15:57:02 -05:00
Kefu Chai
a826efc52f cmake: detect linker support
check the linker support of `--version-script` and `--exclude-libs`
flags. and use them only if they are supported.

llvm-ld does not support `--version-script` at this moment. and it's
reported that old versions of llvm-ld do not support `--exclude-libs`
as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-08 21:53:03 +08:00
Sage Weil
165f09d7e7 cmake/modules/GetGitRevisionDescription: update to work with git-worktree
Found this here: https://phabricator.kde.org/D9172

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-07 14:26:07 -05:00
Patrick Donnelly
bbe3a33775
Merge PR #30552 into master
* refs/pull/30552/head:
	cmake: fix libtsan detection
	test: librados startup/shutdown racer test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-09-30 20:46:14 -07:00
Jeff Layton
09bd258a9f cmake: fix libtsan detection
This variable is case-sensitive.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-30 14:49:12 -04:00
Ulrich Weigand
d1b9d14324 cmake: Test for 16-byte atomic support on IBM Z
On IBM Z the Boost tagged pointer implementation cannot use
"pointer compression" as there are no unused bits in an address;
the whole 64-bit address space is available to user space code.

Instead, Boost uses 16-byte atomics.  This is always supported
on IBM Z, but depending on the particular compiler (version)
it may require linking against libatomic.  The existing checks
in CheckCxxAtomic.cmake do not catch this, however, as they only
test for (up to) 8-byte atomic support.

Fixed by adding a test for 16-byte atomic support on IBM Z.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-29 16:22:52 +02:00
Ulrich Weigand
0065f76d53 headers: Remove __le16/__le32/__le64
These types should no longer be used; use ceph_le16/32/64 instead.

Also removes the cmake check whether those types are provided by
kernel headers (likewise for the __be types, which already were
not used anywhere).

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-18 13:35:21 +02:00
Kefu Chai
01ffda8fff
Merge pull request #30362 from uweigand/fix-unaligned-check
cmake: Fix unaligned check on big-endian systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-14 09:46:41 +08:00
Kefu Chai
8e9400db9c
Merge pull request #29761 from wjwithagen/wjw-fix-venv-python-version
test/venv: do not hardwire to py2.7 for tox tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-13 16:59:27 +08:00
Ulrich Weigand
5a189bdd41 cmake: Fix unaligned check on big-endian systems
On big-endian systems, ntohl is a no-op, so "good" never does
any conversion.  Fix this by keeping the test constants in
big-endian (network) order and using ntohl to convert them
to native (host) byte order.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-12 15:46:06 +02:00
Kefu Chai
34b33d447f cmake: check for unaligned access
C and C++ allow us to cast a pointer with an alignment requirement to
one with a different alignment requirement. but if the destination
type's alignment is greater/stricter than the source type, the behavior
is undefined. on amd64, it's allowed. and on aarch64, GCC offers an
option named `-munaligned-access` to enable this behavior. so we cannot
take this as guaranteed.

in denc.h and messenger implementations, we just cast the pointers to
the type we want to access without memcpy(), which is, strictly
speaking, not quite standard compliant.

so, to be on the safe side, we detect the support of unaligned access at
compile time.

please note, this would fail the cross-compile. as the target platform
might not support unaligned access, and the produced executable does not
run on the build platform. so this test is skipped if the building
host's architecture is different than that of target host.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-11 12:47:26 +08:00
Kefu Chai
67bf182aa5 cmake: extract checks into CephChecks.cmake
also use case chosen in the macro definition as per suggested by cmake
document. see
https://cmake.org/cmake/help/latest/command/macro.html#invocation

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-11 12:47:26 +08:00
Willem Jan Withagen
c70e7af4a6 test/venv: Fix the python version of the test environment
Otherwise it will result in:
[ 59%] preparing venv for mgr-dashboard
New python executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python2.7
Also creating executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /usr/local/bin/python2.7

Even though the Cmake options contained:
  '-DWITH_PYTHON2=OFF' '-DWITH_PYTHON3=3' '-DMGR_PYTHON_VERSION=3'

Mostly due to the fact that
  src/tools/setup-virtualenv.sh
default to:
  PYTHON_BINARY="python2.7"

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-08-27 15:10:21 +02:00
Ernesto Puerta
1e07237d92
mgr/dashboard: clean-up tox and unit tests
Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.

Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)

Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst

Doc:
- Update HACKING.rst

Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.

Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-16 19:17:27 +02:00
Kefu Chai
793308f82d cmake: pass tox envs to run_tox.sh explicitly
* add two optional keyword parameters to `add_tox_test()`
* use the default `TOX_PATH` parameter instead passing it explicitly,
  as it is always `${CMAKE_CURRENT_SOURCE_DIR}`
* drop the code to guess the tox envs in `run_tox.sh`, and always pass
  them explicitly in CMake script using the `--tox-envs` argument.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-06 00:06:29 +08:00
Kefu Chai
d68a84b0fd cmake: do not add gtest-parallel_ext if already added
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-05 19:11:51 +08:00
Kefu Chai
d359227ec0 cmake/modules/AddCephTest.cmake: make ${name}-venv a target
so we can prepare the venv for tox with `make tests`. and it's optional,
as run_tox.sh will check for it and setup a venv if it's not around.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-05 18:47:08 +08:00
Kefu Chai
bbb69fe793 cmake: require CMake v3.10.2
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-02 22:09:12 +08:00
Kefu Chai
95fd792be1
Merge pull request #29396 from wjwithagen/wjw-fix-boost-1.70
cmake: update FindBoost.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:49:34 +08:00
Kefu Chai
b9c9f74f06 cmake: add add_tox_test()
to consolidate the tox tests

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-01 21:33:30 +08:00
Willem Jan Withagen
fdc0e2baeb cmake: update FindBoost.cmake
FreeBSD is moving to Boost 1.70
we should allow user to use boost 1.70.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-07-30 16:00:04 +02:00
Kefu Chai
de9a04027f cmake/modules/Distutils: do not add ${name}-clone if already added
if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:

add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-26 12:11:25 +02:00
Kefu Chai
492efbd694 cmake: use python2 by default
and s/PYTHON_EXECUTABLE/Python_EXECUTABLE/

it's a regression introduced by 5e2bd7fc4d

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-22 16:07:56 +08:00
Kefu Chai
606b9c1822 cmake: pass PYTHON_VERSION explicitly
allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-21 00:47:39 +08:00
Kefu Chai
5e2bd7fc4d cmake: update FindPython* modules
use the ones shiped from the latest cmake. which

* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
  simpler this way and less error prone.

and to accomodate this change:

* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
  `Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
  `find_package(Python3...)` to find python2 or python3 instead of using
  `find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
  defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
  variables. because we find the python interpreter and development
  files using `find_pacakge(Python...)` for ceph-mgr.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-20 22:35:07 +08:00