instead of appending compile flags to CMAKE_C_FLAGS, use
add_compile_options(), as COMPILE_OPTIONS is a list, it'd simpler to
append options to it and to access it in a structured way.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/34719/head:
ceph-fuse: compatible with libfuse3.5 or higher
cmake: to get the header and library from specified path
libfuse: check the libfuse version from the pkconfig/fuse{3}.pc file
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Since the pkg_search_module will help us set the header and lib
path, which we should get the headers and lib from. To check from
other paths will make no sense.
Fixes: https://tracker.ceph.com/issues/45396
Signed-off-by: Xiubo Li <xiubli@redhat.com>
Since libfuse 3.2 to 3.8 the minor version for FUSE library has
stopped updating together with the releases. So we cannot check
version by using the FUSE_VERSION macro in the fuse_common.h header
file directly.
This will check the major/minor version from the fuse{3}.pc pkconfig
file.
Fixes: https://tracker.ceph.com/issues/45396
Signed-off-by: Xiubo Li <xiubli@redhat.com>
1. WITH_LIBURING is used to set HAVE_LIBURING to decide
use liburing in KernelDevice or not.
2. WITH_SYSTEM_LIBURING is to choose use system installed
liburing or build the liburing from source code.
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
in 0437adc33a, we stop right before
reaching $top_srcdir, but we should stop at its parent directory.
in this change, instead of trying to be smart and to walk all the way
up to the root directory or $top_srcdir, we just check $top_srcdir/.git
directly, as we just know it's there or it does not exist at all.
Signed-off-by: Kefu Chai <kchai@redhat.com>
make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we cannot assume that we are using `make` as the cmake generatator,
for instance, if ninja is used, `$(MAKE)` won't be substituted by ninja.
so we need to check if Make is used as generator, if that's the case, we
can just use `$(MAKE)` so we can benefit from the job control of `make`,
otherwise, `make` is used, because currently, PMDK uses Makefile to
build.
Signed-off-by: Kefu Chai <kchai@redhat.com>
it's a shorthand for finding "make" or "gmake" (for FreeBSD), and set
the path to the executable and the command to use in the generated
"Makefile" or whatever build script generated by cmake.
Signed-off-by: Kefu Chai <kchai@redhat.com>
To build Boost.Context (and other libraries) with support to allow
them to be valground usefully, and to include the define to link
against them.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
In order to be able to cross compile rocksdb using mingw, we'll
have to do the following:
* pass on specified toolchain files
* pass the CMAKE_PREFIX_PATH. We'll have to replace the ";" separator
with something else (we went with "!"), otherwise CMake won't be
able to properly handle the command arguments
* pass on the received LZ4 lib location since it cannot be properly
detected when cross compiling.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
We'll update the cmake files in order to be able to build ceph
components for Windows targets.
Cross compiling using MINGW is the easiest approach for now.
Subsequently, we'll add support for Clang and MSVC.
This patch provides the following changes:
* include winsock2.h, which provides ntohl on Windows
* avoid unsupported compiler flags when using msvc
* add a custom toolchain file for mingw
* update install command for ceph-common in order to work with mingw
* avoid running test sample when cross compiling
* link against the ws_32 lib
* set argeted Windows version
* skip yasm checks when targeting Windows
* allow multiple redefinitions when using mingw, picking the first one.
this is a workaround for a mingw TLS bug:
https://sourceforge.net/p/mingw-w64/bugs/816/
Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Commit d1b9d14324 added a check for
16-byte atomics without library support on IBM Z. Unfortunately
it turns out this test only works correctly when the test case
is built without optimization. (This is normally true, but it
may not be the case when passing explicit CXXFLAGS to cmake.)
The underlying reason is that GCC may choose to use either an
inline implementation of the 16-byte atomics or library calls,
depending on whether or not it is able to prove the atomic
variable is properly aligned. At -O0 it is never able to prove
that, but at higher optimization levels it depends on the complexity
of the expression (in particular, whether GCC can track down the
definition of the underlying object).
As the test case uses a very simple expression, it may happen that
this test can be built without requiring library support, but some
of the "real" uses of atomics in Ceph code cannot. This defeats
the whole purpose of the test at configure time.
Fixed by making the access pattern in the test more complex, so
that the test fails even at high optimization levels.
Fixes: https://tracker.ceph.com/issues/43747
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
BuildBoost.cmake (used when we're building the submodule) doesn't
provide parity with FindBoost.cmake (used with system Boost).
Specifically, it doesn't set the _FOUND variables for the various
components, making it hard to depend on finding those features.
Set Boost_<component>_FOUND for all the components we're building in
BuildBoost.cmake to make using these variables possible.
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
python version for python binding and ceph-mgr embedded python
interpreter anymore. as before switching to python3-only build,
we can build python3 and python2 python bindings, and ceph-mgr
can use either of them. but after switching to python3-only
build, ceph-mgr has to use whatever python version used to
build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
parameter anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
so we can use exported targets like "Python3::Python", which is defined
only if CMAKE_ROLE is "PROJECT". but this global property was introduced
by CMake 3.14, but the minimum required version of CMake is 3.10.2 in
Ceph project at this moment. let's set it if it's not defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* remove messages spit when these tools are not found
they are not necessary for building Ceph. and the messages in the cmake
output are distracting.
* fix a typo
Signed-off-by: Kefu Chai <kchai@redhat.com>
CMake CMP0093 was introduced by CMake 3.15, but we only require CMake
v3.10.2. so use the old `Boost_VERSION_MACRO` variable by default,
unless specified otherwise.
Signed-off-by: Kefu Chai <kchai@redhat.com>
CMake CMP0074 was introduced by CMake 3.12, but we only require CMake
v3.10.2. so default to check `Boost_ROOT` unless specified otherwise.
Signed-off-by: Kefu Chai <kchai@redhat.com>
librte_eal.a refers to libpthread.so and libdl.so, and libceph-common.so
refers to librte_eal.a. Many test application link libceph-common.so
without linking pthread, it will cause link error.
Such as unittest_interval_set, target_link_libraries (unittest_interval_set
ceph-common)
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Find and link the numa library for DPDK.
Fixes: https://tracker.ceph.com/issues/42275
Signed-off-by: Hu Ye <yehu5@huawei.com>
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
* refs/pull/31781/head:
cmake/modules: look for GNU make first instead of BSD make
cmake: support parallel build for rocksdb
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
* BuildDPDK.cmake, BuildSPDK.cmake:
* give priority to build with gmake
* throw error if make not found
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* cmake/modules/BuildRocksDB.cmake: if "make" detected
use concurrent processing, by passing the jobserver to
sub-make using `$(MAKE)`
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
per clock_gettime(3),
> On POSIX systems on which these functions are available, the symbol
> _POSIX_TIMERS is defined in <unistd.h> to a value greater than 0.
and
> Most systems require the program be linked with the librt library to
> use these functions.
so, we should detect this symbol and link against librt for using these
functions.
in this change, librt is linked for checking the existence of
clock_gettime(), if it exists. RT_LIBRARY is defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
src/spdk/dpdk and src/seastar/dpdk are both at their private branches
with project-specific modifications, so select proper dpdk source
directory according to flags WITH_SPDK and Seastar_DPDK.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.
Signed-off-by: Kefu Chai <kchai@redhat.com>
dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
as the name of this target should be hidden from the caller of this
function.
Signed-off-by: Kefu Chai <kchai@redhat.com>
check the linker support of `--version-script` and `--exclude-libs`
flags. and use them only if they are supported.
llvm-ld does not support `--version-script` at this moment. and it's
reported that old versions of llvm-ld do not support `--exclude-libs`
as well.
Signed-off-by: Kefu Chai <kchai@redhat.com>
On IBM Z the Boost tagged pointer implementation cannot use
"pointer compression" as there are no unused bits in an address;
the whole 64-bit address space is available to user space code.
Instead, Boost uses 16-byte atomics. This is always supported
on IBM Z, but depending on the particular compiler (version)
it may require linking against libatomic. The existing checks
in CheckCxxAtomic.cmake do not catch this, however, as they only
test for (up to) 8-byte atomic support.
Fixed by adding a test for 16-byte atomic support on IBM Z.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
These types should no longer be used; use ceph_le16/32/64 instead.
Also removes the cmake check whether those types are provided by
kernel headers (likewise for the __be types, which already were
not used anywhere).
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
On big-endian systems, ntohl is a no-op, so "good" never does
any conversion. Fix this by keeping the test constants in
big-endian (network) order and using ntohl to convert them
to native (host) byte order.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
C and C++ allow us to cast a pointer with an alignment requirement to
one with a different alignment requirement. but if the destination
type's alignment is greater/stricter than the source type, the behavior
is undefined. on amd64, it's allowed. and on aarch64, GCC offers an
option named `-munaligned-access` to enable this behavior. so we cannot
take this as guaranteed.
in denc.h and messenger implementations, we just cast the pointers to
the type we want to access without memcpy(), which is, strictly
speaking, not quite standard compliant.
so, to be on the safe side, we detect the support of unaligned access at
compile time.
please note, this would fail the cross-compile. as the target platform
might not support unaligned access, and the produced executable does not
run on the build platform. so this test is skipped if the building
host's architecture is different than that of target host.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Otherwise it will result in:
[ 59%] preparing venv for mgr-dashboard
New python executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python2.7
Also creating executable in /home/jenkins/workspace/ceph-master/build/mgr-dashboard-virtualenv/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /usr/local/bin/python2.7
Even though the Cmake options contained:
'-DWITH_PYTHON2=OFF' '-DWITH_PYTHON3=3' '-DMGR_PYTHON_VERSION=3'
Mostly due to the fact that
src/tools/setup-virtualenv.sh
default to:
PYTHON_BINARY="python2.7"
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.
Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)
Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst
Doc:
- Update HACKING.rst
Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.
Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
* add two optional keyword parameters to `add_tox_test()`
* use the default `TOX_PATH` parameter instead passing it explicitly,
as it is always `${CMAKE_CURRENT_SOURCE_DIR}`
* drop the code to guess the tox envs in `run_tox.sh`, and always pass
them explicitly in CMake script using the `--tox-envs` argument.
Signed-off-by: Kefu Chai <kchai@redhat.com>
so we can prepare the venv for tox with `make tests`. and it's optional,
as run_tox.sh will check for it and setup a venv if it's not around.
Signed-off-by: Kefu Chai <kchai@redhat.com>
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.
- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5
so in this change,
* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
`VERSION` explicitly defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:
add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.
Signed-off-by: Kefu Chai <kchai@redhat.com>
allow `distutils_*()` functions to choose the python according to this
parameter, instead of implicitly using the variable of ${PYTHON_VERSION}
inherited from the caller.
Signed-off-by: Kefu Chai <kchai@redhat.com>
use the ones shiped from the latest cmake. which
* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
simpler this way and less error prone.
and to accomodate this change:
* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
`Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
`find_package(Python3...)` to find python2 or python3 instead of using
`find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
variables. because we find the python interpreter and development
files using `find_pacakge(Python...)` for ceph-mgr.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this option was introduced by cmake 3.1, and we require cmake 3.5.1, so
we can use it without using the homebrew solution.
Signed-off-by: Kefu Chai <kchai@redhat.com>
use the combination of add_custom_command() and add_custom_target() to
build cython extensions to avoid rebuilding the cython extension
everytime we run "make"
Signed-off-by: Kefu Chai <kchai@redhat.com>
so add_custom_target() has better idea what is generated. if any target
will use it, it can be used as a dependency.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The issue has been fixed upstream under lockfree commit
7e23dac52d08ed1a099de9a6fb8bcdefbb06d2da but is not yet available in
a boost release.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
