Libzbc maintainers recommend switching to libzbd, which is lighter and supports
both ZNS SSDs and HM-SMR HDDs.
Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
* cmake/modules/FindSanitizers.cmake: do not pollute CMAKE_REQUIRED_FLAGS
* cmake/modules/FindSanitizers.cmake: expose Sanitizers_COMPILE_OPTIONS
as a list
* CMakeLists.txt: append Sanitizers_COMPILE_OPTIONS to
*_LINKER_FLAGS after replacing ";" with " " in it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Boost::iostreams links against zlib. and FindBoost.cmake
does not add this linkage to Boost::iostreams target, let's
do this after `find_package(Boost...)`. in theory, it'd be
better to have this change in FindBoost.cmake, but it's
error-prone, and increases the risk of regression when
we sync our own copy of FindBoost.cmake with CMake upstream.
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0xf7): undefined reference to `crc32'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x165): undefined reference to
`deflateReset'../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x185): undefined reference to `inflateEnd'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x1a1): undefined reference to
`inflateReset'../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x1b3): undefined reference to `deflateEnd'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x38d): undefined reference to `inflateInit2_'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x3d5): undefined reference to `deflateInit2_'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x134): undefined reference to `deflate'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x144): undefined reference to `inflate'
collect2: error: ld returned 1 exit status
src/CMakeFiles/ceph-common.dir/build.make:441: recipe for target 'bin/libceph-common.dll' failed
Signed-off-by: Kefu Chai <kchai@redhat.com>
For things like cephadm, where there is a lot of "from X import Y",
the import tags become cumbersome. .tox dirs and
python-common/build are just repeats of source files found elsewhere
so result in duplicate tags
Signed-off-by: Dan Mick <dmick@redhat.com>
WITH_PYTHON2 option was deprecated in
5fc657b40d, that commit was included by
Ceph v15.1.0 and up, assuming all downstream packagings have removed
WITH_PYTHON2 option, we are now removing it completely.
Signed-off-by: Kefu Chai <kchai@redhat.com>
OFED_PREFIX was added to compile with xio messenger for supporting RDMA
using the Accelio and/or MLNX_OFED package. but xio messenger was removed in
in cc9a9142fd
let remove the leftover in CMakeLists
Signed-off-by: Kefu Chai <kchai@redhat.com>
instead of appending compile flags to CMAKE_C_FLAGS, use
add_compile_options(), as COMPILE_OPTIONS is a list, it'd simpler to
append options to it and to access it in a structured way.
Signed-off-by: Kefu Chai <kchai@redhat.com>
so systemd/CMakeLists.txt can have access to the variables defined by
it. quote from https://cmake.org/cmake/help/latest/command/include.html.
> Variable reads and writes access the scope of the caller (dynamic
> scoping).
Signed-off-by: Kefu Chai <kchai@redhat.com>
Windows provides socket and other related structures that are mostly
following the Posix standards, but using different headers.
Rather than adding lots of platform checks and require future commits
to do so as well, we're adding headers with the same names, which
in turn include the required Windows headers.
In a few cases, some functions declared by unistd.h are actually
defined by different Windows headers, so we'll need additional includes.
One thing to note here is that boost requires us to include aio.hpp
before the Windows socket headers.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
an option named "mgr_disabled_modules" is added in this change to
prevent mgr from loading modules listed in this option. because mgr
loads *all* modules found in the configured path, and per
https://tracker.ceph.com/issues/45147, python subinterpreter could hang
when loading numpy, so this behavior practically creates a deadlock
in mgr.
this issue is found when mgr uses python3.8 runtime. in development
environment, it'd be inconvenient to disable the offending mgr module
without changing the source code, even if we can choose to not install
them, for instance, the enduser can workaround this issue by
uninstalling `ceph-mgr-diskprediction-local`.
an option would be useful in this case, so we can add the module to the
list before mgr tries to load it.
as this issue is found with python3.8 + diskprediction_local (numpy), so
this mgr module is disabled by default if mgr is compiled with python3.8
runtime.
Fixes: https://tracker.ceph.com/issues/45147
Signed-off-by: Kefu Chai <kchai@redhat.com>
1. WITH_LIBURING is used to set HAVE_LIBURING to decide
use liburing in KernelDevice or not.
2. WITH_SYSTEM_LIBURING is to choose use system installed
liburing or build the liburing from source code.
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
To build Boost.Context (and other libraries) with support to allow
them to be valground usefully, and to include the define to link
against them.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
We'll update the cmake files in order to be able to build ceph
components for Windows targets.
Cross compiling using MINGW is the easiest approach for now.
Subsequently, we'll add support for Clang and MSVC.
This patch provides the following changes:
* include winsock2.h, which provides ntohl on Windows
* avoid unsupported compiler flags when using msvc
* add a custom toolchain file for mingw
* update install command for ceph-common in order to work with mingw
* avoid running test sample when cross compiling
* link against the ws_32 lib
* set argeted Windows version
* skip yasm checks when targeting Windows
* allow multiple redefinitions when using mingw, picking the first one.
this is a workaround for a mingw TLS bug:
https://sourceforge.net/p/mingw-w64/bugs/816/
Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
python version for python binding and ceph-mgr embedded python
interpreter anymore. as before switching to python3-only build,
we can build python3 and python2 python bindings, and ceph-mgr
can use either of them. but after switching to python3-only
build, ceph-mgr has to use whatever python version used to
build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
parameter anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
There were a couple of problems found by flake8 in the qa/
directory (most of them fixed now). Enabling flake8 during the usual
check runs hopefully avoids adding new issues in the future.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
This implements low-level IO engine, which utilizes brand-new
io_uring IO interface: https://lwn.net/Articles/776428/
By default libaio is used. If bluestore_ioring=true is set but kernel
does not support io_uring or architecture is not x86-64, libaio will be
used instead.
In current patch liburing library is used in order not to open code
everything.
In order to compile with liburing WITH_LIBURING=ON should be specified.
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
error: ‘class rocksdb::Env’ has no member named ‘SetAllowNonOwnerAccess’
opt.env->SetAllowNonOwnerAccess(false);
^~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Ilsoo Byun <ilsoobyun@linecorp.com>
* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
as the name of this target should be hidden from the caller of this
function.
Signed-off-by: Kefu Chai <kchai@redhat.com>
on FreeBSD keyutil absence prevents building KRBD and mount.
But on FreeBSD icephfs, ceph-fuse and ceph-mds do work, so we need
ceph-mds to be build.
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
Because EVENTTRACE now implies LTTNG, also cleanup EVENTTRACE
conditions to assume LTTNG.
Also add missing eventtrace dependencies to rbd and
test/objectstore.
Signed-off-by: Samuel Just <sjust@redhat.com>
but we still enable it in `run-make-check.sh`
* cmake: disable SPDK by default
* run-make-check.sh: enable WITH_SPDK so at least we can ensure it
builds
* deb,rpm: add uuid-dev / libuuid-devel as a "make check" dependency
Fixes: https://tracker.ceph.com/issues/41330
Signed-off-by: Kefu Chai <kchai@redhat.com>
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.
- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5
so in this change,
* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
`VERSION` explicitly defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
`find_package(Python ${version}...)` tries to find the greater python version
which is greater than `${version}`, on fc30, at the time of writing, both
python3.8 and python3.7 are offered. but `python3-Cython` is packaged only
for python3.7. so if user installs python3.8, this will prevent user from
building Ceph. as Ceph will not be able to find Cython python module, as it
will try to run `python3.8 -m cython --version`, where python3.8 is the
greatest python version available in the system. but since cython module is
not available to python3.8, cmake will fail to find cython even if is available
to python3.7.
in this change, if user specifies a python version with minor version, we
will use the exact specified version instead of trying to use a version
greater than the specified one.
Signed-off-by: Kefu Chai <kchai@redhat.com>
use the ones shiped from the latest cmake. which
* enables us to find the recent python intepreter and development files,
* find intepreter and development in a single `find_pacakge()` command,
simpler this way and less error prone.
and to accomodate this change:
* all `PYTHON${PYTHON_VERSION}_*` variables are renamed to
`Python${PYTHON_VERSION}_*` if we use `find_package(Python2...)` or
`find_package(Python3...)` to find python2 or python3 instead of using
`find_package(Python...)`.
* use "2" explicitly when using python2, as `Python_*` variables are not
defined anymore
* when compiling python support of ceph-mgr, continue using `Python_*`
variables. because we find the python interpreter and development
files using `find_pacakge(Python...)` for ceph-mgr.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* rename genl to nl:
"genl" is not very specific without more context. and libnl is actually
a library suite. so it would be better if we can rename the library to
be found to "libnl", and by following the naming convention of CMake,
in this change, Findgenl.cmake is renamed to Findnl.cmake
* Findnl.cmake: support the component argument, in our specific case,
what we want is "libnl-genl" library, which is one of the libraries
offered by libnl. so let's just make it a component.
* Findnl.cmake: should pass the names of the required variables instead of
their values to `find_package_handle_standard_args()`. before this
change, we pass `GENL_LIBRARIES` to this function. it was correct at the
first glance. but it is not able to handle the case where case where
libnl-genl is not installed. so the fix is to pass all the names of
required library paths to this function. in this change, their name
are concatenated to a single variable -- `nl_LIBRARIES`, and the
value of this variable is passed to
`find_package_handle_standard_args()`. and the error message would
be more specific this way if libnl-genl is not found:
Could NOT find nl (missing: nl_genl_LIBRARY)
* Findnl.cmake: add nl::<component> as imported library, it helps the
consumer of these libraries to import them -- no need to
specify the `target_include_directories()` separately anymore.
* move the find_package() code to where it is used. it helps to improve
the readability.
* remove `HAVE_GENL` variable: it's not used anywhere.
* drop the messages of "Not using foobar", they do not help.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The next patch adds netlink support to rbd-nbd. This patch just adds the
build related changes to bring in the netlink lib and test and build
against it.
Signed-off-by: Mike Christie <mchristi@redhat.com>
The next patch will add a WITH_RBD check to the LINUX check block so we
only check for the netlink libs when those 2 conditions are met, so this
moves the WIT_RBD option setup to before the LINUX check.
Signed-off-by: Mike Christie <mchristi@redhat.com>
If tcmalloc is in-use as the allocator and its version is less than 2.6.2,
it might be missing support for 'aligned_alloc'. This can result in the
glibc version of 'aligned_alloc' being used to allocate memory that is
then freed by tcmalloc -- resulting in a crash.
Fixes: http://tracker.ceph.com/issues/39703
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
There is a lot of good stuff going on here, but nobody is investing in xio
and it is not expected to be the path forward for RDMA. If that ever
changes, we can resurrect the code. Until then, let's clean up the tree
and reduce friction for changes going forward.
Signed-off-by: Sage Weil <sage@redhat.com>
cmake,rgw: make amqp support optional
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
rabbitmq-c is not a header-only library. so we need to ensure that the
library is also around before linking against it. in this change:
* move the detection down to where librabbitmq is used
* add FindRabbitMQ.cmake for find_package(RabbitMQ)
* s/rabbitmq/RabbitMQ::RabbitMQ/ as the latter is an alias library
target, which has more information attached to it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
We can re-enable after we branch. For N, though, there is no point in dragging
in the dependencies and implying there is something that works.
Signed-off-by: Sage Weil <sage@redhat.com>
This patch introduces new FIO engine which main goal is to test
bare Ceph messenger transport layer performance. Engine submits
requests in async manner and polls for completion. Completions
are kept in lockless ring buffer so on hot path no muteces or
conditions are used in order to get maximum bandwidth and less
latency.
Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
It is rather had to overide a once set timeout during CTesting.
This is the best next compromise. Disadvantage is that a rebuild
is required.
But it helps in my Jenkins testing timely catching run-away programs.
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
We are currently hosting the grafana dashboards in our repo but we do
not install them. This patch adds the cmake support.
Signed-off-by: Boris Ranto <branto@redhat.com>
Ceph requires C++17 support from compiler. This means gcc 7.x being
minimal version supported.
This also allows to fail quick on Debian 'stretch' with it's gcc 6.3
compiler.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
* refs/pull/24020/head:
*: set missing CLOEXEC on opened fds
msg: set O_NONBLOCK on file status flags
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
If CEPH_DEBUG_MUTEX is defined, use the [recursive_]mutex_debug classes
that implement lockdep and a bucnh of other random debug checks. Also
typedef ceph::condition_variable to std::condition_variable_debug, which
adds addition assertions and debug checks.
If CEPH_DEBUG_MUTEX is not defined, then use the bare-bones C++ std::mutex
primitives... or as close as we can get to them.
Since the [recursive_]mutex_debug classes take a string argument for the
lockdep piece, define factory functions ceph::make_[recursive_]mutex that
either pass arguments to the debug implementations or toss them out.
Signed-off-by: Sage Weil <sage@redhat.com>
Otherwise these descriptors may leak across execve() during e.g. MDS respawn.
Fixes: http://tracker.ceph.com/issues/35850
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
