Commit Graph

425 Commits

Author SHA1 Message Date
Kefu Chai
17ed4b4b32 cmake: zbd is REQUIRED if demanded
it was broken in b1e133468f

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-02 21:08:10 +08:00
Kefu Chai
c754297672 cmake: enable LIBURING bluestore backend by default
so users interested in io_uring can test it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-02 21:08:10 +08:00
Kefu Chai
d162c00c66
Merge pull request #37788 from tchaikov/wip-zbd
rpm,cmake: s/WITH_LIBZBD/WITH_ZBD/ and enable ZBD on demand

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
2020-10-30 11:17:56 +08:00
Kefu Chai
b1e133468f cmake: set HAVE_LIBZBD before creating "acconfig.h"
`acconfig.h` is generated using

configure_file(
  ${CMAKE_SOURCE_DIR}/src/include/config-h.in.cmake
  ${CMAKE_BINARY_DIR}/include/acconfig.h
)

in `config-h.in.cmake`, the cmake variable of `HAVE_LIBZBD` is checked.
so we need to ensure that this variable is visible from this
`configure_file()` statement.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-29 11:19:40 +08:00
Mahati Chamarthy
01400fedc3 librbd/cache: init functionality for SSD Cache
Adds build option and implements init functionality for SSD cache

Signed-off-by: Lisa Li <xiaoyan.li@intel.com>
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2020-10-28 11:16:41 +05:30
Adam C. Emerson
b132d68930 {rgw, cmake}: Remove WITH_BOOST_CONTEXT
As Boost.Context now supports s390, there is no longer any need for
the conditional compilation.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-23 11:21:53 -04:00
Adam C. Emerson
713efb66e8 cmake: Bump boost version to 1.72
This gives us s390 support for Boost.Context.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-22 21:39:50 -04:00
Lucian Petrut
5fb90bb796 common: Convert remote errors on win32
errno.h values aren't the same across platforms, for which reason
we'll have to convert remote errors.

We'll use the cmake "add_definitions" function instead of
"add_compile_options", which generates incorrect compiler params
while "add_compile_definitions" isn't available at all when
using CMake 3.10, which is by default available on Ubuntu 18.04.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-09-21 12:34:15 +00:00
Kefu Chai
23e3358e3d
Merge pull request #36955 from agayev/zbc-to-zbd
os/bluestore: Switch from libzbc library to libzbd library

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-08 21:24:11 +08:00
Kefu Chai
d536386306 cmake: add "WITH_ZBD" option
for enabling Zoned Block Device support using libzbd

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-06 12:35:05 -04:00
Abutalib Aghayev
bb488b06c9 os/bluestore: Switch from libzbc library to libzbd library
Libzbc maintainers recommend switching to libzbd, which is lighter and supports
both ZNS SSDs and HM-SMR HDDs.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
2020-09-06 12:35:05 -04:00
Kefu Chai
fc6f35a840 cmake: expose Sanitizers_COMPILE_OPTIONS as a list
* cmake/modules/FindSanitizers.cmake: do not pollute CMAKE_REQUIRED_FLAGS
* cmake/modules/FindSanitizers.cmake: expose Sanitizers_COMPILE_OPTIONS
  as a list
* CMakeLists.txt: append Sanitizers_COMPILE_OPTIONS to
  *_LINKER_FLAGS after replacing ";" with " " in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-31 16:11:33 +08:00
Kefu Chai
bf045f5400
Merge pull request #36696 from tchaikov/wip-cmake-boost-zlib
cmake: link against ZLIB if statically linked against Boost

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-08-22 12:49:15 +08:00
Kefu Chai
a95576a143
Merge pull request #36530 from tchaikov/wip-cmake-python2
cmake: drop WITH_PYTHON2 option

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-08-18 23:18:05 +08:00
Kefu Chai
58b04c5f21 cmake: link against ZLIB if statically linked agasint Boost
Boost::iostreams links against zlib. and FindBoost.cmake
does not add this linkage to Boost::iostreams target, let's
do this after `find_package(Boost...)`. in theory, it'd be
better to have this change in FindBoost.cmake, but it's
error-prone, and increases the risk of regression when
we sync our own copy of FindBoost.cmake with CMake upstream.

../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0xf7): undefined reference to `crc32'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x165): undefined reference to
`deflateReset'../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x185): undefined reference to `inflateEnd'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x1a1): undefined reference to
`inflateReset'../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x1b3): undefined reference to `deflateEnd'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x38d): undefined reference to `inflateInit2_'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x3d5): undefined reference to `deflateInit2_'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x134): undefined reference to `deflate'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x144): undefined reference to `inflate'
collect2: error: ld returned 1 exit status
src/CMakeFiles/ceph-common.dir/build.make:441: recipe for target 'bin/libceph-common.dll' failed

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-18 14:03:53 +08:00
Dan Mick
e7e950aa36 cmake: don't include tags for Python imports, .tox, build/ dirs
For things like cephadm, where there is a lot of "from X import Y",
the import tags become cumbersome.  .tox dirs and
python-common/build are just repeats of source files found elsewhere
so result in duplicate tags

Signed-off-by: Dan Mick <dmick@redhat.com>
2020-08-10 22:53:03 +00:00
Kefu Chai
583482829b cmake: drop WITH_PYTHON2 option
WITH_PYTHON2 option was deprecated in
5fc657b40d, that commit was included by
Ceph v15.1.0 and up, assuming all downstream packagings have removed
WITH_PYTHON2 option, we are now removing it completely.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-08 16:55:32 +08:00
Kefu Chai
22d042442a cmake: use add_compile_options for passing compile options
add_definitions() is for, well, adding definitions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-01 13:33:25 +08:00
Kefu Chai
62cde26417 kv: always enable rocksdb
RocksDB is included as a submodule of Ceph, there is no need to check
for its existence anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-01 13:29:53 +08:00
Kefu Chai
8a7c8015c8 cmake: do not check OFED_PREFIX anymore
OFED_PREFIX was added to compile with xio messenger for supporting RDMA
using the Accelio and/or MLNX_OFED package. but xio messenger was removed in
in cc9a9142fd

let remove the leftover in CMakeLists

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-01 13:23:18 +08:00
Kefu Chai
289d43f400
Merge pull request #35416 from yanghonggang/master
os/bluestore: add blkin traces to BlueStore

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-06-24 11:09:49 +08:00
Yang Honggang
da4105b5df os/bluestore: add blkin traces to BlueStore
Signed-off-by: Yang Honggang <yanghonggang@kuaishou.com>
2020-06-20 10:05:37 +08:00
Kefu Chai
3a0f015d28
Merge pull request #35631 from tchaikov/wip-cmake-cleanup
cmake: use add_compile_options() and add_definitions()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-06-18 23:06:06 +08:00
Kefu Chai
91bccedc32 cmake: use add_compile_options()
instead of appending compile flags to CMAKE_C_FLAGS, use
add_compile_options(), as COMPILE_OPTIONS is a list, it'd simpler to
append options to it and to access it in a structured way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-18 16:40:14 +08:00
Kefu Chai
e6a773bc7a cmake: move include(GNUInstallDirs) up
so systemd/CMakeLists.txt can have access to the variables defined by
it. quote from https://cmake.org/cmake/help/latest/command/include.html.

> Variable reads and writes access the scope of the caller (dynamic
> scoping).

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-18 06:33:15 +08:00
Lucian Petrut
889247d557 include/compat: Add win32 compatibility layer
Windows provides socket and other related structures that are mostly
following the Posix standards, but using different headers.

Rather than adding lots of platform checks and require future commits
to do so as well, we're adding headers with the same names, which
in turn include the required Windows headers.

In a few cases, some functions declared by unistd.h are actually
defined by different Windows headers, so we'll need additional includes.

One thing to note here is that boost requires us to include aio.hpp
before the Windows socket headers.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-06-04 15:52:16 +00:00
Abutalib Aghayev
ab865ca002 os/bluestore: Enable writing data to HM-SMR hard drives.
Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
2020-05-22 12:37:24 -04:00
Kefu Chai
47ce69aa57 cmake: remove cppcheck and iwyu targets
these targets are not used by any other targets, and are not integrated with CI.
so let's drop them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-13 20:34:27 +08:00
Kefu Chai
067adbf9a0 mgr: do not load disabled modules
an option named "mgr_disabled_modules" is added in this change to
prevent mgr from loading modules listed in this option. because mgr
loads *all* modules found in the configured path, and per
https://tracker.ceph.com/issues/45147, python subinterpreter could hang
when loading numpy, so this behavior practically creates a deadlock
in mgr.

this issue is found when mgr uses python3.8 runtime. in development
environment, it'd be inconvenient to disable the offending mgr module
without changing the source code, even if we can choose to not install
them, for instance, the enduser can workaround this issue by
uninstalling `ceph-mgr-diskprediction-local`.

an option would be useful in this case, so we can add the module to the
list before mgr tries to load it.

as this issue is found with python3.8 + diskprediction_local (numpy), so
this mgr module is disabled by default if mgr is compiled with python3.8
runtime.

Fixes: https://tracker.ceph.com/issues/45147
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-30 19:11:22 +08:00
Kefu Chai
b557467df0
Merge pull request #34535 from changchengx/refine_build_liburing
cmake: refactor liburing support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-18 11:39:32 +08:00
Changcheng Liu
36918d7f78 cmake: use cmake module to integrate liburing
1. WITH_LIBURING is used to set HAVE_LIBURING to decide
   use liburing in KernelDevice or not.
2. WITH_SYSTEM_LIBURING is to choose use system installed
   liburing or build the liburing from source code.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-04-17 23:22:56 +08:00
Kefu Chai
2b3adb5c5e cmake: rename Findfuse.cmake to FindFUSE.cmake
so its name is consistent with upstream name, and with the variables
exposed by FindFUSE.cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-17 14:10:55 +08:00
Sage Weil
6d09963ec5 CMakeLists: 16.0.0
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:09:41 -05:00
Jianpeng Ma
9d76123fdf cmake: remove duplicated code.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2020-04-08 11:08:02 +08:00
Sage Weil
895b856aee Merge remote-tracking branch 'gh/octopus' 2020-03-23 20:09:41 -05:00
Jenkins Build Slave User
dc6a0b5c3c 15.2.0 2020-03-23 17:47:45 +00:00
Adam C. Emerson
7f573bb334 build: Add WITH_BOOST_VALGRIND option
To build Boost.Context (and other libraries) with support to allow
them to be valground usefully, and to include the define to link
against them.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-20 00:51:49 -04:00
Jenkins Build Slave User
4536217610 15.1.1 2020-03-13 17:05:38 +00:00
Kefu Chai
f6389baff6 cmake: reduce the indent level
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-24 12:45:35 +08:00
Kefu Chai
a4e9566d23 cmake: use loop for applying new policies
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-24 12:32:54 +08:00
Lucian Petrut
d15481f1a5 cmake: [win32] Update cmake files
We'll update the cmake files in order to be able to build ceph
components for Windows targets.

Cross compiling using MINGW is the easiest approach for now.
Subsequently, we'll add support for Clang and MSVC.

This patch provides the following changes:
* include winsock2.h, which provides ntohl on Windows
* avoid unsupported compiler flags when using msvc
* add a custom toolchain file for mingw
* update install command for ceph-common in order to work with mingw
* avoid running test sample when cross compiling
* link against the ws_32 lib
* set argeted Windows version
* skip yasm checks when targeting Windows
* allow multiple redefinitions when using mingw, picking the first one.
  this is a workaround for a mingw TLS bug:
  https://sourceforge.net/p/mingw-w64/bugs/816/

Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-02-19 08:24:21 +00:00
Sebastian Wagner
615381659c cmake: Integrate Rook client generation
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-13 11:33:55 +01:00
Jenkins Build Slave User
49b0421165 15.1.0 2020-01-29 22:10:24 +00:00
Sage Weil
ef6ed98766 ceph_release: octopus rc 15.1.0
Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-13 14:44:05 -06:00
Kefu Chai
5fc657b40d cmake: drop WITH_PYTHON2 option
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
  python version for python binding and ceph-mgr embedded python
  interpreter anymore. as before switching to python3-only build,
  we can build python3 and python2 python bindings, and ceph-mgr
  can use either of them. but after switching to python3-only
  build, ceph-mgr has to use whatever python version used to
  build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
  and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
  parameter anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 10:58:58 +08:00
Thomas Bechtold
0127cd1e88 qa: Enable flake8 tox and fix failures
There were a couple of problems found by flake8 in the qa/
directory (most of them fixed now). Enabling flake8 during the usual
check runs hopefully avoids adding new issues in the future.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-12-12 10:21:01 +01:00
Roman Penyaev
2268be9ff9 os/bluestore: introduce new io_uring IO engine
This implements low-level IO engine, which utilizes brand-new
io_uring IO interface: https://lwn.net/Articles/776428/

By default libaio is used.  If bluestore_ioring=true is set but kernel
does not support io_uring or architecture is not x86-64, libaio will be
used instead.

In current patch liburing library is used in order not to open code
everything.

In order to compile with liburing WITH_LIBURING=ON should be specified.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
2019-12-10 17:22:15 +08:00
Jason Dillaman
1d866ef37d
Merge pull request #31279 from lixiaoy1/rwl-image-writeback-cache-seq1
rbd/cache: Replicated Write Log core codes part 1

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-12-02 09:16:10 -05:00
Sage Weil
0e0a3853c9 Merge PR #31579 into master
* refs/pull/31579/head:
	cmake: add cppcheck and iwyu static analysis targets

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-11-25 21:49:18 -06:00
lixiaoy1
242d391408 librbd: add the framework for RWL writeback cache
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2019-11-21 04:02:50 -05:00