Commit Graph

439 Commits

Author SHA1 Message Date
Jan Fajerski
2fa311a462 Merge PR #32776 into master
* refs/pull/32776/head:
	win32*.sh: fetch WNBD dependency
	win32*.sh: bump win32 boost dependency
	cmake: skip osd, Lua and neorados on Windows
	win32*.sh: improve lib handling
	win32*.sh: update openssl location
	win32*.sh: cleanup dependency build dir when targetting Windows
	win32*.sh: Strip binaries individually
	win32*.sh: Cleanup zip archive creation
	cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
	win32*.sh: Use nproc for CPU count when targeting win32
	test: Include compat.h to find uid_t/gid_t/gidlist when cross compiling
	win32*.sh: improve win32 dependency check
	win32*.sh: isolate network operations when building win32 dependencies
	cmake: drop Python dependency when targeting Windows
	win32*.sh: cleanup Windows build script
	win32*.sh: fix patch command
	win32*.sh: update Windows build scripts and readme

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-05 12:50:37 +01:00
Yuval Lifshitz
c1af849a0d
Merge pull request #37924 from yuvalif/allow_packages_in_lua
rgw/lua: support packages via luarocks
2020-12-17 19:53:00 +02:00
Josh Durgin
8a74a937dd
Merge pull request #31358 from ideepika/wip-jaegertracing-in-ceph
jaegertracing build/ops integration

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-15 16:29:26 -08:00
Deepika Upadhyay
85ca7b36a0 jaeger: make tracing default turned off, until teuthology integration
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 10:08:09 +00:00
Deepika Upadhyay
4b69d3a184 jaeger, build/op: add jaeger and it's dependencies as submodules
* This commit introduces Jaegertracing library as package libjaeger,
  pickwhich would be consumed by other ceph pacakges such as ceph-common0

* adds the following dependencies, which would be build from source
  using ExternalProjectHelper.cmake +IncludeJaeger.cmake +
  Build<package>.cmake scripts:

  jaegertracing: v0.6.0 [added as a submodule]
  opentracing: v1.6.0 [added as a submodule]
  thrift: 0.13.0 [added as a submodule]
  yaml-cpp: 0.6.0
  json(optional)

* updates Boost to be installed instead of being build only, because
  jaegertracing them during their build process.

* ceph.spec.in: introduces a default enabled jaeger packaging option,
  which could be disabled using --without-jaeger flag during rpmbuild

* note: libjaeger package if enabled will be a dependency on ceph-common, ceph-mon, rgw_common and transitively will be a dependency for modules that have them as  a dependency.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 07:55:46 +00:00
Lucian Petrut
a149c0bebd cmake: drop Python dependency when targeting Windows
The Python bindings aren't supported on Windows for the time being,
for which reason we're dropping the Python dependency.

When cross-compiling, Cmake sometimes complains about not finding
the Python files instead of just using the host files.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-12-09 18:01:02 +00:00
Yuval Lifshitz
6ef3e52a3f rgw/lua: support packages via luarocks
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2020-12-04 09:55:56 +02:00
Kefu Chai
48b3ef5c70 cmake: pass REQUIRED to find_package(StdFilesystem)
otherwise there is chance that we could link to
StdFilesystem::filesystem which is not defined anywhere.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-02 19:42:34 +08:00
Kefu Chai
6c4dbb2e48
Merge pull request #38263 from tchaikov/wip-boost-1.74-asio
src,cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT for Boost.Asio users and cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-26 18:18:22 +08:00
Kefu Chai
3d70821909 cmake: define BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT for Boost.Asio users
see also
https://www.boost.org/doc/libs/1_74_0/doc/html/boost_asio/std_executors.html#boost_asio.std_executors.polymorphic_i_o_executor

we could use `asio::any_io_executor` later on though for better
performance.

also, define CMP0093, so FindBoost reports Boost_VERSION in x.y.z
format. it is simpler to use `VERSION_GREATER_EQUAL` to compare its
version with 1.74 instead of its C macro version ("107000").

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-24 15:50:19 +08:00
Or Ozeri
ea3c1bfb9e librbd: add LUKS support
This commit introduces internal (not yet part of the api) librbd functions for:
1. formating an RBD image in LUKS format
2. parsing an RBD image in LUKS format

 The actual implementation of the LUKS format is done via libcryptsetup, which is added as a new dependency.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2020-11-23 10:32:14 -05:00
Kefu Chai
8eeb7da944 cmake: enable CMP0074
to search prefixes specified by the <PackageName>_ROOT CMake
variable and the <PackageName>_ROOT environment variable. find_package()
in CMake 3.12 started to use these prefixes if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-16 15:36:08 +08:00
Kefu Chai
c976f142f0 cmake: expose FIO_INCLUDE_DIR via libfio
always use the fio built from source for better consistency.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 16:01:59 +08:00
Kefu Chai
d16f7fb524 cmake: drop WITH_SYSTEM_FIO option
there is no well-known distro packaging fio.h, so drop this option.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 16:01:59 +08:00
Kefu Chai
17ed4b4b32 cmake: zbd is REQUIRED if demanded
it was broken in b1e133468f

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-02 21:08:10 +08:00
Kefu Chai
c754297672 cmake: enable LIBURING bluestore backend by default
so users interested in io_uring can test it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-02 21:08:10 +08:00
Kefu Chai
d162c00c66
Merge pull request #37788 from tchaikov/wip-zbd
rpm,cmake: s/WITH_LIBZBD/WITH_ZBD/ and enable ZBD on demand

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
2020-10-30 11:17:56 +08:00
Kefu Chai
b1e133468f cmake: set HAVE_LIBZBD before creating "acconfig.h"
`acconfig.h` is generated using

configure_file(
  ${CMAKE_SOURCE_DIR}/src/include/config-h.in.cmake
  ${CMAKE_BINARY_DIR}/include/acconfig.h
)

in `config-h.in.cmake`, the cmake variable of `HAVE_LIBZBD` is checked.
so we need to ensure that this variable is visible from this
`configure_file()` statement.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-29 11:19:40 +08:00
Mahati Chamarthy
01400fedc3 librbd/cache: init functionality for SSD Cache
Adds build option and implements init functionality for SSD cache

Signed-off-by: Lisa Li <xiaoyan.li@intel.com>
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2020-10-28 11:16:41 +05:30
Adam C. Emerson
b132d68930 {rgw, cmake}: Remove WITH_BOOST_CONTEXT
As Boost.Context now supports s390, there is no longer any need for
the conditional compilation.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-23 11:21:53 -04:00
Adam C. Emerson
713efb66e8 cmake: Bump boost version to 1.72
This gives us s390 support for Boost.Context.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-22 21:39:50 -04:00
Lucian Petrut
5fb90bb796 common: Convert remote errors on win32
errno.h values aren't the same across platforms, for which reason
we'll have to convert remote errors.

We'll use the cmake "add_definitions" function instead of
"add_compile_options", which generates incorrect compiler params
while "add_compile_definitions" isn't available at all when
using CMake 3.10, which is by default available on Ubuntu 18.04.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-09-21 12:34:15 +00:00
Kefu Chai
23e3358e3d
Merge pull request #36955 from agayev/zbc-to-zbd
os/bluestore: Switch from libzbc library to libzbd library

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-08 21:24:11 +08:00
Kefu Chai
d536386306 cmake: add "WITH_ZBD" option
for enabling Zoned Block Device support using libzbd

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-06 12:35:05 -04:00
Abutalib Aghayev
bb488b06c9 os/bluestore: Switch from libzbc library to libzbd library
Libzbc maintainers recommend switching to libzbd, which is lighter and supports
both ZNS SSDs and HM-SMR HDDs.

Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
2020-09-06 12:35:05 -04:00
Kefu Chai
fc6f35a840 cmake: expose Sanitizers_COMPILE_OPTIONS as a list
* cmake/modules/FindSanitizers.cmake: do not pollute CMAKE_REQUIRED_FLAGS
* cmake/modules/FindSanitizers.cmake: expose Sanitizers_COMPILE_OPTIONS
  as a list
* CMakeLists.txt: append Sanitizers_COMPILE_OPTIONS to
  *_LINKER_FLAGS after replacing ";" with " " in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-31 16:11:33 +08:00
Kefu Chai
bf045f5400
Merge pull request #36696 from tchaikov/wip-cmake-boost-zlib
cmake: link against ZLIB if statically linked against Boost

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-08-22 12:49:15 +08:00
Kefu Chai
a95576a143
Merge pull request #36530 from tchaikov/wip-cmake-python2
cmake: drop WITH_PYTHON2 option

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-08-18 23:18:05 +08:00
Kefu Chai
58b04c5f21 cmake: link against ZLIB if statically linked agasint Boost
Boost::iostreams links against zlib. and FindBoost.cmake
does not add this linkage to Boost::iostreams target, let's
do this after `find_package(Boost...)`. in theory, it'd be
better to have this change in FindBoost.cmake, but it's
error-prone, and increases the risk of regression when
we sync our own copy of FindBoost.cmake with CMake upstream.

../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0xf7): undefined reference to `crc32'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x165): undefined reference to
`deflateReset'../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x185): undefined reference to `inflateEnd'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x1a1): undefined reference to
`inflateReset'../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x1b3): undefined reference to `deflateEnd'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x38d): undefined reference to `inflateInit2_'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x3d5): undefined reference to `deflateInit2_'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x134): undefined reference to `deflate'
../../build.deps/mingw/boost/lib/libboost_iostreams.a(zlib.o):zlib.cpp:(.text+0x144): undefined reference to `inflate'
collect2: error: ld returned 1 exit status
src/CMakeFiles/ceph-common.dir/build.make:441: recipe for target 'bin/libceph-common.dll' failed

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-18 14:03:53 +08:00
Dan Mick
e7e950aa36 cmake: don't include tags for Python imports, .tox, build/ dirs
For things like cephadm, where there is a lot of "from X import Y",
the import tags become cumbersome.  .tox dirs and
python-common/build are just repeats of source files found elsewhere
so result in duplicate tags

Signed-off-by: Dan Mick <dmick@redhat.com>
2020-08-10 22:53:03 +00:00
Kefu Chai
583482829b cmake: drop WITH_PYTHON2 option
WITH_PYTHON2 option was deprecated in
5fc657b40d, that commit was included by
Ceph v15.1.0 and up, assuming all downstream packagings have removed
WITH_PYTHON2 option, we are now removing it completely.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-08 16:55:32 +08:00
Kefu Chai
22d042442a cmake: use add_compile_options for passing compile options
add_definitions() is for, well, adding definitions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-01 13:33:25 +08:00
Kefu Chai
62cde26417 kv: always enable rocksdb
RocksDB is included as a submodule of Ceph, there is no need to check
for its existence anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-01 13:29:53 +08:00
Kefu Chai
8a7c8015c8 cmake: do not check OFED_PREFIX anymore
OFED_PREFIX was added to compile with xio messenger for supporting RDMA
using the Accelio and/or MLNX_OFED package. but xio messenger was removed in
in cc9a9142fd

let remove the leftover in CMakeLists

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-01 13:23:18 +08:00
Kefu Chai
289d43f400
Merge pull request #35416 from yanghonggang/master
os/bluestore: add blkin traces to BlueStore

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-06-24 11:09:49 +08:00
Yang Honggang
da4105b5df os/bluestore: add blkin traces to BlueStore
Signed-off-by: Yang Honggang <yanghonggang@kuaishou.com>
2020-06-20 10:05:37 +08:00
Kefu Chai
3a0f015d28
Merge pull request #35631 from tchaikov/wip-cmake-cleanup
cmake: use add_compile_options() and add_definitions()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-06-18 23:06:06 +08:00
Kefu Chai
91bccedc32 cmake: use add_compile_options()
instead of appending compile flags to CMAKE_C_FLAGS, use
add_compile_options(), as COMPILE_OPTIONS is a list, it'd simpler to
append options to it and to access it in a structured way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-18 16:40:14 +08:00
Kefu Chai
e6a773bc7a cmake: move include(GNUInstallDirs) up
so systemd/CMakeLists.txt can have access to the variables defined by
it. quote from https://cmake.org/cmake/help/latest/command/include.html.

> Variable reads and writes access the scope of the caller (dynamic
> scoping).

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-18 06:33:15 +08:00
Lucian Petrut
889247d557 include/compat: Add win32 compatibility layer
Windows provides socket and other related structures that are mostly
following the Posix standards, but using different headers.

Rather than adding lots of platform checks and require future commits
to do so as well, we're adding headers with the same names, which
in turn include the required Windows headers.

In a few cases, some functions declared by unistd.h are actually
defined by different Windows headers, so we'll need additional includes.

One thing to note here is that boost requires us to include aio.hpp
before the Windows socket headers.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2020-06-04 15:52:16 +00:00
Abutalib Aghayev
ab865ca002 os/bluestore: Enable writing data to HM-SMR hard drives.
Signed-off-by: Abutalib Aghayev <agayev@cs.cmu.edu>
2020-05-22 12:37:24 -04:00
Kefu Chai
47ce69aa57 cmake: remove cppcheck and iwyu targets
these targets are not used by any other targets, and are not integrated with CI.
so let's drop them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-13 20:34:27 +08:00
Kefu Chai
067adbf9a0 mgr: do not load disabled modules
an option named "mgr_disabled_modules" is added in this change to
prevent mgr from loading modules listed in this option. because mgr
loads *all* modules found in the configured path, and per
https://tracker.ceph.com/issues/45147, python subinterpreter could hang
when loading numpy, so this behavior practically creates a deadlock
in mgr.

this issue is found when mgr uses python3.8 runtime. in development
environment, it'd be inconvenient to disable the offending mgr module
without changing the source code, even if we can choose to not install
them, for instance, the enduser can workaround this issue by
uninstalling `ceph-mgr-diskprediction-local`.

an option would be useful in this case, so we can add the module to the
list before mgr tries to load it.

as this issue is found with python3.8 + diskprediction_local (numpy), so
this mgr module is disabled by default if mgr is compiled with python3.8
runtime.

Fixes: https://tracker.ceph.com/issues/45147
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-30 19:11:22 +08:00
Kefu Chai
b557467df0
Merge pull request #34535 from changchengx/refine_build_liburing
cmake: refactor liburing support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-18 11:39:32 +08:00
Changcheng Liu
36918d7f78 cmake: use cmake module to integrate liburing
1. WITH_LIBURING is used to set HAVE_LIBURING to decide
   use liburing in KernelDevice or not.
2. WITH_SYSTEM_LIBURING is to choose use system installed
   liburing or build the liburing from source code.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-04-17 23:22:56 +08:00
Kefu Chai
2b3adb5c5e cmake: rename Findfuse.cmake to FindFUSE.cmake
so its name is consistent with upstream name, and with the variables
exposed by FindFUSE.cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-17 14:10:55 +08:00
Sage Weil
6d09963ec5 CMakeLists: 16.0.0
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:09:41 -05:00
Jianpeng Ma
9d76123fdf cmake: remove duplicated code.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2020-04-08 11:08:02 +08:00
Sage Weil
895b856aee Merge remote-tracking branch 'gh/octopus' 2020-03-23 20:09:41 -05:00
Jenkins Build Slave User
dc6a0b5c3c 15.2.0 2020-03-23 17:47:45 +00:00