Commit Graph

473 Commits

Author SHA1 Message Date
Kefu Chai
8c44876be4 cmake: s/HAVE_MSGHDR/WITH_SYSTEMD/
as FreeBSD also has msghdr but it does not have systemd, or
flags like MFD_ALLOW_SEALING, O_TMPFILE or F_SEAL_GROW. so
use WITH_SYSTEMD for enabling journald backend of logging system.

also move the option of "WITH_SYSTEMD" up so that the src/CMakeLists.txt
is able to see the variable of WITH_SYSTEMD defined by it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-08 16:58:50 +08:00
Kefu Chai
c6e8fc5933
Merge pull request #40487 from tchaikov/wip-boost-1.75
install-deps.sh: install boost 1.75

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-07 15:10:34 +08:00
Kefu Chai
80c4a1cd37 common: disable journald logging backend if struct msghdr is not found
* cmake/modules/CephChecks.cmake: detect the existence of struct msghdr,
  define HAVE_MSGHDR if it is found
* src/common/CMakeLists.txt: do not compile journald.cc if HAVE_MSGHDR
  is FALSE. as in that case, we cannot use sendmsg() to write to
  journald unix domain socket
* src/test/CMakeLists.txt, src/test/common/CMakeLists.txt: disable test
  exercising journald logging backend if HAVE_MSGHDR is not defined
* src/common/Journald.h: define a dummy JournaldLogger and a dummy
  JournaldClusterLogger when HAVE_MSGHDR is not defined, in order to
  minimize the change in Log.h and Log.cc, otherwise the source code of
  Log.h and Log.cc would be segmented into smaller chunks by
  `ifdef HAVE_MSGHDR` macros.
* src/include/config-h.in.cmake: define a new macro named
  HAVE_MSGHDR.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-06 14:46:33 +08:00
Kefu Chai
37a015a9ed cmake: dedup components when finding pmem
otherwise we always find libpmem twice if libpmem is explicitly
specified when find_package()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 17:33:10 +08:00
Kefu Chai
89121189ac cmake,install-deps,make-dist: use boost 1.75
so we can use the latest boost

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 15:09:29 +08:00
Kefu Chai
7e1ef0c0c9 cmake: require libpmem 1.7
detect the libpmem libraries' version using pkg-config

please note, the version is not identical to the ones defined in
libpmem.h or libpmemobj/base.h

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 23:59:32 +08:00
Kefu Chai
6dfc80737d cmake: consolidate the find logic in Findpmem.cmake
less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 23:58:53 +08:00
Kefu Chai
bf56cddce5 cmake: use .pc for hints for finding pmem libraries
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:00:17 +08:00
Kefu Chai
16fd07244d cmake: allow use libzstd in system
since we are moving the test nodes from bionic to focal, we are able to
use the prebuilt libzstd libraries when running "make check". to speed
up the build and test, in this change:

* add FindZstd.cmake which allows us to use the libzstd in system
* extract BuildZstd.cmake for better readability
* add an option named "WITH_SYSTEM_ZSTD", which defaults to "OFF",
  so user can enable it on demand.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-31 12:53:39 +08:00
Kefu Chai
65776d3112 cmake: pass unparsed args to add_ceph_test()
this allows us to pass extra argument to the script or executable to
further customize its behavior when testing.

also fix all the callers of add_ceph_unittest(), as there is no need to
repeat the executable, which is also used as the name of the test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-30 17:33:56 +08:00
Kefu Chai
1a422d4478 cmake: pass PARALLEL to add_ceph_unittest()
and parse it using cmake_parse_arguments(), more flexible this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-30 16:44:32 +08:00
Kefu Chai
6152076c22
Merge pull request #40336 from tchaikov/wip-cmake-fio
cmake: pass compile options by fio interface library

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-03-26 09:28:31 +08:00
Kefu Chai
47cec18613 cmake: use fixture for preparing venv
this change should allow us to decouple "ninja tests" from "ctest".
in other words, we can just run

ctest -R run-tox-python-common -V

without running "ninja tests" first. before this change

${name}-venv is added as a dependency of "tests" target.

after this change,

setup-venv-for-${name} is added as a test, which is in turn a test of
run-tox-${name}, so we can just

ctest -R run-tox-${name}

now for preparing the venv and then testing the tox test of ${name}.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-24 22:19:31 +08:00
Kefu Chai
c190fee53d cmake: pass compile options by fio interface library
for better readability and maintainability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-23 16:25:47 +08:00
Patrick Donnelly
f7fa3b50c1
libcephsqlite: rework architecture and backend
This is a complete rewrite on top of SimpleRADOSStriper as well as the
API. The VFS is now a loadable extension as well.

Fixes: https://tracker.ceph.com/issues/40609
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:55 -07:00
Patrick Donnelly
eb52c869d5
cmake: improve build inst for cephsqlite
Notably, find SQLite and build the test code.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:54 -07:00
Kefu Chai
33d2964ae2 cmake: adapt FindBoost.cmake to our needs
the vanilla FindBoost.cmake pulled from cmake has couple assumptions
which do not hold in our environment. so address them case by case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Kefu Chai
3aaedc8fdc cmake: add 1.75 to known versions
sync with
507710438d/Modules/FindBoost.cmake

for v1.75 support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Kefu Chai
cfa20c33a7 cmake: silence "You are in 'detached HEAD' state" warning
git warns us when the repo is in 'detached HEAD' state, like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in
this state without impacting any branches by switching back to a branch.
...
Turn off this advice by setting config variable advice.detachedHead to
false

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:22 +08:00
Kefu Chai
aea8187cb3 cmake: define BUILD_BYPRODUCTS for Boost
so if a cmake generator needs any one of them, it knowns how to build
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:22 +08:00
Kefu Chai
609644c645 cmake: do not specify CMAKE_GENERATOR for external projects
as per CMake document,

> Without this option, the same generator as the main build will be
> used.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:22 +08:00
Kefu Chai
c471d0081b cmake: use "" for no-op INSTALL_COMMAND
as suggested by CMake document, see
https://cmake.org/cmake/help/latest/module/ExternalProject.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:21 +08:00
Kefu Chai
0e83f0d333 cmake: define BUILD_BYPRODUCTS for spdk-ext
so if a cmake generator needs any one of them, it knowns how to build
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:21 +08:00
Kefu Chai
aeb22b47a6 cmake: link libpmemobj against libpmem
libpmemobj should link against libpmem, but, in CMake, imported library
does not allow PRIVATE linkage. so pmem::pmem is added to the list of
INTERFACE_LINK_LIBRARIES.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-06 14:37:24 +08:00
Kefu Chai
48e4cd190f cmake: support COMPONENTS param in Findpmem.cmake
add two components: pmem and pmemobj to this package. so we can find
them and link against them in a more intuitive way.

before this change the COMPONENTS parameter passed to

find_package(pmem ...)

is dropped on the floor and ignored.

after this change, it is checked and taken into consideration.

also, in this change, the exposed variables are renamed from

PMEM_* to pmem_*

to be consistent with the package name. it's encouraged to be consistent
with the package name when it comes to the INCLUDE_DIR and LIBRARIES
variable names.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-06 12:25:37 +08:00
Kefu Chai
b502f69ca2 cmake: use NUMA for package name in FindNUMA.cmake
to address the warning from cmake, like

CMake Warning (dev) at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
  The package name passed to `find_package_handle_standard_args` (numa) does
  not match the name of the calling package (NUMA).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-14 23:11:33 +08:00
Sheng Mao
59a5f333fa cmake: don't pass ccache argument to RocksDB build
RocksDB's cmake has a similar logic to turn on ccache as the main cmake.
Currently BuildRocksDB.cmake explicitly passes ccache to RocksDB cmake
arguments as CMAKE_CXX_COMPILER_LAUNCHER. This causes a compilation error
on ccache 4.0, especially in make -j mode.

ccache: error: Recursive invocation (the name of the ccache binary must
be "ccache")

This commit lets cmake from RocksDB decide RULE_LAUNCH_COMPILE by itself.

Signed-off-by: Sheng Mao <shngmao@gmail.com>
2021-02-09 20:17:53 -07:00
Jan Fajerski
2fa311a462 Merge PR #32776 into master
* refs/pull/32776/head:
	win32*.sh: fetch WNBD dependency
	win32*.sh: bump win32 boost dependency
	cmake: skip osd, Lua and neorados on Windows
	win32*.sh: improve lib handling
	win32*.sh: update openssl location
	win32*.sh: cleanup dependency build dir when targetting Windows
	win32*.sh: Strip binaries individually
	win32*.sh: Cleanup zip archive creation
	cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
	win32*.sh: Use nproc for CPU count when targeting win32
	test: Include compat.h to find uid_t/gid_t/gidlist when cross compiling
	win32*.sh: improve win32 dependency check
	win32*.sh: isolate network operations when building win32 dependencies
	cmake: drop Python dependency when targeting Windows
	win32*.sh: cleanup Windows build script
	win32*.sh: fix patch command
	win32*.sh: update Windows build scripts and readme

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-05 12:50:37 +01:00
Kefu Chai
d72c9205dc cmake: only display the result of checking nasm once
if nasm is able to emit AVX512 instructions, we can assume that it's
able to generate AVX2 instructions as well. so no need to print "Found
nasm" multiple times.

and by chaining mutual exclusive branches with "elseif", we can have
better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-04 01:34:46 +08:00
Kefu Chai
de78439775 cmake: drop duplicated variable
not_arch_x32 replicates ${_support_x64}. and a variable starts with
"not" is just difficult to parse. so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-04 01:34:46 +08:00
Kefu Chai
6cdf25b4ab cmake: install python scripts into /usr/sbin even if DESTDIR is empty
it's reported that on Alphine, "make install" installs
`ceph-volume-systemd` into `/usr/bin` instead of `/usr/sbin`, and this
breaks `systemd/ceph-volume@.service`. so in this change, we always
install this script into `/usr/sbin`, even if `DESTDIR` is not defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-21 01:26:46 +08:00
Josh Durgin
8a74a937dd
Merge pull request #31358 from ideepika/wip-jaegertracing-in-ceph
jaegertracing build/ops integration

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-15 16:29:26 -08:00
Deepika Upadhyay
4b69d3a184 jaeger, build/op: add jaeger and it's dependencies as submodules
* This commit introduces Jaegertracing library as package libjaeger,
  pickwhich would be consumed by other ceph pacakges such as ceph-common0

* adds the following dependencies, which would be build from source
  using ExternalProjectHelper.cmake +IncludeJaeger.cmake +
  Build<package>.cmake scripts:

  jaegertracing: v0.6.0 [added as a submodule]
  opentracing: v1.6.0 [added as a submodule]
  thrift: 0.13.0 [added as a submodule]
  yaml-cpp: 0.6.0
  json(optional)

* updates Boost to be installed instead of being build only, because
  jaegertracing them during their build process.

* ceph.spec.in: introduces a default enabled jaeger packaging option,
  which could be disabled using --without-jaeger flag during rpmbuild

* note: libjaeger package if enabled will be a dependency on ceph-common, ceph-mon, rgw_common and transitively will be a dependency for modules that have them as  a dependency.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 07:55:46 +00:00
Kefu Chai
3f1aa66ee2
Merge pull request #38501 from tchaikov/wip-cmake-crimson-test
test/crimson: do not link against crimson-{os,common}

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-12-11 15:52:22 +08:00
Kefu Chai
461ad210b3 cmake: exclude unit tests from the "all" target
no need to build unit tests when "make all", unless required explicitly,
like "make tests".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-10 18:26:39 +08:00
Kefu Chai
244784fb36 cmake: stop rebuilding rocksdb everytime
this change was originally introduced as a part of
418bfd7bb5, and latter migrated / changed
in the current form. but the idea is the same: to rebuild rocksdb even
if the stamp file shows that it has been built. there is no need to do
so, as we don't hack RocksDB as we used to. also,it is distracting to
check this log message when rebuilding the tree. so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-10 12:26:00 +08:00
Mike Latimer
f42e947a85 cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
Distributions such as SUSE and Ubuntu differ significantly in their mingw
environments. This adds an OS=(ubuntu|suse) flag, which can be used to
specify which environment is being used for the build. Unless set explicitly,
the scripts will try to automatically detect it.

Depending on the OS selected, various mingw configuration options (binaries,
library paths, etc.) as well as required packages are determined.

Due to these options being configured at runtime, corresponding cmake
files are generated on the fly.

Signed-off-by: Mike Latimer <mlatimer@suse.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Signed-off-by: Alessandro Pilotti <apilotti@cloudbasesolutions.com>
2020-12-09 18:01:02 +00:00
Brad Hubbard
6e20e5ed5a cmake: Alter GIT_REPOSITORY for liburing
https://git.kernel.dk/liburing appears, at least currently, to be
unavailable. This could be reverted once it comes back.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-12-03 11:56:43 +10:00
Kefu Chai
a92a27a5b0
Merge pull request #38320 from tchaikov/wip-cmake-std-fs
cmake: do not always require extra link flags for std::filesystem

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-02 22:03:26 +08:00
Kefu Chai
6cf7ebd0ca cmake: do not always require extra link flags for std::filesystem
since GCC-9

> Using the types and functions in <filesystem> does not require linking
> with -lstdc++fs now.

see https://gcc.gnu.org/gcc-9/changes.html#libstdcxx

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-02 19:42:34 +08:00
Kefu Chai
3429c2b586
Merge pull request #38380 from lixiaoy1/pkg_fix
cmake: fix unknown pkg_search_module

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-02 15:38:31 +08:00
lixiaoy1
e8fd8a31b2 cmake: fix unknown pkg_search_module
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2020-12-02 06:18:52 -05:00
Brad Hubbard
e0240b9af4 cmake: Stop 'configure' running every time
Currently the configure script is run on 'liburing_ext' not only for the
initial build but for subsequent incremental builds.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-11-24 13:38:37 +10:00
Or Ozeri
ea3c1bfb9e librbd: add LUKS support
This commit introduces internal (not yet part of the api) librbd functions for:
1. formating an RBD image in LUKS format
2. parsing an RBD image in LUKS format

 The actual implementation of the LUKS format is done via libcryptsetup, which is added as a new dependency.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2020-11-23 10:32:14 -05:00
Kefu Chai
504da5d5dd cmake: set advice.detachedHead=false when cloning liburing
silence "detached HEAD" warning like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-18 17:53:43 +08:00
Kefu Chai
b6a94da614 cmake: add 1.74 to known versions
sync with
685fa8bec0/Modules/FindBoost.cmake

for boost 1.74 support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-16 15:36:09 +08:00
Yanfei Xu
8f0a3fa1ff cmake: add support for python3.9
add support for python3.9.

Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com>
2020-11-11 10:43:37 +08:00
Kefu Chai
dd073a0061
Merge pull request #37984 from tchaikov/wip-fio
cmake: drop WITH_SYSTEM_FIO option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-11-09 21:08:13 +08:00
Kefu Chai
3510433f9c cmake: build liburing with -fPIC
in liburing,
75cad68b95
partially reverts
4e360f7113,
which builds liburing.a with -fPIC.

so we need to pass -fPIC by ourselves. otherwise we'd have

/usr/bin/ld: ../../liburing/src/liburing.a(setup.ol): relocation R_X86_64_PC32 against symbol `io_uring_queue_mmap' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/build.make:154: recipe for target 'lib/libfio_ceph_objectstore.so' failed

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 17:10:59 +08:00
Kefu Chai
c976f142f0 cmake: expose FIO_INCLUDE_DIR via libfio
always use the fio built from source for better consistency.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 16:01:59 +08:00