Commit Graph

464 Commits

Author SHA1 Message Date
Kefu Chai
65776d3112 cmake: pass unparsed args to add_ceph_test()
this allows us to pass extra argument to the script or executable to
further customize its behavior when testing.

also fix all the callers of add_ceph_unittest(), as there is no need to
repeat the executable, which is also used as the name of the test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-30 17:33:56 +08:00
Kefu Chai
1a422d4478 cmake: pass PARALLEL to add_ceph_unittest()
and parse it using cmake_parse_arguments(), more flexible this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-30 16:44:32 +08:00
Kefu Chai
6152076c22
Merge pull request #40336 from tchaikov/wip-cmake-fio
cmake: pass compile options by fio interface library

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-03-26 09:28:31 +08:00
Kefu Chai
47cec18613 cmake: use fixture for preparing venv
this change should allow us to decouple "ninja tests" from "ctest".
in other words, we can just run

ctest -R run-tox-python-common -V

without running "ninja tests" first. before this change

${name}-venv is added as a dependency of "tests" target.

after this change,

setup-venv-for-${name} is added as a test, which is in turn a test of
run-tox-${name}, so we can just

ctest -R run-tox-${name}

now for preparing the venv and then testing the tox test of ${name}.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-24 22:19:31 +08:00
Kefu Chai
c190fee53d cmake: pass compile options by fio interface library
for better readability and maintainability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-23 16:25:47 +08:00
Patrick Donnelly
f7fa3b50c1
libcephsqlite: rework architecture and backend
This is a complete rewrite on top of SimpleRADOSStriper as well as the
API. The VFS is now a loadable extension as well.

Fixes: https://tracker.ceph.com/issues/40609
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:55 -07:00
Patrick Donnelly
eb52c869d5
cmake: improve build inst for cephsqlite
Notably, find SQLite and build the test code.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:54 -07:00
Kefu Chai
33d2964ae2 cmake: adapt FindBoost.cmake to our needs
the vanilla FindBoost.cmake pulled from cmake has couple assumptions
which do not hold in our environment. so address them case by case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Kefu Chai
3aaedc8fdc cmake: add 1.75 to known versions
sync with
507710438d/Modules/FindBoost.cmake

for v1.75 support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Kefu Chai
cfa20c33a7 cmake: silence "You are in 'detached HEAD' state" warning
git warns us when the repo is in 'detached HEAD' state, like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in
this state without impacting any branches by switching back to a branch.
...
Turn off this advice by setting config variable advice.detachedHead to
false

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:22 +08:00
Kefu Chai
aea8187cb3 cmake: define BUILD_BYPRODUCTS for Boost
so if a cmake generator needs any one of them, it knowns how to build
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:22 +08:00
Kefu Chai
609644c645 cmake: do not specify CMAKE_GENERATOR for external projects
as per CMake document,

> Without this option, the same generator as the main build will be
> used.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:22 +08:00
Kefu Chai
c471d0081b cmake: use "" for no-op INSTALL_COMMAND
as suggested by CMake document, see
https://cmake.org/cmake/help/latest/module/ExternalProject.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:21 +08:00
Kefu Chai
0e83f0d333 cmake: define BUILD_BYPRODUCTS for spdk-ext
so if a cmake generator needs any one of them, it knowns how to build
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:21 +08:00
Kefu Chai
aeb22b47a6 cmake: link libpmemobj against libpmem
libpmemobj should link against libpmem, but, in CMake, imported library
does not allow PRIVATE linkage. so pmem::pmem is added to the list of
INTERFACE_LINK_LIBRARIES.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-06 14:37:24 +08:00
Kefu Chai
48e4cd190f cmake: support COMPONENTS param in Findpmem.cmake
add two components: pmem and pmemobj to this package. so we can find
them and link against them in a more intuitive way.

before this change the COMPONENTS parameter passed to

find_package(pmem ...)

is dropped on the floor and ignored.

after this change, it is checked and taken into consideration.

also, in this change, the exposed variables are renamed from

PMEM_* to pmem_*

to be consistent with the package name. it's encouraged to be consistent
with the package name when it comes to the INCLUDE_DIR and LIBRARIES
variable names.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-06 12:25:37 +08:00
Kefu Chai
b502f69ca2 cmake: use NUMA for package name in FindNUMA.cmake
to address the warning from cmake, like

CMake Warning (dev) at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
  The package name passed to `find_package_handle_standard_args` (numa) does
  not match the name of the calling package (NUMA).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-14 23:11:33 +08:00
Sheng Mao
59a5f333fa cmake: don't pass ccache argument to RocksDB build
RocksDB's cmake has a similar logic to turn on ccache as the main cmake.
Currently BuildRocksDB.cmake explicitly passes ccache to RocksDB cmake
arguments as CMAKE_CXX_COMPILER_LAUNCHER. This causes a compilation error
on ccache 4.0, especially in make -j mode.

ccache: error: Recursive invocation (the name of the ccache binary must
be "ccache")

This commit lets cmake from RocksDB decide RULE_LAUNCH_COMPILE by itself.

Signed-off-by: Sheng Mao <shngmao@gmail.com>
2021-02-09 20:17:53 -07:00
Jan Fajerski
2fa311a462 Merge PR #32776 into master
* refs/pull/32776/head:
	win32*.sh: fetch WNBD dependency
	win32*.sh: bump win32 boost dependency
	cmake: skip osd, Lua and neorados on Windows
	win32*.sh: improve lib handling
	win32*.sh: update openssl location
	win32*.sh: cleanup dependency build dir when targetting Windows
	win32*.sh: Strip binaries individually
	win32*.sh: Cleanup zip archive creation
	cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
	win32*.sh: Use nproc for CPU count when targeting win32
	test: Include compat.h to find uid_t/gid_t/gidlist when cross compiling
	win32*.sh: improve win32 dependency check
	win32*.sh: isolate network operations when building win32 dependencies
	cmake: drop Python dependency when targeting Windows
	win32*.sh: cleanup Windows build script
	win32*.sh: fix patch command
	win32*.sh: update Windows build scripts and readme

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-05 12:50:37 +01:00
Kefu Chai
d72c9205dc cmake: only display the result of checking nasm once
if nasm is able to emit AVX512 instructions, we can assume that it's
able to generate AVX2 instructions as well. so no need to print "Found
nasm" multiple times.

and by chaining mutual exclusive branches with "elseif", we can have
better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-04 01:34:46 +08:00
Kefu Chai
de78439775 cmake: drop duplicated variable
not_arch_x32 replicates ${_support_x64}. and a variable starts with
"not" is just difficult to parse. so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-04 01:34:46 +08:00
Kefu Chai
6cdf25b4ab cmake: install python scripts into /usr/sbin even if DESTDIR is empty
it's reported that on Alphine, "make install" installs
`ceph-volume-systemd` into `/usr/bin` instead of `/usr/sbin`, and this
breaks `systemd/ceph-volume@.service`. so in this change, we always
install this script into `/usr/sbin`, even if `DESTDIR` is not defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-21 01:26:46 +08:00
Josh Durgin
8a74a937dd
Merge pull request #31358 from ideepika/wip-jaegertracing-in-ceph
jaegertracing build/ops integration

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-15 16:29:26 -08:00
Deepika Upadhyay
4b69d3a184 jaeger, build/op: add jaeger and it's dependencies as submodules
* This commit introduces Jaegertracing library as package libjaeger,
  pickwhich would be consumed by other ceph pacakges such as ceph-common0

* adds the following dependencies, which would be build from source
  using ExternalProjectHelper.cmake +IncludeJaeger.cmake +
  Build<package>.cmake scripts:

  jaegertracing: v0.6.0 [added as a submodule]
  opentracing: v1.6.0 [added as a submodule]
  thrift: 0.13.0 [added as a submodule]
  yaml-cpp: 0.6.0
  json(optional)

* updates Boost to be installed instead of being build only, because
  jaegertracing them during their build process.

* ceph.spec.in: introduces a default enabled jaeger packaging option,
  which could be disabled using --without-jaeger flag during rpmbuild

* note: libjaeger package if enabled will be a dependency on ceph-common, ceph-mon, rgw_common and transitively will be a dependency for modules that have them as  a dependency.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 07:55:46 +00:00
Kefu Chai
3f1aa66ee2
Merge pull request #38501 from tchaikov/wip-cmake-crimson-test
test/crimson: do not link against crimson-{os,common}

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-12-11 15:52:22 +08:00
Kefu Chai
461ad210b3 cmake: exclude unit tests from the "all" target
no need to build unit tests when "make all", unless required explicitly,
like "make tests".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-10 18:26:39 +08:00
Kefu Chai
244784fb36 cmake: stop rebuilding rocksdb everytime
this change was originally introduced as a part of
418bfd7bb5, and latter migrated / changed
in the current form. but the idea is the same: to rebuild rocksdb even
if the stamp file shows that it has been built. there is no need to do
so, as we don't hack RocksDB as we used to. also,it is distracting to
check this log message when rebuilding the tree. so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-10 12:26:00 +08:00
Mike Latimer
f42e947a85 cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
Distributions such as SUSE and Ubuntu differ significantly in their mingw
environments. This adds an OS=(ubuntu|suse) flag, which can be used to
specify which environment is being used for the build. Unless set explicitly,
the scripts will try to automatically detect it.

Depending on the OS selected, various mingw configuration options (binaries,
library paths, etc.) as well as required packages are determined.

Due to these options being configured at runtime, corresponding cmake
files are generated on the fly.

Signed-off-by: Mike Latimer <mlatimer@suse.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Signed-off-by: Alessandro Pilotti <apilotti@cloudbasesolutions.com>
2020-12-09 18:01:02 +00:00
Brad Hubbard
6e20e5ed5a cmake: Alter GIT_REPOSITORY for liburing
https://git.kernel.dk/liburing appears, at least currently, to be
unavailable. This could be reverted once it comes back.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-12-03 11:56:43 +10:00
Kefu Chai
a92a27a5b0
Merge pull request #38320 from tchaikov/wip-cmake-std-fs
cmake: do not always require extra link flags for std::filesystem

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-02 22:03:26 +08:00
Kefu Chai
6cf7ebd0ca cmake: do not always require extra link flags for std::filesystem
since GCC-9

> Using the types and functions in <filesystem> does not require linking
> with -lstdc++fs now.

see https://gcc.gnu.org/gcc-9/changes.html#libstdcxx

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-02 19:42:34 +08:00
Kefu Chai
3429c2b586
Merge pull request #38380 from lixiaoy1/pkg_fix
cmake: fix unknown pkg_search_module

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-02 15:38:31 +08:00
lixiaoy1
e8fd8a31b2 cmake: fix unknown pkg_search_module
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2020-12-02 06:18:52 -05:00
Brad Hubbard
e0240b9af4 cmake: Stop 'configure' running every time
Currently the configure script is run on 'liburing_ext' not only for the
initial build but for subsequent incremental builds.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-11-24 13:38:37 +10:00
Or Ozeri
ea3c1bfb9e librbd: add LUKS support
This commit introduces internal (not yet part of the api) librbd functions for:
1. formating an RBD image in LUKS format
2. parsing an RBD image in LUKS format

 The actual implementation of the LUKS format is done via libcryptsetup, which is added as a new dependency.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2020-11-23 10:32:14 -05:00
Kefu Chai
504da5d5dd cmake: set advice.detachedHead=false when cloning liburing
silence "detached HEAD" warning like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-18 17:53:43 +08:00
Kefu Chai
b6a94da614 cmake: add 1.74 to known versions
sync with
685fa8bec0/Modules/FindBoost.cmake

for boost 1.74 support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-16 15:36:09 +08:00
Yanfei Xu
8f0a3fa1ff cmake: add support for python3.9
add support for python3.9.

Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com>
2020-11-11 10:43:37 +08:00
Kefu Chai
dd073a0061
Merge pull request #37984 from tchaikov/wip-fio
cmake: drop WITH_SYSTEM_FIO option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-11-09 21:08:13 +08:00
Kefu Chai
3510433f9c cmake: build liburing with -fPIC
in liburing,
75cad68b95
partially reverts
4e360f7113,
which builds liburing.a with -fPIC.

so we need to pass -fPIC by ourselves. otherwise we'd have

/usr/bin/ld: ../../liburing/src/liburing.a(setup.ol): relocation R_X86_64_PC32 against symbol `io_uring_queue_mmap' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/build.make:154: recipe for target 'lib/libfio_ceph_objectstore.so' failed

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 17:10:59 +08:00
Kefu Chai
c976f142f0 cmake: expose FIO_INCLUDE_DIR via libfio
always use the fio built from source for better consistency.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 16:01:59 +08:00
Kefu Chai
d16f7fb524 cmake: drop WITH_SYSTEM_FIO option
there is no well-known distro packaging fio.h, so drop this option.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 16:01:59 +08:00
Kefu Chai
b6bae7ba13 cmake: use make explicitly to build fio
we cannot assume that user uses "make" as the generator of cmake, if,
for instance, ninja is used, `$(MAKE)` is not a valid variable in the
generated `build.ninja`. so we should use "make" explicitly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 15:49:26 +08:00
Kefu Chai
68eb5bcdce cmake: use src/liburing for building liburing if it exists
so we can build with liburing enabled if the dist tarball contains
liburing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-09 13:11:35 +08:00
Kefu Chai
3f2a844da4 cmake: set GIT_SHALLOW and UPDATE_DISCONNECTED
* GIT_SHALLOW=TRUE, so we don't pull the full git history,
  as we don't care about it.
* UPDATE_DISCONNECTED=TRUE, to skip the UPDATE step, this change
  somehow works around
  https://gitlab.kitware.com/cmake/cmake/-/issues/19703. otherwise
  cmake keeps building liburing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-06 01:20:25 +08:00
Kefu Chai
eb81ab4d1c blk/kernel/io_uring: bump liburing to v0.7
* use functions exposed by liburing instead of using syscalls
* v0.7 is the latest release at the time of writing, as liburing is under
  active development. it'd be better to use a newer release.
* also use https://git.kernel.dk/liburing instead of
  http://git.kernel.dk/liburing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-06 01:20:06 +08:00
Changcheng Liu
497b4d6f72 cmake: remove yasm since it's been replaced by nasm
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 12:36:20 +08:00
Changcheng Liu
bf5794b2f0 cmake: define cmake macro to check nasm assembler
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 12:34:38 +08:00
Changcheng Liu
a62f8d8ea0 cmake: check whether yasm support AVX512
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 08:57:54 +08:00
Changcheng Liu
c98a925053 common: rename variable name to improve readability
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 08:57:54 +08:00