To build with DAOS backend, use -DWITH_RADOSGW_DAOS=YES cmake
option. `daos-devel` rpm should be installed beforehand.
To connect to DAOS pool, add the following configuration
parameters to ceph.conf:
```
[client]
...
rgw backend store = daos
daos pool = tank
```
A pool could be created using the following command:
```
dmg pool create --size=<size> <pool_name>
```
To install `daos-devel` do:
```
sudo wget -O /etc/yum.repos.d/daos-packages.repo https://packages.daos.io/v2.0/EL8/packages/x86_64/daos_packages.repo
sudo rpm --import https://packages.daos.io/RPM-GPG-KEY
sudo yum install -y epel-release daos-server daos-client daos-devel
```
Co-authored-by: Walter Warniaha <walter.warniaha@seagate.com>
Signed-off-by: Zuhair AlSader <zuhair.alsader@seagate.com>
In 4a3afcf, the $PATH is set for the test, but we cannot set multiple
properties with a single `set_property()` cmake command. We fix that by
adding the installation path of jsonnet-bundler
(CMAKE_CURRENT_BINARY_DIR) to the $PATH used for every tox test.
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
Co-Authored-By: Kefu Chai <tchaikov@gmail.com>
cmake should fail when generating the building system, if the necessary
development libraries or the header files are missing.
in this change, Findmotr.cmake is added to detect the motr library
and header so that we can use it to find the motr package before using
it.
See-also: https://tracker.ceph.com/issues/57079
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
we are facing following FTBFS on jammy + GCC-11.2 + Cython 0.29 +
CMake 3.22:
creating /home/jenkins-build/build/workspace/ceph-api/build/lib/cython_modules/temp.linux-x86_64-3.10/home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs
compile options: '-I/usr/include/python3.10 -I/usr/include/python3.10 -c'
extra options: '-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -iquote/home/jenkins-build/build/workspace/ceph-api/src/include -w -Dvoid0=dead_function(void) -D__Pyx_check_single_interpreter(ARG)=ARG ## 0 -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2'
cc: /home/jenkins-build/build/workspace/ceph-api/build/src/pybind/cephfs/cephfs.c
cc: warning: ##: linker input file unused because linking not done
cc: error: ##: linker input file not found: No such file or directory
cc: warning: 0: linker input file unused because linking not done
cc: error: 0: linker input file not found: No such file or directory
it seems cython is not able to escape the space in the "extra options"
anymore, so the "##" and "0" are considered as object files passed to
compiler in addition to cephfs.c.
in this change the spaces are removed to help cython to make the right
decision.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
In order to enable the pmem character device, add ndctl=y to the parameter
of compiling the pmdk library when WITH_BLUESTORE_PMEM is ON. Because
after find_ndctl and find_daxctl are added, the dependency packages
required by WITH_BLUESTORE_PMEM and WITH_RBD_RWL become more different.
So separated these two. libpmem has no version required. libpmemobj
required version >=1.8.
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
In order to support the character device of pmem usage in bluestore via
libpmem built by Ceph itself, we need to enable daxctl and ndctl
dependency. add the installation of ndctl and find it. the version of
ndctl and daxctl library requires >63. "apt-get install" meet the version
under ubuntu focal.
the installation of ndctl-devel in ceph.spec.in has not been verified.
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
At first, libpmem was the only library. Later, pmem related libraries
such as libpmemobj and libpmem2 were gradually added. These libraries
were also integrated into one named pmdk. So rename to pmdk.
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
We were using git fetch and an unofficial remote to fetch
opentelemetry-cpp, as it needed additional patch to include boost
headers, with the merge of
open-telemetry/opentelemetry-cpp#1100open-telemetry/opentelemetry-cpp#1020
we do not require to maintain our own patched version. Also, we were
using git fetch as a temporary change, now that tracing is always on, it
should be right time to adapt to using opentelemetry-cpp as a submodule.
This removes compile time fetch for opentelemetry-cpp instead use
official opentelemetry-cpp lib as a submodule.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
It's hitting pacific with a nuisance -Werror=array-parameter= const
char * vs const char[37] mismatch. Follow commit 91a616b26e ("cmake:
pass RTE_DEVEL_BUILD=n when building dpdk") and just disable -Werror.
Fixes: https://tracker.ceph.com/issues/55977
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
before this change, we assume that "ALLOCATOR" is the library name
which can be found by linker with "ld -l <namespec>". but ideally,
cmake can find a library in a more sophiscated way used by
its "find_library()" implementation. so, in this change, instead of
relying on the default paths looked up by "ld", use the path
found by cmake.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
This commit also explicilty sets the toolset when calling
boost's bootstrap.sh, because the latter is only capable
of autodetecting gcc if you have an unversioned `g++` in
the path, which may not be the case if only gcc11-c++ is
installed (the unversioned gcc-c++ package includes the
/usr/bin/g++ symlink, but depending on which version of
openSUSE we build on, that might point to a different gcc
version, so we don't want to use that).
Fixes: https://tracker.ceph.com/issues/55237
Signed-off-by: Tim Serong <tserong@suse.com>
* CMakeLists.txt:
always pass "EXACT" to find_package(Python3).
because per cmake document, "EXACT" only takes effect when
<Package>_FIND_VERSION_COUNT is greater than 1, where <Package>
is "Python3". see also cmake/modules/FindPython/Support.cmake
* cmake/modules/AddCephTest.cmake:
drop redundant find_package(Python3) calls. since Python3 is
a mandatory requirement for building Ceph, we only need a
single call of find_package(Python3..) in the top of the source
tree. the only possible case to repeat it is to ensure that we
have the correct version of Python3 used in following CMake
script. but there is no need to repeat it if we just want to
ensure that we have a python3 interpretor in place.
* cmake/modules/Distutils.cmake:
always pass "EXACT" to find_package(Python3).
we should always pass EXACT to find_package() when finding python3,
this is a follow-up of e2babdfae8
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
The purpose of this patch is to add the initial support to
offload memory/pmem operations by sync usage through hardware path
in DML library.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
if another python3 with higher version is found by
find_package(Python3), the cmake's install script would just
install the python modules/extensions into that python3's
dist-package directory, and the packaging script would fail
to find these artifacts when trying to package them.
so we need to ensure that the install directories for python
modeules/extensions are always "versioned" with WITH_PYTHON3
cmake option.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
to address following warning from python 3.9:
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
adds utf8proc submodule, needed by the arrow submodule in centos. add a
WITH_SYSTEM_UTF8PROC option that controls whether or not utf8proc is
built from submodule
non-system utf8proc is built as a static library to avoid conflicts with
system-provided libraries
ceph.spec.in sets WITH_SYSTEM_UTF8PROC=OFF until it's available in
centos
Signed-off-by: Casey Bodley <cbodley@redhat.com>
adds an arrow submodule. when WITH_RADOSGW_SELECT_PARQUET is enabled,
the submodule is built as an external project and rgw links against its
imported Arrow::Parquet target
Signed-off-by: Casey Bodley <cbodley@redhat.com>
we have following error when trying to compile the C++ source
file on mipsel:
/usr/bin/c++ -DHAVE_CXX11_ATOMIC -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -g1 -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++11 -fPIE -o CMakeFiles/cmTC_9f34f.dir/src.cxx.o -c /<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.cxx
/<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.cxx:15:44: error: template argument 1 is invalid
15 | bool atomic16(std::atomic<unsigned __int128> *ptr)
| ^
/<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.cxx:15:44: error: template argument 1 is invalid
/<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.cxx:15:44: error: template argument 1 is invalid
/<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.cxx:15:44: error: template argument 1 is invalid
/<<PKGBUILDDIR>>/obj-mipsel-linux-gnu/CMakeFiles/CMakeTmp/src.cxx:15:47: error: ‘ptr’ was not declared in this scope
15 | bool atomic16(std::atomic<unsigned __int128> *ptr)
| ^~~
the compiler was GCC 11.2.0.
in this change, instead of detecting the architecture, check for
the macro of `__SIZEOF_INT128__`, which is respected by GCC and Clang.
so it's more readable. also, instead of using the `unsigned __int128`
type provided as a GCC extension, use a struct to mimic the use case
of boost::lockfree library.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ceph is still using the Makefile based building system for building
DPDK. and DPDK enables -Werror if RTE_DEVEL_BUILD is 'y' which is
enabled by default when the dpdk is built from a git repo.
but newer GCC is more picky than the older versions, to prevent
the possible FTBFS when we switch to newer GCC for building old
branches whose dpdk submodule might be include the changes addressing
those warnings. let's just disable this option.
the only effect of this option is to add -Werror to CFLAGS. but
the building warnings from DPDK is not our focus when developing
Ceph in the most cases. so it should be fine.
see also
eac901ce29/doc/build-sdk-quick.txt (L18)
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
* since the minimum required cmake version is now 3.16, there is no
need to check for cmake version for using multiple URLs for
downloading external project.
* use string(JOIN ..) to compose URL option for downloading boost
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
because QAT driver with version v1.7.l.4.14.0 or higher cannot be dowmloaded
directly by URL, FindQatDrv.cmake can find the locally installed QAT package and libraries
Signed-off-by: Miaomiao Liu <miaomiao.liu@intel.com>
Signed-off-by: Hualong Feng <hualong.feng@intel.com>
We build spdk as static library, linking against them requires the
use of `-Wl,--whole-archive` as argument, otherwise we will have error
`nvme.c: nvme_probe_internal: *ERROR*: NVMe trtype 256 not available`.
This is due to the use of constructor functions in spdk to register
NVMe transports. So we need to do so to ensure we call all the
constructors.
Signed-off-by: Tongliang Deng <dengtongliang@sensetime.com>
* add find_package() support for detecting the existence of
linux/blkzoned.h before using it.
* link against Linux::ZNS for adding the compilation definition of
HAVE_ZNS instead of including it in the global config.h
* move the CMake option closer to where it is used for
better readability. as this option takes effect only if
crimson is compiled.
* make WITH_ZNS an option which depends on the value of
CMAKE_SYSTEM_NAME. it is hidden and off if CMAKE_SYSTEM_NAME
is not "Linux".
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
migrate from using opentracing-cpp to opentelemetry-cpp static as distributed tracing API
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
replaced by Opentelemetry project, remove building and linking of jaeger
libraries.
* remove externalProject building
* linking of jaegertracing dependencies to ceph project
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
we no longer need to package libjaeger, as we would be using
libopentelemetry static libraries(until we support shared).
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
we build and use targets needed for jaeger to work with opentelemetry
referring:
https://github.com/open-telemetry/opentelemetry-cpp/blob/main/examples/jaeger/CMakeLists.txt
right now we need a patch for using thrift from distro packages, hence
we are using GIT_FETCH based externalProject method, for development
purpose, which will be replace by more reliable method in future.
NOTE: phase 1, still needs to be tested.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
it's reported that a mips64el build host is able to pass the test of
CheckCxxAtomic without linking against libatomic, while librbd.so
fails to link due to failures like
/usr/bin/ld: ../../../lib/librbd.so.1.16.0: undefined reference to `__atomic_store_16'
/usr/bin/ld: ../../../lib/librbd.so.1.16.0: undefined reference to `__atomic_load_16'
/usr/bin/ld: ../../../lib/librbd.so.1.16.0: undefined reference to `__atomic_compare_exchange_16'
so we have to check the existence of __atomic_load_16 instruction on
mips architecture.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
now that python3.10 has two digits as its minor version, we should be
able to match it.
this change fixes the build with python3.10. without this change, we'd
have
error: wrong library name 'python310' in the --with-<library> option.
when building with python3.10
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
