Ceph requires C++17 support from compiler. This means gcc 7.x being
minimal version supported.
This also allows to fail quick on Debian 'stretch' with it's gcc 6.3
compiler.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
* refs/pull/24020/head:
*: set missing CLOEXEC on opened fds
msg: set O_NONBLOCK on file status flags
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
If CEPH_DEBUG_MUTEX is defined, use the [recursive_]mutex_debug classes
that implement lockdep and a bucnh of other random debug checks. Also
typedef ceph::condition_variable to std::condition_variable_debug, which
adds addition assertions and debug checks.
If CEPH_DEBUG_MUTEX is not defined, then use the bare-bones C++ std::mutex
primitives... or as close as we can get to them.
Since the [recursive_]mutex_debug classes take a string argument for the
lockdep piece, define factory functions ceph::make_[recursive_]mutex that
either pass arguments to the debug implementations or toss them out.
Signed-off-by: Sage Weil <sage@redhat.com>
Otherwise these descriptors may leak across execve() during e.g. MDS respawn.
Fixes: http://tracker.ceph.com/issues/35850
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
by default, debian packaging creates ${prefix}/lib/x86_64-linux-gnu for
multi-arch support. but FindBoost.cmake does not check in such a
directory if $BOOST_ROOT is set. in this change, this directory is added
to the search path.
Signed-off-by: Kefu Chai <kchai@redhat.com>
FindBoost.cmake from upstream cmake now finds python libraries like
find_package(Boost 1.67 python36)
and it export targets like Boost::python36
but we are still linking against Boost::python, so to be compatible
with FindBoost.cmake, we need to update BuildBoost.cmake and
mgr/CMakeLists.txt accordingly. in other words, to export
Boost::python36 and to link Boost::python36.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/23432/head:
Revert "rocksdb: pickup change to link against libsnappy.a"
cmake,make-dist: revert "build gperftools if WITH_STATIC_LIBSTDCXX"
cmake: cleanup snappy related script
debian: strip "-Wl,-Bsymbolic-functions" from LDFLAGS
* rename Findrocksdb.cmake to FindRocksDB.cmake to match its name
* add RocksDB::RocksDB target to BuildRocksDB.cmake and
FindRocksDB.cmake
* use RocksDB::RocksDB target instead of accessing its property
directly, and do not link against its dependencies explicitly.
let its INTERFACE_LINK_LIBRARIES do the job.
Signed-off-by: Kefu Chai <kchai@redhat.com>
as the higher version of libstdc++ is backward compatible with the lower
ones. so there is no need to statically link against C++ libraries. they
can always use the libstdc++ ships with the distro.
This reverts commit a6c73b6ac1
Signed-off-by: Kefu Chai <kchai@redhat.com>
also, we don't need to pass '-lasan' or '-ltsan' to linker. it's
suggested to use the "-fsanitize=${sanitizer}".
please note, this module is compatible with the one used in seastar on
purpose, as seastar is included in ceph using add_subdirectory(), and it
in turn add its own cmake modules directory using list(APPEND ...), so
cmake/modules/FindSantitizers.cmake is prefered over
src/seastar/cmake/FindSantitizers.cmake.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we could create a mini project to build a shared library, and use
try_compile() to test if the found gperftools is compiled with -fPIC.
but as we are targeting mostly xenial when enabling
WITH_STATIC_LIBSTDCXX, and google-perftools on xenial by default
is built without -fPIC. so let's keep it simple.
Signed-off-by: Kefu Chai <kchai@redhat.com>
see
https://libcxx.llvm.org/docs/UsingLibcxx.html#using-libc-experimental-and-experimental,
> Note that as of libc++ 7.0 using the <experimental/filesystem>
> requires linking libc++fs instead of libc++experimental.
do not build ceph_test_admin_socket_output if we are not able to find
the library for std::experimental::filesystem . it is a workaround of
https://reviews.freebsd.org/D10840 where FreeBSD 11.2 does not ship
libc++experimental.a .
Signed-off-by: Kefu Chai <kchai@redhat.com>
- do not link libkv with ALLOC_LIBS, it turns out that if we link
tcmalloc *before* -static-libstdc++ -static-libgcc, libstdc++ and gcc
libs will show up in `ldd` output
- add `-static-libstdc++ -static-libgcc` to CMAKE_SHARED_LINKER_FLAGS
and CMAKE_EXE_LINKER_FLAGS instead of adding them to all shared
libraries and executable. simpler this way.
- link against libtcmalloc statically, because libtcmalloc is a C++
library, linking against it dynamically and linking against C++ runtime
statically will pull in depdencies on two versions of C++ runtime, which
will bring down the app at run-time.
- do not pass '-pie' to linker when building executable if
`WITH_STATIC_LIBSTDCXX` and tcmalloc is used, because the static tcmalloc
is not compiled with PIC.
- only apply '-pie' if ENABLE_SHARED is enabled.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* instead expose the paths, it'd better to expose a library target.
* remove HAVE_LIBJEMALLOC, as it not used anywhere.
Signed-off-by: Kefu Chai <kchai@redhat.com>
for instance, GCC-8 on riscv64 does not offer atomic ops like
__atomic_fetch_or_1, so we need to link against libatomic to get access
to these symbols.
Signed-off-by: Kefu Chai <kchai@redhat.com>
unit_test_framework is an alias of test, and Boost::test is only
necessary if seastar's tests are built. but SEASTAR_ENABLE_TESTS is
reset by us, so drop tests/unit_test_framework here.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* add unit_test_framework for appease seastar's find_package() call,
even we don't build seastar's tests
* some seastar functions declare their return value like:
const size_t str_len(...). and GCC does not like the "const" in it.
so silence it
Signed-off-by: Kefu Chai <kchai@redhat.com>
we should reference liboath by the $name in Find${name}.cmake, also the
$name should be consistent when calling find_package_handle_standard_args().
in this change
* rename Findliboath.cmake to FindOATH.cmake to be consistent with other
find_package() moduless.
* use "OATH" in find_package_handle_standard_args() instead of "oath"
* set the interface properties for OATH::OATH, so the target linking
against it can reference its header directories and libraries automatically.
* remove the stale comment for find_package_handle_standard_args()
* set OATH_INCLUDE_DIRS and OATH_LIBRARIES to follow the convention of
find_package(), even they are not used directly in this project.
Reported-by: Erwan Velu <erwan@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
the new behavior of get_target_property(<var> <target> SOURCES) will be
enforced in future versions of cmake, so let ready for this change now
by removing the generator expressions from the returned source file
list.
Signed-off-by: Kefu Chai <kchai@redhat.com>
so cmake will error on "non-existent dependency in add_dependencies", we
add dependency on CONFIGURE_FILE() output, which is wrong, as its output
is not a target, and is generated when cmake runs. so remove them from
dependencies.
regarding to ceph_ver.h, its path is also wrong. it is created under
${CMAKE_BINARY_DIR}/src/include. so this is another reason to remove it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
This patch adds new QATzip plugin to support QAT for compression.
QATZip is a user space library which builds on top of the Intel
QAT (QuickAssist Technology) user space library, to provide extended
accelerated compression and decompression services by offloading the
actual compression and decompression request(s) to the hardware
QAT accelerators, which are more efficient in terms of cost and power
than general purpose CPUs for those specific compute-intensive
workloads.
Based on QAT accelerators, QATZip can support several compression
algorithm, including deflate, snappy, lz4, etc..
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com>
According to 9535165149
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.
Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>
mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
* Added new CMake flag `WITH_MGR_DASHBOARD_V2_FRONTEND`:
Build the mgr/dashboard_v2 frontend using `npm install && npm run build`
* Set this flag to `OFF` when building packages.
* Removed creation of the frontend from `vstart.sh`
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
since we use http manager which in turn uses curl and uses curl multi
interfaces. While curl is initialized at the first call of curl_easy_init() this
method isn't guaranteed to be safe when multiple threads may call the function
since curl_global_init isn't reentrant. Calling curl_global_init via
rgw::curl::setup_curl which additionally sets up ssl interfaces etc. when
openssl is used as curl's ssl backend. Similarly moving rgw target link to
accomodate this change.
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
openssl <= 1.02 requires explicit callbacks for locking which libcurl doesn't
set. This causes random segmentation faults when openssl uses its global
structures across multiple threads. Providing a simple mutex lock/unlock
functions as a callback. We determine whether openssl is used for libcurl via
curl-config utility which should be installed as a part of our curl development
headers package. We also additionally check that the openssl version is < 1.1.0
which alleviates the need for these callbacks. In this patchset we have done the
following:
- move all curl related global init functionality under rgw::curl namespace
since libcurl may need to set up various ssl libraries etc during its init
- introduce WITH_CURL_OPENSSL in cmake
this checks the backend curl is deployed with using curl-config. Since curl
devel is expected to be installed anyway, this binary should be available and
can help identify the ssl backend curl was compiled with.
- we only setup the locks if beast/civetweb aren't terminated with ssl, since
these libraries setup the locks anyway and we want to prevent double
initialization of openssl. Also we pass in ~CURL_GLOBAL_SSL making curl not
initialize openssl if civetwb/beast is initializing them. Unfortunately this
flag is a noop from curl >= 7.57 wherein both the libraries will end up
initializing openssl anyway, which might override certain settings like error
strings if using openssl < 1.1
https://curl.haxx.se/libcurl/c/threadsafe.htmlhttps://www.openssl.org/docs/man1.0.2/crypto/threads.html#DESCRIPTION
Fixes: http://tracker.ceph.com/issues/22951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Jesse Williamson <jwilliamson@suse.com>
Test whether we're building with libstdc++ or libc++.
Use stdc++filesystem if the former and c++experimental if the later.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* add an option named "MGR_PYTHON_VERSION", so we can build ceph-mgr
which use py3 for running plugins
* also drop the line to specify the "Python_ADDITIONAL_VERSIONS", because
2.7 is listed by all the the FindPythonInterp and FindPythonLibs in
cmake 2.8.12 and up.
* use ${MGR_PYTHON_EXECUTABLE} for holding the path to the python
interpreter used by mgr. because this variable might be overwritten by
"find_package(PythonInterp 2 REQUIRED)" when checking for building env
of pybinding for python2.
Signed-off-by: Kefu Chai <kchai@redhat.com>
adds a more specific option for this boost::context dependency, which was
previously only used by the radosgw beast frontend. see
http://tracker.ceph.com/issues/20048 for more background
Signed-off-by: Casey Bodley <cbodley@redhat.com>
to link libceph-common with libstdc++ if it is enabled.
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4c2216de6a08f048374292a567dbabd083d64e87)
pthread_rwlockattr_setkind_np() is a GNU extension of libpthread. and
Tianshan Qu pointed out, we cannot use
ifdef(PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP) to detect the
availability of this function, because it's an enum not a macro. so,
like other *_np() extensions, we check this one also using cmake at
the configure phase.
Reported-by: Tianshan Qu <tianshan@xsky.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
bluestore,cmake: enable building bluestore without aio
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
KernelDevice is tightly coupled with libaio. more work is needed to
decouple aio from it. but by guarding KernelDevice with HAVE_LIBAIO, we
can enable bluestore on platforms w/o libaio.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* move the check of `USE_CRYPTOPP` to $top_srcdir/CMakeLists.txt
* remove reference of DPDK_LIBRARY, it's defined nowhere
* move the dpdk code to a single place
Signed-off-by: Kefu Chai <kchai@redhat.com>
For CONFIG_BASE_FULL Linux kernels, the maximum number of proc/thread
IDs is set to 32768 by default. This default limit can be quite easily
hit during recovery on nodes with high OSD counts.
To avoid hitting the pid_max default limit, attempt to configure it to
4194304, which corresponds to the maximum limit possible on 64-bit
CONFIG_BASE_FULL kernels.
Fixes: http://tracker.ceph.com/issues/21929
Signed-off-by: David Disseldorp <ddiss@suse.de>
It is not really our business to debug python, boost, or our other
dependencies. Mark them as system includes.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
the second param of enable_launage() is not used, we should call it
multiple times to enable more than one language. switch to project()
command for simplicity.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we've updated the rockdb wrapper on ceph side to be compatible with
the latest version of rocksdb upstream. so ceph is not compatible with
older version of rocksdb.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Tracing with LTTng doesn't work if an application clone()s other
threads. This is an LTTng-related issue and the fix is LD_PRELOAD'ing
liblttng-ust-fork, or linking against it.
We don't have this issue with "regular" tracepoints (see WITH_LTTNG)
because the tracepoints are loaded using dlopen() at runtime by each
Ceph process.
Signed-off-by: Mohamad Gebai <mgebai@suse.com>
All options to en/disable inclusion of libraries or other software
are of the format WITH_ so that the Cmake commaind like ahs all
WITH_* options. The WITH_=ON option will result in a HAVE_ setting
in CMAKE so that tests can use that variable.
Last "abusers" to actually follow this format.
- HAVE_BABELTRACE
- HAVE_ZFSLIB is fixed in #15907
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
* do not REQUIRE libzfs if it is enabled, this follows the way how we
handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
little bit.
Signed-off-by: Kefu Chai <kchai@redhat.com>
build/ops: conditionalize rgw Beast frontend so it isn't built on s390x architecture
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
boost::context is currently (1.63) unsupported for s390x and anyway
it makes sense to conditionalize Boost components so they are only
built with the Ceph components that need them (like is already being
done for mgr).
Fixes: http://tracker.ceph.com/issues/20048
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
when using WITH_SYSTEM_BOOST dont set header-only packages
for BOOST_COMPONENTS. On some distros these packages dont
exist.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
