According to 9535165149
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.
Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>
mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
* Added new CMake flag `WITH_MGR_DASHBOARD_V2_FRONTEND`:
Build the mgr/dashboard_v2 frontend using `npm install && npm run build`
* Set this flag to `OFF` when building packages.
* Removed creation of the frontend from `vstart.sh`
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
since we use http manager which in turn uses curl and uses curl multi
interfaces. While curl is initialized at the first call of curl_easy_init() this
method isn't guaranteed to be safe when multiple threads may call the function
since curl_global_init isn't reentrant. Calling curl_global_init via
rgw::curl::setup_curl which additionally sets up ssl interfaces etc. when
openssl is used as curl's ssl backend. Similarly moving rgw target link to
accomodate this change.
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
openssl <= 1.02 requires explicit callbacks for locking which libcurl doesn't
set. This causes random segmentation faults when openssl uses its global
structures across multiple threads. Providing a simple mutex lock/unlock
functions as a callback. We determine whether openssl is used for libcurl via
curl-config utility which should be installed as a part of our curl development
headers package. We also additionally check that the openssl version is < 1.1.0
which alleviates the need for these callbacks. In this patchset we have done the
following:
- move all curl related global init functionality under rgw::curl namespace
since libcurl may need to set up various ssl libraries etc during its init
- introduce WITH_CURL_OPENSSL in cmake
this checks the backend curl is deployed with using curl-config. Since curl
devel is expected to be installed anyway, this binary should be available and
can help identify the ssl backend curl was compiled with.
- we only setup the locks if beast/civetweb aren't terminated with ssl, since
these libraries setup the locks anyway and we want to prevent double
initialization of openssl. Also we pass in ~CURL_GLOBAL_SSL making curl not
initialize openssl if civetwb/beast is initializing them. Unfortunately this
flag is a noop from curl >= 7.57 wherein both the libraries will end up
initializing openssl anyway, which might override certain settings like error
strings if using openssl < 1.1
https://curl.haxx.se/libcurl/c/threadsafe.htmlhttps://www.openssl.org/docs/man1.0.2/crypto/threads.html#DESCRIPTION
Fixes: http://tracker.ceph.com/issues/22951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Jesse Williamson <jwilliamson@suse.com>
Test whether we're building with libstdc++ or libc++.
Use stdc++filesystem if the former and c++experimental if the later.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
* add an option named "MGR_PYTHON_VERSION", so we can build ceph-mgr
which use py3 for running plugins
* also drop the line to specify the "Python_ADDITIONAL_VERSIONS", because
2.7 is listed by all the the FindPythonInterp and FindPythonLibs in
cmake 2.8.12 and up.
* use ${MGR_PYTHON_EXECUTABLE} for holding the path to the python
interpreter used by mgr. because this variable might be overwritten by
"find_package(PythonInterp 2 REQUIRED)" when checking for building env
of pybinding for python2.
Signed-off-by: Kefu Chai <kchai@redhat.com>
adds a more specific option for this boost::context dependency, which was
previously only used by the radosgw beast frontend. see
http://tracker.ceph.com/issues/20048 for more background
Signed-off-by: Casey Bodley <cbodley@redhat.com>
pthread_rwlockattr_setkind_np() is a GNU extension of libpthread. and
Tianshan Qu pointed out, we cannot use
ifdef(PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP) to detect the
availability of this function, because it's an enum not a macro. so,
like other *_np() extensions, we check this one also using cmake at
the configure phase.
Reported-by: Tianshan Qu <tianshan@xsky.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
bluestore,cmake: enable building bluestore without aio
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
KernelDevice is tightly coupled with libaio. more work is needed to
decouple aio from it. but by guarding KernelDevice with HAVE_LIBAIO, we
can enable bluestore on platforms w/o libaio.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* move the check of `USE_CRYPTOPP` to $top_srcdir/CMakeLists.txt
* remove reference of DPDK_LIBRARY, it's defined nowhere
* move the dpdk code to a single place
Signed-off-by: Kefu Chai <kchai@redhat.com>
For CONFIG_BASE_FULL Linux kernels, the maximum number of proc/thread
IDs is set to 32768 by default. This default limit can be quite easily
hit during recovery on nodes with high OSD counts.
To avoid hitting the pid_max default limit, attempt to configure it to
4194304, which corresponds to the maximum limit possible on 64-bit
CONFIG_BASE_FULL kernels.
Fixes: http://tracker.ceph.com/issues/21929
Signed-off-by: David Disseldorp <ddiss@suse.de>
It is not really our business to debug python, boost, or our other
dependencies. Mark them as system includes.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
the second param of enable_launage() is not used, we should call it
multiple times to enable more than one language. switch to project()
command for simplicity.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we've updated the rockdb wrapper on ceph side to be compatible with
the latest version of rocksdb upstream. so ceph is not compatible with
older version of rocksdb.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Tracing with LTTng doesn't work if an application clone()s other
threads. This is an LTTng-related issue and the fix is LD_PRELOAD'ing
liblttng-ust-fork, or linking against it.
We don't have this issue with "regular" tracepoints (see WITH_LTTNG)
because the tracepoints are loaded using dlopen() at runtime by each
Ceph process.
Signed-off-by: Mohamad Gebai <mgebai@suse.com>
All options to en/disable inclusion of libraries or other software
are of the format WITH_ so that the Cmake commaind like ahs all
WITH_* options. The WITH_=ON option will result in a HAVE_ setting
in CMAKE so that tests can use that variable.
Last "abusers" to actually follow this format.
- HAVE_BABELTRACE
- HAVE_ZFSLIB is fixed in #15907
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
* do not REQUIRE libzfs if it is enabled, this follows the way how we
handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
little bit.
Signed-off-by: Kefu Chai <kchai@redhat.com>
build/ops: conditionalize rgw Beast frontend so it isn't built on s390x architecture
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
boost::context is currently (1.63) unsupported for s390x and anyway
it makes sense to conditionalize Boost components so they are only
built with the Ceph components that need them (like is already being
done for mgr).
Fixes: http://tracker.ceph.com/issues/20048
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
when using WITH_SYSTEM_BOOST dont set header-only packages
for BOOST_COMPONENTS. On some distros these packages dont
exist.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
We are unconditionally linking lttng-ust, which make selinux complain. We
should either
- fix selinux rules and unconditionally link
- dlopen at runtime based on an option (like we do for the current
tracepoints)
- ???
Signed-off-by: Sage Weil <sage@redhat.com>
so instead of excluding all submodules, one can just exclude boost and
rocksdb using:
cmake -DCTAG_EXCLUDES="boost;rocksdb" ..
Signed-off-by: Kefu Chai <kchai@redhat.com>
mallinfo(3) doesnt have 64bit compatible version so when
malloc size > 4GB, either wrapped value or a negtive value
returned, which is really misleading.
Also bump up all int to long to prevent overflow oneday we
have > 2TB memory.
Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
New function and oid lttng tracing enhancements that provide function
and oid level latency breakdown. Requires WITH_EVENTTRACE=ON option to enable.
Signed-off-by: Anjaneya Chagam <anjaneya.chagam@intel.com>
fcgi can now be compiled out of the binary. this reduces the system
dependencies when fcgi is not needed or used.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
prior to this change, libcommon is a convenient library which gets
linked into librados, librbd and libcephfs and all ceph executables.
this incurs some problems:
- double dose of libcommon in memory space and HDD: waste of memory
and disk space.
- if an application links to two libraries including libcommon at the
same time. take librados and libcephfs as an example, they could
interfere with each other by changing the other guy's status.
after this change, libcommon is tuned into a shared library and
renamed to libceph-common. it will be installed into $prefix/lib/ceph,
and packaged in librados2.
ceph.spec.in,debian/librados2.install: package libceph-common in
librados2.
CMakeLists.txt:
- do not link against libboost-* if not necessary.
- s/common/ceph-common/g
- install libceph-common into $prefix/lib/ceph
- set rpath to $prefix/lib/ceph
- link against ceph-common if an executable needs access to non public
symbols in ceph.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The options are taken from rocksdb, but updated to match the
instructions here:
https://gist.github.com/kwk/4171e37f4bcdf7705329
Added option to explicitly enable leak checking via
-fsanitize-leak, after review. The behavior I've observed from
-fsanitize-address and libasan linkage included leak checking,
but perhaps this is not general or changing behavior.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
jemalloc does not implement mallinfo. Only check for existence
of mallinfo when the allocator is not jemalloc.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
this is an un-modified message from dev-util/cmake-3.7.1
/usr/share/cmake/Modules/FindPythonLibs.cmake
call ``find_package(PythonInterp)`` first to
get the currently active Python version by default with a consistent
version of PYTHON_LIBRARIES.
Signed-off-by: Yixun Lan <dlan@gentoo.org>
---
the detail error message we've met here
Found PythonInterp: python (found suitable version "2.7.12", minimum required is "2")
CMake Error at /usr/share/cmake/Modules/FindPythonLibs.cmake:54 (get_filename_component):
get_filename_component called with incorrect number of arguments
Call Stack (most recent call first):
src/CMakeLists.txt:209 (find_package)
* find_package(keyutils REQUIRED) if (WITH_LIBCEPHFS OR WITH_RBD)
prior to this change, we detect keyutils if the building platform is not
FreeBSD, we should instead check the WITH_* options, and let the
maintainer to decided what is the best for his/her platform, and error
out if the building host cannot fulfill the requirement to build the
asserts.
* build krbd.cc if (WITH_RBD)
Signed-off-by: Kefu Chai <kchai@redhat.com>
libcephd is a library that contains ceph daemon code
that can be statically linked in other applications.
Added MergeStaticLibraries.cmake that can merge static libraries
to form a bigger one. This approach avoids the need to mess with
STATIC libraries all over the code base.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
If the host system has boost version 1.61 or higher (as in the case
for ubuntu 16.10 yakkety) the ceph build is currently broken. it
will pickup the system boost libraries but use the headers from the
submodule.
This commit ensure that when WITH_SYSTEM_BOOST is OFF we always
use the boost libraries and headers built from the submodule.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
We're changing the default value because the previous one was
a makeshift solution to not fail Ceph compilation due to
the Beast's dependency on Boost >= 1.54 that wasn't available
on CentoOS 7. As we got the in-tree Boost we can compile
the ASIO front-end by default.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
Intended to remove an apparent race. The two effects are
1. replace top-level command callouts w/file builtins
2. do them in the src/rgw sub-cmake
This is cleaner, and ideally avoids the race.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
now that monitors support rocksdb, fix the build system
to enable compiling without leveldb. kv_backend would need
to be set correctly if you were to to use rocksdb for the
monitors.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
Build Boost using the src/boost submodule, unless overridden
by -DWITH_SYSTEM_BOOST.
If -DBOOST_J=<n> is provided, builds Boost with <n> jobs.
Boost builds in the configured Build directory.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
msg/async: ibverbs/rdma support
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Mellanox <adirl@mellanox.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Fixes here:
Pull civetweb from new branch/commit.
Build civetweb.c w/ USE_IPV6 (w/o depending on extra ceph-only build
logic inside of civetweb.)
Make several things "const" to please the compiler.
Remap port string at '+' -> ,; to avoid , overload.
Use new civetweb options validate_http_method=no canonicalize_url_path=no
Signed-off-by: Marcus Watts <mwatts@redhat.com>
This is because of versioning issues between Boost and Beast
on CentOS 7. It is intended that this patch will be reverted
after merging the in-tree Boost facility.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
The fcgi and expat libraries are only used by rgw, so only make them
hard requirements if WITH_RADOSGW is set.
Signed-off-by: David Disseldorp <ddiss@suse.de>
* renamed target to fio_ceph_objectstore
* moved into src/test/fio subdirectory
* added to cmake build
* added support for DDIR_READ
* added required fio option 'conf' to load a ceph configuration file
* added multiple collections per job to simulate parallelism from pgs
* creates objects of the appropriate size on setup
* added support for multiple jobs that share an ObjectStore instance
Signed-off-by: Casey Bodley <cbodley@redhat.com>
find_package(lttng-ust REQUIRED) fails to find the lttng library without
this change. because find_path(LTTNG_LIBRARY_DIR ...) does not search
in the default library paths. and the second mode of
FIND_PACKAGE_HANDLE_STANDARD_ARGS() does not stop the cmake with a fatal
message, even some of the required vars are missing. so use the
implemetantion from cmake upstream.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Include Python 3 bindings into the cmake build and make packages for them
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
* remove Findtcmalloc.cmake, use Findgperftools.cmake instead.
* tcmalloc is packaged in gperftools-dev, so we can check them in a single
place.
* fix the check of heap-profiler.h, we should enable the check as long
as tcmalloc is enabled or profiler is enabled.
* only check gperftools headers in "include/gperftools", and do not
"include/google" anymore. as we only support the distros which
shipping recent gperftools-dev package with "include/gperftools".
and "google/*.h" are deprecated.
* set ALLOC_LIBS with GPERFTOOLS_TCMALLOC_LIBRARY, so we can link
against tcmalloc with full path.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* AIO_LIBS is now named AIO_LIBARIES, and there is not point to print
out its path
* USE_NSS is not defined if NSS is not checked, so if(USE_NSS) is better
here.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this behaviour matches the autotools. and we should not enable profiling
unless asked to do so.
Fixes: http://tracker.ceph.com/issues/16804
Signed-off-by: Kefu Chai <kchai@redhat.com>
for example
```
include_directories(${LEVELDB_PREFIX}/include)
```
does not make sense if `LEVELDB_PREFIX` is not defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* use find_package_handle_standard_args() and remove unused bits.
* rename OPENLDAP_LIBS to OPENLDAP_LIBRARIES, to be consistent with
find_package() modules.
Signed-off-by: Kefu Chai <kchai@redhat.com>
libuuid dependency is not found in ceph.spec.in or debian/control.
and we are not using libuuid since 62bfc7a.
Signed-off-by: Kefu Chai <kchai@redhat.com>
please note "make test" is used by cmake to run tests, so we cannot just
repurpose it to *build* them.
* AddCephTest.cmake: depends on "tests"
* CMakeLists.txt: let "check" depend on "tests"
* src/CMakeLists.txt: update the run-tox tests
* run-make-check.sh: use "make tests" and "ctest" instead of "make check"
* ceph-detect-init/CMakeLists.txt: let "tests" depend on
"ceph-detect-init"
* ceph-disk/CMakeLists.txt: let "tests" depend on "ceph-disk"
Signed-off-by: Kefu Chai <kchai@redhat.com>
The cmake build should select NSS or CryptoPP consistently with
current autoconf logic, as used in our RHEL builds (-DWITH_NSS),
and the SSL linkage must match (for NSS, -lssl3).
With this change, there is no direct linkage with libssl when NSS
is selected (I still see indirect linkage via libcurl, in turn
via libssh2, looking into that).
With guidance from upstream and downstream, NSS builds are now
selected by default.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
so it would be easier to figure out what allocator we are using later,
instead of poking around ${Foo}_FOUND variables.
Signed-off-by: Kefu Chai <kchai@redhat.com>
