this option was introduced in GCC-8.1, and is enabled by default.
otherwise we will have:
src/rocksdb/util/status.cc:28:15: error: ‘char* strncpy(char*, const
char*, size_t)’ output truncated before terminating nul copying as many
bytes from a string as its length [-Werror=string
op-truncation]
std::strncpy(result, state, cch - 1);
~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
/var/ssd/ceph/src/rocksdb/util/status.cc:19:18: note: length computed
here
std::strlen(state) + 1; // +1 for the null terminator
~~~~~~~~~~~^~~~~~~
cc1plus: all warnings being treated as errors
Signed-off-by: Kefu Chai <kchai@redhat.com>
and remove src/test/gtest-parallel submodule, because gtest-parallel is
only useful for running tests. and not all end-users are interested in
running test not to mention running them in parallel. so, to avoid
including gtest-parallel scripts in the dist tarball. it'd be better to
make it optional, and an external project.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Unittests are run sequentially and could take a long while to run.
This commit is about using gtest-parallel on some of them which are
known to be very slow due to this sequentiality.
To enable the parallel features, the 'parallel' argument just have to be
added to the add_ceph_unittest() call like in :
-add_ceph_unittest(unittest_throttle)
+add_ceph_unittest(unittest_throttle parallel)
This commit impact the following tests :
Test name Before After (in seconds)
unittest_erasure_code_shec_all: 212 43
unittest_throttle 15 5
unittest_crush 9 6
unittest_rbd_mirror 79 21
Total 315 75
This commit saves 240 seconds (4 minutes) per build.
Note it exist several other long tests but can't be parallelized since
there is explicit dependencies in the order to run the subtests.
Those stay sequential.
Signed-off-by: Erwan Velu <erwan@redhat.com>
in spdk v18.05, libuuid is linked by libspdk_util.a, in which,
it is used by lib/util/uuid.c. and libspdk_vol.a uses the wrapper
function exposed by libspdk_util.a, so update the CMakefile script to
reflect the change.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we should reference liboath by the $name in Find${name}.cmake, also the
$name should be consistent when calling find_package_handle_standard_args().
in this change
* rename Findliboath.cmake to FindOATH.cmake to be consistent with other
find_package() moduless.
* use "OATH" in find_package_handle_standard_args() instead of "oath"
* set the interface properties for OATH::OATH, so the target linking
against it can reference its header directories and libraries automatically.
* remove the stale comment for find_package_handle_standard_args()
* set OATH_INCLUDE_DIRS and OATH_LIBRARIES to follow the convention of
find_package(), even they are not used directly in this project.
Reported-by: Erwan Velu <erwan@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
we have 'DH_VERBOSE=1' in debian/rules, which instructs debhelper to
pass -DCMAKE_VERBOSE_MAKEFILE=ON to cmake. but the verbose output for
building boost does not really help if something goes wrong while
building the deb packages. if we do want to enable the verbose output,
we can always enable it by passing `-d <N>' to it. see
https://boostorg.github.io/build/manual/develop/index.html#bbv2.overview.invocation.options
.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we should enable them if they are found.
currently, we don't have bzip2 compressor plugin, so it's not detected
in the cmake script. we can always enable it for rocksdb in future.
Fixes: http://tracker.ceph.com/issues/24025
Signed-off-by: Kefu Chai <kchai@redhat.com>
Fio commit(d04f1d5b342e7733) make external ioengine work.
This need when build libfio_ceph_objectstore.so.
Meanwhile fix a comiler error:
/home/ceph/src/test/fio/fio_ceph_objectstore.cc: In constructor ‘{anonymous}::ceph_ioengine::ceph_ioengine()’:
/home/ceph/src/test/fio/fio_ceph_objectstore.cc:733:19: error: invalid conversion from ‘int (*)(thread_data*, io_u*)’ to ‘fio_q_status (*)(thread_data*, io_u*)’ [-fpermissive]
queue = fio_ceph_os_queue;
^~~~~~~~~~~~~~~~~
src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/build.make:62: recipe for target 'src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/fio_ceph_objectstore.cc.o' failed
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
* after upgrading to gcc-8, seems dpdk requires more settings to compile.
in which, CONFIG_RTE_MAX_VFIO_GROUPS=64 is copied from config/common_base.
* librte_bus_pci.a depends on librte_pci.a, so reorder the libraries
in DPDK_LIBRARIES.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the new behavior of get_target_property(<var> <target> SOURCES) will be
enforced in future versions of cmake, so let ready for this change now
by removing the generator expressions from the returned source file
list.
Signed-off-by: Kefu Chai <kchai@redhat.com>
so cmake will error on "non-existent dependency in add_dependencies", we
add dependency on CONFIGURE_FILE() output, which is wrong, as its output
is not a target, and is generated when cmake runs. so remove them from
dependencies.
regarding to ceph_ver.h, its path is also wrong. it is created under
${CMAKE_BINARY_DIR}/src/include. so this is another reason to remove it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* to pick up the fix of https://svn.boost.org/trac10/ticket/11622
* also the boost::python's library name now includes the version suffix
of python version, so update BuildBoost.cmake accordingly.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to silence the warnings like
CMake Warning at CMakeLists.txt:73 (find_package):
By not providing "Findgflags.cmake" in CMAKE_MODULE_PATH this project
has
asked CMake to find a package configuration file provided by "gflags",
but
CMake did not find one.
Could not find a package configuration file provided by "gflags" with
any
of the following names:
gflagsConfig.cmake
gflags-config.cmake
Add the installation prefix of "gflags" to CMAKE_PREFIX_PATH or set
"gflags_DIR" to a directory containing one of the above files. If
"gflags"
provides a separate development package or SDK, be sure it has been
installed.
Signed-off-by: Kefu Chai <kchai@redhat.com>
This patch adds new QATzip plugin to support QAT for compression.
QATZip is a user space library which builds on top of the Intel
QAT (QuickAssist Technology) user space library, to provide extended
accelerated compression and decompression services by offloading the
actual compression and decompression request(s) to the hardware
QAT accelerators, which are more efficient in terms of cost and power
than general purpose CPUs for those specific compute-intensive
workloads.
Based on QAT accelerators, QATZip can support several compression
algorithm, including deflate, snappy, lz4, etc..
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com>
because FindBoost.cmake defines Boost::boost as the target for header
only dependencies.
due to the limit of cmake < 3.1, this only applies to cmake > 3.1
Signed-off-by: Kefu Chai <kchai@redhat.com>
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
so the dependencies can be included when linking against the one which
depends on them. for example, libboost_filesystem depends on
libboost_system, if we don't link against the latter, linker will bail
out when linking an executable using Boost::filesystem.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the boost override for add_library() does not work for ALIAS targets:
CMake Error at cmake/modules/BuildBoost.cmake:227 (add_dependencies):
Cannot add target-level dependencies to alias target "Seastar::seastar".
Signed-off-by: Casey Bodley <cbodley@redhat.com>
after bumping up the required boost version, old copy of boost in
src/boost will not work. so we need to error out early.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* user-config.jam is the suggested way for adding toolset for Boost.Build.
* always specify toolset when running b2, so we can build boost with
specified compiler instead of the default one guessed by boost building system.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* cmake/modules/BuildSPDK.cmake: add lvol
* cmake/modules/BuildDPDK.cmake: add pci and bus_pci
* ceph.spec.in, cmake/modules/BuildSPDK.cmake, debian/control:
re-introduce libuuid dependency, as 17.07 added lvol. and the latter
depends on uuid.
* cmake/modules/BuildSPDK.cmake: avoid introducing local variable of
`iface_libs`.
* cmake/modules/patch-dpdk-conf.sh: disable
CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES, this option introduces the
balanced allocation of memory. but it also requires libnuma-dev.
let's disable it for now.
Signed-off-by: Kefu Chai <kchai@redhat.com>
config.hpp does not exist if boost is not configured. so a boost source
directory unpacked from dist tarball does not have this file. the
bootstrap.sh instead, it is used to create the necessary config files
for jam.
Signed-off-by: Kefu Chai <kchai@redhat.com>
DPDK requires GNU Make. see
spdk/dpdk/doc/guides/freebsd_gsg/build_dpdk.rst. so, we should use the
configured make program to do the job.
Signed-off-by: Kefu Chai <kchai@redhat.com>
otherwise, the default gcc will be used, and the $CMAKE_C_COMPILER
passed to the outer CMakeLists.txt won't kick in. moreover, if the
building script (ceph.spec for instance) could set the $PATH, and
expect that the CMakeLists.txt will use the toolchain executables
in the $PATH to build Ceph, distutils will continue using the default
$CC for linking the python bindings, on UNIX it will be gcc in the
new shell's $PATH, because we are using `install(CODE "... execute_process(
...))` for installing the python bindings. apparently, this is not
expected. because the new shell's $PATH is very likely different
from the one changed by the building script. to address this, we
should always specify the `$LDSHARED` env var explicitly.
also, pass env vars using `ENV{}` instead of the `env` command to
workaround the issue of https://cmake.org/pipermail/cmake/2015-December/062216.html,
because it's not straightforward to set environment variables with
spaces in the them using cmake. and because one cannot use add_custom_target()
in the script mode of cmake. this leave me only limited options to
fix this issue.
Signed-off-by: Kefu Chai <kchai@redhat.com>
otherwise it's always true, even if the libaries are not found. also
the cmake output will be wrong without this change:
without this change, it is:
-- Found gperftools: GPERFTOOLS_TCMALLOC_LIBRARY;GPERFTOOLS_TCMALLOC_MINIMAL_LIBRARY;GPERFTOOLS_PROFILER_LIBRARY (found version "2.5.93")
with this change, it would be:
-- Found gperftools: /tmp/gperf/lib/libtcmalloc.so (found version "2.5.93")
Signed-off-by: Kefu Chai <kchai@redhat.com>
there is chance that its "patch" version is empty. in that case, do not
include it in its version string. otherwise, we will have something
like:
Found gperftools:
GPERFTOOLS_TCMALLOC_LIBRARY;GPERFTOOLS_TCMALLOC_MINIMAL_LIBRARY;GPERFTOOLS_PROFILER_LIBRARY
(found version "2.4.#define TC_VERSION_PATCH """)
when running cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
the commit d406f228 in gperf implements a c11 feature used by a
recent change in rocksdb: 16e03882, which uses aligned_alloc().
and 16e03882 in rocksdb was merged after v5.7 was tagged, while
16e03882 in gperf was merged after v2.6.1 was tagged.
because aligned_alloc() is not implemented by tcmalloc until the
not-yet-released 2.6.2, if we call aligned_alloc() in an application
linked against tcmalloc, what gets called will be the glibc's
aligned_alloc(). but if we free() the memory chunk allocated by
aligned_alloc(), the tcmalloc's implementation kicks in, then
InvalidFree() is called, because the memory chunk being freed was
allocated by tcmalloc. in short, "mixing allocators", quote from
Dan Mick.
in rocksdb, aligned_alloc() is used if _ISOC11_SOURCE is defined, this
makes sense, because aligned_alloc() is a C11 function. we could avoid
using it by not defining _ISOC11_SOURCE. but as long as _GNU_SOURCE is
defined, glibc defines _ISOC11_SOURCE. and libstdc++ requires
_GNU_SOURCE, because it uses a fair amount of GNU extensions.
Fixes: http://tracker.ceph.com/issues/21422
Signed-off-by: Kefu Chai <kchai@redhat.com>
Skip nonexistent 'expected' Python packaging files, and add bin/ as
a possible file for symlinking during build (for Python projects with
slightly different forms, currently ceph-volume)
Signed-off-by: Dan Mick <dan.mick@redhat.com>
The ppc64 support requires a couple of changes:
- adding the ppc64 support to cmake
- changing optimized crc32 code to compile on ppc64le only
- moving ifdef condition before crc32_align to avoid defined but not
used warning
Signed-off-by: Boris Ranto <branto@redhat.com>
* do not REQUIRE libzfs if it is enabled, this follows the way how we
handle `WITH_XFS` option.
* also refactor the cmake script related to libxfs backend support a
little bit.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Scope SIMD CPU flag detection base on target architecture,
resolving build failures in gf-complete on i386; this is
inline with the previous autotools based feature detection.
Only compile crypto/isa-l if yasm 64 bit complier is found,
effectively limiting scope to x86_64.
Signed-off-by: James Page <james.page@ubuntu.com>
The cmake package names of libraries are case-sensitive. The name used in
FIND_PACKAGE_HANDLE_STANDARD_ARGS - e.g. "cryptopp" - must match the name used
in the module name - "Findcryptopp.cmake" - as well as the name used in
find_package() call - "find_package(cryptopp REQUIRED)"
Note that the term "package" here refers to a cmake abstraction, *not* an
actual RPM or Debian package.
Fixes: http://tracker.ceph.com/issues/19853
Signed-off-by: Nathan Cutler <ncutler@suse.com>
so instead of excluding all submodules, one can just exclude boost and
rocksdb using:
cmake -DCTAG_EXCLUDES="boost;rocksdb" ..
Signed-off-by: Kefu Chai <kchai@redhat.com>
Test not only for -march support, but also the actual
presence of the intrinsic routines. Not sure why, but gcc
4.8.5 passes the first but not the second.
Fixes: http://tracker.ceph.com/issues/19386
Signed-off-by: Dan Mick <dan.mick@redhat.com>
ARMv8 defines PMULL crypto instruction.
This patch optimizes crc32c calculate with the instruction when
available rather than original linear crc instructions.
ceph crc32c performance unit test shows that the optimization get
~ x3.90 speedups on ThunderX ARM Core@2.0GHz (Cavium)
~ x1.45 speedups on ARM Cortex-A57@2.1GHz (Huaiwei)
~ x1.16 speedups on ARM Cortex-A57@2.0GHz (Softiron)
Jira: ENTLLT-358
Change-Id: I657422cd20c9ca78237cd060210a5383f4122575
Signed-off-by: wei xiao <wei.xiao@linaro.org>
This makes it more portable
Remove /usr/include as it is searched by find_path anyway.
Remove unnecessary include that weren't used.
Signed-off-by: Pascal Bach <pascal.bach@nextrem.ch>
simplified the package to find jemalloc. this one matches
the one in src/rocksdb/cmake/modules. The existing one
was struggling to find jemalloc on some hosts.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
