* Findthrift: thrift_FOUND fails to set if called before
find_package_handle_standard_arg, hence reorder adding
thrift::libthrift target after it
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
we use Findthrift.cmake method for adding thrift dependencies, the
cleanup 80e82686eb missed removing these
methods intended to buildthrift from source.
this address the missing failure due to ,
```
CMake Error at cmake/modules/BuildJaeger.cmake:61 (include):
include could not find load file:
Buildthrift
```
this fixes regression introduced by: 80e82686eb
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
debian uses debian/tmp as destination dir for installing build files,
but since we are using common path($build_dir/external) available both
for rpm and debian based dependency installation, it becomes far more
complicated to maintain include/link path for these external projects.
elaborating on it:
path we are configuring for both rpm and debian installing including,
and linking of external librarires:
/build/ceph-17.0.0-5779-g928f9e55/obj-x86_64-linux-gnu/external/
debian appends DESTDIR to this path, and hence our predefined target
artificats cannot find correct path for external libs, I tried adding
ENV${DESTDIR} so that it could include correct external lib install
path, but it still cannot find them:
failed to link in case of:
- install(DIRECTORY $ENV{DESTDIR}${CMAKE_BINARY_DIR}/external/include/jaegertracing
- $ENV{DESTDIR}${CMAKE_BINARY_DIR}/external/include/opentracing
- DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
- include_directories(SYSTEM ${CMAKE_INSTALL_INCLUDEDIR}/jaegertracing)
- include_directories(SYSTEM ${CMAKE_INSTALL_INCLUDEDIR}/opentracing)
-- Installing: /build/ceph-17.0.0-5790-g6bc03cbd/debian/tmp/build/ceph-17.0.0-5790-g6bc03cbd/obj-x86_64-linux-gnu/external/include/jaegertracing/Tracer.h
cd /build/ceph-17.0.0-5790-g6bc03cbd/obj-x86_64-linux-gnu/src && /usr/bin/c++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/build/ceph-17.0.0-5790-g6bc03cbd/obj-x86_64-linux-gnu/src/include -I/build/ceph-17.0.0-5790-g6bc03cbd/src -isystem /build/ceph-17.0.0-5790-g6bc03cbd/obj-x86_64-linux-gnu/boost/include -isystem /build/ceph-17.0.0-5790-g6bc03cbd/obj-x86_64-linux-gnu/include -isystem /build/ceph-17.0.0-5790-g6bc03cbd/src/xxHash -isystem /build/ceph-17.0.0-5790-g6bc03cbd/src/rapidjson/include -isystem /build/ceph-17.0.0-5790-g6bc03cbd/src/include/jaegertracing -isystem /build/ceph-17.0.0-5790-g6bc03cbd/src/include/opentracing -g -O2 -fdebug-prefix-map=/build/ceph-17.0.0-5790-g6bc03cbd=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -std=c++17 -o CMakeFiles/common-objs.dir/mds/mdstypes.cc.o -c /build/ceph-17.0.0-5790-g6bc03cbd/src/mds/mdstypes.cc
In file included from /build/ceph-17.0.0-5790-g6bc03cbd/src/osd/OpRequest.h:21,
from /build/ceph-17.0.0-5790-g6bc03cbd/src/osd/OpRequest.cc:3:
/build/ceph-17.0.0-5790-g6bc03cbd/src/common/tracer.h:10:10: fatal error: jaegertracing/Tracer.h: No such file or directory
Since the install path is in our build environment for these librarires,
skipping DESTDIR looks to me hacky fix, but does the job.
with empty destdir:
-- Installing: /build/ceph-17.0.0-5791-gb97b9640/obj-x86_64-linux-gnu/external/include/jaegertracing/Tracer.h
cd /build/ceph-17.0.0-5791-gb97b9640/obj-x86_64-linux-gnu/src/mon && /usr/bin/c++ -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -D__CEPH__ -D__STDC_FORMAT_MACROS -D__linux__ -I/build/ceph-17.0.0-5791-gb97b9640/obj-x86_64-linux-gnu/src/include -I/build/ceph-17.0.0-5791-gb97b9640/src -isystem /build/ceph-17.0.0-5791-gb97b9640/obj-x86_64-linux-gnu/boost/include -isystem /build/ceph-17.0.0-5791-gb97b9640/obj-x86_64-linux-gnu/include -isystem /build/ceph-17.0.0-5791-gb97b9640/src/xxHash -isystem /build/ceph-17.0.0-5791-gb97b9640/src/rapidjson/include -isystem /build/ceph-17.0.0-5791-gb97b9640/obj-x86_64-linux-gnu/external/include -isystem /build/ceph-17.0.0-5791-gb97b9640/src/rocksdb/include -g -O2 -fdebug-prefix-map=/build/ceph-17.0.0-5791-gb97b9640=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -U_FORTIFY_SOURCE -Wall -fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -Wnon-virtual-dtor -Wno-ignored-qualifiers -ftemplate-depth-1024 -Wpessimizing-move -Wredundant-move -Wstrict-null-sentinel -Woverloaded-virtual -fno-new-ttp-matching -fstack-protector-strong -fdiagnostics-color=auto -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -std=c++17 -o CMakeFiles/mon.dir/MgrMonitor.cc.o -c /build/ceph-17.0.0-5791-gb97b9640/src/mon/MgrMonitor.cc
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* adds BUILD_BYPRODUCT which tells ninja which library will be generated
after the build(needed for dependent build libs)
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* Boost is a dependency for jaeger, to use the right version, we pass
ceph build boost path to cmake jaeger build step
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
since jaegertracing is the original target that jaeger submodule uses in
it's cmake, cmake build complained if named otherwise
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
set_library_properties_for_external_project assists with setting right target
properties for all jaeger dependencies.
IncludeJaeger would take care of linking and creating these targets
having them spread out when they are highly coupled seems not optimal.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
adds Findthrift.cmake which is used to find thrift 0.13, as a dependency
for building jaegertracing
* bump up submodule version for jaeger-client-cpp for thrift compiler
removal
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
the change to build and ship libthift was added when we didn't have 0.13.0
version shipped via distro pkgs, now that centos 8 and F34 supports req.
version, we do not need to build and ship it with jaeger library.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* since focal and centos both have yaml-cpp 0.6 available, which dropped
having boost as it's dependency, moving to 0.6 seems a good upgrade.
* cmake: delete Buildyaml, since distro suppilies v0.6 this is not needed
This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* affects cmake builds for: yaml-cpp, thrift, jaeger, opentracing external
projects
changes done:
* artificat BUILD_BYPRODUCT which tells ninja which library will be generated
after the build(needed for dependent build libs)
* use cmake_command if `make` not found so that we use default generator, cmake
will identify and use ninja directly in this case.
* minor reorder of boost path in jaeger build external projet
fixes: https://tracker.ceph.com/issues/51029
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
since we've dropped the support of GCC older than v8.0, there is no need
to detect <experimental/filesystem>
Signed-off-by: Kefu Chai <kchai@redhat.com>
for better C++17 support, for instance for a better std::filesystem
support.
the reason why 8.1 is required is that ubuntu focal provides GCC-8.1,
and RHEL/CentOS8 provides GCC-8.4.1. so we only test the build on
GCC-8.1 and up so far.
Signed-off-by: Kefu Chai <kchai@redhat.com>
based on recent observation, quite a few C++ source file take
around more than 3.0GiB to compile. for instance,
test_mock_HttpClient.cc could take up to 6270MiB memory to compile.
so increase MAX_{LINK,COMPILE}_MEM accordingly.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change make it more explicit how we override CC, CFLAGS and
CPPFLAGS:
* CC only for the compiler
* CFLAGS only for the flags consumed by C compiler
* CPPFLAGS only used for the preprocessor, it is used by both C and C++
compilers.
this change does not address any specific test failure. it just
improves the correctness and readability.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the __asm__(".asmver ..") is a support provided by the compiler, so
would be better to detect it by either checking the compiler identifer
or just try it out.
in this change, instead of checking the building platform, we check this
feature using check_c_source_compiles().
in future, we could support versioned symbols using function attriubte
or symbol tables or version-script.
on platform where symbol versioning is not supported, we might need to
go with a different approach.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the behavior of cmake 3.17 is a little bit different as it consider
NOT _library STREQUAL ""
as true, if ${_library} is "".
Signed-off-by: Kefu Chai <kchai@redhat.com>
to silence the warning like:
/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/rados/rados.c: In function '__pyx_pymod_exec_rados':
/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/rados/rados.c:78903:30: note: variable tracking size limit exceeded with '-fvar-tracking-assignments', retrying without
78903 | static CYTHON_SMALL_CODE int __pyx_pymod_exec_rados(PyObject *__pyx_pyinit_module)
| ^~~~~~~~~~~~~~~~~~~~~~
cc: warning: /usr/bin/cc: linker input file unused because linking not done
Signed-off-by: Kefu Chai <kchai@redhat.com>
since liburing's configure checks the CC and CXX env variables, let's
pass them when calling "configure". otherwise "make" still uses gcc and
g++ by default as they are specified in the "configure" script. this
also matches with the way how debian/rules calls "make".
Signed-off-by: Kefu Chai <kchai@redhat.com>
before this change, `pmem_VERSION_STRING` is not set if it is not able
to fulfill the specified version requirement. the intention was to check
if the version is able to satisfy the requirement. but actually, passing
an empty `pmem_VERSION_STRING` to `find_package_handle_standard_args()`
as the option of `VERSION_VAR` does not fail this check. on the
contrary, it prints
-- Found pmem: pmem_pmemobj_INCLUDE_DIR;pmem_pmem_INCLUDE_DIR (Required
is at least version "1.17")
if we requires pmem 1.17, while the found version is, for instance,
1.10.
if the required version is 1.7, and the found version is 1.10, the
output from cmake is:
-- Found pmem: pmem_pmemobj_INCLUDE_DIR;pmem_pmem_INCLUDE_DIR (found
suitable version "1.10", minimum required is "1.7")
in this change, the version spec is not specified when calling
`pkg_check_modules()`. so, `PKG_${component}_VERSION` is always set.
and we can always delegate the version checking to
`find_package_handle_standard_args()`. please note, we use the lower
version returned by pkg-config if multiple components are required and
both pkg-config settings return their versions.
Signed-off-by: Kefu Chai <kchai@redhat.com>
as FreeBSD also has msghdr but it does not have systemd, or
flags like MFD_ALLOW_SEALING, O_TMPFILE or F_SEAL_GROW. so
use WITH_SYSTEMD for enabling journald backend of logging system.
also move the option of "WITH_SYSTEMD" up so that the src/CMakeLists.txt
is able to see the variable of WITH_SYSTEMD defined by it.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* cmake/modules/CephChecks.cmake: detect the existence of struct msghdr,
define HAVE_MSGHDR if it is found
* src/common/CMakeLists.txt: do not compile journald.cc if HAVE_MSGHDR
is FALSE. as in that case, we cannot use sendmsg() to write to
journald unix domain socket
* src/test/CMakeLists.txt, src/test/common/CMakeLists.txt: disable test
exercising journald logging backend if HAVE_MSGHDR is not defined
* src/common/Journald.h: define a dummy JournaldLogger and a dummy
JournaldClusterLogger when HAVE_MSGHDR is not defined, in order to
minimize the change in Log.h and Log.cc, otherwise the source code of
Log.h and Log.cc would be segmented into smaller chunks by
`ifdef HAVE_MSGHDR` macros.
* src/include/config-h.in.cmake: define a new macro named
HAVE_MSGHDR.
Signed-off-by: Kefu Chai <kchai@redhat.com>
detect the libpmem libraries' version using pkg-config
please note, the version is not identical to the ones defined in
libpmem.h or libpmemobj/base.h
Signed-off-by: Kefu Chai <kchai@redhat.com>
since we are moving the test nodes from bionic to focal, we are able to
use the prebuilt libzstd libraries when running "make check". to speed
up the build and test, in this change:
* add FindZstd.cmake which allows us to use the libzstd in system
* extract BuildZstd.cmake for better readability
* add an option named "WITH_SYSTEM_ZSTD", which defaults to "OFF",
so user can enable it on demand.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this allows us to pass extra argument to the script or executable to
further customize its behavior when testing.
also fix all the callers of add_ceph_unittest(), as there is no need to
repeat the executable, which is also used as the name of the test.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change should allow us to decouple "ninja tests" from "ctest".
in other words, we can just run
ctest -R run-tox-python-common -V
without running "ninja tests" first. before this change
${name}-venv is added as a dependency of "tests" target.
after this change,
setup-venv-for-${name} is added as a test, which is in turn a test of
run-tox-${name}, so we can just
ctest -R run-tox-${name}
now for preparing the venv and then testing the tox test of ${name}.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to limit the parallel level of compile jobs which consumes lots of memory
to compile and/or link.
compile following targets with heavy_compile_pool.
* unittest_librbd
* crimson*
* ceph-dencoder
some of the .cc files in the target above use up to 6 GiB memory to
compile.
Signed-off-by: Kefu Chai <kchai@redhat.com>
This is a complete rewrite on top of SimpleRADOSStriper as well as the
API. The VFS is now a loadable extension as well.
Fixes: https://tracker.ceph.com/issues/40609
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
the vanilla FindBoost.cmake pulled from cmake has couple assumptions
which do not hold in our environment. so address them case by case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
