cython 0.29 introduced a check which prevents multiple python
subinterpreters from loading the same module:
https://github.com/cython/cython/commit/7e27c7c
Unfortunately, this completely breaks ceph-mgr. Until we can
figure out a better long term solution, this commit removes
cython's subinterpreter check, via some careful abuse of the
C preprocessor.
This works because when cython is invoked, it first generates
some C code, then compiles it. We know it's going to generate
C code including:
int __Pyx_check_single_interpreter(void) { ... }
and:
if (__Pyx_check_single_interpreter())
return NULL;
So, we can do the following:
#define void0 dead_function(void)
#define __Pyx_check_single_interpreter(ARG)=ARG ## 0
This replaces the call to __Pyx_check_single_interpreter()
with a literal 0, removing the subinterpreter check.
The void0 dead_function(void) thing is necessary because
the __Pyx_check_single_interpreter() macro also clobbers
that function definition, so we need to make sure it's
replaced with something that works as a function definition.
Fixes: https://tracker.ceph.com/issues/37472
Signed-off-by: Tim Serong <tserong@suse.com>
--single-version-externally-managed is used for packaging python eggs
to deb. so the metadata can be stored in the egg, and can be accessed
by user if necessary, see [1]. but this does not apply to users who
just want to "make install" without packaging Ceph.
the problem is, if we pass --single-version-externally-managed to
setup.py, we need to pass --record or --root also. otherwise setuptools
complains:
error: You must specify --record or --root when building system packages
so in this change, we choose not to pass
"--single-version-externally-managed", unless $DESTDIR is set.
both "ceph.spec.in" and "debian/rules" set $DESTDIR when running "make
install". so we can always check $DESTDIR to tell if current "make
install" is launched by the packaging script or not. this might not be
accurate, but good enough. assuming enduser does not use DESTDIR when
running plain "make install".
--
[1] https://wiki.debian.org/Python/FAQ#How_should_we_package_Python_eggs.3F
Signed-off-by: Kefu Chai <kchai@redhat.com>
It is rather had to overide a once set timeout during CTesting.
This is the best next compromise. Disadvantage is that a rebuild
is required.
But it helps in my Jenkins testing timely catching run-away programs.
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
in python's distutils.ccompiler, linker_exe is composed using CC instead
of LDFLAGS. the latter only effects how it builds (shared) library.
and put CMAKE_C_FLAGS into the cflags for the compiler for building
python C extensions, it's more consistent this way. more importantly,
if we build with ASan enabled, the canary program, a.k.a. rados_dummy.c,
won't link without proper CFLAGS.
without this change, rados.so fails to build with errors like:
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_stack_free_10'
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_report_exp_store8'
...
...
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
Link Error: RADOS library not found
make[3]: ***
[src/pybind/rados/CMakeFiles/cython_rados.dir/build.make:57:
src/pybind/rados/CMakeFiles/cython_rados] Error 1
Signed-off-by: Kefu Chai <kchai@redhat.com>
When building with ccache, distcc, and other compiler wrappers (such
as STLFilt):
CC='ccache gcc' CXX='ccache g++' cmake /path/to/ceph
make
python modules fail to compile since distutils try to execute the
wrapper itself without specifying the actual compiler.
Although cmake has a special magic switch for compiling with ccache
(cmake -DWITH_CCACHE=ON) other tools (distcc) are not supported, and
specifying the compiler as
CC=/whatever/compiler/is
used to work for decades, and it's a good idea to keep it working
Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
Make sure the mbuf and ethdev libraries are created as they export
required symbols.
Fixes: http://tracker.ceph.com/issues/36341
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
we are using GCC7 and up for C++17 support since mimic, and per
https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/AArch64-Options.html ,
GCC 4.9 and up should be able to support crc and crypto features if
these archs are enabled at GCC's configure-time. so we should always use
the -march for detecting the compiler's support instead of using the
inline assembly now.
GCC 4.8 is an ancient compiler, and per
https://gcc.gnu.org/gcc-4.8/changes.html, it was the the first GCC
release which offers AArch64 support. so we don't need to cater for
this GCC version. and we can trust GCC-7 and up.
Fixes: http://tracker.ceph.com/issues/17516
Signed-off-by: Kefu Chai <kchai@redhat.com>
the FindBoost.cmake shipped by cmake 3.5.1 is not new enough to identify
the dependencies of boost 1.63 and up. so we need to copy the
FindBoost.cmake from cmake upstream. it supports up to boost 1.68.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we build dpdk as static library, linking against them without specifying
--whole-archive will get the constructor symbols droppped by the linker
as they are not referenced directly by the excecutable. so we need to do
so to make sure we call all the constructors.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* add Findcryptopp.cmake back
cryptopp support was dropped in #20015, but it's required by
src/msg/async/dpdk/TCP.h, which `#include <cryptopp/md5.h>`
so, to fix the FTBFS of WITH_DPDK=ON, we need to bring
Findcryptopp.cmake back. it was also removed in #20015.
* pass "-march=core2" when building sources which include dpdk headers.
i was wrong that the headers shipped by distro are generic.
the headers use the sse instructions for speedup memcpy, see
/usr/include/x86_64-linux-gnu/dpdk/rte_memcpy.h .
* also, we need to include the arch specific include directory
for building with dpdk.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* add FindGMock.cmake which allows user to use the libgtest-dev
shipped by distro
* add GMock::{GMock,Main}, GTest::{GTest,Main} targets to be
compatible with FindGTest.cmake and FindGMock.cmake, which
expose the built libraries with properties adhered to
them. so the consumer of them can import them in a better way.
* update tests to drop the commands like
set_target_properties(foo PROPERTIES COMPILE_FLAGS ${UNITTEST_CXX_FLAGS}),
as they are already linked against gmock and gtest.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* add dpdk::dpdk if dpdk is built or found, as seastar checks for
it before adding its component libraries. if user installs libdpdk
and builds WITH_SEASTAR=ON or WITH_DPDK=ON, cmake fails to configure
the building system without this fix.
* add dpdk::cflags target for populating the -march=<arch> compile
option.
* also use pkg-config for finding dpdk.
* link common_async_dpdk against dpdk::dpdk instead
please note, the reason why we can remove the "-march=native" compile
option from Finddpdk.cmake, is that the distro shipped header files
are "generic" in the sense of "-march=<ARCH>", they do not enable
the arch specific intrinsic by default, hence the source files
including them do not need specific compiler options.
Signed-off-by: Kefu Chai <kchai@redhat.com>
-march=native causes gcc to use opcodes according to whichever CPU happens to
be installed in the build host, which can be different for every build. This
makes it impossible to achieve a reproducible build.
Also, if the build host has a very new CPU, running the resulting binaries on
older CPUs (of the same family, i.e. x86_64) could result in segmentation
fault.
References: http://tracker.ceph.com/issues/24948
Signed-off-by: Nathan Cutler <ncutler@suse.com>
* refs/pull/23500/head:
debian: mark python-ceph-argparse Arch = all
rpm: package cephfs-shell for fedora
tools/cephfs,deb: package cephfs-shell
cmake: install script and egg-info files of cephfs-shell
tools/cephfs: add setup.py for cephfs-shell
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
FindBoost.cmake from upstream cmake now finds python libraries like
find_package(Boost 1.67 python36)
and it export targets like Boost::python36
but we are still linking against Boost::python, so to be compatible
with FindBoost.cmake, we need to update BuildBoost.cmake and
mgr/CMakeLists.txt accordingly. in other words, to export
Boost::python36 and to link Boost::python36.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* rename Findrocksdb.cmake to FindRocksDB.cmake to match its name
* add RocksDB::RocksDB target to BuildRocksDB.cmake and
FindRocksDB.cmake
* use RocksDB::RocksDB target instead of accessing its property
directly, and do not link against its dependencies explicitly.
let its INTERFACE_LINK_LIBRARIES do the job.
Signed-off-by: Kefu Chai <kchai@redhat.com>
as the higher version of libstdc++ is backward compatible with the lower
ones. so there is no need to statically link against C++ libraries. they
can always use the libstdc++ ships with the distro.
This reverts commit a6c73b6ac1
Signed-off-by: Kefu Chai <kchai@redhat.com>
cmake: fix std::filesystem detection and extract sanitizer detection into its own module
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
FMT_VERSION is now living in fmt/core.h in libfmt >= 5.0, so we cannot
parse fmt/format.h for libfmt's version anymore. also we don't need
Findfmt.cmake now. because libfmt-dev 4.0 and up is required by seastar,
also it's libfmt-dev 3.x's cmake module which fails to offer fmt::fmt
target. in other words, it's safe to drop libfmt 3.x support. but 5.0.0
is not compatible with seastar, we will have failures like:
In file included from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/aligned_buffer.hh:25:0,
from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/reactor.hh:26,
from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/alien.hh:35,
from
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/src/core/alien.cc:23:
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/print.hh:
In function 'seastar::sstring seastar::format(const char*, A&& ...)':
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/print.hh:135:10:
error: 'MemoryWriter' is not a member of 'fmt'
fmt::MemoryWriter out;
^~~~~~~~~~~~
/home/smithfarm/src/ceph/smithfarm/ceph/src/seastar/include/seastar/core/print.hh:136:5:
error: 'out' was not declared in this scope
out.write(fmt, std::forward<A>(a)...);
^~~
so, we should build libfmt even if we have libfmt-dev 5.x
Signed-off-by: Kefu Chai <kchai@redhat.com>
also, we don't need to pass '-lasan' or '-ltsan' to linker. it's
suggested to use the "-fsanitize=${sanitizer}".
please note, this module is compatible with the one used in seastar on
purpose, as seastar is included in ceph using add_subdirectory(), and it
in turn add its own cmake modules directory using list(APPEND ...), so
cmake/modules/FindSantitizers.cmake is prefered over
src/seastar/cmake/FindSantitizers.cmake.
Signed-off-by: Kefu Chai <kchai@redhat.com>
before this change, we fails to detect std::filesystem with clang++,
because the cmake project created by try_compile() only expands
following options passed from its caller:
- COMPILE_DEFINITIONS
- INCLUDE_DIRECTORIES
- LINK_DIRECTORIES
- LINK_LIBRARIES
which do not include CMAKE_CXX_FLAGS, so either we need to (ab)use
COMPILE_DEFINITIONS for passing -std=c++17, or we can change the
CMAKE_CXX_FLAGS in the parent env, as it turns out the created cmake
project does inherit this flag from current project. in this change,
we use the COMPILE_DEFINITIONS approach: simpler this way. and we
can drop it once cmake 3.8 is required.
Signed-off-by: Kefu Chai <kchai@redhat.com>
When exec: ./do_cmake.sh -DALLOCATOR=jemalloc. Met the following
messages:
>> CMake Error at cmake/modules/FindJeMalloc.cmake:28 (endforeach):
endforeach An ENDFOREACH command was found outside of a proper FOREACH
ENDFOREACH structure. Or its arguments did not match the opening FOREACH
command.
>> Call Stack (most recent call first):
CMakeLists.txt:326 (find_package)
This bug introduce commit 8db629a14b.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
