Commit Graph

267 Commits

Author SHA1 Message Date
Yuri Weinstein
39f7377556
Merge pull request #20390 from theanalyst/rgw/openssl-init
rgw: setup locks for libopenssl

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
Reviewed-by: Marcus Watts <mwatts@redhat.com>
2018-03-08 07:34:29 -08:00
Kefu Chai
23eb14de52
Merge pull request #20103 from openattic/wip-mgr-dashboard_v2
mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 19:08:57 +08:00
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Igor Fedotov
939805165e test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-03-05 20:35:08 +03:00
Sebastian Wagner
24e216b519
mgr/dashboard_v2: Add CMake target to build the frontend.
* Added new CMake flag `WITH_MGR_DASHBOARD_V2_FRONTEND`:
  Build the mgr/dashboard_v2 frontend using `npm install && npm run build`

* Set this flag to `OFF` when building packages.

* Removed creation of the frontend from `vstart.sh`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-05 13:07:17 +00:00
Abhishek Lekshmanan
112ba0b4da rgw: rgw_admin also inits curl
since we use http manager which in turn uses curl and uses curl multi
interfaces. While curl is initialized at the first call of curl_easy_init() this
method isn't guaranteed to be safe when multiple threads may call the function
since curl_global_init isn't reentrant. Calling curl_global_init via
rgw::curl::setup_curl which additionally sets up ssl interfaces etc. when
openssl is used as curl's ssl backend. Similarly moving rgw target link to
accomodate this change.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-03-02 17:42:00 +01:00
Abhishek Lekshmanan
4e30277e4b rgw: setup locks for libopenssl
openssl <= 1.02 requires explicit callbacks for locking which libcurl doesn't
set. This causes random segmentation faults when openssl uses its global
structures across multiple threads. Providing a simple mutex lock/unlock
functions as a callback. We determine whether openssl is used for libcurl via
curl-config utility which should be installed as a part of our curl development
headers package. We also additionally check that the openssl version is < 1.1.0
which alleviates the need for these callbacks. In this patchset we have done the
following:

- move all curl related global init functionality under rgw::curl namespace
  since libcurl may need to set up various ssl libraries etc during its init
- introduce WITH_CURL_OPENSSL in cmake
  this checks the backend curl is deployed with using curl-config. Since curl
  devel is expected to be installed anyway, this binary should be available and
  can help identify the ssl backend curl was compiled with.
- we only setup the locks if beast/civetweb aren't terminated with ssl, since
  these libraries setup the locks anyway and we want to prevent double
  initialization of openssl. Also we pass in ~CURL_GLOBAL_SSL making curl not
  initialize openssl if civetwb/beast is initializing them. Unfortunately this
  flag is a noop from curl >= 7.57 wherein both the libraries will end up
  initializing openssl anyway, which might override certain settings like error
  strings if using openssl < 1.1

https://curl.haxx.se/libcurl/c/threadsafe.html
https://www.openssl.org/docs/man1.0.2/crypto/threads.html#DESCRIPTION

Fixes: http://tracker.ceph.com/issues/22951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Jesse Williamson <jwilliamson@suse.com>
2018-03-02 17:41:42 +01:00
Nathan Cutler
dba3ebec29 build/ops: change WITH_SYSTEMD default to ON
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-02-12 18:11:31 +01:00
Adam C. Emerson
ba5ce157bd build: Use the right experimental library
Test whether we're building with libstdc++ or libc++.

Use stdc++filesystem if the former and c++experimental if the later.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-09 19:54:02 -05:00
Kefu Chai
53c3b4496f test/admin_socket_output: switch to std::experimental::filesystem
so we don't need to compile boost::filesystem anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-05 17:12:19 +08:00
Kefu Chai
7fe6917ba6
Merge pull request #20134 from tchaikov/wip-spdk-aarch64
spdk: update SPDK to fix the build failure on aarch64

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-27 00:29:23 +08:00
Tone Zhang
95a83580df spdk: enable SPDK on AArch64 by default
Enable SPDK in Ceph on AArch64 by default.

Change-Id: I81ad8e0b860968d62fd2740ae04923f477dab75a
Signed-off-by: Tone Zhang <tone.zhang@arm.com>
2018-01-26 21:05:02 +08:00
Kefu Chai
9e45edaab2 cmake: build ceph-mgr with specified version of python
* add an option named "MGR_PYTHON_VERSION", so we can build ceph-mgr
  which use py3 for running plugins
* also drop the line to specify the "Python_ADDITIONAL_VERSIONS", because
  2.7 is listed by all the the FindPythonInterp and FindPythonLibs in
  cmake 2.8.12 and up.
* use ${MGR_PYTHON_EXECUTABLE} for holding the path to the python
  interpreter used by mgr. because this variable might be overwritten by
  "find_package(PythonInterp 2 REQUIRED)" when checking for building env
  of pybinding for python2.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-26 20:24:54 +08:00
Casey Bodley
ba85740955 cmake: remove cryptopp option
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-19 10:25:41 -05:00
Yuri Weinstein
392314d980
Merge pull request #19549 from bi-shun/brotli-compressor
compressor: Add Brotli Compressor

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-06 09:28:58 -08:00
Casey Bodley
b589b73c70 cmake: add WITH_BOOST_CONTEXT option
adds a more specific option for this boost::context dependency, which was
previously only used by the radosgw beast frontend. see
http://tracker.ceph.com/issues/20048 for more background

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-05 14:55:23 -05:00
Casey Bodley
0e47a339df cmake: update minimum boost version to 1.66
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-05 13:52:47 -05:00
Sage Weil
819a3578fa Merge tag 'v13.0.1' 2018-01-03 10:04:20 -06:00
Sage Weil
9cce242776 v13.0.1
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-03 10:03:39 -06:00
BI SHUN KE
f0aa22660c Compressor: Add Brotli Compressor
Signed-off-by: BI SHUN KE <aionshun@livemail.tw>
2017-12-29 18:22:20 +08:00
Kefu Chai
c869054645 cmake: add WITH_STATIC_LIBSTDCXX option
to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4c2216de6a)
2017-12-15 19:16:12 +08:00
Kefu Chai
4231cfbc99 cmake: DPDK works w/o CRYPTOPP
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-13 14:54:17 +08:00
Kefu Chai
4c2216de6a cmake: add WITH_STATIC_LIBSTDCXX option
to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-12 00:26:52 +08:00
Kefu Chai
a5190c9887
Merge pull request #19202 from tchaikov/wip-check-pthread-ext
cmake,common/RWLock: check for libpthread extensions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-12-01 10:36:21 +08:00
Kefu Chai
12dc5733c9
Merge pull request #18938 from tchaikov/wip-cmake
cmake: check gcc version not release date for libstdc++ saneness

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-29 15:38:16 +08:00
Kefu Chai
cf50fb941e cmake,common/RWLock: check for libpthread extensions
pthread_rwlockattr_setkind_np() is a GNU extension of libpthread. and
Tianshan Qu pointed out, we cannot use
ifdef(PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP) to detect the
availability of this function, because it's an enum not a macro. so,
like other *_np() extensions, we check this one also using cmake at
the configure phase.

Reported-by: Tianshan Qu <tianshan@xsky.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-28 23:50:36 +08:00
Kefu Chai
764c2c0e1c cmake: silence CMP0054 warning
see https://gitlab.kitware.com/cmake/cmake/issues/17381 and
a8be8b1b54,
so before the updated cmake is released and packaged. we should
add this setting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-27 12:50:38 +08:00
Kefu Chai
68c52a669c
Merge pull request #19017 from tchaikov/wip-bluestore-without-aio
bluestore,cmake: enable building bluestore without aio

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-11-22 12:26:34 +08:00
Kefu Chai
89a48189ea cmake: only create sysctl file on linux
and check 64bit platform by using the sizeof(void*)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 15:45:03 +08:00
Kefu Chai
586da3da4f
Merge pull request #18544 from ddiss/pid_max
sysctl.d: set kernel.pid_max=4194304

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-20 14:37:08 +08:00
Kefu Chai
346fd43bf8 cmake: complain if spdk/pmem is enabled w/o bluestore
and complain if bluestore is enabled w/o any backend found.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 13:04:07 +08:00
Kefu Chai
57e792bcae bluestore: enable building bluestore w/o libaio
KernelDevice is tightly coupled with libaio. more work is needed to
decouple aio from it. but by guarding KernelDevice with HAVE_LIBAIO, we
can enable bluestore on platforms w/o libaio.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 11:18:02 +08:00
Kefu Chai
a69bd61b8c cmake: use BuildDPDK if libdpdk not found
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:45:26 +08:00
Kefu Chai
dbe04b4a85 cmake: refactor dpdk related code
* move the check of `USE_CRYPTOPP` to $top_srcdir/CMakeLists.txt
* remove reference of DPDK_LIBRARY, it's defined nowhere
* move the dpdk code to a single place

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:43:33 +08:00
David Disseldorp
14a0c2a727 sysctl.d: set kernel.pid_max=4194304 on 64-bit systems
For CONFIG_BASE_FULL Linux kernels, the maximum number of proc/thread
IDs is set to 32768 by default. This default limit can be quite easily
hit during recovery on nodes with high OSD counts.
To avoid hitting the pid_max default limit, attempt to configure it to
4194304, which corresponds to the maximum limit possible on 64-bit
CONFIG_BASE_FULL kernels.

Fixes: http://tracker.ceph.com/issues/21929

Signed-off-by: David Disseldorp <ddiss@suse.de>
2017-11-17 14:40:23 +01:00
Kefu Chai
df446f06cb cmake: WITH_SPDK=ON by default
Signed-off-by: wanjun.lp <wanjun.lp@alibaba-inc.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-17 15:40:00 +08:00
Kefu Chai
01a9f17825 cmake: build spdk/dpdk in cmake modules
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: wanjun.lp <wanjun.lp@alibaba-inc.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
2017-11-17 15:37:53 +08:00
Kefu Chai
10ab96be4d
Merge pull request #18711 from adamemerson/wip-system-includes
cmake: System Includes to silence warnings from submodules and libraries!

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-04 16:03:37 +08:00
Kefu Chai
2ce53a7b30
Merge pull request #18597 from tchaikov/wip-cmake-cleanup
cmake: cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2017-11-04 16:00:47 +08:00
Adam C. Emerson
8d4a07f89f build: Mark dependency includes as SYSTEM
It is not really our business to debug python, boost, or our other
dependencies. Mark them as system includes.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-11-03 23:58:41 -04:00
Kefu Chai
c43bac48b0 cmake: fix indent
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-03 16:23:26 +08:00
Kefu Chai
4c2dd1607f cmake: remove unused definitions
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-03 16:23:12 +08:00
Casey Bodley
b904b57504 rgw: remove boost::coroutine and context deps
the beast frontend no longer uses stackful coroutines, so these
dependencies aren't necessary

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-27 11:28:40 -04:00
Kefu Chai
4ec719bf9e Merge pull request #18357 from asomers/have_libaio
Build: create a proper WITH_BLUESTORE option

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-By: Kefu Chai <kchai@redhat.com>
2017-10-21 10:38:37 +08:00
Ali Maredia
2e477f3c05 Merge pull request #17673 from mogeb/blkin
blkin: link against lttng-ust-fork
2017-10-20 15:52:55 -04:00
Jianpeng Ma
b4cc9a9aa8 cmake: Build libpmem from source code.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-20 00:44:09 +08:00
Alan Somers
1430f36f67 Build: create a proper WITH_BLUESTORE option
Using Bluestore is not the same thing as having libaio, especially on
non-Linux platforms.

Signed-off-by: Alan Somers <asomers@gmail.com>
2017-10-18 10:20:06 -06:00
Ali Maredia
756d539d92 Merge pull request #18285 from tchaikov/wip-cmake
cmake: bump up required cmake version to 2.8.12
2017-10-13 10:47:34 -04:00
Kefu Chai
3f27dbbdb0 cmake: s/LINK_PRIVATE/PRIVATE/
as 2.8.12 is required not, no need to be compatible with 2.8.11 anymore.

this reverts 457f023.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-13 14:32:32 +08:00
Kefu Chai
77ab5eb821 cmake: refactor -fsanitize=... options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-13 14:32:32 +08:00