Commit Graph

282 Commits

Author SHA1 Message Date
Sage Weil
6ae51aed13 Merge remote-tracking branch 'gh/mimic' 2018-05-10 12:18:29 -05:00
Kefu Chai
50a052d79e
Merge pull request #19714 from qwren/qatzip-compressor
compressor: add QAT support

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-05-04 09:17:17 +08:00
Jenkins Build Slave User
1f43eda5fd 13.1.0 2018-05-03 17:57:34 +00:00
Kefu Chai
791af5459e cmake: enable CMP0051
the new behavior of get_target_property(<var> <target> SOURCES) will be
enforced in future versions of cmake, so let ready for this change now
by removing the generator expressions from the returned source file
list.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-26 16:47:54 +08:00
Kefu Chai
4aea3bed7f cmake: enable CMP0046
so cmake will error on "non-existent dependency in add_dependencies", we
add dependency on CONFIGURE_FILE() output, which is wrong, as its output
is not a target, and is generated when cmake runs. so remove them from
dependencies.

regarding to ceph_ver.h, its path is also wrong. it is created under
${CMAKE_BINARY_DIR}/src/include. so this is another reason to remove it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-26 16:44:09 +08:00
Kefu Chai
750c301e1c cmake: use CMP0028 NEW
it's intended behavior to use imported or alias target for dependencies
with name with "::" in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-18 15:10:54 +08:00
Qiaowei Ren
9f3965aef3 compressor: add QAT support
This patch adds new QATzip plugin to support QAT for compression.

QATZip is a user space library which builds on top of the Intel
QAT (QuickAssist Technology) user space library, to provide extended
accelerated compression and decompression services by offloading the
actual compression and decompression request(s) to the hardware
QAT accelerators, which are more efficient in terms of cost and power
than general purpose CPUs for those specific compute-intensive
workloads.

Based on QAT accelerators, QATZip can support several compression
algorithm, including deflate, snappy, lz4, etc..

Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com>
2018-04-18 12:05:44 +08:00
Grant Slater
bb2da9302c cmake: Enable LZ4 if required library is available.
Signed-off-by: Grant Slater <grant.slater@wearefriday.com>
2018-04-12 09:35:24 -04:00
Yehuda Sadeh
6976202360 cls/otp, rgw-admin: use older liboath api, parse seed once
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:02:17 -07:00
Yehuda Sadeh
c3a6f7588a rgw: mfa - initial work
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:01:02 -07:00
Casey Bodley
247e074338 rgw: add ssl support to beast frontend
adds frontend options ssl_certificate, ssl_private_key, ssl_port, ssl_endpoint

Fixes: http://tracker.ceph.com/issues/22832

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-04-05 15:30:22 -04:00
Sage Weil
643253c326 Merge tag 'v13.0.2'
v13.0.2
2018-04-03 10:08:22 -05:00
Sage Weil
ed9182c266 CMakeLists: v13.0.2
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-02 10:03:02 -05:00
Nathan Cutler
a95eacbe48 cmake: add chrono to BOOST_COMPONENTS
According to 9535165149
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.

Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-20 16:14:11 +01:00
Ricardo Dias
aa205a6731
mgr/dashboard: renamed dashboard_v2 to dashboard
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-03-16 07:23:08 +00:00
Yuri Weinstein
39f7377556
Merge pull request #20390 from theanalyst/rgw/openssl-init
rgw: setup locks for libopenssl

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
Reviewed-by: Marcus Watts <mwatts@redhat.com>
2018-03-08 07:34:29 -08:00
Kefu Chai
23eb14de52
Merge pull request #20103 from openattic/wip-mgr-dashboard_v2
mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 19:08:57 +08:00
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Igor Fedotov
939805165e test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-03-05 20:35:08 +03:00
Sebastian Wagner
24e216b519
mgr/dashboard_v2: Add CMake target to build the frontend.
* Added new CMake flag `WITH_MGR_DASHBOARD_V2_FRONTEND`:
  Build the mgr/dashboard_v2 frontend using `npm install && npm run build`

* Set this flag to `OFF` when building packages.

* Removed creation of the frontend from `vstart.sh`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-05 13:07:17 +00:00
Abhishek Lekshmanan
112ba0b4da rgw: rgw_admin also inits curl
since we use http manager which in turn uses curl and uses curl multi
interfaces. While curl is initialized at the first call of curl_easy_init() this
method isn't guaranteed to be safe when multiple threads may call the function
since curl_global_init isn't reentrant. Calling curl_global_init via
rgw::curl::setup_curl which additionally sets up ssl interfaces etc. when
openssl is used as curl's ssl backend. Similarly moving rgw target link to
accomodate this change.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-03-02 17:42:00 +01:00
Abhishek Lekshmanan
4e30277e4b rgw: setup locks for libopenssl
openssl <= 1.02 requires explicit callbacks for locking which libcurl doesn't
set. This causes random segmentation faults when openssl uses its global
structures across multiple threads. Providing a simple mutex lock/unlock
functions as a callback. We determine whether openssl is used for libcurl via
curl-config utility which should be installed as a part of our curl development
headers package. We also additionally check that the openssl version is < 1.1.0
which alleviates the need for these callbacks. In this patchset we have done the
following:

- move all curl related global init functionality under rgw::curl namespace
  since libcurl may need to set up various ssl libraries etc during its init
- introduce WITH_CURL_OPENSSL in cmake
  this checks the backend curl is deployed with using curl-config. Since curl
  devel is expected to be installed anyway, this binary should be available and
  can help identify the ssl backend curl was compiled with.
- we only setup the locks if beast/civetweb aren't terminated with ssl, since
  these libraries setup the locks anyway and we want to prevent double
  initialization of openssl. Also we pass in ~CURL_GLOBAL_SSL making curl not
  initialize openssl if civetwb/beast is initializing them. Unfortunately this
  flag is a noop from curl >= 7.57 wherein both the libraries will end up
  initializing openssl anyway, which might override certain settings like error
  strings if using openssl < 1.1

https://curl.haxx.se/libcurl/c/threadsafe.html
https://www.openssl.org/docs/man1.0.2/crypto/threads.html#DESCRIPTION

Fixes: http://tracker.ceph.com/issues/22951
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Jesse Williamson <jwilliamson@suse.com>
2018-03-02 17:41:42 +01:00
Nathan Cutler
dba3ebec29 build/ops: change WITH_SYSTEMD default to ON
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-02-12 18:11:31 +01:00
Adam C. Emerson
ba5ce157bd build: Use the right experimental library
Test whether we're building with libstdc++ or libc++.

Use stdc++filesystem if the former and c++experimental if the later.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-09 19:54:02 -05:00
Kefu Chai
53c3b4496f test/admin_socket_output: switch to std::experimental::filesystem
so we don't need to compile boost::filesystem anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-05 17:12:19 +08:00
Kefu Chai
7fe6917ba6
Merge pull request #20134 from tchaikov/wip-spdk-aarch64
spdk: update SPDK to fix the build failure on aarch64

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-27 00:29:23 +08:00
Tone Zhang
95a83580df spdk: enable SPDK on AArch64 by default
Enable SPDK in Ceph on AArch64 by default.

Change-Id: I81ad8e0b860968d62fd2740ae04923f477dab75a
Signed-off-by: Tone Zhang <tone.zhang@arm.com>
2018-01-26 21:05:02 +08:00
Kefu Chai
9e45edaab2 cmake: build ceph-mgr with specified version of python
* add an option named "MGR_PYTHON_VERSION", so we can build ceph-mgr
  which use py3 for running plugins
* also drop the line to specify the "Python_ADDITIONAL_VERSIONS", because
  2.7 is listed by all the the FindPythonInterp and FindPythonLibs in
  cmake 2.8.12 and up.
* use ${MGR_PYTHON_EXECUTABLE} for holding the path to the python
  interpreter used by mgr. because this variable might be overwritten by
  "find_package(PythonInterp 2 REQUIRED)" when checking for building env
  of pybinding for python2.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-26 20:24:54 +08:00
Casey Bodley
ba85740955 cmake: remove cryptopp option
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-19 10:25:41 -05:00
Yuri Weinstein
392314d980
Merge pull request #19549 from bi-shun/brotli-compressor
compressor: Add Brotli Compressor

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-06 09:28:58 -08:00
Casey Bodley
b589b73c70 cmake: add WITH_BOOST_CONTEXT option
adds a more specific option for this boost::context dependency, which was
previously only used by the radosgw beast frontend. see
http://tracker.ceph.com/issues/20048 for more background

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-05 14:55:23 -05:00
Casey Bodley
0e47a339df cmake: update minimum boost version to 1.66
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-05 13:52:47 -05:00
Sage Weil
819a3578fa Merge tag 'v13.0.1' 2018-01-03 10:04:20 -06:00
Sage Weil
9cce242776 v13.0.1
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-03 10:03:39 -06:00
BI SHUN KE
f0aa22660c Compressor: Add Brotli Compressor
Signed-off-by: BI SHUN KE <aionshun@livemail.tw>
2017-12-29 18:22:20 +08:00
Kefu Chai
c869054645 cmake: add WITH_STATIC_LIBSTDCXX option
to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 4c2216de6a)
2017-12-15 19:16:12 +08:00
Kefu Chai
4231cfbc99 cmake: DPDK works w/o CRYPTOPP
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-13 14:54:17 +08:00
Kefu Chai
4c2216de6a cmake: add WITH_STATIC_LIBSTDCXX option
to link libceph-common with libstdc++ if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-12 00:26:52 +08:00
Kefu Chai
a5190c9887
Merge pull request #19202 from tchaikov/wip-check-pthread-ext
cmake,common/RWLock: check for libpthread extensions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-12-01 10:36:21 +08:00
Kefu Chai
12dc5733c9
Merge pull request #18938 from tchaikov/wip-cmake
cmake: check gcc version not release date for libstdc++ saneness

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-11-29 15:38:16 +08:00
Kefu Chai
cf50fb941e cmake,common/RWLock: check for libpthread extensions
pthread_rwlockattr_setkind_np() is a GNU extension of libpthread. and
Tianshan Qu pointed out, we cannot use
ifdef(PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP) to detect the
availability of this function, because it's an enum not a macro. so,
like other *_np() extensions, we check this one also using cmake at
the configure phase.

Reported-by: Tianshan Qu <tianshan@xsky.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-28 23:50:36 +08:00
Kefu Chai
764c2c0e1c cmake: silence CMP0054 warning
see https://gitlab.kitware.com/cmake/cmake/issues/17381 and
a8be8b1b54,
so before the updated cmake is released and packaged. we should
add this setting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-27 12:50:38 +08:00
Kefu Chai
68c52a669c
Merge pull request #19017 from tchaikov/wip-bluestore-without-aio
bluestore,cmake: enable building bluestore without aio

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-11-22 12:26:34 +08:00
Kefu Chai
89a48189ea cmake: only create sysctl file on linux
and check 64bit platform by using the sizeof(void*)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 15:45:03 +08:00
Kefu Chai
586da3da4f
Merge pull request #18544 from ddiss/pid_max
sysctl.d: set kernel.pid_max=4194304

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-11-20 14:37:08 +08:00
Kefu Chai
346fd43bf8 cmake: complain if spdk/pmem is enabled w/o bluestore
and complain if bluestore is enabled w/o any backend found.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 13:04:07 +08:00
Kefu Chai
57e792bcae bluestore: enable building bluestore w/o libaio
KernelDevice is tightly coupled with libaio. more work is needed to
decouple aio from it. but by guarding KernelDevice with HAVE_LIBAIO, we
can enable bluestore on platforms w/o libaio.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-20 11:18:02 +08:00
Kefu Chai
a69bd61b8c cmake: use BuildDPDK if libdpdk not found
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:45:26 +08:00
Kefu Chai
dbe04b4a85 cmake: refactor dpdk related code
* move the check of `USE_CRYPTOPP` to $top_srcdir/CMakeLists.txt
* remove reference of DPDK_LIBRARY, it's defined nowhere
* move the dpdk code to a single place

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-18 10:43:33 +08:00
David Disseldorp
14a0c2a727 sysctl.d: set kernel.pid_max=4194304 on 64-bit systems
For CONFIG_BASE_FULL Linux kernels, the maximum number of proc/thread
IDs is set to 32768 by default. This default limit can be quite easily
hit during recovery on nodes with high OSD counts.
To avoid hitting the pid_max default limit, attempt to configure it to
4194304, which corresponds to the maximum limit possible on 64-bit
CONFIG_BASE_FULL kernels.

Fixes: http://tracker.ceph.com/issues/21929

Signed-off-by: David Disseldorp <ddiss@suse.de>
2017-11-17 14:40:23 +01:00