Commit Graph

1366 Commits

Author SHA1 Message Date
Nathan Cutler
8b5bae19e5
Merge pull request #39813 from smithfarm/wip-49583
rpm: disable parallel compression on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-05 14:28:08 +01:00
Nathan Cutler
30df650e64 rpm: use PMDK system libraries on SUSE
As of a49d1dbb32, when the rbd_rwl_cache and
rbd_ssd_cache bconds are enabled and WITH_SYSTEM_PMDK is disabled (as it is by
default), the RPM build attempts to

    git clone https://github.com/ceph/pmdk.git

but of course that won't work in the OBS, where the build workers have no
Internet connectivity.

Fortunately, the openSUSE/SLE versions targeted by Ceph master and pacific ship
the necessary PMDK libraries as RPM packages.

Fixes: a49d1dbb32
Fixes: https://tracker.ceph.com/issues/49550
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 20:46:20 +01:00
Nathan Cutler
ef0384dc50
Merge pull request #39765 from smithfarm/wip-ncpus-for-obs
rpm: set build parallelism from memory on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
2021-03-03 20:44:20 +01:00
Nathan Cutler
19f492014b rpm: drop old SUSE-specific OOM fix
This fix was needed some years ago, but no longer.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:52:04 +01:00
Nathan Cutler
7efcc72483 rpm: limit build jobs by system memory on SUSE
43b441f9a3 removed a bunch of code which the SUSE
builds were relying on to avoid OOM. This commit brings back that code in
a much-streamlined form: the SUSE-specific %limit_build macro.

This also has the advantage of not breaking the build on older RPMs which only
know about %_smp_mflags, and not the newer %_smp_build_ncpus etc. macros.

Fixes: 43b441f9a3
Fixes: https://tracker.ceph.com/issues/49556
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:52:04 +01:00
Nathan Cutler
a63a94714a rpm: disable parallel compression on SUSE
This code causes the Ceph build in OBS to fail due to OOM, because the typical
setting of %_smp_build_ncpus in the OBS is 16, but available memory is
insufficient to sustain such a high degree of parallelism for the in-memory
compression operation.

Fixes: b50fc9e61c
Fixes: https://tracker.ceph.com/issues/49583
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:48:05 +01:00
Nathan Cutler
f27484e84c rpm/luarocks: simplify conditional and support Leap 15.3
The luarocks conditional had gotten hard to read, and the openSUSE Leap 15.3
build needs lua53 as well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-01 21:20:45 +01:00
Nathan Cutler
ffd202a086 rpm: re-disable SUSE lttng build on z390x
This partially reverts 2b1e646f7a which
mistakenly changed a line inside an "%if 0%{?suse_version}" conditional.

Fixes: 2b1e646f7a
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-02-25 19:16:59 +01:00
Kefu Chai
ad46841715
Merge pull request #39427 from ktdreyer/rpm-whitespace
ceph.spec.in: remove trailing whitespace

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-02-24 17:00:04 +08:00
Kyr Shatskyy
1850dd508f ceph.spec.in: use lua53 for openSUSE 15.2
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2021-02-22 16:57:27 +01:00
Kefu Chai
e08fcf8900
Merge pull request #39552 from smithfarm/wip-49356
rpm: openSUSE Tumbleweed has lua54 now

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-21 19:50:20 +08:00
Jason Dillaman
963d854aba
Merge pull request #39539 from lixiaoy1/enable_pwl
cmake: enable write-back cache in spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-02-20 12:59:04 -05:00
Kefu Chai
71aea12bac ceph.spec.in: s/%cmake/cmake/
this change partially reverts da7030db79
which use %cmake rpm macro in the place of "cmake". but

%cmake sets BUILD_SHARED_LIBS=ON. so quite a few internal libraries
defined using add_library() are now compiled into shared libraries which
are not installed or packagesd. when we are installing the rpm packages
compiled with this option, rpm compiles because the linked libraries are
missing, for instance, `libgmock.so.1.10.0` was compiled as a static
library before da7030db79, and was
included by the test executables. but after that change it's compiled
as a shared library.

so we need to either package the linked shared libraries or just link
against them statically. at this moment, the latter approach is simpler,
albeit larger size of exectuable and dbg symbols.

Fixes: https://tracker.ceph.com/issues/49395
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-20 11:43:30 +08:00
lixiaoy1
a49d1dbb32 cmake: enable write-back cache in spec
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2021-02-19 20:22:50 -05:00
Nathan Cutler
d8ee565b64 rpm: openSUSE Tumbleweed has lua54 now
Until someone figures out an easier way, we are stuck with the minor Lua version
in the package names.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-02-19 09:58:51 +01:00
Kefu Chai
0c079852cd
Merge pull request #39506 from tchaikov/wip-rpm-multi-thread-xz
ceph.spec.in: use multi-threaded xz compression and cleanups

Reviewed-by: David Galloway <dgallowa@redhat.com>
2021-02-18 09:32:27 +08:00
Kefu Chai
144bf31087 ceph.spec.in: use %make_install macro
hide buildroot in the implementation of this macro

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 21:06:18 +08:00
Kefu Chai
0e511973f7 ceph.spec.in: use %make_build instead of "make"
so we can hide the `_smp_mflags` in `%make_build` macro

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 21:06:18 +08:00
Kefu Chai
43b441f9a3 ceph.spec.in: use a macro helper to limit number of jobs
so it can work with the rpm macros better.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 21:06:18 +08:00
Kefu Chai
35d1ba003d ceph.spec.in: specify cmake variable type
for better readablity

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 12:39:55 +08:00
Kefu Chai
da7030db79 ceph.spec.in: use %cmake macro
%cmake passes a bunch of default flags, so we can save some efforts.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 12:39:55 +08:00
Kefu Chai
bd4da136ba ceph.spec.in: use RPM_BUILD_NCPUS for number of jobs
this env variable is defined by /usr/lib/rpm/macros. let's used it
instead of inventing our own version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 12:39:55 +08:00
Kefu Chai
b50fc9e61c ceph.spec.in: use multi-threaded xz compression
so we can build the compressed src rpm and binary rpms with smaller
latency.

see also
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.3_release_notes/technology_previews_installation_and_booting

by default, rpm uses

w9.gzdio

which means: gzip level 9. but since xz is able to compress using
multiple threads, let's switch to xz.

xz's default compression level is 6, but ceph's source and binary
packages are relatively large. let's use a higher compression level.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 11:25:37 +08:00
Ken Dreyer
1a6c3cffaa ceph.spec.in: remove trailing whitespace
RPM's parseSpec() Python method internally strips this whitespace
character.

Tools that process this spec file with parseSpec() and evaluate
RPMTAG_DESCRIPTION cannot match this exact %description string as
written here.

Strip the trailing whitespace so that the RPMTAG_DESCRIPTION header
matches what we've written in the spec.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-02-11 11:01:19 -07:00
Yaakov Selkowitz
0f783b7e3b ceph.spec.in: bump gperftools requirement for ppc64le
gperftools' libprofiler did not build on ppc64le until 2.7.90.
The EPEL 8 package is being updated accordingly.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2021-02-09 11:30:43 -05:00
Yaakov Selkowitz
2b1e646f7a ceph.spec.in: enable tcmalloc and lttng on s390x
The necessary prerequisites are already in RHEL+EPEL 8.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2021-02-09 11:06:52 -05:00
luo.runbing
893924c20d
ceph.spec.in: fix missing backslash for line continuation
it errors while building without lua_packages:

+ -DBOOST_J=102 -DWITH_GRAFANA=ON
/var/tmp/rpm-tmp.aq2X3J: line 91: -DBOOST_J=102: command not found

Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
2021-02-09 09:39:41 +08:00
J. Eric Ivancich
8e03dbce42
Merge pull request #39176 from ivancich/wip-rgw-gap-list
rgw: tooling to locate rgw objects with missing rados components

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-04 15:16:48 -05:00
J. Eric Ivancich
3c5154f010 rgw: add rgw-gap-list-comparator tool
The rgw-gap-list tool can produce a number of false positives when the
cluster is being used during its run. One technique to minimize the
number of false positives is to run the tool twice and look for the
objects that appear in both lists. The rgw-gap-list-comparator tool is
designed to do this comparison.

Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-03 16:57:50 -05:00
J. Eric Ivancich
07b42195fb rgw: add rgw-gap-list tool
Due to a prior bug (pr: 38228) tail rados objects of some RGW objects
could have been incorrectly deleted. This tool is designed to look for
such cases. It essentially does the opposite of rgw-orphan-list,
looking for rados objects that RGW expects to be there, but which are
not to be found.

IMPORTANT: This is very experimental at this point in time, and any
"results" produced should be verified by other means.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-03 16:57:50 -05:00
Sebastian Wagner
46188c4caa
Merge pull request #39007 from mgfritch/cephadm-podman-2.0
cephadm: require podman >= 2.0.0

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-29 10:26:03 +01:00
Michael Fritch
a899227895
spec, debian: recommend podman >= 2.0.2
Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-01-26 08:01:55 -07:00
Jason Dillaman
6aba1c7bba
Merge pull request #39049 from lixiaoy1/fix_spec
spec: enable RWL/SSD caches

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-26 09:46:44 -05:00
lixiaoy1
12ca24fc26 spec: enable RWL/SSD caches
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2021-01-26 04:21:30 -05:00
Venky Shankar
b7acf7fc77 pybind/mgr/mirroring: interface to mirror CephFS directory snapshots
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-19 01:06:43 -05:00
Patrick Donnelly
78a7df1500
Merge PR #38846 into master
* refs/pull/38846/head:
	*: remove legacy ceph_volume_client.py library

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-14 08:10:19 -08:00
Patrick Donnelly
a3db265ad5
*: remove legacy ceph_volume_client.py library
This library is obsolete with the mgr volumes plugin since Nautilus.

The last remaining user of this library was Manila which will be using
the volumes plugin with Pacific and onwards.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-12 06:54:29 -08:00
Venky Shankar
1227752983 systemd: cephfs-mirror systemd unit files
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-12 05:57:32 -05:00
Venky Shankar
42233bb5df doc, man: man page for cephfs-mirror tool
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-12 05:57:32 -05:00
Venky Shankar
d5062c60f6 spec, deb: package cephfs-top utility
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-11 06:15:53 -05:00
Patrick Donnelly
d03064d930
Merge PR #38642 into master
* refs/pull/38642/head:
	rpm/deb/cmake: install mount.fuse.ceph man page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-08 15:12:42 -08:00
Nathan Cutler
ea851e71fd
Merge pull request #38603 from smithfarm/wip-48604
rpm: require smartmontools on SUSE

Reviewed-by: Volker Theile <vtheile@suse.com>
2021-01-04 10:58:54 +01:00
Ken Dreyer
ad85bdf8de rpm/deb/cmake: install mount.fuse.ceph man page
Install the mount.fuse.ceph man page and ship it in the ceph-fuse
packaging along with the corresponding mount.fuse.ceph binary.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2020-12-17 16:30:07 -07:00
Yuval Lifshitz
c1af849a0d
Merge pull request #37924 from yuvalif/allow_packages_in_lua
rgw/lua: support packages via luarocks
2020-12-17 19:53:00 +02:00
Nathan Cutler
c7e160a381 rpm: require smartmontools on SUSE
SUSE container images are built using a process that excludes dependencies that
are merely recommended.

Fixes: https://tracker.ceph.com/issues/48604
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-12-16 11:46:46 +01:00
Kefu Chai
5af17733c7 rpm: package crimson-store-nbd in ceph-test package
when building with seastar enabled, package crimson-store-ndb in
"ceph-test" package. "crimson-store-nbd" is a new tool for testing
seastore using nbd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-16 15:55:48 +08:00
Josh Durgin
8a74a937dd
Merge pull request #31358 from ideepika/wip-jaegertracing-in-ceph
jaegertracing build/ops integration

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-15 16:29:26 -08:00
Deepika Upadhyay
85ca7b36a0 jaeger: make tracing default turned off, until teuthology integration
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 10:08:09 +00:00
Deepika Upadhyay
3c5abab7ea ceph.spec.in: refactor s/Cephs/Ceph
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 08:35:22 +00:00
Deepika Upadhyay
4b69d3a184 jaeger, build/op: add jaeger and it's dependencies as submodules
* This commit introduces Jaegertracing library as package libjaeger,
  pickwhich would be consumed by other ceph pacakges such as ceph-common0

* adds the following dependencies, which would be build from source
  using ExternalProjectHelper.cmake +IncludeJaeger.cmake +
  Build<package>.cmake scripts:

  jaegertracing: v0.6.0 [added as a submodule]
  opentracing: v1.6.0 [added as a submodule]
  thrift: 0.13.0 [added as a submodule]
  yaml-cpp: 0.6.0
  json(optional)

* updates Boost to be installed instead of being build only, because
  jaegertracing them during their build process.

* ceph.spec.in: introduces a default enabled jaeger packaging option,
  which could be disabled using --without-jaeger flag during rpmbuild

* note: libjaeger package if enabled will be a dependency on ceph-common, ceph-mon, rgw_common and transitively will be a dependency for modules that have them as  a dependency.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 07:55:46 +00:00