Commit Graph

1375 Commits

Author SHA1 Message Date
Kefu Chai
01a7ecaba2
Merge pull request #40163 from ktdreyer/resource-agents-noarch
rpm: ceph-resource-agents package is noarch

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-18 11:02:22 +08:00
Ken Dreyer
79808963a7 rpm: ceph-resource-agents package is noarch
The ceph-resource-agents package contains an architecture-independent
bash script and parent directories. There are no architecture-dependent
files here, so we can use a single noarch RPM across all host
architectures.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-03-16 16:18:11 -06:00
Kyr Shatskyy
a435380dc3 ceph.spec.in: use ninja instead of ninja-build for openSUSE
And since the dependency is now distro-conditional, moving it down
to the 'distro-conditional make check dependencies' section.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2021-03-16 12:45:26 +01:00
Kefu Chai
75b58115ea ceph.spec: build with system libpmem on fedora and el8
* build with WITH_SYSTEM_PMDK=ON on fedora, as f32 and f33 ship
  libpmem1.8 and libpmem1.9 respectively. and we need libpmem v1.7
* build with WITH_SYSTEM_PMDK=ON on el8, as el8 and CentOS8 AppStream
  ships libpmem v1.6,

quote from nvml.spec:

> By design, PMDK does not support any 32-bit architecture.
> Due to dependency on some inline assembly, PMDK can be compiled only
> on these architectures:
> - x86_64
> - ppc64le (experimental)
> - aarch64 (unmaintained, supporting hardware doesn't exist?)

so far, only x86_64 and ppc64le packages are built.

see also,
https://src.fedoraproject.org/rpms/nvml/blob/rawhide/f/nvml.spec

this change addresses a regression introduced by
a49d1dbb32

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-12 23:04:11 +08:00
Kefu Chai
80f8f0956c ceph.spec: include ninja as BuildRequires
to ease the build for developers using SUSE, Fedora, CentOS or RHEL.
so install-deps.sh can install ninja for them.

Fixes: https://tracker.ceph.com/issues/49694
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-10 18:19:49 +08:00
Kefu Chai
6de97d4434 ceph.spec: prepare for out-of-source build
see also
https://fedoraproject.org/wiki/Changes/CMake_to_do_out-of-source_builds

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-09 19:13:19 +08:00
Kefu Chai
624be364c6
Merge pull request #39909 from smithfarm/wip-revert-make-build-on-suse
rpm: refrain from using %make_build macro on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-09 16:58:58 +08:00
Nathan Cutler
40cbd283bd rpm: refrain from using %make_build macro on SUSE
The recently merged commit 0e511973f7 replaced

    make %{_smp_mflags}

with

    %make_build

for the stated purpose of hiding the %_smp_mflags macro in a higher-level macro.

But, on SUSE, the higher-level macro (%make_build) expands to:

    make -O %{_smp_mflags}

The addition of the -O flag makes the build considerably slower and increases
the memory requirement. The exact reason for this is unknown - possibly it's due
to a bug in make, although the same slowness was observed with ninja as well.
In any event, this is a deal-breaker when building in the OBS, because the build
infrastructure there is optimized for builds that do not require huge amounts of
memory and we would rather have a fast build with mixed up compiler messages
than a very slow one with synced compiler messages.

Fixes: 0e511973f7
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-08 11:52:09 +01:00
Marcus Watts
8afd92425b rgw/kms/vault - need libicu to make canonical json for encryption contexts.
for encryption, aws s3 provides an "encryption" context to vary per-object
keys.  The encryption context is a base64 encoded json structure, which
must be converted to a determinstic form -- "canonical json".  This
requires converting all strings to a normalized canonical form: "utf-8 nfc",
it also requires thta keys in objects be sorted in a fixed order; so some
form of sorting based on nfc.

It turns out that libicu was the best way to produce utf-8 nfc (boost also
provides a mechanism, but it has many quirks).  So, here are the hooks
to pull the system libicu into the build.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2021-03-05 17:15:24 -05:00
Nathan Cutler
8b5bae19e5
Merge pull request #39813 from smithfarm/wip-49583
rpm: disable parallel compression on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-05 14:28:08 +01:00
Nathan Cutler
30df650e64 rpm: use PMDK system libraries on SUSE
As of a49d1dbb32, when the rbd_rwl_cache and
rbd_ssd_cache bconds are enabled and WITH_SYSTEM_PMDK is disabled (as it is by
default), the RPM build attempts to

    git clone https://github.com/ceph/pmdk.git

but of course that won't work in the OBS, where the build workers have no
Internet connectivity.

Fortunately, the openSUSE/SLE versions targeted by Ceph master and pacific ship
the necessary PMDK libraries as RPM packages.

Fixes: a49d1dbb32
Fixes: https://tracker.ceph.com/issues/49550
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 20:46:20 +01:00
Nathan Cutler
ef0384dc50
Merge pull request #39765 from smithfarm/wip-ncpus-for-obs
rpm: set build parallelism from memory on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
2021-03-03 20:44:20 +01:00
Nathan Cutler
19f492014b rpm: drop old SUSE-specific OOM fix
This fix was needed some years ago, but no longer.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:52:04 +01:00
Nathan Cutler
7efcc72483 rpm: limit build jobs by system memory on SUSE
43b441f9a3 removed a bunch of code which the SUSE
builds were relying on to avoid OOM. This commit brings back that code in
a much-streamlined form: the SUSE-specific %limit_build macro.

This also has the advantage of not breaking the build on older RPMs which only
know about %_smp_mflags, and not the newer %_smp_build_ncpus etc. macros.

Fixes: 43b441f9a3
Fixes: https://tracker.ceph.com/issues/49556
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:52:04 +01:00
Nathan Cutler
a63a94714a rpm: disable parallel compression on SUSE
This code causes the Ceph build in OBS to fail due to OOM, because the typical
setting of %_smp_build_ncpus in the OBS is 16, but available memory is
insufficient to sustain such a high degree of parallelism for the in-memory
compression operation.

Fixes: b50fc9e61c
Fixes: https://tracker.ceph.com/issues/49583
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:48:05 +01:00
Nathan Cutler
f27484e84c rpm/luarocks: simplify conditional and support Leap 15.3
The luarocks conditional had gotten hard to read, and the openSUSE Leap 15.3
build needs lua53 as well.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-01 21:20:45 +01:00
Nathan Cutler
ffd202a086 rpm: re-disable SUSE lttng build on z390x
This partially reverts 2b1e646f7a which
mistakenly changed a line inside an "%if 0%{?suse_version}" conditional.

Fixes: 2b1e646f7a
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-02-25 19:16:59 +01:00
Kefu Chai
ad46841715
Merge pull request #39427 from ktdreyer/rpm-whitespace
ceph.spec.in: remove trailing whitespace

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-02-24 17:00:04 +08:00
Kyr Shatskyy
1850dd508f ceph.spec.in: use lua53 for openSUSE 15.2
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2021-02-22 16:57:27 +01:00
Kefu Chai
e08fcf8900
Merge pull request #39552 from smithfarm/wip-49356
rpm: openSUSE Tumbleweed has lua54 now

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-21 19:50:20 +08:00
Jason Dillaman
963d854aba
Merge pull request #39539 from lixiaoy1/enable_pwl
cmake: enable write-back cache in spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-02-20 12:59:04 -05:00
Kefu Chai
71aea12bac ceph.spec.in: s/%cmake/cmake/
this change partially reverts da7030db79c5dbd480040eb249a76638caf52707
which use %cmake rpm macro in the place of "cmake". but

%cmake sets BUILD_SHARED_LIBS=ON. so quite a few internal libraries
defined using add_library() are now compiled into shared libraries which
are not installed or packagesd. when we are installing the rpm packages
compiled with this option, rpm compiles because the linked libraries are
missing, for instance, `libgmock.so.1.10.0` was compiled as a static
library before da7030db79c5dbd480040eb249a76638caf52707, and was
included by the test executables. but after that change it's compiled
as a shared library.

so we need to either package the linked shared libraries or just link
against them statically. at this moment, the latter approach is simpler,
albeit larger size of exectuable and dbg symbols.

Fixes: https://tracker.ceph.com/issues/49395
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-20 11:43:30 +08:00
lixiaoy1
a49d1dbb32 cmake: enable write-back cache in spec
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2021-02-19 20:22:50 -05:00
Nathan Cutler
d8ee565b64 rpm: openSUSE Tumbleweed has lua54 now
Until someone figures out an easier way, we are stuck with the minor Lua version
in the package names.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-02-19 09:58:51 +01:00
Kefu Chai
0c079852cd
Merge pull request #39506 from tchaikov/wip-rpm-multi-thread-xz
ceph.spec.in: use multi-threaded xz compression and cleanups

Reviewed-by: David Galloway <dgallowa@redhat.com>
2021-02-18 09:32:27 +08:00
Kefu Chai
144bf31087 ceph.spec.in: use %make_install macro
hide buildroot in the implementation of this macro

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 21:06:18 +08:00
Kefu Chai
0e511973f7 ceph.spec.in: use %make_build instead of "make"
so we can hide the `_smp_mflags` in `%make_build` macro

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 21:06:18 +08:00
Kefu Chai
43b441f9a3 ceph.spec.in: use a macro helper to limit number of jobs
so it can work with the rpm macros better.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 21:06:18 +08:00
Kefu Chai
35d1ba003d ceph.spec.in: specify cmake variable type
for better readablity

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 12:39:55 +08:00
Kefu Chai
da7030db79 ceph.spec.in: use %cmake macro
%cmake passes a bunch of default flags, so we can save some efforts.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 12:39:55 +08:00
Kefu Chai
bd4da136ba ceph.spec.in: use RPM_BUILD_NCPUS for number of jobs
this env variable is defined by /usr/lib/rpm/macros. let's used it
instead of inventing our own version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 12:39:55 +08:00
Kefu Chai
b50fc9e61c ceph.spec.in: use multi-threaded xz compression
so we can build the compressed src rpm and binary rpms with smaller
latency.

see also
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.3_release_notes/technology_previews_installation_and_booting

by default, rpm uses

w9.gzdio

which means: gzip level 9. but since xz is able to compress using
multiple threads, let's switch to xz.

xz's default compression level is 6, but ceph's source and binary
packages are relatively large. let's use a higher compression level.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-17 11:25:37 +08:00
Ken Dreyer
1a6c3cffaa ceph.spec.in: remove trailing whitespace
RPM's parseSpec() Python method internally strips this whitespace
character.

Tools that process this spec file with parseSpec() and evaluate
RPMTAG_DESCRIPTION cannot match this exact %description string as
written here.

Strip the trailing whitespace so that the RPMTAG_DESCRIPTION header
matches what we've written in the spec.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-02-11 11:01:19 -07:00
Yaakov Selkowitz
0f783b7e3b ceph.spec.in: bump gperftools requirement for ppc64le
gperftools' libprofiler did not build on ppc64le until 2.7.90.
The EPEL 8 package is being updated accordingly.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2021-02-09 11:30:43 -05:00
Yaakov Selkowitz
2b1e646f7a ceph.spec.in: enable tcmalloc and lttng on s390x
The necessary prerequisites are already in RHEL+EPEL 8.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2021-02-09 11:06:52 -05:00
luo.runbing
893924c20d
ceph.spec.in: fix missing backslash for line continuation
it errors while building without lua_packages:

+ -DBOOST_J=102 -DWITH_GRAFANA=ON
/var/tmp/rpm-tmp.aq2X3J: line 91: -DBOOST_J=102: command not found

Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
2021-02-09 09:39:41 +08:00
J. Eric Ivancich
8e03dbce42
Merge pull request #39176 from ivancich/wip-rgw-gap-list
rgw: tooling to locate rgw objects with missing rados components

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-04 15:16:48 -05:00
J. Eric Ivancich
3c5154f010 rgw: add rgw-gap-list-comparator tool
The rgw-gap-list tool can produce a number of false positives when the
cluster is being used during its run. One technique to minimize the
number of false positives is to run the tool twice and look for the
objects that appear in both lists. The rgw-gap-list-comparator tool is
designed to do this comparison.

Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-03 16:57:50 -05:00
J. Eric Ivancich
07b42195fb rgw: add rgw-gap-list tool
Due to a prior bug (pr: 38228) tail rados objects of some RGW objects
could have been incorrectly deleted. This tool is designed to look for
such cases. It essentially does the opposite of rgw-orphan-list,
looking for rados objects that RGW expects to be there, but which are
not to be found.

IMPORTANT: This is very experimental at this point in time, and any
"results" produced should be verified by other means.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-03 16:57:50 -05:00
Sebastian Wagner
46188c4caa
Merge pull request #39007 from mgfritch/cephadm-podman-2.0
cephadm: require podman >= 2.0.0

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-29 10:26:03 +01:00
Michael Fritch
a899227895
spec, debian: recommend podman >= 2.0.2
Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-01-26 08:01:55 -07:00
Jason Dillaman
6aba1c7bba
Merge pull request #39049 from lixiaoy1/fix_spec
spec: enable RWL/SSD caches

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-26 09:46:44 -05:00
lixiaoy1
12ca24fc26 spec: enable RWL/SSD caches
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2021-01-26 04:21:30 -05:00
Venky Shankar
b7acf7fc77 pybind/mgr/mirroring: interface to mirror CephFS directory snapshots
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-19 01:06:43 -05:00
Patrick Donnelly
78a7df1500
Merge PR #38846 into master
* refs/pull/38846/head:
	*: remove legacy ceph_volume_client.py library

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-14 08:10:19 -08:00
Patrick Donnelly
a3db265ad5
*: remove legacy ceph_volume_client.py library
This library is obsolete with the mgr volumes plugin since Nautilus.

The last remaining user of this library was Manila which will be using
the volumes plugin with Pacific and onwards.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-12 06:54:29 -08:00
Venky Shankar
1227752983 systemd: cephfs-mirror systemd unit files
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-12 05:57:32 -05:00
Venky Shankar
42233bb5df doc, man: man page for cephfs-mirror tool
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-12 05:57:32 -05:00
Venky Shankar
d5062c60f6 spec, deb: package cephfs-top utility
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-11 06:15:53 -05:00
Patrick Donnelly
d03064d930
Merge PR #38642 into master
* refs/pull/38642/head:
	rpm/deb/cmake: install mount.fuse.ceph man page

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-08 15:12:42 -08:00