Commit Graph

1311 Commits

Author SHA1 Message Date
Kefu Chai
e96c45d9d2
Merge pull request #37751 from changchengx/nasm_avx512
common: use nasm to build isa-l and isa-l_crypto to prepare for AVX512

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-31 11:41:19 +08:00
Kefu Chai
d162c00c66
Merge pull request #37788 from tchaikov/wip-zbd
rpm,cmake: s/WITH_LIBZBD/WITH_ZBD/ and enable ZBD on demand

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
2020-10-30 11:17:56 +08:00
Changcheng Liu
497b4d6f72 cmake: remove yasm since it's been replaced by nasm
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 12:36:20 +08:00
Changcheng Liu
a944232ad4 install-deps.sh: install nasm for isa-l library
nasm support build isa-l:AVX512 algorithm implementation while yasm
doens't support it. Install nasm assembler to build isa-l
refer to: https://github.com/yasm/yasm/issues/101

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 12:35:55 +08:00
Kefu Chai
98c06b534f rpm: pass -DWITH_ZBD=ON to cmake if zbd is enabled
this is a leftover of fd1b02a341

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-29 11:19:40 +08:00
Kefu Chai
b28174f13f
Merge pull request #37761 from tchaikov/wip-ceph.spec-with-seastar
ceph.spec.in: install gcc-toolset on RHEL only

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-10-28 14:43:49 +08:00
Adam C. Emerson
c9d20eb1a0 rpm: Remove use of no-longer-existing WITH_BOOST_CONTEXT option
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-23 11:21:53 -04:00
Kefu Chai
36759b5363 ceph.spec.in: install gcc-toolset on RHEL only
there is no need to use GTS for compiling crimson, as the GCC-10 shipped by
fedora32 is good enough.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-23 10:49:08 +08:00
Nathan Cutler
b8168d9e8a Revert "spec: Podman (temporarily) requires apparmor-abstractions on suse"
This reverts commit 070e5c3e35.

This commit was working around a bug that was present in old (pre-1.8.0)
versions of podman. Since we are no longer running cephadm with versions
that old, we can safely drop this workaround.

Fixes: https://tracker.ceph.com/issues/47862
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-10-15 11:21:21 +02:00
Nathan Cutler
f5fc346bb2 Revert "rpm: on SUSE, podman is required for cephadm to work"
This reverts commit 009ade0c3e.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-10-15 11:21:21 +02:00
Kefu Chai
ade1a6e394
Merge pull request #37455 from smithfarm/wip-spec-file-cleanup
rpm: three spec file cleanups

Reviewed-by: Tim Serong <tserong@suse.com>
2020-10-14 15:43:30 +08:00
Venky Shankar
7523aef6e8 mgr/stats: mds performance stats module
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Nathan Cutler
3b53003f01 rpm: cleanup: drop useless conditional block in %postun base
The "meat" of this conditional was ripped out by
328807f80b, leaving only an empty shell
behind.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-09-29 15:54:19 +02:00
Nathan Cutler
f69aa5abfb rpm: cleanup: drop %service_del_postun_without_restart
SUSE needs %service_del_postun (with or without restart) *only* if there
is a possibility that the RPM containing the unit file will be upgraded
from a version that packaged SysVinit scripts instead of systemd unit
files. (Which is not the case here.)

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-09-29 15:54:12 +02:00
Franck Bui
7d99e786df rpm: cleanup: drop use of DISABLE_RESTART_ON_UPDATE
This SUSE-specific variable is deprecated and use of
%service_del_postun_without_restart macro should be preferred these
days.

Signed-off-by: Franck Bui <fbui@suse.com>
2020-09-29 15:53:29 +02:00
Kefu Chai
a42fd5c21e
Merge pull request #37281 from tchaikov/wip-fmt
rpm: add fmt-devel as build deps

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-29 08:14:34 +08:00
Kefu Chai
4aced0cba9
Merge pull request #37211 from tchaikov/wip-zbd
rpm,install-deps,run-make-check: enable WITH_ZBD build on demand

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-22 09:42:09 +08:00
Kefu Chai
fd779c3600 rpm: add fmt-devel as build deps
since libfmt > 5.0.0 is packaged by CentOS/RHEL8 and openSUSE/SLE
we can just use it instead of our own version. also, since libfmt is
used out side of crimson project, no need to guard it using "with
seastar" anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-22 01:01:51 +08:00
Yaarit Hatuka
d5528a7e8e ceph.spec.in, debian/control: add smartmontools and nvme-cli dependencies
These packages are needed in order to scrape device health metrics from
devices used by OSD and MON daemons.

smartmontools' smartctl is what we use in order to scrape devices' SMART
attributes and general health metrics.
In addition, we use nvme-cli tool on NVMe devices, which fetches
vendor specific NVMe related health metrics.

Ceph rely on these tools for proper functioning of the underlying layers
of devicehealth mgr module, and other mgr modules which use devicehealth
functionality (such as diskprediction_local, telemetry, dashboard).

Essentially, most of devicehealth commands rely on proper functioning of
smartctl, otherwise they lack the device health metrics.

For example, in case smartctl is missing, the commands:
    ceph device scrape-daemon-health-metrics <who>
    ceph device scrape-health-metrics [<devid>]
will not be able to scrape health metrics, and the command:
    ceph device predict-life-expectancy <devid>
will not provide any meaningful output (since there are no metrics).

In short, when we scrape a device by its daemon (be it an OSD or a MON):
  ceph device scrape-daemon-health-metrics <who>
The devicehealth module command eventually invokes a
block_device_get_metrics() call in either osd/OSD.cc or mon/Monitor.cc,
which wraps calls to both
    block_device_run_smartctl()       (spawns smartctl)
    block_device_run_vendor_nvme()    (spawns nvme)
in common/blkdev.cc.

Minimum version requirements:
'smartmontools' is the package name, which contains two utility
programs: 'smartd' and 'smartctl'. Ceph uses the latter.

Version 6.7 of smartctl first introduced the --json option (beta), which
allows to output the metrics in a JSON format. Since then a few
adjustments were made and the feature officially launched in smartctl
version 7.0.
Since we rely on the JSON format to process the metrics, we must have
smartmontools' smartctl version >= 7.

That said, we choose not to specify smartmontools version here on
purpose, since there might be a scenario where:
We specified smartmontools version to be >= 7.
smartmontools 7 is not available yet in rhel 8 / centos 8.
A user installs via rpm ceph-osd, for example.
smartmontools will not be installed (since version >= 7 is not available
in this repo yet).
Then the user upgrades to 8.3 (which should have smartmontools >= 7),
but smartmontools will not get upgraded (since it's not installed).
In the scenario where we do not specify a version, smartmontools 6.6
will be installed, but it will be upgraded to >= 7 when a user upgrades
(and if it's a fresh installation - version >= 7 would be installed
anyway).

nvme-cli does not have a minimum version.

We use 'Recommends' for both rpm and deb packages since we do not want
the installation to fail in case of conflicts. 'Recommends' weakens the
dependency to be installed in case possible, but ignores it in cases of
conflicts with other dependencies.

It's worth mentioning that smartmontools and nvme-cli dependencies exist
in ceph-container builds.
We add them here for the cases of bare metal installations.

In the future we will add a separate package (with smartmontools and
nvme-cli dependencies) that can be installed on any node (running
rbd-mirror, rgw, mds, mgr, etc.), in order to be able to collect the
health metrics of its devices and offer their life expectancy
prediction.

Fixes: https://tracker.ceph.com/issues/47479
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2020-09-18 03:25:56 +00:00
Kefu Chai
fd1b02a341 rpm: add --with-zbd option
libzbd-devel is not available in any distros official repo at the time
of writing. but we packaged it for RHEL8/CentOS8. and uploaded its
packages to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/. to
build test it to avoid the bitrot of ZBD bluestore backend at a minimal
level. but we cannot do more than this. because

- the kernel shipped by RHEL/CentOS8 does not have zoned block device
  support enabled. see https://zonedstorage.io/distributions/linux/
- and we don't have zoned block device for testing in our lab.

so, in this change, "--with-zbd" option is added so we can pull
in the libzbd-devel package on demand.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-17 14:45:11 +08:00
Nathan Cutler
839c8ec6ba
Merge pull request #36994 from ktdreyer/boost-context-s390x
rpm: enable WITH_BOOST_CONTEXT for s390x

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-09-07 12:39:12 +02:00
Sebastian Wagner
b54e270930
Merge pull request #36995 from smithfarm/wip-47305
rpm: on SUSE, podman is required for cephadm to work

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-09-07 10:39:25 +02:00
Kefu Chai
bc4255ba2c
Merge pull request #36972 from smithfarm/wip-47112
rpm,deb: drop /etc/sudoers.d/cephadm

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-07 14:44:44 +08:00
Ken Dreyer
121cde8c0a rpm: enable WITH_BOOST_CONTEXT for s390x
We've upgraded to Boost version 1.73
(1169dee259), and this version provides
working support for boost::context for s390x. Enable this in the RPM
packaging.

Fixes: https://tracker.ceph.com/issues/47304
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2020-09-04 09:52:53 -06:00
Nathan Cutler
009ade0c3e rpm: on SUSE, podman is required for cephadm to work
On SUSE, cephadm is tested with podman only.

Fixes: https://tracker.ceph.com/issues/47305
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-09-04 17:51:07 +02:00
Nathan Cutler
c1783d3d8f rpm,deb: drop /etc/sudoers.d/cephadm
Current behavior (without this patch) is:

1. cephadm package installs cephadm at /usr/sbin/cephadm
2. cephadm package installs /etc/sudoers.d/cephadm
3. !!! BUT this file refers to a non-existent executable (/usr/bin/cephadm) !!!
4. the PR that introduced this sudoers file (and this discrepancy) was merged in 2019
5. nobody noticed the discrepancy until now

My conclusion: the file /etc/sudoers.d/cephadm is not needed for cephadm to
work.

Fixes: https://tracker.ceph.com/issues/47112
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-09-03 16:31:30 +02:00
Kefu Chai
9ea12cea31
Merge pull request #36932 from theMarix/patch-1
rpm: Fix typo in summary of prometheus-alerts package

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-03 22:05:06 +08:00
Sebastian Wagner
7f21e1ee02
Merge pull request #36949 from sebastian-philipp/mgr-re-add_osd_support
Revert "mgr/osd_support: remove module and all traces"

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2020-09-03 14:41:25 +02:00
Sebastian Wagner
43f1bdb6ef Revert "mgr/osd_support: remove module and all traces"
This reverts commit a55c1dd0fa.

Kept to keep upgrades from older point releases working.
This module can be removed as soon as we no longer
support upgrades from old octopus point releases.

Revert "build/debian: remove osd_support"

This reverts commit 8ff2824beb.

Fixes: https://tracker.ceph.com/issues/47109
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-09-02 16:07:49 +02:00
Venky Shankar
ad4e611924 ceph.spec, debian: changes for building cephfs-mirror daemon
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-09-02 06:50:08 -04:00
Matthias Bach
4d1e30470c rpm: Fix typo in summary of prometheus-alerts package
Signed-off-by: Matthias Bach <marix@marix.org>
2020-09-01 21:18:17 +02:00
Venky Shankar
29a6f72131 spec: include snap_schedule in ceph spec file
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-08-27 15:55:45 +02:00
Kefu Chai
2644766d0f pybind/mgr/diskprediction_cloud: remove mgr-diskprediction-cloud
the service offered by https://www.diskprophet.com is not accessible
anymore. before we have a replacement of it, let's drop
mgr-diskprediction-cloud. and remove its packaging.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-19 11:08:38 +08:00
Kefu Chai
5c038b99b3 ceph.spec.in: use gcc-toolset-9 for building crimson
since seastar dropped support from C++14, we have to move to a compiler
with a decent C++17 support.

in this change, gcc-toolset-9 is used for compiling ceph if seastar is
enabled. use version >= 9.2.1-2.2, because of
https://bugzilla.redhat.com/show_bug.cgi?id=1853900

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-15 17:43:40 +08:00
Patrick Donnelly
305e1b3445
Merge PR #32731 into master
* refs/pull/32731/head:
	mgr/mds_autoscaler: plugin to configure mds instances

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-10 12:47:47 -07:00
Milind Changire
f69abe6291 mgr/mds_autoscaler: plugin to configure mds instances
mgr plugin to deploy and configure MDSs in response to degraded file system

MDS instance management as per changes to:
* 'max_mds' option
* 'standby_count_wanted' option
* mds liveness and transitions from standby to active

mds_autoscaler plugin test credit goes to Sebastian Wagner.

Fixes: https://tracker.ceph.com/issues/40929
Signed-off-by: Milind Changire <mchangir@redhat.com>
2020-08-07 17:09:10 +05:30
Jan Fajerski
42f47baab6
Merge pull request #36372 from jan--f/c-v-fix-package-common-dep
ceph-volume: dependency on python-ceph-common
2020-08-05 13:37:50 +02:00
Jan Fajerski
cb432fe41d ceph-volume: dependency on python-ceph-common
Since e5b585d15d ceph-volume depends on
python-ceph-common. This commit introduces this dependency for the
ceph-osd rpm (which includes ceph-volume) and installs the dependency
for tox runs.

Fixes: https://tracker.ceph.com/issues/46772
Fixes: e5b585d15d

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-03 17:14:41 +02:00
Joshua Schmid
a55c1dd0fa mgr/osd_support: remove module and all traces
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-07-31 10:11:45 +02:00
Sebastian Wagner
0ada965faf python-common, spec.in: Add missing pyyaml dependency
Fixes: cd6a488ab2
Fixes: https://tracker.ceph.com/issues/46759
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-07-30 09:55:08 +02:00
Kefu Chai
db20cfd095 ceph.spec.in: cull _FORTIFY_SOURCE macro from CXXFLAGS for seastar
seastar uses setjmp() and longjmp() to implement coroutine, but
longjmp() is defined as ____longjmp_chk() by GCC if _FORTIFY_SOURC is
defined. ____longjmp_chk() simply bails out with an error message if
the dest stack pointer is higher than the src stack pointer, or the dest
stack pointer is not in the sigaltstack. in the case of seastar, the dst
%sp is not necessarily higher than src stack pointer, and it's not
handling a signal for switching the thread context. that's why we have
the "longjmp causes uninitialized stack frame" error when running
crimson-osd on RHEL/CentOS 8 using the prebuilt rpm packages.

the optflags rpm macro adds -D_FORTIFY_SOURCE=2 to CFLAGS and CXXFLAGS,
so even seastar tries to pass -U_FORTIFY_SOURCE to GCC, there is chance
that cmake append CXXFLAGS at the end of the option list passed to GCC.
and this renders seastar's attempt to undefine _FORTIFY_SOURCE useless.

another way to address this issue is to undefine this macro in
seastar:src/core/thread.cc. but since seastar tries neutralize the macro
in its cmake script instead of source file, i assume they have their
considerations. let's drop it in the rpm recipe instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-23 16:19:56 +08:00
Kefu Chai
3902a5eede rpm,deb: drop python3-six dependency
it's not used anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-07-22 11:55:50 +08:00
Kefu Chai
8f1620b3c9 rpm: package crimson-osd in ceph-osd
in hope to simplify the enabling of crimson-osd. as we will use cephadm
as the way for deploying crimson-osd in future. it's unlikely we will
have classic osd and crimson osd co-located in the same system. so
this is most simple way to enable crimson-osd with minimum efforts.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-30 16:14:18 +08:00
Kefu Chai
338f47fe94 ceph.spec.in: Requires python3-scikit-learn only on fedora and suse
so far, python3-scikit-learn is not packaged by RHEL8/CentOS8 or EPEL8.

see
https://src.fedoraproject.org/rpms/python-scikit-learn/branches?branchname=master

this change should address the regression introduced by
d13b441cdb

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-25 18:19:55 +08:00
Kefu Chai
db4d3ca3aa
Merge pull request #35648 from ideepika/fix-44948
ceph.spec: fix mgr diskprediction_local dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-06-25 16:10:05 +08:00
Deepika Upadhyay
62093a7eb2 ceph.spec: drop support for el7 for rpm builds
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-06-23 16:35:31 +05:30
Deepika Upadhyay
d13b441cdb ceph.spec: fix mgr diskprediction_local dependency
fixes: https://tracker.ceph.com/issues/44948
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-06-23 14:58:51 +05:30
Neal Gompa
ed6552d506 rpm: Drop override on %_libexecdir
We no longer need it, as distributions are converging on /usr/libexec
since FHS 3.0 has specified it as the acceptable path for libexecdir.

Fedora has been using /usr/libexec since the beginning, and
openSUSE has accepted software installed into /usr/libexec since
September 2019 in openSUSE:Factory. openSUSE Leap 15.x still uses
/usr/lib for %_libexecdir, but the rest of the spec should handle the
change gracefully.

Signed-off-by: Neal Gompa <ngompa13@gmail.com>
2020-06-08 13:15:04 -04:00
Kefu Chai
491fa555f9 ceph.spec.in: use %{_presetdir} for preset settings
the systemd preset settings should not be located under /usr/libexec
```
$ rpm --eval "%{_presetdir}"
/usr/lib/systemd/system-preset
```

see also
https://www.freedesktop.org/software/systemd/man/systemd.preset.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-08 13:15:04 -04:00
Neal Gompa
7c8e5de9c7 rpm: Add missing CMake parameter to path to systemd unit dir
This mimics the setting passed to CMake in other package builds,
and ensures that systemd units are installed into the right place.

Signed-off-by: Neal Gompa <ngompa13@gmail.com>
2020-06-08 12:55:28 -04:00