Commit Graph

1460 Commits

Author SHA1 Message Date
Yuri Weinstein
6e408e557b
Merge pull request #43993 from SMIL-Infra/cmake-systemd-cleanup
cmake: cleanups about systemd and install

Reviewed-by: Kefu Chai <kchai@redhat.com>
2022-01-07 15:35:42 -08:00
Alfonso Martínez
9600c73d48
Merge pull request #43763 from rhcs-dashboard/cephadm_box
cephadm/box: Cephadm Docker in Docker dev box

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-01-04 17:22:56 +01:00
Josh Durgin
facf0d35b1
Merge pull request #43598 from ideepika/wip-opentelemetry
migrate from using opentracing-cpp to opentelemetry-cpp static as distributed tracing API

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-12-22 07:25:13 -08:00
Tim Serong
ff7b5d727a
Merge pull request #44269 from SUSE/wip-fix-mgr-cephadm-suse-cherrypy-requires
ceph.spec.in: fix mgr-cephadm CherryPy requirement for SUSE builds

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-12-16 16:15:13 +11:00
Pere Diaz Bou
f31f53b8c4 cephadm/box: Cephadm Docker in Docker dev box
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-12-15 15:36:33 +01:00
Tim Serong
eca838dfab ceph.spec.in: fix mgr-cephadm CherryPy requirement for SUSE builds
Commit 78983ad0d0 added cherrypy to ceph-mgr-cephadm's Requires,
but this needs to be split out into distro-specific sections due
to subtle/irritating naming differences.

Fixes: 78983ad0d0
Signed-off-by: Tim Serong <tserong@suse.com>
2021-12-13 15:50:45 +11:00
Deepika Upadhyay
ab9470fa94 ceph.spec: remove libjaeger package
we no longer need to package libjaeger, as we would be using
libopentelemetry static libraries(until we support shared).

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-11-24 19:59:18 +05:30
胡玮文
8e49d36170 cmake: fix default systemd unit install dir
The original usage of CMAKE_INSTALL_LIBEXECDIR is incorrect.

Also try to get the path from pkg-config

Signed-off-by: 胡玮文 <huww98@outlook.com>
2021-11-18 15:07:26 +08:00
Adam C. Emerson
447564e4db build: Remove LevelDB support
Whereas new LevelDB packages are build with -fno-rtti and break our
attempts to compile against them,

and whereas LevelDB has been deprecated for some time, with the Ceph
team expending great effort to migrating OSDs off of it,

let it be therefore removed.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2021-11-01 12:42:17 -04:00
Sage Weil
b9fcbf2c03 Revert "Merge pull request #41325 from rhcs-dashboard/wip-cephadm_box-master"
This reverts commit 78d6ac0d35, reversing
changes made to 9a8de97f7a.

This broke cephadm (infers config when it should not)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-29 09:53:43 -05:00
Pere Diaz Bou
87aeb1c8ae cephadm/box: add cluster ssh key to hosts
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 17:12:55 +02:00
Pere Diaz Bou
772db8c1b6 cephadm/box: create osds with cephadm and cleanups
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-18 17:12:55 +02:00
Nathan Cutler
1a3a1b0958 rpm: enable system_pmdk bcond for SUSE builds
ecb8d2cae2 disabled the system_pmdk bcond for all
build targets based on the fact that pmdk 1.10 was not available on any of them.

Now that openSUSE Tumbleweed ships pmdk 1.11, we re-enable the system_pmdk bcond
to fix the master build for openSUSE Tumbleweed.

Since openSUSE Tumbleweed is the *only* SUSE build target master supports, there
is no need for greater granularity in the distro conditional here.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-09-27 12:05:45 +02:00
Brad Hubbard
e7a8d094bc
Merge pull request #43291 from badone/wip-missing-hostname-dependency
rpm, debian: We depend on 'hostname'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-27 14:21:14 +10:00
Brad Hubbard
cf932436c8 rpm, debian: Add hostname as a "make check" dependency
test_hostname.cc depends on the hostname binary.

Fixes: https://tracker.ceph.com/issues/52695

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-09-27 11:03:55 +10:00
Adam King
78983ad0d0 mgr/cephadm: cephadm agent 2.0
Creates an http endpoint in mgr/cephadm to receive
http requests and an agent that can be deployed on
each host that will gather metadata on the host and
send it to the mgr/cephadm http endpoint. Should save the
cephadm mgr module a lot of time it would have to spend
repeatedly ssh-ing into each host to gather the metadata
and help performance on larger clusters.

Fixes: https://tracker.ceph.com/issues/51004

Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:50 -04:00
Brad Hubbard
9630b2d8df Revert "rpm, debian: drop xmlstarlet from build deps"
This reverts commit 707edc0b2d.

This commit was somewhat premature in removing the xmlstarlet
dependency.

Fixes: https://tracker.ceph.com/issues/52681

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-09-21 13:20:13 +10:00
Kefu Chai
707edc0b2d rpm, debian: drop xmlstarlet from build deps
xmlstarlet is not used anymore while performing "make check", so drop it
from the build dependency list.

it is still used by the teuthology test though, so keep it as a
dependency of ceph-test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
c5e49f5190 ceph.spec: remove parted from ceph-base deps
parted is only used by ceph-volume

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
94945b6408 ceph.spec, debian: move deps from base to ceph-volume
this change makes util-linux, xfsprogs and e2fsprogs runtime deps of ceph-volume

ceph-volume uses blkid and lsblk, which are in turn packaged by
util-linux.

util-linux were added as build dependency to fulfill the needs of
ceph-disk. and we tested ceph-disk as part of "make check", since
ceph-disk was dropped, there is no need to have util-linux as
build dependency anymore.

the same applies to e2fsprogs and xfsprogs

see also a991778341

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
a4d9280c0c ceph.spec.in: make cryptsetup a ceph-volume runtime dep.
cryptsetup is solely used by ceph-volume, so it should be a runtime
dependency of ceph-volume, not ceph-base.

and remove it fomr build-dependency list, as we don't use this tool when
building ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
8e0e9ef382 ceph.spec.in: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:17 +08:00
Kefu Chai
c1d60c4d00 rpm,deb: make parted runtime dependency of ceph-osd
we don't run parted when building ceph, instead, it is a runtime
dependency of ceph-volume, which is packaged in ceph-osd.

so in this change

* parted is removed from build dependency list
* parted is added as a runtime depedency of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:18:05 +08:00
Neha Ojha
de8a68b768
Merge pull request #41912 from trociny/wip-crushdiff
tools/crushdiff: new tool to test crushmap change

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-08-31 14:18:37 -07:00
Deepika Upadhyay
879f2056cb
Merge pull request #42913 from yaarith/update-sudoers-smartctl
rpm, debian: move smartmontools and nvme-cli to ceph-base

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-08-30 14:26:03 +05:30
Deepika Upadhyay
69a5ae539f
Merge pull request #42770 from ktdreyer/no-lsb-release
Remove dependency on lsb_release

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-08-30 14:13:02 +05:30
Yaarit Hatuka
7ca39fa92b rpm, debian: move smartmontools and nvme-cli to ceph-base
We wish to be able to scrape SMART and NVMe metrics from OSD and MON
nodes. For this we require / recommend smartmontools and nvme-cli
dependencies for both the ceph-osd and ceph-mon packages.  However, the
sudoers file (which is required for invoking `smartctl` by user 'ceph')
was installed only in the ceph-osd package.  Since different packages
cannot own the same file, and because we want to be able to scrape from
every daemon, we move the dependencies and the sudoers installation to
ceph-base. For generalization, we rename:
	sudoers.d/ceph-osd-smartctl -> sudoers.d/ceph-smartctl

Fixes: https://tracker.ceph.com/issues/50657
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2021-08-25 13:20:23 +00:00
Mykola Golub
8ecae46345 doc: add crushdiff(8) man page
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2021-08-24 11:09:08 +03:00
Mykola Golub
6c73184b21 tools/crushdiff: new tool to test crushmap change
A tool to test the effect (number of pgs, objects, bytes moved)
of a crushmap change. This is a wrapper around osdmaptool, hardly
relying on its --test-map-pgs-dump option to get the list of
changed pgs.  Additionally it uses pg stats to calculate the
numbers of objects and bytes moved.

Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2021-08-24 09:20:28 +03:00
Sebastian Wagner
fb7cb4b50d
Merge pull request #42051 from melissa-kun-li/asyncssh
mgr/cephadm: replace execnet and remoto with asyncssh

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-23 14:29:01 +02:00
Melissa
791d8be5c0 mgr/cephadm: replace instances of remoto/execnet with asyncssh
Fixes: https://tracker.ceph.com/issues/44676
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-08-20 14:27:46 -04:00
Ken Dreyer
cbc0127f34 rpm, debian: remove lsb-release
Nothing requires this any more, so we can remove it from the RPM
and Debian packaging.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-08-18 13:08:24 -04:00
Kefu Chai
9d05d73374 ceph.spec.in: drop gdbm from build deps
neither gdbm nor libgdbm is used when building ceph, so let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-18 01:04:42 +08:00
Aashish Sharma
4907c78bb7 mgr/dashboard: fix grafonnet build error
This PR tends to fix the issue caused by #42194

Fixes:https://tracker.ceph.com/issues/52238
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-08-12 17:48:33 +05:30
Ernesto Puerta
afadfede0d
Merge pull request #42194 from rhcs-dashboard/add-grafonnet-grafana
mgr/dashboard: monitoring: replace Grafana JSON with Grafonnet based code
2021-08-11 18:11:59 +02:00
Aashish Sharma
e9bd94515f mgr/dashboard: monitoring: replace Grafana JSON with Grafonnet based Code
This PR intends to add grafonnet to generate grafana JSON files

Fixes: https://tracker.ceph.com/issues/45184
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-08-11 19:23:54 +05:30
Kefu Chai
8b94e73e12
Merge pull request #42602 from b-ranto/wip-lto
rados+rpm: Update symver defs and re-enable LTO

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-06 13:45:01 +08:00
Boris Ranto
381507a31c rpm: Re-enable LTO on supported systems
We can now use LTO when building ceph. The symver issue was fixed by
using the gcc __symver__ attribute. The systems that support it can now
re-enable LTO.

Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
2021-08-05 15:46:47 +02:00
Kefu Chai
a17ebc0406
Merge pull request #42575 from tchaikov/wip-venv
*: s/virtualenv/python -m venv/

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-04 18:37:45 +08:00
Kefu Chai
a485fdf161
Merge pull request #42571 from tchaikov/wip-disable-kvs
cmake: disable kvs rados cls by default

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-08-04 18:31:40 +08:00
Kefu Chai
065ff536a1 cmake: disable kvs rados cls by default
libcls_kvs was introduced back in
73d016fdb3, but we don't have an internal
user so far. to reduce the build time. let's disable the build of it by
default.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-04 10:21:10 +08:00
Kefu Chai
3741f57eee rpm, debian: drop virtualenv from build deps
since we've replaced "virtualenv" with "python3 -m venv", there is no
need to have it in the build deps list.

since, on ubuntu, venv modules is not available by default, we need to
install python3-venv.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-31 22:34:05 +08:00
Kefu Chai
83cd444a41 rpm: enable dwz
this change reverts 9132269421
back then, we were using rpm < 4.13, which does not support
the feature of "Debugsource and debuginfo sub-packages", but per
https://bugzilla.redhat.com/show_bug.cgi?id=185590. rpm >= 4.13
has this feature. see also http://rpm.org/wiki/Releases/4.13.0

in CentOS 8, RPM v4.14.3 is available. and by inspecting the log
when building ceph packages on CentOS 8, we have:

Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-debugsource-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-base-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-common-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-mds-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
....
build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-test-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm

so, rpmbuild does generate debuginfo package for each binary
package. this should make the life of valgrind a lot easier
when reading the dwz -- no need to read the debuginfo of all
the packages, only the .dwz of the related subpackage is read.

this change should help to decrease the size of debuginfo
rpm packages a little bit. see https://tracker.ceph.com/issues/19099#note-7

this change was inspired by Yuanming Chai <ychai@redhat.com>

See-also: https://tracker.ceph.com/issues/19099
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-30 16:35:08 +08:00
Kefu Chai
fa8f07561c
Merge pull request #42286 from dvanders/dvanders_selinux
ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Wido den Hollander <wido@widodh.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:57:19 +08:00
Kefu Chai
ab7e753ab5
Merge pull request #42452 from smithfarm/wip-first-arg-immutable
rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:53:33 +08:00
Kefu Chai
5419131499 ceph.spec.in: build with header only fmt on RHEL
because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue, we should compile librados either with static library
or with header-only library of fmt. but because the fedora packaging
guideline does no encourage us to package static libraries, and it would
be complicated to package both static and dynamic library for fmt.

the simpler solution would be to compile Ceph with the header-only
version of fmt.

in this change, we compile ceph with the header-only version of fmt
on RHEL to address the runtime dependency issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Nathan Cutler
1cb84a1160 rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.

Thanks to Franck Bui for providing the original patch
926433f5d4 that this patch is modeled after.

NOTE: the use of FIRST_ARG had already been eliminated by
926433f5d4 but was re-introduced later by
9466d70985

Fixes: 9466d70985
Fixes: https://tracker.ceph.com/issues/51797
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-07-22 22:52:39 +02:00
Kefu Chai
7048172cad
Merge pull request #38783 from ideepika/wip-osd-tracing
cmake/ninja: support ninja for jaegertracing

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-22 15:33:13 +08:00
Deepika Upadhyay
80e82686eb ceph.spec, cmake, debian: use thrift 0.13+ from distro pkg
the change to build and ship libthift was added when we didn't have 0.13.0
version shipped via distro pkgs, now that centos 8 and F34 supports req.
version, we do not need to build and ship it with jaeger library.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 10:59:45 +05:30
Deepika Upadhyay
e92b4745ba ceph.spec: enable WITH_JAEGER=ON if --with-jaeger passed
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 10:58:12 +05:30