ecb8d2cae2 disabled the system_pmdk bcond for all
build targets based on the fact that pmdk 1.10 was not available on any of them.
Now that openSUSE Tumbleweed ships pmdk 1.11, we re-enable the system_pmdk bcond
to fix the master build for openSUSE Tumbleweed.
Since openSUSE Tumbleweed is the *only* SUSE build target master supports, there
is no need for greater granularity in the distro conditional here.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Creates an http endpoint in mgr/cephadm to receive
http requests and an agent that can be deployed on
each host that will gather metadata on the host and
send it to the mgr/cephadm http endpoint. Should save the
cephadm mgr module a lot of time it would have to spend
repeatedly ssh-ing into each host to gather the metadata
and help performance on larger clusters.
Fixes: https://tracker.ceph.com/issues/51004
Signed-off-by: Adam King <adking@redhat.com>
This reverts commit 707edc0b2d.
This commit was somewhat premature in removing the xmlstarlet
dependency.
Fixes: https://tracker.ceph.com/issues/52681
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
xmlstarlet is not used anymore while performing "make check", so drop it
from the build dependency list.
it is still used by the teuthology test though, so keep it as a
dependency of ceph-test.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change makes util-linux, xfsprogs and e2fsprogs runtime deps of ceph-volume
ceph-volume uses blkid and lsblk, which are in turn packaged by
util-linux.
util-linux were added as build dependency to fulfill the needs of
ceph-disk. and we tested ceph-disk as part of "make check", since
ceph-disk was dropped, there is no need to have util-linux as
build dependency anymore.
the same applies to e2fsprogs and xfsprogs
see also a991778341
Signed-off-by: Kefu Chai <kchai@redhat.com>
cryptsetup is solely used by ceph-volume, so it should be a runtime
dependency of ceph-volume, not ceph-base.
and remove it fomr build-dependency list, as we don't use this tool when
building ceph.
Signed-off-by: Kefu Chai <kchai@redhat.com>
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.
Signed-off-by: Kefu Chai <kchai@redhat.com>
we don't run parted when building ceph, instead, it is a runtime
dependency of ceph-volume, which is packaged in ceph-osd.
so in this change
* parted is removed from build dependency list
* parted is added as a runtime depedency of ceph-osd.
Signed-off-by: Kefu Chai <kchai@redhat.com>
We wish to be able to scrape SMART and NVMe metrics from OSD and MON
nodes. For this we require / recommend smartmontools and nvme-cli
dependencies for both the ceph-osd and ceph-mon packages. However, the
sudoers file (which is required for invoking `smartctl` by user 'ceph')
was installed only in the ceph-osd package. Since different packages
cannot own the same file, and because we want to be able to scrape from
every daemon, we move the dependencies and the sudoers installation to
ceph-base. For generalization, we rename:
sudoers.d/ceph-osd-smartctl -> sudoers.d/ceph-smartctl
Fixes: https://tracker.ceph.com/issues/50657
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
A tool to test the effect (number of pgs, objects, bytes moved)
of a crushmap change. This is a wrapper around osdmaptool, hardly
relying on its --test-map-pgs-dump option to get the list of
changed pgs. Additionally it uses pg stats to calculate the
numbers of objects and bytes moved.
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
mgr/cephadm: replace execnet and remoto with asyncssh
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
We can now use LTO when building ceph. The symver issue was fixed by
using the gcc __symver__ attribute. The systems that support it can now
re-enable LTO.
Fixes: https://tracker.ceph.com/issues/40060
Signed-off-by: Boris Ranto <branto@redhat.com>
libcls_kvs was introduced back in
73d016fdb3, but we don't have an internal
user so far. to reduce the build time. let's disable the build of it by
default.
Signed-off-by: Kefu Chai <kchai@redhat.com>
since we've replaced "virtualenv" with "python3 -m venv", there is no
need to have it in the build deps list.
since, on ubuntu, venv modules is not available by default, we need to
install python3-venv.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change reverts 9132269421
back then, we were using rpm < 4.13, which does not support
the feature of "Debugsource and debuginfo sub-packages", but per
https://bugzilla.redhat.com/show_bug.cgi?id=185590. rpm >= 4.13
has this feature. see also http://rpm.org/wiki/Releases/4.13.0
in CentOS 8, RPM v4.14.3 is available. and by inspecting the log
when building ceph packages on CentOS 8, we have:
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-debugsource-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-base-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-common-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-mds-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
....
build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-test-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
so, rpmbuild does generate debuginfo package for each binary
package. this should make the life of valgrind a lot easier
when reading the dwz -- no need to read the debuginfo of all
the packages, only the .dwz of the related subpackage is read.
this change should help to decrease the size of debuginfo
rpm packages a little bit. see https://tracker.ceph.com/issues/19099#note-7
this change was inspired by Yuanming Chai <ychai@redhat.com>
See-also: https://tracker.ceph.com/issues/19099
Signed-off-by: Kefu Chai <kchai@redhat.com>
because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue, we should compile librados either with static library
or with header-only library of fmt. but because the fedora packaging
guideline does no encourage us to package static libraries, and it would
be complicated to package both static and dynamic library for fmt.
the simpler solution would be to compile Ceph with the header-only
version of fmt.
in this change, we compile ceph with the header-only version of fmt
on RHEL to address the runtime dependency issue.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.
Thanks to Franck Bui for providing the original patch
926433f5d4 that this patch is modeled after.
NOTE: the use of FIRST_ARG had already been eliminated by
926433f5d4 but was re-introduced later by
9466d70985
Fixes: 9466d70985
Fixes: https://tracker.ceph.com/issues/51797
Signed-off-by: Nathan Cutler <ncutler@suse.com>
the change to build and ship libthift was added when we didn't have 0.13.0
version shipped via distro pkgs, now that centos 8 and F34 supports req.
version, we do not need to build and ship it with jaeger library.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* since focal and centos both have yaml-cpp 0.6 available, which dropped
having boost as it's dependency, moving to 0.6 seems a good upgrade.
* cmake: delete Buildyaml, since distro suppilies v0.6 this is not needed
This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
In /etc/sysconfig/ceph we allow operators to define if ceph daemons
should be restarted on upgrade: CEPH_AUTO_RESTART_ON_UPGRADE.
But the post selinux scripts will stop ceph.target regardless if this
is set to `no`, leading to operators adding various hacks to prevent
these unexpected or inconvenient daemon restarts. By now, if users
are using rpms directly, they are likely orchestrating their own
daemon restarts so should not rely on the rpm itself to do this.
Fixes: https://tracker.ceph.com/issues/21672
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
In RPM spec files, comment lines should not include macro invocations,
because RPM can and will expand them, with unpredictable results.
Fixes: https://tracker.ceph.com/issues/51622
Signed-off-by: Nathan Cutler <ncutler@suse.com>
since Seastar has dropped the protobuf dependencies, there is no
need to prepare them for building crimson anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in the KVM instance offered by OBS, we have
[ 346s] + cat /proc/meminfo
[ 347s] MemTotal: 10167736 kB
[ 347s] MemFree: 4983964 kB
[ 347s] MemAvailable: 9826800 kB
[ 347s] Buffers: 85856 kB
[ 347s] Cached: 4615192 kB
[ 347s] SwapCached: 0 kB
...
[ 347s] SwapTotal: 2097148 kB
and its number of hardware threads is
[ 346s] ++ /usr/bin/getconf _NPROCESSORS_ONLN
[ 346s] + _threads=8
so ($MemTotal+$SwapTotal)/1024/2600 = 4.6, which is less
than the # of threads, so "4" was used for the number of jobs.
but per our recent observation in
38be14bc0f, some compiling jobs could
take up to 3GB. in the OOM failure in OBS, we had
[24915s] [24848.843594] Out of memory: Killed process 16894 (cc1plus) total-vm:4293756kB, anon-rss:2970012kB, file-rss:0kB, shmem-rss:0kB, UID:399 pgtables:8324kB oom_score_adj:0
where 4GiB memory was allocated, in which 3GiB was mapped into
memory. this matches with our findings.
in this change, the memory per core is bumped up to 3000MB
in hope to address the OOB. the downside of this change is
that it would take even longer to finish the build if the
building host is limited in memory.
Signed-off-by: Kefu Chai <kchai@redhat.com>
unlike rbd_rwl_cache, rbd_ssd_cache does not depend on pmdk (libpmem),
so let's enable it on all supported architecture and rpm based distros.
Signed-off-by: Kefu Chai <kchai@redhat.com>
ceph-deploy is not actively maintained anymore, and it was replaced by
ceph-volume and other high-level tools.
so there is no point to package its manpage anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the ceph-volume tool is composed of the cli frontend and ceph_volume
python module. in 02bc369e05, its cli
frontend is moved from ceph-base package to ceph-osd. but the python
module was left in ceph-base.
since the only consumer of ceph_volume python package is ceph-volume,
better off moving this python module into ceph-osd. this also aligns
the rpm packaging with the deb packaging, where ceph-osd deb package
also include ceph_volume python module.
we could extract ceph-volumne into its own package, so it can be an
arch-independent package. let's leave it as a follow-up change.
Signed-off-by: Kefu Chai <kchai@redhat.com>
python3-setuptools was originally added to ceph-base as a dependency of
ceph-detect-init, see https://tracker.ceph.com/issues/14864. but since
ceph-disk and ceph-detect-init were replaced by ceph-volume, and were
removed from the debian packaging in
ee6bc23e89.
there is no need to have python3-setuptools in the ceph-base packages
anymore.
but since we are still using pkg_resources module provided by setuptools
in ceph-volume, we need to preserve this runtime dependency in ceph-osd.
as ceph-osd packages ceph-volume.
please note, pkg_resources module is also used by cephadm to poke around
ceph_iscsi python module installed in a container, so python-setuptools
should be installed along with ceph-iscsi if we need a better
interoperability between ceph-iscsi and cephadm. this is not in the
scope of this change.
Signed-off-by: Kefu Chai <kchai@redhat.com>
