this change reverts 9132269421
back then, we were using rpm < 4.13, which does not support
the feature of "Debugsource and debuginfo sub-packages", but per
https://bugzilla.redhat.com/show_bug.cgi?id=185590. rpm >= 4.13
has this feature. see also http://rpm.org/wiki/Releases/4.13.0
in CentOS 8, RPM v4.14.3 is available. and by inspecting the log
when building ceph packages on CentOS 8, we have:
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-debugsource-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-base-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-common-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
Wrote: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-mds-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
....
build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-6524-g4e868f9a/rpm/el8/RPMS/x86_64/ceph-test-debuginfo-17.0.0-6524.g4e868f9a.el8.x86_64.rpm
so, rpmbuild does generate debuginfo package for each binary
package. this should make the life of valgrind a lot easier
when reading the dwz -- no need to read the debuginfo of all
the packages, only the .dwz of the related subpackage is read.
this change should help to decrease the size of debuginfo
rpm packages a little bit. see https://tracker.ceph.com/issues/19099#note-7
this change was inspired by Yuanming Chai <ychai@redhat.com>
See-also: https://tracker.ceph.com/issues/19099
Signed-off-by: Kefu Chai <kchai@redhat.com>
because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue, we should compile librados either with static library
or with header-only library of fmt. but because the fedora packaging
guideline does no encourage us to package static libraries, and it would
be complicated to package both static and dynamic library for fmt.
the simpler solution would be to compile Ceph with the header-only
version of fmt.
in this change, we compile ceph with the header-only version of fmt
on RHEL to address the runtime dependency issue.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.
Thanks to Franck Bui for providing the original patch
926433f5d4 that this patch is modeled after.
NOTE: the use of FIRST_ARG had already been eliminated by
926433f5d4 but was re-introduced later by
9466d70985
Fixes: 9466d70985
Fixes: https://tracker.ceph.com/issues/51797
Signed-off-by: Nathan Cutler <ncutler@suse.com>
the change to build and ship libthift was added when we didn't have 0.13.0
version shipped via distro pkgs, now that centos 8 and F34 supports req.
version, we do not need to build and ship it with jaeger library.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
* since focal and centos both have yaml-cpp 0.6 available, which dropped
having boost as it's dependency, moving to 0.6 seems a good upgrade.
* cmake: delete Buildyaml, since distro suppilies v0.6 this is not needed
This fixes the build failure, as jaegertracing requires yaml-cpp v0.6+
```
Could NOT find yaml-cpp: Found unsuitable version "", but required is at
least "0.5.1" (found yaml-cpp_LIBRARY-NOTFOUND)
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
In /etc/sysconfig/ceph we allow operators to define if ceph daemons
should be restarted on upgrade: CEPH_AUTO_RESTART_ON_UPGRADE.
But the post selinux scripts will stop ceph.target regardless if this
is set to `no`, leading to operators adding various hacks to prevent
these unexpected or inconvenient daemon restarts. By now, if users
are using rpms directly, they are likely orchestrating their own
daemon restarts so should not rely on the rpm itself to do this.
Fixes: https://tracker.ceph.com/issues/21672
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
In RPM spec files, comment lines should not include macro invocations,
because RPM can and will expand them, with unpredictable results.
Fixes: https://tracker.ceph.com/issues/51622
Signed-off-by: Nathan Cutler <ncutler@suse.com>
since Seastar has dropped the protobuf dependencies, there is no
need to prepare them for building crimson anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in the KVM instance offered by OBS, we have
[ 346s] + cat /proc/meminfo
[ 347s] MemTotal: 10167736 kB
[ 347s] MemFree: 4983964 kB
[ 347s] MemAvailable: 9826800 kB
[ 347s] Buffers: 85856 kB
[ 347s] Cached: 4615192 kB
[ 347s] SwapCached: 0 kB
...
[ 347s] SwapTotal: 2097148 kB
and its number of hardware threads is
[ 346s] ++ /usr/bin/getconf _NPROCESSORS_ONLN
[ 346s] + _threads=8
so ($MemTotal+$SwapTotal)/1024/2600 = 4.6, which is less
than the # of threads, so "4" was used for the number of jobs.
but per our recent observation in
38be14bc0f, some compiling jobs could
take up to 3GB. in the OOM failure in OBS, we had
[24915s] [24848.843594] Out of memory: Killed process 16894 (cc1plus) total-vm:4293756kB, anon-rss:2970012kB, file-rss:0kB, shmem-rss:0kB, UID:399 pgtables:8324kB oom_score_adj:0
where 4GiB memory was allocated, in which 3GiB was mapped into
memory. this matches with our findings.
in this change, the memory per core is bumped up to 3000MB
in hope to address the OOB. the downside of this change is
that it would take even longer to finish the build if the
building host is limited in memory.
Signed-off-by: Kefu Chai <kchai@redhat.com>
unlike rbd_rwl_cache, rbd_ssd_cache does not depend on pmdk (libpmem),
so let's enable it on all supported architecture and rpm based distros.
Signed-off-by: Kefu Chai <kchai@redhat.com>
ceph-deploy is not actively maintained anymore, and it was replaced by
ceph-volume and other high-level tools.
so there is no point to package its manpage anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the ceph-volume tool is composed of the cli frontend and ceph_volume
python module. in 02bc369e05, its cli
frontend is moved from ceph-base package to ceph-osd. but the python
module was left in ceph-base.
since the only consumer of ceph_volume python package is ceph-volume,
better off moving this python module into ceph-osd. this also aligns
the rpm packaging with the deb packaging, where ceph-osd deb package
also include ceph_volume python module.
we could extract ceph-volumne into its own package, so it can be an
arch-independent package. let's leave it as a follow-up change.
Signed-off-by: Kefu Chai <kchai@redhat.com>
python3-setuptools was originally added to ceph-base as a dependency of
ceph-detect-init, see https://tracker.ceph.com/issues/14864. but since
ceph-disk and ceph-detect-init were replaced by ceph-volume, and were
removed from the debian packaging in
ee6bc23e89.
there is no need to have python3-setuptools in the ceph-base packages
anymore.
but since we are still using pkg_resources module provided by setuptools
in ceph-volume, we need to preserve this runtime dependency in ceph-osd.
as ceph-osd packages ceph-volume.
please note, pkg_resources module is also used by cephadm to poke around
ceph_iscsi python module installed in a container, so python-setuptools
should be installed along with ceph-iscsi if we need a better
interoperability between ceph-iscsi and cephadm. this is not in the
scope of this change.
Signed-off-by: Kefu Chai <kchai@redhat.com>
6.2.1 is the version packaged by EPEL8, in other words, this is the
version we've been testing. so to be more consistent with the
known-to-be-good version, let's bump up the required version.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to lower the number of jobs, we are experiencing build failures on
a builder with 48c96t, 193 free mem. the failures were caused by
OOM killer which kills the c++ compiler
[498376.128969] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/jenkins.service,task=cc1plus,pid=1387895,uid=1110
[498376.145288] Out of memory: Killed process 1387895 (cc1plus) total-vm:3323312kB, anon-rss:3164568kB, file-rss:0kB, shmem-rss:0kB, UID:1110
[498376.315185] oom_reaper: reaped process 1387895 (cc1plus), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[498377.882072] cc1plus invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
before this change, we use the total memory to calculate the number
of jobs, and assume that each job takes at most 2.5GiB mem. in the
case above, the # of job is 96.
after this change, we use the free memory, and increse the mem per job
to 3.0GiB. in the case above, the # of job would be 85.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Otherwise fedora 33 complains there is no gcc-toolset-9-gcc-c++
when running "WITH_SEASTAR=true ./install_deps.sh"
Related to: 36759b5363
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
we need to use libpmem 1.10 in #40493.
without enabling the module stream offering libpmem 1.9.2, we can only
have access to libpmem 1.6.1. and fedora 33 only has libpmem 1.9
packaged. the same applies to openSUSE Tumbleweed and openSUSE Leap. so
let's stop using libpmem packaged by distro by default, until these
distros include libpmem 1.10.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/40526/head:
spec: add nfs to spec file
mgr/nfs: Don't enable nfs module by default
mgr/nfs: check for invalid chars in cluster id
mgr/nfs: Use CLICommand wrapper
mgr/nfs: reorg nfs files
mgr/nfs: Check if transport or protocol are list instance
mgr/nfs: reorg cluster class and common helper methods
mgr/nfs: move common export helper methods to ExportMgr class
mgr/nfs: move validate methods into new ValidateExport class
mgr/nfs: add custom exception module
mgr/nfs: create new module for export utils
mgr/nfs: rename fs dir to export
mgr/volumes/nfs: Move nfs code out of volumes plugin
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
extract the options in common/options.cc into separate .yaml.in
files, and preprocess them using CMake before translating them into .cc
files using a python script.
this change paves the road to render the options using sphinx, and
will allow us to further annotate the options to include more metadata.
also, a this YAML file can be consumed by applications like dashboard
and Sphinx to consume these metadata in a simpler way.
* use @variable-name@ for substituting the variables in .yaml.in file
* use cmake variable of `mgr_disabled_modules` instead of C macro
to define `mgr_disabled_modules` in global.yaml.in
* debian/control, ceph.spec.in, win32_deps_build.sh: add python3-yaml
as build dep
* add y2c.py (short for YAML to C++) to translate .yaml to .cc file
* common/options/*.yaml.in: extract and split options into .yaml.in
files, the subvars in it is then replaced with CMake variables,
and copied to the corresponding .yaml files
* include/config-h.in.cmake: remove MGR_DISABLED_MODULES, as it
is not a CMake variable.
Signed-off-by: Kefu Chai <kchai@redhat.com>
This daemon has a systemd service which starts it with --setuser ceph
--setgroup ceph. "ceph" user and group are created by ceph-common and
won't be there unless ceph-common is installed.
Fixes: https://tracker.ceph.com/issues/50207
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
de6c8250a6 added an explicit %dir directive for
a new directory added to the ceph-common package, but -- due to a typo --
neglected to include the "%". As a result, RPM builds started to fail with:
Processing files: ceph-common-17.0.0-2787.gde6c8250.el8.x86_64
error: File must begin with "/": {_libdir}/ceph/denc/
RPM build errors:
File must begin with "/": {_libdir}/ceph/denc/
Fixes: de6c8250a6
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2d3c6561b4 introduced a new library directory
"%{_libdir}/ceph/denc/" in ceph-common but did not explicitly state that it
should be owned by the package. This caused OBS builds to fail as follows:
[ 5515s] ceph-common-17.0.0-2786.1.x86_64.rpm: directories not owned by a package:
[ 5515s] - /usr/lib64/ceph/denc
Fixes: 2d3c6561b4
Signed-off-by: Nathan Cutler <ncutler@suse.com>
to reduce the memory footprint when linking ceph-dencoder.
* src/tools/ceph-dencoder:
* build dencoders as shared libraries named with the prefix of
"den-mod-". so ceph-dencoder can find them
* install dencoders into $prefix/lib/ceph/denc, so ceph-dencoder
can find them
* only expose "register_dencoders()" function from plugins.
* load plugins in specified directory
* ceph.spec.in: package plugins
* debian: package plugins
Signed-off-by: Kefu Chai <kchai@redhat.com>
Commit 75980798f1 introduced a new package,
libcephsqlite, with a hard RPM dependency on a package "sqlite-libs" which
does not exist in openSUSE.
Since the runtime library dependencies of libcephsqlite are handled by RPM
transparently, this line is not needed.
Fixes: https://tracker.ceph.com/issues/50007
Signed-off-by: Nathan Cutler <ncutler@suse.com>
The ceph-resource-agents package contains an architecture-independent
bash script and parent directories. There are no architecture-dependent
files here, so we can use a single noarch RPM across all host
architectures.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
And since the dependency is now distro-conditional, moving it down
to the 'distro-conditional make check dependencies' section.
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
* build with WITH_SYSTEM_PMDK=ON on fedora, as f32 and f33 ship
libpmem1.8 and libpmem1.9 respectively. and we need libpmem v1.7
* build with WITH_SYSTEM_PMDK=ON on el8, as el8 and CentOS8 AppStream
ships libpmem v1.6,
quote from nvml.spec:
> By design, PMDK does not support any 32-bit architecture.
> Due to dependency on some inline assembly, PMDK can be compiled only
> on these architectures:
> - x86_64
> - ppc64le (experimental)
> - aarch64 (unmaintained, supporting hardware doesn't exist?)
so far, only x86_64 and ppc64le packages are built.
see also,
https://src.fedoraproject.org/rpms/nvml/blob/rawhide/f/nvml.spec
this change addresses a regression introduced by
a49d1dbb32
Signed-off-by: Kefu Chai <kchai@redhat.com>
to ease the build for developers using SUSE, Fedora, CentOS or RHEL.
so install-deps.sh can install ninja for them.
Fixes: https://tracker.ceph.com/issues/49694
Signed-off-by: Kefu Chai <kchai@redhat.com>
The recently merged commit 0e511973f7 replaced
make %{_smp_mflags}
with
%make_build
for the stated purpose of hiding the %_smp_mflags macro in a higher-level macro.
But, on SUSE, the higher-level macro (%make_build) expands to:
make -O %{_smp_mflags}
The addition of the -O flag makes the build considerably slower and increases
the memory requirement. The exact reason for this is unknown - possibly it's due
to a bug in make, although the same slowness was observed with ninja as well.
In any event, this is a deal-breaker when building in the OBS, because the build
infrastructure there is optimized for builds that do not require huge amounts of
memory and we would rather have a fast build with mixed up compiler messages
than a very slow one with synced compiler messages.
Fixes: 0e511973f7
Signed-off-by: Nathan Cutler <ncutler@suse.com>
for encryption, aws s3 provides an "encryption" context to vary per-object
keys. The encryption context is a base64 encoded json structure, which
must be converted to a determinstic form -- "canonical json". This
requires converting all strings to a normalized canonical form: "utf-8 nfc",
it also requires thta keys in objects be sorted in a fixed order; so some
form of sorting based on nfc.
It turns out that libicu was the best way to produce utf-8 nfc (boost also
provides a mechanism, but it has many quirks). So, here are the hooks
to pull the system libicu into the build.
Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
As of a49d1dbb32, when the rbd_rwl_cache and
rbd_ssd_cache bconds are enabled and WITH_SYSTEM_PMDK is disabled (as it is by
default), the RPM build attempts to
git clone https://github.com/ceph/pmdk.git
but of course that won't work in the OBS, where the build workers have no
Internet connectivity.
Fortunately, the openSUSE/SLE versions targeted by Ceph master and pacific ship
the necessary PMDK libraries as RPM packages.
Fixes: a49d1dbb32
Fixes: https://tracker.ceph.com/issues/49550
Signed-off-by: Nathan Cutler <ncutler@suse.com>
43b441f9a3 removed a bunch of code which the SUSE
builds were relying on to avoid OOM. This commit brings back that code in
a much-streamlined form: the SUSE-specific %limit_build macro.
This also has the advantage of not breaking the build on older RPMs which only
know about %_smp_mflags, and not the newer %_smp_build_ncpus etc. macros.
Fixes: 43b441f9a3
Fixes: https://tracker.ceph.com/issues/49556
Signed-off-by: Nathan Cutler <ncutler@suse.com>
This code causes the Ceph build in OBS to fail due to OOM, because the typical
setting of %_smp_build_ncpus in the OBS is 16, but available memory is
insufficient to sustain such a high degree of parallelism for the in-memory
compression operation.
Fixes: b50fc9e61c
Fixes: https://tracker.ceph.com/issues/49583
Signed-off-by: Nathan Cutler <ncutler@suse.com>
The luarocks conditional had gotten hard to read, and the openSUSE Leap 15.3
build needs lua53 as well.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
This partially reverts 2b1e646f7a which
mistakenly changed a line inside an "%if 0%{?suse_version}" conditional.
Fixes: 2b1e646f7a
Signed-off-by: Nathan Cutler <ncutler@suse.com>
this change partially reverts da7030db79
which use %cmake rpm macro in the place of "cmake". but
%cmake sets BUILD_SHARED_LIBS=ON. so quite a few internal libraries
defined using add_library() are now compiled into shared libraries which
are not installed or packagesd. when we are installing the rpm packages
compiled with this option, rpm compiles because the linked libraries are
missing, for instance, `libgmock.so.1.10.0` was compiled as a static
library before da7030db79, and was
included by the test executables. but after that change it's compiled
as a shared library.
so we need to either package the linked shared libraries or just link
against them statically. at this moment, the latter approach is simpler,
albeit larger size of exectuable and dbg symbols.
Fixes: https://tracker.ceph.com/issues/49395
Signed-off-by: Kefu Chai <kchai@redhat.com>
java packaging tools are now offered by "javapackages-tools" module in
el8. so let's re-enable it. this change partially reverts
dd133840b9d35ca7d8b1788836f6076b06c33087
Signed-off-by: Kefu Chai <kchai@redhat.com>
RPM's parseSpec() Python method internally strips this whitespace
character.
Tools that process this spec file with parseSpec() and evaluate
RPMTAG_DESCRIPTION cannot match this exact %description string as
written here.
Strip the trailing whitespace so that the RPMTAG_DESCRIPTION header
matches what we've written in the spec.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
gperftools' libprofiler did not build on ppc64le until 2.7.90.
The EPEL 8 package is being updated accordingly.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
it errors while building without lua_packages:
+ -DBOOST_J=102 -DWITH_GRAFANA=ON
/var/tmp/rpm-tmp.aq2X3J: line 91: -DBOOST_J=102: command not found
Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
The rgw-gap-list tool can produce a number of false positives when the
cluster is being used during its run. One technique to minimize the
number of false positives is to run the tool twice and look for the
objects that appear in both lists. The rgw-gap-list-comparator tool is
designed to do this comparison.
Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
Due to a prior bug (pr: 38228) tail rados objects of some RGW objects
could have been incorrectly deleted. This tool is designed to look for
such cases. It essentially does the opposite of rgw-orphan-list,
looking for rados objects that RGW expects to be there, but which are
not to be found.
IMPORTANT: This is very experimental at this point in time, and any
"results" produced should be verified by other means.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
This library is obsolete with the mgr volumes plugin since Nautilus.
The last remaining user of this library was Manila which will be using
the volumes plugin with Pacific and onwards.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Install the mount.fuse.ceph man page and ship it in the ceph-fuse
packaging along with the corresponding mount.fuse.ceph binary.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
SUSE container images are built using a process that excludes dependencies that
are merely recommended.
Fixes: https://tracker.ceph.com/issues/48604
Signed-off-by: Nathan Cutler <ncutler@suse.com>
when building with seastar enabled, package crimson-store-ndb in
"ceph-test" package. "crimson-store-nbd" is a new tool for testing
seastore using nbd.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* This commit introduces Jaegertracing library as package libjaeger,
pickwhich would be consumed by other ceph pacakges such as ceph-common0
* adds the following dependencies, which would be build from source
using ExternalProjectHelper.cmake +IncludeJaeger.cmake +
Build<package>.cmake scripts:
jaegertracing: v0.6.0 [added as a submodule]
opentracing: v1.6.0 [added as a submodule]
thrift: 0.13.0 [added as a submodule]
yaml-cpp: 0.6.0
json(optional)
* updates Boost to be installed instead of being build only, because
jaegertracing them during their build process.
* ceph.spec.in: introduces a default enabled jaeger packaging option,
which could be disabled using --without-jaeger flag during rpmbuild
* note: libjaeger package if enabled will be a dependency on ceph-common, ceph-mon, rgw_common and transitively will be a dependency for modules that have them as a dependency.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
This commit introduces internal (not yet part of the api) librbd functions for:
1. formating an RBD image in LUKS format
2. parsing an RBD image in LUKS format
The actual implementation of the LUKS format is done via libcryptsetup, which is added as a new dependency.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
The cephadm package contains an architecture-independent Python script,
empty directories, and an empty authorized_keys file. There are no
architecture-dependent files here, so we can use a single noarch RPM
across all host architectures.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
change sudoers file mode to 440 to match recommended defaults.
From the sudoers man page.
> the default file mode is 0440 (read‐able by owner and group, writable
by none).
> The default mode may be changed via the “sudoers_mode” option to the
sudoers
> Plugin line in the sudo.conf(5) file.
Fixes: https://tracker.ceph.com/issues/48169
Signed-off-by: David Turner <drakonstein@gmail.com>
nasm support build isa-l:AVX512 algorithm implementation while yasm
doens't support it. Install nasm assembler to build isa-l
refer to: https://github.com/yasm/yasm/issues/101
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
This reverts commit 070e5c3e35ea476815ff9fa4f71aed147fe1ea79.
This commit was working around a bug that was present in old (pre-1.8.0)
versions of podman. Since we are no longer running cephadm with versions
that old, we can safely drop this workaround.
Fixes: https://tracker.ceph.com/issues/47862
Signed-off-by: Nathan Cutler <ncutler@suse.com>
SUSE needs %service_del_postun (with or without restart) *only* if there
is a possibility that the RPM containing the unit file will be upgraded
from a version that packaged SysVinit scripts instead of systemd unit
files. (Which is not the case here.)
Signed-off-by: Nathan Cutler <ncutler@suse.com>
This SUSE-specific variable is deprecated and use of
%service_del_postun_without_restart macro should be preferred these
days.
Signed-off-by: Franck Bui <fbui@suse.com>
since libfmt > 5.0.0 is packaged by CentOS/RHEL8 and openSUSE/SLE
we can just use it instead of our own version. also, since libfmt is
used out side of crimson project, no need to guard it using "with
seastar" anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
These packages are needed in order to scrape device health metrics from
devices used by OSD and MON daemons.
smartmontools' smartctl is what we use in order to scrape devices' SMART
attributes and general health metrics.
In addition, we use nvme-cli tool on NVMe devices, which fetches
vendor specific NVMe related health metrics.
Ceph rely on these tools for proper functioning of the underlying layers
of devicehealth mgr module, and other mgr modules which use devicehealth
functionality (such as diskprediction_local, telemetry, dashboard).
Essentially, most of devicehealth commands rely on proper functioning of
smartctl, otherwise they lack the device health metrics.
For example, in case smartctl is missing, the commands:
ceph device scrape-daemon-health-metrics <who>
ceph device scrape-health-metrics [<devid>]
will not be able to scrape health metrics, and the command:
ceph device predict-life-expectancy <devid>
will not provide any meaningful output (since there are no metrics).
In short, when we scrape a device by its daemon (be it an OSD or a MON):
ceph device scrape-daemon-health-metrics <who>
The devicehealth module command eventually invokes a
block_device_get_metrics() call in either osd/OSD.cc or mon/Monitor.cc,
which wraps calls to both
block_device_run_smartctl() (spawns smartctl)
block_device_run_vendor_nvme() (spawns nvme)
in common/blkdev.cc.
Minimum version requirements:
'smartmontools' is the package name, which contains two utility
programs: 'smartd' and 'smartctl'. Ceph uses the latter.
Version 6.7 of smartctl first introduced the --json option (beta), which
allows to output the metrics in a JSON format. Since then a few
adjustments were made and the feature officially launched in smartctl
version 7.0.
Since we rely on the JSON format to process the metrics, we must have
smartmontools' smartctl version >= 7.
That said, we choose not to specify smartmontools version here on
purpose, since there might be a scenario where:
We specified smartmontools version to be >= 7.
smartmontools 7 is not available yet in rhel 8 / centos 8.
A user installs via rpm ceph-osd, for example.
smartmontools will not be installed (since version >= 7 is not available
in this repo yet).
Then the user upgrades to 8.3 (which should have smartmontools >= 7),
but smartmontools will not get upgraded (since it's not installed).
In the scenario where we do not specify a version, smartmontools 6.6
will be installed, but it will be upgraded to >= 7 when a user upgrades
(and if it's a fresh installation - version >= 7 would be installed
anyway).
nvme-cli does not have a minimum version.
We use 'Recommends' for both rpm and deb packages since we do not want
the installation to fail in case of conflicts. 'Recommends' weakens the
dependency to be installed in case possible, but ignores it in cases of
conflicts with other dependencies.
It's worth mentioning that smartmontools and nvme-cli dependencies exist
in ceph-container builds.
We add them here for the cases of bare metal installations.
In the future we will add a separate package (with smartmontools and
nvme-cli dependencies) that can be installed on any node (running
rbd-mirror, rgw, mds, mgr, etc.), in order to be able to collect the
health metrics of its devices and offer their life expectancy
prediction.
Fixes: https://tracker.ceph.com/issues/47479
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
libzbd-devel is not available in any distros official repo at the time
of writing. but we packaged it for RHEL8/CentOS8. and uploaded its
packages to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/. to
build test it to avoid the bitrot of ZBD bluestore backend at a minimal
level. but we cannot do more than this. because
- the kernel shipped by RHEL/CentOS8 does not have zoned block device
support enabled. see https://zonedstorage.io/distributions/linux/
- and we don't have zoned block device for testing in our lab.
so, in this change, "--with-zbd" option is added so we can pull
in the libzbd-devel package on demand.
Signed-off-by: Kefu Chai <kchai@redhat.com>
rpm,deb: drop /etc/sudoers.d/cephadm
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
We've upgraded to Boost version 1.73
(1169dee259), and this version provides
working support for boost::context for s390x. Enable this in the RPM
packaging.
Fixes: https://tracker.ceph.com/issues/47304
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
Current behavior (without this patch) is:
1. cephadm package installs cephadm at /usr/sbin/cephadm
2. cephadm package installs /etc/sudoers.d/cephadm
3. !!! BUT this file refers to a non-existent executable (/usr/bin/cephadm) !!!
4. the PR that introduced this sudoers file (and this discrepancy) was merged in 2019
5. nobody noticed the discrepancy until now
My conclusion: the file /etc/sudoers.d/cephadm is not needed for cephadm to
work.
Fixes: https://tracker.ceph.com/issues/47112
Signed-off-by: Nathan Cutler <ncutler@suse.com>
This reverts commit a55c1dd0fa.
Kept to keep upgrades from older point releases working.
This module can be removed as soon as we no longer
support upgrades from old octopus point releases.
Revert "build/debian: remove osd_support"
This reverts commit 8ff2824beb.
Fixes: https://tracker.ceph.com/issues/47109
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
the service offered by https://www.diskprophet.com is not accessible
anymore. before we have a replacement of it, let's drop
mgr-diskprediction-cloud. and remove its packaging.
Signed-off-by: Kefu Chai <kchai@redhat.com>
since seastar dropped support from C++14, we have to move to a compiler
with a decent C++17 support.
in this change, gcc-toolset-9 is used for compiling ceph if seastar is
enabled. use version >= 9.2.1-2.2, because of
https://bugzilla.redhat.com/show_bug.cgi?id=1853900
Signed-off-by: Kefu Chai <kchai@redhat.com>
mgr plugin to deploy and configure MDSs in response to degraded file system
MDS instance management as per changes to:
* 'max_mds' option
* 'standby_count_wanted' option
* mds liveness and transitions from standby to active
mds_autoscaler plugin test credit goes to Sebastian Wagner.
Fixes: https://tracker.ceph.com/issues/40929
Signed-off-by: Milind Changire <mchangir@redhat.com>
Since e5b585d15de8b07e0a179344d4187582a5c069f2 ceph-volume depends on
python-ceph-common. This commit introduces this dependency for the
ceph-osd rpm (which includes ceph-volume) and installs the dependency
for tox runs.
Fixes: https://tracker.ceph.com/issues/46772
Fixes: e5b585d15de8b07e0a179344d4187582a5c069f2
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
seastar uses setjmp() and longjmp() to implement coroutine, but
longjmp() is defined as ____longjmp_chk() by GCC if _FORTIFY_SOURC is
defined. ____longjmp_chk() simply bails out with an error message if
the dest stack pointer is higher than the src stack pointer, or the dest
stack pointer is not in the sigaltstack. in the case of seastar, the dst
%sp is not necessarily higher than src stack pointer, and it's not
handling a signal for switching the thread context. that's why we have
the "longjmp causes uninitialized stack frame" error when running
crimson-osd on RHEL/CentOS 8 using the prebuilt rpm packages.
the optflags rpm macro adds -D_FORTIFY_SOURCE=2 to CFLAGS and CXXFLAGS,
so even seastar tries to pass -U_FORTIFY_SOURCE to GCC, there is chance
that cmake append CXXFLAGS at the end of the option list passed to GCC.
and this renders seastar's attempt to undefine _FORTIFY_SOURCE useless.
another way to address this issue is to undefine this macro in
seastar:src/core/thread.cc. but since seastar tries neutralize the macro
in its cmake script instead of source file, i assume they have their
considerations. let's drop it in the rpm recipe instead.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in hope to simplify the enabling of crimson-osd. as we will use cephadm
as the way for deploying crimson-osd in future. it's unlikely we will
have classic osd and crimson osd co-located in the same system. so
this is most simple way to enable crimson-osd with minimum efforts.
Signed-off-by: Kefu Chai <kchai@redhat.com>
We no longer need it, as distributions are converging on /usr/libexec
since FHS 3.0 has specified it as the acceptable path for libexecdir.
Fedora has been using /usr/libexec since the beginning, and
openSUSE has accepted software installed into /usr/libexec since
September 2019 in openSUSE:Factory. openSUSE Leap 15.x still uses
/usr/lib for %_libexecdir, but the rest of the spec should handle the
change gracefully.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
This mimics the setting passed to CMake in other package builds,
and ensures that systemd units are installed into the right place.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
mgr/cephadm: use jinja2 to render templates
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
otherwise, "osc build" fails like:
[ 5545s] libcephfs-devel-16.0.0-2172.ga59fb9ffac.x86_64.rpm: directories not owned by a package:
[ 5545s] - /usr/include/cephfs/metrics
this addresses a regression introduced by
c669de12e0
Signed-off-by: Kefu Chai <kchai@redhat.com>
The librbd parent cache is an initial testbed for librbd plugin
support. The shared library will be packaged with librbd for the
time being.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
* refs/pull/34596/head:
test: add a new program for testing ino_release_cb
client: add a new inode release request callback
client: expose ceph_ll_register_callbacks via libcephfs
client: move callback typedefs and arg struct into ceph_ll_client.h
client: rename ceph_statx.h to ceph_ll_client.h
client: make client_dentry_callback_t more friendly for C
client: only override umask_cb with non-NULL values
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Adds a `radosgw-admin` subcommand and walks the associated bucket
indices and manifests to generate the list of rados objects that
represent the rgw objects in the bucket(s).
Also adds a tool named `rgw-orphan-list`, which uses the radoslist
subcommand, that produces a list in a local file of what appear to be
rgw orphans.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
...so we can use it to include other definitions and types that need to
be shared with in-tree code that doesn't want to include libcephfs.h.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
`python.*-saml` is required for Ceph-Dashboard SSO support (optional
feature).
Fixes: https://tracker.ceph.com/issues/44721
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
it's a regression introduced by
6158bcfdef, which dropped the change to
make Sanitizers optional
since we've switched from xenial to bionic. there is no need to disable
this anymore. we ran into an issue caused by the ancient linker shipped
by xenial before.
Fixes: https://tracker.ceph.com/issues/44658
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/33885/head:
Merge pull request #33848 from mchangir/octopus-tests-remove-suprious-whitespace
Merge PR #33746 into octopus
Merge PR #33830 into octopus
Merge PR #33732 into octopus
Merge PR #33620 into octopus
Merge pull request #33876 from tchaikov/octopus-cephadm-mypy
cephadm: add "assert foo is not None" for mypy check
Merge pull request #33067 from tspmelo/wip-rbd-delete-with-snapshot
cephadm: add grafana adopt
Merge PR #33771 into octopus
Merge PR #33850 into octopus
Merge PR #33853 into octopus
Merge PR #33857 into octopus
Merge PR #32990 into octopus
Merge PR #33713 into octopus
Merge PR #33838 into octopus
qa/tasks/cephadm: no default mon|mgr|crash service specs
qa/suites/rados/cephadm/upgrade: upgrade start point that supports the no-spec option
Merge PR #33832 into octopus
cephadm: bootstrap: wait for mgr to restart after enabling a module
mgr: add 'mgr_status' tell command
Merge pull request #33839 from rhcs-dashboard/44538-fix-rgw-grafana-get-put-latencies
Merge pull request #33743 from votdev/issue_43869_fix_qa_test
cephadm: create initial mon and mgr service specs too
cephadm: no need to pregenerate a crash key for the bootstrap host
mgr/cephadm: do not complain when we don't have enough hosts
mgr/cephadm: remove orphan daemons
mgr/cephadm: report size=0 for fabricated ServiceDescription
mgr/cephadm: safety check to prevent removing all mon|mgr daemons
mgr/cephadm: prevent scaling mon|mgr below count=1
mgr/cephadm: do not remove daemons from remove_service
Merge pull request #33805 from tchaikov/wip-44500
spec: Podman (temporarily) requires apparmor-abstractions on suse
mgr/cephadm: Make sure we don't co-locate the same daemon
monitoring: fix RGW grafana chart 'Average GET/PUT Latencies'
tests: remove spurious whitespace
mgr/cephadm: fix service list filtering
Merge PR #33825 into octopus
Merge PR #33811 into octopus
Revert "Merge pull request #33673 from cbodley/wip-denc-enum"
mgr/cephadm: fix upgrade order
Merge PR #33801 into octopus
Merge PR #33822 into octopus
cephadm: bootstrap: tolerate error return from -h
Merge PR #33809 into octopus
Merge PR #32678 into octopus
cephadm: use `sh` instead of `bash` during enter
ceph.in: only shut down rados on clean exit
common/ceph_timer: Pass reference to waited time on stack
common/ceph_timer: Add test
common/ceph_timer: Use unique_function, allowing noncopyable events
common/ceph_timer: Couple cleanups
common/ceph_timer: Fix namespaces
common/ceph_timer: Add missing includes
common/ceph_timer.h: Don't indent contents of a namespace
mgr/dashboard: Crush rule modal
mgr/dashboard: Preserve rule selection on pool type change
mgr/dashboard: Crush rule is only send during replicated pool creation
mgr/dashboard: Explicit returns in pool form
mgr/dashboard: Removes fork join in pool form
mgr/dashboard: Hide ECP actions during ec pool edit
mgr/dashboard: Pool form erasure/replicated boolean
mgr/dashboard: Change pool info API endpoint
mgr/dashboard: Moves ECP info endpoint to UI-API
mgr/cephadm: add _remove_osds_bg back to main loop
mgr/cephadm/osd: update removal report immediately
qa/tasks/ceph_manager: use StringIO for capturing COT output
qa/standalone/scrub/osd-scrub-repair: force osdmap prop to osds
qa/standalone/scrub/osd-scrub-test: wait longer for update
qa/tasks/ceph_manager: capture stderr for COT
qa/suites/rados/ceph: drop opensuse for now
mon/MonClient: send logs to mon on separate schedule than pings
mgr/dashboard: Fix missing ImageSpec usage
mgr/dashboard: Allow removing RBD with snapshots
mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user
mgr/dashboard: support multiple DriveGroups when creating OSDs
mon/MonClient: send logs to mon even if we have no keelalive2
cephadm: flag dashboard user to change password
Reviewed-by: Sebastian Wagner <swagner@suse.com>
`apparmor-abstractions` contains a profile that is required to run podman containers.
Fixes: https://tracker.ceph.com/issues/44272
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Until now, "ocf" and "libradosstriper" were disabled on SLE, but not
openSUSE.
Leaving them enabled for openSUSE makes it appear as if these features
are expected to do something useful on SUSE.
Dropping the "is_opensuse" conditional has the desirable side effect of
streamlining the SUSE bcond block, and in the spirit of "and that's not
all", we take the opportunity to put the bconds in alphabetical order
for comforting cosmetic effect.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
ceph-mgr depends on it in the sense that these plugins are
a hard dependency of ceph-mgr. like cmake depends on cmake-data,
even if the later cannot function on its own without the former.
the reason why we need a separated package is that mgr modules
are written in python, and should be arch independent. this change
helps the downstream to understand this and help to minimize the
divergence between upstream and downstream, because distros like
debian and its derivatives encourage splitting architecture independent
data into its own package.
see
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.html#architecture-independent-data
also, the python related runtime dependencies are also moved into
ceph-mgr-modules-core.
if a python dependency is shared by another ceph-mgr-* package and
ceph-mgr-modules-core. it's only added to ceph-mgr-modules-core for
simplicity and consistency. as the name of dependency might vary
on different distros, so it's tedious and error-prone to repeat
these conditons and checks.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The coverage package under openSUSE (and other distros) are named as
python{major_version}-coverage (without minor version).
Fixes: https://tracker.ceph.com/issues/44164
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
This switch sets CMAKE_VERBOSE_MAKEFILE and causes the CMakeOutput.log
and CMakeError.log files to be dumped after cmake execution finishes.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
