We no longer need it, as distributions are converging on /usr/libexec
since FHS 3.0 has specified it as the acceptable path for libexecdir.
Fedora has been using /usr/libexec since the beginning, and
openSUSE has accepted software installed into /usr/libexec since
September 2019 in openSUSE:Factory. openSUSE Leap 15.x still uses
/usr/lib for %_libexecdir, but the rest of the spec should handle the
change gracefully.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
This mimics the setting passed to CMake in other package builds,
and ensures that systemd units are installed into the right place.
Signed-off-by: Neal Gompa <ngompa13@gmail.com>
mgr/cephadm: use jinja2 to render templates
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
otherwise, "osc build" fails like:
[ 5545s] libcephfs-devel-16.0.0-2172.ga59fb9ffac.x86_64.rpm: directories not owned by a package:
[ 5545s] - /usr/include/cephfs/metrics
this addresses a regression introduced by
c669de12e0
Signed-off-by: Kefu Chai <kchai@redhat.com>
The librbd parent cache is an initial testbed for librbd plugin
support. The shared library will be packaged with librbd for the
time being.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
* refs/pull/34596/head:
test: add a new program for testing ino_release_cb
client: add a new inode release request callback
client: expose ceph_ll_register_callbacks via libcephfs
client: move callback typedefs and arg struct into ceph_ll_client.h
client: rename ceph_statx.h to ceph_ll_client.h
client: make client_dentry_callback_t more friendly for C
client: only override umask_cb with non-NULL values
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Adds a `radosgw-admin` subcommand and walks the associated bucket
indices and manifests to generate the list of rados objects that
represent the rgw objects in the bucket(s).
Also adds a tool named `rgw-orphan-list`, which uses the radoslist
subcommand, that produces a list in a local file of what appear to be
rgw orphans.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
...so we can use it to include other definitions and types that need to
be shared with in-tree code that doesn't want to include libcephfs.h.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
`python.*-saml` is required for Ceph-Dashboard SSO support (optional
feature).
Fixes: https://tracker.ceph.com/issues/44721
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
it's a regression introduced by
6158bcfdef91cc2930c57ff2bbe2bfae37da7363, which dropped the change to
make Sanitizers optional
since we've switched from xenial to bionic. there is no need to disable
this anymore. we ran into an issue caused by the ancient linker shipped
by xenial before.
Fixes: https://tracker.ceph.com/issues/44658
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/33885/head:
Merge pull request #33848 from mchangir/octopus-tests-remove-suprious-whitespace
Merge PR #33746 into octopus
Merge PR #33830 into octopus
Merge PR #33732 into octopus
Merge PR #33620 into octopus
Merge pull request #33876 from tchaikov/octopus-cephadm-mypy
cephadm: add "assert foo is not None" for mypy check
Merge pull request #33067 from tspmelo/wip-rbd-delete-with-snapshot
cephadm: add grafana adopt
Merge PR #33771 into octopus
Merge PR #33850 into octopus
Merge PR #33853 into octopus
Merge PR #33857 into octopus
Merge PR #32990 into octopus
Merge PR #33713 into octopus
Merge PR #33838 into octopus
qa/tasks/cephadm: no default mon|mgr|crash service specs
qa/suites/rados/cephadm/upgrade: upgrade start point that supports the no-spec option
Merge PR #33832 into octopus
cephadm: bootstrap: wait for mgr to restart after enabling a module
mgr: add 'mgr_status' tell command
Merge pull request #33839 from rhcs-dashboard/44538-fix-rgw-grafana-get-put-latencies
Merge pull request #33743 from votdev/issue_43869_fix_qa_test
cephadm: create initial mon and mgr service specs too
cephadm: no need to pregenerate a crash key for the bootstrap host
mgr/cephadm: do not complain when we don't have enough hosts
mgr/cephadm: remove orphan daemons
mgr/cephadm: report size=0 for fabricated ServiceDescription
mgr/cephadm: safety check to prevent removing all mon|mgr daemons
mgr/cephadm: prevent scaling mon|mgr below count=1
mgr/cephadm: do not remove daemons from remove_service
Merge pull request #33805 from tchaikov/wip-44500
spec: Podman (temporarily) requires apparmor-abstractions on suse
mgr/cephadm: Make sure we don't co-locate the same daemon
monitoring: fix RGW grafana chart 'Average GET/PUT Latencies'
tests: remove spurious whitespace
mgr/cephadm: fix service list filtering
Merge PR #33825 into octopus
Merge PR #33811 into octopus
Revert "Merge pull request #33673 from cbodley/wip-denc-enum"
mgr/cephadm: fix upgrade order
Merge PR #33801 into octopus
Merge PR #33822 into octopus
cephadm: bootstrap: tolerate error return from -h
Merge PR #33809 into octopus
Merge PR #32678 into octopus
cephadm: use `sh` instead of `bash` during enter
ceph.in: only shut down rados on clean exit
common/ceph_timer: Pass reference to waited time on stack
common/ceph_timer: Add test
common/ceph_timer: Use unique_function, allowing noncopyable events
common/ceph_timer: Couple cleanups
common/ceph_timer: Fix namespaces
common/ceph_timer: Add missing includes
common/ceph_timer.h: Don't indent contents of a namespace
mgr/dashboard: Crush rule modal
mgr/dashboard: Preserve rule selection on pool type change
mgr/dashboard: Crush rule is only send during replicated pool creation
mgr/dashboard: Explicit returns in pool form
mgr/dashboard: Removes fork join in pool form
mgr/dashboard: Hide ECP actions during ec pool edit
mgr/dashboard: Pool form erasure/replicated boolean
mgr/dashboard: Change pool info API endpoint
mgr/dashboard: Moves ECP info endpoint to UI-API
mgr/cephadm: add _remove_osds_bg back to main loop
mgr/cephadm/osd: update removal report immediately
qa/tasks/ceph_manager: use StringIO for capturing COT output
qa/standalone/scrub/osd-scrub-repair: force osdmap prop to osds
qa/standalone/scrub/osd-scrub-test: wait longer for update
qa/tasks/ceph_manager: capture stderr for COT
qa/suites/rados/ceph: drop opensuse for now
mon/MonClient: send logs to mon on separate schedule than pings
mgr/dashboard: Fix missing ImageSpec usage
mgr/dashboard: Allow removing RBD with snapshots
mgr/dashboard: Refactor and cleanup tasks.mgr.dashboard.test_user
mgr/dashboard: support multiple DriveGroups when creating OSDs
mon/MonClient: send logs to mon even if we have no keelalive2
cephadm: flag dashboard user to change password
Reviewed-by: Sebastian Wagner <swagner@suse.com>
`apparmor-abstractions` contains a profile that is required to run podman containers.
Fixes: https://tracker.ceph.com/issues/44272
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Until now, "ocf" and "libradosstriper" were disabled on SLE, but not
openSUSE.
Leaving them enabled for openSUSE makes it appear as if these features
are expected to do something useful on SUSE.
Dropping the "is_opensuse" conditional has the desirable side effect of
streamlining the SUSE bcond block, and in the spirit of "and that's not
all", we take the opportunity to put the bconds in alphabetical order
for comforting cosmetic effect.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
ceph-mgr depends on it in the sense that these plugins are
a hard dependency of ceph-mgr. like cmake depends on cmake-data,
even if the later cannot function on its own without the former.
the reason why we need a separated package is that mgr modules
are written in python, and should be arch independent. this change
helps the downstream to understand this and help to minimize the
divergence between upstream and downstream, because distros like
debian and its derivatives encourage splitting architecture independent
data into its own package.
see
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.html#architecture-independent-data
also, the python related runtime dependencies are also moved into
ceph-mgr-modules-core.
if a python dependency is shared by another ceph-mgr-* package and
ceph-mgr-modules-core. it's only added to ceph-mgr-modules-core for
simplicity and consistency. as the name of dependency might vary
on different distros, so it's tedious and error-prone to repeat
these conditons and checks.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The coverage package under openSUSE (and other distros) are named as
python{major_version}-coverage (without minor version).
Fixes: https://tracker.ceph.com/issues/44164
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
This switch sets CMAKE_VERBOSE_MAKEFILE and causes the CMakeOutput.log
and CMakeError.log files to be dumped after cmake execution finishes.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
1- Strictly speaking, docker is okay too.
2- We don't want podman (or docker) installed inside the container image,
but we do need the cephadm package.
Fixes: https://tracker.ceph.com/issues/44065
Signed-off-by: Sage Weil <sage@redhat.com>
pkgconfig() is cross-distro and allows to 'not having to care' for distro specific
packaging names. On openSUSE/SUSE distros, for udev/libudev, we gain the
benefit of being able to use -mini flavors which are sooner in the build queues.
Signed-off-by: Dominique Leuenberger <dimstar@opensuse.org>
When running 'FOR_MAKE_CHECK=1 ./install-deps.sh' in CentOS 8
these dependencies were not being installed.
Missing dependencies are provided by
https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
We are currently running into issues when installing the test builds
produced on centos 8 as these might have older selinux-policy-base
version than is available to the rhel 8 nodes.
We still want the builds produced on RHEL 8 to use the proper version
condition so the %{?centos} macro should help us achieve that, here.
Signed-off-by: Boris Ranto <branto@redhat.com>
* package .pyc files on el7
per
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_byte_compiling
it's required to package .pyc files
* use python3 to bytecompile .py files on el7
per
https://fedoraproject.org/wiki/Changes/No_more_automagic_Python_bytecompilation#Status_quo,
`brp-python-bytecompile` script bytecompile .py files automatically, the
.py files under interpreter-specific directories, such as `/usr/lib/python3.6/`,
are compiled using the appropriate compiler. but the mgr modules are
located under `/usr/share/ceph/mgr/`. they are compiled using
`${__python}`, which is still python2 in RHEL/CentOS 7. since we've
dropped the support of python2 in octopus. we should either stop
bytecompiling the .py source files or start using python3 to compile
them. otherwise python2 will fail to compile source files using python3
specific syntax like:
```
Compiling /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/15.0.0-9242-g400d98b/rpm/el7/BUILDROOT/ceph-15.0.0-9242.g400d98b.el7.x86_64/usr/share/ceph/mgr/telemetry/module.py
...
File "/usr/share/ceph/mgr/telemetry/module.py", line 387
anon_devid = f"{devid.rsplit('_', 1)[0]}_{uuid.uuid1()}"
^
SyntaxError: invalid syntax
error: Bad exit status from /var/tmp/rpm-tmp.4oRsox (%install)
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change partially reverts e92cb7a0. as these packages are now
available in AppStream, BaseOS or PowerTools in el8, in this change,
they are re-enabled.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/32232/head:
qa: no need to exclude ceph-mgr-diskprediction-cloud from package list to be installed
qa/packages: do not install ceph-mgr-diskprediction-cloud by default
ceph.spec.in: add runtime deps for mgr-diskprediction-cloud
Reviewed-by: Sage Weil <sage@redhat.com>
Majority of radosgw is contained in libradosgw.so. (/usr)/bin/radosgw
is now a few lines that calls radosgw_Main() in libradosgw.so.
The "zipper" work to modularize storage back-ends that will require
linking to the shared library to resolve the methods that they reference.
Putting the bulk of the implementation in a shared lib also allows for
unit testing to link with the shared lib for testing.
radosgw_Main() is the C++ implementation. For C linkage, radosgw_main()
is provided.
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
We need to figure out if/how we can make this as tolerant as possible for
the curl users so that it can still run on a python2-only host.
Maybe.
Signed-off-by: Sage Weil <sage@redhat.com>
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
python version for python binding and ceph-mgr embedded python
interpreter anymore. as before switching to python3-only build,
we can build python3 and python2 python bindings, and ceph-mgr
can use either of them. but after switching to python3-only
build, ceph-mgr has to use whatever python version used to
build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
parameter anymore.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The copyright notice should be the first thing in the file.
The line being dropped was added, perhaps unintentionally, by
unrelated commit c52eb995e0
"Add initial SELinux support"
Signed-off-by: Nathan Cutler <ncutler@suse.com>
cephadm mounts `/run/lvm` into OSD containers, which only exists,
if `lvm2` is installed on the container host. Otherwise we cannot
start ceph-volume or osd containers.
Fixes: https://tracker.ceph.com/issues/43368
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
since we've moved to py3-only world, there is no need to depend on
pyOpenSSL anymore.
also, _python_buildid was removed in
3bf8b4d7d6 .
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/32252/head:
qa/cephfs/begin: libaio-devel on el8
qa/tasks: nosetests -> python -m nose
qa/tasks/rbd_fio: fio 2.21 -> 3.16
src/test/cli-integration/rbd/snap-diff.t: python -> python
qa/workunits: use nose 3
qa/tasks/cbt: install python3 deps
qa/tasks/ceph_manager.py: do not use python to write a file
test/pybind/test_rados: execute takes a bytes (not str) payload
qa/packages/packages: python[3]-ceph is no more
qa: use python3 for venvs etc
packaging: remove python3-ipaddres, as it is part of the stdlib in py3
qa/packages: python-ceph -> python3-ceph
qa/distros: centos7 -> centos8, rhel7 -> rhel8
spec: remove _python_buildid in favor of python3_pkgversion macro
spec: remove python2 packages and conditions
debian: remove python >= 2.7 requirement
debian: add mgr python versions
debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
debian: update control file to use python3 dependency names
debian: remove all python2 overrides and declarations
debian: remove all python2 install files
Reviewed-by: Alfredo Deza <adeza@redhat.com>
please note, these dependencies are not available in el7/el8, and
python2-{grpcio,protobuf} do not exist in fedora core. also, the
`google.api` module is still missing even with these dependencies
installed. so consider it as a partial fix.
See-also: https://tracker.ceph.com/issues/42655
Signed-off-by: Kefu Chai <kchai@redhat.com>
This commit fixes RPMLINT issue:
cephadm.x86_64: E: summary-not-capitalized (Badness: 20) C cephadm utility to bootstrap Ceph clusters
Summary doesn't begin with a capital letter.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Also introduce a cephadm group. Since the cephadm package does not
require ceph-common, the ceph group is not available to be used.
Drop the -r option to useradd, because cephadm does not qualify as a system
user.
Finally, fix the SUSE RPM build. SUSE RPM builds using "osc" implement a check
that compares the cephadm package's "before" and "after" file lists, to enforce
idempotency of the scriptlets. The "userdel cephadm" in the %postun breaks this
check, so omit it from the SUSE RPM builds.
Fixes: https://tracker.ceph.com/issues/43285
Signed-off-by: Nathan Cutler <ncutler@suse.com>
The binary /usr/bin/rpmspec was recently moved to rpm-build, breaking
install-deps.sh on openSUSE Tumbleweed. The package is not strictly
needed for SLE-15-SP* and openSUSE Leap 15.*, but it doesn't hurt to
have it, and will future-proof these distros from this regression.
Putting the dependency in the spec file does not address the issue,
because /usr/bin/rpmspec must be available before install-deps.sh runs it to
determine the dependencies, but it's nice to have it explicitly listed there,
since it *is* a dependency of "make check" on SUSE distros.
SUSE versions < 15 are of no interest in master/octopus+.
Fixes: https://tracker.ceph.com/issues/42612
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Moving ceph-daemon into src/ceph-daemon/ makes it simpler to add extra
code (eg. tox.ini, README, unittests, ...) specific to ceph-daemon.
That way related files are in a single directory.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
otherwise we might have:
```
ceph/src/pybind/mgr/k8sevents/__init__.py", line 1, in <module>
from .module import Module
File "/home/kchai/ceph/src/pybind/mgr/k8sevents/module.py", line 28, in <module>
import yaml
ImportError: No module named yaml
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
RHEL/CentOS 8 now provide librabbitmq-devel so we can enable it as a
build requirement.
Fixes: https://tracker.ceph.com/issues/38466
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
to be consistent with other python2 packages, and their python3
counterparts
the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel
Signed-off-by: Kefu Chai <kchai@redhat.com>
our python3 bindings are now named `python3-<modname>` after python3 is
now maintained by RHEL/CentOS instead EPEL. to help the users using
`python36-<modname>`, we should "Provide" `python36-<modname>`.
the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel. and we do install
`python36-devel` in install-deps.sh, and install `python3-devel` in
ceph-*build/build/setup_rpm
see also
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/30738/head:
mgr/alerts: raise health alert if smtplib has a problem
mgr/alerts: simple module to send health alerts
Reviewed-by: Tim Serong <tserong@suse.com>
* refs/pull/30603/head:
ceph-daemon: -n type.id instead of -i id
ceph-daemon: drop unused VERSION
ceph-daemon: clean up dir helpers, tighten up permissions
ceph-daemon: fchmod before writing to keyring file
test_ceph_daemon.sh: skip ssh until container image has remoto
ceph-daemon: decode utf-8 in run() helper
mgr/ssh: clean up debug cruft
mgr/ssh: clean up bare except: block
ceph-daemon: clean up bare except: blocks
ceph-daemon: all imports to top
ceph-volume: no_tmpfs -> tmpfs
doc/bootstrap: add new bootstrap documentation
ceph-daemon: add --output-pub-ssh-key for bootstrap
ceph-daemon: make 'shell' easier to use
ceph-daemon: support docker; prefer podman
qa: add ceph-daemon
debian: ceph-daemon package, required by ceph-mgr-ssh
ceph.spec.in: ceph-daemon package, required by ceph-mgr
common/options: cleanup whitespace
mgr/ssh: simplify getting the cluster fsid
mgr/ssh: pipe ceph-daemon script to stdin of python3
ceph-daemon: add support for args and/or stdin from top of script
ceph-daemon: make ceph-volume use get_config_and_keyring
ceph-daemon: ls: behave if /var/log/ceph doesn't exist
ceph-daemon: implement 'adopt' for legacy style daemons
ceph-daemon: fix fsid detection for legacy osds
ceph-daemon: make rm-cluster clean up system-ceph*.slice too
ceph-daemon: configure ssh orchestrator
ceph-daemon: be more restrictive with file permissions
mgr/ssh: create osd with ceph-daemon
mgr/ssh: pass daemon id separately to _create_daemon
ceph-daemon: add --config-and-keyring to ceph-volume command
ceph-daemon: create log path for shell (if needed)
mgr/ssh: use _run_ceph_daemon for _create_daemon
mgr/ssh: factor _run_ceph_daemon out of _get_device_inventory
mon/ConfigMonitor: allow entity type only for 'config get'
ceph-daemon: add ceph-volume subcommand
ceph-daemon: remove unused CephContainer dname property
ceph-daemon: drop useless uid/gid checks
mgr/ssh: deploy new mgrs with ceph-daemon
mgr/ssh: factor _create_daemon out of create_mon
mon/MonCap: allow mgr to create new auth keys
mgr/ssh: run c-v with podman when getting inventory
mgr/ssh: simplify ssh connection management
mgr/ssh: use ceph-daemon for deploying mon
ceph-daemon: allow --mon-network for deploying new mon (vs specifying IP)
ceph-daemon: --config-and-keyring (not key)
common/options: add 'image' config option
test_ceph_daemon: specify image name
vstart.sh: add --ssh to enable+configure ssh orchestrator
mgr/ssh: use ssh identity from config-key, if present
mgr/ssh: hardcode default ssh_config
ceph-daemon: store ssh identity in mon config-key store
ceph-daemon: --privileged arg for 'exec'
ceph-daemon: make deploy work for osd (do a c-v prepare)
ceph-daemon: make shell privileged
ceph-daemon: move get_container_mounts to a helper
ceph-daemon: pass full path for entrypoint
ceph-daemon: make id portion of 'shell' optional
ceph-volume: accept --no-tmpfs argument for bluestore
ceph-daemon: 'unit' command
ceph-daemon: fix run command to use call(), not check_output()
src/ceph-daemon: whitespace
ceph-daemon: add 'enter', 'exec' commands
ceph-daemon: bind config to default location
test_ceph_daemon.sh: test deploy mds too
ceph-daemon: generate ssh keys
ceph-daemon: --config, not --conf
ceph-daemon: long lines
ceph-daemon: add --config to bootstrap
ceph-daemon: add 'shell' command
ceph-daemon: do not import subprocess symbols directly
ceph-daemon: add mons with 'deploy mon.x ...'
ceph-daemon: add 'ls'
ceph-daemon: simplify uid/gid a bit
ceph-daemon: fix libudev
ceph-daemon: autodetect uid/gid from container image
ceph-daemon: default to empty log files, log to stderr (systemd journal)
ceph-daemon: rm-{daemon,cluster}
ceph-daemon: fix bootstrap config
ceph-daemon: fix args.fsid usage
ceph-daemon: be careful overwriting live files
ceph-daemon: slurp some options over from the standard systemd unit
ceph-daemon: add ceph.target and ceph-$fsid.target units
test_ceph_daemon.sh: stupid test script
ceph-daemon: bootstrap and deploy (mgr) work
ceph-daemon: initial checkin
ceph-mon: fix debug print of public_addr
Initialy SMTP support is implemented; nothing else.
This is just smart enough for me to get emails from my home cluster when
something goes wrong. No bells and whistled at this point.
Signed-off-by: Sage Weil <sage@redhat.com>
