Commit Graph

1220 Commits

Author SHA1 Message Date
Nathan Cutler
6dbe449efe
Merge pull request #32805 from smithfarm/wip-cmake-logging
rpm: add cmake_verbose_logging switch

Reviewed-by: David Disseldorp <ddiss@suse.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2020-02-13 20:00:27 +01:00
Nathan Cutler
5e98d22380 rpm: add cmake_verbose_logging switch
This switch sets CMAKE_VERBOSE_MAKEFILE and causes the CMakeOutput.log
and CMakeError.log files to be dumped after cmake execution finishes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-02-13 15:13:24 +01:00
Sebastian Wagner
615381659c cmake: Integrate Rook client generation
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-13 11:33:55 +01:00
Sage Weil
070df40773 Merge PR #33229 into master
* refs/pull/33229/head:
	rpm: define weak_deps for el8

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-12 08:28:51 -06:00
Kefu Chai
5c70f36111 rpm: define weak_deps for el8
RHEL/CentOS 8 comes with rpm 4.14, see
https://centos.pkgs.org/8/centos-baseos-x86_64/rpm-4.14.2-25.el8.x86_64.rpm.html
and
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/packaging_and_distributing_software/new-features-in-rhel-8_packaging-and-distributing-software

and since "Recommends" was introduced by rpm 4.12, see
https://fedoraproject.org/wiki/Changes/RPM-4.12 .
so we are able to use "Recommends" in el8 as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-12 12:28:35 +08:00
Sage Weil
b16f19efe7 ceph.spec: Recommend (but do not require) podman
1- Strictly speaking, docker is okay too.
2- We don't want podman (or docker) installed inside the container image,
but we do need the cephadm package.

Fixes: https://tracker.ceph.com/issues/44065
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-11 19:51:23 -06:00
Sage Weil
2e3ed251a6 mgr/deepsea: remove
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-09 09:45:36 -06:00
Sage Weil
01c36d9db7 mgr/ansible: remove
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 09:45:12 -06:00
Greg Farnum
1adda08a4b rpm: fix up a specfile syntax error
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-02-04 09:10:17 -08:00
Dominique Leuenberger
110135cd16
ceph.spec.in: Use pkgconfig() style BuildRequires for udev/libudev-devel
pkgconfig() is cross-distro and allows to 'not having to care' for distro specific
packaging names. On openSUSE/SUSE distros, for udev/libudev, we gain the
benefit of being able to use -mini flavors which are sooner in the build queues.

Signed-off-by: Dominique Leuenberger <dimstar@opensuse.org>
2020-01-28 14:13:04 +01:00
Alfonso Martínez
e5fa448229 ceph.spec.in: fix 'make check' deps for centos8
When running 'FOR_MAKE_CHECK=1 ./install-deps.sh' in CentOS 8
these dependencies were not being installed.
Missing dependencies are provided by
https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-01-23 11:16:27 +01:00
Boris Ranto
f4cf78dc56 rpm: Relax the selinux policy version for centos builds
We are currently running into issues when installing the test builds
produced on centos 8 as these might have older selinux-policy-base
version than is available to the rhel 8 nodes.

We still want the builds produced on RHEL 8 to use the proper version
condition so the %{?centos} macro should help us achieve that, here.

Signed-off-by: Boris Ranto <branto@redhat.com>
2020-01-17 15:24:46 +01:00
Kefu Chai
567a94e63d ceph.spec.in: package .pyc files on el7
* package .pyc files on el7
  per
  https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_byte_compiling

  it's required to package .pyc files

* use python3 to bytecompile .py files on el7

  per
  https://fedoraproject.org/wiki/Changes/No_more_automagic_Python_bytecompilation#Status_quo,
  `brp-python-bytecompile` script bytecompile .py files automatically, the
  .py files under interpreter-specific directories, such as `/usr/lib/python3.6/`,
  are compiled using the appropriate compiler. but the mgr modules are
  located under `/usr/share/ceph/mgr/`. they are compiled using
  `${__python}`, which is still python2 in RHEL/CentOS 7. since we've
  dropped the support of python2 in octopus. we should either stop
  bytecompiling the .py source files or start using python3 to compile
  them. otherwise python2 will fail to compile source files using python3
  specific syntax like:

```
Compiling /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/15.0.0-9242-g400d98b/rpm/el7/BUILDROOT/ceph-15.0.0-9242.g400d98b.el7.x86_64/usr/share/ceph/mgr/telemetry/module.py
...
  File "/usr/share/ceph/mgr/telemetry/module.py", line 387
    anon_devid = f"{devid.rsplit('_', 1)[0]}_{uuid.uuid1()}"
                                                           ^
SyntaxError: invalid syntax

error: Bad exit status from /var/tmp/rpm-tmp.4oRsox (%install)
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-13 11:09:37 +08:00
Nathan Cutler
0988cb0d2f
Merge pull request #32331 from smithfarm/wip-spec-drop-vim-header
rpm: drop vim-specific header

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-01-08 14:51:33 +01:00
Kefu Chai
dfd90da59c
Merge pull request #32412 from tchaikov/wip-spec-more-deps-for-el8
ceph.spec.in: re-enable "make check" deps for el8

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2020-01-07 22:53:14 +08:00
Daniel Gryniewicz
2ee55c2cf7
Merge pull request #32404 from kalebskeithley/master
rgw: build radosgw daemon as a shared lib + small executable
2020-01-07 08:05:21 -05:00
Kefu Chai
2a27084e1d ceph.spec.in: re-enable "make check" deps for el8
this change partially reverts e92cb7a0. as these packages are now
available in AppStream, BaseOS or PowerTools in el8, in this change,
they are re-enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-26 14:03:08 +08:00
Sage Weil
5ec92e79a2 Merge PR #32232 into master
* refs/pull/32232/head:
	qa: no need to exclude ceph-mgr-diskprediction-cloud from package list to be installed
	qa/packages: do not install ceph-mgr-diskprediction-cloud by default
	ceph.spec.in: add runtime deps for mgr-diskprediction-cloud

Reviewed-by: Sage Weil <sage@redhat.com>
2019-12-24 08:17:35 -06:00
Kaleb S. KEITHLEY
f528f173e4 rgw: build radosgw daemon as a shared lib + small executable
Majority of radosgw is contained in libradosgw.so. (/usr)/bin/radosgw
is now a few lines that calls radosgw_Main() in libradosgw.so.

The "zipper" work to modularize storage back-ends that will require
linking to the shared library to resolve the methods that they reference.

Putting the bulk of the implementation in a shared lib also allows for
unit testing to link with the shared lib for testing.

radosgw_Main() is the C++ implementation. For C linkage, radosgw_main()
is provided.

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-12-23 12:34:44 -05:00
Sage Weil
074f1ff246 cephadm: python3 shebang
We need to figure out if/how we can make this as tolerant as possible for
the curl users so that it can still run on a python2-only host.

Maybe.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 12:47:56 -06:00
Kefu Chai
5fc657b40d cmake: drop WITH_PYTHON2 option
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
  python version for python binding and ceph-mgr embedded python
  interpreter anymore. as before switching to python3-only build,
  we can build python3 and python2 python bindings, and ceph-mgr
  can use either of them. but after switching to python3-only
  build, ceph-mgr has to use whatever python version used to
  build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
  and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
  parameter anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 10:58:58 +08:00
Sage Weil
9fc9197db0 Merge PR #32323 into master
* refs/pull/32323/head:
	spec, debian: cephadm requires lvm2

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-12-18 17:10:09 -06:00
Nathan Cutler
48b1753a8a rpm: drop vim-specific header
The copyright notice should be the first thing in the file.

The line being dropped was added, perhaps unintentionally, by
unrelated commit c52eb995e0
"Add initial SELinux support"

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-12-18 18:05:14 +01:00
Sebastian Wagner
b2cf2c7a8d spec, debian: cephadm requires lvm2
cephadm mounts `/run/lvm` into OSD containers, which only exists,
if `lvm2` is installed on the container host. Otherwise we cannot
start ceph-volume or osd containers.

Fixes: https://tracker.ceph.com/issues/43368

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-12-18 14:40:17 +01:00
Kefu Chai
904650262c ceph.spec.in: always depends on python3.6-pyOpenSSL
since we've moved to py3-only world, there is no need to depend on
pyOpenSSL anymore.

also, _python_buildid was removed in
3bf8b4d7d6 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-18 15:01:27 +08:00
Sage Weil
3268ec7ac8 Merge PR #32252 into master
* refs/pull/32252/head:
	qa/cephfs/begin: libaio-devel on el8
	qa/tasks: nosetests -> python -m nose
	qa/tasks/rbd_fio: fio 2.21 -> 3.16
	src/test/cli-integration/rbd/snap-diff.t: python -> python
	qa/workunits: use nose 3
	qa/tasks/cbt: install python3 deps
	qa/tasks/ceph_manager.py: do not use python to write a file
	test/pybind/test_rados: execute takes a bytes (not str) payload
	qa/packages/packages: python[3]-ceph is no more
	qa: use python3 for venvs etc
	packaging: remove python3-ipaddres, as it is part of the stdlib in py3
	qa/packages: python-ceph -> python3-ceph
	qa/distros: centos7 -> centos8, rhel7 -> rhel8
	spec: remove _python_buildid in favor of python3_pkgversion macro
	spec: remove python2 packages and conditions
	debian: remove python >= 2.7 requirement
	debian: add mgr python versions
	debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
	debian: update control file to use python3 dependency names
	debian: remove all python2 overrides and declarations
	debian: remove all python2 install files

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-12-17 15:23:27 -06:00
Kefu Chai
702880dcf4 ceph.spec.in: add runtime deps for mgr-diskprediction-cloud
please note, these dependencies are not available in el7/el8, and
python2-{grpcio,protobuf} do not exist in fedora core. also, the
`google.api` module is still missing even with these dependencies
installed. so consider it as a partial fix.

See-also: https://tracker.ceph.com/issues/42655
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-17 21:52:18 +08:00
Nathan Cutler
88ad653874 rpm/cephadm: start description and summary with capital letter
This commit fixes RPMLINT issue:

cephadm.x86_64: E: summary-not-capitalized (Badness: 20) C cephadm utility to bootstrap Ceph clusters
Summary doesn't begin with a capital letter.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-12-16 17:03:07 +01:00
Nathan Cutler
f05a769e0d rpm/cephadm: move HOMEDIR to /var/lib and make scriptlets idempotent on SUSE
Also introduce a cephadm group. Since the cephadm package does not
require ceph-common, the ceph group is not available to be used.

Drop the -r option to useradd, because cephadm does not qualify as a system
user.

Finally, fix the SUSE RPM build. SUSE RPM builds using "osc" implement a check
that compares the cephadm package's "before" and "after" file lists, to enforce
idempotency of the scriptlets. The "userdel cephadm" in the %postun breaks this
check, so omit it from the SUSE RPM builds.

Fixes: https://tracker.ceph.com/issues/43285
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-12-16 17:03:01 +01:00
Alfredo Deza
7382221989 packaging: remove python3-ipaddres, as it is part of the stdlib in py3
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-12-12 19:16:33 -05:00
Alfredo Deza
3bf8b4d7d6 spec: remove _python_buildid in favor of python3_pkgversion macro
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-12-12 13:10:07 -06:00
Alfredo Deza
29d1c34424 spec: remove python2 packages and conditions
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-12-12 13:07:12 -06:00
Nathan Cutler
be25c6ed0c
Merge pull request #32080 from smithfarm/wip-43171
ceph.spec.in: move distro-conditional deps to dedicated section

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-12-12 10:18:13 +01:00
Sage Weil
cd1c05acbb mgr/ssh -> mgr/cephadm
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-11 19:14:24 -06:00
Sage Weil
c8750b7066 files,rpm,deb: rename ceph-daemon -> cephadm
This is just renaming the files and adjusting the packages.  Lots of
cleanup to do still.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-11 19:14:09 -06:00
Nathan Cutler
e80d14a176 ceph.spec.in: move distro-conditional deps to dedicated section
Post e92cb7a033 cleanup. Restore previous
structure of "distro-conditional make check dependencies" section.

Fixes: https://tracker.ceph.com/issues/43171
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-12-11 15:17:13 +01:00
Nathan Cutler
6378fa45ac rpm: add rpm-build to SUSE-specific make check deps
The binary /usr/bin/rpmspec was recently moved to rpm-build, breaking
install-deps.sh on openSUSE Tumbleweed. The package is not strictly
needed for SLE-15-SP* and openSUSE Leap 15.*, but it doesn't hurt to
have it, and will future-proof these distros from this regression.

Putting the dependency in the spec file does not address the issue,
because /usr/bin/rpmspec must be available before install-deps.sh runs it to
determine the dependencies, but it's nice to have it explicitly listed there,
since it *is* a dependency of "make check" on SUSE distros.

SUSE versions < 15 are of no interest in master/octopus+.

Fixes: https://tracker.ceph.com/issues/42612
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-12-10 18:31:16 +01:00
Sage Weil
582d713e46 Merge PR #31806 into master
* refs/pull/31806/head:
	spec,debian: ceph-mgr-ssh depends on openssh{-client{s}}

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2019-11-23 20:29:38 -06:00
Sage Weil
378fed3aa0 Merge PR #31698 into master
* refs/pull/31698/head:
	rpm,deb: add cephdaemon user and sudoers file
	ceph-daemon: use mgr/ssh command to create the ssh key
	mgr/ssh: add 'ssh generate-key' and 'ssh clear-key' commands
	mgr/ssh: add mode option
	mgr/ssh: add 'ssh get-pub-key' and 'ssh get-user' commands
	mgr/ssh: convert to command decorations
	mgr/orchestrator: move command annotation helpers to orchestrator.py

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-22 11:24:27 -06:00
Sebastian Wagner
e6909311bd spec,debian: ceph-mgr-ssh depends on openssh{-client{s}}
Container images don't include ssh by default. Thus we
have to explictly depend on it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-11-22 10:32:14 +01:00
Sage Weil
660233935e rpm,deb: add cephdaemon user and sudoers file
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-21 15:34:07 -06:00
Casey Bodley
8aa638b0fa
Merge pull request #30960 from yuvalif/wip-yuval-add-kafka-notif-endpoint
rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-11-21 14:42:23 -05:00
Thomas Bechtold
4258c4772a ceph-daemon: Move ceph-daemon executable to own directory
Moving ceph-daemon into src/ceph-daemon/ makes it simpler to add extra
code (eg. tox.ini, README, unittests, ...) specific to ceph-daemon.
That way related files are in a single directory.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-11-08 17:05:57 +01:00
Yuval Lifshitz
354d775e04 rgw/pubsub: add kafka notification endpoint
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-05 19:25:03 +02:00
Kefu Chai
97bf619e1c ceph.spec.in: run pathfix.py
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-01 10:26:23 -05:00
Kefu Chai
8adbb86fb8 ceph.spec.in: add missing python-yaml dependency for mgr-k8sevents
otherwise we might have:

```
ceph/src/pybind/mgr/k8sevents/__init__.py", line 1, in <module>
    from .module import Module
  File "/home/kchai/ceph/src/pybind/mgr/k8sevents/module.py", line 28, in <module>
    import yaml
ImportError: No module named yaml
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 12:46:15 +08:00
Brad Hubbard
c44c140dfa ceph.spec.in: Enable amqp_endpoint on RHEL8 by default
RHEL/CentOS 8 now provide librabbitmq-devel so we can enable it as a
build requirement.

Fixes: https://tracker.ceph.com/issues/38466

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-10-25 18:18:23 +10:00
Nathan Cutler
15bfb819ff rpm: make librados2, libcephfs2 own (create) /etc/ceph
Fixes: https://tracker.ceph.com/issues/42352
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-10-22 10:52:45 +02:00
Kefu Chai
8ae022a3ce
Merge pull request #30830 from tchaikov/wip/rpm-install-deps/python_provide
install-deps, rpm: use python_provide macro and cleanups

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-10-15 00:05:06 +08:00
Kefu Chai
fb6290b1fa ceph.spec.in: provide python2-<modname>
to be consistent with other python2 packages, and their python3
counterparts

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-10 10:35:59 +08:00