Commit Graph

1183 Commits

Author SHA1 Message Date
Sage Weil
582d713e46 Merge PR #31806 into master
* refs/pull/31806/head:
	spec,debian: ceph-mgr-ssh depends on openssh{-client{s}}

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2019-11-23 20:29:38 -06:00
Sage Weil
378fed3aa0 Merge PR #31698 into master
* refs/pull/31698/head:
	rpm,deb: add cephdaemon user and sudoers file
	ceph-daemon: use mgr/ssh command to create the ssh key
	mgr/ssh: add 'ssh generate-key' and 'ssh clear-key' commands
	mgr/ssh: add mode option
	mgr/ssh: add 'ssh get-pub-key' and 'ssh get-user' commands
	mgr/ssh: convert to command decorations
	mgr/orchestrator: move command annotation helpers to orchestrator.py

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-22 11:24:27 -06:00
Sebastian Wagner
e6909311bd spec,debian: ceph-mgr-ssh depends on openssh{-client{s}}
Container images don't include ssh by default. Thus we
have to explictly depend on it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-11-22 10:32:14 +01:00
Sage Weil
660233935e rpm,deb: add cephdaemon user and sudoers file
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-21 15:34:07 -06:00
Casey Bodley
8aa638b0fa
Merge pull request #30960 from yuvalif/wip-yuval-add-kafka-notif-endpoint
rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-11-21 14:42:23 -05:00
Thomas Bechtold
4258c4772a ceph-daemon: Move ceph-daemon executable to own directory
Moving ceph-daemon into src/ceph-daemon/ makes it simpler to add extra
code (eg. tox.ini, README, unittests, ...) specific to ceph-daemon.
That way related files are in a single directory.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2019-11-08 17:05:57 +01:00
Yuval Lifshitz
354d775e04 rgw/pubsub: add kafka notification endpoint
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-05 19:25:03 +02:00
Kefu Chai
97bf619e1c ceph.spec.in: run pathfix.py
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-01 10:26:23 -05:00
Kefu Chai
8adbb86fb8 ceph.spec.in: add missing python-yaml dependency for mgr-k8sevents
otherwise we might have:

```
ceph/src/pybind/mgr/k8sevents/__init__.py", line 1, in <module>
    from .module import Module
  File "/home/kchai/ceph/src/pybind/mgr/k8sevents/module.py", line 28, in <module>
    import yaml
ImportError: No module named yaml
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-27 12:46:15 +08:00
Brad Hubbard
c44c140dfa ceph.spec.in: Enable amqp_endpoint on RHEL8 by default
RHEL/CentOS 8 now provide librabbitmq-devel so we can enable it as a
build requirement.

Fixes: https://tracker.ceph.com/issues/38466

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-10-25 18:18:23 +10:00
Nathan Cutler
15bfb819ff rpm: make librados2, libcephfs2 own (create) /etc/ceph
Fixes: https://tracker.ceph.com/issues/42352
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-10-22 10:52:45 +02:00
Kefu Chai
8ae022a3ce
Merge pull request #30830 from tchaikov/wip/rpm-install-deps/python_provide
install-deps, rpm: use python_provide macro and cleanups

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-10-15 00:05:06 +08:00
Kefu Chai
fb6290b1fa ceph.spec.in: provide python2-<modname>
to be consistent with other python2 packages, and their python3
counterparts

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-10 10:35:59 +08:00
Kefu Chai
50b19e673d ceph.spec.in: use python_provide macro
our python3 bindings are now named `python3-<modname>` after python3 is
now maintained by RHEL/CentOS instead EPEL. to help the users using
`python36-<modname>`, we should "Provide" `python36-<modname>`.

the `python_provide` macro is offered by `python-rpm-macros` package,
which is in turn required by python*-devel. and we do install
`python36-devel` in install-deps.sh, and install `python3-devel` in
ceph-*build/build/setup_rpm

see also
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-10 10:34:47 +08:00
Sage Weil
5f15829174 Merge PR #30738 into master
* refs/pull/30738/head:
	mgr/alerts: raise health alert if smtplib has a problem
	mgr/alerts: simple module to send health alerts

Reviewed-by: Tim Serong <tserong@suse.com>
2019-10-09 14:13:40 -05:00
Sage Weil
6fd67f19e3 Merge PR #30603 into master
* refs/pull/30603/head:
	ceph-daemon: -n type.id instead of -i id
	ceph-daemon: drop unused VERSION
	ceph-daemon: clean up dir helpers, tighten up permissions
	ceph-daemon: fchmod before writing to keyring file
	test_ceph_daemon.sh: skip ssh until container image has remoto
	ceph-daemon: decode utf-8 in run() helper
	mgr/ssh: clean up debug cruft
	mgr/ssh: clean up bare except: block
	ceph-daemon: clean up bare except: blocks
	ceph-daemon: all imports to top
	ceph-volume: no_tmpfs -> tmpfs
	doc/bootstrap: add new bootstrap documentation
	ceph-daemon: add --output-pub-ssh-key for bootstrap
	ceph-daemon: make 'shell' easier to use
	ceph-daemon: support docker; prefer podman
	qa: add ceph-daemon
	debian: ceph-daemon package, required by ceph-mgr-ssh
	ceph.spec.in: ceph-daemon package, required by ceph-mgr
	common/options: cleanup whitespace
	mgr/ssh: simplify getting the cluster fsid
	mgr/ssh: pipe ceph-daemon script to stdin of python3
	ceph-daemon: add support for args and/or stdin from top of script
	ceph-daemon: make ceph-volume use get_config_and_keyring
	ceph-daemon: ls: behave if /var/log/ceph doesn't exist
	ceph-daemon: implement 'adopt' for legacy style daemons
	ceph-daemon: fix fsid detection for legacy osds
	ceph-daemon: make rm-cluster clean up system-ceph*.slice too
	ceph-daemon: configure ssh orchestrator
	ceph-daemon: be more restrictive with file permissions
	mgr/ssh: create osd with ceph-daemon
	mgr/ssh: pass daemon id separately to _create_daemon
	ceph-daemon: add --config-and-keyring to ceph-volume command
	ceph-daemon: create log path for shell (if needed)
	mgr/ssh: use _run_ceph_daemon for _create_daemon
	mgr/ssh: factor _run_ceph_daemon out of _get_device_inventory
	mon/ConfigMonitor: allow entity type only for 'config get'
	ceph-daemon: add ceph-volume subcommand
	ceph-daemon: remove unused CephContainer dname property
	ceph-daemon: drop useless uid/gid checks
	mgr/ssh: deploy new mgrs with ceph-daemon
	mgr/ssh: factor _create_daemon out of create_mon
	mon/MonCap: allow mgr to create new auth keys
	mgr/ssh: run c-v with podman when getting inventory
	mgr/ssh: simplify ssh connection management
	mgr/ssh: use ceph-daemon for deploying mon
	ceph-daemon: allow --mon-network for deploying new mon (vs specifying IP)
	ceph-daemon: --config-and-keyring (not key)
	common/options: add 'image' config option
	test_ceph_daemon: specify image name
	vstart.sh: add --ssh to enable+configure ssh orchestrator
	mgr/ssh: use ssh identity from config-key, if present
	mgr/ssh: hardcode default ssh_config
	ceph-daemon: store ssh identity in mon config-key store
	ceph-daemon: --privileged arg for 'exec'
	ceph-daemon: make deploy work for osd (do a c-v prepare)
	ceph-daemon: make shell privileged
	ceph-daemon: move get_container_mounts to a helper
	ceph-daemon: pass full path for entrypoint
	ceph-daemon: make id portion of 'shell' optional
	ceph-volume: accept --no-tmpfs argument for bluestore
	ceph-daemon: 'unit' command
	ceph-daemon: fix run command to use call(), not check_output()
	src/ceph-daemon: whitespace
	ceph-daemon: add 'enter', 'exec' commands
	ceph-daemon: bind config to default location
	test_ceph_daemon.sh: test deploy mds too
	ceph-daemon: generate ssh keys
	ceph-daemon: --config, not --conf
	ceph-daemon: long lines
	ceph-daemon: add --config to bootstrap
	ceph-daemon: add 'shell' command
	ceph-daemon: do not import subprocess symbols directly
	ceph-daemon: add mons with 'deploy mon.x ...'
	ceph-daemon: add 'ls'
	ceph-daemon: simplify uid/gid a bit
	ceph-daemon: fix libudev
	ceph-daemon: autodetect uid/gid from container image
	ceph-daemon: default to empty log files, log to stderr (systemd journal)
	ceph-daemon: rm-{daemon,cluster}
	ceph-daemon: fix bootstrap config
	ceph-daemon: fix args.fsid usage
	ceph-daemon: be careful overwriting live files
	ceph-daemon: slurp some options over from the standard systemd unit
	ceph-daemon: add ceph.target and ceph-$fsid.target units
	test_ceph_daemon.sh: stupid test script
	ceph-daemon: bootstrap and deploy (mgr) work
	ceph-daemon: initial checkin
	ceph-mon: fix debug print of public_addr
2019-10-07 15:31:14 -05:00
Sage Weil
d7223938f8 mgr/alerts: simple module to send health alerts
Initialy SMTP support is implemented; nothing else.

This is just smart enough for me to get emails from my home cluster when
something goes wrong.  No bells and whistled at this point.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-07 09:04:39 -05:00
Kefu Chai
73ee20ea26
Merge pull request #30656 from tchaikov/wip-install-deps-for-el8
install-deps.sh: enable PowerTool repo for EL8

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-07 16:06:30 +08:00
Sage Weil
171fecc38a ceph.spec.in: ceph-daemon package, required by ceph-mgr
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 20:33:35 -05:00
Kefu Chai
0098b42927 ceph.spec.in: install python3-Cython on el8
EL8 has python3 support. no need to wait for python3-Cython to populate
to its mirrors.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-01 18:28:17 +08:00
Kefu Chai
e92cb7a033 ceph.spec.in: disable "make check" for rhel8
following packages are not available on RHEL/CentOS 8, they are for
dashboard unit tests. so disable all python packages for dashboard
tests for now:

python-cherrypy
python-coverage
python-pecan
python-routes
python-tox
xmlsec1-devel

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-28 22:20:35 +08:00
Kefu Chai
dd133840b9 ceph.spec.in: disable cephfs_java and amqp_endpoint if RHEL8
RHEL/CentOS 8 does not offer junit and sharutils yet, so disable
cephfs_java on rhel8 at this moment.

RHEL/CentOS 8 does not offer librabbitmq-devel yet, so disable
amqp_endpoint on rhel8 at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-28 22:20:35 +08:00
Jeff Layton
40c5a18f81 ceph.spec.in: fix Cython package dependency for Fedora
Fedora distros do not have python3?-Cython packages, but they do have
python3-Cython ones. Fix the BuildRequires so that we only use the
python3_version_nodots based version string for RHEL.

Fixes: https://tracker.ceph.com/issues/42032
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-26 08:20:57 -04:00
Kefu Chai
b43cbefc05
Merge pull request #30420 from runsisi/wip-remove-requests
rpm,deb: python-requests is not needed for ceph-common

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-25 16:31:13 +08:00
Kefu Chai
6d2f66846d ceph.spec.in: s/pkgversion/version_nodots/
`python3_pkgversion` is now defined as 3, while we don't have packages
like python3-Cython yet in EPEL7. but we do have `python36-Cython`. so
let's use `python3_version_nodots` instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-24 23:23:35 +08:00
luo.runbing
998a3025c0
rpm,deb: python-requests is not needed for ceph-common
Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
2019-09-23 09:56:19 +08:00
Jeff Layton
eae0127513 mount.ceph: fork a child to get info from local configuration
When a secret and/or the mon addrs are not specified by the admin,
then mmap a MAP_SHARED buffer and spawn a child process to get that
info. For safety reasons, the child drops all capabilities other than
CAP_DAC_READ_SEARCH (to ensure that it'll be able to read the keyring,
should one be found). To achieve this, we add a new dependency on
libcap-ng.

Add a new C++ file with a single routine that will create a CephContext,
get a list of monitor addresses and scrape the keyring for a secret for
the specified cephx user.

If that info is found, then it is copied to fixed-length buffers in the
MAP_SHARED area and the child exits successfully.

The parent will then vet the returned info and copy it into the
appropriate fields if they are currently blank.

Fixes: https://tracker.ceph.com/issues/16656
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-13 08:14:48 -04:00
Kefu Chai
4d72fba539
Merge pull request #30126 from dvanders/dvanders_spec
ceph.spec.in: reserve more memory per build job

Reviewed-By: Nathan Cutler <nculter@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-10 16:30:09 +08:00
Sebastian Wagner
199d6418f4
Merge pull request #29520 from pcuzner/add-events-mgr-module
mgr/k8sevents: Add mgr module for kubernetes event integration

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2019-09-06 17:09:21 +02:00
Dan van der Ster
e466c43b23 ceph.spec.in: reserve 2500MB per build job
On our 14GB build machines, 8 parallel jobs results in the compiler
going OOM. Reserving 2500MB per job allows it to succeed with 5 jobs.

Fixes: https://tracker.ceph.com/issues/41659
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2019-09-06 08:52:21 +02:00
Paul Cuzner
2166676288 mgr/k8sevents: Initial ceph -> k8s events integration
Adds a mgr module to provide integration between
Ceph and the kubernetes events API within the rook-ceph
namespace. It provides several commands to view event
status including;

ceph k8sevents list .... show all k8s related events
ceph k8sevents ceph .... show events generated from this
                         module
ceph k8sevents status .. show status of the tracker threads
                         and counts of tracked events

Events sent to kubernetes are sourced from the
a) clog: ceph healtchecks and admin commands
b) explicit checks: hosts, pools and OSD states are checked
   every 'n' seconds

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2019-09-05 14:45:52 +12:00
Kefu Chai
33dfffc841 ceph.spec.in: use g++ >= 8.3.1-3.1
since https://bugzilla.redhat.com/show_bug.cgi?id=1726630 has been
fixed. and we have devtoolset-8-gcc-c++-8.3.1-3.1.el7.x86_64.rpm.

no reason to stick with devtoolset-8-gcc-c++-8.2.1-3 anymore.

Fixes: https://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-03 14:19:31 +08:00
Kefu Chai
5d6d770e2f install-deps.sh,deb,rpm: move python-saml deps into debian/control and ceph.spec.in
these dependencies are only used for building python-saml which is in
turn used for the SAML support. this feature is tested using
`test_sso.py` while performing dashboard tests. we do not package or
ship python-saml along with other Ceph packages. so let's move these
dependencies to the "make check" sections in ceph.spec.in and
debian/control for simplifying install-deps.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-24 02:11:51 +08:00
Kefu Chai
3234fb3982 cmake,run-make-check.sh,deb,rpm: disable SPDK by default
but we still enable it in `run-make-check.sh`

* cmake: disable SPDK by default
* run-make-check.sh: enable WITH_SPDK so at least we can ensure it
  builds
* deb,rpm: add uuid-dev / libuuid-devel as a "make check" dependency

Fixes: https://tracker.ceph.com/issues/41330
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-20 15:38:33 +08:00
Nathan Cutler
b9c7443010 rpm: always build ceph-test package
Fixes: https://tracker.ceph.com/issues/41296
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-08-15 13:28:51 +02:00
Boris Ranto
d9f8a50a89
Merge pull request #28997 from b-ranto/wip-push-dash
Make ceph-dashboard require grafana dashboards

Reviewed-by: Zack Cerza <zcerza@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-08-15 11:02:43 +02:00
Kefu Chai
bbb69fe793 cmake: require CMake v3.10.2
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-02 22:09:12 +08:00
Kefu Chai
0e35c6f641
Merge pull request #29349 from smithfarm/wip-40975
rpm: put librgw lttng SOs in the librgw-devel package

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:52:07 +08:00
Sebastian Wagner
475159ec73
Merge pull request #28915 from sebastian-philipp/python-common-package-2
python-common:  Python common package

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2019-07-30 09:31:34 +02:00
Kefu Chai
3169da7a1a
Merge pull request #29347 from smithfarm/wip-40974
rpm: improve ceph-mgr plugin package summaries

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-30 12:46:37 +08:00
Nathan Cutler
7929af71d9 rpm: put librgw lttng SOs in the librgw-devel package
Shared objects belong in the devel package.

Fixes: http://tracker.ceph.com/issues/40975
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-07-26 13:10:03 +02:00
Nathan Cutler
ea40019c5c rpm: improve ceph-mgr plugin package summaries
Fixes: http://tracker.ceph.com/issues/40974
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-07-26 13:01:05 +02:00
Nathan Cutler
876d6bf6dc rpm: fdupes in SUSE builds to conform with packaging guidelines
Quoting from https://en.opensuse.org/openSUSE:Packaging_Conventions_RPM_Macros

"%fdupes is generally safe for use on /bin, /lib*, /usr (= %_prefix) and /sbin.
Do note however that files with same content but different ownership, when
hardlinked, will get the ownership of any of its content siblings."

Fixes: http://tracker.ceph.com/issues/40973
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-07-26 12:42:09 +02:00
Sebastian Wagner
ea15a67b98 ceph.spec, debian: Add python-common to ceph-common
Co-authored-by: Tim Serong <tserong@suse.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-07-26 12:10:52 +02:00
Matthias Gerstner
d472e88441
packaging: remove SuSEfirewall2 support
removal of SuSEfirewall2 service, since SuSEfirewall2 has been replaced
by firewalld, see [1].

[1]: https://lists.opensuse.org/opensuse-factory/2019-01/msg00490.html

Fixes: http://tracker.ceph.com/issues/40738
Signed-off-by: Matthias Gerstner <matthias.gerstner@suse.de>
2019-07-12 09:55:35 +02:00
Boris Ranto
7b3a01cdfb rpm: Require ceph-grafana-dashboards
The recently introduced 'dashboard grafana dashboards update' command
requires the grafana dashboards for it to work properly. Adding the
dependency.

Signed-off-by: Boris Ranto <branto@redhat.com>
2019-07-11 17:21:55 +02:00
Kefu Chai
00d03f99d0
Merge pull request #28884 from DimStar77/depchain
ceph.spec.in: Drop systemd BuildRequires in case of building for SUSE

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-07-09 16:04:09 +08:00
Kefu Chai
47d4351d3a ceph.spec.in,install-dep.sh: pin to gcc-c++-8.2.1
as an intermediate solution before
https://bugzilla.redhat.com/show_bug.cgi?id=1726630 gets fixed.

Fixes: https://tracker.ceph.com/issues/40646
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-05 23:59:21 +08:00
Dominique Leuenberger
b48ea142de
Packaging: Drop systemd BuildRequires in case of building for SUSE
There is already pkgconfig(systemd) present, which, in SUSE's packaging,
translates to the same package (systemd.pc is shipped as part of the main
systemd package). Not explicitly mentioning 'systemd' as package name allows
the openSUSE Build Service though to find shortcuts by using the
bootstrap packages, i.e. systemd-mini (ABI/API complete).

Signed-off-by: Dominique Leuenberger <dimstar@opensuse.org>
2019-07-05 09:15:20 +02:00
Sage Weil
18eacb6ed8 Merge PR #22446 into master
* refs/pull/22446/head:
	relicense LGPL-2.1 code as LGPL-2.1 or LGPL-3.0
	COPYING: add reference to a few Apache-2.0 licensed source files
	debian/copyright: sync with COPYING
	COPYING: LGPL2.1 -> LGPL-2.1
	COPYING: extra bits from debian/copyright

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-06-12 09:11:49 -05:00