Commit Graph

227 Commits

Author SHA1 Message Date
Willem Jan Withagen
1851621793 common: make y2c.py work on FreeBSD
1) make reference to python3 indepedant of explicit path
2) add required py-yaml module to install list

fixes: #40731
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-13 10:17:01 +02:00
Kefu Chai
17d2bc3707 install-deps.sh: install libpmem libraries if WITH_PMEM is set
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-08 17:15:15 +08:00
Kefu Chai
89121189ac cmake,install-deps,make-dist: use boost 1.75
so we can use the latest boost

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 15:09:29 +08:00
Brad Hubbard
2e1ee019d2 install-deps: Use dnf config-manager
Using subscription-manager will fail in a container so use dnf
config-manager which should work on bare metal as well as in a
container.

Fixes: https://tracker.ceph.com/issues/50118

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-04-03 16:57:50 +10:00
Kefu Chai
e33e3a931d ceph.spec,install-deps: use golang-github-prometheus for promtools
instead of installing docker for using promtools, install
golang-github-prometheus.

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-22 14:13:17 +08:00
Kefu Chai
939b147a55 install-deps.sh: remove existing ceph-libboost of different version
we install different versions of precompiled ceph-libboost packages
for different branches when building and testing them on ubuntu test
nodes. for instance,

- nautilus: v1.72
- octopus, pacific: v1.73

they share the same set of test nodes. and these ceph-libboost packages
conflict with each other, because they install files to the same places.

in order to avoid the confliction, we should uninstall existing packages
before installing a different version of ceph-libboost packages.

ceph-libboost${version}-dev is a package providing the shared headers of
boost library, so, in this change we check if it is installed before
returning or removing the existing packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-20 13:06:08 +08:00
Kefu Chai
f381aa8bf0 test: run promtool test without docker on ubuntu/focal
before this change, we use docker for running promtools offered by
a docker image, but this is not efficient, and quite a few developers
do not want to use docker for running "make check". this change was
introduced by #39246, the reason was that, in Ceph's CI process, we
are using Ubuntu/Bionic for running "make check" jobs, but prometheus
packaged by Bionic does not offer the "test rules" command. so, to
address problem, we are using "dnanexus/promtool:2.9.2" docker image
for verifying monitoring/prometheus/alerts/test_alerts.yml.

after this change, we use prometheus packaged by debian derivatives
instead of pulling a docker image.

* debian/control: add prometheus as a "make check" dependency
* install-deps.sh: partially revert
  53a5816ded, as we don't need to
  pull docker or start docker service for using promtool anymore.
* cmake: check if promtool is capable of running "test rules"
  command, bail out if it is not.

see also: https://tracker.ceph.com/issues/49653

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Kefu Chai
1cc6174a3a install-deps.sh: install boost 1.75 on focal
we bump boost on regular basis. let's take the opportunity of moving to
focal to use boost v1.75.

v1.73 was used before this change. since both boost 1.75 and boost 1.73
install some files at the same places, we need to remove boost 1.73
before installing boost 1.75.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Kefu Chai
f75a36fe9a install-deps.sh: install libzbd on focal
WITH_ZBD is enabled for testing the build of zbd bluestore backend, and
we plan to migrate to Ubuntu/Focal for testing "make check", so need to
install libzbd when the distro version is focal.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-19 20:35:51 +08:00
Aashish Sharma
53a5816ded mgr/dashboard:test prometheus rules through promtool
This PR intends to add unit testing for prometheus rules using promtool. To run the tests run 'run-promtool-unittests.sh' file.

Fixes: https://tracker.ceph.com/issues/45415
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-03-08 10:16:22 +05:30
Sebastian Wagner
f171237378 install-deps: Cope with nonexistent /etc/apt/sources.list.d/ceph-libboost1.72.list
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-15 11:42:03 +01:00
Kefu Chai
af43300146 install-deps.sh: set codename when installing libzbd
this was an oversight in e7e3b86762

also use updated build with sha1 of
1fadde94b08fab574b17637c2bebd2b1e7f9127b, the older version has an issue
where both libzbd and libzbd-dev packaged .pc file with the same path,
so they conflicted with each other. the new version addressed this
issue.

and install libzbd-dev instead of libzbd and libzbd-dev, as the package
name of the former could change if its so version is bumped, so use the
*-dev package is a safer choice from the perspective of testing the
dev build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-26 18:45:09 +08:00
Kefu Chai
e7e3b86762 install-deps.sh: install libzbd-dev when WITH_ZBD is set
to avoid zbd backend bitrot, and for build test it in our "make check"
builds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-25 14:13:15 +08:00
Kefu Chai
cb471023d7 install-deps.sh: do not use deb.rug.nl as a mirror anymore
seems it does not mirror bionic packages for ubuntu-toolchain-r PPA repo anymore.
see http://deb.rug.nl/ppa/mirror/ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu/dists/

without this change, `sudo apt-get update` fails like:

Err:5 http://deb.rug.nl/ppa/mirror/ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu bionic Release
  404  Not Found [IP: 129.125.36.99 80]
...

which fails the build which runs install-deps.sh, like:

Build step 'Execute shell' marked build as failure

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-28 08:18:32 +08:00
Kefu Chai
594fc8c89c install-deps.sh: do not install nlohmann-json{3}?-dev
"mk-build-deps --install" should be able to take care of this.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-17 19:05:04 +08:00
Kefu Chai
99f6644e4f install-deps.sh: do not install libboost-all-dev for old distros
ubuntu disco and ubuntu focal do not ship libboost 1.72 and up, and
we depend on libboost 1.72 or up, so it does not help to install
liboost 1.67 or libboost 1.71 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-17 19:05:04 +08:00
Changcheng Liu
5e193e2bfe install-deps.sh: install debian package with proper right
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-12-17 11:13:35 +08:00
Deepika Upadhyay
29a932c292 install-deps.sh: add with_jaeger options for installing it's dependencies
* if WITH_JAEGER flag is specified, install-deps should mangle and update
  debian/control + ceph.spec the way we do for adding crimson dependencies
  with WITH_SEASTAR flag.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-16 18:17:01 +05:30
Josh Durgin
8a74a937dd
Merge pull request #31358 from ideepika/wip-jaegertracing-in-ceph
jaegertracing build/ops integration

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-15 16:29:26 -08:00
Deepika Upadhyay
4b69d3a184 jaeger, build/op: add jaeger and it's dependencies as submodules
* This commit introduces Jaegertracing library as package libjaeger,
  pickwhich would be consumed by other ceph pacakges such as ceph-common0

* adds the following dependencies, which would be build from source
  using ExternalProjectHelper.cmake +IncludeJaeger.cmake +
  Build<package>.cmake scripts:

  jaegertracing: v0.6.0 [added as a submodule]
  opentracing: v1.6.0 [added as a submodule]
  thrift: 0.13.0 [added as a submodule]
  yaml-cpp: 0.6.0
  json(optional)

* updates Boost to be installed instead of being build only, because
  jaegertracing them during their build process.

* ceph.spec.in: introduces a default enabled jaeger packaging option,
  which could be disabled using --without-jaeger flag during rpmbuild

* note: libjaeger package if enabled will be a dependency on ceph-common, ceph-mon, rgw_common and transitively will be a dependency for modules that have them as  a dependency.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-11 07:55:46 +00:00
Kefu Chai
28ba547c19 install-deps.sh: do not use gcc-toolset-9 from copr
since both CentOS and RHEL now include the updated
gcc-toolset-9-gcc-9.2.1-2.3.el8.

see also https://bugzilla.redhat.com/show_bug.cgi?id=1853900

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-10 18:50:45 +08:00
Brad Hubbard
7a68845579 install-deps.sh: Make powertools repo case insensitive
It looks like CentOS 8.3 will see all repos converted to lower case and
this has been pre-empted in the CentOS stream repos so we need to be
able to enable a repo called 'PowerTools' or 'powertools'

See https://git.centos.org/rpms/centos-repos/c/b759b17

Fixes: https://tracker.ceph.com/issues/48174

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-11-11 08:28:38 +10:00
Changcheng Liu
497b4d6f72 cmake: remove yasm since it's been replaced by nasm
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 12:36:20 +08:00
Changcheng Liu
a944232ad4 install-deps.sh: install nasm for isa-l library
nasm support build isa-l:AVX512 algorithm implementation while yasm
doens't support it. Install nasm assembler to build isa-l
refer to: https://github.com/yasm/yasm/issues/101

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-29 12:35:55 +08:00
Kefu Chai
6e673cad6d install-deps.sh: install libzbd-devel on RHEL/CentOS8 on demand
just like WITH_SEASTAR, WITH_ZBD is passed to install-deps.sh as an env
var, and it's disabled by default, as libzbd-devel is not yet packaged
in any distro supported by this script. we packaged and uploaded it
to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/ for build test
the libzbd bluestore backend. so "with_zbd" is always false on non-el8
distros. and should only be enabled for upstream "make check" test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-17 14:45:11 +08:00
Kefu Chai
c0802c98c4 alpine: remove stale Alpine build recipe
alpine build recipe is stale and does not work with the latest Ceph,
also the APKBUILD for Ceph can be found at alphine's aports repo, see
https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/ceph.

so instead keeping a stale version, let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-18 14:23:50 +08:00
Kefu Chai
10086983d9 install-deps.sh: add tchaikov's copr repo for crimson build on el8
we need it before the updated version of gcc-toolset9 addressing
https://bugzilla.redhat.com/show_bug.cgi?id=1853900 is available in
RHEL8/CentOS8.

see https://copr.fedorainfracloud.org/coprs/tchaikov/gcc-toolset-9

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-15 17:43:40 +08:00
Kefu Chai
1169dee259
Merge pull request #35787 from tchaikov/wip-46208
install-deps,cmake: use boost v1.73

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
2020-06-28 21:19:44 +08:00
Kefu Chai
a0c91f940b install-deps.sh: use ceph-libboost1.73
Fixes: https://tracker.ceph.com/issues/46208
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-26 17:20:05 +08:00
Kefu Chai
2d8da82b23 install-deps.sh: always use python3-sphinx to build the docs
python-sphinx creates a symlink pointing ../share/sphinx/scripts/python2/sphinx-build
to /usr/bin/sphinx-build even if python3-sphinx is already installed.
in that case, if python-sphinx is installed after python3-sphinx,
sphinx-build is in python2. and it breaks the build of master, as we are
using python3 syntax in conf.py since
e9e17b9cef.

and we are still using python2 as well as "python-sphinx" when building nautilus,
so if a build slave happen to compile nautilus before it is scheduled to
build master or a wip- branch, the doc build fails.

in this change, python-sphinx is uninstalled, if /usr/bin/sphinx-build
is found to be written in python2.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-25 19:59:41 +08:00
Kefu Chai
c70415b897 install-deps.sh: always use dnf on RHEL/CentOS
as we don't build master on el7 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-25 19:59:41 +08:00
Kefu Chai
95f6f61534 install-deps.sh: drop bits enabling DTS repo
since GCC offered by RHEL8/CentOS8 is able to compile ceph, there is no
need to add repos providing devtoolset anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-25 19:59:41 +08:00
Deepika Upadhyay
66e81b8b08 install-deps.sh: drop devtoolset(DTS) check for centos
we needed DTS for ensuring centos 7 had decent GCC version, since we won't be
supporting el7 anymore this check becomes outdated.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-06-23 16:35:37 +05:30
Kefu Chai
e42f308782
Merge pull request #35569 from sungjunyoung/master
install_deps.sh: Add debian based elementary os

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-06-20 21:27:06 +08:00
Kefu Chai
55b8707097 install-deps.sh: do not enable unavailable repos
should be specific when enabling codeready-builder repos, there is
chance that some repos are just not available, while the required one
is. for instance, "codeready-builder-for-rhel-8-x86_64-eus-debug-rpms"
might not be available. and in that case, `install-deps.sh` just fails.

so in this change, only the required one is enabled. see also
https://fedoraproject.org/wiki/EPEL

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-17 16:03:04 +08:00
Junyoung, Sung
3e7495d863 install_deps.sh: Add debian based elementary os
Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
2020-06-15 21:10:45 +09:00
Kefu Chai
030bce643f install-deps.sh: drop ktdreyer/ceph-el8 copr repo
as all python3-cherrypy and its deps are included by EPEL8 now

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-15 01:09:07 +08:00
Kefu Chai
e69ec07c71 install-deps.sh: remove the workaround to bootstrap a new venv
el7 now ships python36-virtualenv-15.1.0-4 which contains the fix of
https://github.com/pypa/virtualenv/issues/463

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-12 14:52:13 +08:00
Kefu Chai
1f8433f33c install-deps.sh: install additional repo for RHEL8
these home-brew repos are shared by CentOS8 and RHEL8

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2020-03-12 14:52:13 +08:00
Kefu Chai
25b47150f7 install-deps.sh: install rhel-7-server-optional-rpms on el7 only
RHEL8 uses appstream and supplementary repos for those optional-rpms.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-12 14:36:01 +08:00
Brad Hubbard
8ee9f47b9b install-deps.sh: Actually set gpgcheck to false
The current command appears to do nothing.

Fixes: https://tracker.ceph.com/issues/44344

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-02-28 12:20:18 +10:00
Kefu Chai
915fe2a7cb install-deps.sh: pass "-y" to "dnf copr enable"
otherwise the copr repo won't be enabled. and dnf will bail out with
following message:

Do you really want to enable copr.fedorainfracloud.org/ktdreyer/ceph-el8? [y/N]: Error: Safe and good
answer. Exiting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-26 15:07:53 +08:00
Kefu Chai
f91adafa4b install-deps.sh: use "apt" for checking install status
as "dpkg -s <package-name>" always returns 0 even if the <package> is
removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Kefu Chai
d560af7d5c install-deps.sh: install boost-1.72 for bionic
and uninstall boost-1.67 before installing boost-1.72

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-23 15:21:47 +08:00
Kefu Chai
a6f4234b04 install-deps.sh: drop trusty and xenial support
since we only build on bionic on ubuntu distros, there is no need
to build on trusty and xenial anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-23 00:02:17 +08:00
Kefu Chai
5c990ddd8e install-deps.sh: do not echo version of git
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:56 +08:00
Kefu Chai
e06b7bdd4f install-deps.sh: silence popd and pushd
otherwise they will echo the pushed/popped directory

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:56 +08:00
Kefu Chai
5d4c2c71bd install-deps.sh: revert a5a44cf4
we don't prepare wheels for python2 anymore, and we don't test for
python2. so no need to test for python2

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:56 +08:00
Kefu Chai
7fec5f7a67 install-deps.sh: stop preparing wheels for python2
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 11:52:56 +08:00
Kefu Chai
7f6e30d628 install-deps.sh: add ken's copr repo for el8 build
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-18 21:57:38 +08:00