some build dependencies are still missing in PowerTools and EPEL8, so we
built and pushed them to sepia so it can be used before they are ready
in these repos.
Signed-off-by: Kefu Chai <kchai@redhat.com>
python3-devel depends on python-rpm-macros, which in turn depends on
python-srpm-macros. the latter offers `python3_pkgversion` macro, which
is necessary for prepare the build dependencies of rpm packages.
this change avoids the unnecessary `$yumdnf` calls, and to simplify the
code.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the reason why we need to install these macros is to solve the
egg-chicken problem -- to set `_python_buildid` and `python3_pkgversion`
so that we can prepare the build dependencies and install them. in which,
`_python_buildid` is defined using `python3_pkgversion`. this macro is
offered by python-srpm-macros.
the other macros, like `python3_sitelib` and `__python3` are offered by
`python3-rpm-macros`.
this change also avoid the issue if we install `*rpm-macros` on CentOS8:
Error:
Problem: package R-rpm-macros-1.1.0-2.el8.noarch requires /usr/bin/Rscript, but none of the providers can be installed
- package R-rpm-macros-1.1.0-2.el8.noarch requires R-core, but none of the providers can be installed
- conflicting requests
- nothing provides libRblas.so()(64bit) needed by R-core-3.6.1-1.el8.x86_64
- nothing provides openblas-Rblas needed by R-core-3.6.1-1.el8.x86_64
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
Signed-off-by: Kefu Chai <kchai@redhat.com>
there is chance that the system already have epel-release-latest-7.noarch.rpm
installed, in that case, install-deps.sh should not fail.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in 087ea813, we installed '*rpm-macros' for the macros, so we can have
access to the latest python packaging related macros for preparing the
build dependencies.
but we could run into https://bugs.centos.org/view.php?id=16379, if
we already have an old version of python-devel installed. as the newer
version of python-rpm-macros conflicts with it.
it was a chicken-and-egg problem, as we don't know the exact name of
*rpm-macros packages. that's why we chose to install all of them. but
we have to upgrade the existing python-devel package to resolve the
conflict. but the since there is no python3-devel in RHEL7/CentOS7,
what they have is python36-devel. so we have to hardwire the
`%{python3_pkgversion}` to "36" even before we have access to this
macro, and upgrade the python36-devel package beforehand. but this
renders installing the rpm-macro package less useful -- we intend to
use the macro offered by the package to figure out "36".
as a workaround, we pretend that we know the "main" version of python3
in current RHEL/CentOS. and always install python36-devel for
python-rpm-macros. as the former requires the latter.
once all python3*-devel on all builders are upgraded, we will be safe
to install '*rpm-macros' again without installing python36-devel first.
by then, we could revert this change, or continue installing
python36-devel until the distro bumps up the "main" python version to 3.7
Fixes: https://tracker.ceph.com/issues/41603
Signed-off-by: Kefu Chai <kchai@redhat.com>
* move `for_make_check` to the beginning of script, as FreeBSD will also
use this variable
* extract `preload_wheels_for_tox()` function out to improve readability
* call `preload_wheels_for_tox()` only if `for_make_check` is true
Signed-off-by: Kefu Chai <kchai@redhat.com>
otherwise we will fail to install the build dependencies of
`lazy-object-proxy` from the wheelhouse. as `lazy-object-proxy` does not
add `setuptools_scm` in its `setup.py`, instead it lists
`setuptools_scm` in `setup.cfg` and `pyproject.toml` as a `build-system`
requires. but unfortunately, `pip download` only downloads the
install/run-time dependencies at this moment. and `lazy-object-proxy`
does not offer binary package for at least python2.7.
ideally, `pip download` should collects its dependencies like
Collecting setuptools_scm>=3.3.1 (from lazy-object-proxy->astroid<3,>=2.2.0->pylint->-r requirements-lint.txt (line 1))
so we need to use `pip wheel` do download build-time dependencies
see also https://github.com/pypa/pip/issues/6222
Signed-off-by: Kefu Chai <kchai@redhat.com>
these dependencies are only used for building python-saml which is in
turn used for the SAML support. this feature is tested using
`test_sso.py` while performing dashboard tests. we do not package or
ship python-saml along with other Ceph packages. so let's move these
dependencies to the "make check" sections in ceph.spec.in and
debian/control for simplifying install-deps.sh.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Refactor CMake add_tox_test to automatically add py27 and/or py3 to
provided toxenvs.
Refactor tox.ini:
- Remove requirements-{py27,py3}.txt, as python release dependant
packages can be handled with PEP 508 syntax.
- Remove develepment dependencies from requirements.
- Move pycodestyle settings to separate section.
- Add flake8 check and other checkers (rst, naming, etc). Some of them
are commented out for future clean-ups (Ceph trackers have been opened)
- Pycodestyle removed, as flake8 is a wrapper for pycodestyle.
- Add instafail plugin to report failures immediately
- Add timeout plugin to limit max run time (sometimes test_tasks hangs)
- Remove unused dependencies (lru_cache, pluggy)
Test and code linting fixes:
- Unused imports
- Fixes to HACKING.rst
Doc:
- Update HACKING.rst
Add conftest.py to mock imported modules (rados, rbd, cephfs), and mock
also rados Error and OSError Exceptions.
Fixes: https://tracker.ceph.com/issues/40487
Fixes: https://tracker.ceph.com/issues/41152
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
two reasons:
- do not rely on kitware repo so we have better control of the
cmake3: the `debian` directory is tracked by
https://github.com/tchaikov/ceph-cmake
- chacra repo also offers aarch64 build of cmake3
Signed-off-by: Kefu Chai <kchai@redhat.com>
to enable us to build on xenial, install newer cmake.
cmake 3.10.2 is the version offered by bionic.
with this change, we can safely require cmake 3.10.2 in our cmake
script. as EPEL7 offers cmake 3.13
Signed-off-by: Kefu Chai <kchai@redhat.com>
two reasons
* GCC-9 features more checks. so let's use it!
* crimson targets the hardware + toolchain + kernel after 1+ years,
so it would be great if we can compile and test crimson using
the toolchain which is ubiquitous on most mainstream distos.
Signed-off-by: Kefu Chai <kchai@redhat.com>
because `install-deps.sh` is executed using `source`, we have to pass
these options using env variables. but before this change, `WITH_SEASTAR` is used directly,
while `FOR_MAKE_CHECK` is checked and translated to a local variable
`for_make_check`. which, in my opinion, has better readability.
so, in this change, `WITH_SEASTAR` is translated to `with_seastar`
variable in `install-deps.sh`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
The FreeBSD dependency install specifies devel/kbr5 instead of devel/krb5
Fixes: https://tracker.ceph.com/issues/40050
Signed-off-by: Thomas Johnson <NTmatter@gmail.com>
Prebuilt boost-1.67 packages are only available for limited arch on Ubuntu
(x86_64 for bionic, x86_64 and arm64 for xenial). This will cause install
failure on missing platforms.
Add a new env variable to allow user to build and install from source
while the default option is to install pkgs from ceph repo.
Change-Id: I4259733dd40a638d1bd5d1578a64ecaaa6490121
Signed-off-by: Jun He <jun.he@arm.com>
so `yum-builddep` can have access to the latest macros for preparing the
build dependencies
Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>
seems devtools-8 is not packaged on aarch64. and we are suffering from
ICE on centos7 when building with GCC-7.3. so let's try GCC-8.2!
Fixes: http://tracker.ceph.com/issues/38892
Signed-off-by: Kefu Chai <kchai@redhat.com>
There is a few essentail differences between the way versioning
and a version-map is interpreted for binutils ld and LLVM ld.
The easy way out for the time being is to use the one that works.
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
this change reverts "install-deps.sh: workaround setuptools' dependency on six"
This reverts commit 30ce5e5546.
tl;dr: this change addresses the failures of "make check" runs on arm64
builders when they try to build `mgr-dashboard-test-venv` target.
long story: without this change, we will fail to pull in
setuptools >= 36, and as a result pip will fail to import
`setuptools.build_meta` in `pip/_vendor/pep517/_in_process.py`. and will
a `BackendUnavailable` exception thrown by `_call_hook()` in
`pip/_vendor/pep517/wrappers.py`. since the issue addressed by 30ce5e55
has been addressed since setuptools >= 36.0.1, we should be safe to
upgrade to the latest setuptools now.
Signed-off-by: Kefu Chai <kchai@redhat.com>
as centos-sclo-rh-source leads us to 404 at this moment. and we are not
using the source repo for building ceph. so we can just skip any
unavailable repo.
Fixes: http://tracker.ceph.com/issues/37707
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
