Work around errors related to protobuf package locations on centos.
This is triggered by our squid jobs failing with errors like:
```
22.40 Problem: protobuf-3.14.0-13.el9.i686 from appstream does not
belong to a distupgrade repository
22.40 - package protobuf-compiler-3.14.0-13.el9.x86_64 from @System
requires protobuf = 3.14.0-13.el9, but none of the providers can be
installed
22.40 - cannot install both protobuf-3.14.0-14.el9.x86_64 from
appstream and protobuf-3.14.0-13.el9.x86_64 from @System
22.40 - cannot install both protobuf-3.14.0-14.el9.x86_64 from
appstream and protobuf-3.14.0-13.el9.x86_64 from appstream
22.40 - cannot install the best update candidate for package
protobuf-3.14.0-13.el9.x86_64
22.40 - problem with installed package
protobuf-compiler-3.14.0-13.el9.x86_64
22.40 (try to add '--allowerasing' to command line to replace
conflicting packages or '--skip-broken' to skip uninstallable packages
or '--nobest' to use not only best candidate packages)
------
```
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add libradosstriper devel and debug packages to the test container
image. This is in preparation for adding a future `rados/striper`
package to go-ceph that will wrap these APIs.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Reorganize the RUN command in the Dockerfile that sets up distro based
repositories and ceph packages. Move the `&&` to the beginning of the
lines to highlight when commands begin (and end). Change the indent
level of items that are arguments to commands to further clarify
what is part of what.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The centos 8 stream distribution is now EOL and the yum repo mirrors are
now offline. However, our CI is designed to test older released versions
of ceph where the container images are still using centos 8 stream.
Borrow and adapt a line from the ceph-csi project in order to use the
centos 8 archives (vault) until either we stop testing these versions of
ceph or the ceph project changes the base distro for these release
versions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
With CentOS Stream 8 reaching end of builds phase most of our ceph
images are migrating their bases to CentOS Stream 9. Since they come
with curl by default in a minimal rpm package we switch to attempt the
install by directly linking the binary than the package name. This is
due to the conflict between curl and curl-minimal packages in CentOS
Stream 9 images.
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
This bumps the Go version of the test container to 1.21.8, which is the
oldest supported Go version as of now.
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
Currently the tests on the released reef image fails due to an RPM
transaction error when updating nfs-ganesha. Since go-ceph doesn't
need ganesha packages to be up-to-date to ensure that
lib{cephfs,rados,rbd}-devel are installed we _temporarily_ disable
ganesha repo when running yum update.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Previously, commit 102616b158 tried to
expose GOPROXY env var from developer environment by specifying it as
environment variable or build argument to container build process. But
using --build-arg to container build command would require the same to
be referenced inside Dockerfile/Containerfile so that the following
warning can be avoided:
[Warning] one or more build args were not consumed: [GOPROXY]
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
This bumps the Go version of the test container to 1.19.5, which is
the oldest supported Go version as of now.
The go.mod file (including an upgrade of its dependencies) and the
revive environment is bumped to 1.19 as well.
Signed-off-by: Sven Anderson <sven@redhat.com>
Use the GOARCH variable to define to proper version of go binaries to
download. Without this, the end result image can not run properly on
local host machine.
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Ceph version is already available from CEPH_VERSION env variable.
CEPH_TAG on the other hand is only defined to determine the tag for
ceph base image and seems to be unused in the resulting go-ceph
container image.
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
The Go 1.18 release was done in march 2022 and so we should
continue to use the older of the two supported releases which
is now Go 1.17.x.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This issue should have been fixed in ceph-containers already but for
whatever reason we don't seem to be getting the fix.
For now, disable the apache-arrow-centos repo.
Based on: 8b4c8dc730
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Return to trying to install castxml into the container image when
possible. This now appears to work (and function) on the centos 8 based
images.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
With the recent release of go1.17 the oldest supported go version is
1.16. Update to the newest 1.16.z and clean up the structure of
the go install process a bit.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
First off, ceph is no longer using docker hub for the canonical image
hosting. We have also been recommended to use ceph/ceph instead of
ceph/daemon-base. Thus we switch to 'quay.io/ceph/ceph'.
However, ceph/ceph doesn't use codenames for tags so we also have to use
version number tags. These are set as build args now. To retain
compatibility with the way we were working previously we have add a
best-effort "assertion" to ensure the code name we provide matches with
the codename embedded in the base image.
While working on this I discovered that our previous build args were
probably not working as intended, but only by accident. See:
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
This is why we now name our build arg GO_CEPH_VERSION and it is located
after the 'FROM' line.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
If the make variable USE_CACHE is set, the test container will mount
the /go directory from a named volume that is reused in following
test runs, so that go dependencies don't have to be installed again.
Signed-off-by: Sven Anderson <sven@redhat.com>
Sometimes the version in the RPM repository don't match the version
of the base image. This change updates the packages before installing
the libraries in order to make them match.
Signed-off-by: Sven Anderson <sven@redhat.com>
We're adding a fair amount of octopus (and later) only features now, and
at the time of this writing, pacific is already out. Stop defaulting to
nautilus and default to octopus instead.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The current version we are testing with is no longer supported.
This updates Go to a supported released version.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Bump the golang version to 1.14.7. Go 1.14 is now the older supported
version, with the recent release of Go.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The castxml install from centos8+epel has been rather fraught with
errors including broken/failed installs and, at the time of this commit,
it installs but can not run due to llvm integration/cli issues.
On top of this it was not available on centos7 anyway and so would,
in theory, only have worked on the octopus container image.
At this point it seem that running the implements tool is not worth
all the failures that trying to install castxml causes.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Now that ceph/daemon-base:latest-octopus is on v15.2.4 we can
stop the need to update already installed packages in the container.
To make things "safer" and fail early with errors rather than mysterious
compatibility problems, I am adding explicit versions to the
lib{cephfs,rados,rbd}-devel-* packages. Now if the container can't find
these packages it will fail rather than partially upgrade the ceph
packages.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
On octopus taking in the -devel packages causes *some* of the packages
to be updated but not all. This apparently makes ceph-fuse fail to
start correctly by blocking forever. This works around that issue.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Currently, this tool is only readily available on centos8 based
containers (octopus). But as it is being requested by binary name
it will be taken in if any repo used by the container makes it
available.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Ceph containers [1] are built by the ceph team and for our purposes,
only lack build tools and Go making the delta we need to do smaller.
1: https://github.com/ceph/ceph-container
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan
Anoop C S
Sven Anderson
Shachar Sharon