Commit Graph

61 Commits

Author SHA1 Message Date
Sven Anderson fa8e6c803c CI: enable and collect core dumps in tests
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-11-16 16:26:46 +00:00
Sven Anderson 50668e7216 CI: disable VCS stamp for go builds
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
John Mulligan 0a036df85a entrypoint.sh: add support for using an alternate file system
With micro-osd.sh supporting a 2nd alternate fs we add basic support to
entrypoint.sh to help the tests find and use that alternate fs.
The command line option is flexible, you can supply it with a literal fs
name or, when prefixed with an `@`, it will read the name from a file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan 86802199f8 entrypoint: don't change the build tags variable
Don't change the build tags variable to include the -tags option at the
start. It's a bit confusing. Replace it with a function that returns
the option + the tags themselves for use on a go command line.
This fixes the remaining shellcheck errors in entrypoint.sh.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-07-19 06:55:28 +00:00
John Mulligan 8e8e24a81d entrypoint: treat pkgs as a bash array
Because the package name should be a proper "list" and is used in
multiple places, take shellcheck's warning to heart and replace it
with a proper bash array. Use a common function for generating said
array.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-07-19 06:55:28 +00:00
John Mulligan f2f2c5447b entrypoint: silence a shellcheck warning
For this one single case, it feels like overkill to use an array
or other fancy technique. Just silence shellcheck on this instance.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-07-19 06:55:28 +00:00
John Mulligan 24f02ecb2f entrypoint: quote various shell variables
Small and straightforward fixes for a bunch of items found by
shellcheck.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-07-19 06:55:28 +00:00
Anoop C S 9897bd034d entrypoint: Accept BUILD_TAGS from execution environment
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-07-18 13:19:55 +00:00
Sanford Miller 15d71fe192 rbd: add mirroring support for nautilus
The librbd API for mirroring-related operations changed substantially between
Nautilus and Octopus. Due to this, go-ceph had previously only implemented
mirroring functionality if built against Octopus client libraries. This patch
implements equivalent functionality for use with Nautilus clients.

Signed-off-by: Sanford Miller <smiller@digitalocean.com>
2022-07-13 13:00:48 +00:00
Anoop C S 6dad78e5af entrypoint: Store bootstrap token in variable
Following the mirroring setup there used to be a file named 'token'
holding the bootstrap token left out in sources. We could instead
provide the token via standard output and display it for any further
debugging purpose.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-07-10 15:06:12 +00:00
Sven Anderson 05b4e70014 cutil: enable PtrGuard by default
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-04-28 14:39:48 +00:00
Sven Anderson 4ef3865bb3 implements: analyze all but internal packages
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-04-12 17:53:06 +00:00
Sven Anderson 2ae01ab6bc CI: remove castxml from scripts and docs
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-02-21 14:45:42 +00:00
John Mulligan f27408f8bf contrib: add common/admin/manager to implements
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-02-08 15:55:02 +00:00
John Mulligan a46d9cbd62 entrypoint: ensure implements tool is up to date
When running things by hand it's nice to reuse the same build of
implements over and over, but for the casual contributor
who may need to run implements as part of adding a new API we want to
ensure the tool being run is up-to-date with the repo.
Cleaning the binary first is a quick and dirty way of getting there.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-11-09 02:02:08 +00:00
John Mulligan fc3ffc95e4 entrypoint: support executing only implements tool
The implements tool just does static code (ast) analysis. Support
running it in the container, but on its own in the case you only
want the implements tool output.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-11-09 02:02:08 +00:00
John Mulligan 16ff05194c entrypoint: scan the sources of cephfs/admin, rbd/admin, rgw/admin
Now that 'implements' has rudimentary support for these packages we can
gather information on them just like the C based ones.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 970b2eecda entrypoint: don't print results of 'command' when testing for castxml
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan b425523f44 entrypoint: handle adding build tags through a function
The previous code was somewhat dependent on always having the ceph
version build tag set first. Change the code to use a function so that
constructing the build tags is no longer order dependent and relocate
the actual tag stetting if-statements to below the various function
definitions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
John Mulligan ee90ae5906 entrypoint: default to running tests with 'ceph_preview' build tag
As this script is the heart of our ci and we generally want to test all
of go-ceph in our ci, we default to enabling ceph_preview build tag. If
we need to test only non-preview functions one can set "NO_PREVIEW" env
var to skip setting the build tag.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
Sven Anderson c8cf747e39 entrypoint: show return values of tests
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 19:01:04 +00:00
John Mulligan 58289e2547 entrypoint: add --mirror-state option
The --mirror-state option specifies a path to a file in which the script
will record the state of (rbd) mirroring. This can then later be reused
in order to skip the somewhat time consuming mirroring setup if the
statefile indicates mirroring is already functional.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-25 15:15:48 +00:00
Sven Anderson 1c719b199e tests: add benchmarks for PtrGuard related types
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-06-01 09:19:59 +00:00
John Mulligan 8ed302c7a0 entrypoint.sh: add a vim modeline to help set formatting parameters
For better or worse, unlike Go, shell does not have a standard formatting style.
Add a vim modeline that should match the basic current formatting style of the
scripts. This ought to help fellow vim users when editing the scripts.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-05-17 14:24:14 +00:00
Sven Anderson 93a4fe6d61 tests: make sure that all tests are run
Some tests were not run, because entrypoint.sh used a fixed list of package,
which is easily forgotten to update.  This change creates the list of
packages dynamically.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-05-03 15:07:32 +00:00
John Mulligan cf29cdbb27 entrypoint.sh: replace uses of rbd-fuse with rbd command line tool
Replace uses of the rbd-fuse virtual file system with command line
invocations of `rbd import` and `rbd export`. This allows us to drop
the dependency on rbd-fuse (and fuse entirely, again). Fuse requires
additional privs and special stuff so not using it makes running the
container simpler and more secure.

The only downside that I see to this change is that `rbd import` refuses
to overwrite existing images. This means the logic of the setup
function needed to be slightly changed but I don't consider it all
that different.

I did also slightly decrease the size of the image to ~500K. I didn't
think we needed to be copying so much zeros over the wire nor did
mirroring a lot of zeros demonstrate much IMO.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-23 13:07:21 +00:00
John Mulligan 0946b9e0e2 entrypoint.sh: export MIRROR_CONF for use by go test code
Later changes to the test Go code will make use of the MIRROR_CONF
variable to access the secondary ceph cluster.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-13 09:47:43 +00:00
Sven Anderson 0cc1dcb90d ci: add optional mirroring setup entrypoint.sh
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
Sven Anderson 362929f8d1 ci: add wait option to entrypoint script
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
John Mulligan 2d85e6f235 entrypoint: remove GO_CEPH_TEST_REQUIRE_MOUNT environment var
This env var is no longer used, we can drop it from the entrypoint
script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
Sven Anderson 5768911be7 ci: add a test run for Iovec based on PtrGuard
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
John Mulligan 1e4c2d09ae makefile: add new package to makefile and entrypoint.sh
Integrate the new 'cephfs/admin' package with existing makefile rules
and entrypoint.sh for testing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan 792f8cf02b entrypoint: get api implementation coverage when possible
When castxml is available, run the implements tool to generate
text and json implements output that can be later examined to
see what go-ceph does and does not cover in the ceph apis.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-05 11:35:03 -04:00
John Mulligan 7256dbb27b entrypoint: enable building and testing of internal/cutil package
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
Sven Anderson e01eb03284 retry: add a helper lib for retrying common operations
Our first operation is generating sizes for various caller-allocated
buffers.  The WithSizes function "suggests" sizes to an anonymous
function that takes a size as an argument and returns a hint how
sucessful the attempt was.  If required the same function is called
again with a larger size.  Errors and other results of that anonymous
function are "communicated" with variables of the surrounding scope
in wich the function is defined.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
John Mulligan a5e5ae6206 entrypoint: mention env vars that are not customized by script
Mention the env vars that the go test code looks for but does not change
from the "outer" environment as a way of documenting those variables.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan 900302b1ce entrypoint: set default env var for requiring cephfs mount point
When running cephfs tests locally there is no strong urge to require
an environment exactly like that of the test container. However, we do
want tests run under the container image (especially our default ci) to
use the cephfs mountpoint as they were previously. Set a default value
in the entrypoint script to require the mountpoint so we don't
accidentally miss the tests if something unexpected goes wrong.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan 5b76299c40 entrypoint: allow configuring ceph config source
This further improves the ability to use the script outside the
container itself and against alternate ceph cluster configs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-20 11:06:25 +01:00
John Mulligan 3afcc68ac7 entrypoint: enable go vet checking on packages
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-12 09:02:53 +01:00
John Mulligan 944510bd8c entrypoint: add a --results option for alternate results dir
This can be useful when running the script outside of the
container or doing other sneaky things. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-11 13:31:22 +01:00
John Mulligan 6b396f70b2 errutil: move errutil pkg into internal subdir
Moving the errutil pkg to "internal/errutil" makes errutil private-like
and accessible to only other go-ceph packages. The functions it provided
were always meant to be used only by go-ceph, this just makes it more
official. This is a breaking change but it was only available to
outside users for 1 release and it is somewhat doubtful users outside
of go-ceph would have reached for these functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-10 08:41:44 +01:00
John Mulligan b3f7c6af15 callbacks: create a helper type for managing callbacks between C and Go
As per the Go Wiki page on cgo [1] passing callback with data between C
and Go is non-trivial due to the pointer passing rules [2]. This change
creates a new helper type, based on the examples on the wiki, that uses
opaque indexes to exchange between C and Go.

For now, this type is simple as we're only going to have a few callbacks
initially but this could easily be improved and made more general in the
future as needed. Because we anticipate having to use this type in all
of our core packages (rados and cephfs as well as rbd) we are creating
the type in an "internal" package so that we don't treat the new type
as public despite the capitalized method names.

[1]: https://github.com/golang/go/wiki/cgo
[2]: https://golang.org/cmd/cgo/#hdr-Passing_pointers

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-04 08:10:48 -05:00
Niels de Vos 86a3a66849 tests: parameterize Ceph version for container build and build-tags
Pass `CEPH_VERSION=minic` (or 'luminous') on the `make test-docker`
command to select building a container with a different Ceph version and
running the tests in it.

This passes the given CEPH_VERSION on to the `docker build` command as a
--build-arg, which gets used for selecting the Ceph repository during
container build.

It also sets the environment variable CEPH_VERSION, which in turn is
consumed by `entrypoint.sh` to set the `go build -tags ${CEPH_VERSION}`
option that includes/excludes certain *.go files.

See-also: https://golang.org/pkg/go/build/#hdr-Build_Constraints
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-02-12 13:48:21 -05:00
John Mulligan 2f7493bed1 entrypoint: make var references consistently use curly braces
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00
John Mulligan ee2c26906e entrypoint: support disabling code coverage
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00
John Mulligan b7ea1de2e2 entrypoint: add options to run cpu and mem profiling on tests
Add options --cpuprofile and --memprofile which enables the respective
profile options for the go test command.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00
John Mulligan 7d12f0bbcd entrypoint: highlight important commands in script output
Add a small wrapper function 'show' that prints a prefix and the
command that will be run. Helps the reader jump to important parts of
the script output.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00
John Mulligan 7790594ee4 entrypoint: fix pausing after test failure
The --pause option for the entrypoint script is useful for interactive
debugging after a test failure, but the script was not pausing if the
test suite failed. Add code to explicitly handle test failures including
pausing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00
John Mulligan 691cd8d99b entrypoint: move pre- and post- test code into functions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00
John Mulligan 3ca24f63c5 entrypoint: move go test setup an invocation into a function
A little refactoring to move the setup of the pkg test calls into a
function to ease future changes.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 09:22:03 +01:00