When running things by hand it's nice to reuse the same build of
implements over and over, but for the casual contributor
who may need to run implements as part of adding a new API we want to
ensure the tool being run is up-to-date with the repo.
Cleaning the binary first is a quick and dirty way of getting there.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The implements tool just does static code (ast) analysis. Support
running it in the container, but on its own in the case you only
want the implements tool output.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Now that 'implements' has rudimentary support for these packages we can
gather information on them just like the C based ones.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The previous code was somewhat dependent on always having the ceph
version build tag set first. Change the code to use a function so that
constructing the build tags is no longer order dependent and relocate
the actual tag stetting if-statements to below the various function
definitions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
As this script is the heart of our ci and we generally want to test all
of go-ceph in our ci, we default to enabling ceph_preview build tag. If
we need to test only non-preview functions one can set "NO_PREVIEW" env
var to skip setting the build tag.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The --mirror-state option specifies a path to a file in which the script
will record the state of (rbd) mirroring. This can then later be reused
in order to skip the somewhat time consuming mirroring setup if the
statefile indicates mirroring is already functional.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
For better or worse, unlike Go, shell does not have a standard formatting style.
Add a vim modeline that should match the basic current formatting style of the
scripts. This ought to help fellow vim users when editing the scripts.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Some tests were not run, because entrypoint.sh used a fixed list of package,
which is easily forgotten to update. This change creates the list of
packages dynamically.
Signed-off-by: Sven Anderson <sven@redhat.com>
Replace uses of the rbd-fuse virtual file system with command line
invocations of `rbd import` and `rbd export`. This allows us to drop
the dependency on rbd-fuse (and fuse entirely, again). Fuse requires
additional privs and special stuff so not using it makes running the
container simpler and more secure.
The only downside that I see to this change is that `rbd import` refuses
to overwrite existing images. This means the logic of the setup
function needed to be slightly changed but I don't consider it all
that different.
I did also slightly decrease the size of the image to ~500K. I didn't
think we needed to be copying so much zeros over the wire nor did
mirroring a lot of zeros demonstrate much IMO.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Later changes to the test Go code will make use of the MIRROR_CONF
variable to access the secondary ceph cluster.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Integrate the new 'cephfs/admin' package with existing makefile rules
and entrypoint.sh for testing.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
When castxml is available, run the implements tool to generate
text and json implements output that can be later examined to
see what go-ceph does and does not cover in the ceph apis.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Our first operation is generating sizes for various caller-allocated
buffers. The WithSizes function "suggests" sizes to an anonymous
function that takes a size as an argument and returns a hint how
sucessful the attempt was. If required the same function is called
again with a larger size. Errors and other results of that anonymous
function are "communicated" with variables of the surrounding scope
in wich the function is defined.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
Mention the env vars that the go test code looks for but does not change
from the "outer" environment as a way of documenting those variables.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
When running cephfs tests locally there is no strong urge to require
an environment exactly like that of the test container. However, we do
want tests run under the container image (especially our default ci) to
use the cephfs mountpoint as they were previously. Set a default value
in the entrypoint script to require the mountpoint so we don't
accidentally miss the tests if something unexpected goes wrong.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This further improves the ability to use the script outside the
container itself and against alternate ceph cluster configs.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This can be useful when running the script outside of the
container or doing other sneaky things. :-)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Moving the errutil pkg to "internal/errutil" makes errutil private-like
and accessible to only other go-ceph packages. The functions it provided
were always meant to be used only by go-ceph, this just makes it more
official. This is a breaking change but it was only available to
outside users for 1 release and it is somewhat doubtful users outside
of go-ceph would have reached for these functions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
As per the Go Wiki page on cgo [1] passing callback with data between C
and Go is non-trivial due to the pointer passing rules [2]. This change
creates a new helper type, based on the examples on the wiki, that uses
opaque indexes to exchange between C and Go.
For now, this type is simple as we're only going to have a few callbacks
initially but this could easily be improved and made more general in the
future as needed. Because we anticipate having to use this type in all
of our core packages (rados and cephfs as well as rbd) we are creating
the type in an "internal" package so that we don't treat the new type
as public despite the capitalized method names.
[1]: https://github.com/golang/go/wiki/cgo
[2]: https://golang.org/cmd/cgo/#hdr-Passing_pointers
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Pass `CEPH_VERSION=minic` (or 'luminous') on the `make test-docker`
command to select building a container with a different Ceph version and
running the tests in it.
This passes the given CEPH_VERSION on to the `docker build` command as a
--build-arg, which gets used for selecting the Ceph repository during
container build.
It also sets the environment variable CEPH_VERSION, which in turn is
consumed by `entrypoint.sh` to set the `go build -tags ${CEPH_VERSION}`
option that includes/excludes certain *.go files.
See-also: https://golang.org/pkg/go/build/#hdr-Build_Constraints
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add options --cpuprofile and --memprofile which enables the respective
profile options for the go test command.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a small wrapper function 'show' that prints a prefix and the
command that will be run. Helps the reader jump to important parts of
the script output.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The --pause option for the entrypoint script is useful for interactive
debugging after a test failure, but the script was not pausing if the
test suite failed. Add code to explicitly handle test failures including
pausing.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
A little refactoring to move the setup of the pkg test calls into a
function to ease future changes.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
As preparation for upcoming changes, move the main code of entrypoint.sh
into functions. Best viewed by ignore whitespace.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
When building the container image for running tests take the older of
the two current releases from Go upstream. We use the older version
in order to make sure we are testing with something supported but maybe
not everyone is on 1.13 yet.
This also enables the use of go modules to ensure we get the proper
versions of dependencies.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Run the tests for each sub-package as a separate invocation.
Make the package selectable.
This has a few small advantages:
* When iterating on a single package the other packages do not run
* Better coverage summary line printed after each test
* Slightly better output with verbose test output
Signed-off-by: John Mulligan <jmulligan@redhat.com>
According to the 'go test' help, "The idiomatic way to disable test
caching explicitly is to use -count=1." This removes GOCACHE=off in
favor of using -count=1 as GOCACHE=off breaks other tooling including
the use of go modules.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Adds cli parsing to the entrypoint script allowing for:
1. Custom path to micro-osd script
This allows easier testing of the script itself or quick
and dirty customization for special one-off tests
2. Selecting tests to run (or "ALL" or "NONE")
This allows the user to run only a subset of the tests as needed.
3. Option to pause indefinitely after tests complete
This keeps the containerized environment alive after tests
have run for debugging or just for creating a quick and dirty
ceph environment for playing around.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
When tests are run generate a coverage report.
To access this coverage report mount any directory to /results in the
container. Coverage output files will be written to /results/coverage
w/in the container.
Later we can do other interesting things with the coverage detected
such as a test pass/fail criteria or submitted to a service such as
coveralls, etc.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Sven Anderson
John Mulligan
Niels de Vos
Lincoln Thurlow
Noah Watkins