Commit Graph

273 Commits

Author SHA1 Message Date
John Mulligan 72ea0669c2 cephfs: use common error extraction function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
John Mulligan 887dec1c50 errutil: common error handling functions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
Niels de Vos f15486e0a2 test/rbd: check for expected errors with Image.Clone()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos f611d4a986 test/rbd: check for expected errors with Image.Open()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos 22c15e5a37 rbd: fix problems with Image.Discard()
The return value of rbd_discard() is the number of bytes that have been
discarded, or in case of an error, a negative errno value.

It seems Image.Discard() never can have worked as intended, unless the
error was disregarded. Just like Image.Write(), Image.Discard() should
return a tuple of the number of bytes discarded and an error.

Fixes: ceph/go-ceph#123
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos 9381485fac test/rbd: Add tests for Image.Seek()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos 2faaf93287 test/rbd: add test for opening an Image read-only
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos 30fdf64bdc test/rbd: add test for Image.Resize()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos a5cc9ff302 test/rbd: add tests for getting image properties
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos dbfc54d61e rbd: RbdFeature* constants should be uint64
rbd_get_features() returns a uint64_t with the features that are set in
the image. That means combining (with logical OR) features should result
in a uint64 too. Marking all features explicitly uint64 makes it easier
and cleaner to set/test features.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos 0b8f4eb7e5 test/rbd: add test for Image.Rename()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
Niels de Vos 65d1e9c763 rbd: include rbd_test.go in rbd package
Keeping the _test.go code in the same package allows for the testing
of private functions. AFAWK it is more conventional as well.

See-also: ceph/go-ceph#135
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-16 10:12:51 -05:00
John Mulligan 7c1e4f2ca9 rbd: convert test to use errors with conventional names
We recently created errors in the rbd that match the golang convention
but left aliases for external code. Switch the test code to use the
newer more standard form instead of the "aliases."

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 11:32:21 +01:00
John Mulligan 36fc3510d8 rbd: use conventional naming style for errors
A recent PR added new error values that were similar to the existing
naming convention but were not the normal go convention. Change all the
errors to match typical convention but leave the older names as aliases
such that older code will continue to work (for now).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 11:32:21 +01:00
John Mulligan a2d1c64180 rbd: separate standard lib imports from pkg imports
By convention many projects are separating the golang stdlib imports
from other imports. Do that as well for rbd.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 10:03:58 +01:00
John Mulligan 8a2d18be87 rados: include rados_test.go in rados package
Keeping the _test.go code in the same package allows for the testing
of private functions. AFAIK it is more conventional as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 09:49:47 +01:00
Yue Zhu 750fe1830e rados: add support for sending PG commands 2019-12-13 16:30:00 +01:00
John Mulligan ce5d3ef04d cephfs: remove use of logging in package
After discussion we decided to entirely drop logging from the library.
Future users should rely instead of the error types and values the
function calls return.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-13 15:17:27 +01:00
John Mulligan e07f6a6050 rados: replace Println calls with a proper error
Replace the odd calls to Println with a comparable error value.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-13 15:17:27 +01:00
John Mulligan fdc33a8e69 github: configure a pull request template
Template provides brief reminders on what the project expects on
all prs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 16:17:31 -05:00
Niels de Vos a713db983a rbd: return more useful error messages
Translate the return value (errno) from the C rbd_*() functions into the
standardized strerror() error message.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-12 13:56:53 -05:00
John Mulligan dcfe969b09 rbd: use correct style for package doc comment
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:34:51 -05:00
John Mulligan 5d0b9b17a0 rados: use correct style for package doc comment
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:34:51 -05:00
John Mulligan b3deb28b94 makefile: use revive for style checking
Switch use of style checker from golint to revive for both the makefile
and travis configuration.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:22:11 -05:00
John Mulligan 75fdc694be testing: add a project-wide toml config file for revive
Revive is a tool similar to golint but can be configured. Add an initial
configuration for revive.
This configuration reduces the initial issue list to something more
manageable and that can be worked on incrementally rather than trying to
boil to ocean to make the checker happy.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:22:11 -05:00
John Mulligan 5ea85a9419
Merge pull request #89 from CodeLingoBot/rewrite
Fix function comments based on best practices from Effective Go
2019-12-11 19:56:20 -05:00
Niels de Vos 2d52744470 rbd: improve error checking
Make it possible to easily validate the *Image and *Snapshot objects
before using them. This makes the use of go-ceph for RBD functions more
stable, and it is less likely that incorrect use causes Go panics.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 19:47:21 -05:00
John Mulligan 99de05854c ceph: remove unnecessary imports in doc.go
The package imports in doc.go were not accomplishing anything and also
were not complete (missing cephfs). This removes them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
John Mulligan ab02cd27c0 ceph: expand doc to go a bit
Make the doc comment in doc.go a bit more fleshed out with pointers
to the real functionality of the library.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
John Mulligan 9460b11d88 ceph: rename package for go files in root dir
Oddly, probably a historical artifact, the .go files in the root of the
go-ceph tree were named "rados". As rados is one of our sub-packages it
seemed more correct to call the package "ceph".  This was chosen as
"go-ceph" is not a valid identifier in go and since this is already go
code prefixing with "go" seems a bit redundant.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
Niels de Vos b86563db7e rbd: add Create4() for more advanced RBD Image creation
This allows to set additional options while creating RBD images.
Ceph-CSI will initially consume this to configure an optionally
different pool for data.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos c06593eef7 rbd: add support for RbdImageOptions
There are several RbdImageOptions that can be used to configure features
of an RBD image while creating, cloning, migrating and copying. A
follow-up patch will add Create4() where the RbdImageOptions can be
used.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos 1a06d47a0d tests: use Ceph Nautilus
Prepare for adding support for RbdImageOptions with
RBD_IMAGE_OPTION_FLATTEN that has been introduced in Ceph Mimic.
Nautilus is the most recent supported versions, so use that for testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos 6a17d59988 rbd: split Create() off into Create2() ad Create3() following librbd.h
The C library offers rbd_create2() and rbd_create3() and does not try to
cover all options with rbd_create(). For users that are familiar with
the C API, the Create2() and Create3() functions have been added.

The existing Create() API still handles the complete set of
rbd_create*() functions, so backwards compatibility is covered.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 09:09:54 -05:00
zhangyue de3bd20fe3 bugfix: use GetError instead of RDBError to return
Signed-off-by: zhangyue <zy675793960@yeah.net>
2019-12-11 08:44:00 -05:00
John Mulligan fa319f99bb rbd: use uuid library, rather than shelling out, in rbd test
Previously, the tests were shelling out to run 'uuidgen' which
is not particularly efficient and a bit ugly. Switch this test
to use the same uuid library already in use by the rados test code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-06 11:23:03 -05:00
Sven Anderson 472f6dd5bb Tests: avoid segmentation faults in tests
This change replaces a couple of functions from the "assert" package
with the corresponding functions from the "require", where the
following code relies on a non-nil object.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-06 11:12:34 -05:00
John Mulligan 20d12ba7bc test: disable test results caching with -count flag
According to the 'go test' help, "The idiomatic way to disable test
caching explicitly is to use -count=1." This removes GOCACHE=off in
favor of using -count=1 as GOCACHE=off breaks other tooling including
the use of go modules.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
John Mulligan 5b6bad390a test: make entrypoint script behavior configurable
Adds cli parsing to the entrypoint script allowing for:
1. Custom path to micro-osd script
   This allows easier testing of the script itself or quick
   and dirty customization for special one-off tests
2. Selecting tests to run (or "ALL" or "NONE")
   This allows the user to run only a subset of the tests as needed.
3. Option to pause indefinitely after tests complete
   This keeps the containerized environment alive after tests
   have run for debugging or just for creating a quick and dirty
   ceph environment for playing around.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
John Mulligan ac1d6a6554 dockerfile: use exec form for entrypoint script
Using the exec form will allow us to pass arguments to the
container/entrypoint in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
Sven Anderson 1bc4d888b8 Makefile: Fix CONTAINER_OPTS for Fedora
Fedora based distributions don't use AppArmor but SELinux instead.
This change detects the distribution and sets the --security-opts
accordingly.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-05 13:32:55 -05:00
Niels de Vos 09b6977dc9 rados: free cluster runtime resources automatically
Release resources that are allocated while configuring the connection to
the cluster. rados_shutdown() should only be needed after a successful
call to rados_connect(), however if the connection has been configured
with non-default parameters, some of the parameters may be allocated
before connecting. rados_shutdown() will free the allocated resources,
even if there has not been a connection yet.

Note that the finalizers get executed during garbage collection, which
can be forced by calling runtime.GC() for testing.

Fixes: #109
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-26 09:59:19 -05:00
Niels de Vos cb575930e6 make: add support for running test-docker on SElinux enabled systems
When running 'make test-docker' on Fedora with SElinux in Enforcing
mode, fetching dependencies for the project fails. It seems the volume
is not accessible to the container. Enabling SElinux for the volume (by
passing the :z flag) makes it work.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-04 13:18:19 -05:00
Niels de Vos 72c262c493 Docker: prevent blocking prompt with add-apt-repository
A blocking message is shown when running 'make test-docker':

    STEP 6: RUN add-apt-repository ppa:gophers/archive
     Co-installable packages of Go 1.4 through 1.11 for supported versions
    of Ubuntu -- currently Trusty, Xenial, Bionic, Cosmic and Disco.

    Installing golang-1.X installs the latest point release of Go 1.X. I add
    the odd patch to keep older versions of Go building with new toolchains,
    but I do *not* make any effort to backport security fixes to Go releases
    that are not supported by upstream. Buyer beware!

    I aim to start uploading a release when its first beta is released, so
    e.g. I uploaded a golang-1.7  package after 1.7 beta 1 was released. But
    I don't always keep up with this.

    So that these packages can be co-installable, they do not install
    anything to /usr/bin. Once you have installed the golang-1.X package,
    you will need to add /usr/lib/go-1.X/bin (or maybe /usr/lib/go-tip/bin)
    to your $PATH, or you can just invoke /usr/lib/go-1.X/bin/go directly.

    The packages all Recommend: appropriate versions of the runtime race
    detector support on amd64, which are also built in this PPA
    (eventually).
     More info: https://launchpad.net/~gophers/+archive/ubuntu/archive
    Press [ENTER] to continue or ctrl-c to cancel adding it

By passing the -y switch to add-apt-repository, this message is ignored
and the process continues.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-04 13:18:19 -05:00
John Mulligan 93d8176a10 travis: add ceph nautilus to the test matrix
Add ceph nautilus to the test matrix as well as simplifying the setup
script in the travis yaml file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 13:15:30 -05:00
John Mulligan a9a7137d59 test: update micro-osd.sh to support nautilus
Apparently 'rados mkpool' and 'rados rmpool' no longer function when
using nautilus.
Switch to ceph osd pool subcommands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 13:15:30 -05:00
John Mulligan fb17c78c11 tests: generate html coverage report
When tests are run generate a coverage report.

To access this coverage report mount any directory to /results in the
container. Coverage output files will be written to /results/coverage
w/in the container.

Later we can do other interesting things with the coverage detected
such as a test pass/fail criteria or submitted to a service such as
coveralls, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 11:04:15 -05:00
John Mulligan 971027692f makefile: rebuild test container as needed
If the Dockerfile or entrypoint.sh changes the "ci image" should be
rebuilt automatically.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 11:04:15 -05:00
Niels de Vos 43a863d1cb Add GetPoolByID() for calling rados_pool_reverse_lookup()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-01 14:40:47 -04:00
Niels de Vos bcf44fc782 Add GetPoolByName() for calling rados_pool_lookup()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-01 14:40:47 -04:00