Commit Graph

246 Commits

Author SHA1 Message Date
Niels de Vos
2d52744470 rbd: improve error checking
Make it possible to easily validate the *Image and *Snapshot objects
before using them. This makes the use of go-ceph for RBD functions more
stable, and it is less likely that incorrect use causes Go panics.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 19:47:21 -05:00
John Mulligan
99de05854c ceph: remove unnecessary imports in doc.go
The package imports in doc.go were not accomplishing anything and also
were not complete (missing cephfs). This removes them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
John Mulligan
ab02cd27c0 ceph: expand doc to go a bit
Make the doc comment in doc.go a bit more fleshed out with pointers
to the real functionality of the library.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
John Mulligan
9460b11d88 ceph: rename package for go files in root dir
Oddly, probably a historical artifact, the .go files in the root of the
go-ceph tree were named "rados". As rados is one of our sub-packages it
seemed more correct to call the package "ceph".  This was chosen as
"go-ceph" is not a valid identifier in go and since this is already go
code prefixing with "go" seems a bit redundant.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
Niels de Vos
b86563db7e rbd: add Create4() for more advanced RBD Image creation
This allows to set additional options while creating RBD images.
Ceph-CSI will initially consume this to configure an optionally
different pool for data.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos
c06593eef7 rbd: add support for RbdImageOptions
There are several RbdImageOptions that can be used to configure features
of an RBD image while creating, cloning, migrating and copying. A
follow-up patch will add Create4() where the RbdImageOptions can be
used.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos
1a06d47a0d tests: use Ceph Nautilus
Prepare for adding support for RbdImageOptions with
RBD_IMAGE_OPTION_FLATTEN that has been introduced in Ceph Mimic.
Nautilus is the most recent supported versions, so use that for testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos
6a17d59988 rbd: split Create() off into Create2() ad Create3() following librbd.h
The C library offers rbd_create2() and rbd_create3() and does not try to
cover all options with rbd_create(). For users that are familiar with
the C API, the Create2() and Create3() functions have been added.

The existing Create() API still handles the complete set of
rbd_create*() functions, so backwards compatibility is covered.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 09:09:54 -05:00
zhangyue
de3bd20fe3 bugfix: use GetError instead of RDBError to return
Signed-off-by: zhangyue <zy675793960@yeah.net>
2019-12-11 08:44:00 -05:00
John Mulligan
fa319f99bb rbd: use uuid library, rather than shelling out, in rbd test
Previously, the tests were shelling out to run 'uuidgen' which
is not particularly efficient and a bit ugly. Switch this test
to use the same uuid library already in use by the rados test code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-06 11:23:03 -05:00
Sven Anderson
472f6dd5bb Tests: avoid segmentation faults in tests
This change replaces a couple of functions from the "assert" package
with the corresponding functions from the "require", where the
following code relies on a non-nil object.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-06 11:12:34 -05:00
John Mulligan
20d12ba7bc test: disable test results caching with -count flag
According to the 'go test' help, "The idiomatic way to disable test
caching explicitly is to use -count=1." This removes GOCACHE=off in
favor of using -count=1 as GOCACHE=off breaks other tooling including
the use of go modules.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
John Mulligan
5b6bad390a test: make entrypoint script behavior configurable
Adds cli parsing to the entrypoint script allowing for:
1. Custom path to micro-osd script
   This allows easier testing of the script itself or quick
   and dirty customization for special one-off tests
2. Selecting tests to run (or "ALL" or "NONE")
   This allows the user to run only a subset of the tests as needed.
3. Option to pause indefinitely after tests complete
   This keeps the containerized environment alive after tests
   have run for debugging or just for creating a quick and dirty
   ceph environment for playing around.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
John Mulligan
ac1d6a6554 dockerfile: use exec form for entrypoint script
Using the exec form will allow us to pass arguments to the
container/entrypoint in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
Sven Anderson
1bc4d888b8 Makefile: Fix CONTAINER_OPTS for Fedora
Fedora based distributions don't use AppArmor but SELinux instead.
This change detects the distribution and sets the --security-opts
accordingly.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-05 13:32:55 -05:00
Niels de Vos
09b6977dc9 rados: free cluster runtime resources automatically
Release resources that are allocated while configuring the connection to
the cluster. rados_shutdown() should only be needed after a successful
call to rados_connect(), however if the connection has been configured
with non-default parameters, some of the parameters may be allocated
before connecting. rados_shutdown() will free the allocated resources,
even if there has not been a connection yet.

Note that the finalizers get executed during garbage collection, which
can be forced by calling runtime.GC() for testing.

Fixes: #109
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-26 09:59:19 -05:00
Niels de Vos
cb575930e6 make: add support for running test-docker on SElinux enabled systems
When running 'make test-docker' on Fedora with SElinux in Enforcing
mode, fetching dependencies for the project fails. It seems the volume
is not accessible to the container. Enabling SElinux for the volume (by
passing the :z flag) makes it work.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-04 13:18:19 -05:00
Niels de Vos
72c262c493 Docker: prevent blocking prompt with add-apt-repository
A blocking message is shown when running 'make test-docker':

    STEP 6: RUN add-apt-repository ppa:gophers/archive
     Co-installable packages of Go 1.4 through 1.11 for supported versions
    of Ubuntu -- currently Trusty, Xenial, Bionic, Cosmic and Disco.

    Installing golang-1.X installs the latest point release of Go 1.X. I add
    the odd patch to keep older versions of Go building with new toolchains,
    but I do *not* make any effort to backport security fixes to Go releases
    that are not supported by upstream. Buyer beware!

    I aim to start uploading a release when its first beta is released, so
    e.g. I uploaded a golang-1.7  package after 1.7 beta 1 was released. But
    I don't always keep up with this.

    So that these packages can be co-installable, they do not install
    anything to /usr/bin. Once you have installed the golang-1.X package,
    you will need to add /usr/lib/go-1.X/bin (or maybe /usr/lib/go-tip/bin)
    to your $PATH, or you can just invoke /usr/lib/go-1.X/bin/go directly.

    The packages all Recommend: appropriate versions of the runtime race
    detector support on amd64, which are also built in this PPA
    (eventually).
     More info: https://launchpad.net/~gophers/+archive/ubuntu/archive
    Press [ENTER] to continue or ctrl-c to cancel adding it

By passing the -y switch to add-apt-repository, this message is ignored
and the process continues.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-04 13:18:19 -05:00
John Mulligan
93d8176a10 travis: add ceph nautilus to the test matrix
Add ceph nautilus to the test matrix as well as simplifying the setup
script in the travis yaml file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 13:15:30 -05:00
John Mulligan
a9a7137d59 test: update micro-osd.sh to support nautilus
Apparently 'rados mkpool' and 'rados rmpool' no longer function when
using nautilus.
Switch to ceph osd pool subcommands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 13:15:30 -05:00
John Mulligan
fb17c78c11 tests: generate html coverage report
When tests are run generate a coverage report.

To access this coverage report mount any directory to /results in the
container. Coverage output files will be written to /results/coverage
w/in the container.

Later we can do other interesting things with the coverage detected
such as a test pass/fail criteria or submitted to a service such as
coveralls, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 11:04:15 -05:00
John Mulligan
971027692f makefile: rebuild test container as needed
If the Dockerfile or entrypoint.sh changes the "ci image" should be
rebuilt automatically.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 11:04:15 -05:00
Niels de Vos
43a863d1cb Add GetPoolByID() for calling rados_pool_reverse_lookup()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-01 14:40:47 -04:00
Niels de Vos
bcf44fc782 Add GetPoolByName() for calling rados_pool_lookup()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-01 14:40:47 -04:00
John Mulligan
ef655aa904 makefile: support using other container runtimes
Add new variables to the Makefile that allow the use of other container
runtimes (that have a docker compatible cli) and allow setting extra
opts to work on platforms that don't run apparmor.
All defaults continue to be the same as before.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-01 14:35:28 -04:00
Noah Watkins
e32f9f0f2e
Merge pull request #81 from MalloZup/golint
Enable golint in travis and Makefile
2018-12-17 14:15:54 -08:00
dmaiocchi
157d0c73b3 Enable golint in travis and Makefile 2018-12-11 23:38:36 +01:00
Noah Watkins
b8f75ed68d
Merge pull request #80 from ZYecho/fix-func-depred
bugfix: fix rados_read_op_omap_get_vals deprecated
2018-12-03 12:21:57 -08:00
zhangyue
7884e8a8af bugfix: fix function deprecated rados_read_op_omap_get_vals
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-12-02 11:04:21 +08:00
Noah Watkins
e1762cd8b3
Merge pull request #74 from mergetb/cephfs-add-rmdir
cephfs: add unmount, release, chmod, chown, etc
2018-10-11 14:59:13 -07:00
Lincoln Thurlow
5a74b46a87 cephfs: modified guard clauses + golint'd 2018-10-11 11:36:26 -07:00
Lincoln Thurlow
ecf9a99249 cephfs: add unmount, release, chmod, chown, etc
This commit adds the following cephfs functions:

* Unmount // Unmounting is necessary to cleanup mounts
* Release // Release destroys the cmount ~ end of transaction
* RemoveDir // inverse of MakeDir
* Chown // change ownership of file or directory
* Chmod // change permissions of file or directory

Tests are included for each function.

In addition to these changes modifications to:

.travis.yml, Dockerfile, and Makefile

were made to accomodate tests to mount the ceph volume.  Tests use
fuse to mount the volume which requires adding:

--device /dev/fuse --cap-add SYS_ADMIN --security-opt \
apparmor:unconfined

to the docker container (alternatively --privileged works but adds
additional permissions).

Changes to README add the above docker changes as well as point
users to the necessary ceph development libraries.
2018-10-11 11:09:16 -07:00
Lincoln Thurlow
f3df337fab cephfs: add cephfs commands, expand logging
* CephError uses syscall for string version of error
* Add error logging for every function
* Add RemoveDir() function
* Add Unmount() function
* Add Release() function
* Add Chown() function
* Add Chmod() function
2018-10-11 11:09:16 -07:00
Noah Watkins
768c5507d1
Merge pull request #73 from mergetb/cephfs-fix-docker
go-ceph: add cephfs to docker

fixes: #35
2018-10-09 21:20:50 -07:00
Lincoln Thurlow
b7c7659ec4 go-ceph: add cephfs to docker
* Adds metadata server (mds) for cephfs to Dockerfile
* Enables cephfs tests
2018-10-09 20:56:52 -07:00
Noah Watkins
21c4240aa3
Merge pull request #69 from MalloZup/Nitpick
Use Guardclause for function
2018-10-09 10:24:23 -07:00
Dario Maiocchi
caf0f7ed68 Use Guardclause for function 2018-10-09 10:22:02 +02:00
Noah Watkins
92d7415a65
Merge pull request #71 from MalloZup/examplego
Make example more golang
2018-10-07 07:23:36 -07:00
dmaiocchi
f34b1cc12a Make example more golang 2018-10-07 09:36:09 +02:00
Noah Watkins
336289075e
Merge pull request #68 from dswarbrick/code-cleanups
Code cleanups
2018-09-18 08:55:02 -07:00
Daniel Swarbrick
951118e855
Drop unnecessary typecasts
Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
2018-09-18 11:12:32 +02:00
Daniel Swarbrick
3023eb281d
Go style cleanups
- Use C errno constants in place of hardcoded ints
- Use type inference where possible (especially C.CString
- Don't explicitly specify zero values where value would be default anway
- Rewrap some unnecessarily wrapped short lines
- Use if/else one-liners where value is not needed afterwards

Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
2018-09-18 11:03:41 +02:00
Noah Watkins
c521b5aa72
Merge pull request #67 from ceph/ci-go-fmt
Ci go fmt
2018-09-17 16:06:29 -07:00
Noah Watkins
208695a8c1 fmt: apply gofmt updates
Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-09-17 15:57:31 -07:00
Noah Watkins
6107372c34 ci: check go formatting
Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-09-17 15:41:24 -07:00
Noah Watkins
fd4036e498
Merge pull request #66 from dswarbrick/trash-wrappers
Implement wrappers around rbd_trash_* functions
2018-09-17 15:28:21 -07:00
Daniel Swarbrick
bdc1c100fe
Force trash remove to override deferment time
Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
2018-09-17 23:33:23 +02:00
Daniel Swarbrick
837181cb94
Add unit test for rbd_trash functionality
Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
2018-09-17 23:07:18 +02:00
Daniel Swarbrick
4a88b5da12
Implement wrappers around rbd_trash_* functions
Signed-off-by: Daniel Swarbrick <daniel.swarbrick@gmail.com>
2018-09-17 20:46:23 +02:00
Noah Watkins
28db807c97
Merge pull request #65 from ceph/connected-state
Connected state
2018-09-14 15:10:35 -07:00