Commit Graph

259 Commits

Author SHA1 Message Date
John Mulligan
a2d1c64180 rbd: separate standard lib imports from pkg imports
By convention many projects are separating the golang stdlib imports
from other imports. Do that as well for rbd.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 10:03:58 +01:00
John Mulligan
8a2d18be87 rados: include rados_test.go in rados package
Keeping the _test.go code in the same package allows for the testing
of private functions. AFAIK it is more conventional as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 09:49:47 +01:00
Yue Zhu
750fe1830e rados: add support for sending PG commands 2019-12-13 16:30:00 +01:00
John Mulligan
ce5d3ef04d cephfs: remove use of logging in package
After discussion we decided to entirely drop logging from the library.
Future users should rely instead of the error types and values the
function calls return.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-13 15:17:27 +01:00
John Mulligan
e07f6a6050 rados: replace Println calls with a proper error
Replace the odd calls to Println with a comparable error value.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-13 15:17:27 +01:00
John Mulligan
fdc33a8e69 github: configure a pull request template
Template provides brief reminders on what the project expects on
all prs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 16:17:31 -05:00
Niels de Vos
a713db983a rbd: return more useful error messages
Translate the return value (errno) from the C rbd_*() functions into the
standardized strerror() error message.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-12 13:56:53 -05:00
John Mulligan
dcfe969b09 rbd: use correct style for package doc comment
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:34:51 -05:00
John Mulligan
5d0b9b17a0 rados: use correct style for package doc comment
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:34:51 -05:00
John Mulligan
b3deb28b94 makefile: use revive for style checking
Switch use of style checker from golint to revive for both the makefile
and travis configuration.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:22:11 -05:00
John Mulligan
75fdc694be testing: add a project-wide toml config file for revive
Revive is a tool similar to golint but can be configured. Add an initial
configuration for revive.
This configuration reduces the initial issue list to something more
manageable and that can be worked on incrementally rather than trying to
boil to ocean to make the checker happy.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-12 11:22:11 -05:00
John Mulligan
5ea85a9419
Merge pull request #89 from CodeLingoBot/rewrite
Fix function comments based on best practices from Effective Go
2019-12-11 19:56:20 -05:00
Niels de Vos
2d52744470 rbd: improve error checking
Make it possible to easily validate the *Image and *Snapshot objects
before using them. This makes the use of go-ceph for RBD functions more
stable, and it is less likely that incorrect use causes Go panics.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 19:47:21 -05:00
John Mulligan
99de05854c ceph: remove unnecessary imports in doc.go
The package imports in doc.go were not accomplishing anything and also
were not complete (missing cephfs). This removes them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
John Mulligan
ab02cd27c0 ceph: expand doc to go a bit
Make the doc comment in doc.go a bit more fleshed out with pointers
to the real functionality of the library.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
John Mulligan
9460b11d88 ceph: rename package for go files in root dir
Oddly, probably a historical artifact, the .go files in the root of the
go-ceph tree were named "rados". As rados is one of our sub-packages it
seemed more correct to call the package "ceph".  This was chosen as
"go-ceph" is not a valid identifier in go and since this is already go
code prefixing with "go" seems a bit redundant.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-11 14:44:54 -05:00
Niels de Vos
b86563db7e rbd: add Create4() for more advanced RBD Image creation
This allows to set additional options while creating RBD images.
Ceph-CSI will initially consume this to configure an optionally
different pool for data.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos
c06593eef7 rbd: add support for RbdImageOptions
There are several RbdImageOptions that can be used to configure features
of an RBD image while creating, cloning, migrating and copying. A
follow-up patch will add Create4() where the RbdImageOptions can be
used.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos
1a06d47a0d tests: use Ceph Nautilus
Prepare for adding support for RbdImageOptions with
RBD_IMAGE_OPTION_FLATTEN that has been introduced in Ceph Mimic.
Nautilus is the most recent supported versions, so use that for testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 13:43:58 -05:00
Niels de Vos
6a17d59988 rbd: split Create() off into Create2() ad Create3() following librbd.h
The C library offers rbd_create2() and rbd_create3() and does not try to
cover all options with rbd_create(). For users that are familiar with
the C API, the Create2() and Create3() functions have been added.

The existing Create() API still handles the complete set of
rbd_create*() functions, so backwards compatibility is covered.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-12-11 09:09:54 -05:00
zhangyue
de3bd20fe3 bugfix: use GetError instead of RDBError to return
Signed-off-by: zhangyue <zy675793960@yeah.net>
2019-12-11 08:44:00 -05:00
John Mulligan
fa319f99bb rbd: use uuid library, rather than shelling out, in rbd test
Previously, the tests were shelling out to run 'uuidgen' which
is not particularly efficient and a bit ugly. Switch this test
to use the same uuid library already in use by the rados test code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-06 11:23:03 -05:00
Sven Anderson
472f6dd5bb Tests: avoid segmentation faults in tests
This change replaces a couple of functions from the "assert" package
with the corresponding functions from the "require", where the
following code relies on a non-nil object.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-06 11:12:34 -05:00
John Mulligan
20d12ba7bc test: disable test results caching with -count flag
According to the 'go test' help, "The idiomatic way to disable test
caching explicitly is to use -count=1." This removes GOCACHE=off in
favor of using -count=1 as GOCACHE=off breaks other tooling including
the use of go modules.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
John Mulligan
5b6bad390a test: make entrypoint script behavior configurable
Adds cli parsing to the entrypoint script allowing for:
1. Custom path to micro-osd script
   This allows easier testing of the script itself or quick
   and dirty customization for special one-off tests
2. Selecting tests to run (or "ALL" or "NONE")
   This allows the user to run only a subset of the tests as needed.
3. Option to pause indefinitely after tests complete
   This keeps the containerized environment alive after tests
   have run for debugging or just for creating a quick and dirty
   ceph environment for playing around.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
John Mulligan
ac1d6a6554 dockerfile: use exec form for entrypoint script
Using the exec form will allow us to pass arguments to the
container/entrypoint in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-05 14:04:15 -05:00
Sven Anderson
1bc4d888b8 Makefile: Fix CONTAINER_OPTS for Fedora
Fedora based distributions don't use AppArmor but SELinux instead.
This change detects the distribution and sets the --security-opts
accordingly.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-05 13:32:55 -05:00
Niels de Vos
09b6977dc9 rados: free cluster runtime resources automatically
Release resources that are allocated while configuring the connection to
the cluster. rados_shutdown() should only be needed after a successful
call to rados_connect(), however if the connection has been configured
with non-default parameters, some of the parameters may be allocated
before connecting. rados_shutdown() will free the allocated resources,
even if there has not been a connection yet.

Note that the finalizers get executed during garbage collection, which
can be forced by calling runtime.GC() for testing.

Fixes: #109
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-26 09:59:19 -05:00
Niels de Vos
cb575930e6 make: add support for running test-docker on SElinux enabled systems
When running 'make test-docker' on Fedora with SElinux in Enforcing
mode, fetching dependencies for the project fails. It seems the volume
is not accessible to the container. Enabling SElinux for the volume (by
passing the :z flag) makes it work.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-04 13:18:19 -05:00
Niels de Vos
72c262c493 Docker: prevent blocking prompt with add-apt-repository
A blocking message is shown when running 'make test-docker':

    STEP 6: RUN add-apt-repository ppa:gophers/archive
     Co-installable packages of Go 1.4 through 1.11 for supported versions
    of Ubuntu -- currently Trusty, Xenial, Bionic, Cosmic and Disco.

    Installing golang-1.X installs the latest point release of Go 1.X. I add
    the odd patch to keep older versions of Go building with new toolchains,
    but I do *not* make any effort to backport security fixes to Go releases
    that are not supported by upstream. Buyer beware!

    I aim to start uploading a release when its first beta is released, so
    e.g. I uploaded a golang-1.7  package after 1.7 beta 1 was released. But
    I don't always keep up with this.

    So that these packages can be co-installable, they do not install
    anything to /usr/bin. Once you have installed the golang-1.X package,
    you will need to add /usr/lib/go-1.X/bin (or maybe /usr/lib/go-tip/bin)
    to your $PATH, or you can just invoke /usr/lib/go-1.X/bin/go directly.

    The packages all Recommend: appropriate versions of the runtime race
    detector support on amd64, which are also built in this PPA
    (eventually).
     More info: https://launchpad.net/~gophers/+archive/ubuntu/archive
    Press [ENTER] to continue or ctrl-c to cancel adding it

By passing the -y switch to add-apt-repository, this message is ignored
and the process continues.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-04 13:18:19 -05:00
John Mulligan
93d8176a10 travis: add ceph nautilus to the test matrix
Add ceph nautilus to the test matrix as well as simplifying the setup
script in the travis yaml file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 13:15:30 -05:00
John Mulligan
a9a7137d59 test: update micro-osd.sh to support nautilus
Apparently 'rados mkpool' and 'rados rmpool' no longer function when
using nautilus.
Switch to ceph osd pool subcommands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 13:15:30 -05:00
John Mulligan
fb17c78c11 tests: generate html coverage report
When tests are run generate a coverage report.

To access this coverage report mount any directory to /results in the
container. Coverage output files will be written to /results/coverage
w/in the container.

Later we can do other interesting things with the coverage detected
such as a test pass/fail criteria or submitted to a service such as
coveralls, etc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 11:04:15 -05:00
John Mulligan
971027692f makefile: rebuild test container as needed
If the Dockerfile or entrypoint.sh changes the "ci image" should be
rebuilt automatically.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-04 11:04:15 -05:00
Niels de Vos
43a863d1cb Add GetPoolByID() for calling rados_pool_reverse_lookup()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-01 14:40:47 -04:00
Niels de Vos
bcf44fc782 Add GetPoolByName() for calling rados_pool_lookup()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-11-01 14:40:47 -04:00
John Mulligan
ef655aa904 makefile: support using other container runtimes
Add new variables to the Makefile that allow the use of other container
runtimes (that have a docker compatible cli) and allow setting extra
opts to work on platforms that don't run apparmor.
All defaults continue to be the same as before.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-11-01 14:35:28 -04:00
CodeLingo Bot
9e2fbe369f Fix function comments based on best practices from Effective Go
Signed-off-by: CodeLingo Bot <bot@codelingo.io>
2019-03-06 03:14:57 +00:00
Noah Watkins
e32f9f0f2e
Merge pull request #81 from MalloZup/golint
Enable golint in travis and Makefile
2018-12-17 14:15:54 -08:00
dmaiocchi
157d0c73b3 Enable golint in travis and Makefile 2018-12-11 23:38:36 +01:00
Noah Watkins
b8f75ed68d
Merge pull request #80 from ZYecho/fix-func-depred
bugfix: fix rados_read_op_omap_get_vals deprecated
2018-12-03 12:21:57 -08:00
zhangyue
7884e8a8af bugfix: fix function deprecated rados_read_op_omap_get_vals
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-12-02 11:04:21 +08:00
Noah Watkins
e1762cd8b3
Merge pull request #74 from mergetb/cephfs-add-rmdir
cephfs: add unmount, release, chmod, chown, etc
2018-10-11 14:59:13 -07:00
Lincoln Thurlow
5a74b46a87 cephfs: modified guard clauses + golint'd 2018-10-11 11:36:26 -07:00
Lincoln Thurlow
ecf9a99249 cephfs: add unmount, release, chmod, chown, etc
This commit adds the following cephfs functions:

* Unmount // Unmounting is necessary to cleanup mounts
* Release // Release destroys the cmount ~ end of transaction
* RemoveDir // inverse of MakeDir
* Chown // change ownership of file or directory
* Chmod // change permissions of file or directory

Tests are included for each function.

In addition to these changes modifications to:

.travis.yml, Dockerfile, and Makefile

were made to accomodate tests to mount the ceph volume.  Tests use
fuse to mount the volume which requires adding:

--device /dev/fuse --cap-add SYS_ADMIN --security-opt \
apparmor:unconfined

to the docker container (alternatively --privileged works but adds
additional permissions).

Changes to README add the above docker changes as well as point
users to the necessary ceph development libraries.
2018-10-11 11:09:16 -07:00
Lincoln Thurlow
f3df337fab cephfs: add cephfs commands, expand logging
* CephError uses syscall for string version of error
* Add error logging for every function
* Add RemoveDir() function
* Add Unmount() function
* Add Release() function
* Add Chown() function
* Add Chmod() function
2018-10-11 11:09:16 -07:00
Noah Watkins
768c5507d1
Merge pull request #73 from mergetb/cephfs-fix-docker
go-ceph: add cephfs to docker

fixes: #35
2018-10-09 21:20:50 -07:00
Lincoln Thurlow
b7c7659ec4 go-ceph: add cephfs to docker
* Adds metadata server (mds) for cephfs to Dockerfile
* Enables cephfs tests
2018-10-09 20:56:52 -07:00
Noah Watkins
21c4240aa3
Merge pull request #69 from MalloZup/Nitpick
Use Guardclause for function
2018-10-09 10:24:23 -07:00
Dario Maiocchi
caf0f7ed68 Use Guardclause for function 2018-10-09 10:22:02 +02:00