Commit Graph

485 Commits

Author SHA1 Message Date
Sven Anderson
6072f41bd6 update more code to use the retry lib.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
Sven Anderson
e8da761769 update various parts of the code to use the retry lib.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
Sven Anderson
e01eb03284 retry: add a helper lib for retrying common operations
Our first operation is generating sizes for various caller-allocated
buffers.  The WithSizes function "suggests" sizes to an anonymous
function that takes a size as an argument and returns a hint how
sucessful the attempt was.  If required the same function is called
again with a larger size.  Errors and other results of that anonymous
function are "communicated" with variables of the surrounding scope
in wich the function is defined.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
John Mulligan
5aadca02bf cephfs: split basic permissions related funcs into new files
Continue organizing the cephfs functionality by creating new files
for chmod and chown functionality.
Similarly, the dedicated test functions for those items are moved
into a new file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
bbf5fecf90 cephfs: split basic path management funcs into new files
Continue organizing the cephfs functionality by creating new files
for the most basic path management functions:
  MakeDir, RemoveDir, ChangeDir, CurrentDir
Similarly, the dedicated test functions for those items are moved into
a new file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
ce64ef536a cephfs: split (mds) command functions and tests into new files
Start organizing some of the cephfs functionality into functions grouped
by files by moving the MDS command functions into a new file. Move the
tests dedicated to those functions into a new matching file too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
29c88f4b2d cephfs: add functions for Init and SetMountPerms
The Init functions initializes the connection without mounting the fs.
This function must be called before setting the perms but before
creating the mount.

SetMountPerms accepts a UserPerm object to set the fs ownership info
on the mount. The corresponding tests verifies that the UserPerm
can be applied and it effects the gid of newly created dirs for that
mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
e1bbd45b5c cephfs: add a UserPerm type wrapping libcephfs UserPerm
The UserPerm type will be used later to set the owner credentials on the
mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
673c60bc51 cephfs: have test cases remove dirs they create
To prevent test failures re-reunning the suite against the same fs we
need to make sure tests clean up after themselves.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
175cba3388 revive: enable rule to check for unhandled errors
Enable the revive rule to ensure we check the return values of various
function calls. Put common printing functions in the skip list of the
check, because no-one ever checks the error of those. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
dd46304a07 revive: enable rules to limit argument and return counts
We already have a few functions with unwieldy number of arguments. Cap
the max number of arguments to allow at our current max (7) and cap
the number of return values at 3.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
9764170d5c revive: enable some additional code quality rules
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
c9acb56afe revive: enable indent-error-flow check
Now that we have no code that triggers this check, re-enable the
indent-error-flow check in the revive config.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
b7a3914f3b ceph: remove use of boolean literal in expression
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
20311caff5 rados: check errors in test when deleting pools
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
a1de43cd5d rados: improve clean up of temporary file used in test
Clean up the temporary file using a defer and make sure that the
errors returned by the functions in the defer func are checked.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
c4f8f0445a rados: check errors returned by Write calls
Don't ignore errors returned by ioctx Write function calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
38adf053dd rados: check errors returned when reading config
Don't ignore the error return from ReadDefaultConfigFile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
0faccb56e8 rbd: check that encode calls dont return error
Don't ignore the error returned by json encode func call.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
8ce5fa5fc7 rbd: remove a redundant else statement
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
9f6d4d48bb cephfs: add a RewindDir and List function for simpler dir listings
The RewindDir is used by List and can be used independently. It
resets the open dir to the "start".
The List function and the accompanying DirEntries type allow one
to simply grab all the dir entries at once. The Names method on the
DirEntries type allows the user to get the most commonly wanted items
from the dir entries in one shot.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
fd275e2b9a cephfs: add ReadDir implementing ceph_readdir_r function
Adds a method to ReadDir that implements ceph_readdir_r returning
a single dir entry at at time.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
e98648f76f cephfs: add Directory type and funcs to open and close it
Add the Directory type and the OpenDir function to get an open
Directory and a Close function to close/free it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
e1fd17df8b readme: advertise maintainer "office hours"
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-08 10:31:23 -04:00
John Mulligan
973def5658 makefile: remove shell loop that was ignoring errors
revive would only fail if rbd had errors due to the loop being in shell.
Now we expand the modules list as arguments to revive which then
invokes revive only once and properly handles errors.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-07 13:55:51 -04:00
John Mulligan
71d533d14f makefile: no need to require a tty for running tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-07 13:55:51 -04:00
John Mulligan
fee7ff61ce ci: add a github workflow config for CI
This should basically run similarly to the travis ci configuration
but use the native github ci infrastructure.
In order to avoid messing around with various env vars I had to use the
v2-beta version of the go setup library. Once v2 has been released
we can switch away from the beta.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-07 13:55:51 -04:00
John Mulligan
be7a90a2be rados: use getError helper in general error conversion cases
For more consisistency use getError where possible.
Changes produced using `gofmt -r 'RadosError(int(x)) -> getError(x)`,
followed by a few manual fixups.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 17:10:09 -04:00
John Mulligan
5394d1c99f rados: update error conversion func to be consistent with rbd & cephfs
Rename the getRadosError to getError and update it to match the other
packages behavior.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 17:10:09 -04:00
John Mulligan
f584aef60f cephfs: consolidate error types & values in an errors.go file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 17:10:09 -04:00
John Mulligan
054ae8ddff rbd: consolidate error types & values in an errors.go file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 17:10:09 -04:00
John Mulligan
1f4543e835 rados: consolidate error types & values in an errors.go file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 17:10:09 -04:00
John Mulligan
a5e5ae6206 entrypoint: mention env vars that are not customized by script
Mention the env vars that the go test code looks for but does not change
from the "outer" environment as a way of documenting those variables.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
2ad8361692 cephfs: make the Release function more idempotent to callers
Previously, calling Release more than once for the same MountInfo would
abort due to a double free in the ceph libs. As this is somewhat user
hostile we add some simple state tracking in our wrapper type such that
one can make redundant calls to Release without crashing the
application.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
4277d40137 cephfs: remove directory created in the ChangeDir test
The test function that exercises ChangeDir was creating a directory
and leaving it behind. Add some basic cleanup to the test to try
and avoid leaking stuff between test runs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
36196c5e60 cephfs: release fs resources in remaining test cases
In a previous commit we added fsDisconnect to match with fsConnect.  For
tests that do not use fsConnect but still acquire resources we need to
free up those resources with direct calls to the appropriate functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
1842ad248d cephfs: pair fsConnect with a fsDisconnect function
For every (most?) call to fsConnect we should pair it with a (deferred)
call to fsDisconnect to close connection and release resources
associated with the mount object we have created for the test.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
bdee9f4aab cephfs: TestMountWithRoot should only try unmounting after mounting
The defer function in TestMountWithRoot was always trying to call
unmount before release. This is not correct if the fs was not mounted
so we reorganize this code to more closely match what should be done
in normal practice.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
6a08124775 cephfs: TestCreateMount should not try to mount twice
Somehow we managed to introduce extra code into TestCreateMount that
had it create two mounts. This is undesirable for this test so this
change eliminates the extra unwanted invocation.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
ab178bc7ed cephfs: allow passing mds name into test through env var
Previously, the mds name was hardcoded in the tests to match that set up
by "micro-osd.sh". We continue to default to that value, but allow the
test to be supplied a custom value by the environment.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
900302b1ce entrypoint: set default env var for requiring cephfs mount point
When running cephfs tests locally there is no strong urge to require
an environment exactly like that of the test container. However, we do
want tests run under the container image (especially our default ci) to
use the cephfs mountpoint as they were previously. Set a default value
in the entrypoint script to require the mountpoint so we don't
accidentally miss the tests if something unexpected goes wrong.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
0b658baebf cephfs: restrict running of tests that need a mounted fs instance
Add a utility function to test cases that expect the cephfs file system
to be mounted locally and accessible.
Add environment vars to configure the location of the mount point as
well whether the function causes the test to be skipped (the default)
or force an early but clear failure.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
2c249311ef travis: use "octopus" release by name
Within the last 24hrs the "octopus" release of ceph is available, and
matching tags have been created in the ceph containers project. Consume
the release name rather than master.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-25 17:37:43 +01:00
John Mulligan
eb6386e416 testing: set "exported" rule failures back to errors
We had a lot of "exported" badness previously but that either been
resolved or disabled on a case-by-case basis. Make the rule fail the
check so that new cases like this are not brought in.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00
John Mulligan
eee19d642f rbd: rename uses of RbdImageOptions to ImageOptions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00
John Mulligan
a29025a893 rbd: drop the Rbd prefix from ImageOptions and ImageOption
Previously, the types were named RbdImageOptions and RbdImageOption.
Now they are named ImageOptions and ImageOption.
Type aliases have been added to prevent existing code from
breaking. As these types were new in v0.2, I suspect we can drop
the aliases post-v0.3.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00
John Mulligan
51827ead4e rbd: convert uses of RbdImageOptionX to ImageOptionX
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00
John Mulligan
6d77dc97dc rbd: remove Rbd prefix from constants and provide aliases
Image option constants are now named ImageOptionX, without an Rbd
prefix. Aliases to the old names are provided and doc comments for both.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00
John Mulligan
c2d0bc90ef rbd: add comments to features go file to pass check tool
Make the features.go file clean to the revive checker tool.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00
John Mulligan
8f2671fee9 rbd: add doc comments for (Get|Set|Remove)Metadata funcs
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-23 10:09:25 -04:00