Commit Graph

509 Commits

Author SHA1 Message Date
John Mulligan
9a621942f3 gitignore: ignore .test and .json files
The .test files are generated by some makefile rules and should
never be checked in.
There's also no reason to track json files which may be generated
by running tools.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-14 14:35:01 +02:00
John Mulligan
e2a78eec02 cutil: allow passing free functions to command output type
The *_command functions in librados and libcephfs document the use
of specific free functions for data allocated. These functions are
currently just wrappers around C's free() function. However, to be
more strictly compliant this change adds a free-function callback
to the CommandOutput type and the specific free functions are now
used outside the unit tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
f2d0bb4692 cephfs: replace copy-n-paste *_command functions argument handling
Similar to the functions in the rados pkg, cephfs package has a function
MdsCommand that is mostly boilerplate that to set up the C function's
arguments. Replace it with the types from the new cutil internal
package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
817b6b68c2 rados: replace copy-n-paste *_command functions argument handling
Replace the argument handling (wrangling?) around *_command functions,
such as MonCommand, with the new command input and output helper types
from the internal/cutil package.
No external change should be visible.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
7256dbb27b entrypoint: enable building and testing of internal/cutil package
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
e40744fdf6 cutil: add a new internal package for c+go utility functions
Now we have sufficient boilerplate in our code for interacting with
various types and ceph calls with similar needs we establish a new
internal package, "cutil" (C utilities).

Note that many of the return types are wrapped. This is due to the
limits placed on us by cgo.  Despite the irritating limitations Go
places on "exporting" C types it still ought to help in the long run for
patterns that are very common or patterns that are subtle and we want to
write specific tests for.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
d4079e3949 cephfs: add path based Statx function implmenting ceph_statx
Add a Statx wrapper for ceph_statx.
Add a type wrapping the statx status info that exposes the various
fields from the C-struct.
Add a type wrapping struct timespec, based on golang's x/sys, for the
time fields in the struct.
Note that the ceph struct is not the same as the linux statx struct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-07 13:44:19 -04:00
muagarwa
27a1824ca0 cephfs: implement wrappers for fchmod and fchown
Fchmod() implemenets ceph_fchmod()
Fchown() implements ceph_fchown()
To fix: https://github.com/ceph/go-ceph/issues/240

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-07 09:43:05 +02:00
muagarwa
fa7300fff8 cephfs: Validate mount.mount usage in cephfs/path.go
ceph MountInfo pointer is being passed in the functions without a validation check, if it is nil there may be a crash. Fixed the functions to first validate mount.mount

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-06 10:04:43 -04:00
root
e89344cae1 fix typos 2020-05-01 09:37:09 -04:00
muagarwa
9e43e5119d cephfs: adding Link, Symlink and Readlink functions
Link function implements ceph_link().
Symlink function implements ceph_symlink().
Readlink function implements ceph_readlink().
To fix https://github.com/ceph/go-ceph/issues/218

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-04-30 13:28:51 -04:00
John Mulligan
fde15e439e cephfs: add DType type and getter function for directory entries
CephFS returns Linux-style struct dirent that has a d_type field.
It appears to properly make use of the field. This change exposes
the field using a DType type and associated constants as well
as a DType() function for the DirEntry.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-30 12:54:37 -04:00
John Mulligan
7b2e18b5e1 cephfs: reject negative offsets for ReadAt/WriteAt
Negative offsets that are passed through ReadAt/WriteAt to the ceph
calls cause them to behave like Read/Write and this is undesirable
at the go layer. Additionally, the os packages' file functions
reject negative offsets as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
f7936e4ac4 cephfs: add a test to ensure behavior of mixed use of Read + ReadAt
Add a test case to ensure the behavior of a mixed used [1] of Read
and ReadAt calls is consistent and ReadAt doesn't side effect
the file position used by Read.

1 - Not recommended, but we should not break now or in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
938506e1d3 cephfs: have Read/ReadAt return io.EOF error when nothing is read
Have Read and ReadAt functions return io.EOF when nothing is read
from the file so that it matches the current behavior of file types
in Go standard library.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
1899072b27 cephfs: add additional error handling and state checks to file funcs
Now, the close function is idempotent wrt being called multiple times
on a valid file. Additionally, check the state of the file in various
functions to produce sensible errors, matching those cephfs itself
returns, if the object was not constructed properly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
c37ec2104a cephfs: add an Unlink func implmenting ceph_unlink
The unlink function can now be used to clean up files from the other
existing test cases.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
854a834b1c cephfs: implement file IO functions for open/close/read/write/seek
Implement core file I/O functions based on a file handle wrapper.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
fb2736dd0a rados: add MgrCommand and MgrCommandWithInputBuffer functions
Add functions for issuing commands to ceph mgrs with
MgrCommand and MgrCommandWithInputBuffer, implemented simliarly
to the other *Command style functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-27 14:38:16 -04:00
John Mulligan
ff805ee1e1 rados: remove named return values in command.go
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-27 14:38:16 -04:00
John Mulligan
79dfe8e631 rados: move *Command functions into a separate file
Organize the rados package a bit more by moving functions that wrap
rados_xyz_command functions into a dedicated file. Move related test to
a separate file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-27 14:38:16 -04:00
muagarwa
301d8e427f Makefile: Fix CONTAINER_OPTS for Fedora
make test-container is failing on Fedora based distribution because of bad label option "disabled".
Change label option to "disable"

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-04-23 08:22:07 -04:00
John Mulligan
1eb81971a6 workflows: get coverage result from ci container and archive it
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-21 15:56:40 -04:00
John Mulligan
204a7558d4 makefile: support getting results locally from container test
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-21 15:56:40 -04:00
Sven Anderson
6072f41bd6 update more code to use the retry lib.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
Sven Anderson
e8da761769 update various parts of the code to use the retry lib.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
Sven Anderson
e01eb03284 retry: add a helper lib for retrying common operations
Our first operation is generating sizes for various caller-allocated
buffers.  The WithSizes function "suggests" sizes to an anonymous
function that takes a size as an argument and returns a hint how
sucessful the attempt was.  If required the same function is called
again with a larger size.  Errors and other results of that anonymous
function are "communicated" with variables of the surrounding scope
in wich the function is defined.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
John Mulligan
5aadca02bf cephfs: split basic permissions related funcs into new files
Continue organizing the cephfs functionality by creating new files
for chmod and chown functionality.
Similarly, the dedicated test functions for those items are moved
into a new file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
bbf5fecf90 cephfs: split basic path management funcs into new files
Continue organizing the cephfs functionality by creating new files
for the most basic path management functions:
  MakeDir, RemoveDir, ChangeDir, CurrentDir
Similarly, the dedicated test functions for those items are moved into
a new file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
ce64ef536a cephfs: split (mds) command functions and tests into new files
Start organizing some of the cephfs functionality into functions grouped
by files by moving the MDS command functions into a new file. Move the
tests dedicated to those functions into a new matching file too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
29c88f4b2d cephfs: add functions for Init and SetMountPerms
The Init functions initializes the connection without mounting the fs.
This function must be called before setting the perms but before
creating the mount.

SetMountPerms accepts a UserPerm object to set the fs ownership info
on the mount. The corresponding tests verifies that the UserPerm
can be applied and it effects the gid of newly created dirs for that
mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
e1bbd45b5c cephfs: add a UserPerm type wrapping libcephfs UserPerm
The UserPerm type will be used later to set the owner credentials on the
mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
673c60bc51 cephfs: have test cases remove dirs they create
To prevent test failures re-reunning the suite against the same fs we
need to make sure tests clean up after themselves.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
175cba3388 revive: enable rule to check for unhandled errors
Enable the revive rule to ensure we check the return values of various
function calls. Put common printing functions in the skip list of the
check, because no-one ever checks the error of those. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
dd46304a07 revive: enable rules to limit argument and return counts
We already have a few functions with unwieldy number of arguments. Cap
the max number of arguments to allow at our current max (7) and cap
the number of return values at 3.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
9764170d5c revive: enable some additional code quality rules
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
c9acb56afe revive: enable indent-error-flow check
Now that we have no code that triggers this check, re-enable the
indent-error-flow check in the revive config.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
b7a3914f3b ceph: remove use of boolean literal in expression
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
20311caff5 rados: check errors in test when deleting pools
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
a1de43cd5d rados: improve clean up of temporary file used in test
Clean up the temporary file using a defer and make sure that the
errors returned by the functions in the defer func are checked.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
c4f8f0445a rados: check errors returned by Write calls
Don't ignore errors returned by ioctx Write function calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
38adf053dd rados: check errors returned when reading config
Don't ignore the error return from ReadDefaultConfigFile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
0faccb56e8 rbd: check that encode calls dont return error
Don't ignore the error returned by json encode func call.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
8ce5fa5fc7 rbd: remove a redundant else statement
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 13:11:05 -04:00
John Mulligan
9f6d4d48bb cephfs: add a RewindDir and List function for simpler dir listings
The RewindDir is used by List and can be used independently. It
resets the open dir to the "start".
The List function and the accompanying DirEntries type allow one
to simply grab all the dir entries at once. The Names method on the
DirEntries type allows the user to get the most commonly wanted items
from the dir entries in one shot.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
fd275e2b9a cephfs: add ReadDir implementing ceph_readdir_r function
Adds a method to ReadDir that implements ceph_readdir_r returning
a single dir entry at at time.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
e98648f76f cephfs: add Directory type and funcs to open and close it
Add the Directory type and the OpenDir function to get an open
Directory and a Close function to close/free it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
e1fd17df8b readme: advertise maintainer "office hours"
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-08 10:31:23 -04:00
John Mulligan
973def5658 makefile: remove shell loop that was ignoring errors
revive would only fail if rbd had errors due to the loop being in shell.
Now we expand the modules list as arguments to revive which then
invokes revive only once and properly handles errors.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-07 13:55:51 -04:00
John Mulligan
71d533d14f makefile: no need to require a tty for running tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-07 13:55:51 -04:00