Commit Graph

169 Commits

Author SHA1 Message Date
Niels de Vos
c4714165a6 timespec: move to internal pkg
There are functions from librbd that can use the Timespec type too.
Instead of consuming 'cephfs.Timespec' in the rbd package, create an
internal type that can be exposed through both packages.

This looks a bit like duplication, but it does not break the current
users of Timespec in the cephfs package.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-22 09:22:18 -04:00
John Mulligan
3a561218d4 cephfs: add Rename function
Rename, it does what it says on the tin. Tests included.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-15 13:59:27 -04:00
John Mulligan
624230cdbd cephfs: add Fsync and Sync calls for open files
Add wrappers for ceph_fsync. The Fsync call directly wraps ceph_fsync
which provides options to behave more like fsync or more like fdatasync.
Add Sync, a simpler wrapper over Fsync, to match any interfaces that
make use of `File.Sync` from Go's os package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-11 11:21:11 -04:00
Mudit Agarwal
092c0b7832 cephfs: add ceph_flock() function
Added ceph_flock() function which applies or removes an advisory lock on an open file.

Fixes: https://github.com/ceph/go-ceph/issues/248

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-27 10:40:52 -04:00
Mudit Agarwal
1d30b636ff cephfs: add ceph_fallocate() function
Added ceph_fallocate() which  preallocates or releases disk space
for the file for the given byte range.

Fixes: https://github.com/ceph/go-ceph/issues/246

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-20 13:12:55 -04:00
John Mulligan
80833a6789 cephfs: add Fstatx function to file type
Use the CephStatx type previously added to the library to implement a
wrapper for the ceph_fstatx function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-19 17:06:44 -04:00
John Mulligan
807eafb9d5 cephfs: add ReadDirPlus function and associated types
Add the ReadDirPlus function call, wrapping ceph_readdirplus_r. Add
DirEntryPlus which is a DirEntry plus a getter for the statx field.
Add a test similar to the ReadDir test and (since it is possible to
induce an error with ceph_readdirplus_r) a test for the error handling
path to improve code coverage.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-19 16:35:58 -04:00
John Mulligan
ce8e45541b cephfs: add test coverage for invalid file state in Fchown
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-14 09:27:39 -04:00
John Mulligan
92a85b4385 cephfs: add test coverage for invalid file state in Fchmod
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-14 09:27:39 -04:00
John Mulligan
e2a78eec02 cutil: allow passing free functions to command output type
The *_command functions in librados and libcephfs document the use
of specific free functions for data allocated. These functions are
currently just wrappers around C's free() function. However, to be
more strictly compliant this change adds a free-function callback
to the CommandOutput type and the specific free functions are now
used outside the unit tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
f2d0bb4692 cephfs: replace copy-n-paste *_command functions argument handling
Similar to the functions in the rados pkg, cephfs package has a function
MdsCommand that is mostly boilerplate that to set up the C function's
arguments. Replace it with the types from the new cutil internal
package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan
d4079e3949 cephfs: add path based Statx function implmenting ceph_statx
Add a Statx wrapper for ceph_statx.
Add a type wrapping the statx status info that exposes the various
fields from the C-struct.
Add a type wrapping struct timespec, based on golang's x/sys, for the
time fields in the struct.
Note that the ceph struct is not the same as the linux statx struct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-07 13:44:19 -04:00
muagarwa
27a1824ca0 cephfs: implement wrappers for fchmod and fchown
Fchmod() implemenets ceph_fchmod()
Fchown() implements ceph_fchown()
To fix: https://github.com/ceph/go-ceph/issues/240

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-07 09:43:05 +02:00
muagarwa
fa7300fff8 cephfs: Validate mount.mount usage in cephfs/path.go
ceph MountInfo pointer is being passed in the functions without a validation check, if it is nil there may be a crash. Fixed the functions to first validate mount.mount

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-06 10:04:43 -04:00
muagarwa
9e43e5119d cephfs: adding Link, Symlink and Readlink functions
Link function implements ceph_link().
Symlink function implements ceph_symlink().
Readlink function implements ceph_readlink().
To fix https://github.com/ceph/go-ceph/issues/218

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-04-30 13:28:51 -04:00
John Mulligan
fde15e439e cephfs: add DType type and getter function for directory entries
CephFS returns Linux-style struct dirent that has a d_type field.
It appears to properly make use of the field. This change exposes
the field using a DType type and associated constants as well
as a DType() function for the DirEntry.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-30 12:54:37 -04:00
John Mulligan
7b2e18b5e1 cephfs: reject negative offsets for ReadAt/WriteAt
Negative offsets that are passed through ReadAt/WriteAt to the ceph
calls cause them to behave like Read/Write and this is undesirable
at the go layer. Additionally, the os packages' file functions
reject negative offsets as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
f7936e4ac4 cephfs: add a test to ensure behavior of mixed use of Read + ReadAt
Add a test case to ensure the behavior of a mixed used [1] of Read
and ReadAt calls is consistent and ReadAt doesn't side effect
the file position used by Read.

1 - Not recommended, but we should not break now or in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
938506e1d3 cephfs: have Read/ReadAt return io.EOF error when nothing is read
Have Read and ReadAt functions return io.EOF when nothing is read
from the file so that it matches the current behavior of file types
in Go standard library.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
1899072b27 cephfs: add additional error handling and state checks to file funcs
Now, the close function is idempotent wrt being called multiple times
on a valid file. Additionally, check the state of the file in various
functions to produce sensible errors, matching those cephfs itself
returns, if the object was not constructed properly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
c37ec2104a cephfs: add an Unlink func implmenting ceph_unlink
The unlink function can now be used to clean up files from the other
existing test cases.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan
854a834b1c cephfs: implement file IO functions for open/close/read/write/seek
Implement core file I/O functions based on a file handle wrapper.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
Sven Anderson
e8da761769 update various parts of the code to use the retry lib.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
John Mulligan
5aadca02bf cephfs: split basic permissions related funcs into new files
Continue organizing the cephfs functionality by creating new files
for chmod and chown functionality.
Similarly, the dedicated test functions for those items are moved
into a new file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
bbf5fecf90 cephfs: split basic path management funcs into new files
Continue organizing the cephfs functionality by creating new files
for the most basic path management functions:
  MakeDir, RemoveDir, ChangeDir, CurrentDir
Similarly, the dedicated test functions for those items are moved into
a new file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
ce64ef536a cephfs: split (mds) command functions and tests into new files
Start organizing some of the cephfs functionality into functions grouped
by files by moving the MDS command functions into a new file. Move the
tests dedicated to those functions into a new matching file too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-17 13:40:01 -04:00
John Mulligan
29c88f4b2d cephfs: add functions for Init and SetMountPerms
The Init functions initializes the connection without mounting the fs.
This function must be called before setting the perms but before
creating the mount.

SetMountPerms accepts a UserPerm object to set the fs ownership info
on the mount. The corresponding tests verifies that the UserPerm
can be applied and it effects the gid of newly created dirs for that
mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
e1bbd45b5c cephfs: add a UserPerm type wrapping libcephfs UserPerm
The UserPerm type will be used later to set the owner credentials on the
mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
673c60bc51 cephfs: have test cases remove dirs they create
To prevent test failures re-reunning the suite against the same fs we
need to make sure tests clean up after themselves.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-15 14:22:30 +02:00
John Mulligan
9f6d4d48bb cephfs: add a RewindDir and List function for simpler dir listings
The RewindDir is used by List and can be used independently. It
resets the open dir to the "start".
The List function and the accompanying DirEntries type allow one
to simply grab all the dir entries at once. The Names method on the
DirEntries type allows the user to get the most commonly wanted items
from the dir entries in one shot.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
fd275e2b9a cephfs: add ReadDir implementing ceph_readdir_r function
Adds a method to ReadDir that implements ceph_readdir_r returning
a single dir entry at at time.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
e98648f76f cephfs: add Directory type and funcs to open and close it
Add the Directory type and the OpenDir function to get an open
Directory and a Close function to close/free it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-09 11:18:08 -04:00
John Mulligan
f584aef60f cephfs: consolidate error types & values in an errors.go file
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 17:10:09 -04:00
John Mulligan
2ad8361692 cephfs: make the Release function more idempotent to callers
Previously, calling Release more than once for the same MountInfo would
abort due to a double free in the ceph libs. As this is somewhat user
hostile we add some simple state tracking in our wrapper type such that
one can make redundant calls to Release without crashing the
application.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
4277d40137 cephfs: remove directory created in the ChangeDir test
The test function that exercises ChangeDir was creating a directory
and leaving it behind. Add some basic cleanup to the test to try
and avoid leaking stuff between test runs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
36196c5e60 cephfs: release fs resources in remaining test cases
In a previous commit we added fsDisconnect to match with fsConnect.  For
tests that do not use fsConnect but still acquire resources we need to
free up those resources with direct calls to the appropriate functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
1842ad248d cephfs: pair fsConnect with a fsDisconnect function
For every (most?) call to fsConnect we should pair it with a (deferred)
call to fsDisconnect to close connection and release resources
associated with the mount object we have created for the test.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
bdee9f4aab cephfs: TestMountWithRoot should only try unmounting after mounting
The defer function in TestMountWithRoot was always trying to call
unmount before release. This is not correct if the fs was not mounted
so we reorganize this code to more closely match what should be done
in normal practice.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
6a08124775 cephfs: TestCreateMount should not try to mount twice
Somehow we managed to introduce extra code into TestCreateMount that
had it create two mounts. This is undesirable for this test so this
change eliminates the extra unwanted invocation.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
ab178bc7ed cephfs: allow passing mds name into test through env var
Previously, the mds name was hardcoded in the tests to match that set up
by "micro-osd.sh". We continue to default to that value, but allow the
test to be supplied a custom value by the environment.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
0b658baebf cephfs: restrict running of tests that need a mounted fs instance
Add a utility function to test cases that expect the cephfs file system
to be mounted locally and accessible.
Add environment vars to configure the location of the mount point as
well whether the function causes the test to be skipped (the default)
or force an early but clear failure.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-06 16:29:26 -04:00
John Mulligan
2e5a2c3b9e cephfs: temporarily disable revive caring about CephFSError stuttering
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-17 09:56:49 -04:00
John Mulligan
6b396f70b2 errutil: move errutil pkg into internal subdir
Moving the errutil pkg to "internal/errutil" makes errutil private-like
and accessible to only other go-ceph packages. The functions it provided
were always meant to be used only by go-ceph, this just makes it more
official. This is a breaking change but it was only available to
outside users for 1 release and it is somewhat doubtful users outside
of go-ceph would have reached for these functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-10 08:41:44 +01:00
John Mulligan
33422fc2cb cephfs: add SetConfigOption and GetConfigOption functions
These are implemented similarly to the rados equivalents.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-10 08:15:16 +01:00
John Mulligan
fa5bb0fd69 cephfs: add GetFsCid to implement ceph_get_fs_cid
This function can be used to uniquely id the file system that the
mount object is connecting to.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-24 10:18:48 +01:00
John Mulligan
8948b20f5a cephfs: add MountWithRoot to implement the root arg to ceph_mount
The existing Mount() method did not provide a way to access the root
path argument to ceph_mount. This new function allows the caller to
specify an alternate root dir for the mount.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-24 10:18:48 +01:00
John Mulligan
522142665b cephfs: improve doc comments for mount, unmount, release funcs
Try to improve the doc comments for clarity and less redundancy.
Add "implements" subsections for the three functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-17 08:22:04 +01:00
John Mulligan
0c1b895566 cephfs: add doc comments to error type and func
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-17 08:22:04 +01:00
John Mulligan
2240c56614 cephfs: update TestCreateMountWithId to check custom id is used
Now that we have mds command function, we can use it to check for the id
we passed to CreateMountWithId appears in the session list.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 14:01:33 +01:00
John Mulligan
34f1507992 cephfs: add wrapper funcs for ceph_mds_command
Adding a wrapper for ceph_mds_command allows the client to make
queries against the mds server.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-02-11 14:01:33 +01:00
John Mulligan
15a55c9000 cephfs: add a doc.go file documenting the package
The minimal doc.go file matches the style currently used for rados and
rbd.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-01-28 16:10:39 +01:00
John Mulligan
c433abf787 cephfs: allow creating mount with an id
The ceph api function supports passing a string to identify the
client doing the mount. Expose that aspect of the api with a
new function CreateMountWithId.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-01-28 08:52:51 +01:00
John Mulligan
296c551613 cephfs: test creating a cephfs mount from a rados Conn
Verify the cephfs CreateFromRados function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-01-27 18:30:10 +01:00
John Mulligan
9929f63b13 cephfs: wrap ceph_create_from_rados function
Support creating a cephfs 'mount' from an existing rados connection
using the ceph_create_from_rados function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-01-27 18:30:10 +01:00
John Mulligan
b177121473 cephfs: replace tests mount setup boilerplate with func
In the cephfs tests the setup of the mount/connection was largely being
done as a copy-n-paste. This change adds a common setup function for
most of the test cases that need an active connection.
Similar to other recent changes this also adds a timeout such that
if the test fails to get a connection it will fail quickly rather
than block forever.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-01-20 16:52:45 +01:00
John Mulligan
4589af9dfd cephfs: fix linting errors found in cephfs_test.go
Now that cephfs_test.go is part of the cephfs package, the linter is
checking it and finding errors. Fix them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
John Mulligan
73753d5482 cephfs: test error type and error handling functions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
John Mulligan
9f46b4621b cephfs: include cephfs_test.go in cephfs package
This will allow us to test private functions from the cephfs package
int our test file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
John Mulligan
b6ad17881a cephfs: convert cephfs error type to exported type
Convert cephfs packages previously unexposed type to a public one and
replace a bunch of the existing boilerplate error handling with a
convenience function similar to those found ind rbd and rados.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
John Mulligan
72ea0669c2 cephfs: use common error extraction function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-16 16:59:25 +01:00
John Mulligan
ce5d3ef04d cephfs: remove use of logging in package
After discussion we decided to entirely drop logging from the library.
Future users should rely instead of the error types and values the
function calls return.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-12-13 15:17:27 +01:00
Sven Anderson
472f6dd5bb Tests: avoid segmentation faults in tests
This change replaces a couple of functions from the "assert" package
with the corresponding functions from the "require", where the
following code relies on a non-nil object.

Signed-off-by: Sven Anderson <sven@redhat.com>
2019-12-06 11:12:34 -05:00
Lincoln Thurlow
5a74b46a87 cephfs: modified guard clauses + golint'd 2018-10-11 11:36:26 -07:00
Lincoln Thurlow
ecf9a99249 cephfs: add unmount, release, chmod, chown, etc
This commit adds the following cephfs functions:

* Unmount // Unmounting is necessary to cleanup mounts
* Release // Release destroys the cmount ~ end of transaction
* RemoveDir // inverse of MakeDir
* Chown // change ownership of file or directory
* Chmod // change permissions of file or directory

Tests are included for each function.

In addition to these changes modifications to:

.travis.yml, Dockerfile, and Makefile

were made to accomodate tests to mount the ceph volume.  Tests use
fuse to mount the volume which requires adding:

--device /dev/fuse --cap-add SYS_ADMIN --security-opt \
apparmor:unconfined

to the docker container (alternatively --privileged works but adds
additional permissions).

Changes to README add the above docker changes as well as point
users to the necessary ceph development libraries.
2018-10-11 11:09:16 -07:00
Lincoln Thurlow
f3df337fab cephfs: add cephfs commands, expand logging
* CephError uses syscall for string version of error
* Add error logging for every function
* Add RemoveDir() function
* Add Unmount() function
* Add Release() function
* Add Chown() function
* Add Chmod() function
2018-10-11 11:09:16 -07:00
Noah Watkins
3f52ea219a repo: update new repo path
noahdesu -> ceph

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2015-08-31 07:34:42 -07:00
Abhishek Lekshmanan
d63c6698c2 gofmt all the source files
Running gofmt ./... on the repository

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2015-07-08 14:04:54 +05:30
Noah Watkins
0f4cf26919 cephfs: add mkdir
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2015-05-01 12:57:07 -07:00
Noah Watkins
1efdf980f0 cephfs: begin cephfs wrappers
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2015-05-01 12:41:47 -07:00