The json key sent to retain snapshots on subvolume remove was misspelled
and had a dash in it rather than an underscore. Fix the incorrect key
and add a test, that needs to be able to run on version of cephfs w/o
snapshot retention, and verifies that flag does the correct thing.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Ceph issue https://tracker.ceph.com/issues/46084 has been fixed with
backports to nautilus and octopus. Re-enable the relevant test data
and fix up the tests to match.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Now that the new year has arrived this test assertion has started to
fail. This is because, although I anticipated the year would eventually
change and used GreaterOrEqual, the argument order is wrong. My excuse
is that I find the argument order in testify's Equal function confusing,
it causes me to assume other functions in testify are also confusing
(even when it's the "normal" order) and/or I copy and pasted from an
Equal line and didn't change the args (both are true!).
Instead of allowing any old year, we assume that the clocks where the
test are run are in sync enough that the current year is correct for
the assertion (thanks for the suggestion Sven).
Happy new year, go-ceph!
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This flagSet type allows the user to also set the value of the retain
snapshots flag, something unique to subvolume removal (all others had
the force flag only).
Signed-off-by: John Mulligan <jmulligan@redhat.com>
To prepare for multiple flags types, we create a flagSet interface and
break out the functionality that was in Update into a per-type method
and then the common "merger" function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a Truncate method to File type. Note the doc comment as I hit
a bug in ceph when writing the first test cases. Go-ceph will behave
as ceph here - so using go-ceph with an unfixed version of the
ceph libs will also exhibit the bug.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Due to a new ceph nautilus the workflow test was breaking because the
existing code failed to anticipate the "subvolume uuid" in the path
to the snapshot data. This is the curse of writing code without anything
to test it against. I also add a very short delay just to give us
a chance to avoid issues in the future in case the snapshot creation
is not quite synchronous.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Apparently, this feature is being removed from all active ceph versions
and will not be present in future versions. Since we won't be able to
test it any more, and the api is provisional still, we just remove it
entirely.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This doesn't touch every call in cephfs admin, but does try to act like
a real application, or set of applications would: reading and writing
data as well as using the api to create and manage subvolumes,
snapshots, and clones.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Some versions of ceph return a list of supported features for a
subvolume. On versions that don't return this field the list will always
be empty.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
For users who need to know the pools (by name) as well as the files
systems, we add ListFileSystems which is similar to 'ceph fs ls', rather
than the existing ListVolumes as that function is simlar to 'ceph fs
volume ls' and that does not return pool information.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Return not implemented errors when "older" versions of ceph return
formatted text for output instead of JSON, even tho we asked for json.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Rather than compile things in and out, which really should only be for
the client version of the ceph libs, add vars that tests can use to
check the server. Also, a "sentinel" test is added to make sure the
tests are being fed a version it understands.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Previously, I was sort of abusing the build tags system to just get our
tests working. That was incorrect because you could be building go-ceph
against nautilus libs but using an octopus server and vice versa. In
addition, I've learned that some of the snapshot and clone behaviors are
being changed as items are backported in ceph. This means that build
tags themselves are not good indicators of what a particular version of
ceph supports.
This patch is the first step to removing these restrictions and making
cephfs admin work more smoothly across more versions of ceph.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This test is added after we add force delete functions as deleting a
canceled clone requires a force remove function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The response type simplifies the error handling as a "pipeline" of
checks that are applied one after another.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add the new to octopus call for subvolume snapshot info. Two fields
appear to return strings "yes"/"no", so could possibly have been
converted to booleans. In the chance that other string values get
returned and simplicity I left them as strings. If we need to, we can add
convenience funcs to return bools later.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a SubVolumeInfo function that works like the `ceph fs subvolume
info` command. This function returns a SubVolumeInfo type from parsing
the response JSON.
In order to re-use the existing QuotaSize type, which is an interface
and thus can't be unmarshaled directly, we use a non-exported wrapper
type for the unmarshaling and then we fix up the BytesQuota field so
that users of the library don't need to directly deal with the quota
size placeholder type, which can unmarshal the value.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The date & time format used by ceph differs from the default used by Go
for time.Time types. Define a new type, embedding time.Time, that can
parse and expressing date+time strings the ceph way.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a new quotaSizePlaceholder type which is a non-interface type that
can unmarshal JSON with a varying type into one of our types that meets
the QuotaSize interface.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This clarifies what the size values are for when this interface is used
and hopefully clarifies what the size actually does to a subvolume. This
also generalizes the type a bit in preparation for it being used in
other upcoming functions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The SubVolumeGroupPath function maps the subvolumegroup to a path from
the root of the cephfs, and works like `ceph fs subvolumegroup getpath
...`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The SubVolumePath function maps the subvolume to a path from the root
of the cephfs, and works like `ceph fs subvolume getpath ...`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The EnumerateVolumes function returns a list of Name & ID pairs,
to identify the file systems in the cluster. The enumeration is based
on `ceph fs dump` like JSON, but for implementation only. The dump
feature itself is not currently exported in a raw form.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Some of the queries we need to make are not mgr commands but (still?)
mon commands. Extend the interface and helper funcs to support issuing
mon commands.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Removes some old reminder comments.
Adds doc comments with the keyword "Similar To" followed by block quoted
text that shows a cli command the function will behave similarly to.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The environment variable to turn on tracing will now be parsed using
strconv.ParseBool rather than just a quick-n-dirty str compare.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The volume status command does not appear to be able to return JSON
on nautilus. Disable it on anything other than octopus for now.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
When the ceph client code has not yet (asynchronously) gotten the mgr
map, it can return -EACCES when we run the MgrCommand function. We
before continuing with the test function to get the fsadmin we delay a
bit so that we are more likely to have the mgr map before continuing.
This change also fixes the caching of the fs admin.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a function for resizing a subvolume as well as types needed to
create the JSON command and parse the response JSON. The newSize
argument can be passed a numeric value (as a ByteCount) or the special
constant `Infinite` much like the ceph cli command. This is accomplished
using the interface type `NewSize`.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Allow the environment variable GO_CEPH_TEST_DEBUG_TRACE=yes to enable
tracing of the input & output json from the manager commands. This is
both useful for debugging the tests themselves and demonstrates the use
of something other than rados.Conn meeting the interface required by
FSAdmin.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add type to parse the pool info from the volume status response.
Use new general response unmarshal function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Adds the central admin type `FSAdmin` and some helper functions.
Adds basic volume list and info functions.
Adds tests for the volume functions and common fsadmin functions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
These functions are not part of the cephfs APIs but will be based on the
various command-json features of the 'ceph' command line tool.
This change only adds a doc.go to stub in the package.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Extend the Preadv/Pwritev test cases by adding assertions for the
behavior of the call in the end-of-file condition, and add another
test case to clarify the behavior in an explicit short-read test.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
As pointed out in a review of Pwritev/Preadv functions, someone could
be using the tests as example code and not realize the "wide open"
permissions used in the test. Because it doesn't actually impact the
behavior of the test there's no reason not use use more restricted
permissions.
This change makes the older functions in the file use the restricted
perms too.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add Preadv implementing ceph_preadv.
Add Pwritev implementing ceph_pwritev.
These calls act similarly to ReadAt and WriteAt but take multiple
buffers at once.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The name "samples" was too generic. Rename it to "xattrSamples".
This variable will be used outside of this file in a future change.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Use the newly added SplitSparseBuffer cutil function to break the buffer
cephfs returns into a slice of strings with xattr names.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a section to the doc comment for SetXattr that notes the behavior
I observed while working on the tests: that passing an empty value
(or null on the C side) causes the xattr to be removed rather than
set with an "empty" value. This issue has been reported in the
ceph tracker (see link in comment).
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add SetXattr implementing ceph_fsetxattr.
Add GetXattr implementing ceph_fgetxattr.
Add test function to exercise both.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Similar to the rados function of the same name, ParseDefaultConfigEnv
uses the environment to configure the cephfs mount.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The ParseConfigArgv behaves similarly to the same named function in
rados. The command line argv value is parsed by ceph libs and used to
configure the mount object.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
In order to avoid external dependencies on implementation details,
this change replaces CephFSError with the unexported cephFSError. In case
some application really needs access to the integer value, it can use
the pattern
var errno interface{ Errno() int }
if errors.As(err, errno) { ... errno.Errno() ... }
Signed-off-by: Sven Anderson <sven@redhat.com>
There are functions from librbd that can use the Timespec type too.
Instead of consuming 'cephfs.Timespec' in the rbd package, create an
internal type that can be exposed through both packages.
This looks a bit like duplication, but it does not break the current
users of Timespec in the cephfs package.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add wrappers for ceph_fsync. The Fsync call directly wraps ceph_fsync
which provides options to behave more like fsync or more like fdatasync.
Add Sync, a simpler wrapper over Fsync, to match any interfaces that
make use of `File.Sync` from Go's os package.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Use the CephStatx type previously added to the library to implement a
wrapper for the ceph_fstatx function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add the ReadDirPlus function call, wrapping ceph_readdirplus_r. Add
DirEntryPlus which is a DirEntry plus a getter for the statx field.
Add a test similar to the ReadDir test and (since it is possible to
induce an error with ceph_readdirplus_r) a test for the error handling
path to improve code coverage.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The *_command functions in librados and libcephfs document the use
of specific free functions for data allocated. These functions are
currently just wrappers around C's free() function. However, to be
more strictly compliant this change adds a free-function callback
to the CommandOutput type and the specific free functions are now
used outside the unit tests.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Similar to the functions in the rados pkg, cephfs package has a function
MdsCommand that is mostly boilerplate that to set up the C function's
arguments. Replace it with the types from the new cutil internal
package.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a Statx wrapper for ceph_statx.
Add a type wrapping the statx status info that exposes the various
fields from the C-struct.
Add a type wrapping struct timespec, based on golang's x/sys, for the
time fields in the struct.
Note that the ceph struct is not the same as the linux statx struct.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
ceph MountInfo pointer is being passed in the functions without a validation check, if it is nil there may be a crash. Fixed the functions to first validate mount.mount
Signed-off-by: Mudit Agarwal muagarwa@redhat.com
Link function implements ceph_link().
Symlink function implements ceph_symlink().
Readlink function implements ceph_readlink().
To fix https://github.com/ceph/go-ceph/issues/218
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
CephFS returns Linux-style struct dirent that has a d_type field.
It appears to properly make use of the field. This change exposes
the field using a DType type and associated constants as well
as a DType() function for the DirEntry.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Negative offsets that are passed through ReadAt/WriteAt to the ceph
calls cause them to behave like Read/Write and this is undesirable
at the go layer. Additionally, the os packages' file functions
reject negative offsets as well.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a test case to ensure the behavior of a mixed used [1] of Read
and ReadAt calls is consistent and ReadAt doesn't side effect
the file position used by Read.
1 - Not recommended, but we should not break now or in the future.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Have Read and ReadAt functions return io.EOF when nothing is read
from the file so that it matches the current behavior of file types
in Go standard library.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Now, the close function is idempotent wrt being called multiple times
on a valid file. Additionally, check the state of the file in various
functions to produce sensible errors, matching those cephfs itself
returns, if the object was not constructed properly.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Continue organizing the cephfs functionality by creating new files
for chmod and chown functionality.
Similarly, the dedicated test functions for those items are moved
into a new file as well.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Continue organizing the cephfs functionality by creating new files
for the most basic path management functions:
MakeDir, RemoveDir, ChangeDir, CurrentDir
Similarly, the dedicated test functions for those items are moved into
a new file as well.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
