Commit Graph

929 Commits

Author SHA1 Message Date
John Mulligan
b001f1a4d2 cephfs admin: refactor response handling with a dedicated response type
The response type simplifies the error handling as a "pipeline" of
checks that are applied one after another.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
07722a0111 cephfs admin: add basic functionality for clone support
I think I'm a clone now

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
Sven Anderson
beb3351f24 callbacks: add a benchmark test case
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-10-01 10:03:41 -04:00
John Mulligan
e3c0d271dd cephfs admin: add subvolume group snapshot create, remove, and list funcs
Add CreateSubVolumeGroupSnapshot function.
Add RemoveSubVolumeGroupSnapshot function.
Add ListSubVolumeGroupSnapshots function.
Add tests for the above.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00
John Mulligan
1fec20be83 cephfs admin: add SubVolumeSnapshotInfo function for octopus
Add the new to octopus call for subvolume snapshot info.  Two fields
appear to return strings "yes"/"no", so could possibly have been
converted to booleans. In the chance that other string values get
returned and simplicity I left them as strings. If we need to, we can add
convenience funcs to return bools later.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00
John Mulligan
be55eb32ee cephfs admin: add subvolume snapshot create, remove, and list funcs
Add CreateSubVolumeSnapshot function.
Add RemoveSubVolumeSnapshot function.
Add ListSubVolumeSnapshots function.
Add tests for the above.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00
John Mulligan
3e81d128e6 cephfs admin: add a SubVolumeInfo function
Add a SubVolumeInfo function that works like the `ceph fs subvolume
info` command. This function returns a SubVolumeInfo type from parsing
the response JSON.

In order to re-use the existing QuotaSize type, which is an interface
and thus can't be unmarshaled directly, we use a non-exported wrapper
type for the unmarshaling and then we fix up the BytesQuota field so
that users of the library don't need to directly deal with the quota
size placeholder type, which can unmarshal the value.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan
ed23d71ff8 cephfs admin: add TimeStamp type for JSON unmarshaling
The date & time format used by ceph differs from the default used by Go
for time.Time types. Define a new type, embedding time.Time, that can
parse and expressing date+time strings the ceph way.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan
cdc0264a6b cephfs admin: add quotaSizePlaceholder type for JSON unmarshaling
Add a new quotaSizePlaceholder type which is a non-interface type that
can unmarshal JSON with a varying type into one of our types that meets
the QuotaSize interface.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan
d8143f5bae cephfs admin: rename "NewSize" type to "QuotaSize"
This clarifies what the size values are for when this interface is used
and hopefully clarifies what the size actually does to a subvolume. This
also generalizes the type a bit in preparation for it being used in
other upcoming functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan
fbb4f957eb cephfs admin: add a SubVolumeGroupPath function
The SubVolumeGroupPath function maps the subvolumegroup to a path from
the root of the cephfs, and works like `ceph fs subvolumegroup getpath
...`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-25 13:39:31 -04:00
John Mulligan
b66dcaf565 cephfs admin: add a SubVolumePath function
The SubVolumePath function maps the subvolume to a path from the root
of the cephfs, and works like `ceph fs subvolume getpath ...`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-25 13:39:31 -04:00
John Mulligan
c4e1292da1 cephfs admin: add EnumerateVolumes function
The EnumerateVolumes function returns a list of Name & ID pairs,
to identify the file systems in the cluster. The enumeration is based
on `ceph fs dump` like JSON, but for implementation only. The dump
feature itself is not currently exported in a raw form.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-24 15:50:51 -04:00
John Mulligan
3afac0b0b8 cephfs admin: add support for running MON commands
Some of the queries we need to make are not mgr commands but (still?)
mon commands. Extend the interface and helper funcs to support issuing
mon commands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-24 15:50:51 -04:00
Mudit Agarwal
736858ca7d rbd: add wrapper for rbd_snap_set_by_id()
Added wrapper for rbd_snap_set_by_id() function

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-09-24 14:03:01 -04:00
Mudit Agarwal
2f0905b585 rbd: add wrapper for rbd_snap_get_timestamp()
Added a wrapper for rbd_snap_get_timestamp() function.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-09-24 14:03:01 -04:00
Mudit Agarwal
0e09cd4370 rbd: add snapshot ID related functions
added wrappers for rbd_snap_get_name() and rbd_snap_get_id()

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-09-24 14:03:01 -04:00
Mudit Agarwal
34cab3407f rbd: add a new version of GetParentInfo()
GetParentInfo() currently requires the caller to specify the
string length for various attributes it returns. If the
specified string length is less than required, it can return
ERANGE error. Replacing it with a new function which avoids
this.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-09-21 11:19:38 -04:00
John Mulligan
c362706640 cephfs admin: add "Similar To" doc comments
Removes some old reminder comments.
Adds doc comments with the keyword "Similar To" followed by block quoted
text that shows a cli command the function will behave similarly to.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
7d18596d9d cephfs admin: use strconv to parse env var content
The environment variable to turn on tracing will now be parsed using
strconv.ParseBool rather than just a quick-n-dirty str compare.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
d327bf4f3e cephfs admin: move volume status to octopus specific files
The volume status command does not appear to be able to return JSON
on nautilus. Disable it on anything other than octopus for now.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
c1ff3a6b18 cephfs admin: in the test setup code, delay a bit to fetch mgr map
When the ceph client code has not yet (asynchronously) gotten the mgr
map, it can return -EACCES when we run the MgrCommand function.  We
before continuing with the test function to get the fsadmin we delay a
bit so that we are more likely to have the mgr map before continuing.
This change also fixes the caching of the fs admin.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
f9e1ec5d2a cephfs admin: disable package on 'luminous' and 'mimic'
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
caac65de71 cephfs admin: add ResizeSubVolume and supporting types
Add a function for resizing a subvolume as well as types needed to
create the JSON command and parse the response JSON. The newSize
argument can be passed a numeric value (as a ByteCount) or the special
constant `Infinite` much like the ceph cli command. This is accomplished
using the interface type `NewSize`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
a120c62623 cephfs admin: add optional debug tracing to the tests
Allow the environment variable GO_CEPH_TEST_DEBUG_TRACE=yes to enable
tracing of the input & output json from the manager commands. This is
both useful for debugging the tests themselves and demonstrates the use
of something other than rados.Conn meeting the interface required by
FSAdmin.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
272d34a9dd cephfs admin: parse pools info from volume status func
Add type to parse the pool info from the volume status response.
Use new general response unmarshal function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
4e5424dbfb cephfs admin: make a common func for unmarshaling json responses
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
1e4c2d09ae makefile: add new package to makefile and entrypoint.sh
Integrate the new 'cephfs/admin' package with existing makefile rules
and entrypoint.sh for testing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
fd10d9b756 cephfs admin: add functions to create, list, & remove subvolumes
Adds functions and tests for:
* CreateSubVolume
* ListSubVolumes
* RemoveSubVolume

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
5e6eec6a1a cephfs admin: add functions to create, list, & delete subvolumegroups
Adds functions and tests for:
* CreateSubVolumeGroup
* ListSubVolumeGroups
* RemoveSubVolumeGroup

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
198541cccd cephfs admin: add core functionality and volume functions
Adds the central admin type `FSAdmin` and some helper functions.
Adds basic volume list and info functions.
Adds tests for the volume functions and common fsadmin functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
ea1073dc1d cephfs admin: start a package for convenient administration of cephfs
These functions are not part of the cephfs APIs but will be based on the
various command-json features of the 'ceph' command line tool.
This change only adds a doc.go to stub in the package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00
John Mulligan
c845565e1b readme: clarify why we can't make static binaries
Make the statemant about not being able to statically link a less
general statement. It's not just cgo, we also only have shared objects
from ceph lib{rados,rbd,cephfs}.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
f88b87eaab readme: "Development" sub-title was at the wrong level
All other titles were at level two. Make "Development" consistent.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
4ec7a91da8 readme: move the scattershot api hints to a separate doc
There were various random "documentation" tidbits, all for rados,
on the main readme. As these are neither comprehensive nor complete
I am moving them to a seperate page and calling them "hints" which
seem to be closer to what they are.

Eventually, we could include more comprehensive examples in the
docs/ directory.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
a1641d2838 readme: note that code using go-ceph needs go dynamic libs
Add a caveat that go-ceph requires the ceph dynamically linked libraries
at runtime and that static linking is not available.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
c6113f79f9 readme: improve flow of the Installation section
Emphasize that go-ceph is a library and that the ceph devel packages are
build-time dependencies.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
07243682f2 readme: add brief introduction to what go-ceph is
Like a lot of projects on github the readme didn't really explain what
go-ceph *is*. There is a very brief project description and if you're
clever you may know what "bindings" are. This short introduction should
help clarify and expand that for any newcomers to the project.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
db01338d4a readme: remove irrelevant "badge" from README.md
This "badge" was making use of travis ci. We dropped use of travis ci
months ago. I am opting not to replace the badge with something new
because I don't think badges are useful in the first place.
But I don't want to rock the boat too much and remove all of them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 13:44:24 -04:00
John Mulligan
b24fa281df cephfs: clarify the behavior of the EOF case and short read in Preadv
Extend the Preadv/Pwritev test cases by adding assertions for the
behavior of the call in the end-of-file condition, and add another
test case to clarify the behavior in an explicit short-read test.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 10:07:34 -04:00
John Mulligan
f3d14f90b3 cephfs: change the permissions of open files in file_test.go
As pointed out in a review of Pwritev/Preadv functions, someone could
be using the tests as example code and not realize the "wide open"
permissions used in the test. Because it doesn't actually impact the
behavior of the test there's no reason not use use more restricted
permissions.
This change makes the older functions in the file use the restricted
perms too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 10:07:34 -04:00
John Mulligan
9c1ed2dfc2 cephfs: implement Preadv and Pwritev File methods
Add Preadv implementing ceph_preadv.
Add Pwritev implementing ceph_pwritev.
These calls act similarly to ReadAt and WriteAt but take multiple
buffers at once.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 10:07:34 -04:00
John Mulligan
ebea82dda5 cutil: add Iovec type wrapping C struct iovec arrays
This type is useful for passing disparate buffers to be read or
written in a single call. Functions using this type exist in cephfs
and rbd. Currently this is needed for cephfs calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 10:07:34 -04:00
Sven Anderson
4706453428 rbd: remove unnecessary C wrapper functions
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-08-28 09:51:06 -04:00
John Mulligan
dbf20aea0a rbd: add Sparsify implementing rbd_sparsify
The sparsify function can be used to make an image sparse by replacing
runs of zeros with holes. This change adds a wrapper for the basic
rbd_sparsify function and accompanying tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-27 14:22:52 -04:00
Sven Anderson
176a163c3c callbacks: use uintptr instead of int as index
Since the index of the registered callbacks is basically only used as
void*/unsafe.Pointer, it makes more sense and code simpler, if
the index type is uintptr instead of a simple int.

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-08-27 10:26:12 -04:00
John Mulligan
9cd28289b5 testing: use supported golang version in ci container images
Bump the golang version to 1.14.7. Go 1.14 is now the older supported
version, with the recent release of Go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-25 09:32:07 -04:00
John Mulligan
d0f3a30c57 readme: add a support table for go-ceph <-> ceph versions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-25 09:00:56 -04:00
John Mulligan
d2d2af34b1 readme: another small wording tweak
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-25 09:00:56 -04:00
John Mulligan
76bb869f02 readme: switch example tag use to 'nautilus'
Switch the example of how to compile go-ceph with tags to refer to
nautilus in light of luminous and mimic being deprecated.
Small wording/grammar changes to try and make the sentence clearer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-25 09:00:56 -04:00