Commit Graph

151 Commits

Author SHA1 Message Date
John Mulligan f88a2b13f9 cephfs: remove useMount test helper function
Now that there are no tests using the fuse mount point and the useMount
function that detects an external mount point is needed, we can remove
the useMount function itself and the env var used to control it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan 57a37c6c74 cephfs: remove uses of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan f0142920b5 cephfs: remove uses of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan df9834fc82 cephfs: remove uses of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan 3b9317b608 cephfs: remove a use of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
Sven Anderson 527553718c cephfs: add a missing userMount() to tests
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-03-05 04:30:30 +00:00
Sven Anderson b8a803ccbb cutil: use SyncBuffer for iovec type
The main motivation for PtrGuard was read and write buffers as they
are used in iovec.  This change uses SyncBuffer for the iovec
implementation, so that the no-copy PtrGuard implementation can be
enabled with the with_ptrguard build tag.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Mudit Agarwal 1e15c36ec9 cephfs: add state field to SubVolumeInfo structure.
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-02 15:29:53 +00:00
John Mulligan 5130f55387 cephfs admin: fix the json sent when flag RetainSnapshots is set
The json key sent to retain snapshots on subvolume remove was misspelled
and had a dash in it rather than an underscore. Fix the incorrect key
and add a test, that needs to be able to run on version of cephfs w/o
snapshot retention, and verifies that flag does the correct thing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-01 08:33:28 +00:00
John Mulligan c995d63088 cephfs: enable test cases with ceph issues that have been fixed
Ceph issue https://tracker.ceph.com/issues/46084 has been fixed with
backports to nautilus and octopus. Re-enable the relevant test data
and fix up the tests to match.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-19 08:23:52 +00:00
John Mulligan 6900146fc4 cephfs admin: fix reversed args in greater-or-equal assertion
Now that the new year has arrived this test assertion has started to
fail. This is because, although I anticipated the year would eventually
change and used GreaterOrEqual, the argument order is wrong. My excuse
is that I find the argument order in testify's Equal function confusing,
it causes me to assume other functions in testify are also confusing
(even when it's the "normal" order) and/or I copy and pasted from an
Equal line and didn't change the args (both are true!).

Instead of allowing any old year, we assume that the clocks where the
test are run are in sync enough that the current year is correct for
the assertion (thanks for the suggestion Sven).
Happy new year, go-ceph!

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-05 13:30:50 +00:00
John Mulligan 8fc7765cfe cephfs admin: add test case for new RemoveSubVolumWithFlags function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan e958f299ec cephfs admin: create a custom flags type for subvolume removal
This flagSet type allows the user to also set the value of the retain
snapshots flag, something unique to subvolume removal (all others had
the force flag only).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan 7524b39272 cephfs admin: add flagSet interface and mergeFlags function
To prepare for multiple flags types, we create a flagSet interface and
break out the functionality that was in Update into a per-type method
and then the common "merger" function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan 98afaf6376 cephfs admin: rename rmFlags type to commonRmFlags
This is to clarify that for "normal" cases the private commonRmFlags
type is correct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan 2f1ee70242 cephfs admin: move rmFlags type to a file for flags handling
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan 36abb07dc5 cephfs: add a test case for calling Statx on an invalid mount
Adding this test case gets test coverage to 100% for path.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
John Mulligan 7cbacf9331 cephfs: add Truncate implementing ceph_truncate
Add a Truncate method that operates on paths.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
John Mulligan e2de2b6e58 cephfs: add File method Truncate implementing ceph_ftruncate
Add a Truncate method to File type. Note the doc comment as I hit
a bug in ceph when writing the first test cases. Go-ceph will behave
as ceph here - so using go-ceph with an unfixed version of the
ceph libs will also exhibit the bug.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
John Mulligan 11b6c916ed cephfs admin: fix issues with TestWorkflow
Due to a new ceph nautilus the workflow test was breaking because the
existing code failed to anticipate the "subvolume uuid" in the path
to the snapshot data. This is the curse of writing code without anything
to test it against. I also add a very short delay just to give us
a chance to avoid issues in the future in case the snapshot creation
is not quite synchronous.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-27 10:06:16 -04:00
John Mulligan 55821a9300 cephfs admin: remove subvolumegroup snapshot support
Apparently, this feature is being removed from all active ceph versions
and will not be present in future versions. Since we won't be able to
test it any more, and the api is provisional still, we just remove it
entirely.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-27 10:06:16 -04:00
John Mulligan c7adfc8637 cephfs admin: add a workflow test to check call behaviors
This doesn't touch every call in cephfs admin, but does try to act like
a real application, or set of applications would: reading and writing
data as well as using the api to create and manage subvolumes,
snapshots, and clones.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-09 14:15:56 -04:00
John Mulligan 17cab5c47d cephfs admin: add support for a list of features in SubVolumeInfo
Some versions of ceph return a list of supported features for a
subvolume. On versions that don't return this field the list will always
be empty.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-09 11:20:06 -04:00
John Mulligan 135e624aa5 cephfs admin: add ListFileSystems function for listing fs with pools
For users who need to know the pools (by name) as well as the files
systems, we add ListFileSystems which is similar to 'ceph fs ls', rather
than the existing ListVolumes as that function is simlar to 'ceph fs
volume ls' and that does not return pool information.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-09 10:15:16 +02:00
John Mulligan c23e1a8659 cephfs admin: doc comment improvements for clone
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-07 08:54:34 -04:00
John Mulligan 0fa5143172 cephfs admin: move tests that were previously only built on octopus
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan 5dce7c9014 cephfs admin: return not-impl errors if ceph returns text for volume status
Return not implemented errors when "older" versions of ceph return
formatted text for output instead of JSON, even tho we asked for json.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan 5f64c52223 cephfs admin: add support for returning not-implemented errors
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan 462372c178 cephfs admin: add ability to check server version in tests
Rather than compile things in and out, which really should only be for
the client version of the ceph libs, add vars that tests can use to
check the server. Also, a "sentinel" test is added to make sure the
tests are being fed a version it understands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan a71ec050bf cephfs admin: expose all functions to all build tags >=natuilus
Previously, I was sort of abusing the build tags system to just get our
tests working. That was incorrect because you could be building go-ceph
against nautilus libs but using an octopus server and vice versa. In
addition, I've learned that some of the snapshot and clone behaviors are
being changed as items are backported in ceph. This means that build
tags themselves are not good indicators of what a particular version of
ceph supports.

This patch is the first step to removing these restrictions and making
cephfs admin work more smoothly across more versions of ceph.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan e14bd1e436 cephfs admin: add in a test for canceling clones
This test is added after we add force delete functions as deleting a
canceled clone requires a force remove function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan c9ab11ad12 cephfs admin: add ForceRemove<XYZ> variants for subv and group snaps
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan 6a020daeb7 cephfs admin: add ForceRemove<XYZ> variants for subvolumes and groups
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan dcc4f67f1d cephfs admin: have clone test work on older and newer ceph versions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan e7c7752393 cephfs admin: ignore deprecation warnings for protect/unprotect
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan 9a97852bf2 cephfs admin: add unit tests for response type
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan b001f1a4d2 cephfs admin: refactor response handling with a dedicated response type
The response type simplifies the error handling as a "pipeline" of
checks that are applied one after another.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan 07722a0111 cephfs admin: add basic functionality for clone support
I think I'm a clone now

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan e3c0d271dd cephfs admin: add subvolume group snapshot create, remove, and list funcs
Add CreateSubVolumeGroupSnapshot function.
Add RemoveSubVolumeGroupSnapshot function.
Add ListSubVolumeGroupSnapshots function.
Add tests for the above.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00
John Mulligan 1fec20be83 cephfs admin: add SubVolumeSnapshotInfo function for octopus
Add the new to octopus call for subvolume snapshot info.  Two fields
appear to return strings "yes"/"no", so could possibly have been
converted to booleans. In the chance that other string values get
returned and simplicity I left them as strings. If we need to, we can add
convenience funcs to return bools later.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00
John Mulligan be55eb32ee cephfs admin: add subvolume snapshot create, remove, and list funcs
Add CreateSubVolumeSnapshot function.
Add RemoveSubVolumeSnapshot function.
Add ListSubVolumeSnapshots function.
Add tests for the above.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00
John Mulligan 3e81d128e6 cephfs admin: add a SubVolumeInfo function
Add a SubVolumeInfo function that works like the `ceph fs subvolume
info` command. This function returns a SubVolumeInfo type from parsing
the response JSON.

In order to re-use the existing QuotaSize type, which is an interface
and thus can't be unmarshaled directly, we use a non-exported wrapper
type for the unmarshaling and then we fix up the BytesQuota field so
that users of the library don't need to directly deal with the quota
size placeholder type, which can unmarshal the value.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan ed23d71ff8 cephfs admin: add TimeStamp type for JSON unmarshaling
The date & time format used by ceph differs from the default used by Go
for time.Time types. Define a new type, embedding time.Time, that can
parse and expressing date+time strings the ceph way.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan cdc0264a6b cephfs admin: add quotaSizePlaceholder type for JSON unmarshaling
Add a new quotaSizePlaceholder type which is a non-interface type that
can unmarshal JSON with a varying type into one of our types that meets
the QuotaSize interface.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan d8143f5bae cephfs admin: rename "NewSize" type to "QuotaSize"
This clarifies what the size values are for when this interface is used
and hopefully clarifies what the size actually does to a subvolume. This
also generalizes the type a bit in preparation for it being used in
other upcoming functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:16:34 -04:00
John Mulligan fbb4f957eb cephfs admin: add a SubVolumeGroupPath function
The SubVolumeGroupPath function maps the subvolumegroup to a path from
the root of the cephfs, and works like `ceph fs subvolumegroup getpath
...`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-25 13:39:31 -04:00
John Mulligan b66dcaf565 cephfs admin: add a SubVolumePath function
The SubVolumePath function maps the subvolume to a path from the root
of the cephfs, and works like `ceph fs subvolume getpath ...`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-25 13:39:31 -04:00
John Mulligan c4e1292da1 cephfs admin: add EnumerateVolumes function
The EnumerateVolumes function returns a list of Name & ID pairs,
to identify the file systems in the cluster. The enumeration is based
on `ceph fs dump` like JSON, but for implementation only. The dump
feature itself is not currently exported in a raw form.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-24 15:50:51 -04:00
John Mulligan 3afac0b0b8 cephfs admin: add support for running MON commands
Some of the queries we need to make are not mgr commands but (still?)
mon commands. Extend the interface and helper funcs to support issuing
mon commands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-24 15:50:51 -04:00
John Mulligan c362706640 cephfs admin: add "Similar To" doc comments
Removes some old reminder comments.
Adds doc comments with the keyword "Similar To" followed by block quoted
text that shows a cli command the function will behave similarly to.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-31 10:38:38 -04:00