Commit Graph

103 Commits

Author SHA1 Message Date
Sven Anderson
f5c0b089f1 cephfs/admin: promote preview API for release v0.23.0
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-08-14 14:43:57 +00:00
John Mulligan
db97336f62 cephfs admin: add version check for reef
Reef is a valid target for testing fsadmin.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-08-02 15:26:51 +00:00
guoguangwu
448e4b7f5c cephfs: remove refs to deprecated io/ioutil
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-07-08 14:26:48 +00:00
Anoop C S
37b9a31fd0 cephfs/admin: Make subvolume snapshot metadata APIs stable
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-06-08 17:35:55 +00:00
Anoop C S
c2e03fe3bb cephfs/admin: Make subvolume metadata APIs stable
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-06-08 17:35:55 +00:00
John Mulligan
ad61049254 cephfs admin: deprecate the New function
The New function created a rados connection object but did not expose it
via a public api, making it impossible to deterministically sever / clean up
the connection. This commit assumes that the patch will be merged prior
to the go-ceph v0.21 release so that we can plan to remove the function
three versions later (v0.24).  I chose three versions so that it is one
release longer than our typical stabilization period as this function
has been around for a while.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-27 17:55:51 +00:00
Anoop C S
2eb895d54f cephfs/admin: Add tests for API fetching volume info
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-20 14:17:55 +00:00
Anoop C S
d9b77dfca6 cephfs/admin: Add API to fetch volume info
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-20 14:17:55 +00:00
Anoop C S
8bc6ef4fa5 cephfs/admin: Add tests for Subvolume{Group} pinning API
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-17 20:32:44 +00:00
Anoop C S
11361b05a8 cephfs/admin: Add Subvolume{Group} pinning API
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-17 20:32:44 +00:00
Sven Anderson
87041ee7d8 gofmt: reformat everything with new gofmt of go 1.19
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
John Mulligan
7accfa6b82 cephfs admin: update tests to handle more than one file system
If the ceph cluster we're testing against supports more then one
fs our tests shouldn't fail. Update the tests to ensure a cephfs
exists as expected but don't assume it's the only one (or that
it's the first one returned).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
Anoop C S
8b8e63ebde cephfs/admin: Remove ceph_pre_quincy tags
With an exemption to Pacific, necessary backports are already available
with recent Quincy releases.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-01-23 15:12:52 +00:00
John Mulligan
6f7879428f cephfs admin: add additional delay when module is enabled
There was continued flakiness when the tests that enabled the cephfs
mirroring module were enabled. This change adds another 200 ms delay
to hopefully cover the time that might occur as the mgr has a new
module enabled and restart. But unfortunately this is still a bit of
a wild guess.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-12-08 15:12:38 +00:00
John Mulligan
63370f4df6 cephfs admin: remove test skip in TestCancelClone
This will re-enable the test case on ceph quincy and pacific.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-12-08 15:12:38 +00:00
John Mulligan
6c62acb906 cephfs admin: make CloneStatus.GetFailure stable
As per the stabilization doc, this api is now stable for release 0.18.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-10-17 17:18:49 +00:00
John Mulligan
9f22ac412d cephfs admin: remove unnecessary import C lines
All the C functions needed by cephfs/admin package come from
one of the other go-ceph packages (rados, cephfs, etc) there's
no good reason to have this unused import in these files.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-08-03 14:35:14 +00:00
Anoop C S
62b4a7323d cephfs/admin: Replace 'unstested' with pre-release tags
Now that we have support for testing APIs from pre-release Ceph
versions lets convert the current 'ceph_ci_untested' tag into
corresponding pre-release version tags.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-08-02 14:38:54 +00:00
Anoop C S
31b208ff55 cephfs/admin: Add 'main' to expected Ceph versions
When run against Ceph built out of 'main' development branch our
test-suite fails to find corresponding code name. Therefore we
include 'main' also to the expected list of versions.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-08-02 12:38:57 +00:00
Anoop C S
db3a21808f cephfs/admin: Fix compile errors
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-06-30 08:58:01 +00:00
Sven Anderson
6c459dc039 CI: also disable TestCancelClone for pacific
The code causing this issue has been packported to pacific and blocks
the CI. We need to temporarily disable for pacific as well.

Signed-off-by: Sven Anderson <sven@redhat.com>
2022-06-29 13:57:41 +00:00
John Mulligan
d31582cf06 cephfs admin: remove deprecated EnableModule, DisableModule funcs
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-14 13:10:09 +00:00
Prasanna Kumar Kalever
ec40a6c6c3 cephfs: add subvolume snapshot metadata APIs
Exports below APIs:
SetSnapshotMetadata(),
GetSnapshotMetadata(),
RemoveSnapshotMetadata(),
ForceRemoveSnapshotMetadata() and
ListSnapshotMetadata()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-13 15:16:36 +00:00
Prasanna Kumar Kalever
7d224905ca cephfs: add subvolume metadata APIs
Exports below APIs:
SetMetadata(),
GetMetadata(),
RemoveMetadata(),
ForceRemoveMetadata() and
ListMetadata()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-13 10:44:05 -04:00
Niels de Vos
e8c006b3af cephfs/admin: add CloneStatus.GetFailure()
In case cloning failed, Ceph Mgr returns an additional attribute in the
response. The "failed" attribute contains an errno (as string?!) and a
description. It is useful for applications using go-ceph to report the
error back to the user.

As the CloneStatus is on the same level as the CloneFailure object in
the response, keeping strictly to the CephFS Mgr API would break
compatibility. Instead, a new CloneStatus.GetFailure() method has been
introduced, returning a CloneFailure type (or nil) object with details
about the failure.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-06-09 19:47:04 +00:00
John Mulligan
e0369cdf88 cephfs admin: disable test TestCancelClone on quincy
This test feels very flaky, only passing 2-3 times out of ten (by
my vague guesstimate). This needs investigation.
However I don't want to block all of CI against quincy for this this
now.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
13bdb3e26c cephfs admin: remove explicit subvol delete from snap retention test
On ceph quincy, removing the subvolume explicitly (even with --force)
triggered an error. Due to the discussion in
https://tracker.ceph.com/issues/54625
I was informed that the explicit delete was not necessary, and while
the error on quincy was not correct, even on previous versions the
call is basically a no-op.

Rather than wait for this to be changed back to a no-op on quincy,
we remove the unneeded call. While we're at it, change the polling
loop to use the assert lib's Eventually call which is much nicer
to read and expires based on a duration avoiding the need to
think about loop iterations. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
ae44f69791 cephfs admin: flexibly decode the MDSVersion field in volume status
It appears the JSON being returned from `ceph fs status` has changed.
The field returning the mds version was previously "just" a string.
Now its a more complex thing. This of course breaks the old version
of the go code which is statically typed to expect a string.

In order not to break backwards compatibility we replace the object
used for parsing the JSON with a private object that has a custom
unmarshaller function for the mds version field. If it sees the string
it uses the string. Otherwise, it looks for this new structure and,
if possible, extracts the version string from the first item.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
ac1401b4e2 cephfs admin: add quincy to valid server editions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
c4cfd15e12 cephfs admin: increase timeout waiting for mirroring module
Again.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-02-08 15:55:02 +00:00
John Mulligan
f705d41c96 cephfs admin: mark general mgr module funcs as deprecated
The versions in cluster/admin/manager should be used instead.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-02-08 15:55:02 +00:00
John Mulligan
cc1cb6ed24 cephfs admin: use the new admintest module to set up tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-02-08 15:55:02 +00:00
John Mulligan
85208d213f cephfs admin: reimplement cephfs mgr module calls via common admin
Now that we've moved the logic over to common admin, replace the
implementation in cephfs admin.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-02-08 15:55:02 +00:00
John Mulligan
1d9c3f8533 cephfs admin: poll mgr to see if mirroring module is enabled
When testing the mirroing workflow, we will not explicitly check
on the mgr for the presence of the mirroring module before proceeding
with the tests. The plan is to have this cut down on test flakes.

The previous workaround also had a bug in that the 2nd sleep was in the
wrong location. Perhaps, fixing this alone would resolve the test
flakes, but this is more proper anyway.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-12-07 18:00:56 +00:00
John Mulligan
a26d4b8563 cephfs admin: add a private function for getting mgr modules
Add a function for getting the lists of enabled/disabled mgr modules
from ceph. This is private for the moment as general module management
probably shouldn't be part of cephfs admin. It's bad enough we hacked
in the functions for enabling mirroring. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-12-07 18:00:56 +00:00
John Mulligan
e79f1d786f cephfs: remove luminous and mimic build tags
Luminous and mimic have not been supported for a few releases now.
There's no need to keep these build tags any more.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-11-09 01:24:01 +00:00
John Mulligan
c0852d43d0 cephfs admin: add sleeps after mgr module is enabled (workaround)
This is a workaround for the common test flake:
```
=== RUN   TestMirroring
    mirror_workflow_test.go:63:
            Error Trace:    mirror_workflow_test.go:63
            Error:          Received unexpected error:
                            rados: ret=-95, Operation not supported:
"Module 'mirroring' is not enabled (required by command 'fs snapshot
mirror enable'): use `ceph mgr module enable mirroring` to enable it"
            Test:           TestMirroring
```
at even though we've certainly requested the mode be enabled the mgr
has apparently not fully processed the request and the module is
not yet available. We add a half-second sleep after each call
to hopefully give the mgr time enough to completely enable the module.

A future improvement would be to poll the mgr until we know it is ready to
use the cephfs mirroring module, but that would take some research and
(probably) new API calls. So this is a temporary workaround just
to try and reduce the number of test flakes sooner rather than later.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-11-01 15:45:03 +00:00
Sven Anderson
87f4563118 format: run gofmt on all files
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-10-04 18:58:35 +00:00
John Mulligan
bf6cd07204 cephfs admin: import path package as pathpkg
This avoids import shadowing elsewhere in workflow_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
53d742d7ce cephfs admin: tweak naming of DaemonStatusPeer
It was previously DaemonStatusPeerRemote but this struck the reviewers
as redundant so it is now simply DaemonStatusPeer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-05 13:53:22 +00:00
John Mulligan
9eaff23200 cephfs admin: add a full workflow test for cephfs mirroring
This rather large test case tests the mirroring functions in an
end-to-end manner. It sets up the mirroring peers, enables mirroring on
directories and verifies that mirroring was set up by writing data and
verifying that data was propagated from site a to site b.

Ceph pacific is required to use cephfs mirroring and thus run this test.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
5ab987da0a cephfs admin: add data parsing tests for cephfs mirroring functions
Add test cases that exercise the parsing of data returned by ceph for
the informational calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
ed5fad41e6 cephfs admin: add informational functions for cephfs mirroring
Add the functions DaemonStatus and PeerList which provide information
about the cephfs mirroring daemon and mirroring peers.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
d847769649 cephfs admin: add core functions needed to work with cephfs mirroring
These functions require support in the ceph mgr and pertain to setting
up mirroring in the file system, adding directories to mirror, and
setting up mirroring peers.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
2e8b52be8a cephfs admin: add EnableMirroringModule and DisableMirroringModule funcs
Add functions EnableMirroringModule and DisableMirroringModule for
turning on an off the mirroring module at the ceph mgr.
This also adds the lower level EnableModule and DisableModule that take
a string parameter naming the module, and are the base for
EnableMirroringModule and DisableMirroringModule.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
e56f10f3b4 cephfs admin: make fsConnect utility function more flexible
The fsConnect function is used by the tests to support getting a
data-path cephfs connection, in order to support those fs admin tests
that need to make changes in the file system directly. The new second
argument to the call allows for the cephfs connection to be made with a
non-default configuration.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
df3c6dae2e cephfs admin: add newFSAdmin func for setting up fsadmin test instances
Previously, the getFSAdmin function always fetched a cached (if
available) fsadmin instance set up for the default configuration of
ceph. This change makes getFSAdmin based on newFSAdmin a function
that always returns a new instance and allows a custom configuration
to be passed.

Otherwise, newFSAdmin works like getFSAdmin did previously, supporting
debugging via an environment variable.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-03 07:25:28 +00:00
John Mulligan
bbd88f0b85 cephfs admin: swap from internal package to common packge for interfaces
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-06-01 11:06:19 +00:00
John Mulligan
1f02b52df2 cephfs admin: update test helper functions to work on ceph "pacific"
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-30 15:23:40 +02:00
John Mulligan
dfa74510aa cephfs admin: replace mon and mgr command helpers with common versions
Remove the logic previously ported over to internal/commands and just
leave the functions in place as to not have to excessively touch the
existing code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00