Commit Graph

725 Commits

Author SHA1 Message Date
Yang Honggang
f3e39224ec Makefile: fix "No such file or directory" error
$ make test-docker
grep: /sys/kernel/security/lsm: No such file or directory

Signed-off-by: Yang Honggang <yanghonggang@kuaishou.com>
2020-12-23 13:29:22 +00:00
John Mulligan
2870202d06 github: remove mergify rule checking for mimic as we have removed it
The test case is gone from the CI. Remove it from the merigify config
too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-16 15:18:33 +01:00
John Mulligan
574b3584fa readme: document v0.7.0 and the ceph versions it supports
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-14 09:54:45 -05:00
John Mulligan
4cf40e8e0b ci: remove "mimic" from testing matrix
Mimic is not a supported build target in v0.7 and as such there's no
need to keep testing it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-10 09:41:28 -05:00
John Mulligan
8fc7765cfe cephfs admin: add test case for new RemoveSubVolumWithFlags function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
e958f299ec cephfs admin: create a custom flags type for subvolume removal
This flagSet type allows the user to also set the value of the retain
snapshots flag, something unique to subvolume removal (all others had
the force flag only).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
7524b39272 cephfs admin: add flagSet interface and mergeFlags function
To prepare for multiple flags types, we create a flagSet interface and
break out the functionality that was in Update into a per-type method
and then the common "merger" function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
98afaf6376 cephfs admin: rename rmFlags type to commonRmFlags
This is to clarify that for "normal" cases the private commonRmFlags
type is correct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
2f1ee70242 cephfs admin: move rmFlags type to a file for flags handling
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
36abb07dc5 cephfs: add a test case for calling Statx on an invalid mount
Adding this test case gets test coverage to 100% for path.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
John Mulligan
7cbacf9331 cephfs: add Truncate implementing ceph_truncate
Add a Truncate method that operates on paths.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
John Mulligan
e2de2b6e58 cephfs: add File method Truncate implementing ceph_ftruncate
Add a Truncate method to File type. Note the doc comment as I hit
a bug in ceph when writing the first test cases. Go-ceph will behave
as ceph here - so using go-ceph with an unfixed version of the
ceph libs will also exhibit the bug.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
Niels de Vos
0afcf98b34 Add rbd_writesame()
In order to provide the ability to do thick provisioning of RBD images,
Image.WriteSame() can be used. The `rbd` commandline utility calls the
rbd_writesame() function when an image is created with the
`--thick-provision` option.

See-also: ceph/ceph-csi#1675
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-12 12:50:03 +00:00
Niels de Vos
460837e900 rados: add LIBRADOS_OP_FLAG_* constants
The LIBRADOS_OP_FLAG_* constants can be passed to rbd_writesame() and
other operations that will be added in the future.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-12 12:50:03 +00:00
Sven Anderson
c8f26a841b rbd: eliminate invalid unsafe.Pointers
Although it seems to be stable with the current Go versions,
unsafe.Pointers that are created by applying arithmetic on nil
pointers are explicitly not allowed[1], and could potentially lead to
panics by the garbage collector.  To be on the safe side, this change
goes back to minimal inline wrappers for the callback registrations,
which should be zero-cost.  For the callbacks themselves fortunately
no wrappers are required, because CGo happily converts void* to
uintptr arguments.  The potentially problematic VoidPtr helper is
removed again.

[1] https://golang.org/pkg/unsafe/#Pointer

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-11-09 08:58:22 +00:00
Sven Anderson
b0ffc1afe8 callbacks: replace index with unique ID
The former code assumed a LIFO usage, and otherwise always produced
collisions.  This change uses unique IDs instead, that as a
side-effect might make debugging easier.

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-11-05 17:16:32 +00:00
John Mulligan
7b1d735f38 workflows: move to setup-go@v2
We had been using setup-go@v2-beta for a while, before v2 was out.
Now v2-beta is out of data and generating warnings in the github
actions ui.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-04 10:02:24 -05:00
Niels de Vos
a5c777bf33 Use Smart-mode for "Strict Merge" to enable automatic rebase
Enabling Smart-mode for "Strict Merge" causes Mergify to do a rebase of
the PR before merging. The rebase will trigger CI runs again, and
merging will be done only when the CI results are successful.

See-also: https://docs.mergify.io/actions.html#merge
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 10:01:55 -05:00
John Mulligan
e7791c9e06 rbd: add ListMetadata implementing rbd_metadata_list
The function is named ListMetadata but returns a map of all
the metadata assigned to the rbd image. It is named ListMetadata to
match the C librbd api call.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-04 14:01:51 +00:00
John Mulligan
6bc6c7c3b1 rbd: move TestImageMetadata function to a new test file
Like the metadata functions were recently moved to a separate file,
move TestImageMetadata to a matching metadata_test.go file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-04 14:01:51 +00:00
John Mulligan
eb84f5aba1 rbd: move image metadata functions to new go file
As rbd.go is very long, and the get-/set-/remove- metadata functions in
rbd.go make a nice logical unit, move them to a new metadata.go file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-04 14:01:51 +00:00
Mudit Agarwal
5f107f3f5e Add configuration for Github stale action
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-11-04 08:20:46 -05:00
John Mulligan
b756c84956 rados: add GetNamespace implementing rados_ioctx_get_namespace
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-03 17:50:44 +00:00
John Mulligan
bd77bccc68 rados: add implements section to doc comment for SetNamespace
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-03 17:50:44 +00:00
Niels de Vos
92b10d8699 Include all status checks explicitly in the Mergify configuration
Unfortunately it is not recommended to use a regular expression to check
for the CI status of multiple tests. In case some tests do not run and
fail to report the status, Mergify will not know something is wrong and
PRs might get merged.

The only way to make sure all status checks have passed successfully, is
to list them all separately.

See-also: https://docs.mergify.io/conditions.html#validating-all-status-check
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-03 12:36:46 -05:00
John Mulligan
c7bab1c25c Enable a daily run of the CI
Recently, changes to ceph nautilus reminded me that due to the low rate of PRs we don't always catch things that change in our dependencies right away. I propose adding a daily CI run that help catch these kind of things sooner and can help distinguish between a PR related failure and random flakiness and upstream changes.
2020-11-02 10:19:56 -05:00
Niels de Vos
7160ceaa3a Add configuration for Mergify
These rules for Mergify do the following:

1. dismiss review +1's when a PR is updated
   but keep any -1 reviews, so the reviewer needs to confirm the update
   addresses the comment(s)

2. merge the PR when there are 2 (or more) positive reviews, and no
   negative ones
   don't merge when the 'do-not-merge' label is added to the PR
   also require that the status checks (CI) has passed

3. in case there are merge conflicts, Mergify will leave a message
   asking for a rebase

Documentation: https://docs.mergify.io/configuration.html
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-02 09:11:43 -05:00
John Mulligan
11b6c916ed cephfs admin: fix issues with TestWorkflow
Due to a new ceph nautilus the workflow test was breaking because the
existing code failed to anticipate the "subvolume uuid" in the path
to the snapshot data. This is the curse of writing code without anything
to test it against. I also add a very short delay just to give us
a chance to avoid issues in the future in case the snapshot creation
is not quite synchronous.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-27 10:06:16 -04:00
John Mulligan
55821a9300 cephfs admin: remove subvolumegroup snapshot support
Apparently, this feature is being removed from all active ceph versions
and will not be present in future versions. Since we won't be able to
test it any more, and the api is provisional still, we just remove it
entirely.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-27 10:06:16 -04:00
John Mulligan
dbd5b9361c ci: remove "luminous" from testing matrix
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-14 10:38:58 -04:00
John Mulligan
28912b8d3c readme: document v0.6.0 and supported ceph versions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-14 09:53:23 +02:00
John Mulligan
c7adfc8637 cephfs admin: add a workflow test to check call behaviors
This doesn't touch every call in cephfs admin, but does try to act like
a real application, or set of applications would: reading and writing
data as well as using the api to create and manage subvolumes,
snapshots, and clones.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-09 14:15:56 -04:00
John Mulligan
17cab5c47d cephfs admin: add support for a list of features in SubVolumeInfo
Some versions of ceph return a list of supported features for a
subvolume. On versions that don't return this field the list will always
be empty.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-09 11:20:06 -04:00
John Mulligan
135e624aa5 cephfs admin: add ListFileSystems function for listing fs with pools
For users who need to know the pools (by name) as well as the files
systems, we add ListFileSystems which is similar to 'ceph fs ls', rather
than the existing ListVolumes as that function is simlar to 'ceph fs
volume ls' and that does not return pool information.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-09 10:15:16 +02:00
John Mulligan
c23e1a8659 cephfs admin: doc comment improvements for clone
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-07 08:54:34 -04:00
John Mulligan
0fa5143172 cephfs admin: move tests that were previously only built on octopus
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan
5dce7c9014 cephfs admin: return not-impl errors if ceph returns text for volume status
Return not implemented errors when "older" versions of ceph return
formatted text for output instead of JSON, even tho we asked for json.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan
5f64c52223 cephfs admin: add support for returning not-implemented errors
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan
462372c178 cephfs admin: add ability to check server version in tests
Rather than compile things in and out, which really should only be for
the client version of the ceph libs, add vars that tests can use to
check the server. Also, a "sentinel" test is added to make sure the
tests are being fed a version it understands.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan
a71ec050bf cephfs admin: expose all functions to all build tags >=natuilus
Previously, I was sort of abusing the build tags system to just get our
tests working. That was incorrect because you could be building go-ceph
against nautilus libs but using an octopus server and vice versa. In
addition, I've learned that some of the snapshot and clone behaviors are
being changed as items are backported in ceph. This means that build
tags themselves are not good indicators of what a particular version of
ceph supports.

This patch is the first step to removing these restrictions and making
cephfs admin work more smoothly across more versions of ceph.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-03 11:08:13 -04:00
John Mulligan
e14bd1e436 cephfs admin: add in a test for canceling clones
This test is added after we add force delete functions as deleting a
canceled clone requires a force remove function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
c9ab11ad12 cephfs admin: add ForceRemove<XYZ> variants for subv and group snaps
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
6a020daeb7 cephfs admin: add ForceRemove<XYZ> variants for subvolumes and groups
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
dcc4f67f1d cephfs admin: have clone test work on older and newer ceph versions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
e7c7752393 cephfs admin: ignore deprecation warnings for protect/unprotect
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
9a97852bf2 cephfs admin: add unit tests for response type
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
b001f1a4d2 cephfs admin: refactor response handling with a dedicated response type
The response type simplifies the error handling as a "pipeline" of
checks that are applied one after another.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
John Mulligan
07722a0111 cephfs admin: add basic functionality for clone support
I think I'm a clone now

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-10-02 09:12:02 -04:00
Sven Anderson
beb3351f24 callbacks: add a benchmark test case
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-10-01 10:03:41 -04:00
John Mulligan
e3c0d271dd cephfs admin: add subvolume group snapshot create, remove, and list funcs
Add CreateSubVolumeGroupSnapshot function.
Add RemoveSubVolumeGroupSnapshot function.
Add ListSubVolumeGroupSnapshots function.
Add tests for the above.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-09-28 13:43:25 -04:00