Commit Graph

1030 Commits

Author SHA1 Message Date
John Mulligan
3fef124e9d docs: add new rbd admin functions to api-status (post rebase)
While working on this patch series new APIs were added to the rbd/admin
package. This patch includes them in the initial set of apis being
tracked.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
3f32efcf56 docs: add initial copies of the api-status.json & markdown doc
The file api-status.json is designed to track changes to the go-ceph
APIs, especially the 'deprecated' and 'preview' apis. It also stores
lists of stable apis so that we can see if something new but not
explicitly stable appears.

The api-status.md file is actually meant to be read by users of go-ceph
so to help make decisions about the use of new preview apis or
deprecated apis. It is generated from api-status.json.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
7ce34854e1 workflows: experiment with apiage check in the ci
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
5d41b671df contrib: add apiage.py - a script to help track the "aging" of apis
This tool can be run as part of our CI as a mechanism to check that
(some) of the policy requirements are being met. It can also be run by
humans as part of the process to update the tracked API information.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
16ff05194c entrypoint: scan the sources of cephfs/admin, rbd/admin, rgw/admin
Now that 'implements' has rudimentary support for these packages we can
gather information on them just like the C based ones.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
970b2eecda entrypoint: don't print results of 'command' when testing for castxml
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
953e3ab495 dockerfile: enable castxml when possible (again)
Return to trying to install castxml into the container image when
possible. This now appears to work (and function) on the centos 8 based
images.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
f986a965ca implements: always include the ceph_preview build tag
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
3590347f60 implements: add support for scanning the api of the "admin" subpackages
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
961ff19bfd implements: don't panic if no counts for C functions are avaiable
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
29c18bcfd0 implements: report on stable functions in json output
Because there will be a lot of stable functions we only report on them
in the json report as the text report is more meant for summaries.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
d1aa76c613 implements: track all public functions, including stable ones
The go source (ast) visitor will now keep track of all public functions it
finds putting them into lists by category: deprecated, preview, stable.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
2ebd4bc1d5 implements: ignore some C package calls that are just noise
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
18a134b535 implements: gather more per-api-function stats into a struct
Do more than collect names, because we're going to need more information
to feed to the eventual check tool.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
12e7178bc0 implements: teach the implements tool to detect deprecated/preview funcs
Teach the implements tool how to detect functions marked deprecated and
marked preview in the go-ceph codebase. Especially helpful will be the
JSON output which can, in the future, be added to a tool-chain that
helps track when deprecated functions are to be removed and when preview
functions should become stable.

Note that this change ONLY implements the detection of
preview/deprecated based on the doc comments.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
f015cc36ca rbd: clearly mark Set as deprecated
Use the notation described in the new API stability doc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan
7f74708d5b rbd: clearly mark open function as deprecated
Use the notation described in the new API stability doc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
Rakshith R
126d6f5747 rbd admin: add rbd task [list|cancel|remove|trash remove|flatten]` apis
These functions let one add various rbd tasks that are done
asynchronously in the background. Also added tests for the same.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-28 13:13:07 +00:00
Rakshith R
890b3619db rbd admin: add type ImageSpec to work with rbd task api
ImageSpec type has been added to work with rbd task api.
It consists of optional pool & namespace and image name/id.
[pool/][namespace/]image[Name/id].
Added unit tests.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-28 13:13:07 +00:00
Jiffin Tony Thottan
62627bf406 rgw/admin: add apis for handling UserCap
Added two apis AddUserCap() and RemoveUserCap() to add/remove the
capabilities from RGW user.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-09-27 12:23:29 +00:00
John Mulligan
b425523f44 entrypoint: handle adding build tags through a function
The previous code was somewhat dependent on always having the ceph
version build tag set first. Change the code to use a function so that
constructing the build tags is no longer order dependent and relocate
the actual tag stetting if-statements to below the various function
definitions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
John Mulligan
8835a26c9b makefile: support NO_PREVIEW env var
Following on the previous change, the NO_PREVIEW var can be set to skip
testing with preview functions. We default to testing with preview
functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
John Mulligan
ee90ae5906 entrypoint: default to running tests with 'ceph_preview' build tag
As this script is the heart of our ci and we generally want to test all
of go-ceph in our ci, we default to enabling ceph_preview build tag. If
we need to test only non-preview functions one can set "NO_PREVIEW" env
var to skip setting the build tag.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
Sven Anderson
c8cf747e39 entrypoint: show return values of tests
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 19:01:04 +00:00
Sven Anderson
af5a9e9906 rados: remove unused constant
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson
605e31fef8 rados: remove gosimple linter warnings
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson
54fe45484a rados: remove errcheck linter warnings
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson
b7f01be284 rados: remove ineffassign linter warnings
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson
ff75b9c2a2 rados: add missing error asserts
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-14 14:14:23 +00:00
yjkim1
bb535d6df2 cephfs: add missing cephfs API components: function ceph_version
Signed-off-by: Kim young jin <oct28-yjkim@gmail.com>
2021-09-13 15:51:10 +00:00
John Mulligan
361598d4d3 revive: enable additional rules
This enables a bunch of rules that are fairly safe for go-ceph.
There's a lot more rules, but many of them are not as good a fit.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
6a9c0b243a rgw admin: import testify suite as tsuite
This avoids import shadowing elsewhere in radosgw_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
c658cc93a9 rados: import testify suite as tsuite
This avoids import shadowing elsewhere in rados_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
bf6cd07204 cephfs admin: import path package as pathpkg
This avoids import shadowing elsewhere in workflow_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
2256a73262 revive: enable var-naming rule (with exceptions)
Revive's var-naming rule will help us avoid adding future violations
of Go's standard naming convention (camelCase, etc.).
We make exceptions for "Id" and "Uid" as they are treated more like
words than initialisms in this codebase and there are already many many
instances if "Id", etc. that need renaming or exceptions and it's just
not worth it. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
89c64b3100 rbd: change instance_ids to instanceIDs
Fix up variable names that don't meet Go standards.
This one crept in while working on older code but before the newer
patch enabling revive to find these was available.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
91efd1f092 rados: naming conventions: disable revive naming check on old struct
This struct has public fields that violate standard naming
conventions.
Disable the check on this struct only.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
25c0699926 rados: naming conventions: disable revive naming check on old struct
This struct has public fields that violate standard naming conventions.
Disable the check on this struct only.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
572de578f6 rados: naming conventions: rename ensure_connected to ensureConnected
Fix up variable names that don't meet Go standards.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
6856f9efa2 implements: naming conventions: remove underscores from c stub vars
Fix up variable names that don't meet Go standards.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
c1e544b8c3 implements: naming conventions: Xml -> XML
Fix up variable names that don't meet Go standards.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan
2456bf70a1 containers: update go version
With the recent release of go1.17 the oldest supported go version is
1.16. Update to the newest 1.16.z and clean up the structure of
the go install process a bit.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan
ac2e61a699 makefile: add a variable for passing additional container build options
The one I often use is "--no-cache" to force a rebuild. Adding a var to
the makefile makes it so I don't have to get too hacky with it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan
c7fb7aaa7c makefile: determine CEPH_TAG based on CEPH_VERSION
For go-ceph's purposes we prefer the series codename so we can simply
derive CEPH_TAG from CEPH_VERSION in the makefile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan
cf6de4b12d containers: use ceph/ceph as our base image
First off, ceph is no longer using docker hub for the canonical image
hosting. We have also been recommended to use ceph/ceph instead of
ceph/daemon-base. Thus we switch to 'quay.io/ceph/ceph'.

However, ceph/ceph doesn't use codenames for tags so we also have to use
version number tags. These are set as build args now. To retain
compatibility with the way we were working previously we have add a
best-effort "assertion" to ensure the code name we provide matches with
the codename embedded in the base image.

While working on this I discovered that our previous build args were
probably not working as intended, but only by accident. See:
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
This is why we now name our build arg GO_CEPH_VERSION and it is located
after the 'FROM' line.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan
1adc3bac8b workflows: simplify ci runs using new make targets
The new make targets allow us to share (mirror-enabled) ceph container
instances. Use this to test both non-ptrguard and prtguard test runs
while dropping the (now redundant) non-mirror-enabled ceph + tests
container run.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-31 17:51:58 +02:00
John Mulligan
d535ef39de cephfs: make statx test more robust
The TestStatxFieldsRootDir test was assuming that the nlink count
returned by statx would be exactly 2. Make the test more robust against
varying content the root dir of the volume might have by only asserting
that the nlink be 2 or more.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-31 17:51:58 +02:00
John Mulligan
528893ce1d cephfs: clean up symlink created in TestSymlink func
Add a defer to clean up the symlink created by the test.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-31 17:51:58 +02:00
Sven Anderson
83c0acf52c docs: add a API stability document
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-08-25 16:51:26 +00:00
John Mulligan
cdbffdc229 makefile: expand test-multi-container into multiple test-containers-* rules
This change adds a whole family of test-containers-* rules. The role
meant to be used directly "test-containers-test" brings up the
required dependencies once, and will reuse them on subsequent runs.
The targets "test-containers-kill" and "test-containers-clean" will
remove just the ceph containers and all dependencies respectively.

The change may look big, and I did shorten some lines, thus lengthening
the makefile vertically, but the logic is basically the same as before.
One can continue using "test-multi-container" and it should behave
pretty much as before, but outside the CI when working on code or
testing things out using the more granular rules can help speed things
up.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-25 15:15:48 +00:00