Commit Graph

1033 Commits

Author SHA1 Message Date
Sven Anderson 97630486ee contrib: implements tool prints stderr if castxml fails
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-10-04 15:49:43 +00:00
Sven Anderson 3210885d21 ptrguard: simplify pointer size assert
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-10-04 06:58:23 +00:00
John Mulligan a451da9e8e github: require explicit "no-API" label for merge on 1 review
It's too easy to forget to triage a PR with the API label, and then
if the PR looks good and is approved the mergify system will merge
it. Move to using an explicit "no-API" label for marking the PR as truly
not having API changes and not requiring a 2nd review.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 09:49:24 -04:00
John Mulligan 3fef124e9d docs: add new rbd admin functions to api-status (post rebase)
While working on this patch series new APIs were added to the rbd/admin
package. This patch includes them in the initial set of apis being
tracked.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 3f32efcf56 docs: add initial copies of the api-status.json & markdown doc
The file api-status.json is designed to track changes to the go-ceph
APIs, especially the 'deprecated' and 'preview' apis. It also stores
lists of stable apis so that we can see if something new but not
explicitly stable appears.

The api-status.md file is actually meant to be read by users of go-ceph
so to help make decisions about the use of new preview apis or
deprecated apis. It is generated from api-status.json.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 7ce34854e1 workflows: experiment with apiage check in the ci
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 5d41b671df contrib: add apiage.py - a script to help track the "aging" of apis
This tool can be run as part of our CI as a mechanism to check that
(some) of the policy requirements are being met. It can also be run by
humans as part of the process to update the tracked API information.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 16ff05194c entrypoint: scan the sources of cephfs/admin, rbd/admin, rgw/admin
Now that 'implements' has rudimentary support for these packages we can
gather information on them just like the C based ones.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 970b2eecda entrypoint: don't print results of 'command' when testing for castxml
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 953e3ab495 dockerfile: enable castxml when possible (again)
Return to trying to install castxml into the container image when
possible. This now appears to work (and function) on the centos 8 based
images.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan f986a965ca implements: always include the ceph_preview build tag
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 3590347f60 implements: add support for scanning the api of the "admin" subpackages
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 961ff19bfd implements: don't panic if no counts for C functions are avaiable
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 29c18bcfd0 implements: report on stable functions in json output
Because there will be a lot of stable functions we only report on them
in the json report as the text report is more meant for summaries.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan d1aa76c613 implements: track all public functions, including stable ones
The go source (ast) visitor will now keep track of all public functions it
finds putting them into lists by category: deprecated, preview, stable.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 2ebd4bc1d5 implements: ignore some C package calls that are just noise
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 18a134b535 implements: gather more per-api-function stats into a struct
Do more than collect names, because we're going to need more information
to feed to the eventual check tool.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 12e7178bc0 implements: teach the implements tool to detect deprecated/preview funcs
Teach the implements tool how to detect functions marked deprecated and
marked preview in the go-ceph codebase. Especially helpful will be the
JSON output which can, in the future, be added to a tool-chain that
helps track when deprecated functions are to be removed and when preview
functions should become stable.

Note that this change ONLY implements the detection of
preview/deprecated based on the doc comments.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan f015cc36ca rbd: clearly mark Set as deprecated
Use the notation described in the new API stability doc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
John Mulligan 7f74708d5b rbd: clearly mark open function as deprecated
Use the notation described in the new API stability doc.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-30 13:27:01 +00:00
Rakshith R 126d6f5747 rbd admin: add rbd task [list|cancel|remove|trash remove|flatten]` apis
These functions let one add various rbd tasks that are done
asynchronously in the background. Also added tests for the same.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-28 13:13:07 +00:00
Rakshith R 890b3619db rbd admin: add type ImageSpec to work with rbd task api
ImageSpec type has been added to work with rbd task api.
It consists of optional pool & namespace and image name/id.
[pool/][namespace/]image[Name/id].
Added unit tests.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-28 13:13:07 +00:00
Jiffin Tony Thottan 62627bf406 rgw/admin: add apis for handling UserCap
Added two apis AddUserCap() and RemoveUserCap() to add/remove the
capabilities from RGW user.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-09-27 12:23:29 +00:00
John Mulligan b425523f44 entrypoint: handle adding build tags through a function
The previous code was somewhat dependent on always having the ceph
version build tag set first. Change the code to use a function so that
constructing the build tags is no longer order dependent and relocate
the actual tag stetting if-statements to below the various function
definitions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
John Mulligan 8835a26c9b makefile: support NO_PREVIEW env var
Following on the previous change, the NO_PREVIEW var can be set to skip
testing with preview functions. We default to testing with preview
functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
John Mulligan ee90ae5906 entrypoint: default to running tests with 'ceph_preview' build tag
As this script is the heart of our ci and we generally want to test all
of go-ceph in our ci, we default to enabling ceph_preview build tag. If
we need to test only non-preview functions one can set "NO_PREVIEW" env
var to skip setting the build tag.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-22 11:42:22 +00:00
Sven Anderson c8cf747e39 entrypoint: show return values of tests
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 19:01:04 +00:00
Sven Anderson af5a9e9906 rados: remove unused constant
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson 605e31fef8 rados: remove gosimple linter warnings
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson 54fe45484a rados: remove errcheck linter warnings
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson b7f01be284 rados: remove ineffassign linter warnings
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-16 13:47:56 +00:00
Sven Anderson ff75b9c2a2 rados: add missing error asserts
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-09-14 14:14:23 +00:00
yjkim1 bb535d6df2 cephfs: add missing cephfs API components: function ceph_version
Signed-off-by: Kim young jin <oct28-yjkim@gmail.com>
2021-09-13 15:51:10 +00:00
John Mulligan 361598d4d3 revive: enable additional rules
This enables a bunch of rules that are fairly safe for go-ceph.
There's a lot more rules, but many of them are not as good a fit.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 6a9c0b243a rgw admin: import testify suite as tsuite
This avoids import shadowing elsewhere in radosgw_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan c658cc93a9 rados: import testify suite as tsuite
This avoids import shadowing elsewhere in rados_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan bf6cd07204 cephfs admin: import path package as pathpkg
This avoids import shadowing elsewhere in workflow_test.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 2256a73262 revive: enable var-naming rule (with exceptions)
Revive's var-naming rule will help us avoid adding future violations
of Go's standard naming convention (camelCase, etc.).
We make exceptions for "Id" and "Uid" as they are treated more like
words than initialisms in this codebase and there are already many many
instances if "Id", etc. that need renaming or exceptions and it's just
not worth it. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 89c64b3100 rbd: change instance_ids to instanceIDs
Fix up variable names that don't meet Go standards.
This one crept in while working on older code but before the newer
patch enabling revive to find these was available.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 91efd1f092 rados: naming conventions: disable revive naming check on old struct
This struct has public fields that violate standard naming
conventions.
Disable the check on this struct only.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 25c0699926 rados: naming conventions: disable revive naming check on old struct
This struct has public fields that violate standard naming conventions.
Disable the check on this struct only.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 572de578f6 rados: naming conventions: rename ensure_connected to ensureConnected
Fix up variable names that don't meet Go standards.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 6856f9efa2 implements: naming conventions: remove underscores from c stub vars
Fix up variable names that don't meet Go standards.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan c1e544b8c3 implements: naming conventions: Xml -> XML
Fix up variable names that don't meet Go standards.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-13 15:13:53 +00:00
John Mulligan 2456bf70a1 containers: update go version
With the recent release of go1.17 the oldest supported go version is
1.16. Update to the newest 1.16.z and clean up the structure of
the go install process a bit.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan ac2e61a699 makefile: add a variable for passing additional container build options
The one I often use is "--no-cache" to force a rebuild. Adding a var to
the makefile makes it so I don't have to get too hacky with it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan c7fb7aaa7c makefile: determine CEPH_TAG based on CEPH_VERSION
For go-ceph's purposes we prefer the series codename so we can simply
derive CEPH_TAG from CEPH_VERSION in the makefile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan cf6de4b12d containers: use ceph/ceph as our base image
First off, ceph is no longer using docker hub for the canonical image
hosting. We have also been recommended to use ceph/ceph instead of
ceph/daemon-base. Thus we switch to 'quay.io/ceph/ceph'.

However, ceph/ceph doesn't use codenames for tags so we also have to use
version number tags. These are set as build args now. To retain
compatibility with the way we were working previously we have add a
best-effort "assertion" to ensure the code name we provide matches with
the codename embedded in the base image.

While working on this I discovered that our previous build args were
probably not working as intended, but only by accident. See:
https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
This is why we now name our build arg GO_CEPH_VERSION and it is located
after the 'FROM' line.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-09-02 13:53:09 +00:00
John Mulligan 1adc3bac8b workflows: simplify ci runs using new make targets
The new make targets allow us to share (mirror-enabled) ceph container
instances. Use this to test both non-ptrguard and prtguard test runs
while dropping the (now redundant) non-mirror-enabled ceph + tests
container run.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-31 17:51:58 +02:00
John Mulligan d535ef39de cephfs: make statx test more robust
The TestStatxFieldsRootDir test was assuming that the nlink count
returned by statx would be exactly 2. Make the test more robust against
varying content the root dir of the volume might have by only asserting
that the nlink be 2 or more.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-08-31 17:51:58 +02:00