Commit Graph

1381 Commits

Author SHA1 Message Date
John Mulligan 065c996fc8 docs: fix version numbers of recently added APIs
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-27 17:55:51 +00:00
Nikhil-Ladha fcd2fcd6f6 rbd: Add mirror peer site API components
**Description:**
Added some API components of mirror peer site

Fixes: #486

Signed-off-by: Nikhil-Ladha <nikhilladha1999@gmail.com>
2023-03-27 08:39:12 +00:00
Anoop C S 48afd419c4 docs: Add volume info fetching API to api-status
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-20 14:17:55 +00:00
Anoop C S 2eb895d54f cephfs/admin: Add tests for API fetching volume info
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-20 14:17:55 +00:00
Anoop C S d9b77dfca6 cephfs/admin: Add API to fetch volume info
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-20 14:17:55 +00:00
Nikhil-Ladha 96408092ab rgw/admin: Add missing uid member to usage struct
**Description:**
Added the uid member to the usage type.
We don't need to handle the uid here, as the ceph funtion handles it in the backend.

Fixes: #758

Signed-off-by: Nikhil-Ladha <nikhilladha1999@gmail.com>
2023-03-20 07:09:27 +00:00
Anoop C S 50926448c5 docs: Add Subvolume{Group} pinning API to api-status
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-17 20:32:44 +00:00
Anoop C S 8bc6ef4fa5 cephfs/admin: Add tests for Subvolume{Group} pinning API
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-17 20:32:44 +00:00
Anoop C S 11361b05a8 cephfs/admin: Add Subvolume{Group} pinning API
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-17 20:32:44 +00:00
John Mulligan 1feedf3bfc rbd: remove stray debugging prints from mirror tests
It is doubtful that these Print(f)s have been useful to any one for a
long time.  Remove them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-17 18:10:53 +00:00
John Mulligan 2270166490 docs: update api-status for new mirroring image replay status funcs
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-17 18:10:53 +00:00
John Mulligan ab7955a236 rbd: add tests for DescriptionReplayStatus and UnmarshalDescriptionJSON
Add unit and simplistic functional tests for new JSON/status description
extraction functions.  Because these are preview functions we also add a
stub file to add a no-op function to be built on non-preview builds.
This avoids having to reimplement large parts of the existing mirroring
tests just to validate this small aspect of the SiteMirrorImageStatus
struct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-17 18:10:53 +00:00
John Mulligan f4f0d6dd9a rbd: add functions to parse json in mirroring status description field
Add UnmarshalDescriptionJSON method to SiteMirrorImageStatus
Add DescriptionReplayStatus method to SiteMirrorImageStatus

Both these functions are meant to help extract the JSON object that is
some times appended to the Description field in the
SiteMirrorImageStatus struct. I have only seen the JSON appear after
"replaying, " but there's little docs for this and I skimmed to code to
even find out what fields the JSON might include.  As such, the
UnmarshalDescriptionJSON tries to unmarshal the appended JSON into an
object the caller provides in case skimming the code was insufficient or
the caller has other needs. For most cases, DescriptionReplayStatus can
be used to parse the JSON into a struct containing known fields.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-17 18:10:53 +00:00
Nick Janus 5bb75604fa rados: free object list results after listing
This fixes a memory leak that becomes substantial across large pools.

Signed-off-by: Nick Janus <github@nondesignated.com>
2023-03-15 13:41:40 +00:00
Anoop C S 7607fd801b Dockerfile: Add missing GOPROXY build argument
Previously, commit 102616b158 tried to
expose GOPROXY env var from developer environment by specifying it as
environment variable or build argument to container build process. But
using --build-arg to container build command would require the same to
be referenced inside Dockerfile/Containerfile so that the following
warning can be avoided:

[Warning] one or more build args were not consumed: [GOPROXY]

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-08 21:10:15 +00:00
dependabot[bot] a9b8301c17 go-ceph: bump github.com/stretchr/testify from 1.8.1 to 1.8.2
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-02 14:50:14 +00:00
dependabot[bot] 3f95731e69 go-ceph: bump github.com/aws/aws-sdk-go from 1.44.197 to 1.44.212
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.197 to 1.44.212.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.197...v1.44.212)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-02 11:05:25 +00:00
Damien 102616b158 build: propagation of GOPROXY env var
Signed-off-by: Damien <maitredede@gmail.com>
2023-02-21 15:16:48 +00:00
Sven Anderson 50668e7216 CI: disable VCS stamp for go builds
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
Sven Anderson 87041ee7d8 gofmt: reformat everything with new gofmt of go 1.19
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
Sven Anderson 1dfd94f9b3 Go: bump tested Go version to 1.19.5
This bumps the Go version of the test container to 1.19.5, which is
the oldest supported Go version as of now.

The go.mod file (including an upgrade of its dependencies) and the
revive environment is bumped to 1.19 as well.

Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
Damien 4ea99c452e cephfs: implementation of mkdirs
Signed-off-by: Damien <maitredede@gmail.com>
2023-02-20 06:00:15 +00:00
Sven Anderson 7063a4b300 README: add v0.20.0 release
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-14 15:20:58 +00:00
Anoop C S 1e6f8891ea docs: fix api versions for new ListBucketWithStat API
This change executes `make api-fix-versions` in order to convert the
placeholder versions into real version numbers.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-02-14 13:41:44 +00:00
John Mulligan 5cefb0937f github: remove deprecated and failing rebase_fallback attribute
As noted in the status on github:
```
The configuration uses the deprecated rebase_fallback attribute of the
queue action.
A brownout is planned on February 13th, 2023.
This option will be removed on March 13th, 2023.
For more information: https://docs.mergify.com/actions/queue/
```

Therefore we'll simply drop this attribute.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-14 11:15:19 +05:30
John Mulligan 45e9a80284 docs: add SelectFilesystem to api status
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan 7accfa6b82 cephfs admin: update tests to handle more than one file system
If the ceph cluster we're testing against supports more then one
fs our tests shouldn't fail. Update the tests to ensure a cephfs
exists as expected but don't assume it's the only one (or that
it's the first one returned).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan b596cb959b cephfs: add test cases for SelectFileystem call
These tests are all scoped within a test function that will only run if
the alternate fs name is supplied through the environment. This way the
tests can be run by hand on older versions if need be.

The tests verify the use of the SelectFilesystem function and that the
file systems are distinct namespaces.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan 1755e31f71 cephfs: add SelectFileystem implementing ceph_select_filesystem
Add SelectFileystem implementing ceph_select_filesystem which can be
used to select a cephfs by name, prior to calling mount, when the ceph
cluster hosts more than one cephfs file system.

Fixes: #262
Fixes: #818
Original-Version-By: "Arvid E. Picciani" <arvid@kraud.cloud>
Original-PR-URL: https://github.com/ceph/go-ceph/pull/819
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan 530fea2032 makefile: add ability to use alternate fs on multi container tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan 0a036df85a entrypoint.sh: add support for using an alternate file system
With micro-osd.sh supporting a 2nd alternate fs we add basic support to
entrypoint.sh to help the tests find and use that alternate fs.
The command line option is flexible, you can supply it with a literal fs
name or, when prefixed with an `@`, it will read the name from a file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan 6f2a33a012 micro-osd.sh: run a second filesystem on pacific and later
Add a function to bring up a 2nd cephfs file system. Execute it on
ceph versions pacific and later.

While the API exists on nautilus and octopus those versions are already
deprecated in go-ceph. Additionally, the ceph docs for "multifs" state,
"Beginning with the Pacific release, multiple file system support is
stable and ready to use." [1] Therefore we won't bother running our test
automation on anything earlier.

1 - https://docs.ceph.com/en/latest/cephfs/multifs/#multiple-ceph-file-systems

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan dada7a32b3 github: fix linebreaks in pr template
The linebreaks within the 2 bottom paragraphs look fine in an editor
such as vim, but look odd when viewed as part of the PR on github.
Improve the visuals and make it more readable in the github UI by
removing linebreaks.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-07 06:15:58 +00:00
dependabot[bot] 6016a49e67 go-ceph: bump github.com/gofrs/uuid
Bumps [github.com/gofrs/uuid](https://github.com/gofrs/uuid) from 4.3.1+incompatible to 4.4.0+incompatible.
- [Release notes](https://github.com/gofrs/uuid/releases)
- [Commits](https://github.com/gofrs/uuid/compare/v4.3.1...v4.4.0)

---
updated-dependencies:
- dependency-name: github.com/gofrs/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-02 23:11:36 +00:00
dependabot[bot] b26adadc9f go-ceph: bump github.com/aws/aws-sdk-go from 1.44.172 to 1.44.192
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.172 to 1.44.192.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.172...v1.44.192)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-02 20:41:26 +00:00
John Mulligan daf48e4aa2 nfs admin: fix handling of non-existing psuedo_paths for export info
Fixes: #815

When updates to the NFS mgr API were recently merged on Ceph main branch
the go-ceph test started failing. This occurred because go-ceph was
unintentionally relying on invalid JSON (an empty string) returned from
the API call for an invalid export pseudo path. On ceph main it now
returns an empty JSON object. This change treats both empty string
and empty JSON object as an error explicitly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-31 05:14:58 +00:00
Shachar Sharon d930ba7157 containers: use proper go arch for local host
Use the GOARCH variable to define to proper version of go binaries to
download. Without this, the end result image can not run properly on
local host machine.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2023-01-25 15:06:48 +00:00
Shachar Sharon 4543bdda46 makefile: pass GOARCH as container's build arg
Pass GOARCH as explicit argument when building container image. Use
local host machine to define desired architecture.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2023-01-25 15:06:48 +00:00
moti 3c45e67b6e rgw: add ListBucketsWithStat api
Add api stability.

Signed-off-by: moti <motaharesdq@gmail.com>
2023-01-24 13:38:48 +00:00
moti dafbe65716 rgw: add ListBucketsWithStat function
Add an rgw api function that combines listing and stat'ing buckets.

Signed-off-by: moti <motaharesdq@gmail.com>
2023-01-24 13:38:48 +00:00
Anoop C S 8960979182 docs: Update status for subvolume and snapshot metadata APIs
Following APIs are being added as preview APIs:

FSAdmin.GetMetadata
FSAdmin.SetMetadata
FSAdmin.RemoveMetadata
FSAdmin.ForceRemoveMetadata
FSAdmin.ListMetadata
FSAdmin.GetSnapshotMetadata
FSAdmin.SetSnapshotMetadata
FSAdmin.RemoveSnapshotMetadata
FSAdmin.ForceRemoveSnapshotMetadata
FSAdmin.ListSnapshotMetadata

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-01-23 15:12:52 +00:00
Anoop C S 8b8e63ebde cephfs/admin: Remove ceph_pre_quincy tags
With an exemption to Pacific, necessary backports are already available
with recent Quincy releases.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-01-23 15:12:52 +00:00
John Mulligan 122159ed21 docs: fix api versions for new rbd migration apis
As of d6bf51115d we've tried to make API
stability less burdensome on contributors by putting in placeholder
values in the status files. However, we haven't used it much yet. This
change executes `make api-fix-versions` in order to convert
the placeholder versions into real version numbers - both for the
real reason of tracking the values and also for a more real world test
of the apiage feature.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 05:44:24 +00:00
John Mulligan 64d5ab403b docs: fix common/admin/nfs section of api-status.md
It now reports "No Preview/Deprecated APIs found. All APIs are
considered stable." rather than leaving a blank section.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 05:44:24 +00:00
John Mulligan 36f65ce31d contrib: fix and simplify condition for reporting only stable apis
When generating the human-readable api-status.md document from the
JSON, the logic used to determine if a package had no Preview/Deprecated
APIs was incorrect. This change fixes and simplifies that condition.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 05:44:24 +00:00
John Mulligan 5e369f2546 github: update PR template with new checklist item and helpful links
Add a new checklist item reminding contributors to run `make
api-update` to track new APIs. Add two short paragraphs that
cover some of the common things I copy and paste or need to look up
when working with contributors' PRs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-11 06:20:56 +00:00
Omid Yoosefi a6a42e9c7b rbd: implement live migration functions
Signed-off-by: Omid Yoosefi <omidyoosefi@ibm.com>
2023-01-11 05:35:52 +00:00
John Mulligan 2d1d51b762 contrib: convert some print calls info info/warning message calls
Adds a info and a warning function that currently just wraps print.
In the future these classified print statements could be customized
for output specific to the message type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-10 16:46:02 +00:00
John Mulligan dd95ce1437 contrib: add a specific warning to display new and unexpected stable api
Previously, apiage.py just print out a problem count when it found
an unexpected stable api in the new apis file. Now the output displays
something like this:
```
found new unexpected stable apis: API.LeakyBucket
error: 1 problems detected
```

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-10 16:46:02 +00:00
John Mulligan 76ca06826b contrib: format apiage.py with black python formatter
Ran: `black -l78 contrib/apiage.py`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-10 16:46:02 +00:00