Commit Graph

1320 Commits

Author SHA1 Message Date
John Mulligan
ab7955a236 rbd: add tests for DescriptionReplayStatus and UnmarshalDescriptionJSON
Add unit and simplistic functional tests for new JSON/status description
extraction functions.  Because these are preview functions we also add a
stub file to add a no-op function to be built on non-preview builds.
This avoids having to reimplement large parts of the existing mirroring
tests just to validate this small aspect of the SiteMirrorImageStatus
struct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-17 18:10:53 +00:00
John Mulligan
f4f0d6dd9a rbd: add functions to parse json in mirroring status description field
Add UnmarshalDescriptionJSON method to SiteMirrorImageStatus
Add DescriptionReplayStatus method to SiteMirrorImageStatus

Both these functions are meant to help extract the JSON object that is
some times appended to the Description field in the
SiteMirrorImageStatus struct. I have only seen the JSON appear after
"replaying, " but there's little docs for this and I skimmed to code to
even find out what fields the JSON might include.  As such, the
UnmarshalDescriptionJSON tries to unmarshal the appended JSON into an
object the caller provides in case skimming the code was insufficient or
the caller has other needs. For most cases, DescriptionReplayStatus can
be used to parse the JSON into a struct containing known fields.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-17 18:10:53 +00:00
Nick Janus
5bb75604fa rados: free object list results after listing
This fixes a memory leak that becomes substantial across large pools.

Signed-off-by: Nick Janus <github@nondesignated.com>
2023-03-15 13:41:40 +00:00
Anoop C S
7607fd801b Dockerfile: Add missing GOPROXY build argument
Previously, commit 102616b158 tried to
expose GOPROXY env var from developer environment by specifying it as
environment variable or build argument to container build process. But
using --build-arg to container build command would require the same to
be referenced inside Dockerfile/Containerfile so that the following
warning can be avoided:

[Warning] one or more build args were not consumed: [GOPROXY]

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-03-08 21:10:15 +00:00
dependabot[bot]
a9b8301c17 go-ceph: bump github.com/stretchr/testify from 1.8.1 to 1.8.2
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-02 14:50:14 +00:00
dependabot[bot]
3f95731e69 go-ceph: bump github.com/aws/aws-sdk-go from 1.44.197 to 1.44.212
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.197 to 1.44.212.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.197...v1.44.212)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-02 11:05:25 +00:00
Damien
102616b158 build: propagation of GOPROXY env var
Signed-off-by: Damien <maitredede@gmail.com>
2023-02-21 15:16:48 +00:00
Sven Anderson
50668e7216 CI: disable VCS stamp for go builds
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
Sven Anderson
87041ee7d8 gofmt: reformat everything with new gofmt of go 1.19
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
Sven Anderson
1dfd94f9b3 Go: bump tested Go version to 1.19.5
This bumps the Go version of the test container to 1.19.5, which is
the oldest supported Go version as of now.

The go.mod file (including an upgrade of its dependencies) and the
revive environment is bumped to 1.19 as well.

Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-20 20:50:24 +00:00
Damien
4ea99c452e cephfs: implementation of mkdirs
Signed-off-by: Damien <maitredede@gmail.com>
2023-02-20 06:00:15 +00:00
Sven Anderson
7063a4b300 README: add v0.20.0 release
Signed-off-by: Sven Anderson <sven@redhat.com>
2023-02-14 15:20:58 +00:00
Anoop C S
1e6f8891ea docs: fix api versions for new ListBucketWithStat API
This change executes `make api-fix-versions` in order to convert the
placeholder versions into real version numbers.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-02-14 13:41:44 +00:00
John Mulligan
5cefb0937f github: remove deprecated and failing rebase_fallback attribute
As noted in the status on github:
```
The configuration uses the deprecated rebase_fallback attribute of the
queue action.
A brownout is planned on February 13th, 2023.
This option will be removed on March 13th, 2023.
For more information: https://docs.mergify.com/actions/queue/
```

Therefore we'll simply drop this attribute.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-14 11:15:19 +05:30
John Mulligan
45e9a80284 docs: add SelectFilesystem to api status
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
7accfa6b82 cephfs admin: update tests to handle more than one file system
If the ceph cluster we're testing against supports more then one
fs our tests shouldn't fail. Update the tests to ensure a cephfs
exists as expected but don't assume it's the only one (or that
it's the first one returned).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
b596cb959b cephfs: add test cases for SelectFileystem call
These tests are all scoped within a test function that will only run if
the alternate fs name is supplied through the environment. This way the
tests can be run by hand on older versions if need be.

The tests verify the use of the SelectFilesystem function and that the
file systems are distinct namespaces.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
1755e31f71 cephfs: add SelectFileystem implementing ceph_select_filesystem
Add SelectFileystem implementing ceph_select_filesystem which can be
used to select a cephfs by name, prior to calling mount, when the ceph
cluster hosts more than one cephfs file system.

Fixes: #262
Fixes: #818
Original-Version-By: "Arvid E. Picciani" <arvid@kraud.cloud>
Original-PR-URL: https://github.com/ceph/go-ceph/pull/819
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
530fea2032 makefile: add ability to use alternate fs on multi container tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
0a036df85a entrypoint.sh: add support for using an alternate file system
With micro-osd.sh supporting a 2nd alternate fs we add basic support to
entrypoint.sh to help the tests find and use that alternate fs.
The command line option is flexible, you can supply it with a literal fs
name or, when prefixed with an `@`, it will read the name from a file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
6f2a33a012 micro-osd.sh: run a second filesystem on pacific and later
Add a function to bring up a 2nd cephfs file system. Execute it on
ceph versions pacific and later.

While the API exists on nautilus and octopus those versions are already
deprecated in go-ceph. Additionally, the ceph docs for "multifs" state,
"Beginning with the Pacific release, multiple file system support is
stable and ready to use." [1] Therefore we won't bother running our test
automation on anything earlier.

1 - https://docs.ceph.com/en/latest/cephfs/multifs/#multiple-ceph-file-systems

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-13 10:11:06 -05:00
John Mulligan
dada7a32b3 github: fix linebreaks in pr template
The linebreaks within the 2 bottom paragraphs look fine in an editor
such as vim, but look odd when viewed as part of the PR on github.
Improve the visuals and make it more readable in the github UI by
removing linebreaks.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-07 06:15:58 +00:00
dependabot[bot]
6016a49e67 go-ceph: bump github.com/gofrs/uuid
Bumps [github.com/gofrs/uuid](https://github.com/gofrs/uuid) from 4.3.1+incompatible to 4.4.0+incompatible.
- [Release notes](https://github.com/gofrs/uuid/releases)
- [Commits](https://github.com/gofrs/uuid/compare/v4.3.1...v4.4.0)

---
updated-dependencies:
- dependency-name: github.com/gofrs/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-02 23:11:36 +00:00
dependabot[bot]
b26adadc9f go-ceph: bump github.com/aws/aws-sdk-go from 1.44.172 to 1.44.192
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.172 to 1.44.192.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.172...v1.44.192)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-02 20:41:26 +00:00
John Mulligan
daf48e4aa2 nfs admin: fix handling of non-existing psuedo_paths for export info
Fixes: #815

When updates to the NFS mgr API were recently merged on Ceph main branch
the go-ceph test started failing. This occurred because go-ceph was
unintentionally relying on invalid JSON (an empty string) returned from
the API call for an invalid export pseudo path. On ceph main it now
returns an empty JSON object. This change treats both empty string
and empty JSON object as an error explicitly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-31 05:14:58 +00:00
Shachar Sharon
d930ba7157 containers: use proper go arch for local host
Use the GOARCH variable to define to proper version of go binaries to
download. Without this, the end result image can not run properly on
local host machine.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2023-01-25 15:06:48 +00:00
Shachar Sharon
4543bdda46 makefile: pass GOARCH as container's build arg
Pass GOARCH as explicit argument when building container image. Use
local host machine to define desired architecture.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2023-01-25 15:06:48 +00:00
moti
3c45e67b6e rgw: add ListBucketsWithStat api
Add api stability.

Signed-off-by: moti <motaharesdq@gmail.com>
2023-01-24 13:38:48 +00:00
moti
dafbe65716 rgw: add ListBucketsWithStat function
Add an rgw api function that combines listing and stat'ing buckets.

Signed-off-by: moti <motaharesdq@gmail.com>
2023-01-24 13:38:48 +00:00
Anoop C S
8960979182 docs: Update status for subvolume and snapshot metadata APIs
Following APIs are being added as preview APIs:

FSAdmin.GetMetadata
FSAdmin.SetMetadata
FSAdmin.RemoveMetadata
FSAdmin.ForceRemoveMetadata
FSAdmin.ListMetadata
FSAdmin.GetSnapshotMetadata
FSAdmin.SetSnapshotMetadata
FSAdmin.RemoveSnapshotMetadata
FSAdmin.ForceRemoveSnapshotMetadata
FSAdmin.ListSnapshotMetadata

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-01-23 15:12:52 +00:00
Anoop C S
8b8e63ebde cephfs/admin: Remove ceph_pre_quincy tags
With an exemption to Pacific, necessary backports are already available
with recent Quincy releases.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2023-01-23 15:12:52 +00:00
John Mulligan
122159ed21 docs: fix api versions for new rbd migration apis
As of d6bf51115d we've tried to make API
stability less burdensome on contributors by putting in placeholder
values in the status files. However, we haven't used it much yet. This
change executes `make api-fix-versions` in order to convert
the placeholder versions into real version numbers - both for the
real reason of tracking the values and also for a more real world test
of the apiage feature.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 05:44:24 +00:00
John Mulligan
64d5ab403b docs: fix common/admin/nfs section of api-status.md
It now reports "No Preview/Deprecated APIs found. All APIs are
considered stable." rather than leaving a blank section.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 05:44:24 +00:00
John Mulligan
36f65ce31d contrib: fix and simplify condition for reporting only stable apis
When generating the human-readable api-status.md document from the
JSON, the logic used to determine if a package had no Preview/Deprecated
APIs was incorrect. This change fixes and simplifies that condition.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-12 05:44:24 +00:00
John Mulligan
5e369f2546 github: update PR template with new checklist item and helpful links
Add a new checklist item reminding contributors to run `make
api-update` to track new APIs. Add two short paragraphs that
cover some of the common things I copy and paste or need to look up
when working with contributors' PRs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-11 06:20:56 +00:00
Omid Yoosefi
a6a42e9c7b rbd: implement live migration functions
Signed-off-by: Omid Yoosefi <omidyoosefi@ibm.com>
2023-01-11 05:35:52 +00:00
John Mulligan
2d1d51b762 contrib: convert some print calls info info/warning message calls
Adds a info and a warning function that currently just wraps print.
In the future these classified print statements could be customized
for output specific to the message type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-10 16:46:02 +00:00
John Mulligan
dd95ce1437 contrib: add a specific warning to display new and unexpected stable api
Previously, apiage.py just print out a problem count when it found
an unexpected stable api in the new apis file. Now the output displays
something like this:
```
found new unexpected stable apis: API.LeakyBucket
error: 1 problems detected
```

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-10 16:46:02 +00:00
John Mulligan
76ca06826b contrib: format apiage.py with black python formatter
Ran: `black -l78 contrib/apiage.py`.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-01-10 16:46:02 +00:00
dependabot[bot]
c690fb96b7 go-ceph: bump golang.org/x/sys from 0.2.0 to 0.4.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.2.0 to 0.4.0.
- [Release notes](https://github.com/golang/sys/releases)
- [Commits](https://github.com/golang/sys/compare/v0.2.0...v0.4.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-05 14:49:06 +00:00
dependabot[bot]
c403910924 go-ceph: bump github.com/aws/aws-sdk-go from 1.44.151 to 1.44.172
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.151 to 1.44.172.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.151...v1.44.172)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-04 19:40:23 +00:00
dependabot[bot]
bd993aea68 go-ceph: bump actions/stale from 6 to 7
Bumps [actions/stale](https://github.com/actions/stale) from 6 to 7.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-04 15:33:25 +00:00
Anoop C S
a0524acd7e rados: Remove 'ceph_preview' tag from test source file
This is a left over from v0.19.0 release where we missed to remove
preview tag when corresponding API was promoted to stable.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-12-19 18:20:48 +00:00
Anoop C S
2c47426469 rgw/admin: Remove 'ceph_preview' tags from test files
These are left overs from v0.13.0 and v0.17.0 releases where we missed
to remove preview tag when corresponding APIs were promoted to stable.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-12-19 18:20:48 +00:00
John Mulligan
d572983696 go-ceph: simplify v0.16.0 quincy note
The v0.16.0 has a "sword" marker noting that quincy was working but not
officially fully supported in v0.16. Now that v0.16 is a few releases in
the past we can simplify that note down to the minimal level of details.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-12-12 15:39:36 +00:00
John Mulligan
ca37a86899 go-ceph: update README.md for v0.19.0 release
This moves ceph octopus to the deprecated column.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-12-12 15:39:36 +00:00
John Mulligan
6f7879428f cephfs admin: add additional delay when module is enabled
There was continued flakiness when the tests that enabled the cephfs
mirroring module were enabled. This change adds another 200 ms delay
to hopefully cover the time that might occur as the mgr has a new
module enabled and restart. But unfortunately this is still a bit of
a wild guess.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-12-08 15:12:38 +00:00
John Mulligan
63370f4df6 cephfs admin: remove test skip in TestCancelClone
This will re-enable the test case on ceph quincy and pacific.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-12-08 15:12:38 +00:00
Anoop C S
747dd7274b rgw/admin: Make API.SetIndividualBucketQuota stable
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-12-06 14:20:29 +00:00
Anoop C S
c11161eb61 rgw/admin: Make API.CreateKey and API.RemoveKey stable
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-12-06 14:20:29 +00:00