Commit Graph

1343 Commits

Author SHA1 Message Date
Sanford Miller
15d71fe192 rbd: add mirroring support for nautilus
The librbd API for mirroring-related operations changed substantially between
Nautilus and Octopus. Due to this, go-ceph had previously only implemented
mirroring functionality if built against Octopus client libraries. This patch
implements equivalent functionality for use with Nautilus clients.

Signed-off-by: Sanford Miller <smiller@digitalocean.com>
2022-07-13 13:00:48 +00:00
Sven Anderson
a2cf1e7262 go-ceph: bump Go version of module to 1.17
Updates the Go version of the module with `go mod tidy -go 1.17`,
which also adds some explicit requirements in order to support lazy
module loading.

Signed-off-by: Sven Anderson <sven@redhat.com>
2022-07-11 16:21:44 +00:00
Anoop C S
6dad78e5af entrypoint: Store bootstrap token in variable
Following the mirroring setup there used to be a file named 'token'
holding the bootstrap token left out in sources. We could instead
provide the token via standard output and display it for any further
debugging purpose.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-07-10 15:06:12 +00:00
Anoop C S
0b06ef208d gitignore: Add vi/vim backup files
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-07-07 13:05:57 +00:00
Quentin Devos
5f38b5422e rados: Implement rados_set_alloc_hint2
Signed-off-by: Quentin Devos <4972091+Okhoshi@users.noreply.github.com>
2022-07-06 17:56:39 +00:00
dependabot[bot]
89899b3833 rebase: bump github.com/aws/aws-sdk-go from 1.44.43 to 1.44.47
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.43 to 1.44.47.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.43...v1.44.47)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-05 14:13:01 +00:00
Joshua Baergen
9e340d1e4e rados: Make sure Conn isn't GC'd while it has dependant IOContexts.
Because we have a finalizer configured for Conn that cleans up Ceph
resources, it's important that we don't let it get GC'd before anything
that depends on it has been cleaned up. Since clients are free to hold a
reference to an IOContext without maintaining a reference to the
underlying Conn, we hold such a Conn reference within the IOContext to
signal this dependency to the Go GC.

Signed-off-by: Joshua Baergen <jbaergen@digitalocean.com>
2022-07-01 15:30:18 +00:00
Anoop C S
1589706287 Makefile: Use ?= for initial variable definitions
:= is not capable of accepting values(for variables) from execution
environment. Therefore replace := with ?= for completeness.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-06-30 18:20:09 +00:00
Sven Anderson
d647433121 CI: remove non-ptrguard tests
We will (most probably) remove non-ptrguard code before the next
release. Let's skip the tests already now, because they double the
likelihood of failing flaky tests.

Signed-off-by: Sven Anderson <sven@redhat.com>
2022-06-30 17:45:00 +00:00
Sven Anderson
517434f084 Add MacOS instructions to README
This is a replacement for #540.
2022-06-30 13:01:11 +00:00
Anoop C S
db3a21808f cephfs/admin: Fix compile errors
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-06-30 08:58:01 +00:00
dependabot[bot]
ee6bb0e6d1 rebase: bump github.com/stretchr/testify from 1.7.2 to 1.8.0
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.2 to 1.8.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.7.2...v1.8.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-30 08:13:23 +00:00
dependabot[bot]
18b5a072b9 rebase: bump github.com/aws/aws-sdk-go from 1.44.28 to 1.44.43
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.28 to 1.44.43.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.28...v1.44.43)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-30 07:39:46 +00:00
Sven Anderson
6c459dc039 CI: also disable TestCancelClone for pacific
The code causing this issue has been packported to pacific and blocks
the CI. We need to temporarily disable for pacific as well.

Signed-off-by: Sven Anderson <sven@redhat.com>
2022-06-29 13:57:41 +00:00
Anoop C S
146ee24547 Dockerfile: Remove an unused CEPH_TAG env variable
Ceph version is already available from CEPH_VERSION env variable.
CEPH_TAG on the other hand is only defined to determine the tag for
ceph base image and seems to be unused in the resulting go-ceph
container image.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-06-28 12:54:48 +00:00
riya-singhal31
eb651c28a4 fix: fixed few typos
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2022-06-27 13:21:51 -04:00
Anoop C S
38862c0c7a makefile: use "pacific" as our default ceph version
"quincy" is already out which will then make "octopus" EOL very soon.
Therefore default to "pacific" instead.

Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2022-06-24 16:01:11 +00:00
Sanford Miller
133e675c8d rbd: refactor GetImageNames to use a for loop instead of recursion
This is done because using a for loop is more idiomatic in Go code.

Signed-off-by: Sanford Miller <smiller@digitalocean.com>
2022-06-14 16:33:29 +00:00
Sanford Miller
36d5c4498e rbd: don't cap the buffer size used in GetImageNames
This removes the limit on the max buffer size GetImageNames is willing to pass
to rbd_list2, which is somewhat arbitrary and is too small for large clusters.
GetImageNames will continue to start with a small buffer size and retry with a
larger buffer if rbd_list2 returns ERANGE (just without a cap on the max buffer
size it's willing to go to).

Signed-off-by: Sanford Miller <smiller@digitalocean.com>
2022-06-14 16:33:29 +00:00
John Mulligan
863d71caa5 rados: various readop & writeop apis are now stable
These apis are now stable as per the stability plan.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-14 16:01:43 +00:00
John Mulligan
ec0b9a1455 rados: watcher apis are now stable
Watcher apis are now stable as per the stability plan.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-14 16:01:43 +00:00
John Mulligan
096745e86b readme: note v0.16.0 has preliminary quincy support
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-14 13:41:11 +00:00
John Mulligan
d31582cf06 cephfs admin: remove deprecated EnableModule, DisableModule funcs
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-14 13:10:09 +00:00
John Mulligan
0722af5375 rbd admin: enable TestMirrorSnapshotScheduleStatus on quincy
The issue has (probably) been fixed in ceph v17.2.0 images.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-14 12:39:59 +00:00
Prasanna Kumar Kalever
ec40a6c6c3 cephfs: add subvolume snapshot metadata APIs
Exports below APIs:
SetSnapshotMetadata(),
GetSnapshotMetadata(),
RemoveSnapshotMetadata(),
ForceRemoveSnapshotMetadata() and
ListSnapshotMetadata()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-13 15:16:36 +00:00
Prasanna Kumar Kalever
7d224905ca cephfs: add subvolume metadata APIs
Exports below APIs:
SetMetadata(),
GetMetadata(),
RemoveMetadata(),
ForceRemoveMetadata() and
ListMetadata()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-13 10:44:05 -04:00
Niels de Vos
e8c006b3af cephfs/admin: add CloneStatus.GetFailure()
In case cloning failed, Ceph Mgr returns an additional attribute in the
response. The "failed" attribute contains an errno (as string?!) and a
description. It is useful for applications using go-ceph to report the
error back to the user.

As the CloneStatus is on the same level as the CloneFailure object in
the response, keeping strictly to the CephFS Mgr API would break
compatibility. Instead, a new CloneStatus.GetFailure() method has been
introduced, returning a CloneFailure type (or nil) object with details
about the failure.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-06-09 19:47:04 +00:00
John Mulligan
9ac8b86285 docs: add API statuses for nfs admin api
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-09 16:24:31 +00:00
John Mulligan
af64f12c62 nfs admin: remove untested build tag
The latest ceph pacific should include the mgr changes needed to test
the NFS apis. We should be able to treat nfs like a normal preview api.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-06-09 16:24:31 +00:00
dependabot[bot]
a12919353c rebase: bump github.com/stretchr/testify from 1.7.1 to 1.7.2
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.7.1...v1.7.2)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-09 14:35:27 +00:00
dependabot[bot]
8d2970ff5a rebase: bump github.com/aws/aws-sdk-go from 1.44.24 to 1.44.28
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.24 to 1.44.28.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.24...v1.44.28)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-09 14:05:18 +00:00
Sven Anderson
f4fb583252 docs: remove all PREVIEW comments
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-06-06 14:42:04 +00:00
Sven Anderson
9f55b65d17 docs: replace references to PREVIEW comment
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-06-06 14:42:04 +00:00
Sven Anderson
423c7ca899 implements: use ceph_preview tag instead of PREVIEW comment
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-06-06 14:42:04 +00:00
dependabot[bot]
40f3799e02 rebase: bump github.com/aws/aws-sdk-go from 1.44.10 to 1.44.24
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.10 to 1.44.24.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.10...v1.44.24)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-06 14:07:16 +00:00
Sven Anderson
005aa73667 rgw/admin: fix tests for quincy
Signed-off-by: Sven Anderson <sven@redhat.com>
2022-05-19 14:09:21 +00:00
Sanford Miller
68ebb0520a rbd: expose rbd_snap_rename to Go clients
This implements the following method:
	(snapshot *Snapshot) Rename(destName string) error

Signed-off-by: Sanford Miller <smiller@digitalocean.com>
2022-05-17 14:40:07 +00:00
dependabot[bot]
fc91a0661f rebase: bump github.com/aws/aws-sdk-go from 1.44.5 to 1.44.10
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.5 to 1.44.10.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.5...v1.44.10)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-13 13:58:54 +00:00
John Mulligan
40f1fa47d9 containers: update the go version used in the CI container
The Go 1.18 release was done in march 2022 and so we should
continue to use the older of the two supported releases which
is now Go 1.17.x.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-11 16:05:23 +00:00
dependabot[bot]
e48b72c98e rebase: bump github.com/aws/aws-sdk-go from 1.44.3 to 1.44.5
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.3 to 1.44.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.3...v1.44.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-09 18:01:44 +00:00
John Mulligan
57bbe83bef rgw admin: disable TestBucket on qunicy
Mysteriously fails without a useful error.  Will need to be properly
debugged later.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
68eff5d9e4 rbd admin: disable TestMirrorSnapshotScheduleStatus on qunicy
Will need to be properly debugged later.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
e0369cdf88 cephfs admin: disable test TestCancelClone on quincy
This test feels very flaky, only passing 2-3 times out of ten (by
my vague guesstimate). This needs investigation.
However I don't want to block all of CI against quincy for this this
now.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
37ba163aae makefile: allow testing with unreleased quincy containers
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
13bdb3e26c cephfs admin: remove explicit subvol delete from snap retention test
On ceph quincy, removing the subvolume explicitly (even with --force)
triggered an error. Due to the discussion in
https://tracker.ceph.com/issues/54625
I was informed that the explicit delete was not necessary, and while
the error on quincy was not correct, even on previous versions the
call is basically a no-op.

Rather than wait for this to be changed back to a no-op on quincy,
we remove the unneeded call. While we're at it, change the polling
loop to use the assert lib's Eventually call which is much nicer
to read and expires based on a duration avoiding the need to
think about loop iterations. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
ae44f69791 cephfs admin: flexibly decode the MDSVersion field in volume status
It appears the JSON being returned from `ceph fs status` has changed.
The field returning the mds version was previously "just" a string.
Now its a more complex thing. This of course breaks the old version
of the go code which is statically typed to expect a string.

In order not to break backwards compatibility we replace the object
used for parsing the JSON with a private object that has a custom
unmarshaller function for the mds version field. If it sees the string
it uses the string. Otherwise, it looks for this new structure and,
if possible, extracts the version string from the first item.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
ac1401b4e2 cephfs admin: add quincy to valid server editions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
0f29fdd510 micro-osd.sh: fix rgw connecting to mon without cephx
See also:
* https://tracker.ceph.com/issues/52103
* https://github.com/ceph/ceph/pull/42718

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
d01392853c micro-osd.sh: make rgw log location be consistent with every other daemon
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00
John Mulligan
61826a4af5 micro-osd.sh: make shellcheck happier
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 14:14:25 +00:00