Commit Graph

764 Commits

Author SHA1 Message Date
Sven Anderson
48bff9548a ci: wait before starting ceph-osd
This is another attempt to fix #423.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-04 07:38:38 +00:00
John Mulligan
0b4ed9dbe8 rbd: add GetGroup implementing rbd_get_group
This method is useful to determine what group an image is part of.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-02 15:48:00 +00:00
John Mulligan
cbf2a51698 rbd: add GroupImageList implementing rbd_group_image_list
This function can be used to list the images in a group.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-02 15:48:00 +00:00
John Mulligan
cd0a5bfe01 rbd: add GroupImageAdd, GroupImageRemove, and GroupImageRemoveByID
* Add GroupImageAdd implementing rbd_group_image_add
* Add GroupImageRemove implementing rbd_group_image_remove
* Add GroupImageRemoveByID implementing rbd_group_image_remove_by_id

These functions allow managing the images that are part of a group.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-02 15:48:00 +00:00
Mudit Agarwal
1e15c36ec9 cephfs: add state field to SubVolumeInfo structure.
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-02 15:29:53 +00:00
John Mulligan
1748ad781c rbd: test that cephIoctx helper panics when fed invalid inputs
As per go-ceph issue #427, and a discussion in #399 this change and
test ensure that the cephIoctx helper function, that extracts the
C.rados_ioctx_t from a Go rados.IOContext, never returns a nil/null
C type.
As these are always "programming errors" panicking is reasonable
behavior for this case.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-01 15:31:17 +00:00
John Mulligan
5130f55387 cephfs admin: fix the json sent when flag RetainSnapshots is set
The json key sent to retain snapshots on subvolume remove was misspelled
and had a dash in it rather than an underscore. Fix the incorrect key
and add a test, that needs to be able to run on version of cephfs w/o
snapshot retention, and verifies that flag does the correct thing.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-01 08:33:28 +00:00
Niels de Vos
05a8b48688 rbd: Add TestWriteSame/zerofill test case
Using WriteSame() to implement a zerofill function is less straight
forward than expected. `rbd_discard_on_zeroed_write_same` is a Ceph
option than affects the behaviour of WriteSame(). By default the option
is enabled, causing WriteSame() to call discard instead of writing
zeros.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-01-28 16:04:12 +00:00
Mudit Agarwal
3032c539a0 rbd: add GetImageMirrorMode() to get the mirroing mode of a RBD image
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-01-25 20:35:15 +00:00
Mudit Agarwal
6ff40f828f rbd: add function to fetch mirror status of a RBD image.
Add GetMirrorImageInfo() to fetch the mirror status of a RBD image.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-01-25 20:35:15 +00:00
John Mulligan
25a5086184 rbd: fix typo in comment & improve comment consistency
RBD was misspelled as RDB. While I was there I made the comments
for both constants more consistently worded.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-25 14:38:39 +00:00
John Mulligan
1e3b9a2cd9 rbd: add an initial set of basic mirroring functions
Add functions:
* SetMirrorMode implementing rbd_mirror_mode_set
* GetMirrorMode implementing rbd_mirror_mode_get
* MirrorEnable implementing rbd_mirror_image_enable2
* MirrorDisable implementing rbd_mirror_image_disable
* MirrorPromote implementing rbd_mirror_image_promote
* MirrorDemote implementing rbd_mirror_image_demote
* MirrorResync implementing rbd_mirror_image_resync
* MirrorInstanceId implementing rbd_mirror_image_get_instance_id

For now, these mirroring related functions are only supported on octopus
builds. Right now the demand is for snapshot based mirroring, which is
new in octopus. We can always relax this in the future and add the
necessary support for nautilus, for journaling only based mirroring.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-22 09:27:06 -05:00
John Mulligan
a40638e947 rbd: add GroupList implementing rbd_group_list
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-22 08:13:20 +00:00
John Mulligan
7eeb5f79ab rbd: add GroupCreate, GroupRemove, & GroupRename functions
Add GroupCreate implementing rbd_group_create
Add GroupRemove implementing rbd_group_remove
Add GroupRename implementing rbd_group_rename

Naming was <noun><verb> to better match other functions in rbd
like NamespaceCreate, etc. even though I don't prefer that
ordering ;-).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-22 08:13:20 +00:00
John Mulligan
76bca17e43 rbd: enable test cases with ceph issues that have been fixed
Ceph issue https://tracker.ceph.com/issues/43178 has been fixed for
a while with backports. No reason not to enable these test cases now.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-21 21:40:20 +00:00
Sven Anderson
0130c60281 cutil: add general C type, function and const aliases
This change adds a new pointer alias CPtr, that should always only
contains pointers to C allocated memory and should always used for
such pointers in order to keep them separate from Go pointers more
easily. This is helpful, since only CPtr are allowed to be stored in
C allocated memory, and Go pointers can only be passed to C functions
if they don't point to memory not containing Go pointers again, that
is all pointers in such memory must be CPtr.

The change also adds aliases for C.malloc and C.free that return and
accept the CPtr type as a little safety measure.

Additionally there are some useful constants defined.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-01-20 14:46:03 +00:00
John Mulligan
c995d63088 cephfs: enable test cases with ceph issues that have been fixed
Ceph issue https://tracker.ceph.com/issues/46084 has been fixed with
backports to nautilus and octopus. Re-enable the relevant test data
and fix up the tests to match.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-19 08:23:52 +00:00
Sven Anderson
134c628df1 ci: run ceph-osd in foreground during mkfs
This is an attempt to fix #423.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-01-15 14:31:48 +00:00
John Mulligan
c91f54ef25 rados: add test cases for ReadOp type and functions
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
0a67ff8f5a rados: add an initial set of tests for WriteOp operations
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
315188eaef rados: add AssertExists function to ReadOp
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
c8d8d8c364 rados: add Write, WriteFull, and WriteSame WriteOp functions
Add Write implementing rados_write_op_write.
Add WriteFull implementing rados_write_op_write_full.
Add WriteSame implementing rados_write_op_writesame.

These are some of the basic functions needed to write data to
a rados object using a WriteOp.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
e4e1257768 rados: add AssertExists WriteOp function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
570738b24c rados: add CleanOmap WriteOp function
Reimplement CleanOmap in term of a new CleanOmap function on the
WriteOp.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
f4e0730b8f rados: add RmOmapKeys WriteOp function
Reimplement RmOmapKeys in term of a new RmOmapKeys function on the
WriteOp.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
701bb74897 rados: add GetOmapValues function to the ReadOp
Refactor the previous ListOmapValues logic into a new read op step
that can be used for iteration over the results of the operation. The
previous function is now based on the ReadOp.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
bc6cabbe9f rados: add SetOmap WriteOp function
Reimplement SetOmap in terms of a new SetOmap function on the WriteOp.
Now the actions of the write op can start to be atomically chained the
way WriteOps are supposed to.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
614806ad01 rados: add withRefs type
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
1d6366f008 rados: add an error to be used if data is requested too early
Add a new error that will be returned from a op step method if the
method is called before the step is ready to provide data - like
when the operation has not yet been performed.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
dffdc11f1f rados: add an initial ReadOp type for rados read operations
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
557390fb1d rados: make ioctx create internally based on WriteOp Create
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
926e101171 rados: create a new WriteOp type around rados_write_op_t
The WriteOp type will manage librados write operations and the
corresponding functions that ceph names like rados_write_op_*.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
2a31d6a411 rados: add a function for use as a finalizer for any opStep
This utility function avoids having to have a custom finalizer for every
op step. It calls the free() function to release any allocated C memory
the step may have.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
f1934a5513 rados: add a file to expose librados operation flags to our rados pkg
These flags are passed to the *_op_operate calls in the C API.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
c96da54878 rados: add test cases for basic operation types
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
53d2d83bc4 rados: add generic operation type for basis of data handling in op types
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
29ebb61850 timespec: add a helper function to convert from Timespec to C
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
John Mulligan
6900146fc4 cephfs admin: fix reversed args in greater-or-equal assertion
Now that the new year has arrived this test assertion has started to
fail. This is because, although I anticipated the year would eventually
change and used GreaterOrEqual, the argument order is wrong. My excuse
is that I find the argument order in testify's Equal function confusing,
it causes me to assume other functions in testify are also confusing
(even when it's the "normal" order) and/or I copy and pasted from an
Equal line and didn't change the args (both are true!).

Instead of allowing any old year, we assume that the clocks where the
test are run are in sync enough that the current year is correct for
the assertion (thanks for the suggestion Sven).
Happy new year, go-ceph!

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-05 13:30:50 +00:00
John Mulligan
da34b0aec5 github: do not automatically merge on "extended-review" label
The "extended-review" label indicates that review is likely to take
an extended period of time and for bots to not automatically take
a PR.

This avoid the need to preemptively mark a PR as changes needed or
give a false impression with a "do-not-merge" tag, which has a subtle
lack-of-quality implication.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-04 14:54:24 -05:00
Yang Honggang
f3e39224ec Makefile: fix "No such file or directory" error
$ make test-docker
grep: /sys/kernel/security/lsm: No such file or directory

Signed-off-by: Yang Honggang <yanghonggang@kuaishou.com>
2020-12-23 13:29:22 +00:00
John Mulligan
2870202d06 github: remove mergify rule checking for mimic as we have removed it
The test case is gone from the CI. Remove it from the merigify config
too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-16 15:18:33 +01:00
John Mulligan
574b3584fa readme: document v0.7.0 and the ceph versions it supports
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-14 09:54:45 -05:00
John Mulligan
4cf40e8e0b ci: remove "mimic" from testing matrix
Mimic is not a supported build target in v0.7 and as such there's no
need to keep testing it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-10 09:41:28 -05:00
John Mulligan
8fc7765cfe cephfs admin: add test case for new RemoveSubVolumWithFlags function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
e958f299ec cephfs admin: create a custom flags type for subvolume removal
This flagSet type allows the user to also set the value of the retain
snapshots flag, something unique to subvolume removal (all others had
the force flag only).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
7524b39272 cephfs admin: add flagSet interface and mergeFlags function
To prepare for multiple flags types, we create a flagSet interface and
break out the functionality that was in Update into a per-type method
and then the common "merger" function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
98afaf6376 cephfs admin: rename rmFlags type to commonRmFlags
This is to clarify that for "normal" cases the private commonRmFlags
type is correct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
2f1ee70242 cephfs admin: move rmFlags type to a file for flags handling
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-12-07 14:47:31 +00:00
John Mulligan
36abb07dc5 cephfs: add a test case for calling Statx on an invalid mount
Adding this test case gets test coverage to 100% for path.go.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00
John Mulligan
7cbacf9331 cephfs: add Truncate implementing ceph_truncate
Add a Truncate method that operates on paths.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-30 15:39:09 +00:00