Previous changes added support for operating on omap keys that include
null characters (e.g. rados_omap_get_next2,
rados_write_op_omap_rm_keys2). This change adds some basic test
coverage those operations' null character support.
Signed-off-by: Nick Janus <github@nondesignated.com>
The previous `rados_nobjects_list_open` used for listing could not
return oids that included null characters. Although this isn't a common
case, having go-ceph able to list such objects is useful for bugs such
as https://tracker.ceph.com/issues/48874.
Signed-off-by: Nick Janus <github@nondesignated.com>
Our test container does not support an orchestrator to pre configure
the ".nfs" configuration pool for us. This is something that rook and
cephadm are expected to do before exports are created, etc.
So we support creating a minimal "mock" configuration in rados.
This option can be toggled on and off via the GO_CEPH_TEST_MOCK_NFS
env var. Turning it off can be useful if you want to run the live
suite against a "real" ceph cluster.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This call is used for removing "routine" responses that get sent in
the body of a response rather than the status. Otherwise, it is the same
as the FilterPrefix function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
THIS POTENTIALLY BREAKY DOWNSTREAM CODE (as it changes the types
of exported fields in an existing, exported struct).
The fields of the User structure representing the Subuser information
have been specified to parse this data strictly and typesafe.
The fields SwiftKeys and Subusers need the url:"-" annotation,
because otherwise url-keys in that substructure would clash with
those in the User structure.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Only one of the two code paths of buildQueryPath was tested so far.
This makes the test harness tight by testing the second one (where
the path already contains a query parameter).
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
The following changes have been done:
* Up until now everything in the argument objects was serialized to
the API calls. This was updated to restrict the serialization to
the API call parameters that are parsed in the Ceph RGW source code.
Parameters not yet supported by us are documented as comments.
Note, that a superset of the documented parameters is supported.
Documentation for the API:
<https://docs.ceph.com/en/pacific/radosgw/adminops/>
Link to the used source tree:
<193895ffba/src/rgw>
The argument parsing happens in the rgw_rest_*.cc files.
* The serialization code (valueToURLParams) has been updated to
be more in line with other serialization methods:
- A tag "-" causes the field to be ignored
- Only the first item in a list of tag items is interpreted as
name.
- The handling of pointer and direct data types has been
harmonized (the same rules for the names and value apply now).
* There is still room for improvement to make things more consistent:
A pointer to a non-elementary data type will emit unexpected
request parameters.
* Presence of required parameters is not validated by the library,
this is left to the API.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
WriteOp.RmOmapKeys now uses rados_write_op_omap_rm_keys2().
The entirety of removeOmapKeysStep struct was removed as it's not necessary for
it to own the various C-allocated values passed to
rados_write_op_omap_rm_keys2() as arguments. rados_write_op_omap_rm_keys2()
makes copies of all the string-based args [1], so it's sufficient for them to
be short lived.
[1] 2bd3dd512a/src/librados/librados_c.cc (L3888-L3902)
WriteOp.SetOmap now uses rados_write_op_omap_set2().
The entirety of setOmapStep struct was removed as it's not necessary for it to
own the various C-allocated values passed to rados_write_op_omap_set2()
as arguments. rados_write_op_omap_set2() makes copies of all the string-based
args [1], so it's sufficient for them to be short lived.
[1] 2bd3dd512a/src/librados/librados_c.cc (L3852-L3871)
BufferGroup is a helper structure that holds Go-allocated slices of
C-allocated strings and their respective lengths. Useful for C functions
that consume byte buffers with explicit length instead of null-terminated
strings.
add wrapper to set locator key on IOContext
Signed-off-by: Alexander Pücker <alexander.puecker@hetzner.com>
* mark API as PREVIEW and update docs
Signed-off-by: Alexander Pücker <alexander.puecker@hetzner.com>
* move new API call into own file to add preview build tag
Signed-off-by: Alexander Pücker <alexander.puecker@hetzner.com>
* reset locater with nil instead of a empty string
Signed-off-by: Alexander Pücker <alexander.puecker@hetzner.com>
* rename variables for naming consistency
Signed-off-by: Alexander Pücker <alexander.puecker@hetzner.com>
* update release versions for new API call
Signed-off-by: Alexander Pücker <alexander.puecker@hetzner.com>
This change adds two log packages. One for the external go-ceph
consumers under common/log, and one for the internal use within the
go-ceph code under internal/log. The external package exports two
functions for consumers, SetWarnf() and SetDebugf(), that set a
logger for warnings and debug messages respectively. They take a
log.Printf compatible function as argument.
The internal package exports these functions as Warnf() and Debugf(),
which can be used by the go-ceph code to log warnings and debug
messages.
Signed-off-by: Sven Anderson <sven@redhat.com>
This change implements the bindings for the watch/notify APIs of
librados. Instead of callbacks, the watcher is implemented with more
Go-idiomatic channels. A watcher object exposes two read-only
channels, one to receive the notify events and one to receive
occuring errors.
Signed-off-by: Sven Anderson <sven@redhat.com>
The comments above Read and Write were very old and not up to our
current standards. Update them to be accurate and note that the
offset internal to the image type is not concurrency safe.
This also cleans up some old and unhelpful todos.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This commit implements binding for read_op_omap_get_vals_by_keys
RADOS Read operation.
Includes a unit test.
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
This commit implements binding for rados_write_op_setxattr
RADOS Write operation.
Includes a unit test.
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
These APIs already existed, and were considered stable in cephfs/admin
before getting moved. I initially was going to mark them preview but then
realized that cephfs/admin depends on them and having them marked
preview with the ceph_preview tag would break cephfs/admin. Due to the
above, I think an exception to the policy is warranted here as there
is not much new except the package itself.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Nick Janus
John Mulligan
Sven Anderson
Sebastian Riese
mergify-bot
Robert Vasek