Commit Graph

27 Commits

Author SHA1 Message Date
John Mulligan 0b1e3f5d7e commands: port tracing helper type from cephfs/admin
This debugging code can be made common.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan 6b3b0fdc46 commands: add a test file for new shared Response type
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan 93676ab094 commands: port response type and related to commands pkg
Convert the response type from cephfs/admin to the new package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan 42e9904d0b commands: start a new internal sub-package for working with json commands
This code was part of cephfs/admin but will be shared with a soon-to-be
rbd/admin package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
Sven Anderson 014a3f6308 ptrguard: assert pointer and uintptr have same size
Uintptr are guaranteed to be at least of the size of a pointer.  However, in
theory it could be larger.  Since the code of PtrGuard assumes both have the
same size and to be on the safe side, this change adds a compile-time check,
that the sizes of uintptr and pointers are the same.

Signed-off-by: Sven Anderson <sven@anderson.de>
2021-03-26 11:41:14 +00:00
Sven Anderson 1c72979ce0 ptrguard: assert that //go:uintptrescapes actually works
This adds a test that assures that the special //go:uintptrescapes comment
before the storeUntilRelease() function works as intended, that is the
garbage collector doesn't touch the object referenced by the uintptr until
the function returns after Release() is called.  The test will fail if the
//go:uintptrescapes comment is disabled (removed) or stops working in future
versions of go.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-03-19 09:47:19 -04:00
Sven Anderson b8a803ccbb cutil: use SyncBuffer for iovec type
The main motivation for PtrGuard was read and write buffers as they
are used in iovec.  This change uses SyncBuffer for the iovec
implementation, so that the no-copy PtrGuard implementation can be
enabled with the with_ptrguard build tag.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Sven Anderson 430dea5b7f cutil: add SyncBuffer abstraction layer to PtrGuard
In order to be able to swich on and off PtrGuard usage, SyncBuffer
implements a typical usage pattern of PtrGuard, that is a data buffer
referenced from inside a C allocated struct, in two ways: 1) using a
PtrGuard to dierctly store a pointer to the Go buffer in C memory, or
2) allocating a C buffer of same size and syncing data back with
C.memcpy().  The implementation can be chosen with the with_ptrguard
build tag.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Sven Anderson ac57250a5b cutil: add some useful function aliases
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Sven Anderson 2bfdee2508 cutil: add test for PtrGuard
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 14:50:56 +00:00
Sven Anderson 0451edaf71 cutil: add PtrGuard for storing Go pointers in C memory
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 14:50:56 +00:00
Sven Anderson 0130c60281 cutil: add general C type, function and const aliases
This change adds a new pointer alias CPtr, that should always only
contains pointers to C allocated memory and should always used for
such pointers in order to keep them separate from Go pointers more
easily. This is helpful, since only CPtr are allowed to be stored in
C allocated memory, and Go pointers can only be passed to C functions
if they don't point to memory not containing Go pointers again, that
is all pointers in such memory must be CPtr.

The change also adds aliases for C.malloc and C.free that return and
accept the CPtr type as a little safety measure.

Additionally there are some useful constants defined.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-01-20 14:46:03 +00:00
John Mulligan 29ebb61850 timespec: add a helper function to convert from Timespec to C
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-01-12 14:47:30 +00:00
Sven Anderson c8f26a841b rbd: eliminate invalid unsafe.Pointers
Although it seems to be stable with the current Go versions,
unsafe.Pointers that are created by applying arithmetic on nil
pointers are explicitly not allowed[1], and could potentially lead to
panics by the garbage collector.  To be on the safe side, this change
goes back to minimal inline wrappers for the callback registrations,
which should be zero-cost.  For the callbacks themselves fortunately
no wrappers are required, because CGo happily converts void* to
uintptr arguments.  The potentially problematic VoidPtr helper is
removed again.

[1] https://golang.org/pkg/unsafe/#Pointer

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-11-09 08:58:22 +00:00
Sven Anderson b0ffc1afe8 callbacks: replace index with unique ID
The former code assumed a LIFO usage, and otherwise always produced
collisions.  This change uses unique IDs instead, that as a
side-effect might make debugging easier.

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-11-05 17:16:32 +00:00
Sven Anderson beb3351f24 callbacks: add a benchmark test case
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-10-01 10:03:41 -04:00
John Mulligan ebea82dda5 cutil: add Iovec type wrapping C struct iovec arrays
This type is useful for passing disparate buffers to be read or
written in a single call. Functions using this type exist in cephfs
and rbd. Currently this is needed for cephfs calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-28 10:07:34 -04:00
Sven Anderson 4706453428 rbd: remove unnecessary C wrapper functions
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-08-28 09:51:06 -04:00
Sven Anderson 176a163c3c callbacks: use uintptr instead of int as index
Since the index of the registered callbacks is basically only used as
void*/unsafe.Pointer, it makes more sense and code simpler, if
the index type is uintptr instead of a simple int.

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-08-27 10:26:12 -04:00
John Mulligan 828ea92a51 errutil: replace unused StrError with FormatErrorCode function
This change removes StrError, as it was unused.
Instead we add FormatErrorCode which should replace the common code that
was previously repeated for all error-code types, along with
consistently printing the raw error code value's sign (minus sign when
negative).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-08-10 13:03:24 -04:00
John Mulligan e5d1a53060 cutil: add SplitBuffer and SplitSparseBuffer helper functions
Add SplitBuffer and SplitSparseBuffer functions for extracting a list
of strings from a single buffer, typically returned in C code, from
a single Go buffer. The SplitBuffer variant will return empty strings
if multiple nulls are found in sequence, assuming that the C code
packs data between on single null byte (expect the final byte).
The SplitSparseBuffer variant assumes that the C code may not
tightly pack the data with single null bytes and thus will not
return any empty strings (unless the input buffer is empty
or only contains nulls).
Most of the code in the go-ceph codebase is doing the latter but
probably should have been doing the former. Thus both approaches
are provided.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-07-23 16:38:39 -04:00
Niels de Vos c4714165a6 timespec: move to internal pkg
There are functions from librbd that can use the Timespec type too.
Instead of consuming 'cephfs.Timespec' in the rbd package, create an
internal type that can be exposed through both packages.

This looks a bit like duplication, but it does not break the current
users of Timespec in the cephfs package.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-22 09:22:18 -04:00
John Mulligan e2a78eec02 cutil: allow passing free functions to command output type
The *_command functions in librados and libcephfs document the use
of specific free functions for data allocated. These functions are
currently just wrappers around C's free() function. However, to be
more strictly compliant this change adds a free-function callback
to the CommandOutput type and the specific free functions are now
used outside the unit tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan e40744fdf6 cutil: add a new internal package for c+go utility functions
Now we have sufficient boilerplate in our code for interacting with
various types and ceph calls with similar needs we establish a new
internal package, "cutil" (C utilities).

Note that many of the return types are wrapped. This is due to the
limits placed on us by cgo.  Despite the irritating limitations Go
places on "exporting" C types it still ought to help in the long run for
patterns that are very common or patterns that are subtle and we want to
write specific tests for.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
Sven Anderson e01eb03284 retry: add a helper lib for retrying common operations
Our first operation is generating sizes for various caller-allocated
buffers.  The WithSizes function "suggests" sizes to an anonymous
function that takes a size as an argument and returns a hint how
sucessful the attempt was.  If required the same function is called
again with a larger size.  Errors and other results of that anonymous
function are "communicated" with variables of the surrounding scope
in wich the function is defined.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00
John Mulligan 6b396f70b2 errutil: move errutil pkg into internal subdir
Moving the errutil pkg to "internal/errutil" makes errutil private-like
and accessible to only other go-ceph packages. The functions it provided
were always meant to be used only by go-ceph, this just makes it more
official. This is a breaking change but it was only available to
outside users for 1 release and it is somewhat doubtful users outside
of go-ceph would have reached for these functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-10 08:41:44 +01:00
John Mulligan b3f7c6af15 callbacks: create a helper type for managing callbacks between C and Go
As per the Go Wiki page on cgo [1] passing callback with data between C
and Go is non-trivial due to the pointer passing rules [2]. This change
creates a new helper type, based on the examples on the wiki, that uses
opaque indexes to exchange between C and Go.

For now, this type is simple as we're only going to have a few callbacks
initially but this could easily be improved and made more general in the
future as needed. Because we anticipate having to use this type in all
of our core packages (rados and cephfs as well as rbd) we are creating
the type in an "internal" package so that we don't treat the new type
as public despite the capitalized method names.

[1]: https://github.com/golang/go/wiki/cgo
[2]: https://golang.org/cmd/cgo/#hdr-Passing_pointers

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-03-04 08:10:48 -05:00