revive v1.3.7 added https://github.com/mgechev/revive/pull/966 which
checks for unused parameters in function literals. This caused several
lint errors in go-ceph code.
Signed-off-by: Manish <myathnal@redhat.com>
Since Go 1.21 there is a runtime.Pinner API that allows to safely
pass structures with embedded Go pointers to C code. In earlier Go
version we know that the garbage collector is non-moving, so it is
safe to pass Go pointers to C as well. This change adds two
implementations of PtrGuard, one for pre 1.21 that is basically a
no-op, and one for 1.21+ that uses runtime.Pinner.
Signed-off-by: Sven Anderson <sven@redhat.com>
This call is used for removing "routine" responses that get sent in
the body of a response rather than the status. Otherwise, it is the same
as the FilterPrefix function.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
BufferGroup is a helper structure that holds Go-allocated slices of
C-allocated strings and their respective lengths. Useful for C functions
that consume byte buffers with explicit length instead of null-terminated
strings.
This change adds two log packages. One for the external go-ceph
consumers under common/log, and one for the internal use within the
go-ceph code under internal/log. The external package exports two
functions for consumers, SetWarnf() and SetDebugf(), that set a
logger for warnings and debug messages respectively. They take a
log.Printf compatible function as argument.
The internal package exports these functions as Warnf() and Debugf(),
which can be used by the go-ceph code to log warnings and debug
messages.
Signed-off-by: Sven Anderson <sven@redhat.com>
All of our <foo>/admin/ test suites have similar needs for rados conn
and wrapped debugging. Create a short package that is meant to only
be consumed by our test suites.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This commit implements binding for rados_write_op_cmpext RADOS Write operation.
Includes a unit test.
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
The finalizer of an object is called in a separate go routine, so we
might have to check several times for its completion.
Fixes: #545
Signed-off-by: Sven Anderson <sven@redhat.com>
This change uses slices on top of C allocated array memory in order
to have a simple (no pointer arithmetic) and safe (boundary-checked)
access to its elements.
Signed-off-by: Sven Anderson <sven@redhat.com>
Uintptr are guaranteed to be at least of the size of a pointer. However, in
theory it could be larger. Since the code of PtrGuard assumes both have the
same size and to be on the safe side, this change adds a compile-time check,
that the sizes of uintptr and pointers are the same.
Signed-off-by: Sven Anderson <sven@anderson.de>
This adds a test that assures that the special //go:uintptrescapes comment
before the storeUntilRelease() function works as intended, that is the
garbage collector doesn't touch the object referenced by the uintptr until
the function returns after Release() is called. The test will fail if the
//go:uintptrescapes comment is disabled (removed) or stops working in future
versions of go.
Signed-off-by: Sven Anderson <sven@redhat.com>
The main motivation for PtrGuard was read and write buffers as they
are used in iovec. This change uses SyncBuffer for the iovec
implementation, so that the no-copy PtrGuard implementation can be
enabled with the with_ptrguard build tag.
Signed-off-by: Sven Anderson <sven@redhat.com>
In order to be able to swich on and off PtrGuard usage, SyncBuffer
implements a typical usage pattern of PtrGuard, that is a data buffer
referenced from inside a C allocated struct, in two ways: 1) using a
PtrGuard to dierctly store a pointer to the Go buffer in C memory, or
2) allocating a C buffer of same size and syncing data back with
C.memcpy(). The implementation can be chosen with the with_ptrguard
build tag.
Signed-off-by: Sven Anderson <sven@redhat.com>
This change adds a new pointer alias CPtr, that should always only
contains pointers to C allocated memory and should always used for
such pointers in order to keep them separate from Go pointers more
easily. This is helpful, since only CPtr are allowed to be stored in
C allocated memory, and Go pointers can only be passed to C functions
if they don't point to memory not containing Go pointers again, that
is all pointers in such memory must be CPtr.
The change also adds aliases for C.malloc and C.free that return and
accept the CPtr type as a little safety measure.
Additionally there are some useful constants defined.
Signed-off-by: Sven Anderson <sven@redhat.com>
Although it seems to be stable with the current Go versions,
unsafe.Pointers that are created by applying arithmetic on nil
pointers are explicitly not allowed[1], and could potentially lead to
panics by the garbage collector. To be on the safe side, this change
goes back to minimal inline wrappers for the callback registrations,
which should be zero-cost. For the callbacks themselves fortunately
no wrappers are required, because CGo happily converts void* to
uintptr arguments. The potentially problematic VoidPtr helper is
removed again.
[1] https://golang.org/pkg/unsafe/#Pointer
Signed-off-by: Sven Anderson <sven@redhat.com>
The former code assumed a LIFO usage, and otherwise always produced
collisions. This change uses unique IDs instead, that as a
side-effect might make debugging easier.
Signed-off-by: Sven Anderson <sven@redhat.com>
This type is useful for passing disparate buffers to be read or
written in a single call. Functions using this type exist in cephfs
and rbd. Currently this is needed for cephfs calls.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Since the index of the registered callbacks is basically only used as
void*/unsafe.Pointer, it makes more sense and code simpler, if
the index type is uintptr instead of a simple int.
Signed-off-by: Sven Anderson <sven@redhat.com>
This change removes StrError, as it was unused.
Instead we add FormatErrorCode which should replace the common code that
was previously repeated for all error-code types, along with
consistently printing the raw error code value's sign (minus sign when
negative).
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add SplitBuffer and SplitSparseBuffer functions for extracting a list
of strings from a single buffer, typically returned in C code, from
a single Go buffer. The SplitBuffer variant will return empty strings
if multiple nulls are found in sequence, assuming that the C code
packs data between on single null byte (expect the final byte).
The SplitSparseBuffer variant assumes that the C code may not
tightly pack the data with single null bytes and thus will not
return any empty strings (unless the input buffer is empty
or only contains nulls).
Most of the code in the go-ceph codebase is doing the latter but
probably should have been doing the former. Thus both approaches
are provided.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
There are functions from librbd that can use the Timespec type too.
Instead of consuming 'cephfs.Timespec' in the rbd package, create an
internal type that can be exposed through both packages.
This looks a bit like duplication, but it does not break the current
users of Timespec in the cephfs package.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The *_command functions in librados and libcephfs document the use
of specific free functions for data allocated. These functions are
currently just wrappers around C's free() function. However, to be
more strictly compliant this change adds a free-function callback
to the CommandOutput type and the specific free functions are now
used outside the unit tests.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Now we have sufficient boilerplate in our code for interacting with
various types and ceph calls with similar needs we establish a new
internal package, "cutil" (C utilities).
Note that many of the return types are wrapped. This is due to the
limits placed on us by cgo. Despite the irritating limitations Go
places on "exporting" C types it still ought to help in the long run for
patterns that are very common or patterns that are subtle and we want to
write specific tests for.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Our first operation is generating sizes for various caller-allocated
buffers. The WithSizes function "suggests" sizes to an anonymous
function that takes a size as an argument and returns a hint how
sucessful the attempt was. If required the same function is called
again with a larger size. Errors and other results of that anonymous
function are "communicated" with variables of the surrounding scope
in wich the function is defined.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
Moving the errutil pkg to "internal/errutil" makes errutil private-like
and accessible to only other go-ceph packages. The functions it provided
were always meant to be used only by go-ceph, this just makes it more
official. This is a breaking change but it was only available to
outside users for 1 release and it is somewhat doubtful users outside
of go-ceph would have reached for these functions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
As per the Go Wiki page on cgo [1] passing callback with data between C
and Go is non-trivial due to the pointer passing rules [2]. This change
creates a new helper type, based on the examples on the wiki, that uses
opaque indexes to exchange between C and Go.
For now, this type is simple as we're only going to have a few callbacks
initially but this could easily be improved and made more general in the
future as needed. Because we anticipate having to use this type in all
of our core packages (rados and cephfs as well as rbd) we are creating
the type in an "internal" package so that we don't treat the new type
as public despite the capitalized method names.
[1]: https://github.com/golang/go/wiki/cgo
[2]: https://golang.org/cmd/cgo/#hdr-Passing_pointers
Signed-off-by: John Mulligan <jmulligan@redhat.com>
