avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
/*
|
|
|
|
* This file is part of FFmpeg.
|
|
|
|
*
|
|
|
|
* FFmpeg is free software; you can redistribute it and/or
|
|
|
|
* modify it under the terms of the GNU Lesser General Public
|
|
|
|
* License as published by the Free Software Foundation; either
|
|
|
|
* version 2.1 of the License, or (at your option) any later version.
|
|
|
|
*
|
|
|
|
* FFmpeg is distributed in the hope that it will be useful,
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
* Lesser General Public License for more details.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU Lesser General Public
|
|
|
|
* License along with FFmpeg; if not, write to the Free Software
|
|
|
|
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <stdatomic.h>
|
|
|
|
#include <stdint.h>
|
|
|
|
#include <string.h>
|
|
|
|
|
|
|
|
#include "internal.h"
|
|
|
|
#include "refstruct.h"
|
|
|
|
|
|
|
|
#include "libavutil/avassert.h"
|
2022-08-05 16:32:23 +00:00
|
|
|
#include "libavutil/error.h"
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
#include "libavutil/macros.h"
|
|
|
|
#include "libavutil/mem.h"
|
2022-08-05 16:32:23 +00:00
|
|
|
#include "libavutil/thread.h"
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
|
|
|
|
#ifndef REFSTRUCT_CHECKED
|
|
|
|
#ifndef ASSERT_LEVEL
|
|
|
|
#define ASSERT_LEVEL 0
|
|
|
|
#endif
|
|
|
|
#define REFSTRUCT_CHECKED (ASSERT_LEVEL >= 1)
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#if REFSTRUCT_CHECKED
|
|
|
|
#define ff_assert(cond) av_assert0(cond)
|
|
|
|
#else
|
|
|
|
#define ff_assert(cond) ((void)0)
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#define REFSTRUCT_COOKIE AV_NE((uint64_t)MKBETAG('R', 'e', 'f', 'S') << 32 | MKBETAG('t', 'r', 'u', 'c'), \
|
|
|
|
MKTAG('R', 'e', 'f', 'S') | (uint64_t)MKTAG('t', 'r', 'u', 'c') << 32)
|
|
|
|
|
2024-02-05 18:48:26 +00:00
|
|
|
#if __STDC_VERSION__ >= 201112L && !defined(_MSC_VER)
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
#define REFCOUNT_OFFSET FFALIGN(sizeof(RefCount), FFMAX3(STRIDE_ALIGN, 16, _Alignof(max_align_t)))
|
|
|
|
#else
|
|
|
|
#define REFCOUNT_OFFSET FFALIGN(sizeof(RefCount), FFMAX(STRIDE_ALIGN, 16))
|
|
|
|
#endif
|
|
|
|
|
|
|
|
typedef struct RefCount {
|
|
|
|
/**
|
|
|
|
* An uintptr_t is big enough to hold the address of every reference,
|
|
|
|
* so no overflow can happen when incrementing the refcount as long as
|
|
|
|
* the user does not throw away references.
|
|
|
|
*/
|
|
|
|
atomic_uintptr_t refcount;
|
|
|
|
FFRefStructOpaque opaque;
|
|
|
|
void (*free_cb)(FFRefStructOpaque opaque, void *obj);
|
2022-08-05 16:32:23 +00:00
|
|
|
void (*free)(void *ref);
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
|
|
|
|
#if REFSTRUCT_CHECKED
|
|
|
|
uint64_t cookie;
|
|
|
|
#endif
|
|
|
|
} RefCount;
|
|
|
|
|
|
|
|
static RefCount *get_refcount(void *obj)
|
|
|
|
{
|
|
|
|
RefCount *ref = (RefCount*)((char*)obj - REFCOUNT_OFFSET);
|
|
|
|
ff_assert(ref->cookie == REFSTRUCT_COOKIE);
|
|
|
|
return ref;
|
|
|
|
}
|
|
|
|
|
2022-08-04 06:40:57 +00:00
|
|
|
static const RefCount *cget_refcount(const void *obj)
|
|
|
|
{
|
|
|
|
const RefCount *ref = (const RefCount*)((const char*)obj - REFCOUNT_OFFSET);
|
|
|
|
ff_assert(ref->cookie == REFSTRUCT_COOKIE);
|
|
|
|
return ref;
|
|
|
|
}
|
|
|
|
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
static void *get_userdata(void *buf)
|
|
|
|
{
|
|
|
|
return (char*)buf + REFCOUNT_OFFSET;
|
|
|
|
}
|
|
|
|
|
|
|
|
static void refcount_init(RefCount *ref, FFRefStructOpaque opaque,
|
|
|
|
void (*free_cb)(FFRefStructOpaque opaque, void *obj))
|
|
|
|
{
|
|
|
|
atomic_init(&ref->refcount, 1);
|
|
|
|
ref->opaque = opaque;
|
|
|
|
ref->free_cb = free_cb;
|
2022-08-05 16:32:23 +00:00
|
|
|
ref->free = av_free;
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
|
|
|
|
#if REFSTRUCT_CHECKED
|
|
|
|
ref->cookie = REFSTRUCT_COOKIE;
|
|
|
|
#endif
|
|
|
|
}
|
|
|
|
|
|
|
|
void *ff_refstruct_alloc_ext_c(size_t size, unsigned flags, FFRefStructOpaque opaque,
|
|
|
|
void (*free_cb)(FFRefStructOpaque opaque, void *obj))
|
|
|
|
{
|
|
|
|
void *buf, *obj;
|
|
|
|
|
|
|
|
if (size > SIZE_MAX - REFCOUNT_OFFSET)
|
|
|
|
return NULL;
|
|
|
|
buf = av_malloc(size + REFCOUNT_OFFSET);
|
|
|
|
if (!buf)
|
|
|
|
return NULL;
|
|
|
|
refcount_init(buf, opaque, free_cb);
|
|
|
|
obj = get_userdata(buf);
|
|
|
|
if (!(flags & FF_REFSTRUCT_FLAG_NO_ZEROING))
|
|
|
|
memset(obj, 0, size);
|
|
|
|
|
|
|
|
return obj;
|
|
|
|
}
|
|
|
|
|
|
|
|
void ff_refstruct_unref(void *objp)
|
|
|
|
{
|
|
|
|
void *obj;
|
|
|
|
RefCount *ref;
|
|
|
|
|
|
|
|
memcpy(&obj, objp, sizeof(obj));
|
|
|
|
if (!obj)
|
|
|
|
return;
|
|
|
|
memcpy(objp, &(void *){ NULL }, sizeof(obj));
|
|
|
|
|
|
|
|
ref = get_refcount(obj);
|
|
|
|
if (atomic_fetch_sub_explicit(&ref->refcount, 1, memory_order_acq_rel) == 1) {
|
|
|
|
if (ref->free_cb)
|
|
|
|
ref->free_cb(ref->opaque, obj);
|
2022-08-05 16:32:23 +00:00
|
|
|
ref->free(ref);
|
avcodec/refstruct: Add simple API for refcounted objects
For now, this API is supposed to replace all the internal uses
of reference counted objects in libavcodec; "internal" here
means that the object is created in libavcodec and is never
put directly in the hands of anyone outside of it.
It is intended to be made public eventually, but for now
I enjoy the ability to modify it freely.
Several shortcomings of the AVBuffer API motivated this API:
a) The unnecessary allocations (and ensuing error checks)
when using the API. Besides the need for runtime checks it
imposes upon the developer the burden of thinking through
what happens in case an error happens. Furthermore, these
error paths are typically not covered by FATE.
b) The AVBuffer API is designed with buffers and not with
objects in mind: The type for the actual buffers used
is uint8_t*; it pretends to be able to make buffers
writable, but this is wrong in case the buffer is not a POD.
Another instance of this thinking is the lack of a reset
callback in the AVBufferPool API.
c) The AVBuffer API incurs unnecessary indirections by
going through the AVBufferRef.data pointer. In case the user
tries to avoid this indirection and stores a pointer to
AVBuffer.data separately (which also allows to use the correct
type), the user has to keep these two pointers in sync
in case they can change (and in any case has two pointers
occupying space in the containing context). See the following
commit using this API for H.264 parameter sets for an example
of the removal of such syncing code as well as the casts
involved in the parts where only the AVBufferRef* pointer
was stored.
d) Given that the AVBuffer API allows custom allocators,
creating refcounted objects with dedicated free functions
often involves a lot of boilerplate like this:
obj = av_mallocz(sizeof(*obj));
ref = av_buffer_create((uint8_t*)obj, sizeof(*obj), free_func, opaque, 0);
if (!ref) {
av_free(obj);
return AVERROR(ENOMEM);
}
(There is also a corresponding av_free() at the end of free_func().)
This is now just
obj = ff_refstruct_alloc_ext(sizeof(*obj), 0, opaque, free_func);
if (!obj)
return AVERROR(ENOMEM);
See the subsequent patch for the framepool (i.e. get_buffer.c)
for an example.
This API does things differently; it is designed to be lightweight*
as well as geared to the common case where the allocator of the
underlying object does not matter as long as it is big enough and
suitably aligned. This allows to allocate the user data together
with the API's bookkeeping data which avoids an allocation as well
as the need for separate pointers to the user data and the API's
bookkeeping data. This entails that the actual allocation of the
object is performed by RefStruct, not the user. This is responsible
for avoiding the boilerplate code mentioned in d).
As a downside, custom allocators are not supported, but it will
become apparent in subsequent commits that there are enough
usecases to make it worthwhile.
Another advantage of this API is that one only needs to include
the relevant header if one uses the API and not when one includes
the header or some other component that uses it. This is because there
is no RefStruct type analog of AVBufferRef. This brings with it
one further downside: It is not apparent from the pointer itself
whether the underlying object is managed by the RefStruct API
or whether this pointer is a reference to it (or merely a pointer
to it).
Finally, this API supports const-qualified opaque pointees;
this will allow to avoid casting const away by the CBS code.
*: Basically the only exception to the you-only-pay-for-what-you-use
rule is that it always uses atomics for the refcount.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 01:07:58 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
void *ff_refstruct_ref(void *obj)
|
|
|
|
{
|
|
|
|
RefCount *ref = get_refcount(obj);
|
|
|
|
|
|
|
|
atomic_fetch_add_explicit(&ref->refcount, 1, memory_order_relaxed);
|
|
|
|
|
|
|
|
return obj;
|
|
|
|
}
|
|
|
|
|
|
|
|
const void *ff_refstruct_ref_c(const void *obj)
|
|
|
|
{
|
|
|
|
/* Casting const away here is fine, as it is only supposed
|
|
|
|
* to apply to the user's data and not our bookkeeping data. */
|
|
|
|
RefCount *ref = get_refcount((void*)obj);
|
|
|
|
|
|
|
|
atomic_fetch_add_explicit(&ref->refcount, 1, memory_order_relaxed);
|
|
|
|
|
|
|
|
return obj;
|
|
|
|
}
|
|
|
|
|
|
|
|
void ff_refstruct_replace(void *dstp, const void *src)
|
|
|
|
{
|
|
|
|
const void *dst;
|
|
|
|
memcpy(&dst, dstp, sizeof(dst));
|
|
|
|
|
|
|
|
if (src == dst)
|
|
|
|
return;
|
|
|
|
ff_refstruct_unref(dstp);
|
|
|
|
if (src) {
|
|
|
|
dst = ff_refstruct_ref_c(src);
|
|
|
|
memcpy(dstp, &dst, sizeof(dst));
|
|
|
|
}
|
|
|
|
}
|
2022-08-04 06:40:57 +00:00
|
|
|
|
|
|
|
int ff_refstruct_exclusive(const void *obj)
|
|
|
|
{
|
|
|
|
const RefCount *ref = cget_refcount(obj);
|
|
|
|
/* Casting const away here is safe, because it is a load.
|
|
|
|
* It is necessary because atomic_load_explicit() does not
|
|
|
|
* accept const atomics in C11 (see also N1807). */
|
|
|
|
return atomic_load_explicit((atomic_uintptr_t*)&ref->refcount, memory_order_acquire) == 1;
|
|
|
|
}
|
2022-08-05 16:32:23 +00:00
|
|
|
|
|
|
|
struct FFRefStructPool {
|
|
|
|
size_t size;
|
|
|
|
FFRefStructOpaque opaque;
|
|
|
|
int (*init_cb)(FFRefStructOpaque opaque, void *obj);
|
|
|
|
void (*reset_cb)(FFRefStructOpaque opaque, void *obj);
|
|
|
|
void (*free_entry_cb)(FFRefStructOpaque opaque, void *obj);
|
|
|
|
void (*free_cb)(FFRefStructOpaque opaque);
|
|
|
|
|
|
|
|
int uninited;
|
|
|
|
unsigned entry_flags;
|
|
|
|
unsigned pool_flags;
|
|
|
|
|
|
|
|
/** The number of outstanding entries not in available_entries. */
|
|
|
|
atomic_uintptr_t refcount;
|
|
|
|
/**
|
|
|
|
* This is a linked list of available entries;
|
|
|
|
* the RefCount's opaque pointer is used as next pointer
|
|
|
|
* for available entries.
|
|
|
|
* While the entries are in use, the opaque is a pointer
|
|
|
|
* to the corresponding FFRefStructPool.
|
|
|
|
*/
|
|
|
|
RefCount *available_entries;
|
|
|
|
AVMutex mutex;
|
|
|
|
};
|
|
|
|
|
|
|
|
static void pool_free(FFRefStructPool *pool)
|
|
|
|
{
|
|
|
|
ff_mutex_destroy(&pool->mutex);
|
|
|
|
if (pool->free_cb)
|
|
|
|
pool->free_cb(pool->opaque);
|
2022-08-16 12:04:27 +00:00
|
|
|
av_free(get_refcount(pool));
|
2022-08-05 16:32:23 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void pool_free_entry(FFRefStructPool *pool, RefCount *ref)
|
|
|
|
{
|
|
|
|
if (pool->free_entry_cb)
|
|
|
|
pool->free_entry_cb(pool->opaque, get_userdata(ref));
|
|
|
|
av_free(ref);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void pool_return_entry(void *ref_)
|
|
|
|
{
|
|
|
|
RefCount *ref = ref_;
|
|
|
|
FFRefStructPool *pool = ref->opaque.nc;
|
|
|
|
|
|
|
|
ff_mutex_lock(&pool->mutex);
|
|
|
|
if (!pool->uninited) {
|
|
|
|
ref->opaque.nc = pool->available_entries;
|
|
|
|
pool->available_entries = ref;
|
|
|
|
ref = NULL;
|
|
|
|
}
|
|
|
|
ff_mutex_unlock(&pool->mutex);
|
|
|
|
|
|
|
|
if (ref)
|
|
|
|
pool_free_entry(pool, ref);
|
|
|
|
|
|
|
|
if (atomic_fetch_sub_explicit(&pool->refcount, 1, memory_order_acq_rel) == 1)
|
|
|
|
pool_free(pool);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void pool_reset_entry(FFRefStructOpaque opaque, void *entry)
|
|
|
|
{
|
|
|
|
FFRefStructPool *pool = opaque.nc;
|
|
|
|
|
|
|
|
pool->reset_cb(pool->opaque, entry);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int refstruct_pool_get_ext(void *datap, FFRefStructPool *pool)
|
|
|
|
{
|
|
|
|
void *ret = NULL;
|
|
|
|
|
|
|
|
memcpy(datap, &(void *){ NULL }, sizeof(void*));
|
|
|
|
|
|
|
|
ff_mutex_lock(&pool->mutex);
|
|
|
|
ff_assert(!pool->uninited);
|
|
|
|
if (pool->available_entries) {
|
|
|
|
RefCount *ref = pool->available_entries;
|
|
|
|
ret = get_userdata(ref);
|
|
|
|
pool->available_entries = ref->opaque.nc;
|
|
|
|
ref->opaque.nc = pool;
|
|
|
|
atomic_init(&ref->refcount, 1);
|
|
|
|
}
|
|
|
|
ff_mutex_unlock(&pool->mutex);
|
|
|
|
|
|
|
|
if (!ret) {
|
|
|
|
RefCount *ref;
|
|
|
|
ret = ff_refstruct_alloc_ext(pool->size, pool->entry_flags, pool,
|
|
|
|
pool->reset_cb ? pool_reset_entry : NULL);
|
|
|
|
if (!ret)
|
|
|
|
return AVERROR(ENOMEM);
|
|
|
|
ref = get_refcount(ret);
|
|
|
|
ref->free = pool_return_entry;
|
|
|
|
if (pool->init_cb) {
|
|
|
|
int err = pool->init_cb(pool->opaque, ret);
|
|
|
|
if (err < 0) {
|
|
|
|
if (pool->pool_flags & FF_REFSTRUCT_POOL_FLAG_RESET_ON_INIT_ERROR)
|
|
|
|
pool->reset_cb(pool->opaque, ret);
|
|
|
|
if (pool->pool_flags & FF_REFSTRUCT_POOL_FLAG_FREE_ON_INIT_ERROR)
|
|
|
|
pool->free_entry_cb(pool->opaque, ret);
|
|
|
|
av_free(ref);
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
atomic_fetch_add_explicit(&pool->refcount, 1, memory_order_relaxed);
|
2022-08-06 11:04:10 +00:00
|
|
|
|
|
|
|
if (pool->pool_flags & FF_REFSTRUCT_POOL_FLAG_ZERO_EVERY_TIME)
|
|
|
|
memset(ret, 0, pool->size);
|
|
|
|
|
2022-08-05 16:32:23 +00:00
|
|
|
memcpy(datap, &ret, sizeof(ret));
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
void *ff_refstruct_pool_get(FFRefStructPool *pool)
|
|
|
|
{
|
|
|
|
void *ret;
|
|
|
|
refstruct_pool_get_ext(&ret, pool);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2022-08-16 12:04:27 +00:00
|
|
|
/**
|
|
|
|
* Hint: The content of pool_unref() and refstruct_pool_uninit()
|
|
|
|
* could currently be merged; they are only separate functions
|
|
|
|
* in case we would ever introduce weak references.
|
|
|
|
*/
|
|
|
|
static void pool_unref(void *ref)
|
2022-08-05 16:32:23 +00:00
|
|
|
{
|
2022-08-16 12:04:27 +00:00
|
|
|
FFRefStructPool *pool = get_userdata(ref);
|
|
|
|
if (atomic_fetch_sub_explicit(&pool->refcount, 1, memory_order_acq_rel) == 1)
|
|
|
|
pool_free(pool);
|
|
|
|
}
|
2022-08-05 16:32:23 +00:00
|
|
|
|
2022-08-16 12:04:27 +00:00
|
|
|
static void refstruct_pool_uninit(FFRefStructOpaque unused, void *obj)
|
|
|
|
{
|
|
|
|
FFRefStructPool *pool = obj;
|
|
|
|
RefCount *entry;
|
2022-08-05 16:32:23 +00:00
|
|
|
|
|
|
|
ff_mutex_lock(&pool->mutex);
|
|
|
|
ff_assert(!pool->uninited);
|
|
|
|
pool->uninited = 1;
|
|
|
|
entry = pool->available_entries;
|
|
|
|
pool->available_entries = NULL;
|
|
|
|
ff_mutex_unlock(&pool->mutex);
|
|
|
|
|
|
|
|
while (entry) {
|
|
|
|
void *next = entry->opaque.nc;
|
|
|
|
pool_free_entry(pool, entry);
|
|
|
|
entry = next;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
FFRefStructPool *ff_refstruct_pool_alloc(size_t size, unsigned flags)
|
|
|
|
{
|
|
|
|
return ff_refstruct_pool_alloc_ext(size, flags, NULL, NULL, NULL, NULL, NULL);
|
|
|
|
}
|
|
|
|
|
|
|
|
FFRefStructPool *ff_refstruct_pool_alloc_ext_c(size_t size, unsigned flags,
|
|
|
|
FFRefStructOpaque opaque,
|
|
|
|
int (*init_cb)(FFRefStructOpaque opaque, void *obj),
|
|
|
|
void (*reset_cb)(FFRefStructOpaque opaque, void *obj),
|
|
|
|
void (*free_entry_cb)(FFRefStructOpaque opaque, void *obj),
|
|
|
|
void (*free_cb)(FFRefStructOpaque opaque))
|
|
|
|
{
|
2022-08-16 12:04:27 +00:00
|
|
|
FFRefStructPool *pool = ff_refstruct_alloc_ext(sizeof(*pool), 0, NULL,
|
|
|
|
refstruct_pool_uninit);
|
2022-08-05 16:32:23 +00:00
|
|
|
int err;
|
|
|
|
|
|
|
|
if (!pool)
|
|
|
|
return NULL;
|
2022-08-16 12:04:27 +00:00
|
|
|
get_refcount(pool)->free = pool_unref;
|
2022-08-05 16:32:23 +00:00
|
|
|
|
|
|
|
pool->size = size;
|
|
|
|
pool->opaque = opaque;
|
|
|
|
pool->init_cb = init_cb;
|
|
|
|
pool->reset_cb = reset_cb;
|
|
|
|
pool->free_entry_cb = free_entry_cb;
|
|
|
|
pool->free_cb = free_cb;
|
|
|
|
#define COMMON_FLAGS FF_REFSTRUCT_POOL_FLAG_NO_ZEROING
|
|
|
|
pool->entry_flags = flags & COMMON_FLAGS;
|
|
|
|
// Filter out nonsense combinations to avoid checks later.
|
|
|
|
if (!pool->reset_cb)
|
|
|
|
flags &= ~FF_REFSTRUCT_POOL_FLAG_RESET_ON_INIT_ERROR;
|
|
|
|
if (!pool->free_entry_cb)
|
|
|
|
flags &= ~FF_REFSTRUCT_POOL_FLAG_FREE_ON_INIT_ERROR;
|
|
|
|
pool->pool_flags = flags;
|
|
|
|
|
2022-08-06 11:04:10 +00:00
|
|
|
if (flags & FF_REFSTRUCT_POOL_FLAG_ZERO_EVERY_TIME) {
|
|
|
|
// We will zero the buffer before every use, so zeroing
|
|
|
|
// upon allocating the buffer is unnecessary.
|
|
|
|
pool->entry_flags |= FF_REFSTRUCT_FLAG_NO_ZEROING;
|
|
|
|
}
|
|
|
|
|
2022-08-05 16:32:23 +00:00
|
|
|
atomic_init(&pool->refcount, 1);
|
|
|
|
|
|
|
|
err = ff_mutex_init(&pool->mutex, NULL);
|
|
|
|
if (err) {
|
2022-08-16 12:04:27 +00:00
|
|
|
// Don't call ff_refstruct_uninit() on pool, as it hasn't been properly
|
|
|
|
// set up and is just a POD right now.
|
|
|
|
av_free(get_refcount(pool));
|
2022-08-05 16:32:23 +00:00
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
return pool;
|
|
|
|
}
|