This is the first in a series of changes that will introduce a drmprime
hwdec. As our vaapi hwdec is based around exporting surfaces as
drmprime dmabufs, we've actually got a lot of useful code already in
place in the GL/PL interops. I'm going to reorganise and adjust this
code to make the interops usable with the new hwdec as well.
The first step is to rename the files and functions. There are no
functional or other changes here. They will come next.
This is somewhat academic for now, as we explicitly ask for separate
layers and the scenarios where multi-plane images are required also use
complex formats that cannot be decomposed after the fact, but
nevertheless it is possible for us to consume simple multi-plane
images where there is one layer with n planes instead of n layers with
one plane each.
In these cases, we just treat the planes the same as we would if they
were each in a separate layer and everything works out.
It ought to be possible to make this work for OpenGL but I couldn't
wrap my head around how to provide the right DRM fourcc when
pretending a plane is a layer by itself. So I've left that
unimplemented.
Our logging here today is very poor. We don't make it clear what
formats we are probing, or even that a certain format failed in most
cases. In the case where we do log the error, we don't make it clear
which format it was that failed.
The end result is that we have no idea what the possible and final
format spaces are, which makes it very hard to debug whether things are
working correctly, or to work on supporting additional formats.
There's really nothing vulkan-specific about this hwdec wrapper, and it
actually works perfectly fine with an OpenGL-based ra_pl. This is not
hugely important at the time, but I still think it makes sense in case
we ever decide to make vo_gpu_next wrap OpenGL contexts to ra_pl instead
of exposing the underlying ra_gl.
While testing support for the exotic formats used by Intel vaapi for
4:2:2 and 4:4:4 surfaces, I realised that we were enumerating all
endpoints and checking formats for them. The problem with this is
that decoding (VLD) endpoints are only a subset of what vaapi exposes.
All the encoding endpoints are there too, and there is also the
None profile that we don't care about, but which generates ffmpeg
warnings if you try and examine it.
So, let's only look at VLD endpoints. This will speed things up a
little bit and make the logging less noisy.
As documented on struct mp_hwdec_ctx, hw_imgfmt specifies the hardware
surface wrapper format for which supported_formats is valid. If this was
not set, f_hwtransfer ignored supported_formats, and assumed all formats
were supported.
hwdec_vaapi tries to probe all available surface formats in advance. For
that, we iterate over _all_ profiles in an attempt to collect possible
surface formats. This means we try profiles we normally wouldn't use for
decoding or filtering, and which could be "unrelated" services.
It seems some drivers report at least one profile, for which
vaQueryConfigEntrypoints() fails (because the profile is not supported;
not sure why it lists it, then). So turn the error message into a
verbose message to avoid confusing output.
Fixes: #7347
In vaapi 1.1.0 (which confusingly is libva release 2.1.0), they
introduced a new surface export API that is more efficient, and
we've been supporting that and the old API ever since (Feb 2018).
If we drop support for the old API, we can do some fairly nice cleanup
of the code.
Note that the pkgconfig entries are explicitly versioned by the API
version and not the library version. I confirmed the upstream pkgconfig
files.
This makes the condition for including each init match the condition for
compiling the file that defines it.
It's possible to e.g. HAVE_GL and HAVE_VAAPI without HAVE_VAAPI_EGL,
which resulted in "undefined reference to `vaapi_gl_init'" with the old
code.
During probing on a system with AMD GPU, mpv used to output the
following messages if hardware decoding was enabled:
[ffmpeg] AVHWFramesContext: Failed to create surface: 2 (resource
allocation failed).
[ffmpeg] AVHWFramesContext: Unable to allocate a surface from internal
buffer pool.
This commit removed the message, with hopefully no other side effects.
Long explanations follow, better don't read them, it's just tedious
drivel about the details. People should learn to write concise commit
messages, not drone on and on endlessly all while they have no fucking
point.
The code probes supported hardware pixel format, and checks whether they
can be mapped as textures. av_hwdevice_get_hwframe_constraints() returns
a list of hardware pixel formats in the valid_sw_formats field (the "sw"
means software, but they're still hardware pixel formats, makes sense).
This contained the format yuv420p, even though this is not a valid
hardware format. Trying to create a surface of this type results in VA
surface creation failure, upon which FFmpeg prints the error messages
above. We'd be fine with this, except FFmpeg has a global log callback,
and there's no way to suppress these messages without creating other
issues.
It turns out that FFmpeg's vaapi implementation returns all formats from
vaQueryImageFormats() if no "hwconfig" is provided. This list includes
yuv420p, which is probably supported for surface upload/download, but
not as native format. Following FFmpeg's logic, it should not appear in
the valid_sw_formats list, because formats for transfers are returned by
another roundabout API.
Idiotically, there doesn't seem to be any vaapi call that determines
whether a format is a valid surface format. All mechanisms to do this
are bound to a VAConfigID (= video codec or video processor), all while
the actual surface creation API strangely does not take a VAConfigID (a
big WTF).
Also, calling the vaCreateSurfaces() API ourselves for probing is out of
the question, because that functions is utterly and idiotically complex.
Look at the FFmpeg code and how much effort it requires to setup a
complete set of attributes - we can't duplicate this.
So the only way left to do this is the most idiotic and tedious way:
enumerating all VAProfile (and VAEntrypoints) to create all possible
VAConfigIDs. Each of the VAConfigIDs is associated with a list of
formats, which FFmpeg can return (by passing the ID along with the
"hwconfig"), and which is probed separately.
Note that VAConfigID actually refers to a dynamic instance of something,
and creating a VAConfigID takes not only the VAProfile and the
VAEntrypoint, but also an arbitrary attribute array. In theory, this
means our attempt to get to know all possible configurations cannot
work, but in practice this attribute array seems to be pointless for
decoding and video processing, and FFmpeg doesn't use it (though the
encoding path does use it). This probably just makes it _barely_ OK to
do it this way.
Could we discard all this probing shit, and somehow do it another way?
Probably not. The EGL API for mapping surfaces doesn't even seem to
provide a way to enumerate supported formats, we may not even know
whether DRM/dmabuf interop is actually supported (AFAIR the EGL
extensions are present even if they don't work), nor do we know whether
the VAAPI driver supports this interop (not sure). So actually trying is
the only way.
Further, mpv initializes the decoder on a another thread, where you
can't just access OpenGL state. This suckage is mostly to be blamed on
OpenGL itself and its crazy thread boundedness. In theory, this could be
done anyway (see how software decoding "direct rendering" tries to get
around this). But to make it worse, the decoder never cares about the
list of supported formats determined by this code; instead,
f_autoconvert.c tries to deal with it and insert a video processor
(well, good luck with this crap, I bet it doesn't even work). So this
whole endeavor might be pointless, other than the fact that failed
probing can disable use of vaapi (which is correct and necessary). But
if you have a shovel, you don't use it to smash the flat end on the heap
of shit that's piled up before you, or do you?
While this method probably works, it's still orgasmically tedious. It
was tedious before: we had to create a real surface, create a GL
texture, map the surface with it, then destroy everything again. But the
added code is tedious on its own. Highlights include the need to malloc
a FFmpeg struct just to pass a single damn integer, the need to
enumerate "entrypoints" for each VA profile, even though all profiles
have exactly 1 entrypoint, and the kind of obnoxious way how vaapi
requires you to preallocate arrays for returned things, even they could
for example reasonably be returned as immutable arrays or have some
other simpler API.
The main grand fuckup is of course that vaapi requires a VAConfigID to
query surface properties, but not for creating surfaces. This
awkwardness even affected the FFmpeg API design, which has a "hwconfig"
concept that is only used by vaapi (vaapi is only 1 out of 10 hardware
decoding APIs supported by the FFmpeg hwcontext stuff). Maybe I'm just
missing something. It's as if vaapi required setting radioactive shit on
fire. Look how clean the native D3D11 code is instead. (Even the ANGLE
code manages to avoid being this fucked up. Or the VDPAU code, despite
supporting multiple mapping methods.)
Another only barely related change is that the valid_sw_formats field
can be NULL, and the API explicitly documents this. Technically, the mpv
code was buggy for not checking this, although until now the FFmpeg
implementation so far could not return it when we still passed NULL for
the hwconfig parameter.
No functional changes, just preparation for the next commit. Split the
probing into multiple functions. Prepare for the yet unused possibility
to pass AVVAAPIHWConfig to probing. try_format_pixfmt() now assumes it
can be called multiple times with the same format, so it filters the
format.
The format probing is now something like O(n^2) for n formats, but n
will most likely remain something under 50 or so.
Extending the client-allocated mpv_opengl_drm_params struct
constituted a break of ABI that could cause UB.
Create a clean break by deprecating "drm_params" and related structs
and enum values, and replacing it with "drm_params_v2".
Also fix some comments and code that wrongly assumed that open could
return any other negative number than -1 for failure.
This commit updates the libmpv version to 1.104
Like hwdec_cuda, you get a big #ifdef mess if you try and keep the
OpenGL and Vulkan interops in the same file. So, I've refactored
them into separate files in a similar way.
This is documented as required (although we did not do it in
the old GL codepath, with no visible problems) and I have seen
transient artifacts after seeking which _appear_ to have gone
away after introducing this.
We saw a segfault when trying to use the intel-media-driver (iHD)
rather than the normal intel va driver. This happened because the
iHD driver reports P010 (and maybe other formats) with multiple
layers to represent the interleaved UV plane. The normal va driver
reports one UV layer to match the plane.
This threw off my logic which assumed that the number of layers
could not exceed the number of planes.
There's a way one could fix this in a fully generalised form, but
I'm just going to do what the EGL path does and assume that:
* Layer 'n' is on Plane 'n' for n < total number of planes
* These layers always start at offset 0 on the plane
You can imagine ways that these assumptions are violated, but at
least the failure will look the same for both EGL and Vulkan
paths.
Today, we normally see a format error when probing because yuyv422
cannot be used, but it's in the normal set of probed formats.
This error is distracting and confusing, so only log probing errors
at the VERBOSE level.
Fixes#6411
This change introduces a vulkan interop path for the vaapi hwdec.
The basic principles are mostly the same as for EGL, with the
exported dma_buf being imported by Vukan. The biggest difference
is that we cannot reuse the texture as we do with OpenGL - there's
no way to rebind a VkImage to a different piece of memory, as far
as I can see. So, a new texture is created on each map call.
I did not bother implementing a code path for the old libva API as
I think it's safe to assume any system with a working vulkan driver
will have access to a newer libva.
Note that we are using separate layers for the vaapi surface, just
as is done for EGL. This is because libplacebo doesn't support
multiplane images.
This change does not include format negotiation because no driver
implements the vk_ext_image_drm_format_modifier extension that
would be required to do that. In practice, the two formats we care
about (nv12, p010) work correctly, so we are not blocked. A separate
change had to be made in libplacebo to filter out non-fatal validation
errors related to surface sizes due to the lack of format negotiation.
Philip Langdale
Niklas Haas
wm4
Philip Sequeira
Anton Kindestam
Philip Langdale