As documented on struct mp_hwdec_ctx, hw_imgfmt specifies the hardware
surface wrapper format for which supported_formats is valid. If this was
not set, f_hwtransfer ignored supported_formats, and assumed all formats
were supported.
hwdec_vaapi tries to probe all available surface formats in advance. For
that, we iterate over _all_ profiles in an attempt to collect possible
surface formats. This means we try profiles we normally wouldn't use for
decoding or filtering, and which could be "unrelated" services.
It seems some drivers report at least one profile, for which
vaQueryConfigEntrypoints() fails (because the profile is not supported;
not sure why it lists it, then). So turn the error message into a
verbose message to avoid confusing output.
Fixes: #7347
In vaapi 1.1.0 (which confusingly is libva release 2.1.0), they
introduced a new surface export API that is more efficient, and
we've been supporting that and the old API ever since (Feb 2018).
If we drop support for the old API, we can do some fairly nice cleanup
of the code.
Note that the pkgconfig entries are explicitly versioned by the API
version and not the library version. I confirmed the upstream pkgconfig
files.
In theory, using strstr() to search for extensions is a bad idea,
because some extension names might be prefixes for other names, so you
could get false positives. gl_check_extension() avoids this case.
It's not clear whether this is really needed; maybe not. Surely the EGL
committee is aware of these practices (many GL clients do this, which is
why it's widely considered bad practice), and would avoid defining new
extension names which contain existing names as sub-strings, but
whatever.
Adding an ifdef mess to deal with insufficient system headers is kind of
a mess. It's easier to just provide the definitions manually. This sucks
a bit too, but it's the approach we've been using with OpenGL headers in
general, and I think that worked pretty well.
On systems whose EGL headers do not define these extensions, the build
still failed due to missing ..._PLANE3_... defines. Although we supplied
missing EGL_LINUX_DMA_BUF_EXT defines manually, the PLANE3 ones are
actually from a separate extension, which explains why they were not
added to the fallback defines in the first place.
Add them, now it builds without the eglext.h include.
See #6838.
We should only be printing errors that occur when not probing, to
avoid creating the impression that something is wrong - and errors
during probing isn't a problem.
This makes the condition for including each init match the condition for
compiling the file that defines it.
It's possible to e.g. HAVE_GL and HAVE_VAAPI without HAVE_VAAPI_EGL,
which resulted in "undefined reference to `vaapi_gl_init'" with the old
code.
Previously, there appeared to be implicit synchronisation in the
GL interop path, and we never observed any visual glitches. However,
recently, I started seeing stuttering in the GL path and on closer
examination it looked like read-before-write behaviour where GL
would display an old frame again rather than the current one.
After verifying that disabling hwdec made the problem go away,
I tried adding a cuStreamSynchronize() after the memcpys and that
also resolved the problem, so it's clearly sync related.
cuStreamSynchronize() is a CPU sync and so more heavy-weight than
you want, but it's the only tool we have. There is no mechanism
defined for synchronising GL to CUDA (It looks like there is a way
to synchronise CUDA to EGL but it appears one way and so wouldn't
directly address this problem).
Anyway, empirically, the output now looks the same as with hwdec
off.
At first, this code used only 1 plane, so the compatibility stuff was
sufficient. But then use of planes 1 and 2 was added, without extending
the compatibility stuff.
I think I've seen a case recently where this broke the build and caused
users to apply invalid fixes, but I don't remember where.
It's possible that I didn't get all defines that are needed.
During probing on a system with AMD GPU, mpv used to output the
following messages if hardware decoding was enabled:
[ffmpeg] AVHWFramesContext: Failed to create surface: 2 (resource
allocation failed).
[ffmpeg] AVHWFramesContext: Unable to allocate a surface from internal
buffer pool.
This commit removed the message, with hopefully no other side effects.
Long explanations follow, better don't read them, it's just tedious
drivel about the details. People should learn to write concise commit
messages, not drone on and on endlessly all while they have no fucking
point.
The code probes supported hardware pixel format, and checks whether they
can be mapped as textures. av_hwdevice_get_hwframe_constraints() returns
a list of hardware pixel formats in the valid_sw_formats field (the "sw"
means software, but they're still hardware pixel formats, makes sense).
This contained the format yuv420p, even though this is not a valid
hardware format. Trying to create a surface of this type results in VA
surface creation failure, upon which FFmpeg prints the error messages
above. We'd be fine with this, except FFmpeg has a global log callback,
and there's no way to suppress these messages without creating other
issues.
It turns out that FFmpeg's vaapi implementation returns all formats from
vaQueryImageFormats() if no "hwconfig" is provided. This list includes
yuv420p, which is probably supported for surface upload/download, but
not as native format. Following FFmpeg's logic, it should not appear in
the valid_sw_formats list, because formats for transfers are returned by
another roundabout API.
Idiotically, there doesn't seem to be any vaapi call that determines
whether a format is a valid surface format. All mechanisms to do this
are bound to a VAConfigID (= video codec or video processor), all while
the actual surface creation API strangely does not take a VAConfigID (a
big WTF).
Also, calling the vaCreateSurfaces() API ourselves for probing is out of
the question, because that functions is utterly and idiotically complex.
Look at the FFmpeg code and how much effort it requires to setup a
complete set of attributes - we can't duplicate this.
So the only way left to do this is the most idiotic and tedious way:
enumerating all VAProfile (and VAEntrypoints) to create all possible
VAConfigIDs. Each of the VAConfigIDs is associated with a list of
formats, which FFmpeg can return (by passing the ID along with the
"hwconfig"), and which is probed separately.
Note that VAConfigID actually refers to a dynamic instance of something,
and creating a VAConfigID takes not only the VAProfile and the
VAEntrypoint, but also an arbitrary attribute array. In theory, this
means our attempt to get to know all possible configurations cannot
work, but in practice this attribute array seems to be pointless for
decoding and video processing, and FFmpeg doesn't use it (though the
encoding path does use it). This probably just makes it _barely_ OK to
do it this way.
Could we discard all this probing shit, and somehow do it another way?
Probably not. The EGL API for mapping surfaces doesn't even seem to
provide a way to enumerate supported formats, we may not even know
whether DRM/dmabuf interop is actually supported (AFAIR the EGL
extensions are present even if they don't work), nor do we know whether
the VAAPI driver supports this interop (not sure). So actually trying is
the only way.
Further, mpv initializes the decoder on a another thread, where you
can't just access OpenGL state. This suckage is mostly to be blamed on
OpenGL itself and its crazy thread boundedness. In theory, this could be
done anyway (see how software decoding "direct rendering" tries to get
around this). But to make it worse, the decoder never cares about the
list of supported formats determined by this code; instead,
f_autoconvert.c tries to deal with it and insert a video processor
(well, good luck with this crap, I bet it doesn't even work). So this
whole endeavor might be pointless, other than the fact that failed
probing can disable use of vaapi (which is correct and necessary). But
if you have a shovel, you don't use it to smash the flat end on the heap
of shit that's piled up before you, or do you?
While this method probably works, it's still orgasmically tedious. It
was tedious before: we had to create a real surface, create a GL
texture, map the surface with it, then destroy everything again. But the
added code is tedious on its own. Highlights include the need to malloc
a FFmpeg struct just to pass a single damn integer, the need to
enumerate "entrypoints" for each VA profile, even though all profiles
have exactly 1 entrypoint, and the kind of obnoxious way how vaapi
requires you to preallocate arrays for returned things, even they could
for example reasonably be returned as immutable arrays or have some
other simpler API.
The main grand fuckup is of course that vaapi requires a VAConfigID to
query surface properties, but not for creating surfaces. This
awkwardness even affected the FFmpeg API design, which has a "hwconfig"
concept that is only used by vaapi (vaapi is only 1 out of 10 hardware
decoding APIs supported by the FFmpeg hwcontext stuff). Maybe I'm just
missing something. It's as if vaapi required setting radioactive shit on
fire. Look how clean the native D3D11 code is instead. (Even the ANGLE
code manages to avoid being this fucked up. Or the VDPAU code, despite
supporting multiple mapping methods.)
Another only barely related change is that the valid_sw_formats field
can be NULL, and the API explicitly documents this. Technically, the mpv
code was buggy for not checking this, although until now the FFmpeg
implementation so far could not return it when we still passed NULL for
the hwconfig parameter.
No functional changes, just preparation for the next commit. Split the
probing into multiple functions. Prepare for the yet unused possibility
to pass AVVAAPIHWConfig to probing. try_format_pixfmt() now assumes it
can be called multiple times with the same format, so it filters the
format.
The format probing is now something like O(n^2) for n formats, but n
will most likely remain something under 50 or so.
Extending the client-allocated mpv_opengl_drm_params struct
constituted a break of ABI that could cause UB.
Create a clean break by deprecating "drm_params" and related structs
and enum values, and replacing it with "drm_params_v2".
Also fix some comments and code that wrongly assumed that open could
return any other negative number than -1 for failure.
This commit updates the libmpv version to 1.104
Like hwdec_cuda, you get a big #ifdef mess if you try and keep the
OpenGL and Vulkan interops in the same file. So, I've refactored
them into separate files in a similar way.
This is documented as required (although we did not do it in
the old GL codepath, with no visible problems) and I have seen
transient artifacts after seeking which _appear_ to have gone
away after introducing this.
We saw a segfault when trying to use the intel-media-driver (iHD)
rather than the normal intel va driver. This happened because the
iHD driver reports P010 (and maybe other formats) with multiple
layers to represent the interleaved UV plane. The normal va driver
reports one UV layer to match the plane.
This threw off my logic which assumed that the number of layers
could not exceed the number of planes.
There's a way one could fix this in a fully generalised form, but
I'm just going to do what the EGL path does and assume that:
* Layer 'n' is on Plane 'n' for n < total number of planes
* These layers always start at offset 0 on the plane
You can imagine ways that these assumptions are violated, but at
least the failure will look the same for both EGL and Vulkan
paths.
Today, we normally see a format error when probing because yuyv422
cannot be used, but it's in the normal set of probed formats.
This error is distracting and confusing, so only log probing errors
at the VERBOSE level.
Fixes#6411
This change introduces a vulkan interop path for the vaapi hwdec.
The basic principles are mostly the same as for EGL, with the
exported dma_buf being imported by Vukan. The biggest difference
is that we cannot reuse the texture as we do with OpenGL - there's
no way to rebind a VkImage to a different piece of memory, as far
as I can see. So, a new texture is created on each map call.
I did not bother implementing a code path for the old libva API as
I think it's safe to assume any system with a working vulkan driver
will have access to a newer libva.
Note that we are using separate layers for the vaapi surface, just
as is done for EGL. This is because libplacebo doesn't support
multiplane images.
This change does not include format negotiation because no driver
implements the vk_ext_image_drm_format_modifier extension that
would be required to do that. In practice, the two formats we care
about (nv12, p010) work correctly, so we are not blocked. A separate
change had to be made in libplacebo to filter out non-fatal validation
errors related to surface sizes due to the lack of format negotiation.
The amount of code now present that's specific to Vulkan or OpenGL
has reached the point where we really want to split it out to
avoid a mess of #ifdefs.
At the same time, I'm moving the code to an api neutral location.
wm4
Philip Langdale
Philip Sequeira
Anton Kindestam
Philip Langdale