This affects vo_opengl_cb in particular: it'll most likely auto-load
VDA, and then the VideoToolbox decoder won't work. And everything fails.
This is mainly caused by FFmpeg using separate pixfmts for the _same_
thing (CVPixelBuffers), simply because libavcodec's architecture demands
that hwaccel backends are selected by pixfmts. (Which makes no sense,
but now we have the mess.)
So instead of duplicating FFmpeg's misdesign, just change the format to
our own canonical one on the image output by the decoder. Now the GL
interop code is exactly the same for VDA and VT, and we use the VT name
only.
VDA is being deprecated in OS X 10.11 so this is needed to keep hwdec working.
The code needs libavcodec support which was added recently (to FFmpeg git,
libav doesn't support it).
Signed-off-by: Stefano Pigozzi <stefano.pigozzi@gmail.com>
A rather dumb hack to copy the problematic rectangle textures (mandated
by VDA) into 2D ones.
(This isn't done yet for OpenGL 3.0+. We need to make sure the
performance isn't reduced too much by it.)
The hardware always decodes to nv12 so using this image format causes less cpu
usage than uyvy (which we are currently using, since Apple examples and other
free software use that). The reduction in cpu usage can add up to quite a bit,
especially for 4k or high fps video.
This needs an accompaning commit in libavcodec.
This is somewhat imperfect, because detection of hw decoding APIs is
mostly done on demand, and often avoided if not necessary. (For example,
we know very well that there are no hw decoders for certain codecs.)
This also requires every hwdec backend to identify itself (see hwdec.h
changes).
Hardware decoding/displaying with vo_opengl is done by replacing the
normal video textures with textures provided by the hardware decoding
API OpenGL interop code. Often, this changes the format (vaglx and vdpau
return RGBA, vda returns packed YUV).
If the format is changed, there was a chance (or at least a higher
potential for bugs) that the shader generation code could be confused by
the mismatch of formats, and would create incorrect conversions.
Simplify this by requiring the hwdec interop driver to set the format it
will return to us. This affects all fields, not just some (done by
replacing the format with the value of the converted_imgfmt field in
init_format), in particular fields like colorlevels.
Currently, no hwdec interop driver does anything sophisticated, and the
win is mostly from the mp_image_params_guess_csp() function, which will
reset fields like colorlevels to expected value if RGBA is used.
Instead of converting the hw surface to an image in the VO, provide a
generic way to convet hw surfaces, and use this in the screenshot code.
It's all relatively straightforward, except vdpau is being terrible. It
needs a huge chunk of new code, because copying back is not simple.
The way we use rectangle textures (required by VDA for no reason) works
onl in OpenGL 3.0 or higher. Below that, the shader will fail to
compile. We could add support for older OpenGL versions, but that would
be a major pain.
This normally doesn't matter; mpv itself always creates OpenGL 3.2
contexts on OSX. But it could matter if a client API user uses
vo_opengl_cb, and gives it a 2.1 context (which OSX also allows you to
create).
This wasn't done before because there was no advantage in "abstracting"
it. This changed, and putting this into its own files is better than
messing it into gl_common.c/h.
Basically, don't access the vo field.
There's also no reason anymore to access MPGLContext. We still need to
access loaded GL functions though, so add a field for that to gl_hwdec.
Untested.
Previous API worked under the assumption that download_image is always called
after map_image. In practice this is true, but it's better to have a much
generic API that doesn't depend on the order in which the functions are called.
The harder work was done in the previous commits. After that this feature comes
out almost for free.
The only problem is I can't get the textures created with CGLTexImageIOSurface2D
to download properly, thus the code performs download using some CoreVideo APIs.
If someone knows why download of textures created with CGLTexImageIOSurface2D
doesn't work please contact me :)
wm4
Sebastien Zwickert
Stefano Pigozzi