mpv/video/out
wm4 a17337910c vo_gpu: hwdec_vaegl: silence confusing message during probing
During probing on a system with AMD GPU, mpv used to output the
following messages if hardware decoding was enabled:

[ffmpeg] AVHWFramesContext: Failed to create surface: 2 (resource
allocation failed).
[ffmpeg] AVHWFramesContext: Unable to allocate a surface from internal
buffer pool.

This commit removed the message, with hopefully no other side effects.
Long explanations follow, better don't read them, it's just tedious
drivel about the details. People should learn to write concise commit
messages, not drone on and on endlessly all while they have no fucking
point.

The code probes supported hardware pixel format, and checks whether they
can be mapped as textures. av_hwdevice_get_hwframe_constraints() returns
a list of hardware pixel formats in the valid_sw_formats field (the "sw"
means software, but they're still hardware pixel formats, makes sense).
This contained the format yuv420p, even though this is not a valid
hardware format. Trying to create a surface of this type results in VA
surface creation failure, upon which FFmpeg prints the error messages
above. We'd be fine with this, except FFmpeg has a global log callback,
and there's no way to suppress these messages without creating other
issues.

It turns out that FFmpeg's vaapi implementation returns all formats from
vaQueryImageFormats() if no "hwconfig" is provided. This list includes
yuv420p, which is probably supported for surface upload/download, but
not as native format. Following FFmpeg's logic, it should not appear in
the valid_sw_formats list, because formats for transfers are returned by
another roundabout API.

Idiotically, there doesn't seem to be any vaapi call that determines
whether a format is a valid surface format. All mechanisms to do this
are bound to a VAConfigID (= video codec or video processor), all while
the actual surface creation API strangely does not take a VAConfigID (a
big WTF).

Also, calling the vaCreateSurfaces() API ourselves for probing is out of
the question, because that functions is utterly and idiotically complex.
Look at the FFmpeg code and how much effort it requires to setup a
complete set of attributes - we can't duplicate this.

So the only way left to do this is the most idiotic and tedious way:
enumerating all VAProfile (and VAEntrypoints) to create all possible
VAConfigIDs. Each of the VAConfigIDs is associated with a list of
formats, which FFmpeg can return (by passing the ID along with the
"hwconfig"), and which is probed separately.

Note that VAConfigID actually refers to a dynamic instance of something,
and creating a VAConfigID takes not only the VAProfile and the
VAEntrypoint, but also an arbitrary attribute array. In theory, this
means our attempt to get to know all possible configurations cannot
work, but in practice this attribute array seems to be pointless for
decoding and video processing, and FFmpeg doesn't use it (though the
encoding path does use it). This probably just makes it _barely_ OK to
do it this way.

Could we discard all this probing shit, and somehow do it another way?
Probably not. The EGL API for mapping surfaces doesn't even seem to
provide a way to enumerate supported formats, we may not even know
whether DRM/dmabuf interop is actually supported (AFAIR the EGL
extensions are present even if they don't work), nor do we know whether
the VAAPI driver supports this interop (not sure). So actually trying is
the only way.

Further, mpv initializes the decoder on a another thread, where you
can't just access OpenGL state. This suckage is mostly to be blamed on
OpenGL itself and its crazy thread boundedness. In theory, this could be
done anyway (see how software decoding "direct rendering" tries to get
around this). But to make it worse, the decoder never cares about the
list of supported formats determined by this code; instead,
f_autoconvert.c tries to deal with it and insert a video processor
(well, good luck with this crap, I bet it doesn't even work). So this
whole endeavor might be pointless, other than the fact that failed
probing can disable use of vaapi (which is correct and necessary). But
if you have a shovel, you don't use it to smash the flat end on the heap
of shit that's piled up before you, or do you?

While this method probably works, it's still orgasmically tedious. It
was tedious before: we had to create a real surface, create a GL
texture, map the surface with it, then destroy everything again. But the
added code is tedious on its own. Highlights include the need to malloc
a FFmpeg struct just to pass a single damn integer, the need to
enumerate "entrypoints" for each VA profile, even though all profiles
have exactly 1 entrypoint, and the kind of obnoxious way how vaapi
requires you to preallocate arrays for returned things, even they could
for example reasonably be returned as immutable arrays or have some
other simpler API.

The main grand fuckup is of course that vaapi requires a VAConfigID to
query surface properties, but not for creating surfaces. This
awkwardness even affected the FFmpeg API design, which has a "hwconfig"
concept that is only used by vaapi (vaapi is only 1 out of 10 hardware
decoding APIs supported by the FFmpeg hwcontext stuff). Maybe I'm just
missing something. It's as if vaapi required setting radioactive shit on
fire. Look how clean the native D3D11 code is instead. (Even the ANGLE
code manages to avoid being this fucked up. Or the VDPAU code, despite
supporting multiple mapping methods.)

Another only barely related change is that the valid_sw_formats field
can be NULL, and the API explicitly documents this. Technically, the mpv
code was buggy for not checking this, although until now the FFmpeg
implementation so far could not return it when we still passed NULL for
the hwconfig parameter.
2019-09-19 20:37:05 +02:00
..
cocoa cocoa: fix drawing on macOS 10.14 2018-11-13 21:14:29 +02:00
cocoa-cb cocoa-cb: remove an unused variable 2019-09-02 00:39:36 +03:00
d3d11 vo_d3d11/context: fix crash due to ctx->ra is null pointer access 2019-09-14 21:35:49 +10:00
gpu vo_gpu: remove vdpau/GLX backend 2019-09-19 20:37:05 +02:00
hwdec vo_gpu: hwdec_vaegl: silence confusing message during probing 2019-09-19 20:37:05 +02:00
opengl vo_gpu: remove vdpau/GLX backend 2019-09-19 20:37:05 +02:00
placebo video/out/gpu: Add a `storable` flag to ra_format 2019-07-08 00:59:28 +02:00
vulkan vo/gpu: vulkan: Pass the device name option through to libplacebo 2019-08-24 18:38:27 +02:00
win32 vo_gpu: win: remove exclusive-fullscreen detection hack 2017-12-20 14:53:41 +11:00
aspect.c aspect: add video margin options 2019-09-19 20:37:05 +02:00
aspect.h
bitmap_packer.c
bitmap_packer.h
cocoa_cb_common.swift cocoa-cb: migrate to swift 5 with swift 4 fallback 2019-07-21 18:13:07 +03:00
cocoa_common.h
cocoa_common.m {mac,cocoa}: trim trailing null out of macosx_icon when loading it 2018-10-02 00:20:43 +03:00
d3d_shader_420p.h
d3d_shader_yuv.hlsl
dither.c
dither.h
dr_helper.c vo: move DR helper code to a separate source file 2018-04-29 02:21:32 +03:00
dr_helper.h vo: move DR helper code to a separate source file 2018-04-29 02:21:32 +03:00
drm_atomic.c drm: rename plane options to better, invariant, names 2018-12-01 15:42:20 +02:00
drm_atomic.h drm: rename plane options to better, invariant, names 2018-12-01 15:42:20 +02:00
drm_common.c drm_common: Support --drm-mode=<preferred|highest|N|WxH[@R]> 2019-05-04 14:17:11 +02:00
drm_common.h drm_common: Don't export functions only being used internally 2019-05-04 14:17:11 +02:00
drm_prime.c Add DRM_PRIME Format Handling and Display for RockChip MPP decoders 2017-10-23 21:07:24 +02:00
drm_prime.h Add DRM_PRIME Format Handling and Display for RockChip MPP decoders 2017-10-23 21:07:24 +02:00
filter_kernels.c vo_opengl: refactor scaler LUT weight packing/loading 2017-09-04 13:53:14 +02:00
filter_kernels.h vo_opengl: refactor scaler LUT weight packing/loading 2017-09-04 13:53:14 +02:00
libmpv.h vo_libmpv: support render performance data 2018-11-13 20:43:29 +02:00
vo.c vo, vo_gpu, glx: correct GLX_OML_sync_control usage 2018-12-06 10:32:27 +01:00
vo.h vo, vo_gpu, glx: correct GLX_OML_sync_control usage 2018-12-06 10:32:27 +01:00
vo_caca.c Add checks for HAVE_GPL to various GPL-only source files 2017-10-10 15:51:16 +02:00
vo_direct3d.c Add checks for HAVE_GPL to various GPL-only source files 2017-10-10 15:51:16 +02:00
vo_drm.c drm_common: Add proper help option to drm-mode 2019-05-04 14:17:11 +02:00
vo_gpu.c vo: use a struct for vsync feedback stuff 2018-12-06 10:30:25 +01:00
vo_image.c
vo_lavc.c encode: get rid of the output packet queue 2018-05-03 01:08:44 +03:00
vo_libmpv.c vo_libmpv: fix null pointer dereference 2019-03-11 01:55:59 +02:00
vo_mediacodec_embed.c vo_mediacodec_embed: fix forgotten VO_CAP_NOREDRAW→VO_CAP_NORETAIN 2018-02-20 01:59:20 +02:00
vo_null.c
vo_rpi.c vo_gpu: add internal ability to skip osd/subs for rendering 2018-02-11 17:45:51 -08:00
vo_sdl.c vo_sdl: add support for screensaver VOCTRL's 2018-06-02 23:34:38 +03:00
vo_tct.c sws_utils: don't force callers to provide option struct 2018-01-18 00:59:07 -08:00
vo_vaapi.c vo: add warning message to vo_vaapi and vo_vdpau 2019-09-14 13:50:10 +02:00
vo_vdpau.c vo: add warning message to vo_vaapi and vo_vdpau 2019-09-14 13:50:10 +02:00
vo_x11.c vo_x11: fix return value in resize() error paths 2018-11-17 00:53:56 +01:00
vo_xv.c build: remove POSIX/sysv shared memory test 2017-12-02 23:19:13 +01:00
w32_common.c w32_common: avoid unnecessary sprintfs 2019-05-10 20:47:05 +10:00
w32_common.h
wayland_common.c wayland: fix wl_proxy leak 2019-09-19 00:00:19 +03:00
wayland_common.h wayland_common: rename “shell” into “wm_base” 2019-02-17 23:44:34 +02:00
win_state.c video: change some remaining vo_opengl mentions to vo_gpu 2018-01-20 14:43:49 -08:00
win_state.h
x11_common.c x11: fix cursor hiding initial state 2019-03-16 21:17:32 +01:00
x11_common.h x11: fix cursor hiding initial state 2019-03-16 21:17:32 +01:00