RepoMirrors
/
mpv
mirror of https://github.com/mpv-player/mpv
1
0
Fork 0
Code Issues Releases Wiki Activity
mpv/video/out/opengl/hwdec_d3d11egl.c

661 lines
21 KiB
C
Raw Normal View History

vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
/*
* This file is part of mpv.
*
* mpv is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* mpv is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
*/
win32: replace libuuid.a usage with initguid.h Including initguid.h at the top of a file that uses references to GUIDs causes the GUIDs to be declared globally with __declspec(selectany). The 'selectany' attribute tells the linker to consolidate multiple definitions of each GUID, which would be great except that, in Cygwin and MinGW GCC 6.1, this method of linking makes the GUIDs conflict with the ones declared in libuuid.a. Since initguid.h obsoletes libuuid.a in modern compilers that support __declspec(selectany), add initguid.h to all files that use GUIDs and remove libuuid.a from the build. Fixes #3097
2016-04-30 15:49:38 +00:00
#include <initguid.h>
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
#include <assert.h>
#include <windows.h>
d3d11va: store texture/subindex in IMGFMT_D3D11VA plane pointers Basically this gets rid of the need for the accessors in d3d11va.h, and the code can be cleaned up a little bit. Note that libavcodec only defines a ID3D11VideoDecoderOutputView pointer in the last plane pointers, but it tolerates/passes through the other plane pointers we set.
2016-04-27 12:03:30 +00:00
#include <d3d11.h>
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
#include <EGL/egl.h>
#include <EGL/eglext.h>
vo_opengl: angle: dynamically load ANGLE ANGLE is _really_ annoying to build. (Requires special toolchain and a recent MSVC version.) This results in various issues with people having trouble to build mpv against ANGLE (apparently linking it against a prebuilt binary doesn't count, or using binaries from potentially untrusted sources is not wanted). Dynamically loading ANGLE is going to be a huge convenience. This commit implements this, with special focus on keeping it source compatible to a normal build with ANGLE linked at build-time.
2016-05-11 10:33:49 +00:00
#include "angle_dynamic.h"
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
#include "common/common.h"
#include "osdep/timer.h"
#include "osdep/windows_utils.h"
#include "hwdec.h"
#include "video/hwdec.h"
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
#ifndef EGL_D3D_TEXTURE_SUBRESOURCE_ID_ANGLE
#define EGL_D3D_TEXTURE_SUBRESOURCE_ID_ANGLE 0x3AAB
#endif
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
struct priv {
video: refactor how VO exports hwdec device handles The main change is with video/hwdec.h. mp_hwdec_info is made opaque (and renamed to mp_hwdec_devices). Its accessors are mainly thread-safe (or documented where not), which makes the whole thing saner and cleaner. In particular, thread-safety rules become less subtle and more obvious. The new internal API makes it easier to support multiple OpenGL interop backends. (Although this is not done yet, and it's not clear whether it ever will.) This also removes all the API-specific fields from mp_hwdec_ctx and replaces them with a "ctx" field. For d3d in particular, we drop the mp_d3d_ctx struct completely, and pass the interfaces directly. Remove the emulation checks from vaapi.c and vdpau.c; they are pointless, and the checks that matter are done on the VO layer. The d3d hardware decoders might slightly change behavior: dxva2-copy will not use the VO device anymore if the VO supports proper interop. This pretty much assumes that any in such cases the VO will not use any form of exclusive mode, which makes using the VO device in copy mode unnecessary. This is a big refactor. Some things may be untested and could be broken.
2016-05-09 17:42:03 +00:00
struct mp_hwdec_ctx hwctx;
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
ID3D11Device *d3d11_device;
ID3D11VideoDevice *video_dev;
ID3D11VideoContext *video_ctx;
EGLDisplay egl_display;
EGLConfig egl_config;
EGLSurface egl_surface;
ID3D11Texture2D *texture;
ID3D11VideoProcessor *video_proc;
ID3D11VideoProcessorEnumerator *vp_enum;
ID3D11VideoProcessorOutputView *out_view;
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
struct mp_image_params image_params;
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
int c_w, c_h;
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
EGLStreamKHR egl_stream;
GLuint gl_textures[3];
// EGL_KHR_stream
EGLStreamKHR (EGLAPIENTRY *CreateStreamKHR)(EGLDisplay dpy,
const EGLint *attrib_list);
EGLBoolean (EGLAPIENTRY *DestroyStreamKHR)(EGLDisplay dpy,
EGLStreamKHR stream);
// EGL_KHR_stream_consumer_gltexture
EGLBoolean (EGLAPIENTRY *StreamConsumerAcquireKHR)
(EGLDisplay dpy, EGLStreamKHR stream);
EGLBoolean (EGLAPIENTRY *StreamConsumerReleaseKHR)
(EGLDisplay dpy, EGLStreamKHR stream);
// EGL_NV_stream_consumer_gltexture_yuv
EGLBoolean (EGLAPIENTRY *StreamConsumerGLTextureExternalAttribsNV)
(EGLDisplay dpy, EGLStreamKHR stream, EGLAttrib *attrib_list);
// EGL_ANGLE_stream_producer_d3d_texture_nv12
EGLBoolean (EGLAPIENTRY *CreateStreamProducerD3DTextureNV12ANGLE)
(EGLDisplay dpy, EGLStreamKHR stream, const EGLAttrib *attrib_list);
EGLBoolean (EGLAPIENTRY *StreamPostD3DTextureNV12ANGLE)
(EGLDisplay dpy, EGLStreamKHR stream, void *texture,
const EGLAttrib *attrib_list);
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
};
static void destroy_video_proc(struct gl_hwdec *hw)
{
struct priv *p = hw->priv;
if (p->out_view)
ID3D11VideoProcessorOutputView_Release(p->out_view);
p->out_view = NULL;
if (p->video_proc)
ID3D11VideoProcessor_Release(p->video_proc);
p->video_proc = NULL;
if (p->vp_enum)
ID3D11VideoProcessorEnumerator_Release(p->vp_enum);
p->vp_enum = NULL;
}
static void destroy_objects(struct gl_hwdec *hw)
{
struct priv *p = hw->priv;
GL *gl = hw->gl;
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
if (p->egl_stream)
p->DestroyStreamKHR(p->egl_display, p->egl_stream);
p->egl_stream = 0;
for (int n = 0; n < 3; n++) {
gl->DeleteTextures(1, &p->gl_textures[n]);
p->gl_textures[n] = 0;
}
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
if (p->egl_display && p->egl_surface) {
eglReleaseTexImage(p->egl_display, p->egl_surface, EGL_BACK_BUFFER);
eglDestroySurface(p->egl_display, p->egl_surface);
}
p->egl_surface = NULL;
if (p->texture)
ID3D11Texture2D_Release(p->texture);
p->texture = NULL;
destroy_video_proc(hw);
}
static void destroy(struct gl_hwdec *hw)
{
struct priv *p = hw->priv;
destroy_objects(hw);
video: refactor how VO exports hwdec device handles The main change is with video/hwdec.h. mp_hwdec_info is made opaque (and renamed to mp_hwdec_devices). Its accessors are mainly thread-safe (or documented where not), which makes the whole thing saner and cleaner. In particular, thread-safety rules become less subtle and more obvious. The new internal API makes it easier to support multiple OpenGL interop backends. (Although this is not done yet, and it's not clear whether it ever will.) This also removes all the API-specific fields from mp_hwdec_ctx and replaces them with a "ctx" field. For d3d in particular, we drop the mp_d3d_ctx struct completely, and pass the interfaces directly. Remove the emulation checks from vaapi.c and vdpau.c; they are pointless, and the checks that matter are done on the VO layer. The d3d hardware decoders might slightly change behavior: dxva2-copy will not use the VO device anymore if the VO supports proper interop. This pretty much assumes that any in such cases the VO will not use any form of exclusive mode, which makes using the VO device in copy mode unnecessary. This is a big refactor. Some things may be untested and could be broken.
2016-05-09 17:42:03 +00:00
hwdec_devices_remove(hw->devs, &p->hwctx);
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
if (p->video_ctx)
ID3D11VideoContext_Release(p->video_ctx);
p->video_ctx = NULL;
if (p->video_dev)
ID3D11VideoDevice_Release(p->video_dev);
p->video_dev = NULL;
if (p->d3d11_device)
ID3D11Device_Release(p->d3d11_device);
p->d3d11_device = NULL;
}
static int create(struct gl_hwdec *hw)
{
vo_opengl: angle: dynamically load ANGLE ANGLE is _really_ annoying to build. (Requires special toolchain and a recent MSVC version.) This results in various issues with people having trouble to build mpv against ANGLE (apparently linking it against a prebuilt binary doesn't count, or using binaries from potentially untrusted sources is not wanted). Dynamically loading ANGLE is going to be a huge convenience. This commit implements this, with special focus on keeping it source compatible to a normal build with ANGLE linked at build-time.
2016-05-11 10:33:49 +00:00
if (!angle_load())
return -1;
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
EGLDisplay egl_display = eglGetCurrentDisplay();
if (!egl_display)
return -1;
vo_opengl: EGL: fix hwdec probing If ANGLE was probed before (but rejected), the ANGLE API can remain "initialized", and eglGetCurrentDisplay() will return a non-NULL EGLDisplay. Then if a native GL context is used, the ANGLE/EGL API will then (apparently) keep working alongside native OpenGL API. Since GL objects are just numbers, they'll simply fail to interact, and OpenGL will get invalid textures. For some reason this will result in black textures. With VAAPI-EGL, something similar could happen in theory, but didn't in practice.
2016-05-05 11:38:08 +00:00
if (!eglGetCurrentContext())
return -1;
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
const char *exts = eglQueryString(egl_display, EGL_EXTENSIONS);
vo_opengl: d3d11egl: don't require EGL_EXT_device_query Older ANGLE builds don't export this. This change is really only for convenience, and I might revert it at some later point.
2016-05-11 13:08:11 +00:00
if (!exts || !strstr(exts, "EGL_ANGLE_d3d_share_handle_client_buffer"))
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
return -1;
vo_opengl: d3d11egl: don't require EGL_EXT_device_query Older ANGLE builds don't export this. This change is really only for convenience, and I might revert it at some later point.
2016-05-11 13:08:11 +00:00
bool use_native_device = !!strstr(exts, "EGL_EXT_device_query");
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
HRESULT hr;
struct priv *p = talloc_zero(hw, struct priv);
hw->priv = p;
p->egl_display = egl_display;
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
// Optional EGLStream stuff for working without video processor.
// Note that as long as GL_OES_EGL_image_external_essl3 is not available,
// this won't work in ES 3.x mode due to missing GLSL mechanisms.
if (strstr(exts, "EGL_ANGLE_stream_producer_d3d_texture_nv12") &&
vo_opengl: d3d11egl: don't require EGL_EXT_device_query Older ANGLE builds don't export this. This change is really only for convenience, and I might revert it at some later point.
2016-05-11 13:08:11 +00:00
use_native_device && hw->gl->es == 200)
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
{
MP_VERBOSE(hw, "Loading EGL_ANGLE_stream_producer_d3d_texture_nv12\n");
p->CreateStreamKHR = (void *)eglGetProcAddress("eglCreateStreamKHR");
p->DestroyStreamKHR = (void *)eglGetProcAddress("eglDestroyStreamKHR");
p->StreamConsumerAcquireKHR =
(void *)eglGetProcAddress("eglStreamConsumerAcquireKHR");
p->StreamConsumerReleaseKHR =
(void *)eglGetProcAddress("eglStreamConsumerReleaseKHR");
p->StreamConsumerGLTextureExternalAttribsNV =
(void *)eglGetProcAddress("eglStreamConsumerGLTextureExternalAttribsNV");
p->CreateStreamProducerD3DTextureNV12ANGLE =
(void *)eglGetProcAddress("eglCreateStreamProducerD3DTextureNV12ANGLE");
p->StreamPostD3DTextureNV12ANGLE =
(void *)eglGetProcAddress("eglStreamPostD3DTextureNV12ANGLE");
if (!p->CreateStreamKHR || !p->DestroyStreamKHR ||
!p->StreamConsumerAcquireKHR || !p->StreamConsumerReleaseKHR ||
!p->StreamConsumerGLTextureExternalAttribsNV ||
!p->CreateStreamProducerD3DTextureNV12ANGLE ||
!p->StreamPostD3DTextureNV12ANGLE)
{
MP_ERR(hw, "Failed to load some EGLStream functions.\n");
goto fail;
}
}
vo_opengl: d3d11egl: don't require EGL_EXT_device_query Older ANGLE builds don't export this. This change is really only for convenience, and I might revert it at some later point.
2016-05-11 13:08:11 +00:00
if (use_native_device) {
PFNEGLQUERYDISPLAYATTRIBEXTPROC p_eglQueryDisplayAttribEXT =
(void *)eglGetProcAddress("eglQueryDisplayAttribEXT");
PFNEGLQUERYDEVICEATTRIBEXTPROC p_eglQueryDeviceAttribEXT =
(void *)eglGetProcAddress("eglQueryDeviceAttribEXT");
if (!p_eglQueryDisplayAttribEXT || !p_eglQueryDeviceAttribEXT)
goto fail;
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
vo_opengl: d3d11egl: don't require EGL_EXT_device_query Older ANGLE builds don't export this. This change is really only for convenience, and I might revert it at some later point.
2016-05-11 13:08:11 +00:00
EGLAttrib device = 0;
if (!p_eglQueryDisplayAttribEXT(egl_display, EGL_DEVICE_EXT, &device))
goto fail;
EGLAttrib d3d_device = 0;
if (!p_eglQueryDeviceAttribEXT((EGLDeviceEXT)device,
EGL_D3D11_DEVICE_ANGLE, &d3d_device))
{
MP_ERR(hw, "Could not get EGL_D3D11_DEVICE_ANGLE from ANGLE.\n");
goto fail;
}
p->d3d11_device = (ID3D11Device *)d3d_device;
if (!p->d3d11_device)
goto fail;
ID3D11Device_AddRef(p->d3d11_device);
} else {
HANDLE d3d11_dll = GetModuleHandleW(L"d3d11.dll");
if (!d3d11_dll) {
MP_ERR(hw, "Failed to load D3D11 library\n");
goto fail;
}
PFN_D3D11_CREATE_DEVICE CreateDevice =
(void *)GetProcAddress(d3d11_dll, "D3D11CreateDevice");
if (!CreateDevice)
goto fail;
hr = CreateDevice(NULL, D3D_DRIVER_TYPE_HARDWARE, NULL,
D3D11_CREATE_DEVICE_VIDEO_SUPPORT, NULL, 0,
D3D11_SDK_VERSION, &p->d3d11_device, NULL, NULL);
if (FAILED(hr)) {
MP_ERR(hw, "Failed to create D3D11 Device: %s\n",
mp_HRESULT_to_str(hr));
goto fail;
}
}
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
ID3D10Multithread *multithread;
hr = ID3D11Device_QueryInterface(p->d3d11_device, &IID_ID3D10Multithread,
(void **)&multithread);
if (FAILED(hr)) {
ID3D10Multithread_Release(multithread);
MP_ERR(hw, "Failed to get Multithread interface: %s\n",
mp_HRESULT_to_str(hr));
goto fail;
}
ID3D10Multithread_SetMultithreadProtected(multithread, TRUE);
ID3D10Multithread_Release(multithread);
hr = ID3D11Device_QueryInterface(p->d3d11_device, &IID_ID3D11VideoDevice,
(void **)&p->video_dev);
if (FAILED(hr))
goto fail;
ID3D11DeviceContext *device_ctx;
ID3D11Device_GetImmediateContext(p->d3d11_device, &device_ctx);
if (!device_ctx)
goto fail;
hr = ID3D11DeviceContext_QueryInterface(device_ctx, &IID_ID3D11VideoContext,
(void **)&p->video_ctx);
ID3D11DeviceContext_Release(device_ctx);
if (FAILED(hr))
goto fail;
EGLint attrs[] = {
EGL_BUFFER_SIZE, 32,
EGL_RED_SIZE, 8,
EGL_GREEN_SIZE, 8,
EGL_BLUE_SIZE, 8,
EGL_SURFACE_TYPE, EGL_PBUFFER_BIT,
EGL_ALPHA_SIZE, 8,
EGL_BIND_TO_TEXTURE_RGBA, EGL_TRUE,
EGL_NONE
};
EGLint count;
if (!eglChooseConfig(p->egl_display, attrs, &p->egl_config, 1, &count) ||
!count) {
MP_ERR(hw, "Failed to get EGL surface configuration\n");
goto fail;
}
video: refactor how VO exports hwdec device handles The main change is with video/hwdec.h. mp_hwdec_info is made opaque (and renamed to mp_hwdec_devices). Its accessors are mainly thread-safe (or documented where not), which makes the whole thing saner and cleaner. In particular, thread-safety rules become less subtle and more obvious. The new internal API makes it easier to support multiple OpenGL interop backends. (Although this is not done yet, and it's not clear whether it ever will.) This also removes all the API-specific fields from mp_hwdec_ctx and replaces them with a "ctx" field. For d3d in particular, we drop the mp_d3d_ctx struct completely, and pass the interfaces directly. Remove the emulation checks from vaapi.c and vdpau.c; they are pointless, and the checks that matter are done on the VO layer. The d3d hardware decoders might slightly change behavior: dxva2-copy will not use the VO device anymore if the VO supports proper interop. This pretty much assumes that any in such cases the VO will not use any form of exclusive mode, which makes using the VO device in copy mode unnecessary. This is a big refactor. Some things may be untested and could be broken.
2016-05-09 17:42:03 +00:00
p->hwctx = (struct mp_hwdec_ctx){
.type = HWDEC_D3D11VA,
.driver_name = hw->driver->name,
.ctx = p->d3d11_device,
};
hwdec_devices_add(hw->devs, &p->hwctx);
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
return 0;
fail:
destroy(hw);
return -1;
}
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
static int create_egl_stream(struct gl_hwdec *hw, struct mp_image_params *params)
{
struct priv *p = hw->priv;
GL *gl = hw->gl;
if (params->hw_subfmt != IMGFMT_NV12)
return -1;
if (!p->CreateStreamKHR)
return -1; // extensions not available
MP_VERBOSE(hw, "Using EGL_KHR_stream path.\n");
// Hope that the given texture unit range is not "in use" by anything.
// The texture units need to be bound during init only, and are free for
// use again after the initialization here is done.
int texunits = 0; // [texunits, texunits + num_planes)
int num_planes = 2;
int gl_target = GL_TEXTURE_EXTERNAL_OES;
p->egl_stream = p->CreateStreamKHR(p->egl_display, (EGLint[]){EGL_NONE});
if (!p->egl_stream)
goto fail;
for (int n = 0; n < num_planes; n++) {
gl->ActiveTexture(GL_TEXTURE0 + texunits + n);
gl->GenTextures(1, &p->gl_textures[n]);
gl->BindTexture(gl_target, p->gl_textures[n]);
gl->TexParameteri(gl_target, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
gl->TexParameteri(gl_target, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
gl->TexParameteri(gl_target, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
gl->TexParameteri(gl_target, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
}
EGLAttrib attrs[] = {
EGL_COLOR_BUFFER_TYPE, EGL_YUV_BUFFER_EXT,
EGL_YUV_NUMBER_OF_PLANES_EXT, num_planes,
EGL_YUV_PLANE0_TEXTURE_UNIT_NV, texunits + 0,
EGL_YUV_PLANE1_TEXTURE_UNIT_NV, texunits + 1,
EGL_NONE,
};
if (!p->StreamConsumerGLTextureExternalAttribsNV(p->egl_display, p->egl_stream,
attrs))
goto fail;
if (!p->CreateStreamProducerD3DTextureNV12ANGLE(p->egl_display, p->egl_stream,
(EGLAttrib[]){EGL_NONE}))
goto fail;
params->imgfmt = params->hw_subfmt;
for (int n = 0; n < num_planes; n++) {
gl->ActiveTexture(GL_TEXTURE0 + texunits + n);
gl->BindTexture(gl_target, 0);
}
gl->ActiveTexture(GL_TEXTURE0);
return 0;
fail:
MP_ERR(hw, "Failed to create EGLStream\n");
if (p->egl_stream)
p->DestroyStreamKHR(p->egl_display, p->egl_stream);
p->egl_stream = 0;
gl->ActiveTexture(GL_TEXTURE0);
return -1;
}
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
static int reinit(struct gl_hwdec *hw, struct mp_image_params *params)
{
struct priv *p = hw->priv;
GL *gl = hw->gl;
HRESULT hr;
destroy_objects(hw);
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
p->image_params = *params;
// If this does not work, use the video process instead.
if (create_egl_stream(hw, params) >= 0)
return 0;
MP_VERBOSE(hw, "Using ID3D11VideoProcessor path.\n");
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
D3D11_TEXTURE2D_DESC texdesc = {
.Width = params->w,
.Height = params->h,
.Format = DXGI_FORMAT_B8G8R8A8_UNORM,
.MipLevels = 1,
.ArraySize = 1,
.SampleDesc = { .Count = 1 },
.Usage = D3D11_USAGE_DEFAULT,
.BindFlags = D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE,
.MiscFlags = D3D11_RESOURCE_MISC_SHARED,
};
hr = ID3D11Device_CreateTexture2D(p->d3d11_device, &texdesc, NULL, &p->texture);
if (FAILED(hr)) {
MP_ERR(hw, "Failed to create texture: %s\n", mp_HRESULT_to_str(hr));
goto fail;
}
HANDLE share_handle = NULL;
IDXGIResource *res;
hr = IUnknown_QueryInterface(p->texture, &IID_IDXGIResource, (void **)&res);
if (FAILED(hr))
goto fail;
hr = IDXGIResource_GetSharedHandle(res, &share_handle);
if (FAILED(hr))
share_handle = NULL;
IDXGIResource_Release(res);
if (!share_handle)
goto fail;
EGLint attrib_list[] = {
EGL_WIDTH, params->w,
EGL_HEIGHT, params->h,
EGL_TEXTURE_FORMAT, EGL_TEXTURE_RGBA,
EGL_TEXTURE_TARGET, EGL_TEXTURE_2D,
EGL_NONE
};
p->egl_surface = eglCreatePbufferFromClientBuffer(
p->egl_display, EGL_D3D_TEXTURE_2D_SHARE_HANDLE_ANGLE,
share_handle, p->egl_config, attrib_list);
if (p->egl_surface == EGL_NO_SURFACE) {
MP_ERR(hw, "Failed to create EGL surface\n");
goto fail;
}
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
gl->GenTextures(1, &p->gl_textures[0]);
gl->BindTexture(GL_TEXTURE_2D, p->gl_textures[0]);
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
vo_opengl: d3d11egl: minor simplification This should be ok. eglBindTexImage() just associates the texture, and does not make a copy (not even a conceptual one).
2016-04-27 12:35:24 +00:00
eglBindTexImage(p->egl_display, p->egl_surface, EGL_BACK_BUFFER);
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
gl->BindTexture(GL_TEXTURE_2D, 0);
vo_opengl: refactor how hwdec interop exports textures Rename gl_hwdec_driver.map_image to map_frame, and let it fill out a struct gl_hwdec_frame describing the exact texture layout. This gives more flexibility to what the hwdec interop can export. In particular, it can export strange component orders/permutations and textures with padded size. (The latter originating from cropped video.) The way gl_hwdec_frame works is in the spirit of the rest of the vo_opengl video processing code, which tends to put as much information in immediate state (as part of the dataflow), instead of declaring it globally. To some degree this duplicates the texplane and img_tex structs, but until we somehow unify those, it's better to give the hwdec state its own struct. The fact that changing the hwdec struct would require changes and testing on at least 4 platform/GPU combinations makes duplicating it almost a requirement to avoid pain later. Make gl_hwdec_driver.reinit set the new image format and remove the gl_hwdec.converted_imgfmt field. Likewise, gl_hwdec.gl_texture_target is replaced with gl_hwdec_plane.gl_target. Split out a init_image_desc function from init_format. The latter is not called in the hwdec case at all anymore. Setting up most of struct texplane is also completely separate in the hwdec and normal cases. video.c does not check whether the hwdec "mapped" image format is supported. This should not really happen anyway, and if it does, the hwdec interop backend must fail at creation time, so this is not an issue.
2016-05-10 16:29:10 +00:00
params->imgfmt = IMGFMT_RGB0;
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
return 0;
fail:
destroy_objects(hw);
return -1;
}
static int create_video_proc(struct gl_hwdec *hw, struct mp_image_params *params)
{
struct priv *p = hw->priv;
HRESULT hr;
destroy_video_proc(hw);
// Note: we skip any deinterlacing considerations for now.
D3D11_VIDEO_PROCESSOR_CONTENT_DESC vpdesc = {
.InputWidth = p->c_w,
.InputHeight = p->c_h,
.OutputWidth = params->w,
.OutputHeight = params->h,
};
hr = ID3D11VideoDevice_CreateVideoProcessorEnumerator(p->video_dev, &vpdesc,
&p->vp_enum);
if (FAILED(hr))
goto fail;
// Assume RateConversionIndex==0 always works fine for us.
hr = ID3D11VideoDevice_CreateVideoProcessor(p->video_dev, p->vp_enum, 0,
&p->video_proc);
if (FAILED(hr)) {
MP_ERR(hw, "Failed to create D3D11 video processor.\n");
goto fail;
}
D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC outdesc = {
.ViewDimension = D3D11_VPOV_DIMENSION_TEXTURE2D,
};
hr = ID3D11VideoDevice_CreateVideoProcessorOutputView(p->video_dev,
(ID3D11Resource *)p->texture,
p->vp_enum, &outdesc,
&p->out_view);
if (FAILED(hr))
goto fail;
// Note: libavcodec does not support cropping left/top with hwaccel.
RECT src_rc = {
.right = params->w,
.bottom = params->h,
};
ID3D11VideoContext_VideoProcessorSetStreamSourceRect(p->video_ctx,
p->video_proc,
0, TRUE, &src_rc);
// This is supposed to stop drivers from fucking up the video quality.
ID3D11VideoContext_VideoProcessorSetStreamAutoProcessingMode(p->video_ctx,
p->video_proc,
0, FALSE);
vo_opengl: d3d11egl: support full range YUV MSDN documents this as "Introduced in Windows 8.1.". I assume on Windows 7 this field will simply be ignored. Too bad for Windows 7 users. Also, I'm not using D3D11_VIDEO_PROCESSOR_NOMINAL_RANGE_16_235 and D3D11_VIDEO_PROCESSOR_NOMINAL_RANGE_0_255, because these are apparently completely missing from the MinGW headers. (Such a damn pain.)
2016-05-11 13:23:02 +00:00
if (params->colorspace != MP_CSP_BT_601 &&
params->colorspace != MP_CSP_BT_709)
vo_opengl: d3d11egl: print warning on unsupported colorspaces Not much we can do about. If there are many complaints, a mechanism to automatically disable interop in such cases could be added.
2016-04-27 12:23:03 +00:00
{
MP_WARN(hw, "Unsupported video colorspace (%s/%s). Consider disabling "
"hardware decoding, or using --hwdec=d3d11va-copy to get "
"correct output.\n",
m_opt_choice_str(mp_csp_names, params->colorspace),
m_opt_choice_str(mp_csp_levels_names, params->colorlevels));
}
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
D3D11_VIDEO_PROCESSOR_COLOR_SPACE csp = {
.YCbCr_Matrix = params->colorspace != MP_CSP_BT_601,
vo_opengl: d3d11egl: support full range YUV MSDN documents this as "Introduced in Windows 8.1.". I assume on Windows 7 this field will simply be ignored. Too bad for Windows 7 users. Also, I'm not using D3D11_VIDEO_PROCESSOR_NOMINAL_RANGE_16_235 and D3D11_VIDEO_PROCESSOR_NOMINAL_RANGE_0_255, because these are apparently completely missing from the MinGW headers. (Such a damn pain.)
2016-05-11 13:23:02 +00:00
.Nominal_Range = params->colorlevels == MP_CSP_LEVELS_TV ? 1 : 2,
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
};
ID3D11VideoContext_VideoProcessorSetStreamColorSpace(p->video_ctx,
p->video_proc,
0, &csp);
return 0;
fail:
destroy_video_proc(hw);
return -1;
}
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
static int map_frame_video_proc(struct gl_hwdec *hw, ID3D11Texture2D *d3d_tex,
int d3d_subindex, struct gl_hwdec_frame *out_frame)
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
{
struct priv *p = hw->priv;
HRESULT hr;
ID3D11VideoProcessorInputView *in_view = NULL;
D3D11_TEXTURE2D_DESC texdesc;
ID3D11Texture2D_GetDesc(d3d_tex, &texdesc);
if (!p->video_proc || p->c_w != texdesc.Width || p->c_h != texdesc.Height) {
p->c_w = texdesc.Width;
p->c_h = texdesc.Height;
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
if (create_video_proc(hw, &p->image_params) < 0)
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
return -1;
}
D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC indesc = {
.FourCC = 0, // huh?
.ViewDimension = D3D11_VPIV_DIMENSION_TEXTURE2D,
.Texture2D = {
.ArraySlice = d3d_subindex,
},
};
hr = ID3D11VideoDevice_CreateVideoProcessorInputView(p->video_dev,
(ID3D11Resource *)d3d_tex,
p->vp_enum, &indesc,
&in_view);
if (FAILED(hr)) {
MP_ERR(hw, "Could not create ID3D11VideoProcessorInputView\n");
return -1;
}
D3D11_VIDEO_PROCESSOR_STREAM stream = {
.Enable = TRUE,
.pInputSurface = in_view,
};
hr = ID3D11VideoContext_VideoProcessorBlt(p->video_ctx, p->video_proc,
p->out_view, 0, 1, &stream);
ID3D11VideoProcessorInputView_Release(in_view);
if (FAILED(hr)) {
MP_ERR(hw, "VideoProcessorBlt failed.\n");
return -1;
}
vo_opengl: refactor how hwdec interop exports textures Rename gl_hwdec_driver.map_image to map_frame, and let it fill out a struct gl_hwdec_frame describing the exact texture layout. This gives more flexibility to what the hwdec interop can export. In particular, it can export strange component orders/permutations and textures with padded size. (The latter originating from cropped video.) The way gl_hwdec_frame works is in the spirit of the rest of the vo_opengl video processing code, which tends to put as much information in immediate state (as part of the dataflow), instead of declaring it globally. To some degree this duplicates the texplane and img_tex structs, but until we somehow unify those, it's better to give the hwdec state its own struct. The fact that changing the hwdec struct would require changes and testing on at least 4 platform/GPU combinations makes duplicating it almost a requirement to avoid pain later. Make gl_hwdec_driver.reinit set the new image format and remove the gl_hwdec.converted_imgfmt field. Likewise, gl_hwdec.gl_texture_target is replaced with gl_hwdec_plane.gl_target. Split out a init_image_desc function from init_format. The latter is not called in the hwdec case at all anymore. Setting up most of struct texplane is also completely separate in the hwdec and normal cases. video.c does not check whether the hwdec "mapped" image format is supported. This should not really happen anyway, and if it does, the hwdec interop backend must fail at creation time, so this is not an issue.
2016-05-10 16:29:10 +00:00
*out_frame = (struct gl_hwdec_frame){
.planes = {
{
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
.gl_texture = p->gl_textures[0],
vo_opengl: refactor how hwdec interop exports textures Rename gl_hwdec_driver.map_image to map_frame, and let it fill out a struct gl_hwdec_frame describing the exact texture layout. This gives more flexibility to what the hwdec interop can export. In particular, it can export strange component orders/permutations and textures with padded size. (The latter originating from cropped video.) The way gl_hwdec_frame works is in the spirit of the rest of the vo_opengl video processing code, which tends to put as much information in immediate state (as part of the dataflow), instead of declaring it globally. To some degree this duplicates the texplane and img_tex structs, but until we somehow unify those, it's better to give the hwdec state its own struct. The fact that changing the hwdec struct would require changes and testing on at least 4 platform/GPU combinations makes duplicating it almost a requirement to avoid pain later. Make gl_hwdec_driver.reinit set the new image format and remove the gl_hwdec.converted_imgfmt field. Likewise, gl_hwdec.gl_texture_target is replaced with gl_hwdec_plane.gl_target. Split out a init_image_desc function from init_format. The latter is not called in the hwdec case at all anymore. Setting up most of struct texplane is also completely separate in the hwdec and normal cases. video.c does not check whether the hwdec "mapped" image format is supported. This should not really happen anyway, and if it does, the hwdec interop backend must fail at creation time, so this is not an issue.
2016-05-10 16:29:10 +00:00
.gl_target = GL_TEXTURE_2D,
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
.tex_w = p->image_params.w,
.tex_h = p->image_params.h,
vo_opengl: refactor how hwdec interop exports textures Rename gl_hwdec_driver.map_image to map_frame, and let it fill out a struct gl_hwdec_frame describing the exact texture layout. This gives more flexibility to what the hwdec interop can export. In particular, it can export strange component orders/permutations and textures with padded size. (The latter originating from cropped video.) The way gl_hwdec_frame works is in the spirit of the rest of the vo_opengl video processing code, which tends to put as much information in immediate state (as part of the dataflow), instead of declaring it globally. To some degree this duplicates the texplane and img_tex structs, but until we somehow unify those, it's better to give the hwdec state its own struct. The fact that changing the hwdec struct would require changes and testing on at least 4 platform/GPU combinations makes duplicating it almost a requirement to avoid pain later. Make gl_hwdec_driver.reinit set the new image format and remove the gl_hwdec.converted_imgfmt field. Likewise, gl_hwdec.gl_texture_target is replaced with gl_hwdec_plane.gl_target. Split out a init_image_desc function from init_format. The latter is not called in the hwdec case at all anymore. Setting up most of struct texplane is also completely separate in the hwdec and normal cases. video.c does not check whether the hwdec "mapped" image format is supported. This should not really happen anyway, and if it does, the hwdec interop backend must fail at creation time, so this is not an issue.
2016-05-10 16:29:10 +00:00
},
},
};
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
return 0;
}
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
static int map_frame_egl_stream(struct gl_hwdec *hw, ID3D11Texture2D *d3d_tex,
int d3d_subindex, struct gl_hwdec_frame *out_frame)
{
struct priv *p = hw->priv;
EGLAttrib attrs[] = {
EGL_D3D_TEXTURE_SUBRESOURCE_ID_ANGLE, d3d_subindex,
EGL_NONE,
};
if (!p->StreamPostD3DTextureNV12ANGLE(p->egl_display, p->egl_stream,
(void *)d3d_tex, attrs))
return -1;
if (!p->StreamConsumerAcquireKHR(p->egl_display, p->egl_stream))
return -1;
D3D11_TEXTURE2D_DESC texdesc;
ID3D11Texture2D_GetDesc(d3d_tex, &texdesc);
*out_frame = (struct gl_hwdec_frame){
.planes = {
{
.gl_texture = p->gl_textures[0],
.gl_target = GL_TEXTURE_EXTERNAL_OES,
.tex_w = texdesc.Width,
.tex_h = texdesc.Height,
},
{
.gl_texture = p->gl_textures[1],
.gl_target = GL_TEXTURE_EXTERNAL_OES,
.tex_w = texdesc.Width / 2,
.tex_h = texdesc.Height / 2,
.swizzle = "rgba", // even in ES2 mode (no LUMINANCE_ALPHA)
},
},
};
return 0;
}
static int map_frame(struct gl_hwdec *hw, struct mp_image *hw_image,
struct gl_hwdec_frame *out_frame)
{
struct priv *p = hw->priv;
if (!p->gl_textures[0])
return -1;
ID3D11Texture2D *d3d_tex = (void *)hw_image->planes[1];
int d3d_subindex = (intptr_t)hw_image->planes[2];
if (!d3d_tex)
return -1;
return p->egl_stream
? map_frame_egl_stream(hw, d3d_tex, d3d_subindex, out_frame)
: map_frame_video_proc(hw, d3d_tex, d3d_subindex, out_frame);
}
static void unmap(struct gl_hwdec *hw)
{
struct priv *p = hw->priv;
if (p->egl_stream)
p->StreamConsumerReleaseKHR(p->egl_display, p->egl_stream);
}
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
const struct gl_hwdec_driver gl_hwdec_d3d11egl = {
.name = "d3d11-egl",
.api = HWDEC_D3D11VA,
.imgfmt = IMGFMT_D3D11VA,
.create = create,
.reinit = reinit,
vo_opengl: refactor how hwdec interop exports textures Rename gl_hwdec_driver.map_image to map_frame, and let it fill out a struct gl_hwdec_frame describing the exact texture layout. This gives more flexibility to what the hwdec interop can export. In particular, it can export strange component orders/permutations and textures with padded size. (The latter originating from cropped video.) The way gl_hwdec_frame works is in the spirit of the rest of the vo_opengl video processing code, which tends to put as much information in immediate state (as part of the dataflow), instead of declaring it globally. To some degree this duplicates the texplane and img_tex structs, but until we somehow unify those, it's better to give the hwdec state its own struct. The fact that changing the hwdec struct would require changes and testing on at least 4 platform/GPU combinations makes duplicating it almost a requirement to avoid pain later. Make gl_hwdec_driver.reinit set the new image format and remove the gl_hwdec.converted_imgfmt field. Likewise, gl_hwdec.gl_texture_target is replaced with gl_hwdec_plane.gl_target. Split out a init_image_desc function from init_format. The latter is not called in the hwdec case at all anymore. Setting up most of struct texplane is also completely separate in the hwdec and normal cases. video.c does not check whether the hwdec "mapped" image format is supported. This should not really happen anyway, and if it does, the hwdec interop backend must fail at creation time, so this is not an issue.
2016-05-10 16:29:10 +00:00
.map_frame = map_frame,
vo_opengl: d3d11egl: native NV12 sampling support This uses EGL_ANGLE_stream_producer_d3d_texture_nv12 and related extensions to map the D3D textures coming from the hardware decoder directly in GL. In theory this would be trivial to achieve, but unfortunately ANGLE does not have a mechanism to "import" D3D textures as GL textures. Instead, an awkward mechanism via EGL_KHR_stream was implemented, which involves at least 5 extensions and a lot of glue code. (Even worse than VAAPI EGL interop, and very far from the simplicity you get on OSX.) The ANGLE mechanism so far supports only the NV12 texture format, which means 10 bit won't work. It also does not work in ES3 mode yet. For these reasons, the "old" ID3D11VideoProcessor code is kept and used as a fallback.
2016-05-10 18:37:03 +00:00
.unmap = unmap,
vo_opengl: D3D11VA + ANGLE interop This uses ID3D11VideoProcessor to convert the video to a RGBA surface, which is then bound to ANGLE. Currently ANGLE does not provide any way to bind nv12 surfaces directly, so this will have to do. ID3D11VideoContext1 would give us slightly more control about the colorspace conversion, though it's still not good, and not available in MinGW headers yet. The video processor is created lazily, because we need to have the coded frame size, of which AVFrame and mp_image have no concept of. Doing the creation lazily is less of a pain than somehow hacking the coded frame size into mp_image. I'm not really sure how ID3D11VideoProcessorInputView is supposed to work. We recreate it on every frame, which is simple and hopefully doesn't affect performance.
2016-04-27 11:49:47 +00:00
.destroy = destroy,
};