mpv/video/fmt-conversion.c

113 lines
3.4 KiB
C
Raw Normal View History

/*
* This file is part of mpv.
*
* mpv is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* mpv is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
*/
#include <libavutil/pixdesc.h>
#include <libavutil/avutil.h>
#include "video/img_format.h"
#include "fmt-conversion.h"
static const struct {
int fmt;
enum AVPixelFormat pix_fmt;
} conversion_map[] = {
{IMGFMT_ARGB, AV_PIX_FMT_ARGB},
{IMGFMT_BGRA, AV_PIX_FMT_BGRA},
{IMGFMT_BGR24, AV_PIX_FMT_BGR24},
{IMGFMT_RGB565, AV_PIX_FMT_RGB565},
{IMGFMT_ABGR, AV_PIX_FMT_ABGR},
{IMGFMT_RGBA, AV_PIX_FMT_RGBA},
{IMGFMT_RGB24, AV_PIX_FMT_RGB24},
{IMGFMT_PAL8, AV_PIX_FMT_PAL8},
{IMGFMT_UYVY, AV_PIX_FMT_UYVY422},
{IMGFMT_NV12, AV_PIX_FMT_NV12},
{IMGFMT_Y8, AV_PIX_FMT_GRAY8},
{IMGFMT_Y16, AV_PIX_FMT_GRAY16},
{IMGFMT_420P, AV_PIX_FMT_YUV420P},
{IMGFMT_444P, AV_PIX_FMT_YUV444P},
video: decouple internal pixel formats from FourCCs mplayer's video chain traditionally used FourCCs for pixel formats. For example, it used IMGFMT_YV12 for 4:2:0 YUV, which was defined to the string 'YV12' interpreted as unsigned int. Additionally, it used to encode information into the numeric values of some formats. The RGB formats had their bit depth and endian encoded into the least significant byte. Extended planar formats (420P10 etc.) had chroma shift, endian, and component bit depth encoded. (This has been removed in recent commits.) Replace the FourCC mess with a simple enum. Remove all the redundant formats like YV12/I420/IYUV. Replace some image format names by something more intuitive, most importantly IMGFMT_YV12 -> IMGFMT_420P. Add img_fourcc.h, which contains the old IDs for code that actually uses FourCCs. Change the way demuxers, that output raw video, identify the video format: they set either MP_FOURCC_RAWVIDEO or MP_FOURCC_IMGFMT to request the rawvideo decoder, and sh_video->imgfmt specifies the pixel format. Like the previous hack, this is supposed to avoid the need for a complete codecs.cfg entry per format, or other lookup tables. (Note that the RGB raw video FourCCs mostly rely on ffmpeg's mappings for NUT raw video, but this is still considered better than adding a raw video decoder - even if trivial, it would be full of annoying lookup tables.) The TV code has not been tested. Some corrective changes regarding endian and other image format flags creep in.
2012-12-23 19:03:30 +00:00
// YUVJ are YUV formats that use the full Y range. Decoder color range
// information is used instead. Deprecated in ffmpeg.
{IMGFMT_420P, AV_PIX_FMT_YUVJ420P},
{IMGFMT_444P, AV_PIX_FMT_YUVJ444P},
{IMGFMT_BGR0, AV_PIX_FMT_BGR0},
{IMGFMT_0RGB, AV_PIX_FMT_0RGB},
{IMGFMT_RGB0, AV_PIX_FMT_RGB0},
{IMGFMT_0BGR, AV_PIX_FMT_0BGR},
{IMGFMT_RGBA64, AV_PIX_FMT_RGBA64},
#ifdef AV_PIX_FMT_X2RGB10
{IMGFMT_RGB30, AV_PIX_FMT_X2RGB10},
#endif
{IMGFMT_VDPAU, AV_PIX_FMT_VDPAU},
{IMGFMT_VIDEOTOOLBOX, AV_PIX_FMT_VIDEOTOOLBOX},
{IMGFMT_MEDIACODEC, AV_PIX_FMT_MEDIACODEC},
{IMGFMT_VAAPI, AV_PIX_FMT_VAAPI},
{IMGFMT_DXVA2, AV_PIX_FMT_DXVA2_VLD},
video: rewrite filtering glue code Get rid of the old vf.c code. Replace it with a generic filtering framework, which can potentially handle more than just --vf. At least reimplementing --af with this code is planned. This changes some --vf semantics (including runtime behavior and the "vf" command). The most important ones are listed in interface-changes. vf_convert.c is renamed to f_swscale.c. It is now an internal filter that can not be inserted by the user manually. f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is conceptually easy, but a big mess due to the data flow changes). The existing filters are all changed heavily. The data flow of the new filter framework is different. Especially EOF handling changes - EOF is now a "frame" rather than a state, and must be passed through exactly once. Another major thing is that all filters must support dynamic format changes. The filter reconfig() function goes away. (This sounds complex, but since all filters need to handle EOF draining anyway, they can use the same code, and it removes the mess with reconfig() having to predict the output format, which completely breaks with libavfilter anyway.) In addition, there is no automatic format negotiation or conversion. libavfilter's primitive and insufficient API simply doesn't allow us to do this in a reasonable way. Instead, filters can use f_autoconvert as sub-filter, and tell it which formats they support. This filter will in turn add actual conversion filters, such as f_swscale, to perform necessary format changes. vf_vapoursynth.c uses the same basic principle of operation as before, but with worryingly different details in data flow. Still appears to work. The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are heavily changed. Fortunately, they all used refqueue.c, which is for sharing the data flow logic (especially for managing future/past surfaces and such). It turns out it can be used to factor out most of the data flow. Some of these filters accepted software input. Instead of having ad-hoc upload code in each filter, surface upload is now delegated to f_autoconvert, which can use f_hwupload to perform this. Exporting VO capabilities is still a big mess (mp_stream_info stuff). The D3D11 code drops the redundant image formats, and all code uses the hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a big mess for now. f_async_queue is unused.
2018-01-16 10:53:44 +00:00
{IMGFMT_D3D11, AV_PIX_FMT_D3D11},
RPI support This requires FFmpeg git master for accelerated hardware decoding. Keep in mind that FFmpeg must be compiled with --enable-mmal. Libav will also work. Most things work. Screenshots don't work with accelerated/opaque decoding (except using full window screenshot mode). Subtitles are very slow - even simple but huge overlays can cause frame drops. This always uses fullscreen mode. It uses dispmanx and mmal directly, and there are no window managers or anything on this level. vo_opengl also kind of works, but is pretty useless and slow. It can't use opaque hardware decoding (copy back can be used by forcing the option --vd=lavc:h264_mmal). Keep in mind that the dispmanx backend is preferred over the X11 ones in case you're trying on X11; but X11 is even more useless on RPI. This doesn't correctly reject extended h264 profiles and thus doesn't fallback to software decoding. The hw supports only up to the high profile, and will e.g. return garbage for Hi10P video. This sets a precedent of enabling hw decoding by default, but only if RPI support is compiled (which most hopefully it will be disabled on desktop Linux platforms). While it's more or less required to use hw decoding on the weak RPI, it causes more problems than it solves on real platforms (Linux has the Intel GPU problem, OSX still has some cases with broken decoding.) So I can live with this compromise of having different defaults depending on the platform. Raspberry Pi 2 is required. This wasn't tested on the original RPI, though at least decoding itself seems to work (but full playback was not tested).
2015-03-29 13:12:11 +00:00
{IMGFMT_MMAL, AV_PIX_FMT_MMAL},
hwdec/opengl: Add support for CUDA and cuvid/NvDecode Nvidia's "NvDecode" API (up until recently called "cuvid" is a cross platform, but nvidia proprietary API that exposes their hardware video decoding capabilities. It is analogous to their DXVA or VDPAU support on Windows or Linux but without using platform specific API calls. As a rule, you'd rather use DXVA or VDPAU as these are more mature and well supported APIs, but on Linux, VDPAU is falling behind the hardware capabilities, and there's no sign that nvidia are making the investments to update it. Most concretely, this means that there is no VP8/9 or HEVC Main10 support in VDPAU. On the other hand, NvDecode does export vp8/9 and partial support for HEVC Main10 (more on that below). ffmpeg already has support in the form of the "cuvid" family of decoders. Due to the design of the API, it is best exposed as a full decoder rather than an hwaccel. As such, there are decoders like h264_cuvid, hevc_cuvid, etc. These decoders support two output paths today - in both cases, NV12 frames are returned, either in CUDA device memory or regular system memory. In the case of the system memory path, the decoders can be used as-is in mpv today with a command line like: mpv --vd=lavc:h264_cuvid foobar.mp4 Doing this will take advantage of hardware decoding, but the cost of the memcpy to system memory adds up, especially for high resolution video (4K etc). To avoid that, we need an hwdec that takes advantage of CUDA's OpenGL interop to copy from device memory into OpenGL textures. That is what this change implements. The process is relatively simple as only basic device context aquisition needs to be done by us - the CUDA buffer pool is managed by the decoder - thankfully. The hwdec looks a bit like the vdpau interop one - the hwdec maintains a single set of plane textures and each output frame is repeatedly mapped into these textures to pass on. The frames are always in NV12 format, at least until 10bit output supports emerges. The only slightly interesting part of the copying process is that CUDA works by associating PBOs, so we need to define these for each of the textures. TODO Items: * I need to add a download_image function for screenshots. This would do the same copy to system memory that the decoder's system memory output does. * There are items to investigate on the ffmpeg side. There appears to be a problem with timestamps for some content. Final note: I mentioned HEVC Main10. While there is no 10bit output support, NvDecode can return dithered 8bit NV12 so you can take advantage of the hardware acceleration. This particular mode requires compiling ffmpeg with a modified header (or possibly the CUDA 8 RC) and is not upstream in ffmpeg yet. Usage: You will need to specify vo=opengl and hwdec=cuda. Note that hwdec=auto will probably not work as it will try to use vdpau first. mpv --hwdec=cuda --vo=opengl foobar.mp4 If you want to use filters that require frames in system memory, just use the decoder directly without the hwdec, as documented above.
2016-09-04 22:23:55 +00:00
{IMGFMT_CUDA, AV_PIX_FMT_CUDA},
{IMGFMT_P010, AV_PIX_FMT_P010},
{IMGFMT_DRMPRIME, AV_PIX_FMT_DRM_PRIME},
hwdec_vulkan: add Vulkan HW Interop Vulkan Video Decoding has finally become a reality, as it's now showing up in shipping drivers, and the ffmpeg support has been merged. With that in mind, this change introduces HW interop support for ffmpeg Vulkan frames. The implementation is functionally complete - it can display frames produced by hardware decoding, and it can work with ffmpeg vulkan filters. There are still various caveats due to gaps and bugs in drivers, so YMMV, as always. Primary testing has been done on Intel, AMD, and nvidia hardware on Linux with basic Windows testing on nvidia. Notable caveats: * Due to driver bugs, video decoding on nvidia does not work right now, unless you use the Vulkan Beta driver. It can be worked around, but requires ffmpeg changes that are not considered acceptable to merge. * Even if those work-arounds are applied, Vulkan filters will not work on video that was decoded by Vulkan, due to additional bugs in the nvidia drivers. The filters do work correctly on content decoded some other way, and then uploaded to Vulkan (eg: Decode with nvdec, upload with --vf=format=vulkan) * Vulkan filters can only be used with drivers that support VK_EXT_descriptor_buffer which doesn't include Intel ANV as yet. There is an MR outstanding for this. * When dealing with 1080p content, there may be some visual distortion in the bottom lines of frames due to chroma scaling incorporating the extra hidden lines at the bottom of the frame (1080p content is actually stored as 1088 lines), depending on the hardware/driver combination and the scaling algorithm. This cannot be easily addressed as the mechanical fix for it violates the Vulkan spec, and probably requires a spec change to resolve properly. All of these caveats will be fixed in either drivers or ffmpeg, and so will not require mpv changes (unless something unexpected happens) If you want to run on nvidia with the non-beta drivers, you can this ffmpeg tree with the work-around patches: * https://github.com/philipl/FFmpeg/tree/vulkan-nvidia-workarounds
2022-03-12 19:21:29 +00:00
#if HAVE_VULKAN_INTEROP
{IMGFMT_VULKAN, AV_PIX_FMT_VULKAN},
#endif
{0, AV_PIX_FMT_NONE}
};
enum AVPixelFormat imgfmt2pixfmt(int fmt)
{
if (fmt == IMGFMT_NONE)
return AV_PIX_FMT_NONE;
if (fmt >= IMGFMT_AVPIXFMT_START && fmt < IMGFMT_AVPIXFMT_END) {
enum AVPixelFormat pixfmt = fmt - IMGFMT_AVPIXFMT_START;
// Avoid duplicate format - each format must be unique.
int mpfmt = pixfmt2imgfmt(pixfmt);
if (mpfmt == fmt && av_pix_fmt_desc_get(pixfmt))
return pixfmt;
return AV_PIX_FMT_NONE;
}
for (int i = 0; conversion_map[i].fmt; i++) {
if (conversion_map[i].fmt == fmt)
return conversion_map[i].pix_fmt;
}
return AV_PIX_FMT_NONE;
}
int pixfmt2imgfmt(enum AVPixelFormat pix_fmt)
{
if (pix_fmt == AV_PIX_FMT_NONE)
return IMGFMT_NONE;
for (int i = 0; conversion_map[i].pix_fmt != AV_PIX_FMT_NONE; i++) {
if (conversion_map[i].pix_fmt == pix_fmt)
return conversion_map[i].fmt;
}
int generic = IMGFMT_AVPIXFMT_START + pix_fmt;
if (generic < IMGFMT_AVPIXFMT_END && av_pix_fmt_desc_get(pix_fmt))
return generic;
return 0;
}