Commit Graph

111 Commits

Author SHA1 Message Date
sfan5 861e9566f6 video: don't define IMGFMT_VULKAN conditionally
We generally try to avoid that due to the #ifdef mess.
The equivalent format is defined in ffmpeg 4.4 while our interop code
requires a higher version, but that doesn't cause any problems.
2024-02-26 17:37:20 +01:00
Kacper Michajłow 66e451f4e6 csputils: replace mp_colorspace with pl_color_space 2024-01-22 14:54:55 +00:00
NRK b26d2ab01c img_format: remove duplicated macros
these are defined in osdep/endian.h already
2023-10-23 00:10:42 +00:00
Philip Langdale 61e685594d hwdec_vulkan: add Vulkan HW Interop
Vulkan Video Decoding has finally become a reality, as it's now
showing up in shipping drivers, and the ffmpeg support has been
merged.

With that in mind, this change introduces HW interop support for
ffmpeg Vulkan frames. The implementation is functionally complete - it
can display frames produced by hardware decoding, and it can work with
ffmpeg vulkan filters. There are still various caveats due to gaps and
bugs in drivers, so YMMV, as always.

Primary testing has been done on Intel, AMD, and nvidia hardware on
Linux with basic Windows testing on nvidia.

Notable caveats:
* Due to driver bugs, video decoding on nvidia does not work right now,
  unless you use the Vulkan Beta driver. It can be worked around, but
  requires ffmpeg changes that are not considered acceptable to merge.
* Even if those work-arounds are applied, Vulkan filters will not work
  on video that was decoded by Vulkan, due to additional bugs in the
  nvidia drivers. The filters do work correctly on content decoded some
  other way, and then uploaded to Vulkan (eg: Decode with nvdec, upload
  with --vf=format=vulkan)
* Vulkan filters can only be used with drivers that support
  VK_EXT_descriptor_buffer which doesn't include Intel ANV as yet.
  There is an MR outstanding for this.
* When dealing with 1080p content, there may be some visual distortion
  in the bottom lines of frames due to chroma scaling incorporating the
  extra hidden lines at the bottom of the frame (1080p content is
  actually stored as 1088 lines), depending on the hardware/driver
  combination and the scaling algorithm. This cannot be easily
  addressed as the mechanical fix for it violates the Vulkan spec, and
  probably requires a spec change to resolve properly.

All of these caveats will be fixed in either drivers or ffmpeg, and so
will not require mpv changes (unless something unexpected happens)

If you want to run on nvidia with the non-beta drivers, you can this
ffmpeg tree with the work-around patches:

* https://github.com/philipl/FFmpeg/tree/vulkan-nvidia-workarounds
2023-05-28 15:46:05 -07:00
Philip Langdale 48bb17549c filters/f_hwtransfer: remove VAAPI <-> Vulkan mapping for now
This mapping isn't actually relevant until we have the Vulkan interop
merged, and it requires a newer version of libavutil than our minimum
requirement. So I'm going to remove it from master and put it in the
interop PR.

Fixes #10813
2022-10-29 18:49:46 -07:00
Aaron Boxer fbcd9c140f mp_imgfmt: move DRMPRIME format to end of enum
dmabuf-wayland vo supports both DRMPRIME and VAAPI formats.
If upload filter is needed, we want to try VAAPI hwdec first
since most users will be using VAAPI, and there is a several second
delay when DRMPRIME hwdec is tried and fails.
2022-10-26 18:41:47 +00:00
Philip Langdale f221666ade f_hwtransfer: mp_image_pool: support HW -> HW mapping
Certain combinations of hardware formats require the use of hwmap to
transfer frames between the formats, rather than hwupload, which will
fail if attempted.

To keep the usage of vf_format for HW -> HW transfers as intuitive as
possible, we should detect these cases and do the map operation instead
of uploading.

For now, the relevant cases are moving between VAAPI and Vulkan, and
VAAPI and DRM Prime, in both directions. I have introduced the IMGFMT
entry for Vulkan here so that I can put in the complete mapping table.
It's actually not useless, as you can map to Vulkan, use a Vulkan
filter and then map back to VAAPI for display output.
2022-09-21 09:39:34 -07:00
wm4 2525aa3393 img_format: expose another helper 2020-05-21 02:29:05 +02:00
wm4 eb27baf5ec video: remove useless mp_imgfmt_desc.avformat field
Had 1 user; easily replaced.
2020-05-20 18:38:19 +02:00
wm4 086953a9da video: clean up pixel metadata stuff some more
A repeat of the previous useless commits.

Pondered whether to use separate fields or just a flags integer for
color and component types; the latter won for now.

Functions like mp_imgfmt_get_component_type() are now discouraged, and
mp_imgfmt_desc.flags is back for defining all information. Some days ago
I felt like the opposite would be the better design. Fortunately, it
doesn't matter.

With this, I think all image format properties that mpv needs are
exhaustively defined all in one place.
2020-05-20 18:38:15 +02:00
wm4 176f422213 video: shuffle imgfmt metadata code around
I guess I decided to stuff it all into mp_imgfmt_desc (the "old"
struct). This is probably a mistake. At first I was afraid that this
struct would get too fat (probably justified, and hereby happened), but
on the other hand mp_imgfmt_get_desc() (which builds the struct) calls
the former mp_imgfmt_get_layout(), and the separation doesn't make too
much sense anyway. Just merge them.

Still, try to keep out the extra info for packed YUV bullshit. I think
the result is OK, and there's as much information as there was before.

The test output changes a little. There's no independent bits[] array
anymore, so formats which did not previously have set this now show it.
(These formats are mpv-only and are still missing the metadata. To be
added later). Also, the output for the cursed packed formats changes.
2020-05-20 00:02:27 +02:00
wm4 27e5416c12 video: add pixel component location metadata
I thought I'd probably want something like this, so the hardcoded stuff
in repack.c can be removed eventually. Of course this has no purpose at
all, and will not have any. (For now, this provides only metadata, and
nothing uses it, apart from the "test" that dumps it as text.)

This adds full support for AV_PIX_FMT_UYYVYY411 (probably out of spite,
because the format is 100% useless). Support for some mpv-only formats
is missing, ironically.

The code goes through _lengths_ to try to make sense out of the FFmpeg
AVPixFmtDescriptor data. Which is even more amazing that the new
metadata basically mirrors pixdesc, and just adds to it. Considering
code complexity and speed issues (it takes time to crunch through all
this shit all the time), and especially the fact that pixdesc is very
_incomplete_, it would probably better to have our own table to all
formats. But then we'd not scramble every time FFmpeg adds a new format,
which would be annoying. On the other hand, by using pixdesc, we get the
excitement to see whether this code will work, or break everything in
catastrophic ways.

The data structure still sucks a lot. Maybe I'll redo it again. The text
dump is weirdly differently formatted than the C struct - because I'm
not happy with the representation. Maybe I'll redo it all over again.

In summary: this commit does nothing.
2020-05-18 01:54:59 +02:00
wm4 caee8748da video: clean up some imgfmt related stuff
Remove the vaguely defined plane_bits and component_bits fields from
struct mp_imgfmt_desc. Add weird replacements for existing uses. Remove
the bytes[] field, replace uses with bpp[].

Fix some potential alignment issues in existing code. As a compromise,
split mp_image_pixel_ptr() into 2 functions, because I think it's a bad
idea to implicitly round, but for some callers being slightly less
strict is convenient.

This shouldn't really change anything. In fact, it's a 100% useless
change. I'm just cleaning up what I started almost 8 years ago (see
commit 00653a3eb0). With this I've decided to keep mp_imgfmt_desc,
just removing the weird parts, and keeping the saner parts.
2020-05-18 01:54:59 +02:00
wm4 6db890ebab video: remove RGB32/BGR32 aliases
They are sort of confusing, and they hide the fact that they have an
alpha component. Using the actual formats directly is no problem, sicne
these were useful only for big endian systems, something we can't test
anyway.
2020-05-11 19:57:34 +02:00
wm4 56dbbc3847 video: add yuv float formats
Adding all these so I can use them for obscure processing purposes (see
later draw_bmp commit).

There isn't really a reason why they should exist. On the other hand,
they're just labels for formats that can be handled in a generic way,
and this commit adds support for them in the zimg wrapper and vo_gpu
just by making the formats exist. (Well, vo_gpu had to be fixed in the
previous commit.)
2020-05-09 18:02:57 +02:00
wm4 4019c11314 video: fix rgb30 component order
Was broken with a zimg wrapper refucktor before the previous commit. In
addition, it seems this didn't match the vo_drm format, or the format
naming convention. So the order actually changes, and the format is
redefined. (The img_format.h comment was probably wrong.)

Change vo_gpu to the new format as well, so we can still test it.
2020-05-09 18:02:57 +02:00
wm4 d8002f1dde video: separate repacking code from zimg and make it independent
For whatever purpose. If anything, this makes the zimg wrapper cleaner.

The added tests are not particular exhaustive, but nice to have. This
also makes the scale_zimg.c test pretty useless, because it only tests
repacking (going through the zimg wrapper). In theory, the repack_tests
things could also be used on scalers, but I guess it doesn't matter.

Some things are added over the previous zimg wrapper code. For example,
some fringe formats can now be expanded to 8 bit per component for
convenience.
2020-05-09 18:02:57 +02:00
wm4 43591ba96b img_format: make component order in comment more logical 2020-05-09 18:02:57 +02:00
wm4 8767c46873 img_format: add some mpv-only helper formats
Utterly useless, but the intention is to make dealing with corner case
pixel formats (forced upon us by FFmpeg, very rarely) less of a pain.
The zimg wrapper will use them. (It already supports these formats
automatically, but it will help with its internals.)

Y1 is considered RGB, even though gray formats are generally treated as
YUV for various reasons. mpv will default all YUV formats to limited
range internally, which makes no sense for a 1 bit format, so this is a
problem. I wanted to avoid that mp_image_params_guess_csp() (which
applies the default) explicitly checks for an image format, so although
a bit janky, this seems to be a good solution, especially because I
really don't give a shit about these formats, other than having to
handle them. It's notable that AV_PIX_FMT_MONOBLACK (also 1 bit gray,
just packed) already explicitly marked itself as RGB.
2020-04-23 13:24:35 +02:00
wm4 0f8f6a665b video: change chroma_w/chroma_h fields to use shift instead of size
When I added mp_regular_imgfmt, I made the chroma subsampling use the
actual chroma division factor, instead of a shift (log2 of the actual
value). I had some ideas about how this was (probably?) more intuitive
and general. But nothing ever uses non-power of 2 subsampling (except
jpeg in rare cases apparently, because the world is a bad place).

Change the fields back to use shifts and rename them to avoid mistakes.
2020-04-23 13:24:35 +02:00
wm4 7cb83593c2 img_format: add format description table for mpv-only formats
Make this slightly less ad-hoc. Also correct the missing alpha flag for
yap8/yap16.

Despite reduced redundancy, the LOC is going up anyway... whatever.
2020-04-23 13:24:35 +02:00
wm4 a8b84c9a1a zimg: add support for big endian input and output
One of the extremely annoying dumb things in ffmpeg is that most pixel
formats are available as little endian and big endian variants. (The
sane way would be having native endian formats only.) Usually, most of
the real codecs use native formats only, while non-native formats are
used by fringe raw codecs only. But the PNG encoders and decoders
unfortunately use big endian formats, and since PNG it such a popular
format, this causes problems for us. In particular, the current zimg
wrapper will refuse to work (and mpv will fall back to sws) when writing
non-8 bit PNGs.

So add non-native endian support to zimg. This is done in a fairly
"generic" way (which means lots of potential for bugs). If input is a
"regular" format (and just byte-swapped), the rest happens
automatically, which happens to cover all interesting formats.

Some things could be more efficient; for example, unpacking is done on
the data before it's passed to the unpacker. You could make endian
swapping part of the actual unpacking process, which might be slightly
faster. You could avoid copying twice in some cases (such as when
there's no actual repacker, or if alignment needs to be corrected). But
I don't really care. It's reasonably fast for the normal case.

Not entirely sure whether this is correct. Some (but not many) formats
are covered by the tests, some I tested manually. Some I can't even
test, because libswscale doesn't support them (like nv20*).
2020-04-13 15:56:27 +02:00
wm4 0df0a847f4 video: drop NV24 alias
Caused build failures with still supported FFmpeg versions. It's
unreferenced, so it's not needed.

Fixes: #7471
2020-02-18 18:03:42 +01:00
wm4 d699893dbd img_format: add alias for ffmpeg pal8 format
For the next commit.
2020-02-10 18:59:59 +01:00
wm4 a841fe9484 img_format: add gray/alpha planar formats
The zimg wrapper "needs" these formats as intermediary when repacking
the normal gray/alpha packed format. The packed format is used by the
png decoder and encoder, and is thus interesting.

Unfortunately, mpv-only formats are a mess right now, because all the
existing code is focused around using the FFmpeg metadata for pixel
formats. This should be improved, but not now, so make the mess worse.

This commit doesn't add support for it to the zimg wrapper yet.
2020-02-10 17:38:54 +01:00
wm4 bcfabf40a4 img_format: remove some unneeded alpha flag handling
Don't know what this was for, but the result doesn't change.
2019-11-08 21:22:49 +01:00
wm4 98352362ea img_format: remove some unused format flags
They were used at some point, but then fell into disuse. In general,
these old flags are all a bit fuzzy, so it's a good idea to remove them
as much as possible.

The comment about MP_IMGFLAG_PAL isn't true anymore. The old meaning was
deprecated at some point, and the flag was removed from "pseudo
paletted" formats. I think mpv at one point changed its own flag from
AV_PIX_FMT_FLAG_PSEUDOPAL to AV_PIX_FMT_FLAG_PAL, when the former was
deprecated, and it became unnecessary to allocate a palette for
non-paletted formats. (The one who deprecated in FFmpeg was me, if you
wonder.)

MP_IMGFLAG_PLANAR was used in command.c, use a relatively similar flag
as replacement.
2019-11-03 23:18:35 +01:00
wm4 c3cee4b9ec img_format: add function to find image format by layout
This is similar to mp_imgfmt_find(), but probably a bit saner. Used by
the next commit. The previous commit is required to map this
unambiguously between all formats.
2019-11-02 01:02:54 +01:00
wm4 bdd1e1e7ec img_format: add mp_regular_imgfmt.forced_csp field
As the code comment says, this is needed to disambiguate FFmpeg formats.
This struct only describes the "physical" layout of a format, while
FFmpeg also attaches part of the colorspace information to the format.
2019-11-02 01:00:32 +01:00
wm4 337ccc50dc img_format: add more explanations to component_pad field
Weird shit. I thought this was a clever way to elegantly handle two
cases at once, but maybe it's just confusing.
2019-11-02 00:55:55 +01:00
wm4 8f5979c5d8 img_format: add RGB30 format
FFmpeg does not support this from what I can see. This makes supporting
it a bit awkward.

Later commits use this format.
2019-10-20 19:41:18 +02:00
wm4 986daac45e img_format: document a minor guarantee for certain imgfmt metadata 2019-10-20 01:44:22 +02:00
Philip Langdale 1638fa7b46 vo/gpu: hwdec_vdpau: Support direct mode for 4:4:4 content
New releases of VDPAU support decoding 4:4:4 content, and that comes
back as NV24 when using 'direct mode' in OpenGL Interop. That means we
need to be a little bit smarter about how we set up the OpenGL
textures.
2019-07-08 01:11:27 +02:00
wm4 ebf7c04057 img_format.h: cosmetics: fix whitespace 2018-03-15 23:13:53 -07:00
wm4 9f595f3a80 vo_gpu: make screenshots use the GL renderer
Using the GL renderer for color conversion will make sure screenshots
will use the same conversion as normal video rendering. It can do this
for all types of screenshots.

The logic when to write 16 bit PNGs changes. To approximate the old
behavior, we decide by looking whether the source video format has more
than 8 bits per component. We apply this logic even for window
screenshots. Also, 16 bit PNGs now always include an unused alpha
channel. The reason is that FFmpeg has RGB48 and RGBA64 formats, but no
RGB064. RGB48 is 3 bytes and usually not supported by GPUs for
rendering, so we have to use RGBA64, which forces an alpha channel.

Will break for users who use --target-trc and similar options.

I considered creating a new gl_video context, but it could double GPU
memory use, so I didn't.

This uses FBOs instead of glGetTexImage(), because that increases the
chance it could work on GLES (e.g. ANGLE). Untested. No support for the
Vulkan and D3D11 backends yet.

Fixes #5498. Also fixes #5240, because the code for reading back is not
used with the new code path.
2018-02-11 17:45:51 -08:00
wm4 76276c9210 video: rewrite filtering glue code
Get rid of the old vf.c code. Replace it with a generic filtering
framework, which can potentially handle more than just --vf. At least
reimplementing --af with this code is planned.

This changes some --vf semantics (including runtime behavior and the
"vf" command). The most important ones are listed in interface-changes.

vf_convert.c is renamed to f_swscale.c. It is now an internal filter
that can not be inserted by the user manually.

f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed
once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is
conceptually easy, but a big mess due to the data flow changes).

The existing filters are all changed heavily. The data flow of the new
filter framework is different. Especially EOF handling changes - EOF is
now a "frame" rather than a state, and must be passed through exactly
once.

Another major thing is that all filters must support dynamic format
changes. The filter reconfig() function goes away. (This sounds complex,
but since all filters need to handle EOF draining anyway, they can use
the same code, and it removes the mess with reconfig() having to predict
the output format, which completely breaks with libavfilter anyway.)

In addition, there is no automatic format negotiation or conversion.
libavfilter's primitive and insufficient API simply doesn't allow us to
do this in a reasonable way. Instead, filters can use f_autoconvert as
sub-filter, and tell it which formats they support. This filter will in
turn add actual conversion filters, such as f_swscale, to perform
necessary format changes.

vf_vapoursynth.c uses the same basic principle of operation as before,
but with worryingly different details in data flow. Still appears to
work.

The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are
heavily changed. Fortunately, they all used refqueue.c, which is for
sharing the data flow logic (especially for managing future/past
surfaces and such). It turns out it can be used to factor out most of
the data flow. Some of these filters accepted software input. Instead of
having ad-hoc upload code in each filter, surface upload is now
delegated to f_autoconvert, which can use f_hwupload to perform this.

Exporting VO capabilities is still a big mess (mp_stream_info stuff).

The D3D11 code drops the redundant image formats, and all code uses the
hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a
big mess for now.

f_async_queue is unused.
2018-01-30 03:10:27 -08:00
wm4 7533d3c3a8 video: make IMGFMT_IS_HWACCEL() return 0 or 1
Sometimes helps avoiding usage mistakes.
2018-01-18 00:59:07 -08:00
wm4 2115e2306e video: add utility function to pick conversion image format from a list 2018-01-18 00:59:07 -08:00
Lionel CHAZALLON cfcee4cfe7 Add DRM_PRIME Format Handling and Display for RockChip MPP decoders
This commit allows to use the AV_PIX_FMT_DRM_PRIME newly introduced
format in ffmpeg that allows decoders to provide an AVDRMFrameDescriptor
struct.

That struct holds dmabuf fds and information allowing zerocopy rendering
using KMS / DRM Atomic.

This has been tested on RockChip ROCK64 device.
2017-10-23 21:07:24 +02:00
Aman Gupta 61a1612de9 hwdec: add mediacodec hardware decoder for IMGFMT_MEDIACODEC frames 2017-10-09 18:36:54 +02:00
wm4 63b1031ca2 vo_opengl: support float pixel formats
Like AV_PIX_FMT_GBRPF32LE.
2017-08-15 17:00:35 +02:00
wm4 60d0cea310 img_format: fix a comment
This was changed a while ago. Part of it might still apply to the old
D3D hwaccel glue code, though.
2017-07-15 13:04:32 +02:00
wm4 c61520b6bd img_format: drop some unused things 2017-06-30 18:38:23 +02:00
wm4 2aff6f8c95 vo_direct3d: remove non-working nv12 shader support
It never worked. It relied on some obscure texture format to provide the
equivalent of GL_RG or GL_LUMINANCE_ALPHA, but no hardware seemed to
report support for it ever. No idea what's the correct way to do this.
On D3D11 it exists, of course.

(Actually I'd like to remove the whole VO.)
2017-06-30 18:13:01 +02:00
wm4 1ad036a2ef video: get rid of swapped packed YUV
Another legacy annoyance. The only place where packed YUV is still
important is slightly older Apple hardware or drivers, which require
it for efficient hardware decoding.
2017-06-30 18:01:29 +02:00
wm4 300f6a3344 video: drop some more IMGFMT aliases
For vo_opengl and vo_direct3d, these are supported in a generic way.

For vf_vapoursynth, we could probably map its VSFormat struct in a
generic way, but for now do some bullshit.

vf_eq.c actually loses support for these formats. We could add generic
support too (anything that has 8 bit planes will work), but why bother.
The filter is deprecated anyway.
2017-06-29 21:30:10 +02:00
wm4 af5a71f52e video: drop some unused IMGFMT aliases
These formats are supported in a generic way.

To get rid of IMGFMT_NV21, remove support from vo_direct3d.c completely.
2017-06-29 20:56:29 +02:00
wm4 1dffcb0167 vo_opengl: rely on FFmpeg pixdesc a bit more
Add something that allows is to extract the component order from various
RGBA formats. In fact, also handle YUV, GBRP, and XYZ formats with this.

It introduces a new struct mp_regular_imgfmt, that hopefully will
eventually replace struct mp_imgfmt_desc. The latter is still needed by
a lot of code though, especially generic code. Also vo_opengl still uses
the old one, so this commit is sort of incomplete.

Due to its genericness, it's also possible that this commit introduces
rendering bugs, or accepts formats it shouldn't accept.
2017-06-29 20:52:05 +02:00
wm4 5ea851feae video/fmt-conversion, img_format: change license to LGPL
The problem with fmt-conversion.h is that "lucabe", who disagreed with
LGPL, originally wrote it. But it was actually rewritten by "reimar"
later. The original switch statement was replaced with a lookup table.
No code other than the imgfmt2pixfmt() function signature survives.
Neither the format pairs (PIXFMT<->IMGFMT), nor the concept of mapping
them, can be copyrighted.

So changing the license should be fine, because reimar and all other
authors involved with the new code agreed to LGPL.

We also don't consider format pairs added later as copyrightable.

(The direct-mapping idea mentioned in the "Copyright" file seems
attractive, and I might implement in later anyway.)

Likewise, there might be some format names added to img_format.h, which
are not covered by relicensing agreements. These all affect "later"
additions, and they follow either the FFmpeg PIXFMT naming or some other
pre-existing logic, so this should be fine.
2017-06-18 15:15:07 +02:00
wm4 456c2ad222 img_format: drop unused aliases
vo_opengl uses those automatically via pixdesc.
2017-06-18 15:15:07 +02:00