These tests should really be rewritten to be less stupid so they don't
break everytime ffmpeg updates its formats, but that's too much effort
right now. Bump the required libavutil version as well.
8e1ef7c38f
Major release means new formats and failing tests so just add the new
stuff to the ref again and bump the minimum libavutil version. The last
ffmpeg commit that added a new format and is the minimum required
version for these subset of tests is referenced below.
479747645f
A few formats were added recently, some were removed, and some flags
were changed. Maybe this should be better, but just update for now so
the tests stop failing.
Certain combinations of hardware formats require the use of hwmap to
transfer frames between the formats, rather than hwupload, which will
fail if attempted.
To keep the usage of vf_format for HW -> HW transfers as intuitive as
possible, we should detect these cases and do the map operation instead
of uploading.
For now, the relevant cases are moving between VAAPI and Vulkan, and
VAAPI and DRM Prime, in both directions. I have introduced the IMGFMT
entry for Vulkan here so that I can put in the complete mapping table.
It's actually not useless, as you can map to Vulkan, use a Vulkan
filter and then map back to VAAPI for display output.
It may be completely useless, and I can't verify it as no known samples
or other known/accessible software using it, but why not?
Putting this together with he 422 code requires making it slightly more
generic. I'm still staying with a "huge" if tree instead of a table to
select the scanline worker callback, because it's actually small and not
huge (although it not being generic still feels slightly painful).
A repeat of the previous useless commits.
Pondered whether to use separate fields or just a flags integer for
color and component types; the latter won for now.
Functions like mp_imgfmt_get_component_type() are now discouraged, and
mp_imgfmt_desc.flags is back for defining all information. Some days ago
I felt like the opposite would be the better design. Fortunately, it
doesn't matter.
With this, I think all image format properties that mpv needs are
exhaustively defined all in one place.
I guess I decided to stuff it all into mp_imgfmt_desc (the "old"
struct). This is probably a mistake. At first I was afraid that this
struct would get too fat (probably justified, and hereby happened), but
on the other hand mp_imgfmt_get_desc() (which builds the struct) calls
the former mp_imgfmt_get_layout(), and the separation doesn't make too
much sense anyway. Just merge them.
Still, try to keep out the extra info for packed YUV bullshit. I think
the result is OK, and there's as much information as there was before.
The test output changes a little. There's no independent bits[] array
anymore, so formats which did not previously have set this now show it.
(These formats are mpv-only and are still missing the metadata. To be
added later). Also, the output for the cursed packed formats changes.
My previous commit added support for this format, but it was still
broken, and prevented the allocation code from working. It's unknown
whether it's correct now (because this pixfmt is so obscure and useless,
there are no known samples around), but who cares.
I thought I'd probably want something like this, so the hardcoded stuff
in repack.c can be removed eventually. Of course this has no purpose at
all, and will not have any. (For now, this provides only metadata, and
nothing uses it, apart from the "test" that dumps it as text.)
This adds full support for AV_PIX_FMT_UYYVYY411 (probably out of spite,
because the format is 100% useless). Support for some mpv-only formats
is missing, ironically.
The code goes through _lengths_ to try to make sense out of the FFmpeg
AVPixFmtDescriptor data. Which is even more amazing that the new
metadata basically mirrors pixdesc, and just adds to it. Considering
code complexity and speed issues (it takes time to crunch through all
this shit all the time), and especially the fact that pixdesc is very
_incomplete_, it would probably better to have our own table to all
formats. But then we'd not scramble every time FFmpeg adds a new format,
which would be annoying. On the other hand, by using pixdesc, we get the
excitement to see whether this code will work, or break everything in
catastrophic ways.
The data structure still sucks a lot. Maybe I'll redo it again. The text
dump is weirdly differently formatted than the C struct - because I'm
not happy with the representation. Maybe I'll redo it all over again.
In summary: this commit does nothing.
Remove the vaguely defined plane_bits and component_bits fields from
struct mp_imgfmt_desc. Add weird replacements for existing uses. Remove
the bytes[] field, replace uses with bpp[].
Fix some potential alignment issues in existing code. As a compromise,
split mp_image_pixel_ptr() into 2 functions, because I think it's a bad
idea to implicitly round, but for some callers being slightly less
strict is convenient.
This shouldn't really change anything. In fact, it's a 100% useless
change. I'm just cleaning up what I started almost 8 years ago (see
commit 00653a3eb0). With this I've decided to keep mp_imgfmt_desc,
just removing the weird parts, and keeping the saner parts.
Whatever it's worth. Instead of doing a pretty stupid conversion to
float, just blend it directly. This works for most RGB formats that are
8 bits per component or below (the latter because we expand packed
fringe RGB formats for simplicity). For higher bit depth RGB this would
need extra code.
This checked params->color.space for being RGB. If the colorspace is
unset, this did dumb things because even if the imgfmt was a RGB one,
the colorspace was not set to RGB. This actually also happened to the
tests.
(Short-cutting RGB like this is actually wrong, since RGB could still
have strange gamma or primaries, which would warrant a full conversion.
So you'd need to check for these other parameters as well. To be fixed
later.)
draw_bmp.c is the software blender for subtitles and OSD. It's used by
encoding mode (burning subtitles), and some VOs, like vo_drm, vo_x11,
vo_xv, and possibly more.
This changes the algorithm from upsampling the video to 4:4:4 and then
blending to downsampling the OSD and then blending directly to video.
This has far-reaching consequences for its internals, and results in an
effective rewrite.
Since I wanted to avoid un-premultiplying, all blending is done with
premultiplied alpha. That's actually the sane thing to do. The old code
just didn't do it, because it's very weird in YUV fixed point.
Essentially, you'd have to compensate for the chroma centering constant
by subtracting src_alpha/255*128. This seemed so hairy (especially with
correct rounding and high bit depths involved) that I went for using
float.
I think it turned out mostly OK, although it's more complex and less
maintainable than before. reinit() is certainly a bit too long. While it
should be possible to optimize the RGB path more (for example by
blending directly instead of doing the stupid float conversion), this is
probably slower. vo_xv users probably lose in this, because it takes the
slowest path (due to subsampling requirements and using YUV).
Why this rewrite? Nobody knows. I simply forgot the reason. But you'll
have it anyway. Whether or not this would have required a full rewrite,
at least it supports target alpha now (you can for example hard sub
transparent PNGs, if you ever wanted to use mpv for this).
Remove the check in vf_sub. The new draw_bmp.c is not as reliant on
libswscale anymore (mostly uses repack.c now), and osd.c shows an
error message on missing support instead now.
Formats with chroma subsampling of 4 are not supported, because FFmpeg
doesn't provide pixfmt definitions for alpha variants. We could provide
those ourselves (relatively trivial), but why bother.
Adding all these so I can use them for obscure processing purposes (see
later draw_bmp commit).
There isn't really a reason why they should exist. On the other hand,
they're just labels for formats that can be handled in a generic way,
and this commit adds support for them in the zimg wrapper and vo_gpu
just by making the formats exist. (Well, vo_gpu had to be fixed in the
previous commit.)
Was broken with a zimg wrapper refucktor before the previous commit. In
addition, it seems this didn't match the vo_drm format, or the format
naming convention. So the order actually changes, and the format is
redefined. (The img_format.h comment was probably wrong.)
Change vo_gpu to the new format as well, so we can still test it.
For whatever purpose. If anything, this makes the zimg wrapper cleaner.
The added tests are not particular exhaustive, but nice to have. This
also makes the scale_zimg.c test pretty useless, because it only tests
repacking (going through the zimg wrapper). In theory, the repack_tests
things could also be used on scalers, but I guess it doesn't matter.
Some things are added over the previous zimg wrapper code. For example,
some fringe formats can now be expanded to 8 bit per component for
convenience.
This was inconsistent for unknown reason. monob was the way we wanted
it, and handling of monow was missing.
See the previous "img_format: add some mpv-only helper formats" commit.
Matters for the zimg wrapper.
Utterly useless, but the intention is to make dealing with corner case
pixel formats (forced upon us by FFmpeg, very rarely) less of a pain.
The zimg wrapper will use them. (It already supports these formats
automatically, but it will help with its internals.)
Y1 is considered RGB, even though gray formats are generally treated as
YUV for various reasons. mpv will default all YUV formats to limited
range internally, which makes no sense for a 1 bit format, so this is a
problem. I wanted to avoid that mp_image_params_guess_csp() (which
applies the default) explicitly checks for an image format, so although
a bit janky, this seems to be a good solution, especially because I
really don't give a shit about these formats, other than having to
handle them. It's notable that AV_PIX_FMT_MONOBLACK (also 1 bit gray,
just packed) already explicitly marked itself as RGB.
Make this slightly less ad-hoc. Also correct the missing alpha flag for
yap8/yap16.
Despite reduced redundancy, the LOC is going up anyway... whatever.
Again worthless, slow, and only for libswscale parity.
With this, we support all formats libswscale supports, except bayer
input, and rgb4/bgr4 output. We even support some formats libswscale
doesn't.
It's possible that the zimg wrapper isn't always as fast as libswscale.
But there is optimization potential: the inner repack loops are
self-contained enough that they could be reasonably be implemented in
assembler (probably), and doing everything slice-wise should reduce the
overhead of the separate pack/unpack stages.
Just lazily tested.
The comment on AV_PIX_FMT_Y210LE seems to be wrong. It claims it's "like
YUYV422", bit it seems more like YVYU422, at last the way libswscale
input treats it. Maybe Intel pays its developers too much?
The repacker inner lop is probably rather inefficient. In theory we
could optimize it by reading the packed pixels as words, doing the
component reshuffling using compile time values etc., but I'd rather
keep the code size small. It's already bad enough that we have to
support 16 bit per component variants, just because this one Intel guy
couldn't keep it in his pants. In general, I can't be bothered to spend
time on optimizing it; I'm only doing this for fun (i.e. masochistic
obligation).
Generic statement about how this is not really appropriate, etc., and
only useful for temporary debugging things, and how I commit it anyway
despite violating my own principles (and how I'd reject this change if
it came from you).
One of the extremely annoying dumb things in ffmpeg is that most pixel
formats are available as little endian and big endian variants. (The
sane way would be having native endian formats only.) Usually, most of
the real codecs use native formats only, while non-native formats are
used by fringe raw codecs only. But the PNG encoders and decoders
unfortunately use big endian formats, and since PNG it such a popular
format, this causes problems for us. In particular, the current zimg
wrapper will refuse to work (and mpv will fall back to sws) when writing
non-8 bit PNGs.
So add non-native endian support to zimg. This is done in a fairly
"generic" way (which means lots of potential for bugs). If input is a
"regular" format (and just byte-swapped), the rest happens
automatically, which happens to cover all interesting formats.
Some things could be more efficient; for example, unpacking is done on
the data before it's passed to the unpacker. You could make endian
swapping part of the actual unpacking process, which might be slightly
faster. You could avoid copying twice in some cases (such as when
there's no actual repacker, or if alignment needs to be corrected). But
I don't really care. It's reasonably fast for the normal case.
Not entirely sure whether this is correct. Some (but not many) formats
are covered by the tests, some I tested manually. Some I can't even
test, because libswscale doesn't support them (like nv20*).
This explicitly depends on the pixfmt list from FFmpeg (done so to
easily spot regression, incompatible changes, and other unexpected
things).
Some local changes in mpv change some of the output. For pal8 an alias
was added back, and the [GENERIC] markers are removed because the mpv
aliases are not dependent on the mpv config anymore (which was
unnecessary).
The other changes are due to ffmpeg adding some new formats.
The zimg wrapper "needs" these formats as intermediary when repacking
the normal gray/alpha packed format. The packed format is used by the
png decoder and encoder, and is thus interesting.
Unfortunately, mpv-only formats are a mess right now, because all the
existing code is focused around using the FFmpeg metadata for pixel
formats. This should be improved, but not now, so make the mess worse.
This commit doesn't add support for it to the zimg wrapper yet.
libzimg recently added direct alpha support and new API for it. (The API
change is rather minimal, and it turns out we can easily support old and
new zimg versions.)
This does not support _all_ alpha formats. For example, gray + alpha is
not supported yet, because my stupid design in the zimg wrapper would
require a planar gray + alpha format, while ffmpeg provides only a
packed one.
This tests the RGB repacker code in zimg, which deserves to be tested
because it's tricky and there will be more formats.
scale_test.c contains some code that can be used to test any scaler. Or
at least that would be great; currently it can only test repacking of
some byte-aligned-component RGB formats. It should be called
repack_test.c, but I'm too lazy to change the filename now.
The idea is that libswscale is used to cross-check the conversions
performed by the zimg wrapper. This is why it's "OK" that scale_test.c
does libswscale calls.
scale_sws.c is the equivalent to scale_zimg.c, and is of course
worthless (because it tests libswscale by comparing the results with
libswscale), but still might help with finding bugs in scale_test.c.
This borrows a sorted list of image formats from test/img_format.c, for
the same reason that file sorts them.
There's a slight possibility that this can be used to test vo_gpu.c too
some times in the future.
This is fragile enough that it warrants getting "monitored".
This takes the commented test program code from img_format.c, makes it
output to a text file, and then compares it to a "ref" file stored in
git.
Originally, I wanted to do the comparison etc. in a shell or Python
script. But why not do it in C. So mpv calls /usr/bin/diff as a
sub-process now.
This test will start producing different output if FFmpeg adds new pixel
formats or pixel format flags, or if mpv adds new IMGFMT (either aliases
to FFmpeg formats or own formats). That is unavoidable, and requires
manual inspection of the results, and then updating the ref file.
The changes in the non-test code are to guarantee that the format ID
conversion functions only translate between valid IDs.
Dudemanguy
Philip Langdale
wm4