Instead of letting it keep decoding by trying to find a new frame,
"plug" the frame queue by not removing it. (Or actually, by putting
it back instead of discarding it.)
Matters for seamless looping (following commits), and possibly some
other corner cases.
The added function vf_unread_output_frame() is a bit of a sin, but still
reasonable, since its implementation is trivial.
This makes the difference between passing VA_FRAME_PICTURE or
VA_BOTTOM_FIELD for progressive frames (that should be force-
deinterlaced) to VAProcPipelineParameterBuffer.flags. VA-VPP doesn't
really seem to care, and we can get rid of mp_refqueue_is_interlaced()
entirely. It could be argued it's better to pass field flags instead of
the progressive flag.
"Real" frame flag vs. what we pretend it to be. It always used the real
flag, and thus never deinterlaced unflagged frames, even if the
suboption was set to "no".
I made this call up because I sort of thought this makes senssssse. I'm
now convinced that it does not, and even is actively harmful. I'm still
quite in the dark how the DirectD 11 video API is supposed to work with
synchronization, but at least for normal graphics this call would not
make much sense.
For example it should be set to IMGFMT_D3D11NV12 if it isn't already.
Otherwise, an assertion in vf.c could trigger.
This probably couldn't be provoked yet.
This involves multiple changes:
1. Brightness metadata is split into nominal peak and signal peak.
For a quick and dirty explanation: nominal peak is the brightest value
that your color space can represent (i.e. the brightness of an encoded
1.0), and signal peak is the brightest value that actually occurs in
the video (i.e. the brightest thing that's displayed).
2. vo_opengl uses a new decision logic to figure out the right nom_peak
and sig_peak for all situations. It also does a better job of picking
the right target gamut/colorspace to use for the OSD. (Which still is
and still should be treated as sRGB). This change in logic also
fixes#3293 en passant.
3. Since it was growing rapidly, the logic for auto-guessing / inferring
the right colorimetry configuration (in pass_colormanage) was split from
the logic for actually performing the adaptation (now pass_color_map).
Right now, the new logic doesn't do a whole lot since HDR metadata is
still ignored (but not for long).
This has two reasons:
1. I tend to add new fields to this metadata, and every time I've done
so I've consistently forgotten to update all of the dozens of places in
which this colorimetry metadata might end up getting used. While most
usages don't really care about most of the metadata, sometimes the
intend was simply to “copy” the colorimetry metadata from one struct to
another. With this being inside a substruct, those lines of code can now
simply read a.color = b.color without having to care about added or
removed fields.
2. It makes the type definitions nicer for upcoming refactors.
In going through all of the usages, I also expanded a few where I felt
that omitting the “young” fields was a bug.
See previous commit. (The mixing case was never supported, so this has
equivalent functionality.)
This also implicitly fixes a bug: the old code allocated image data for
the cropped surface size only, which means the
video_surface_get_bits_y_cb_cr call could write beyond the allocated
image memory.
Until now, we've always converted vdpau video surfaces to RGB, and then
mapped the resulting RGB texture. Change this so that the surface is
mapped as NV12 plane textures.
The reason this wasn't done until now is because vdpau surfaces are
mapped in an "interlaced" way as separate fields, even for progressive
video. This requires messy reinterleraving. It turns out that even
though it's an extra processing step, the result can be faster than
going through the video mixer for RGB conversion.
Other than some potential speed-gain, doing this has multiple other
advantages. We can apply our own color conversion, which is important in
more complex cases. We can correctly apply debanding and potentially
other processing that requires chroma-specific or in-YUV handling.
If deinterlacing is enabled, this switches back to the old RGB
conversion method. Until we have at least a primitive deinterlacer in
vo_opengl, this will stay this way. The d3d11 and vaapi code paths are
similar. (Of course these don't require any crazy field reinterleaving.)
Otherwise stale references will survive forever. Could leak hardware
video surfaces.
In particular, the mpv vdpau code crashed with an assertion when exiting
after toggling deinterlacing, because not all references were released.
Instead of doing HDR tone mapping on an ad-hoc basis inside
pass_colormanage, the reference peak of an image is now part of the
image params (alongside colorspace, gamma, etc.) and tone mapping is
done whenever peak_src != peak_dst.
To get sensible behavior when mixing HDR and SDR content and displays,
target-brightness is a generic filler for "the assumed brightness of SDR
content".
This gets rid of the weird display_scaled hack, sets the framework
for multiple HDR functions with difference reference peaks, and allows
us to (in a future commit) autodetect the right source peak from
the HDR metadata.
(Apart from metadata, the source peak can also be controlled via
vf_format. For HDR content this adjusts the overall image brightness,
for SDR content it's like simulating a different exposure)
Commit 0348cd08 was too naive/simple, and always inserted the d3d11vpp
filter if any d3d11 output image formats were supported, even if it
makes no sense. For example --vf=format=rgb8 already breaks it.
It needs to take the set of supported input formats into account. the
weird format negotiation makes this hard. As a simple and cheap
solution, make some assumptions about the supported formats of a filter.
I hope to simplify this one day by using another format negotiation
algorithm, but this can probably wait.
We now have a video filter that uses the d3d11 video processor, so it
makes no sense to have one in the VO interop code. The VO uses it for
formats not directly supported by ANGLE (so the video data is converted
to a RGB texture, which ANGLE can take in).
Change this so that the video filter is automatically inserted if
needed. Move the code that maps RGB surfaces to its own inteorp backend.
Add a bunch of new image formats, which are used to enforce the new
constraints, and to automatically insert the filter only when needed.
The added vf mechanism to auto-insert the d3d11vpp filter is very dumb
and primitive, and will work only for this specific purpose. The format
negotiation mechanism in the filter chain is generally not very pretty,
and mostly broken as well. (libavfilter has a different mechanism, and
these mechanisms don't match well, so vf_lavfi uses some sort of hack.
It only works because hwaccel and non-hwaccel formats are strictly
separated.)
The RGB interop is now only used with older ANGLE versions. The only
reason I'm keeping it is because it's relatively isolated (uses only
existing mechanisms and adds no new concepts), and because I want to be
able to compare the behavior of the old code with the new one for
testing. It will be removed eventually.
If ANGLE has NV12 interop, P010 is now handled by converting to NV12
with the video processor, instead of converting it to RGB and using the
old mechanism to import that as a texture.
Main use: deinterlacing.
I'm not sure how to select the deinterlacing mode at all. You can
enumate the available video processors, but at least on Intel, all of
them either signal support for all deinterlacers, or none (the latter is
apparently used for IVTC). I haven't found anything that actually tells
the processor _which_ algorithm to use.
Another strange detail is how to select top/bottom fields and field
dominance. At least I'm getting quite similar results to vavpp on Linux,
so I'm content with it for now.
Future plans include removing the D3D11 video processor use from the
ANGLE interop code.
This makes vf_vdpaupp use the deinterlacer helper code already used by
vf_vavpp. I nice side-effect is that this also removes some traces of
code originating from vo_vdpau.c, so we can switch it to LGPL.
Extend the refqueue helper with a deint setting. If not set,
mp_refqueue_should_deint() always returns false, which slightly
simplifies vf_vdpaupp. It's of no consequence to vf_vavpp (other than it
has to set it to get expected behavior).
Abstracts the annoying framerate-doubling behavior.
Same deal as with refqueue introduction: the code size blows up, but at
least it can be reused for other filters.
Calling this right at start of filter_ext() also fixes a small
regression from previous commit. The change in reference surfaces due to
the first update_pipeline() with deinterlacing enabled changed behavior
of mp_refqueue_next() and mp_refqueue_has_output(). Since
update_pipeline() was called between those, the frame output logic got
inconsistent, and the first deinterlaced frame was duplicated from the
previous non-deinterlaced frame.
Also reset the number of ref-frames when switching back to non-deint
mode.
If the deinterlacer separates fields, the framerate must be doubled.
Since we have no stable and reliably framerate anywhere, we've been
calculating it by taking the time halfway to the next frame.
vf_vavpp actually used the past frame to calculate the frame duration,
which is sort of ok, but will skip the 2nd field in a stream (since the
first frame has no past PTS). This is annoying for testing, so use the
future frame PTS instead, which means the last field of the stream will
be dropped instead.
Move the handling of the future/past frames and the associated dataflow
rules to a separate source file.
While this on its own seems rather questionable and just inflates the
code, I intend to reuse it for other filters. The logic is annoying
enough that it shouldn't be duplicated a bunch of times.
(I considered other ways of sharing this logic, such as an uber-
deinterlace filter, which would access the hardware deinterlacer via a
different API. Although that sounds like kind of the right approach,
this would have other problems, so let's not, at least for now.)
Cropping usually happens by adjusting the plane start pointers and the
image size. The former is obviously not possible for opaque hwaccel
formats, but the latter must work.
Since the code already takes care of aligning the top/left crop origin
to chroma alignment, simply set the crop origin to 0/0 in the hwaccel
case. Also add a message if such an adjustment happens.
Supporting this isn't worth much; the main usefulness is with debugging.
The main change is with video/hwdec.h. mp_hwdec_info is made opaque (and
renamed to mp_hwdec_devices). Its accessors are mainly thread-safe (or
documented where not), which makes the whole thing saner and cleaner. In
particular, thread-safety rules become less subtle and more obvious.
The new internal API makes it easier to support multiple OpenGL interop
backends. (Although this is not done yet, and it's not clear whether it
ever will.)
This also removes all the API-specific fields from mp_hwdec_ctx and
replaces them with a "ctx" field. For d3d in particular, we drop the
mp_d3d_ctx struct completely, and pass the interfaces directly.
Remove the emulation checks from vaapi.c and vdpau.c; they are
pointless, and the checks that matter are done on the VO layer.
The d3d hardware decoders might slightly change behavior: dxva2-copy
will not use the VO device anymore if the VO supports proper interop.
This pretty much assumes that any in such cases the VO will not use any
form of exclusive mode, which makes using the VO device in copy mode
unnecessary.
This is a big refactor. Some things may be untested and could be broken.
Until now, we have made the assumption that a driver will use only 1
hardware surface format. the format is dictated by the driver (you
don't create surfaces with a specific format - you just pass a
rt_format and get a surface that will be in a specific driver-chosen
format).
In particular, the renderer created a dummy surface to probe the format,
and hoped the decoder would produce the same format. Due to a driver
bug this required a workaround to actually get the same format as the
driver did.
Change this so that the format is determined in the decoder. The format
is then passed down as hw_subfmt, which allows the renderer to configure
itself with the correct format. If the hardware surface changes its
format midstream, the renderer can be reconfigured using the normal
mechanisms.
This calls va_surface_init_subformat() each time after the decoder
returns a surface. Since libavcodec/AVFrame has no concept of sub-
formats, this is unavoidable. It creates and destroys a derived
VAImage, but this shouldn't have any bad performance effects (at
least I didn't notice any measurable effects).
Note that vaDeriveImage() failures are silently ignored as some
drivers (the vdpau wrapper) support neither vaDeriveImage, nor EGL
interop. In addition, we still probe whether we can map an image
in the EGL interop code. This is important as it's the only way
to determine whether EGL interop is supported at all. With respect
to the driver bug mentioned above, it doesn't matter which format
the test surface has.
In vf_vavpp, also remove the rt_format guessing business. I think the
existing logic was a bit meaningless anyway. It's not even a given
that vavpp produces the same rt_format for output.
The complex filter support that will be added makes much more complex
use of libavfilter, and I'm not going to bother with adding hacks to
keep libavfilter optional.
This covers source files which were added in mplayer2 and mpv times
only, and where all code is covered by LGPL relicensing agreements.
There are probably more files to which this applies, but I'm being
conservative here.
A file named ao_sdl.c exists in MPlayer too, but the mpv one is a
complete rewrite, and was added some time after the original ao_sdl.c
was removed. The same applies to vo_sdl.c, for which the SDL2 API is
radically different in addition (MPlayer supports SDL 1.2 only).
common.c contains only code written by me. But common.h is a strange
case: although it originally was named mp_common.h and exists in MPlayer
too, by now it contains only definitions written by uau and me. The
exceptions are the CONTROL_ defines - thus not changing the license of
common.h yet.
codec_tags.c contained once large tables generated from MPlayer's
codecs.conf, but all of these tables were removed.
From demux_playlist.c I'm removing a code fragment from someone who was
not asked; this probably could be done later (see commit 15dccc37).
misc.c is a bit complicated to reason about (it was split off mplayer.c
and thus contains random functions out of this file), but actually all
functions have been added post-MPlayer. Except get_relative_time(),
which was written by uau, but looks similar to 3 different versions of
something similar in each of the Unix/win32/OSX timer source files. I'm
not sure what that means in regards to copyright, so I've just moved it
into another still-GPL source file for now.
screenshot.c once had some minor parts of MPlayer's vf_screenshot.c, but
they're all gone.
Instead of handling this on filter chain reinit, do it directly after
the decoder. This makes the code less entangled. In particular, this
gets rid of the really weird "override params" concept in the video
filter code.
The last_format/fixed_formats have some redundance with decoder_output,
but unfortunately the latter has a slightly different use.
This is for the sake of command.c and the "deinterlace" option/property.
Instead of forcing certain "better" defaults when inserting yadif,
change the actual "yadif" defaults.
I pondered not changing vf_yadif, and instead adding a trivial "yadif-
auto" wrapper filter, which would merely have different defaults. But
thinking about it, it doesn't make any sense for "deinterlace" to have
different defaults from vf_yadif, with vf_yadif having the "worse"
defaults. If someone wants the old behavior, the old behavior can be
forced in a backward and forward compatible way by setting the
suboptions.
Fixes#2539 (kind of).
MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
They are evil and should be eradicated. Some of these were pretty dumb
anyway.
There are probably some more around in platform specific code or other
code not enabled by default on Linux.
The vf_format suboption is replaced with --video-output-levels (a global
option and property). In particular, the parameter is removed from
mp_image_params. The mechanism is moved to the "video equalizer", which
also handles common video output customization like brightness and
contrast controls.
The new code is slightly cleaner, and the top-level option is slightly
more user-friendly than as vf_format sub-option.
This can happen if the hw decoder allocates padded surfaces (e.g.
mod16), but the VPP output surface was allocated with the exact size.
Apparently VPP requires matching input and output sizes, or it will add
artifacts. In this case, it added mirrored pixels to the bottom few
pixels.
Note that the previous commit should have fixed this. But it didn't
work, while this commit does.
Fixes#2320.
If not set, VPP will use the whole surface. This is a problem if the
surfaces are padded, and especially if the surfaces are padded by
different amounts.
This is an attempt to fix#2320, but it appears to do nothing at all.
I took this out because I thought the filter chain would auto-negotiate
using nv12 without the explicit hint, and it does in the basic case
with no intermediate filter, but once you start adding filters, it
can end up negotiating a different format and then failing.
Today, vdpaurb will fail if it's used with non-hardware-decoded
content. This created work for the user as they have to explicitly
add or not add it, depending on the content.
As an improvement, we can make vdpaurb pass through any frames
that aren't hardware decoded, so that it can always be present in the
filter chain, if desired.
I see no point in keeping these around. Keeping wrappers for some select
libavfilter filters just because MPlayer had these filters is not a good
reason.
Ultimately, all real filtering work should go to libavfilter, and users
should get used to using vf_lavfi directly. We might even not require
the awful double-nested syntax for using libavfilter one day.
vf_rotate, vf_yadif, vf_stereo3d are kept because mpv uses them
internally. (They all extend the lavfi filters or change their
defaults.) vf_mirror is kept for symmetry with vf_flip. vf_gradfun and
vf_pullup are probably semi-popular, so I'll remove them not yet - only
after some more discussion.
This was requested by someone.
All code was written by myself; some minor changes by 2 contributors who
agreed to general LGPL relicensing. 1 line of code is by someone unknown
who possibly wasn't asked (setting the "display_fps" variable), and
which can be reasonably ignored as it makes up only 0.1% of the file.
The interlaced frame test needs to be aware that the input mpi might be
NULL - this happens at the end of a stream when the input frames have
all been submitted but frames still need to be drained from the
decoder.
Normally, vdpau decoded frames are passed directly to a suitable
vo (vo_vdpau or vo_opengl) without ever touching system memory. This
is efficient for output purposes, but prevents any of the regular
filters from being used with such frames.
This new filter implements a read-back step to pull the frames back
into system memory where they can be acted on by other filters.
Eventually the frames will be sent to the vo as if they were normal
software-decoded frames.
Note that a vdpau compatible vo must still be used to ensure that
the decoder is properly initialised.
Signed-off-by: wm4 <wm4@nowhere>
Drop libva versions below 0.34.0. These are ancient, so I don't care.
Drop the vo_vaapi deinterlacer as well. With 0.34.0, VPP is always
available, and deinterlacing is done with vf_vavpp.
The vaCreateSurfaces() function changes its signature - actually it did
in 0.34.0 or so, and the <va/va_compat.h> defined a macro to make it use
the old signature.
This was missing for extended deinterlacer.
Unfortunately, these deinterlacer still do not work. The provided future
frame (which is all the deinterlacers want) seems to be correct, though.
One minor behavioral change is that this always keeps the previous frame
for PTS computations. This could be avoided (in order to keep exactly
the same behavior as before), but it seems more elegant and should not
do any harm. (Also, if we really cared about reducing hw frame refs,
a more worthy goal is producing the field output incrementally.)
The mapped data (pointed to by the param variable) is not needed before,
so the call can be moved down. Also, this prevents that the buffer
remains mapped forever if the other vaMapBuffer() call above fails (the
cleanup code forgets to unmap the buffer - this commit makes it
unnecessary).
This used a do-while loop, which runs only once, as replacement for a
cleanup goto. While this is ok, doing a goto directly is easier to
follow and is closer to idiomatic C. But mainly remove it so that the
indentation can be reduced.
We do not fill them, so we would pass random IDs to the driver. The code
was originally written to handle bob deinterlacing only, so I guess it
originally passed always 0 anyway, despite having code for reference
surface list allocation.
Also, move down the vaUnmapBuffer() call. This call actually "unmaps"
the param pointer, so accessing it after the unmap call would be
undefined behavior. The "example" in <va/vavpp.h> does this too, but
it's most likely an error.
(Additionally, not even bob deinterlacing worked correctly in my test,
sigh.)
MP_IMGFIELD_TOP/MP_IMGFIELD_BOTTOM were completely unused, and
MP_IMGFIELD_ORDERED was always set (even though vf_vdpaupp.c strangely
checked for the latter).
Vapoursynth made an incompatible API change: clips with unknown length
are not supported anymore. In fact, Vapoursynth abort()s the program
(which by the way invalidate all of its claims of API/ABI stability).
So add some nonsense to make it work again.
It's entirely useless, especially now that vo.c handles screenshots in a
generic way, and requires no special VO support. There are some
potential weird use-cases, but actually I've never seen it being used.
Use OPT_CHOICE_C() instead of the custom parser. The functionality is
pretty much equivalent.
(On a side note, it seems --video-stereo-mode can't be removed, because
it controls whether to "reduce" stereo video to mono, which is also the
default. In fact I'm not sure how this should be handled at all.)
Remove the colorspace-related top-level options, add them to vf_format.
They are rather obscure and not needed often, so it's better to get them
out of the way. In particular, this gets rid of the semi-complicated
logic in command.c (most of which was needed for OSD display and the
direct feedback from the VO). It removes the duplicated color-related
name mappings.
This removes the ability to write the colormatrix and related
properties. Since filters can be changed at runtime, there's no loss of
functionality, except that you can't cycle automatically through the
color constants anymore (but who needs to do this).
This also changes the type of the mp_csp_names and related variables, so
they can directly be used with OPT_CHOICE. This probably ended up a bit
awkward, for the sake of not adding a new option type which would have
used the previous format.
It was "by design" possible to make mpv crash if the parameters didn't
make enough sense, like "format=rgb24:yuv420p". While forcing the format
has some minor (rather questionable) use for debugging, allowing it to
crash is just stupid.
Handling this perfectly with VapourSynth is probably not possible: you
either need to tell it the total number of input frames in advance, or
deliver an infinite stream. With playback, EOF can happen at an
unpredictable point, for which the VapourSynth API has no mechanism at
all. We handle EOF by returning an error to the filter, which will the
filter return all pending frame callbacks.
We still can try to handle it approximately: if the filter requests a
frame past EOF, then send it an error. This seems to work relatively
well with filters which don't request future frames.
There's literally no reason why these functions have to be inline (they
might be performance critical, but then the function call overhead isn't
going to matter at all).
Uninline them and move them to mp_image.c. Drop the header file and fix
all uses of it.
wm4
Niklas Haas
Dmitrij D. Czarkoff
Paul B Mahol
Philip Langdale
Filip Milivojevic
Michael Vetter
xylosper
Marcin Kurczewski