This means vf_vapoursynth doesn't need a hack to work around the filter
code, and libavfilter filters now actually get the frame_rate field on
input pads set.
The libavfilter doxygen says the frame_rate field is only to be set if
the frame rate is known to be constant, and uses the word "must" (which
probably means they really mean it?) - but ffmpeg.c sets the field to
mere guesses anyway, and it looks like this normally won't lead to
problems.
In theory (and practice), this is not needed, because the VS filter get
frame callback will cause the process function to be called again if
there's not enough data. But it's still a bit weird to just add one more
frame on each iteration, so make it cleaner and make it request frames
until the input array is full.
This was obviously nonsense, and a previous "fix" to this code was
nonsense too. What is really needed here is temporarily dropping the
lock while calling destroy_vs()/reinit_vs().
Fixes#5470.
Unknown frames were not freed properly. Although this doesn't really
happen anyway, because we're never going to feed audio frames to a video
filter chain. Since it's theoretically possible, and all other filters
handle this consistently, fix it anyway.
Properly initialize the output frame parameters other than image format
and size. This includes colorspace hints. (We're still not reading them
back from VapourSynth if it sets them, though. Usually it doesn't
anyway.)
VapourSynth can't pass through timestamps, only frame durations. So we
need to remember the timestamp of the very first frame passed to it.
This was accidentally set to 0 instead of NOPTS on init, so inserting
the filter during playback could show strange behavior.
Might be part of #5470.
Get rid of the old vf.c code. Replace it with a generic filtering
framework, which can potentially handle more than just --vf. At least
reimplementing --af with this code is planned.
This changes some --vf semantics (including runtime behavior and the
"vf" command). The most important ones are listed in interface-changes.
vf_convert.c is renamed to f_swscale.c. It is now an internal filter
that can not be inserted by the user manually.
f_lavfi.c is a refactor of player/lavfi.c. The latter will be removed
once --lavfi-complex is reimplemented on top of f_lavfi.c. (which is
conceptually easy, but a big mess due to the data flow changes).
The existing filters are all changed heavily. The data flow of the new
filter framework is different. Especially EOF handling changes - EOF is
now a "frame" rather than a state, and must be passed through exactly
once.
Another major thing is that all filters must support dynamic format
changes. The filter reconfig() function goes away. (This sounds complex,
but since all filters need to handle EOF draining anyway, they can use
the same code, and it removes the mess with reconfig() having to predict
the output format, which completely breaks with libavfilter anyway.)
In addition, there is no automatic format negotiation or conversion.
libavfilter's primitive and insufficient API simply doesn't allow us to
do this in a reasonable way. Instead, filters can use f_autoconvert as
sub-filter, and tell it which formats they support. This filter will in
turn add actual conversion filters, such as f_swscale, to perform
necessary format changes.
vf_vapoursynth.c uses the same basic principle of operation as before,
but with worryingly different details in data flow. Still appears to
work.
The hardware deint filters (vf_vavpp.c, vf_d3d11vpp.c, vf_vdpaupp.c) are
heavily changed. Fortunately, they all used refqueue.c, which is for
sharing the data flow logic (especially for managing future/past
surfaces and such). It turns out it can be used to factor out most of
the data flow. Some of these filters accepted software input. Instead of
having ad-hoc upload code in each filter, surface upload is now
delegated to f_autoconvert, which can use f_hwupload to perform this.
Exporting VO capabilities is still a big mess (mp_stream_info stuff).
The D3D11 code drops the redundant image formats, and all code uses the
hw_subfmt (sw_format in FFmpeg) instead. Although that too seems to be a
big mess for now.
f_async_queue is unused.
I've decided that MP_TRACE means “noisy spam per frame”, whereas
MP_DBG just means “more verbose debugging messages than MSGL_V”.
Basically, MSGL_DBG shouldn't create spam per frame like it currently
does, and MSGL_V should make sense to the end-user and provide mostly
additional informational output.
MP_DBG is basically what I want to make the new default for --log-file,
so the cut-off point for MP_DBG is if we probably want to know if for
debugging purposes but the user most likely doesn't care about on the
terminal.
Also, the debug callbacks for libass and ffmpeg got bumped in their
verbosity levels slightly, because being external components they're a
bit less relevant to mpv debugging, and a bit too over-eager in what
they consider to be relevant information.
I exclusively used the "try it on my machine and remove messages from
MSGL_* until it does what I want it to" approach of refactoring, so
YMMV.
For vo_opengl and vo_direct3d, these are supported in a generic way.
For vf_vapoursynth, we could probably map its VSFormat struct in a
generic way, but for now do some bullshit.
vf_eq.c actually loses support for these formats. We could add generic
support too (anything that has 8 bit planes will work), but why bother.
The filter is deprecated anyway.
This has two reasons:
1. I tend to add new fields to this metadata, and every time I've done
so I've consistently forgotten to update all of the dozens of places in
which this colorimetry metadata might end up getting used. While most
usages don't really care about most of the metadata, sometimes the
intend was simply to “copy” the colorimetry metadata from one struct to
another. With this being inside a substruct, those lines of code can now
simply read a.color = b.color without having to care about added or
removed fields.
2. It makes the type definitions nicer for upcoming refactors.
In going through all of the usages, I also expanded a few where I felt
that omitting the “young” fields was a bug.
MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
This was requested by someone.
All code was written by myself; some minor changes by 2 contributors who
agreed to general LGPL relicensing. 1 line of code is by someone unknown
who possibly wasn't asked (setting the "display_fps" variable), and
which can be reasonably ignored as it makes up only 0.1% of the file.
Vapoursynth made an incompatible API change: clips with unknown length
are not supported anymore. In fact, Vapoursynth abort()s the program
(which by the way invalidate all of its claims of API/ABI stability).
So add some nonsense to make it work again.
Handling this perfectly with VapourSynth is probably not possible: you
either need to tell it the total number of input frames in advance, or
deliver an infinite stream. With playback, EOF can happen at an
unpredictable point, for which the VapourSynth API has no mechanism at
all. We handle EOF by returning an error to the filter, which will the
filter return all pending frame callbacks.
We still can try to handle it approximately: if the filter requests a
frame past EOF, then send it an error. This seems to work relatively
well with filters which don't request future frames.
And remove all uses of the VFCAP_CSP_SUPPORTED* constants. This is
supposed to reduce conversions if many filters are used (with many
incompatible pixel formats), and also for preferring the VO's natively
supported pixel formats (as opposed to conversion).
This is worthless by now. Not only do the main VOs not use software
conversion, but also the way vf_lavfi and libavfilter work mostly break
the way the old MPlayer mechanism worked. Other important filters like
vf_vapoursynth do not support "proper" format negotation either.
Part of this was already removed with the vf_scale cleanup from today.
While I'm touching every single VO, also fix the query_format argument
(it's not a FourCC anymore).
This adds an "auto" choice to the concurrent-frames suboption, and makes
it the default.
I'm not so sure about making this the default, though. It could lead to
excessive buffering with large CPU counts. But we'll see.
This is basically a hack; but apparently a needed one, since many
vapoursynth filters insist on having a FPS set.
We need to apply the FPS override before creating the filters. Also
change some terminal output related to the FPS value.
Most of this is explained in the code comments. This change should
improve performance with vapoursynth, especially if concurrent requests
are used.
This should change nothing if vf_vapoursynth is not in the filter chain,
since non-threaded filters obviously can not asynchronously finish
filtering of frames.
In theory, vsscript should be doing it, but it's not there yet, neither
did there seem to be any interest in making it flexible enough to handle
more than 1 scripting language.
Some VS filters will requests frames from their parent filters while
they're initialized. Thy do this in a blocking manner, and
initialization will not succeed until the frame request is satisfied.
This deadlocked mpv, because we can feed frames to the filter only after
initialization is finished.
Return an error instead of deadlocking.
Note that we (probably) can handle frames being requested during init
fine, as long as the requests don't block initialization. But we can
distinguish this situation, and a simple test seems to indicate VS
usually doesn't do this.
See #1168.
Until now, we always required the playback core to decode a new frame to
get more output from the filter. That seems to be completely
unnecessary, because filtered results may arrive before that.
Add a filter_out callback, and restructure the code such that it can
return any filtered frames, or block if it hasn't read at least one
frame.
In the worst case, it still can happen that bursts of input requests and
output requests happen. (This commit tries to reduce burst-like
behavior, but it's not entirely possible due to the indeterministic
nature of VS threading.)
This is a similar change as with 95bb0bb6.
Add two new script environment variables 'video_in_dw' and
'video_in_dh', representing the display resolution of video. Along
with video resolution, sample ratio aspect can be calculated in
scripts.
Currently it's impossible to change sample ratio aspect with single
vapoursynth filter since '_SARNum' and '_SARDen' frame properties
from output clip will be ignored. A following 'dsize' filter is
necessary for this purpose.
When seeking, we violently destroy the filter, because vapoursynth has
no proper API for terminating a video with unknown frame count. This
looks like an error to vapoursynth, and the error is returned via the
frame callbacks. The bug is that we remember this error state across
reinitialization, so on the first filter call after reinitialization, we
thought filtering the current frame failed. This caused a shift by 1
frame on each seek.
CC: @mpv-player/stable
Until now, failure to allocate image data resulted in a crash (i.e.
abort() was called). This was intentional, because it's pretty silly to
degrade playback, and in almost all situations, the OOM will probably
kill you anyway. (And then there's the standard Linux overcommit
behavior, which also will kill you at some point.)
But I changed my opinion, so here we go. This change does not affect
_all_ memory allocations, just image data. Now in most failure cases,
the output will just be skipped. For video filters, this coincidentally
means that failure is treated as EOF (because the playback core assumes
EOF if nothing comes out of the video filter chain). In other
situations, output might be in some way degraded, like skipping frames,
not scaling OSD, and such.
Functions whose return values changed semantics:
mp_image_alloc
mp_image_new_copy
mp_image_new_ref
mp_image_make_writeable
mp_image_setrefp
mp_image_to_av_frame_and_unref
mp_image_from_av_frame
mp_image_new_external_ref
mp_image_new_custom_ref
mp_image_pool_make_writeable
mp_image_pool_get
mp_image_pool_new_copy
mp_vdpau_mixed_frame_create
vf_alloc_out_image
vf_make_out_image_writeable
glGetWindowScreenshot
Chainik
wm4
Niklas Haas
Philip Sequeira
Niklas Haas
Julian
Ben Boeckel
Bin Jin