MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
This was requested by someone.
All code was written by myself; some minor changes by 2 contributors who
agreed to general LGPL relicensing. 1 line of code is by someone unknown
who possibly wasn't asked (setting the "display_fps" variable), and
which can be reasonably ignored as it makes up only 0.1% of the file.
Vapoursynth made an incompatible API change: clips with unknown length
are not supported anymore. In fact, Vapoursynth abort()s the program
(which by the way invalidate all of its claims of API/ABI stability).
So add some nonsense to make it work again.
Handling this perfectly with VapourSynth is probably not possible: you
either need to tell it the total number of input frames in advance, or
deliver an infinite stream. With playback, EOF can happen at an
unpredictable point, for which the VapourSynth API has no mechanism at
all. We handle EOF by returning an error to the filter, which will the
filter return all pending frame callbacks.
We still can try to handle it approximately: if the filter requests a
frame past EOF, then send it an error. This seems to work relatively
well with filters which don't request future frames.
And remove all uses of the VFCAP_CSP_SUPPORTED* constants. This is
supposed to reduce conversions if many filters are used (with many
incompatible pixel formats), and also for preferring the VO's natively
supported pixel formats (as opposed to conversion).
This is worthless by now. Not only do the main VOs not use software
conversion, but also the way vf_lavfi and libavfilter work mostly break
the way the old MPlayer mechanism worked. Other important filters like
vf_vapoursynth do not support "proper" format negotation either.
Part of this was already removed with the vf_scale cleanup from today.
While I'm touching every single VO, also fix the query_format argument
(it's not a FourCC anymore).
This adds an "auto" choice to the concurrent-frames suboption, and makes
it the default.
I'm not so sure about making this the default, though. It could lead to
excessive buffering with large CPU counts. But we'll see.
This is basically a hack; but apparently a needed one, since many
vapoursynth filters insist on having a FPS set.
We need to apply the FPS override before creating the filters. Also
change some terminal output related to the FPS value.
Most of this is explained in the code comments. This change should
improve performance with vapoursynth, especially if concurrent requests
are used.
This should change nothing if vf_vapoursynth is not in the filter chain,
since non-threaded filters obviously can not asynchronously finish
filtering of frames.
In theory, vsscript should be doing it, but it's not there yet, neither
did there seem to be any interest in making it flexible enough to handle
more than 1 scripting language.
Some VS filters will requests frames from their parent filters while
they're initialized. Thy do this in a blocking manner, and
initialization will not succeed until the frame request is satisfied.
This deadlocked mpv, because we can feed frames to the filter only after
initialization is finished.
Return an error instead of deadlocking.
Note that we (probably) can handle frames being requested during init
fine, as long as the requests don't block initialization. But we can
distinguish this situation, and a simple test seems to indicate VS
usually doesn't do this.
See #1168.
Until now, we always required the playback core to decode a new frame to
get more output from the filter. That seems to be completely
unnecessary, because filtered results may arrive before that.
Add a filter_out callback, and restructure the code such that it can
return any filtered frames, or block if it hasn't read at least one
frame.
In the worst case, it still can happen that bursts of input requests and
output requests happen. (This commit tries to reduce burst-like
behavior, but it's not entirely possible due to the indeterministic
nature of VS threading.)
This is a similar change as with 95bb0bb6.
Add two new script environment variables 'video_in_dw' and
'video_in_dh', representing the display resolution of video. Along
with video resolution, sample ratio aspect can be calculated in
scripts.
Currently it's impossible to change sample ratio aspect with single
vapoursynth filter since '_SARNum' and '_SARDen' frame properties
from output clip will be ignored. A following 'dsize' filter is
necessary for this purpose.
When seeking, we violently destroy the filter, because vapoursynth has
no proper API for terminating a video with unknown frame count. This
looks like an error to vapoursynth, and the error is returned via the
frame callbacks. The bug is that we remember this error state across
reinitialization, so on the first filter call after reinitialization, we
thought filtering the current frame failed. This caused a shift by 1
frame on each seek.
CC: @mpv-player/stable
Until now, failure to allocate image data resulted in a crash (i.e.
abort() was called). This was intentional, because it's pretty silly to
degrade playback, and in almost all situations, the OOM will probably
kill you anyway. (And then there's the standard Linux overcommit
behavior, which also will kill you at some point.)
But I changed my opinion, so here we go. This change does not affect
_all_ memory allocations, just image data. Now in most failure cases,
the output will just be skipped. For video filters, this coincidentally
means that failure is treated as EOF (because the playback core assumes
EOF if nothing comes out of the video filter chain). In other
situations, output might be in some way degraded, like skipping frames,
not scaling OSD, and such.
Functions whose return values changed semantics:
mp_image_alloc
mp_image_new_copy
mp_image_new_ref
mp_image_make_writeable
mp_image_setrefp
mp_image_to_av_frame_and_unref
mp_image_from_av_frame
mp_image_new_external_ref
mp_image_new_custom_ref
mp_image_pool_make_writeable
mp_image_pool_get
mp_image_pool_new_copy
mp_vdpau_mixed_frame_create
vf_alloc_out_image
vf_make_out_image_writeable
glGetWindowScreenshot
It could in theory happen that the filter loop will enter a blocking
wait, even though it could make progress by emptying the list of
already-filtered images. I'm not quite sure if this could actually cause
a real issue - probably not.
VapourSynth won't just filter multiple frames at once on its own. You
have to request multiple frames at once manually. This is what this
commit introduces: a sub-option controls how many frames will be
requested at once. This also changes the semantics of the maxbuffer sub-
option, now renamed to buffered-frames.
We only support them for input. The frame properties of output frames
are ignored (except frame durations).
Properties not set for now: _ChromaLocation, _Field, _FieldBased
Set _DurationNum/_DurationDen on each VS frame, instead of
_AbsoluteTime. The duration is the difference between the timestamp of
the frame and the next frame, and when receiving filtered VS frames, we
convert them back to an absolute PTS by summing them.
We pass the timestamps with microsecond resolution. mpv uses double for
timestamps internally, so we don't know the "real" timebase or FPS. VS
on the other hand uses fractions for frame durations. We can't pass
through the numbers exactly, but microseconds ought to be enough to be
even safe from accumulating rounding errors.
Since this leaks video images, and the player keeps feeding new images
to the fitler even if it fails, this would probably have disastrous
consequences.
Or in other words, add support for properly draining remaining frames
from video filters. vf_yadif is buffering at least one frame, and the
buffered frame was not retrieved on EOF.
For most filters, ignore this for now, and just adjust them to the
changed semantics of filter_ext. But for vf_lavfi (used by vf_yadif),
real support is implemented. libavfilter handles this simply by passing
a NULL frame to av_buffersrc_add_frame(), so we just have to make
mp_to_av() handle NULL arguments.
In load_next_vo_frame(), we first try to output a frame buffered in the
VO, then the filter, and then (if EOF is reached and there's still no
new frame) the VO again, with draining enabled. I guess this was
implemented slightly incorrectly before, because the filter chain still
could have had remaining output frames.
I thought the "_" in "_AbsoluteTime" was part of the documentation
markup.
This still doesn't help us with VS filters that change timing;
apparently you must use frame durations instead.
Make the filter apply the pixel aspect ratio of the input to the output.
This is more useful than forcing 1:1 PAR when playing anamorphic video
such as DVDs.
VapourSynth itself actually allows passing through the aspect ratio, but
it's in a not very useful form for us: it's per video-frame instead of
constant (i.e in VSVideoInfo). As long as we don't have a way to allow a
filter to spontaneously change output parameters, we can't use this.
(And I don't really feel like making this possible.)
We were relying on vsscript_freeScript() to take care of proper
termination. But it doesn't do that: it doesn't wait for the filters to
finish and exit at all. Instead, it just destroys all objects, which
causes the worker threads to crash sometimes.
Also, we're supposed to wait for the frame callback to finish before
freeing the associated node.
Handle this by explicitly waiting as far as we can. Probably fixes
crashes on seeking, although VapourSynth itself might also need some
work to make this case completely stable.
Before this commit, the filter attempted to keep the vsscript state
(p->se) even when the script was reloaded. Change it to destroy the
script state too on reloading. Now no workaround for LoadPlugin is
necessary, and this also fixes a weird theoretical race condition when
destroying and recreating the mpv source filter.
Mainly meant to apply simple VapourSynth filters to video at runtime.
This has various restrictions, which are listed in the manpage.
Additionally, this actually copies video frames when converting frame
references from mpv to VapourSynth, and a second time when going from
VapourSynth to mpv. This is inefficient and could probably be easily
improved. But for now, this is simpler, and in fact I'm not sure if
we even can references VapourSynth frames after the core has been
destroyed.
wm4
Julian
Ben Boeckel
Bin Jin