I guess gl_video->global was originally meant to be optional, but now it
crashes in some newer code with vo_opengl_cb, which tries to init it
with this field set to NULL (because normally it's not needed).
Probably fixes#2542.
This applies to unexpected freezes or deadlocks, not e.g. normal
framedrops. The verbose messages also might remind an API user if the
API usage is incorrect, such as not calling mpv_opengl_cb_draw() when a
redraw request was issued.
vo_opengl_cb is a special case, because we somehow have to render video
asynchronously, all while "trusting" the API user to do it correctly.
This didn't quite work, and a while ago a compromise using a timeout to
prevent theoretically possible deadlocks was added.
Make it even more synchronous. Basically, go all the way, and
synchronize rendering between VO and user renderer thread to the
full extent possible.
This means the silly frame queue is dropped, and we event attempt to
synchronize the GL SwapBuffer call (via mpv_opengl_cb_report_flip()).
The changes introduced with commit dc33eb56 are effectively dropped. I
don't even remember if they mattered.
In the future, we might make all VOs fetch asynchronously from a frame
queue, which would mostly remove the differences between vo_opengl and
vo_opengl_cb, but this will take a while (if it will even be done).
The noframe event is logged whenever there is no new frame. This can
happen due to normal redraws, but also due to video frame queue
underflow.
The mpv_opengl_cb_report_flip() API function is currently pretty
useless, because blocking on the video frame queue is more reliable and
simpler. But at least we can log the actual vsync.
next_vsync/prev_vsync was only used to retrieve the vsync duration. We
can get this in a simpler way.
This also removes the vsync duration estimation from vo_opengl_cb.c,
which is probably worthless anyway. (And once interpolation is made
display-sync only, this won't matter at all.)
The equalizer code as it exists in vo_opengl works perfectly fine. The
situation in vo_opengl_cb is pretty different. The playback thread can't
communicate with the renderer thread synchronously (essentially to give
the API user more flexibility). So the equalizer communication has to be
done in an asynchronous way too.
There were two problems. First, the eq capabilities can change with the
pixel format, and the renderer initializes them on config only. This
means equalizers were disabled on the first config run, and options like
--video-output-levels or --brightness would not work. So we just
initialize the caps with a known superset. The player will not correctly
indicate when setting an eq doesn't work, but we're fine with it, as it
is a relatively cosmetic issue.
Second, it copied back the eq settings in the "wrong" moment (what
for?), which overwrote the settings in some cases.
Third, the eq was not reset correctly on vo init. This is needed to make
it behave the same as vo_opengl.
When seeking, the current frame will have the wrong timestamp, until the
seek is done and a new frame from the seek target is shown. We still use
the "old" frame for redrawing during seeks. This frame has to be
explicitly marked with still=true in order not to confuse the
interpolation queue. If this is not done, it will ignore frames until a
frame with approximately the same timestamp as the "old" frame is
reached. (Does this mean interpolation handles timestamp resets
incorrectly? I have no idea.)
Of course we also have to clear possibly queued frames on seeks.
Also, in pausing, explicitly let the frame redraw.
This parameter has been unused for years (the last flag was removed in
commit d658b115). Get rid of it.
This affects the general VO API, as well as the vo_opengl backend API,
so it touches a lot of files.
The VOFLAGs are still used to control OpenGL context creation, so move
them to the OpenGL backend code.
Just avoid some code duplication. Also, gl_video_set_options() having a
queue size output parameter is weird at best. While I don't appreciate
that this commit suddenly requires gl_video.c to deal with vo.c directly
in a special case, it's simply the best place to put this function.
See manpage additions. This is mainly useful for vo_opengl_cb, but can
also be applied to vo_opengl.
On a side note, gl_hwdec_load_api() should stop using a name string, and
instead always use the IDs. This should be cleaned up another time.
draw_image_timed is renamed to draw_frame. struct frame_timing is
renamed to vo_frame. flip_page_timed is merged into draw_frame (the
additional parameters are part of struct vo_frame). draw_frame also
deprecates VOCTRL_REDRAW_FRAME, and replaces it with a method that
works for both VOs which can cache the current frame, and VOs which
need to redraw it anyway.
This is preparation to making the interpolation and (work in progress)
display sync code saner.
Lots of other refactoring, and also some simplifications.
This should make interpolation work much better in general, although
there still might be some side effects for unusual framerates (eg. 35 Hz
or 48 Hz). Most of the common framerates are tested and working fine.
(24 Hz, 30 Hz, 60 Hz)
The new code doesn't have support for oversample yet, so it's been
removed (and will most likely be reimplemented in a cleaner way if
there's enough demand). I would recommend using something like robidoux
or mitchell instead of oversample, though - they're much
smoother for the common cases.
Now the VO can request a number of future frames with the last parameter
of vo_set_queue_params(). This will be helpful to fix the interpolation
code.
Note that the first frame (after playback start or seeking) will usually
not have any future frames (to make seeking fast). Near the end of the
file, the number of future frames will become lower as well.
This uses the OpenGL frame interpolation code, which before could be
used by vo_opengl only.
Some effort was made to make it behave like vo_opengl, for the better or
the worse. As a consequence, there is a minor duplication of code and
mechanism. Hopefully this can all be wiped as soon as the VO frame
queue/timing mechanism is cleaned up.
This also attempts to use mpv_opengl_cb_report_flip() (as called by the
API user) to determine the vsync interval. This might need refinement as
well.
(In general, we simply expect the API user to work in vsync-blocking
manner.)
(I have no idea why there are different modes.)
Instead of risking to drop frames too early, give it some margin. Since
there are situations this could deadlock, wait with a timeout. This can
happen if e.g. the API user is refusing to render anything, or if
uninitialization is happening.
Currently, the wayland backend needs extra work to avoid drawing more
often than the wayland frame callback allows. (This is not ideal, but
will be fixed at a later time.)
Unify this with the start_frame callback added for cocoa. Some details
change for the better. For example, if a frame is dropped, and a redraw
is done afterwards, the actually correct frame is redrawn, instead
whatever was in the textures from before the dropped frame.
Increase the default queue size. This helps with "missed" frames due to
the asynchronous nature of the API. All the other VOs are synchronous,
so if rendering and displaying takes a while, the common code in vo.c
will be blocked until it can continue. But with opengl-cb, vo.c might
immediately push the next ready frame, which causes the current frame
to be dropped _if_ it wasn't rendered yet.
One could fix this by making vo.c wait a while (until the API user calls
the render function, which pulls the frame). But setting the default
queue size to 2 seems much simpler: instead of dropping the frame, it
will be pushed to the API user once the next renderer call finishes.
(This is still a bit strange, and will hopefully be cleaned up when
video scheduling is redone, but for now this appears to deliver
relatively good results.)
Unlike other VOs, this rendered OSD even while no VO was created
(because the renderer lives as long as the API user wants). Change this,
and refactor the code so that the OSD object is accessible only while
the VO is created.
(There is a short time where the OSD can still be accessed even after VO
destruction - this is not a race condition, though it's inelegant and
unfortunately unavoidable.)
This replaces the old smoothmotion code by a more flexible tscale
option, which essentially allows any scaler to be used for interpolating
frames. (The actual "smoothmotion" scaler which behaves identical to the
old code does not currently exist, but it will be re-added in a later commit)
The only odd thing is that larger filters require a larger queue size
offset, which is currently set dynamically as it introduces some issues
when pausing or framestepping. Filters with a lower radius are not
affected as much, so this is identical to the old smoothmotion if the
smoothmotion interpolator is used.
The basic idea is to use dynamically generated shaders instead of a
single monolithic file + a ton of ifdefs. Instead of having to setup
every aspect of it separately (like compiling shaders, setting uniforms,
perfoming the actual rendering steps, the GLSL parts), we generate the
GLSL on the fly, and perform the rendering at the same time. The GLSL
is regenerated every frame, but the actual compiled OpenGL-level shaders
are cached, which makes it fast again. Almost all logic can be in a
single place.
The new code is significantly more flexible, which allows us to improve
the code clarity, performance and add more features easily.
This commit is incomplete. It drops almost all previous code, and
readds only the most important things (some of them actually buggy).
The next commit will complete it - it's separate to preserve authorship
information.
mpv_opengl_cb_render() is supposed to clear the screen with the
background color if there's no video... I think.
It didn't do this, because although uninit() requested gl_video_config()
to be called, this didn't happen, because this function checks whether
the VO is set - and it's unsert after uninit() releases the lock.
Also call the user wakeup callback in this situation, so the user
actually redraws immediately.
This is somewhat imperfect, because detection of hw decoding APIs is
mostly done on demand, and often avoided if not necessary. (For example,
we know very well that there are no hw decoders for certain codecs.)
This also requires every hwdec backend to identify itself (see hwdec.h
changes).
At the time screenshot support was added, images weren't refcounted yet,
so screenshots required specialized implementations in the VOs. But now
we can handle these things much simpler. Also see commit 5bb24980.
If there are VOs in the future which can't do this (e.g. they need to
write to the image passed to vo_driver->draw_image), this still could be
disabled on a per-VO basis etc., so we lose no potential performance
advantages.
Use different VOCTRLs for "window" and normal screenshot modes. The
normal one will probably be removed, and replaced by generic code in
vo.c, and this commit is preparation for this. (Doing it the other way
around would be slightly simpler, but I haven't decided yet about the
second one, and touching every VO is needed anyway in order to remove
the unneeded crap. E.g. has_osd has been unused for a long time.)
SmoothMotion is a way to time and blend frames made popular by MadVR. It's
intended behaviour is to remove stuttering caused by mismatches between the
display refresh rate and the video fps, while preserving the video's original
artistic qualities (no soap opera effect). It's supposed to make 24fps video
playback on 60hz monitors as close as possible to a 24hz monitor.
Instead of drawing a frame once once it's pts has passed the vsync time, we
redraw at the display refresh rate, and if we detect the vsync is between two
frames we interpolated them (depending on their position relative to the vsync).
We actually interpolate as few frames as possible to avoid a blur effect as
much as possible. For example, if we were to play back a 1fps video on a 60hz
monitor, we would blend at most on 1 vsync for each frame (while the other 59
vsyncs would be rendered as is).
Frame interpolation is always done before scaling and in linear light when
possible (an ICC profile is used, or :srgb is used).
Before this commit, each hw backend had their own specific struct types
for context, and some, like VDA, had none at all. Add a context struct
(mp_hwdec_ctx) that provides a somewhat generic way to pass the hwdec
context around. Some things get slightly better, some slightly more
verbose.
mp_hwdec_info is still around; it's still needed, but is reduced to its
role of handling delayed loading of the hwdec backend.
And remove all uses of the VFCAP_CSP_SUPPORTED* constants. This is
supposed to reduce conversions if many filters are used (with many
incompatible pixel formats), and also for preferring the VO's natively
supported pixel formats (as opposed to conversion).
This is worthless by now. Not only do the main VOs not use software
conversion, but also the way vf_lavfi and libavfilter work mostly break
the way the old MPlayer mechanism worked. Other important filters like
vf_vapoursynth do not support "proper" format negotation either.
Part of this was already removed with the vf_scale cleanup from today.
While I'm touching every single VO, also fix the query_format argument
(it's not a FourCC anymore).
Support for taking screenshots when doing hardware decoding needs to be
added later.
This takes the last image queued to the VO, which is logically the image
the player thinks is on screen (so e.g. subtitles will match).
forget_frames() does not clear this, because seeking does not remove the
current image from the screen (until the next one is drawn).
The previous implementation of opengl-cb kept only latest flipped frame.
This can cause massive frame drops because rendering is done asynchronously
and only the latest frame can be rendered.
This commit introduces frame queue and releated options to opengl-cb.
frame-queue-size: the maximum size of frame queue (1-100, default: 1)
frame-drop-mode: behavior when frame queue is full (pop, clear, default: pop)
The frame queue holds delayed frames and drops frames if the frame queue is
overflowed with next method:
'pop' mode: drops all the oldest frames overflown.
'clear' mode: drops all frames in queue and clear it.
With default options(frame-queue-size=1:frame-drop-mode=pop),
opengl-cb behaves in the same way as previous implementation effectively.
For frame-queue-size > 1, opengl-cb tries to calls update() without waiting
next flip_page() in order to consume queued frames.
Signed-off-by: wm4 <wm4@nowhere>
This was always supposed to work.
Just add the option declaration. Normally I'm not a fan of duplicating
such things, but in this case it's (still) harmless.
This makes vo_opengl_cb respond to controls like "gamma" and
"brightness". The commit includes an awkward refactor for vo_opengl to
make it easier for vo_opengl_cb.
One problem is a logical race condition. The set of supported controls
depends on the pixelformat, which in turn is set by reconfig(). But the
actual reconfig() call (on the renderer) happens asynchronously on the
renderer thread. At the time it happens, the player most likely already
tried to set some controls for command line options (see init_vo() in
video.c). So setting this command line options will fail most of the
time, though it could randomly succeed. This can't be fixed directly,
because the player can't wait on the renderer thread, because the
renderer thread might already wait on the player.
Until now, calling mpv_opengl_cb_uninit_gl() at a "bad moment" could
make the whole thing to explode. The API user was asked to avoid such
situations by calling it only in "good moments". But this was probably a
bit too subtle and could easily be overlooked.
Integrate the approach the qml example uses directly into the
implementation. If the OpenGL context is to be unitialized, forcefully
disable video, and block until this is done.
wm4
Niklas Haas
Stefano Pigozzi
xylosper