Originally when presentation time was implemented, the frame callback
and presentation feedback functions were called in each rendering api's
separate backend (egl and vulkan). This meant that their respective
structs were basically copy and pasted across both files. Plus later
vo_wlshm started using frame callbacks too. Things got refactored a few
times and it turns out there's actually no need to have these things
separate anymore. The frame callback can just be initialized in
vo_wayland_init and then everything else will follow from there. Just
move all of this code to wayland_common and get rid of the duplication.
Sidenote: This means that vo_wlshm can actually receive presentation
feedback now. It's really simple to do so might as well. See the next
commit.
30dcfbc is a workaround for incorrect border sizes that could occur on
sway/wlroots in certain edge cases. This seemed harmless enough, but it
turns out that on mutter the extra wl_surface_commit somehow causes the
window always go to the top left of the screen after you leave
fullscreen. No idea why this occurs, but the original commit is a
workaround a sway bug and causing regressions for other users isn't
right despite the author being biased towards sway/wlroots.
This reverts commit 30dcfbc9cb.
Made possible with 00b9c81. 34b8adc let the wayland surface set an
opaque region depending on if alpha was set by the user or not. However,
there was no attempted detection for runtime changes and it is possible
(at least in wayland vulkan) to toggle the alpha on and off. So this
meant, we could be incorrectly signalling an opaque region if the user
happened to change the alpha. Additionally, add a helper function for
this and use it everywhere we want to set the opaque region.
efb0c5c changed the rendering logic of mpv on wayland and made it skip
rendering when it did not receive frame callback in time. The idea was
to skip rendering when the surface was hidden and be less wasteful. This
unfortunately had issues in certain instances where a frame callback
could be missed (but the window was still in view) due to imprecise
rendering (like the default audio video-sync mode). This would lead to
the video appearing to stutter since mpv would skip rendering in those
cases.
To account for this case, simply re-add an old heuristic for detecting
if a window is hidden or not since the goal is to simply not render when
a window is hidden. If the wait on the frame callback times out enough
times in a row, then we consider the window hidden and thus begin to
skip rendering then. The actual threshold to consider a surface as
hidden is completely arbitrary (greater than your monitor's refresh
rate), but it's safe enough since realistically you're not going to miss
60+ frame callbacks in a row unless the surface actually is hidden.
Fixes#8169.
Apparently a part of the wayland spec. A compositor may use a surface
that has set part of itself as opaque for various optimizations. For
mpv, we simply set the entire surface as opaque as long as the user has
not set alpha=yes (note: alpha is technically broken in the wayland EGL
backend at the time of this commit but oh well). wlshm is always opaque.
Fixes#8125.
Back in the olden days, mpv's wayland backend was driven by the frame
callback. This had several issues and was removed in favor of the
current approach which allowed some advanced features (like
display-resample and presentation time) to actually work properly.
However as a consequence, it meant that mpv always rendered, even if the
surface was hidden. Wayland people consider this "wasteful" (and well
they aren't wrong). This commit aims to avoid wasteful rendering by
doing some additional checks in the swapchain. There's three main parts
to this.
1. Wayland EGL now uses an external swapchain (like the drm context).
Before we start a new frame, we check to see if we are waiting on a
callback from the compositor. If there is no wait, then go ahead and
proceed to render the frame, swap buffers, and then initiate
vo_wayland_wait_frame to poll (with a timeout) for the next potential
callback. If we are still waiting on callback from the compositor when
starting a new frame, then we simple skip rendering it entirely until
the surface comes back into view.
2. Wayland on vulkan has essentially the same approach although the
details are a little different. The ra_vk_ctx does not have support for
an external swapchain and although such a mechanism could theoretically
be added, it doesn't make much sense with libplacebo. Instead,
start_frame was added as a param and used to check for callback.
3. For wlshm, it's simply a matter of adding frame callback to it,
leveraging vo_wayland_wait_frame, and using the frame callback value to
whether or not to draw the image.
It was possible for sway to get incorrectly sized borders if you resized
the mpv window in a creative manner (e.g. open a video in a non-floating
mode, set window scale to 2, then float it and witness wrong border
sizes). This is possibly a sway bug (Plasma doesn't have these border
issues at least), but there's a reasonable workaround for this.
The reason for the incorrect border size is because it is possible for
mpv to ignore the width/height from the toplevel listener and set its
own size. This new size can differ from what sway/wlroots believes the
size is which is what causes the sever side decorations to be drawn on
incorrect dimensions.
A simple trick is to just explicitly commit the surface after a resize
is performed. This is only done if mpv is not fullscreened or maximized
since we always obey the compositor widths/heights in those cases.
Sending the commit signals the compositor of the new change in the
surface and thus sway/wlroots updates its internal coordinates
appropriately and borders are no longer broken.
When using presentation time, we have to be sure to update the ust when
no presentation events are received to make sure playback is still
smooth and in sync. Part of the recent presentation time refactor was to
use the presentation discarded event to signal that the window is
hidden. Evidently, this doesn't work the same everywhere for whatever
reason (drivers?? hardware??) and at least one user experienced issues
with playback getting out of sync since (presumably) the discarded event
didn't occur when hiding the window. Instead, let's just go back to the
old way of checking if the last_ust is equal to the ust value of the
last member in the wayland sync queue. Fixes#8010.
The motivation for this change was a segfault caused by e107342 which
has complicated reasons for occuring (i.e. I'm not 100% sure but I think
it is a really weird race). The major part of this commit is moving the
initialization of presentation listener to the frame_callback function.
Calling it in swap_buffers worked fine but in practice it meant a lot of
meaningless function calls if a window was hidden (the presentation
would just be immediately discarded). By calling it in frame_callback,
we ensure the listener is only created when it is possible to receive a
presentation event.
Of course calling the presentation listener in feedback_presented or
feedback_discarded was considered, but ultimately these events are too
slow. Receiving the ust/msc/sbc triplet here and then passing it to mpv
results in higher vsync judder since there is (likely) not enough time
before the next pageflip. By design, the frame callback is meant to give
us as much time as possible before the next repaint so calling it here
is probably optimal.
Additionally, we can make better use of the feedback_discarded event.
The wp_presentation_feedback should not be destroyed here. It will be
taken care of either when we get feedback again or when the player
quits. Instead what we can do is set a bool that tells wayland_sync_swap
to update itself based on mp_time delta. In practice, the result is not
any different than before, but it should be more understandable what is
going on now.
Of course, the segfault mentioned at the beginning is fixed with this as
well.
Commit 07b0c18 introduced some build breakages. Some breakages
were fixed on c1fc535 and a1adafe. This one is still remaining.
This commit fixes the following build error:
[153/521] Compiling video/out/vulkan/context_wayland.c
../video/out/vulkan/context_wayland.c:26:10: fatal error: video/out/wayland/presentation-time.h: No such file or directory
26 | #include "video/out/wayland/presentation-time.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Relevant to: #7802
One not-so-nice hack in the wayland code is the assumption of when a
window is hidden (out of view from the compositor) and an arbitrary
delay for enabling/disabling the usage of presentation time. Since you
do not receive any presentation feedback when a window is hidden on
wayland (a feature or misfeature depending on who you ask), the ust is
updated based on the refresh_nsec statistic gathered from the previous
feedback event.
The flaw with this is that refresh_nsec basically just reports back the
display's refresh rate (1 / refresh_rate * 10^9). It doesn't tell you
how long the vsync interval really was. So as a video is left playing
out of view, the wl->last_queue_display_time becomes increasingly
inaccurate. This led to a vsync spike when bringing the mpv window back
into sight after it was hidden for a period of time. The hack for
working around this is to just wait a while before enabling presentation
time again. The discrepancy between the "bogus"
wl->last_queue_display_time and the actual value you get from the
feedback only happens initially after a switch. If you just discard
those values, you avoid the dramatic vsync spike.
It turns out that there's a smarter way to do this. Just use mp_time_us
deltas. The whole reason for these hacks is because
wl->last_queue_display_time wasn't close enough to how long it would
take for a frame to actually display if it wasn't hidden. Instead, mpv's
internal timer can be used, and the difference between wayland_sync_swap
calls is a close enough proxy for the vsync interval (certainly better
than using the monitor's refresh rate). This avoids the entire conundrum
of massive vsync spikes when bringing the player back into view, and it
means we can get rid of extra crap like wl->hidden.
This originally existed as a hack for weston. In certain scenarios, a
frame taking too long to render would cause vo_wayland_wait_frame to
timeout which would result in a ton of dropped frames. The naive
solution was to just to add a slight delay to the time value. If a
frame took too long, it would likely to fall under the timeout value and
all was well. This was exposed to the user since the default delay
(1000) was completely arbitrary.
However with presentation time, this doesn't appear to be neccesary.
Fresh frames that take longer than the display's refresh rate (16.666 ms
in most cases) behave well in Weston. In the other two main compositors
without presentation time (GNOME and Plasma), they also do not
experience any ill effects. It's better not to overcomplicate things, so
this "feature" can be removed now.
There's 2 stupid things here that need to be fixed. First of all,
vulkan wasn't actually using presentation time because somehow the
get_vsync function in context.c disappeared. Secondly, if the mpv window
was hidden it was updating the ust time based on the refresh_usec but
really it should simply just not feed any information to the vsync info
structure. So this adds some logic to assume whether or not a window is
hidden.
The old way of using wayland in mpv relied on an external renderloop for
semi-accurate timings. This had multiple issues though. Display sync
would break whenever the window was hidden (since the frame callback
stopped being executed) which was really annoying. Also the entire
external renderloop logic was kind of fragile and didn't play well with
mpv's internal structure (i.e. using presentation time in that old
paradigm breaks stats.lua).
Basically the problem is that swap buffers blocks on wayland which is
crap whenever you hide the mpv window since it looks up the entire
player. So you have to make swap buffers not block, but this has a
different problem. Timings will be terrible if you use the unblocked
swap buffers call.
Based on some discussion in #wayland, the trick here is relatively
simple and works well enough for our purposes. Instead we basically
build a way to block with a timeout in the wayland buffer swap
functions.
A bool is set in the frame callback function that indicates whether or
not mpv is waiting for a frame to be displayed. In the actual buffer
swap function, we enter into a while loop waiting for this flag to be
set. At the same time, the wl_display is polled to block the thread and
wakeup if it receives any events from the compositor. This loop only
breaks if enough time has passed or if the frame callback bool is
received.
In the near future, it is better to set whether or not frame a frame has
been displayed in the presentation feedback. However as a first pass,
doing it in the frame callback is more than good enough.
The "downside" is that we render frames that aren't actually shown on
screen when the player is hidden (it seems like wayland people don't
like that). But who cares. Accurate timings are way more important. It's
probably not too hard to add that behavior back in the player though.
This change introduces a vulkan interop path for the vaapi hwdec.
The basic principles are mostly the same as for EGL, with the
exported dma_buf being imported by Vukan. The biggest difference
is that we cannot reuse the texture as we do with OpenGL - there's
no way to rebind a VkImage to a different piece of memory, as far
as I can see. So, a new texture is created on each map call.
I did not bother implementing a code path for the old libva API as
I think it's safe to assume any system with a working vulkan driver
will have access to a newer libva.
Note that we are using separate layers for the vaapi surface, just
as is done for EGL. This is because libplacebo doesn't support
multiplane images.
This change does not include format negotiation because no driver
implements the vk_ext_image_drm_format_modifier extension that
would be required to do that. In practice, the two formats we care
about (nv12, p010) work correctly, so we are not blocked. A separate
change had to be made in libplacebo to filter out non-fatal validation
errors related to surface sizes due to the lack of format negotiation.
This commit rips out the entire mpv vulkan implementation in favor of
exposing lightweight wrappers on top of libplacebo instead, which
provides much of the same except in a more up-to-date and polished form.
This (finally) unifies the code base between mpv and libplacebo, which
is something I've been hoping to do for a long time.
Note: The ra_pl wrappers are abstract enough from the actual libplacebo
device type that we can in theory re-use them for other devices like
d3d11 or even opengl in the future, so I moved them to a separate
directory for the time being. However, the rest of the code is still
vulkan-specific, so I've kept the "vulkan" naming and file paths, rather
than introducing a new `--gpu-api` type. (Which would have been ended up
with significantly more code duplicaiton)
Plus, the code and functionality is similar enough that for most users
this should just be a straight-up drop-in replacement.
Note: This commit excludes some changes; specifically, the updates to
context_win and hwdec_cuda are deferred to separate commits for
authorship reasons.
This was confusing at best. Change it to output the actual choices.
(Seems like in the end it's always me who has to clean up other people's
bullshit.)
Context names were not unique - but they should be, so fix it. The whole
point of the original --opengl-backend option was to side-step the
tricky auto-detection, so you know exactly what you get. The goal of
this commit is to make --gpu-context work the same way. Fix the
non-unique names by appending "vk" to the names.
Keep in mind that this was not suitable for slecting the "UI" backend
anyway, since "x11" would force GLX, whereas people on not-NVIDIA
actually want "x11egl". Users trying to use --gpu-context=x11 to force
the X11 backend would always end up with GLX, which would at least break
VAAPI hardware decoding for them. Basically the idea that this option
could select the "UI" type is completely broken - it selects an
implementation, which implies a UI. Selecting the UI type This would
require a separate mechanism. (Although in theory this separate
mechanism could be part of the --gpu-context option - in any case,
someone would have to implement it.)
To achieve help output that can actually be understood, just duplicate
the code. Most of that code is duplicated anyway, and trying to share
just the list code with the result of making the output unreadable
doesn't make too much sense. If we wanted to save code/effort, we could
just remove the help output altogether.
--gpu-api has non-unique entries, and it would be nice to group them
(e.g. list all OpenGL capable contexts with "opengl"), but C makes this
simple idea too much of a pain, so don't do it.
Also remove a stray tab from the android entry on the manpage.
This commit:
- Implements output tracking (e.g. monitor plug/unplug)
- Creates the surface during registry (no other dependencies)
- Queues the callback immediately after surface creation
- Cleaner and better event handling (functions return directly)
- Better reconfigure handling (resizes reduced to 1 during init)
- Don't unnecessarily resize (if dimensions match)
Apart from that fixes 2 potential memory leaks (mime type and window
title), 2 string ownership issues (output name and make need to be
dup'd), fixes some style issues (switches were indented) and finally
adds messages when disabling/enabling idle inhibition.
The callback setter function was removed in preparation for the commit
which will use the frame event cb because it was unnecessary.
The wayland code was written more than 4 years ago when wayland wasn't
even at version 1.0. This commit rewrites everything in a more modern way,
switches to using the new xdg v6 shell interface which solves a lot of bugs
and makes mpv tiling-friedly, adds support for drag and drop, adds support
for touchscreens, adds support for KDE's server decorations protocol,
and finally adds support for the new idle-inhibitor protocol.
It does not yet use the frame callback as a main rendering loop driver,
this will happen with a later commit.
Dudemanguy
Daniel Bermond
Philip Langdale
Niklas Haas
wm4
Rostislav Pehlivanov