2017-09-16 02:46:38 +00:00
|
|
|
/*
|
|
|
|
|
* This file is part of mpv.
|
|
|
|
|
*
|
|
|
|
|
* mpv is free software; you can redistribute it and/or
|
|
|
|
|
* modify it under the terms of the GNU Lesser General Public
|
|
|
|
|
* License as published by the Free Software Foundation; either
|
|
|
|
|
* version 2.1 of the License, or (at your option) any later version.
|
|
|
|
|
*
|
|
|
|
|
* mpv is distributed in the hope that it will be useful,
|
|
|
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
* GNU Lesser General Public License for more details.
|
|
|
|
|
*
|
|
|
|
|
* You should have received a copy of the GNU Lesser General Public
|
|
|
|
|
* License along with mpv. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include "video/out/gpu/context.h"
|
|
|
|
|
#include "video/out/wayland_common.h"
|
|
|
|
|
|
|
|
|
|
#include "common.h"
|
|
|
|
|
#include "context.h"
|
|
|
|
|
#include "utils.h"
|
|
|
|
|
|
|
|
|
|
struct priv {
|
|
|
|
|
struct mpvk_ctx vk;
|
|
|
|
|
};
|
|
|
|
|
|
2021-11-04 14:42:54 +00:00
|
|
|
static bool wayland_vk_check_visible(struct ra_ctx *ctx)
|
wayland: only render if we have frame callback
Back in the olden days, mpv's wayland backend was driven by the frame
callback. This had several issues and was removed in favor of the
current approach which allowed some advanced features (like
display-resample and presentation time) to actually work properly.
However as a consequence, it meant that mpv always rendered, even if the
surface was hidden. Wayland people consider this "wasteful" (and well
they aren't wrong). This commit aims to avoid wasteful rendering by
doing some additional checks in the swapchain. There's three main parts
to this.
1. Wayland EGL now uses an external swapchain (like the drm context).
Before we start a new frame, we check to see if we are waiting on a
callback from the compositor. If there is no wait, then go ahead and
proceed to render the frame, swap buffers, and then initiate
vo_wayland_wait_frame to poll (with a timeout) for the next potential
callback. If we are still waiting on callback from the compositor when
starting a new frame, then we simple skip rendering it entirely until
the surface comes back into view.
2. Wayland on vulkan has essentially the same approach although the
details are a little different. The ra_vk_ctx does not have support for
an external swapchain and although such a mechanism could theoretically
be added, it doesn't make much sense with libplacebo. Instead,
start_frame was added as a param and used to check for callback.
3. For wlshm, it's simply a matter of adding frame callback to it,
leveraging vo_wayland_wait_frame, and using the frame callback value to
whether or not to draw the image.
2020-09-18 17:29:53 +00:00
|
|
|
{
|
|
|
|
|
struct vo_wayland_state *wl = ctx->vo->wl;
|
2021-06-27 14:55:33 +00:00
|
|
|
bool render = !wl->hidden || wl->opts->disable_vsync;
|
wayland: simplify render loop
This is actually a very nice simplification that should have been
thought of years ago (sue me). In a nutshell, the story with the
wayland code is that the frame callback and swap buffer behavior doesn't
fit very well with mpv's rendering loop. It's been refactored/changed
quite a few times over the years and works well enough but things could
be better. The current iteration works with an external swapchain to
check if we have frame callback before deciding whether or not to
render. This logic was implemented in both egl and vulkan.
This does have its warts however. There's some hidden state detection
logic which works but is kind of ugly. Since wayland doesn't allow
clients to know if they are actually visible (questionable but
whatever), you can just reasonably assume that if a bunch of callbacks
are missed in a row, you're probably not visible. That's fine, but it is
indeed less than ideal since the threshold is basically entirely
arbitrary and mpv does do a few wasteful renders before it decides that
the window is actually hidden.
The biggest urk in the vo_wayland_wait_frame is the use of
wl_display_roundtrip. Wayland developers would probably be offended by
the way mpv abuses that function, but essentially it was a way to have
semi-blocking behavior needed for display-resample to work. Since the
swap interval must be 0 on wayland (otherwise it will block the entire
player's rendering loop), we need some other way to wait on vsync. The
idea here was to dispatch and poll a bunch of wayland events, wait (with
a timeout) until we get frame callback, and then wait for the compositor
to process it. That pretty much perfectly waits on vsync and lets us
keep all the good timings and all that jazz that we want for mpv. The
problem is that wl_display_roundtrip is conceptually a bad function. It
can internally call wl_display_dispatch which in certain instances,
empty event queue, will block forever. Now strictly speaking, this
probably will never, ever happen (once I was able to to trigger it by
hardcoding an error into a compositor), but ideally
vo_wayland_wait_frame should never infinitely block and stall the
player. Unfortunately, removing that function always lead to problems
with timings and unsteady vsync intervals so it survived many refactors.
Until now, of course. In wayland, the ideal is to never do wasteful
rendering (i.e. don't render if the window isn't visible). Instead of
wrestling around with hidden states and possible missed vblanks, let's
rearrange the wayland rendering logic so we only ever draw a frame when
the frame callback is returned to use (within a reasonable timeout to
avoid blocking forever).
This slight rearrangement of the wait allows for several simplifications
to be made. Namely, wl_display_roundtrip stops being needed. Instead, we
can rely entirely on totally nonblocking calls (dispatch_pending, flush,
and so on). We still need to poll the fd here to actually get the frame
callback event from the compositor, but there's no longer any reason to
do extra waiting. As soon as we get the callback, we immediately draw.
This works quite well and has stable vsync (display-resample and audio).
Additionally, all of the logic about hidden states is no longer needed.
If vo_wayland_wait_frame times out, it's okay to assume immediately that
the window is not visible and skip rendering.
Unfortunately, there's one limitation on this new approach. It will only
work correctly if the compositor implements presentation time. That
means a reduced version of the old way still has to be carried around in
vo_wayland_wait_frame. So if the compositor has no presentation time,
then we are forced to use wl_display_roundtrip and juggle some funny
assumptions about whether or not the window is hidden or not. Plasma is
the only real notable compositor without presentation time at this stage
so perhaps this "legacy" mechanism could be removed in the future.
2021-05-17 19:36:59 +00:00
|
|
|
wl->frame_wait = true;
|
wayland: shuffle around the render loop again
Take two. f4e89dd went wrong by moving vo_wayland_wait_frame before
start_frame was called. Whether or not this matters depends on the
compositor, but some weird things can happen. Basically, it's a
scheduling issue. vo_wayland_wait_frame queues all events and sends them
to the server to process (with no blocking if presentation time is
available). If mpv changes state while rendering (and this function is
called before every frame is drawn), then that event also gets
dispatched and sent to the compositor. This, in some cases, can cause
some funny behavior because the next frame gets attached to the surface
while the old buffer is getting released. It's safer to call this
function after the swap already happens and well before mpv calls its
next draw. There's no weird scheduling of events, and the compositor log
is more normal.
The second part of this is to fix some stuttering issues. This is mostly
just conjecture, but probably what was happening was this thing called
"composition". The easiest way to see this is to play a video on the
default audio sync mode (probably easiest to see on a typical 23.976
video). Have that in a window and float it over firefox (floating
windows are bloat on a tiling wm anyway). Then in firefox, do some short
bursts of smooth scrolling (likely uses egl). Some stutter in video
rendering could be observed, particularly in panning shots.
Compositors are supposed to prevent tearing so what likely was happening
was that the compositor was simply holding the buffer a wee bit longer
to make sure it happened in sync with the smooth scrolling. Because the
mpv code waits precisely on presentation time, the loop would timeout on
occasion instead of receiving the frame callback. This would then lead
to a skipped frame when rendering and thus causing stuttering.
The fix is simple: just only count consecutive timeouts as not receiving
frame callback. If a compositor holds the mpv buffer slightly longer to
avoid tearing, then we will definitely receive frame callback on the
next round of the render loop. This logic also appears to be sound for
plasma (funfact: Plasma always returns frame callback even when the
window is hidden. Not sure what's up with that, but luckily it doesn't
matter to us.), so get rid of the goofy 1/vblank_time thing and just
keep it a simple > 1 check.
2021-05-23 19:36:19 +00:00
|
|
|
|
wayland: only render if we have frame callback
Back in the olden days, mpv's wayland backend was driven by the frame
callback. This had several issues and was removed in favor of the
current approach which allowed some advanced features (like
display-resample and presentation time) to actually work properly.
However as a consequence, it meant that mpv always rendered, even if the
surface was hidden. Wayland people consider this "wasteful" (and well
they aren't wrong). This commit aims to avoid wasteful rendering by
doing some additional checks in the swapchain. There's three main parts
to this.
1. Wayland EGL now uses an external swapchain (like the drm context).
Before we start a new frame, we check to see if we are waiting on a
callback from the compositor. If there is no wait, then go ahead and
proceed to render the frame, swap buffers, and then initiate
vo_wayland_wait_frame to poll (with a timeout) for the next potential
callback. If we are still waiting on callback from the compositor when
starting a new frame, then we simple skip rendering it entirely until
the surface comes back into view.
2. Wayland on vulkan has essentially the same approach although the
details are a little different. The ra_vk_ctx does not have support for
an external swapchain and although such a mechanism could theoretically
be added, it doesn't make much sense with libplacebo. Instead,
start_frame was added as a param and used to check for callback.
3. For wlshm, it's simply a matter of adding frame callback to it,
leveraging vo_wayland_wait_frame, and using the frame callback value to
whether or not to draw the image.
2020-09-18 17:29:53 +00:00
|
|
|
return render;
|
|
|
|
|
}
|
|
|
|
|
|
2020-08-15 21:07:53 +00:00
|
|
|
static void wayland_vk_swap_buffers(struct ra_ctx *ctx)
|
|
|
|
|
{
|
|
|
|
|
struct vo_wayland_state *wl = ctx->vo->wl;
|
wayland: shuffle around the render loop again
Take two. f4e89dd went wrong by moving vo_wayland_wait_frame before
start_frame was called. Whether or not this matters depends on the
compositor, but some weird things can happen. Basically, it's a
scheduling issue. vo_wayland_wait_frame queues all events and sends them
to the server to process (with no blocking if presentation time is
available). If mpv changes state while rendering (and this function is
called before every frame is drawn), then that event also gets
dispatched and sent to the compositor. This, in some cases, can cause
some funny behavior because the next frame gets attached to the surface
while the old buffer is getting released. It's safer to call this
function after the swap already happens and well before mpv calls its
next draw. There's no weird scheduling of events, and the compositor log
is more normal.
The second part of this is to fix some stuttering issues. This is mostly
just conjecture, but probably what was happening was this thing called
"composition". The easiest way to see this is to play a video on the
default audio sync mode (probably easiest to see on a typical 23.976
video). Have that in a window and float it over firefox (floating
windows are bloat on a tiling wm anyway). Then in firefox, do some short
bursts of smooth scrolling (likely uses egl). Some stutter in video
rendering could be observed, particularly in panning shots.
Compositors are supposed to prevent tearing so what likely was happening
was that the compositor was simply holding the buffer a wee bit longer
to make sure it happened in sync with the smooth scrolling. Because the
mpv code waits precisely on presentation time, the loop would timeout on
occasion instead of receiving the frame callback. This would then lead
to a skipped frame when rendering and thus causing stuttering.
The fix is simple: just only count consecutive timeouts as not receiving
frame callback. If a compositor holds the mpv buffer slightly longer to
avoid tearing, then we will definitely receive frame callback on the
next round of the render loop. This logic also appears to be sound for
plasma (funfact: Plasma always returns frame callback even when the
window is hidden. Not sure what's up with that, but luckily it doesn't
matter to us.), so get rid of the goofy 1/vblank_time thing and just
keep it a simple > 1 check.
2021-05-23 19:36:19 +00:00
|
|
|
|
|
|
|
|
if (!wl->opts->disable_vsync)
|
|
|
|
|
vo_wayland_wait_frame(wl);
|
|
|
|
|
|
wayland: only render if we have frame callback
Back in the olden days, mpv's wayland backend was driven by the frame
callback. This had several issues and was removed in favor of the
current approach which allowed some advanced features (like
display-resample and presentation time) to actually work properly.
However as a consequence, it meant that mpv always rendered, even if the
surface was hidden. Wayland people consider this "wasteful" (and well
they aren't wrong). This commit aims to avoid wasteful rendering by
doing some additional checks in the swapchain. There's three main parts
to this.
1. Wayland EGL now uses an external swapchain (like the drm context).
Before we start a new frame, we check to see if we are waiting on a
callback from the compositor. If there is no wait, then go ahead and
proceed to render the frame, swap buffers, and then initiate
vo_wayland_wait_frame to poll (with a timeout) for the next potential
callback. If we are still waiting on callback from the compositor when
starting a new frame, then we simple skip rendering it entirely until
the surface comes back into view.
2. Wayland on vulkan has essentially the same approach although the
details are a little different. The ra_vk_ctx does not have support for
an external swapchain and although such a mechanism could theoretically
be added, it doesn't make much sense with libplacebo. Instead,
start_frame was added as a param and used to check for callback.
3. For wlshm, it's simply a matter of adding frame callback to it,
leveraging vo_wayland_wait_frame, and using the frame callback value to
whether or not to draw the image.
2020-09-18 17:29:53 +00:00
|
|
|
if (wl->presentation)
|
2021-06-26 21:12:03 +00:00
|
|
|
vo_wayland_sync_swap(wl);
|
2019-10-10 19:14:40 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void wayland_vk_get_vsync(struct ra_ctx *ctx, struct vo_vsync_info *info)
|
|
|
|
|
{
|
|
|
|
|
struct vo_wayland_state *wl = ctx->vo->wl;
|
2020-04-09 16:17:03 +00:00
|
|
|
if (wl->presentation) {
|
2019-10-10 19:14:40 +00:00
|
|
|
info->vsync_duration = wl->vsync_duration;
|
|
|
|
|
info->skipped_vsyncs = wl->last_skipped_vsyncs;
|
|
|
|
|
info->last_queue_display_time = wl->last_queue_display_time;
|
|
|
|
|
}
|
2019-10-07 20:58:36 +00:00
|
|
|
}
|
|
|
|
|
|
2017-10-01 20:16:49 +00:00
|
|
|
static void wayland_vk_uninit(struct ra_ctx *ctx)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
|
|
|
|
struct priv *p = ctx->priv;
|
|
|
|
|
|
|
|
|
|
ra_vk_ctx_uninit(ctx);
|
|
|
|
|
mpvk_uninit(&p->vk);
|
|
|
|
|
vo_wayland_uninit(ctx->vo);
|
|
|
|
|
}
|
|
|
|
|
|
2017-10-01 20:16:49 +00:00
|
|
|
static bool wayland_vk_init(struct ra_ctx *ctx)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
|
|
|
|
struct priv *p = ctx->priv = talloc_zero(ctx, struct priv);
|
|
|
|
|
struct mpvk_ctx *vk = &p->vk;
|
|
|
|
|
int msgl = ctx->opts.probing ? MSGL_V : MSGL_ERR;
|
|
|
|
|
|
2018-11-10 11:53:33 +00:00
|
|
|
if (!mpvk_init(vk, ctx, VK_KHR_WAYLAND_SURFACE_EXTENSION_NAME))
|
2017-09-16 02:46:38 +00:00
|
|
|
goto error;
|
|
|
|
|
|
|
|
|
|
if (!vo_wayland_init(ctx->vo))
|
|
|
|
|
goto error;
|
|
|
|
|
|
|
|
|
|
VkWaylandSurfaceCreateInfoKHR wlinfo = {
|
|
|
|
|
.sType = VK_STRUCTURE_TYPE_WAYLAND_SURFACE_CREATE_INFO_KHR,
|
2017-10-01 20:16:49 +00:00
|
|
|
.display = ctx->vo->wl->display,
|
|
|
|
|
.surface = ctx->vo->wl->surface,
|
2017-09-16 02:46:38 +00:00
|
|
|
};
|
|
|
|
|
|
2019-10-07 20:58:36 +00:00
|
|
|
struct ra_vk_ctx_params params = {
|
2021-11-04 14:42:54 +00:00
|
|
|
.check_visible = wayland_vk_check_visible,
|
2019-10-07 20:58:36 +00:00
|
|
|
.swap_buffers = wayland_vk_swap_buffers,
|
2019-10-10 19:14:40 +00:00
|
|
|
.get_vsync = wayland_vk_get_vsync,
|
2019-10-07 20:58:36 +00:00
|
|
|
};
|
|
|
|
|
|
2018-11-10 11:53:33 +00:00
|
|
|
VkInstance inst = vk->vkinst->instance;
|
|
|
|
|
VkResult res = vkCreateWaylandSurfaceKHR(inst, &wlinfo, NULL, &vk->surface);
|
2017-09-16 02:46:38 +00:00
|
|
|
if (res != VK_SUCCESS) {
|
2018-11-10 11:53:33 +00:00
|
|
|
MP_MSG(ctx, msgl, "Failed creating Wayland surface\n");
|
2017-09-16 02:46:38 +00:00
|
|
|
goto error;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Because in Wayland clients render whenever they receive a callback from
|
|
|
|
|
* the compositor, and the fact that the compositor usually stops sending
|
|
|
|
|
* callbacks once the surface is no longer visible, using FIFO here would
|
|
|
|
|
* mean the entire player would block on acquiring swapchain images. Hence,
|
|
|
|
|
* use MAILBOX to guarantee that there'll always be a swapchain image and
|
|
|
|
|
* the player won't block waiting on those */
|
2019-10-07 20:58:36 +00:00
|
|
|
if (!ra_vk_ctx_init(ctx, vk, params, VK_PRESENT_MODE_MAILBOX_KHR))
|
2017-09-16 02:46:38 +00:00
|
|
|
goto error;
|
|
|
|
|
|
2018-12-30 22:59:48 +00:00
|
|
|
ra_add_native_resource(ctx->ra, "wl", ctx->vo->wl->display);
|
|
|
|
|
|
2017-09-16 02:46:38 +00:00
|
|
|
return true;
|
|
|
|
|
|
|
|
|
|
error:
|
2017-10-01 20:16:49 +00:00
|
|
|
wayland_vk_uninit(ctx);
|
2017-09-16 02:46:38 +00:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
2018-11-10 11:53:33 +00:00
|
|
|
static bool resize(struct ra_ctx *ctx)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
2017-10-01 20:16:49 +00:00
|
|
|
struct vo_wayland_state *wl = ctx->vo->wl;
|
2017-09-16 02:46:38 +00:00
|
|
|
|
2017-10-09 01:08:36 +00:00
|
|
|
MP_VERBOSE(wl, "Handling resize on the vk side\n");
|
2017-09-16 02:46:38 +00:00
|
|
|
|
2020-10-05 15:28:37 +00:00
|
|
|
const int32_t width = wl->scaling * mp_rect_w(wl->geometry);
|
|
|
|
|
const int32_t height = wl->scaling * mp_rect_h(wl->geometry);
|
2020-10-01 16:00:58 +00:00
|
|
|
|
2020-10-05 15:28:37 +00:00
|
|
|
vo_wayland_set_opaque_region(wl, ctx->opts.want_alpha);
|
2020-11-08 15:51:52 +00:00
|
|
|
return ra_vk_ctx_resize(ctx, width, height);
|
2017-09-16 02:46:38 +00:00
|
|
|
}
|
|
|
|
|
|
2017-10-01 20:16:49 +00:00
|
|
|
static bool wayland_vk_reconfig(struct ra_ctx *ctx)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
2017-10-01 20:16:49 +00:00
|
|
|
if (!vo_wayland_reconfig(ctx->vo))
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
return true;
|
2017-09-16 02:46:38 +00:00
|
|
|
}
|
|
|
|
|
|
2017-10-01 20:16:49 +00:00
|
|
|
static int wayland_vk_control(struct ra_ctx *ctx, int *events, int request, void *arg)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
|
|
|
|
int ret = vo_wayland_control(ctx->vo, events, request, arg);
|
|
|
|
|
if (*events & VO_EVENT_RESIZE) {
|
2018-11-10 11:53:33 +00:00
|
|
|
if (!resize(ctx))
|
2017-09-16 02:46:38 +00:00
|
|
|
return VO_ERROR;
|
|
|
|
|
}
|
|
|
|
|
return ret;
|
|
|
|
|
}
|
|
|
|
|
|
2017-10-01 20:16:49 +00:00
|
|
|
static void wayland_vk_wakeup(struct ra_ctx *ctx)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
|
|
|
|
vo_wayland_wakeup(ctx->vo);
|
|
|
|
|
}
|
|
|
|
|
|
2017-10-01 20:16:49 +00:00
|
|
|
static void wayland_vk_wait_events(struct ra_ctx *ctx, int64_t until_time_us)
|
2017-09-16 02:46:38 +00:00
|
|
|
{
|
|
|
|
|
vo_wayland_wait_events(ctx->vo, until_time_us);
|
|
|
|
|
}
|
|
|
|
|
|
2020-10-05 15:28:37 +00:00
|
|
|
static void wayland_vk_update_render_opts(struct ra_ctx *ctx)
|
|
|
|
|
{
|
|
|
|
|
struct vo_wayland_state *wl = ctx->vo->wl;
|
|
|
|
|
vo_wayland_set_opaque_region(wl, ctx->opts.want_alpha);
|
|
|
|
|
wl_surface_commit(wl->surface);
|
|
|
|
|
}
|
|
|
|
|
|
2017-09-16 02:46:38 +00:00
|
|
|
const struct ra_ctx_fns ra_ctx_vulkan_wayland = {
|
2020-10-05 15:28:37 +00:00
|
|
|
.type = "vulkan",
|
|
|
|
|
.name = "waylandvk",
|
|
|
|
|
.reconfig = wayland_vk_reconfig,
|
|
|
|
|
.control = wayland_vk_control,
|
|
|
|
|
.wakeup = wayland_vk_wakeup,
|
|
|
|
|
.wait_events = wayland_vk_wait_events,
|
|
|
|
|
.update_render_opts = wayland_vk_update_render_opts,
|
|
|
|
|
.init = wayland_vk_init,
|
|
|
|
|
.uninit = wayland_vk_uninit,
|
2017-09-16 02:46:38 +00:00
|
|
|
};
|
