Based on the implementation of ffmpeg's sixel backend output written
by Hayaki Saito
https://github.com/saitoha/FFmpeg-SIXEL/blob/sixel/libavdevice/sixel.c
Sixel is a protocol to display graphics in a terminal. This commit
adds support to play videos on a sixel enabled terminal using libsixel.
With --vo=sixel, the output will be in sixel format.
The input frame will be scaled to the user specified resolution
(--vo-sixel-width and --vo-sixel-height) using swscaler and then
encoded using libsixel and output to the terminal. This method
requires high cpu and there are high frame drops for 720p and
higher resolution videos and might require using lesser colors and
have drop in quality. Docs have all the supported options listed
to fine tune the output quality.
TODO: A few parameters of libsixel such as the sixel_encode_policy
and the SIXEL_XTERM16 variables are hardcoded, might want to
expose them as command line options. Also the initialization
resolution is not automatic and if the user doesn't specify the
dimensions, it picks 320x240 as the default resolution which is not
optimal. So need to automatically pick the best fit resolution for
the current open terminal window size.
Wayland’s wl_pointer interface describes the button event’s argument as
being taken from linux/input-event-codes.h, so there is no need to
include the more generic linux/input.h.
Testing kwinft out (kwin fork), it was discovered that sometimes it
would return a ust value of 0 which subsequently resulted in incorrect
presentation statistics (i.e. large negative numbers which are obviously
impossible). Arguably, it shouldn't return 0s, but a workaround for mpv
in this case is harmless.
VOCTRL_UPDATE_RENDER_OPTS is supposed to be optional so check if it
actually exists before executing the function. Fixes a segfault when
changing the alpha value at runtime on non-wayland platforms.
7fb972f fixed transparency on x11/EGL/Mesa but happened to also break it
for wayland and nvidia. Ideally on wayland, you should just be able to
pick the right EGLConfig that has alpha but this doesn't seem to work
because reasons. So just go back to setting the EGL_ALPHA_SIZE bit if
the user asks for alpha. Apparently this worked before for nvidia as
well. The hack is to just run an eglQueryString in the x11egl context.
If it picks up Mesa as the EGL_VENDOR, then force ctx->opts.want_alpha
to 0 and let pick_xrgba_config take care of the rest.
Made possible with 00b9c81. 34b8adc let the wayland surface set an
opaque region depending on if alpha was set by the user or not. However,
there was no attempted detection for runtime changes and it is possible
(at least in wayland vulkan) to toggle the alpha on and off. So this
meant, we could be incorrectly signalling an opaque region if the user
happened to change the alpha. Additionally, add a helper function for
this and use it everywhere we want to set the opaque region.
vo_gpu has a small set of options for ra_ctx that can be set. In
practice, runtime toggling doesn't matter for most of these as they have
no effect while a video is playing. However, changing the alpha option
during runtime can actually work depending on the backend used. mpv
already detected when one of these options changed, but it made no
attempt to update the options in the ra_ctx accordingly (likely because
nothing made any use of this information). Another related change is to
add an update_render_opts to the fns and allow invidiual backends to
(optionally) use it.
efb0c5c changed the rendering logic of mpv on wayland and made it skip
rendering when it did not receive frame callback in time. The idea was
to skip rendering when the surface was hidden and be less wasteful. This
unfortunately had issues in certain instances where a frame callback
could be missed (but the window was still in view) due to imprecise
rendering (like the default audio video-sync mode). This would lead to
the video appearing to stutter since mpv would skip rendering in those
cases.
To account for this case, simply re-add an old heuristic for detecting
if a window is hidden or not since the goal is to simply not render when
a window is hidden. If the wait on the frame callback times out enough
times in a row, then we consider the window hidden and thus begin to
skip rendering then. The actual threshold to consider a surface as
hidden is completely arbitrary (greater than your monitor's refresh
rate), but it's safe enough since realistically you're not going to miss
60+ frame callbacks in a row unless the surface actually is hidden.
Fixes#8169.
Pointless feature that can be done with environment variables. It was
also implemented incorrectly and broke autoprobing.
This reverts commit 015b676875.
As per the client API, a client can connect to any arbitrary wayland
socket. mpv has always just passed NULL which connected to the
compositor currently in use, but one could just as easily pass the name
of a different socket (i.e. the value of WAYLAND_DISPLAY). Here, we just
expose this argument as a user configurable option. If the user passes a
socket name that does not exist, then print a warning and fall back to
NULL.
Apparently a part of the wayland spec. A compositor may use a surface
that has set part of itself as opaque for various optimizations. For
mpv, we simply set the entire surface as opaque as long as the user has
not set alpha=yes (note: alpha is technically broken in the wayland EGL
backend at the time of this commit but oh well). wlshm is always opaque.
Fixes#8125.
on macOS 10.15 setting the activation policy behaves quite weirdly. the
call changes the current active App to a nameless process, which
probably also the reason that prevents the not focusing to work.
a workaround for that, is to refocus the previous active app.
Fixes#7725
Back in the olden days, mpv's wayland backend was driven by the frame
callback. This had several issues and was removed in favor of the
current approach which allowed some advanced features (like
display-resample and presentation time) to actually work properly.
However as a consequence, it meant that mpv always rendered, even if the
surface was hidden. Wayland people consider this "wasteful" (and well
they aren't wrong). This commit aims to avoid wasteful rendering by
doing some additional checks in the swapchain. There's three main parts
to this.
1. Wayland EGL now uses an external swapchain (like the drm context).
Before we start a new frame, we check to see if we are waiting on a
callback from the compositor. If there is no wait, then go ahead and
proceed to render the frame, swap buffers, and then initiate
vo_wayland_wait_frame to poll (with a timeout) for the next potential
callback. If we are still waiting on callback from the compositor when
starting a new frame, then we simple skip rendering it entirely until
the surface comes back into view.
2. Wayland on vulkan has essentially the same approach although the
details are a little different. The ra_vk_ctx does not have support for
an external swapchain and although such a mechanism could theoretically
be added, it doesn't make much sense with libplacebo. Instead,
start_frame was added as a param and used to check for callback.
3. For wlshm, it's simply a matter of adding frame callback to it,
leveraging vo_wayland_wait_frame, and using the frame callback value to
whether or not to draw the image.
If the alignment is less than 16, certain libswscale code paths will
silently corrupt memory outside of the target buffer. This actually
affected the libmpv software rendering API (that was fun to debug).
Rather than passing this problem to the next API user, try to avoid it
within libmpv.
It's unclear which alignment libswscale requires for safe operation. I'm
picking 32 (one more than the observed safe value in the case I
experienced), because libavfilter mostly uses this value.
The way to work this around is slow: just make a full copy of the entire
input or output image. Possibly this could be optimized by using the
slice API, but that would be more effort, and would likely expose
further libswscale bugs. Hope that this is a rarely needed path.
The next commit will update the alignment requirement documentation
bits.
Add a property that returns whether the window is focused, currently
only for X11 and Wayland.
My use cause for this is having an equivalent of pause-when-minimize.lua
for tiling window managers: make mpv play only while it's in the current
workspace or is focused (I'm fine with either one but prefer focus).
On X I do this by observing display-names, which is empty when the
rectangles of the display and mpv don't intersect, but on Wayland its
value doesn't change when mpv leaves the current workspace (and the same
check doesn't work since the geometries still intersect).
This could later be made writable as requested in #6252.
Note that on Wayland se shouldn't consider an unactivated window with
keyboard input focused.
The wlroots compositors I tested set activated after changing the
keyboard focus, so if you set wl->focused only in
keyboard_handle_enter() and keyboard_handle_leave() to avoid adding the
"has_keyboard_input" member, focused isn't set to true when first
opening mpv until you focus another window and focus mpv again.
Conversely, if that order can't be assumed for all compositors, we
should toggle wl->focused when necessary in keyboard_handle_enter() and
keyboard_handle_leave() as well as in handle_toplevel_config().
Transparent windows on X11/EGL/native Mesa GL didn't work for various
reasons. From what I remember, the current code did work with nvidia at
least. Mesa has made attempts to fix this, but they never really made it
in.
But it turns out you can make EGL/Mesa list the EGLConfigs that use X11
RGBA visuals, and context_x11egl.c contains code that explicitly selects
them if alpha is requested (see pick_xrgba_config()).
The reason EGL/Mesa did not list them (and thus breaking transparency)
is because we requested a EGL_ALPHA_SIZE != 0 if alpha is requested. But
the transparent EGLConfigs use EGL_ALPHA_SIZE == 0. That's because EGL
doesn't actually support the concept of transparent windows; the alpha
size parameter is something else (memory rendering without FBOs or
something, I don't care enough to look up the real reasons).
This still won't work on Wayland. Every EGL backend needs platform
specific code. (Good job, EGL, such an awesome platform independent
standard.)
Fixes: #6590
Oversight in b0f0be7. The user_sbc value would update but not last_sbc
if no presentation events were received. This would result in an
incorrect sbc_passed value (in practice, this should always be 1 since,
as far I know, all wayland compositors are currently only capable of
double buffering). When bring the window back into view, it would result
in a single frame of very high vsync jitter. Although in most cases it
was imperceptible, rarely I was able to completely break playback (i.e.
constant mistimed/dropped frames). Fix this by simply incrementing
last_sbc by 1 if the window is hidden. The buffer swap call did still
occur. The user just didn't see it.
Why on earth did I ever bother with this dumb crap? If we do not have
any presentation statistics, just set the relevant vo_sync_info values
to -1 to disable it. It's much simpler than using mp deltas and trying
to keep up with mpv's clock. This also appears to fix audio/video
desynchronization if you start a video with the pause flag, move it out
of view, and then unpause it. Technically harmless since the video
wasn't even in view and putting back in view recovered it, but a quieter
terminal is better.
this fixes some race condition where the content of the layer wasn't
updated because the size didn't changed and the layer was already marked
as updated. just force an update on reconfig.
Fixes#7989
this is preparation for new backends. currently this is done via the
libmpv API but for future new new VOs not based on libmpv we need these
VOCTRL events.
remove the libmpv observer for the macOS specific options and use a
config cache + change callback for runtime changes. this is also a
preparation for new backends and generalises even more, since libmpv
functions can't and shouldn't be used in usual vo backends. for feature
parity the config cache is used.
move all backend independent code parts in their own folder and files,
to simplify adding new backends. the goal is to only extend one class
and add the backend dependent parts there. usually only the (un)init,
config and related parts need to be implemented per backend. furthermore
all needed windowing and related events are propagated and can be
overwritten. the other backend dependent part is usually the surface for
rendering, for example the opengl oder metal layer.
in the best case a new backend can be added with only a few hundred
lines.
It was possible for sway to get incorrectly sized borders if you resized
the mpv window in a creative manner (e.g. open a video in a non-floating
mode, set window scale to 2, then float it and witness wrong border
sizes). This is possibly a sway bug (Plasma doesn't have these border
issues at least), but there's a reasonable workaround for this.
The reason for the incorrect border size is because it is possible for
mpv to ignore the width/height from the toplevel listener and set its
own size. This new size can differ from what sway/wlroots believes the
size is which is what causes the sever side decorations to be drawn on
incorrect dimensions.
A simple trick is to just explicitly commit the surface after a resize
is performed. This is only done if mpv is not fullscreened or maximized
since we always obey the compositor widths/heights in those cases.
Sending the commit signals the compositor of the new change in the
surface and thus sway/wlroots updates its internal coordinates
appropriately and borders are no longer broken.
The original goal was to simplify all this logic to make it less fragile
and breaky. Unfortunately, that didn't exactly happen and things might
actually be more complicated in some ways (well in other ways it's
simplier). There's a lot of negotiation back and forth between the
client and the compositor regarding sizes. The client (aka mpv) can do a
resize on its own. But also the compositor can request its own resize
(which we should be nice and listen to of course). The older method had
a lot of breakfalls/edgecases that were gradually patched up as time
went on, but that approach is really fragile. This refactor should,
hopefully, be on a more solid foundation.
Don't call any of the xdg toplevel state changing functions
(fullscreen, maximized, etc.) directly. Use the toggle wrapper
functions. These signal that the state was changed which is later
handled in the toplevel listener.
Introduce a new vdparams variable that stores the actual dimensions of
the video. This does create some new (but neccesary) complexity.
wl->vdparams stores what the actual dimensions of the video are
(according to mpv). wl->window_size stores the last size of the window
(so it includes any manual resizes for instance). wl->geometry is the
actual size of the output that gets displayed on the screen.
We've had some serious issues with GNOME in the past, but since then
their compositor has undergone some major internal improvements. The
most severe one [1], random vsync spikes and mistimed frames, can no
longer be reproduced by the original author of the issue. There are some
minor UI-related things (lack of window decorations for instance since
there is no xdg-decoration support), but users don't seem to complain
about that too much and they aren't revelant to playback.
3.38 isn't out quite yet, but that should also fix playback issues when
on a multimonitor setup (the fix is in the master branch at the moment).
In terms of playback, the only real concerning issue is the lack of idle
inhibit so a warning is still displayed. But GNOME has their own
workaround that users can use for that so if anyone happens to complain,
we can just point them to that.
[1] https://gitlab.gnome.org/GNOME/mutter/-/issues/957
When using presentation time, we have to be sure to update the ust when
no presentation events are received to make sure playback is still
smooth and in sync. Part of the recent presentation time refactor was to
use the presentation discarded event to signal that the window is
hidden. Evidently, this doesn't work the same everywhere for whatever
reason (drivers?? hardware??) and at least one user experienced issues
with playback getting out of sync since (presumably) the discarded event
didn't occur when hiding the window. Instead, let's just go back to the
old way of checking if the last_ust is equal to the ust value of the
last member in the wayland sync queue. Fixes#8010.
The motivation for this change was a segfault caused by e107342 which
has complicated reasons for occuring (i.e. I'm not 100% sure but I think
it is a really weird race). The major part of this commit is moving the
initialization of presentation listener to the frame_callback function.
Calling it in swap_buffers worked fine but in practice it meant a lot of
meaningless function calls if a window was hidden (the presentation
would just be immediately discarded). By calling it in frame_callback,
we ensure the listener is only created when it is possible to receive a
presentation event.
Of course calling the presentation listener in feedback_presented or
feedback_discarded was considered, but ultimately these events are too
slow. Receiving the ust/msc/sbc triplet here and then passing it to mpv
results in higher vsync judder since there is (likely) not enough time
before the next pageflip. By design, the frame callback is meant to give
us as much time as possible before the next repaint so calling it here
is probably optimal.
Additionally, we can make better use of the feedback_discarded event.
The wp_presentation_feedback should not be destroyed here. It will be
taken care of either when we get feedback again or when the player
quits. Instead what we can do is set a bool that tells wayland_sync_swap
to update itself based on mp_time delta. In practice, the result is not
any different than before, but it should be more understandable what is
going on now.
Of course, the segfault mentioned at the beginning is fixed with this as
well.
Nothing major but it's technically possible for the
wp_presentation_feedback struct to still be allocated when quitting the
player. Just destroy it if it exists like all of the other wayland
objects.
In a playlist of videos with different sizes, going to the next video
would not properly resize the window. This actually broke way back in
7170910 (oops), but somehow nobody ever complained. The fix is simple.
If a window isn't maximized, be sure to set the window geometry again.
Just some small changes when creating the xdg_surface. Don't set the
toplevel title (or app id) in create_xdg_surface anymore because it's
entirely pointless. Also make it possible for create_xdg_surface to
return something other than 0 so the error checking is somewhat
meaningful. It's not really clear if these xdg functions can even fail
in the first place (perhaps some weird proxy marshalling crap could
possibly go wrong somehow), but it can't hurt. Note that all app id
stuff has been removed (temporarily) in this commit. See the next commit
which adds it back in.
Sway 1.5 started sending more pointer motion events to mpv which broke
the autohiding behavior. The cursor would appear again if you
fullscreened. Sway had a good reason to do this because certain
applications had inconsistencies between hardware cursor and software
cursor without rebasing on state changes[1]. So mpv needs to take this
special case into consideration.
Initially, simply checking mouse coordinates for changes was considered,
but this doesn't work. All coordinates are surface-local in wayland so
something can appear to move in the local coordinate space but not
globally. You're not allowed to know global mouse coordinates in
wayland, and we don't care about local coordinate changes in mpv so this
approach isn't viable.
Instead, let's just keep track of a local state change. If the toplevel
surface changes in some way (fullscreen, maximized, etc.), then just set
a bool that lets us ignore the mp_input_set_mouse_pos function. This
keeps the cursor from appearing simply because the state was changed
(i.e. fullscreening). For compositors that don't send pointer motion
events on a state change, this does technically mean that the initial
mp_input_set_mouse_pos is never set. In practice, this isn't a
noticeable difference though because moving a mouse generates a ton of
motion events so you'll immediately see it on the second motion event.
[1] https://github.com/swaywm/sway/issues/5594
wl_display_dispatch is dangerous because it will block forever if the
event queue is empty. Any direct calls to this function should just be
replaced with wl_display_dispatch_pending which accomplishes the same
thing for mpv's purposes without any chance of blocking.
The other potential trap is wl_display_roundtrip. It can internally call
wl_display_dispatch which in certain circumstances could potentially
block. There are cases where we need the server to finish processing
client requests before doing anything else so this can not be cleanly
avoided. The dangerous call is the usage of wl_display_roundtrip in
vo_wayland_wait_frame. In the majority of cases, this shouldn't be a
problem because the previous wl_display_read_events should always queue
up some events on the fd for wl_display_roundtrip to send. However, the
compositor could potentially send us an error in the display queue that
could lead to bad behavior when wl_display_roundtrip is called.
The wl_display_roundtrip can't be removed because we are relying on its
semi-blocking capabilities, but the logic can be slightly adjusted to be
safer. The obvious thing to do is to make sure we check the pollfd for
any errors. If one is returned, then we call wl_display_cancel_read and
try again. The less obvious trick is to call wl_display_dispatch_pending
and move wl_display_roundtrip outside of the blocking + timeout loop.
This change has some subtle but important differences. Previously,
vo_wayland_wait_frame would read an event and wait on the server to
process it one-by-one. With this change, the events are dispatched as
soon as possible to the server and then we wait on all of those
(potentially multiple) events to be processed after we have either
received frame callback or the loop times out.
After that is done, we can then check for if there are any errors on the
display. If it's all clear, we can run wl_display_roundtrip without any
worries. If some error happens, then don't execute the function at all.
Previously, the compositor was signaled that a drag-and-drop ended with
wl_data_offer_finish in check_dnd_fd. This is, however, erroneous
because it is outside of the data_device_listener and in some cases
caused errors with certain compositors. check_dnd_fd itself does not
need to know or care about anything that happens in wayland. It just
needs to read data from an fd. The simple fix is to just always signal
the end of a drag-and-drop in data_device_handle_drop. check_dnd_fd can
free memory and close the fd later, but it should not talk to the
compositor. Fixes#7954.
The read of the wayland display fd in vo_wayland_wait_events was
incorrect and technically vulnerable to race conditions. The correct
usage as per the client api is to use wl_display_prepare_read as well as
wl_display_read_events.
Rather than after tone-mapping. This prevents overflow when the
pre-tonemapped signal contains inputs exceeding sig_peak. I also
realized that with this clipping in place, post-clipping no longer needs
to be done, so this isn't even particularly slower.
The only two exceptions to the rule are "clip" and "linear", which
relied on the post-clipping to do their tone mapping properly.
Fixes#7929
This normally gets printed by libplacebo itself when initializing the
context, but due to the way our code is structured (for convenience) we
don't have the log hook enabled by the time this function call is
relevant. So instead just print it manually as an easier work-around
than restructuring the code.
Shreesh Adiga
Jan Ekström
Emmanuel Gil Peyrot
Niklas Haas
Dudemanguy
der richter
sfan5
wm4
Guido Cella