Libass commit f08f8ea5 (between 0.16 and 0.17) changed how PlayResX
affects some aspects of rendering.
The libass change fixes a VSFilter compatibility issue which existed
for about two decades, and there are no libass plans to support the
previous behavior, so ultimately we have to adjust the mpv code, and
we can't guarantee to restore the old behavior in all cases.
Starting at this commit, vector drawing coords, font spacing, border
and shadow widths are all affected by PlayResX (specifically, by
the aspect), while previously they were unaffected by PlayResX.
This changed converted sub border and shadow widths in mpv, because
ffmpeg generates the ass with fixed PlayResX of 384 (aspect of 4:3),
and with libass 0.17, if this doesn't match the display aspect, then
borders and shadow were too wide - because most clips aspect is more
than 4:3.
The fact that ffmpeg uses fixed PlayResX of 384 could be considered
an issue, but for now we have no control over it, and ffmpeg doesn't
have the video resolution when it converts an srt source to ass.
So here we adjust PlayResX accordingly so that border/shadows are
now rendered with correct width.
However, regardless of that commit, changing PlayResX also affects
the margin value, so to compensate, we adjust sub-margins-x too.
According to libass devs, this should cover basic srt-to-ass
conversion by ffmpeg to work correctly with libass 0.17.
However, there could be srt extensions which use more complex ass,
and/or ffmpeg conversion of other sub formats (such as aribb24,
aribcaption and movtext), where more things need adjustments.
As of now we don't know what these are, and so we don't really know
what else might remain broken or get broken.
dbc5d7b7db seems to have originally
introduced this behavior. At the time, wm4 simply reconfigured ass on
every frame in order to accommodate runtime changes in sub options. This
certainly works, but these libass API calls are not free and there is at
least one known performance regression due to a change in libass*.
Regardless of whether or not the libass change is good/bad, there is no
need for mpv to constantly reconfigure this. When wm4 made that commit,
there was no notification mechanism for options changing that could
easily be used so he didn't really have any other choice. But it's
nearly 10 years later now and internally we have all the necessary
pieces to only configure ass again when we need to: on option changes or
resizes. So go ahead and implement that in this commit which simply uses
the already existing SD_CTRL_UPDATE_OPTS and compares osd_res sizes to
determine whether or not an ass configure is needed.
*: https://github.com/libass/libass/issues/698
Essentially the same as d1d2370d07 except
for clear_rgba_overlay. From some empirical testing, the s->x1 value is
either 0 or SLICE_W (256). In the case where it is 256 and s->x0 is 0,
then it writes memory. For most slices, this is fine. However for the
very last slice in the loop, it is possible for the width here to exceed
the total width of the line (p->w). If that occurs, the memset triggers
a buffer overflow. This last case needs to be guarded in the same way as
the previously mentioned commit (total width - SLICE_W * x) and with
MPMIN in case s->x1 is 0 here to preserve the old behavior. It's unknown
if anything other than dmabuf-wayland can possibly hit this, but it's
definitely a problem within mp_draw_sub_overlay itself.
this changes mp_image_new_ref() to handle allocation failure itself
instead of doing it at its many call-sites (some of which never checked
for failure at all).
also remove MP_HANDLE_OOM() from the call sites since this is not
necessary anymore.
not all the call-sites have been touched, since some of the caller might
be relying on `mp_image_new_ref(NULL)` returning NULL.
Fixes: https://github.com/mpv-player/mpv/issues/11840
Upon an option update with an UPDATE_SUB_HARD flag,
the ass_track that stores all the decoded
subtitle packets/events is destroyed and recreated, which means
the packets need to be read and decoded again to refill
the ass_track. This caused issues (no subs displayed) in 2 cases:
1. external sub files
Previously, external sub files were read and decoded only
once when loaded. Since this meant all decoded events were lost
forever when recreating the ass_track, we need to change this
and trigger a new preload during sub reinits.
2. converted subs (non-ASS text subs like srt)
For converted subs, we maintain a list of previously
seen packets to avoid decoding and adding duplicate events
to the ass_track. Previously this list wasn’t synchronized with
the corresponding ass_track, so the sub decoder would reject
any previously seen sub packets, usually meaning only subs sometime
after the current pts would be displayed after sub reinits.
Fix this by resetting the list upon ass_track recreation.
This way we receive such minor details as the profile (necessary for
ARIB captions, among others) during init. This enables decoders
to switch between ARIB caption profile A and profile C streams.
This reworks all of mpv's unit tests so they are compiled as separate
executables (optional) and run via meson test. Because most of the tests
are dependant on mpv's internals, existing compiled objects are
leveraged to create static libs and used when necessary. As an aside, a
function was moved into video/out/gpu/utils for sanity's sake (otherwise
most of vo would have been needed). As a plus, meson multithreads
running tests automatically and also the output no longer pollutes the
source directory. There are tests that can break due to ffmpeg changes,
so they require a specific minimum libavutil version to be built.
These options make it possible to specify the directory that will be
passed to ass_set_fonts_dir(), akin to VLC's `--ssa-fontsdir` and
FFmpeg's `fontsdir`.
Fixes#8338
c784820454 introduced a bool option type
as a replacement for the flag type, but didn't actually transition and
remove the flag type because it would have been too much mundane work.
The `osd-overlay` command didn't trigger a redraw when the overlay was
set to hidden.
This is fine when the overlay was already hidden before that, but
transitioning from not hidden to hidden requires a redraw.
e97819f88e corrected a special case
condition that lead to an out of bounds access if the total width
happened to be an integer multiple of SLICE_W (256) which could cause a
crash in software VOs. However, it turns out that the functions in this
file evaluate quite differently when using encoding mode (and presumably
libmpv as well according to reports although I could not independently
verify it).
The logic here gets complicated but what ends up happening is that, in
blend_overlay_with_video, the value of x + w can be greater than p->w in
certain cases in encoding mode. The x is the positional value of the
slice which remained unchanged from before, but w can take the full
value of SLICE_W (256) which is not necessarily correct. The width of
the final slice here should be the total remaining width. We can handle
this in mark_rect by simply always adjusting x1 of the last slice to be
equal to total width - SLICE_W * x so it can never extend beyond where
it should be. In practice, this value should be the maximum allowed
here. I'm not sure if the existing x1 value can possibly already be
lower than SLICE_W, but just MPMIN it to be on the safe side.
Fixes#10908.
This has been a long standing annoyance - ffmpeg is removing
sizeof(AVPacket) from the API which means you cannot stack-allocate
AVPacket anymore. However, that is something we take advantage of
because we use short-lived AVPackets to bridge from native mpv packets
in our main decoding paths.
We don't think that switching these to `av_packet_alloc` is desirable,
given the cost of heap allocation, so this change takes a different
approach - allocating a single packet in the relevant context and
reusing it over and over.
That's fairly straight-forward, with the main caveat being that
re-initialising the packet is unintuitive. There is no function that
does exactly what we need (what `av_init_packet` did). The closest is
`av_packet_unref`, which additionally frees buffers and side-data.
However, we don't copy those things - we just assign them in from our
own packet, so we have to explicitly clear the pointers before calling
`av_packet_unref`. But at least we can make a wrapper function for
that.
The weirdest part of the change is the handling of the vtt subtitle
conversion. This requires two packets, so I had to pre-allocate two in
the context struct. That sounds excessive, but if allocating the
primary packet is too expensive, then allocating the secondary one for
vtt subtitles must also be too expensive.
This change is not conditional as heap allocated AVPackets were
available for years and years before the deprecation.
It turns out, even xy-VSFilter and XySubFilter do not
mangle colours if the video is native RGB regardless of
the sub's YCbCr header. libass' docs were also updated
to reflect this.
Commit 740b701 introduced handling for subtitles with unknown duration,
but it came with a minor flaw where a track event that shares identical
start time with following track event will has its Duration value set
to 0, we don't want this to happen since it will prevent simultaneous
rendering of multiple track events.
This commit aims to address this issue.
When the width is exactly a multiple of SLICE_W (currently 256),
heap buffer overflow is reported by Address Sanitizer. So adjust
the maximum index for the line array accordingly.
ASS must only automatically break at ASCII spaces (\x20), but other
subtitle formats might expect more breaking oppurtinities.
Especially non-ASS subs in scripts, which typically do not use (ASCII)
spaces to seperate words, like e.g. CJK, might overflow the screen
if the conversion didn't insert additional linebreaks (ffmpeg does not).
Thus try to enable Unicode linebreaking for converted subs and the OSD
if libass is new enough. The feature may still be unavailable at runtime
if libass wasn't build with Unicode linebreaking support.
TL;DR: previously a JavaScript VM was created + destroyed whenever
a sub track was initialized, even if no jsre filter was set.
Now a JS VM is created only if jsre filters were set.
Sub filters are initialized once when a subtitle track is chosen, and
then whenever the sub track changes or when some sub options change.
Sub filters init is synchronous - playback is suspended till it ends.
A filter can abort init early (get disabled) depending on conditions
specific to each filter. The regex and jsre filters aborted early
if the filter is disabled (default is enabled) or if the track is not
ass (relativey rare, e.g. bitmap subs).
The init then iterates over the filter strings, and if the result is
empty (common - no filter was added, but also if all strings failed
regex init) then it's also aborted during init.
While this iteration step is cheap with filter regex, with jsre it
requires instanciating the JS VM (mujs) in advance in order to parse
the filter strings at the list, and the VM is then destroyed if the
list ends up empty.
This VM create+destroy is fast but measurable (0.2 - 0.7 ms, slowest
measured on 2010 MacBook Air), but can be avoided altogether if we
check that the filter list is not empty before we create the VM.
So now we do just that.
this field is used only in a special vo draining edge case.
switching to an atomic reduces osd->lock contention
between the mpv core (in write_video) and vo threads
which are managing osd rendering manually (such as vo_rpi).
Signed-off-by: Aman Karmani <aman@tmm1.net>
Previously, the sub-visibility option changed the visibility of all
subtitles including secondary ones. This meant that it was not possible
to only display secondary subtitles while hiding the primary ones. This
modifies the sub-visibility option so that it only affects primary
subtitles which allows only secondary subtitles to be displayed.
This reverts commit 04f0b0abe4.
It's not a good idea to unify the names only for visibility, while
keeping secondary-* for everything else.
This needs a bit more thought before we allow secondary sub to be
visible on its own.
Adds --sub-visibility choices 'primary-only' for only displaying the
primary subtitle track, and 'secondary-only' for only displaying
secondary subtitle track.
Removes --secondary-sub-visibility and displays a message telling the
user to use --sub-visibility=yes/primary-only instead.
These changes make it so that the default 'sub-visibility' bind 'v'
cycles through all the 'sub-visibility' choices, 'no', 'yes',
'primary-only', and 'secondary-only'.
Since libavcodec major version 59, the requested "ass" format became
the default as the old timing-included format was disabled starting
with that version. Additionally, this option by itself has since
been deprecated as it no longer serves any purpose.
References:
FFmpeg/FFmpeg@1f63665ca5FFmpeg/FFmpeg@176b8d785bFixes#9413
This is an artificial background box with the original colors,
rendered as a plain rectangle instead of libass's opaque-box.
Fixes#9134 (together with the previous commit)
This breaks the rendering in various ways, so first workaround is to
simply disable the opaque box for the bar.
Next commit will add an artificial background box.
29e15e6248 prefixed youtube-dl's subs url with an edl prefix to not
download them until they're selected, which is useful when there are
many sub languages. But this prefix broke the alignment of secondary
subs, which would overlap the primary subs instead of always being
placed at the top. This can be tested with
--sub-file='edl://!no_clip;!delay_open,media_type=sub;secondary_sub.srt'
When a sub is added, sub.c:reinit_sub() is called. This calls in
init_subdec() -> dec_sub.c:sub_create() -> init_decoder() ->
sd_ass:init(). Then reinit_sub() calls
sub_control(track->d_sub, SD_CTRL_SET_TOP, &(bool){!!order}) which sets
sd_ass_priv.on_top = true for secondary subs.
But for EDL subs the real sub is initialized again when in
dec_sub.c:sub_read_packets() is_new_segment() returns true and
update_segment() is called, or when sub_get_bitmaps() calls
update_segment(). update_segment() then calls init_decoder(), which
calls sd_ass:init(), so sd_ass_priv is reinitialized, and its on_top
property is left false. This commit sets it to true again.
For URLs that need to be downloaded it seems that the update_segment()
call that reinitializes sd_ass_priv is always the one in
sub_read_packets(), but with local subs sub_get_bitmaps() is usually
called earlier (though there shouldn't be a reason to use the EDL URL
for local subs), so I added the order parameter to sub_create(), rather
than adding it to all of update_segment(), sub_read_packets() and
sub_get_bitmaps().
Also removes the cast to bool in the other sub_control call, since
sub/sd_ass.c:control already casts arg to bool when cmd is
SD_CTRL_SET_TOP.
Using --sub-filter-regex-plain (default:no)
The ass-to-plaintext functionality already existed at sd_ass.c, but
it's internal and uses a private buffer type, so a trivial utility
wrapper was added with standard char*/bstr interface.
The plaintext can be multi-line, and the multi-line regexp flag is now
always set, but only affects plaintext (the ASS source is one line).
Pretty much identical to filter-regex but with JS expressions and
requires only JS support. Shares the filter-regex-* control options.
The target audience is Windows users - where filter-regex doesn't
work due to missing APIs, but mujs builds cleanly on Windows, and JS
is usually enabled in 3rd party Windows mpv builds.
Lua could have been used with similar effort, however, the JS regex
syntax is more extensive and also much more similar to POSIX.
1. On a pathological case where event_format is NULL, previously the
filter was trying to use it with each new sub - and re-failed. Now
the filter gets disabled on init (event_format doesn't change).
2. Previously, if the filter didn't modify the text or if the text
could not be extracted - it still allocated a new packet with same
content. Now it returns the original, saving a whole lot of no-ops
(there are still few allocations in this case though).
1 above is preparation for the next commit, and 2 was trivial, but
there's more to do if anyone cares (NIH string functions instead of
bstr, unused arguments, messages could be improved, and more).
Add two stand-alone function to help with the text-extraction task
which ass filters need. Makes it easier to add new filters without
cargo-culting this functionality.
Currently, on malformed event (which shouldn't happen), a warning is
printed when a filter tries to extract the text, so if few filters
are enabled, we'll get multiple warnings (like before) - not critical.
The regex filter now uses these utils, the SDH filter not yet.
This allows users to control whether full dialogue subtitles are displayed
with an audio track already in their preferred subtitle language.
Additionally, this improves handling for the forced flag, automatically
selecting between forced and unforced subtitle streams based on the user's
settings and the selected audio.
Seems like this is requested all the time.
It seems libass allows out of range values, but does allows the subtitle
to go out of the screen at the bottom (only when moving it to the top
it's "clamped"). Too bad, don't do that then. The bitmap sub rendering
code on the other hand is under our control, and will not move a
subtitle out of the screen.
Fixes: #7986
Options like --sub-ass-force-style and others could not be changed at
runtime (the changes didn't take any effect). Fix this by using the
brutal approach, and completely reinit the subtitle state when this
happens. Maybe a bit clunky, but for now I'd rather not put more effort
into this.
Fixes: #7689
libass recently switched the default from 1 to 0 for compatibility
with ASS scripts that rely on the historical/VSFilter default of 0.
libass does attempt to detect and avoid breaking scripts that rely
on the historic libass-only default of 1, but it doesn't cover tracks
created directly through the API, so set the header explicitly.
Fixes https://github.com/mpv-player/mpv/issues/7900.
