Enable ASS_FEATURE_{WHOLE_TEXT_LAYOUT, BIDI_BRACKETS} and auto base
detection by default, and add an option to disable this if needed.
This is strictly an improvement for webvtt files as they always use
auto base detection. This _fixes_ right-to-left text rendering for
webvtt files which correctly mark rtl/ltr. Webvtt files obtained from
sources which sideload the RTL information through css also see an
improvement due to the auto detection.
Generally SRT files also want this, but some are also written to
workaround VSFilter quirks.
See also: https://github.com/mpv-player/mpv/pull/12985#issuecomment-1839565138
0b35b4c917 originally introduced sd_filter
to make a more general subtitle filter infrastructure. But when doing
so, it directly sets ft->event_format to ass_track->event_format in the
struct. The lifetime of ass_track and the sd_filter are not equivalent
which makes it easy to trigger undefined behavior. Notably, commit
cda8f1613f introduced assobjects_destroy
which can destroy ass_track anytime during runtime which means that the
string in ft->event_format is actually freed and should never be used.
Remedy this by simply doing a proper strdup when the filter inits with
ft as the parent so we avoid this scenario altogether. Fixex #13525.
An ass event like `{\p1\pos{1,1}}m 0 0 l -3 -7 l 11 -7 l 11 -2` ends
the drawing mode started with `\p1` due to `\pos` gets confused with
`\p`, and thus that line is wrongly considered to be visible text.
LRC subtitles can have lines with multiple timestamps, e.g.
[00:00.00][00:02.00]foo
[00:01.00]bar
Currently mpv shows only the "foo" that was decoded first, because it
compares the packet file position to check if a packet was already seen,
and it is the same for both occurrences of "foo". Fix this by also
comparing the pts.
This keeps comparing the packet position on top of the pts to not break
subtitle lines with the same timestamp, like:
1
00:00:00,000 --> 00:00:01,000
foo
2
00:00:00,000 --> 00:00:01,000
bar
where mpv shows both lines on top of each other. They are common in ASS
subtitles.
Fixes https://github.com/mpv-player/mpv/issues/13497.
The previous commits optimized sub redrawing on still images/terminal so
mpv wouldn't redraw so much. There is a gap though. It only assumes
static subtitles. Since ASS can be animated, those types of subtitles
will always need redraws so we need to build in specific detection for
this. We need to build a whitelist of events in ASS that are considered
animations and then flag the packet. Additionally, there's a bunch of
annoying bookkeeping that has to be done since packets can be dropped on
seeks and so on.
This only affects two special cases: printing subtitles to the terminal
and printing subtitles on a still picture. Previously, mpv was very dumb
here and spammed this logic on every single loop. For terminal
subtitles, this isn't as big of a deal, but for the image case this is
pretty bad. The entire VO constantly redrew even when there was no need
to which can be very expensive depending on user settings.
Instead, let's rework sub_read_packets so that it also tells us whether
or not the subtitle packets update in some way in addition to telling us
whether or not to read more. Since we cache all packets thanks to the
previous commit, we can leverage this information to make a guess
whether or not the current subtitle packet is supposed to be visible on
the screen. Because the redraw now only happens when it is needed, the
mp_set_timeout_hack can be removed.
f9cefbfec4 made it so mp_ass_flush_old_events() is continously called on
subtitles with unknown duration, without explaining why, breaking
sub-seek/step -1 with a VO (the issue does not occur when showing
subtitles in the terminal because get_bitmaps() is not called). I don't
experience any issue after removing the call, so delete it to fix these
commands.
After removing that, you can sub-seek -1 once after regular playback,
but not after seeking and thus not multiple times in a row. This is
caused by a714f8e928 which fixed subtitles with unknown duration being
duplicated when seeking with a VO (it does not happen in the terminal)
by clearing old lines on seeks, which broke sub-seek -1 and sub-step -1
in a second way after any seek. The proper fix is to remove the line
ctx->num_seen_packets = 0 for subtitles with unknown duration instead,
which lets decode() return early when a line has already been shown.
Having removed these 2 lines, I also removed sd->preload_ok = false, and
thus the whole conditional, since according to sub/sd.h preload_ok only
needs to be set to false when old subtitles are discarded, and they are
no longer discarded,
The bug can be reproduced with
mpv --sub-file=<(curl 'https://music.xianqiao.wang/neteaseapiv2/lyric?id=1399616170' | jq -r .lrc.lyric) 'https://www.youtube.com/watch?v=dQw4w9WgXcQ'
Missed in 3250f6e447. Note that the
hardcoded ass alignment value is not used anymore as of that commit, but
we should ideally be moving towards secondary subs actually being
customizable via ASS anyways.
Over the years, we've accumulated several secondary subtitle related
options and properties, but the implementation was not really consistent
and it wasn't clear what the right process for adding more should be. So
to make things nicer, let's refactor all of the subtitle options with
secondary variants (sub-delay, sub-pos, and sub-visibility) and split
them off to a new, separate struct. All of the underlying values are
stored in an array instead for simplicity. Additionally, the
implementation of some secondary-sub-* properties were slightly changed
so there would be less redundancy.
Stretch a subtitle duration so it ends when the next one starts.
Should help with subtitles which erroneously have zero durations.
I found such a subrip substitles stream in the wild.
Signed-off-by: Mohammad AlSaleh <CE.Mohammad.AlSaleh@gmail.com>
In the sub seek code path, there was an arbitrary small offset added to
the pts before the seek. However when seeking backwards, the offset was
an additional subtraction. de6eace6e9
added this logic 10 years ago and perhaps it made sense then, but the
additional subtraction when seeking backwards causes the subtitle seek
to go too far to the previous subtitle if the durations overlap. This
should always be an addition to work correctly. Additionally, the sub
stepping code path also could use this offset for the same reason
(duration overlaps). However, it is only applicable to sd_ass not
sd_lavc. sd_lavc has step_sub support but on a sample it didn't even
work anyway. Perhaps it only works for certain kinds of subtitles
(patches welcome).
Anyways instead of keeping this offset as a magic number, we can define
it in sd.h which is handy for this. For sd_ass, we add the offset when
sub stepping, and the offset is always added for sub seeking like it was
before. Update the comment to be a little more relevant to what actually
happens today. Fixes#11445.
UPDATE_SUB_HARD causes all of the ass objects to reset in order to apply
the new style. UPDATE_SUB_FILT doesn't actually reset the sd, but it
should in order to update the actual filters so that was added here.
Doing this causes the current subtitle to be dropped. In the paused
cause, this concidentally works because command.c forces a video refresh
which then reloads the subtitle essentially. But while playing, the
subtitle will be dropped and you won't get anything until the next one
appears.
Instead of using video refreshes, what we can do is just always save the
last two subtitle packets in a cache and redecode them if needed. This
is much easier and also allows us to get rid of all the video refresh
logic in command.c. Fixes#12386.
This option has exactly the same semantics are other mpv options that
override a particular thing with something from the user. So instead of
the "force-style" name, use "-overrides" which is more consistent.
The plural form is used since it's a list option.
The function called to get sub-start and sub-end returns early when the
subtitle's duration is unknown, but by just removing this check the
properties work fine. The final sub line has a very large sub-end, but
that is much better than not having the properties work at all.
Third try is the charm? I stupidly missed that this option already
existed in my previous commits. Instead, add an auto value to it and
enable it by default for sd_lavc but not sd_ass. On my limited samples,
it seems to fix the gaps issue that can occur but without regressing
some duration timings for sub_lavc subtitles. Well hopefully anyway.
Fixes#12327.
The old name is pretty bad and users mistakenly think it has something
to do with selecting forced subtitles (that would be
--subs-fallback-forced). Instead of giving it such a generic name, make
it clearer that this has to do specifically with forced sub events
which is only relevant for a small minority of subtitles.
First of all, this never worked. Or if it ever did, it was in some
select few scenarios. c9474dc9ed is what
originally added support for the auto choice. However, that commit
worked by propagating a value to a fake option used internally. This
shouldn't have ever worked because the underlying m_config_cache was
never updated so the value shouldn't have been preserved when accessed
in sd_lavc. And indeed with some testing, the value there is always 0
unsurprisingly.
This was later rewritten in ba7cc07106
along with a lot of other sub changes, but with that, it was still
mostly broken. The reason is because one of the key parts of having to
hit this logic (prefer_forced) required `--no-subs-with-matching-audio`
to be set. If the audio language matches the subtitle language (the
requirement also excludes forced subs), the option makes no subtitle
selection in the first place so pick->forced_only_def is not set to true
and nothing even happens. Another way around this would be to attempt to
change your OS language (like with the LANG environment variable) so
that the subtitle track gets selected but then audio_matches mistakenly
becomes false because it compares the OS language to the audio language
which then make preferred_forced 0, so nothing happens. I don't think
there's a scenario where pick->forced_only_def is actually set to true
(thus meaning `auto` is useless), but maybe someone could contrive
something very strange. Regardless, it's definitely not something even
remotely common.
fbe8f99194 changed track selection again
but didn't consider this particular case. The net result is that DVD/PGS
subs become equivalent to --sub-forced-only being yes, so this a change
in behavior and probably not a good one. Note that I wasn't able to
actually observe any difference in a PGS sample. It still displayed
subtitles fine but that sample probably didn't have the right flags to
hit the sub-forced-only logic.
Anyways, the auto feature is extremely questionable at best and in my
view, not actually worth it. It is meant to be used with
`--no-subs-with-matching-audio` to display forced pictures in subtitle
tracks that are not marked as forced, but that contradicts that
particular option's purpose and description in the manual (secretly
selecting a track under certain conditions even though it says not to).
Instead of trying to shove all this logic into select_default_track
which is already insanely complicated as it is, recognize that this is a
trivial lua script. If you absolutely want to turn --sub-forced-only on
under these certain conditions (DVD/PGS subtitles, matching audio and
subtitle languages, etc.), just look at the current-tracks property and
do your thing. The very, very niche behavior that this option tried to
accomplish basically never worked, no user even knows what this option
does, and well it's just not worth supporting in core mpv code. Drop
all this code for sanity's sake and change --sub-forced-only back to a
bool.
mpv makes this option an integer, but the underlying ass API actually
accepts doubles. From some testing, there is no meaningful precision
difference between float or double (it seems to go in roughly 0.05
steps), so just make it a float. sd_lavc also can handle non-integer
values here. Closes#11583.
Libass commit f08f8ea5 (between 0.16 and 0.17) changed how PlayResX
affects some aspects of rendering.
The libass change fixes a VSFilter compatibility issue which existed
for about two decades, and there are no libass plans to support the
previous behavior, so ultimately we have to adjust the mpv code, and
we can't guarantee to restore the old behavior in all cases.
Starting at this commit, vector drawing coords, font spacing, border
and shadow widths are all affected by PlayResX (specifically, by
the aspect), while previously they were unaffected by PlayResX.
This changed converted sub border and shadow widths in mpv, because
ffmpeg generates the ass with fixed PlayResX of 384 (aspect of 4:3),
and with libass 0.17, if this doesn't match the display aspect, then
borders and shadow were too wide - because most clips aspect is more
than 4:3.
The fact that ffmpeg uses fixed PlayResX of 384 could be considered
an issue, but for now we have no control over it, and ffmpeg doesn't
have the video resolution when it converts an srt source to ass.
So here we adjust PlayResX accordingly so that border/shadows are
now rendered with correct width.
However, regardless of that commit, changing PlayResX also affects
the margin value, so to compensate, we adjust sub-margins-x too.
According to libass devs, this should cover basic srt-to-ass
conversion by ffmpeg to work correctly with libass 0.17.
However, there could be srt extensions which use more complex ass,
and/or ffmpeg conversion of other sub formats (such as aribb24,
aribcaption and movtext), where more things need adjustments.
As of now we don't know what these are, and so we don't really know
what else might remain broken or get broken.
dbc5d7b7db seems to have originally
introduced this behavior. At the time, wm4 simply reconfigured ass on
every frame in order to accommodate runtime changes in sub options. This
certainly works, but these libass API calls are not free and there is at
least one known performance regression due to a change in libass*.
Regardless of whether or not the libass change is good/bad, there is no
need for mpv to constantly reconfigure this. When wm4 made that commit,
there was no notification mechanism for options changing that could
easily be used so he didn't really have any other choice. But it's
nearly 10 years later now and internally we have all the necessary
pieces to only configure ass again when we need to: on option changes or
resizes. So go ahead and implement that in this commit which simply uses
the already existing SD_CTRL_UPDATE_OPTS and compares osd_res sizes to
determine whether or not an ass configure is needed.
*: https://github.com/libass/libass/issues/698
Upon an option update with an UPDATE_SUB_HARD flag,
the ass_track that stores all the decoded
subtitle packets/events is destroyed and recreated, which means
the packets need to be read and decoded again to refill
the ass_track. This caused issues (no subs displayed) in 2 cases:
1. external sub files
Previously, external sub files were read and decoded only
once when loaded. Since this meant all decoded events were lost
forever when recreating the ass_track, we need to change this
and trigger a new preload during sub reinits.
2. converted subs (non-ASS text subs like srt)
For converted subs, we maintain a list of previously
seen packets to avoid decoding and adding duplicate events
to the ass_track. Previously this list wasn’t synchronized with
the corresponding ass_track, so the sub decoder would reject
any previously seen sub packets, usually meaning only subs sometime
after the current pts would be displayed after sub reinits.
Fix this by resetting the list upon ass_track recreation.
This way we receive such minor details as the profile (necessary for
ARIB captions, among others) during init. This enables decoders
to switch between ARIB caption profile A and profile C streams.
These options make it possible to specify the directory that will be
passed to ass_set_fonts_dir(), akin to VLC's `--ssa-fontsdir` and
FFmpeg's `fontsdir`.
Fixes#8338
It turns out, even xy-VSFilter and XySubFilter do not
mangle colours if the video is native RGB regardless of
the sub's YCbCr header. libass' docs were also updated
to reflect this.
Commit 740b701 introduced handling for subtitles with unknown duration,
but it came with a minor flaw where a track event that shares identical
start time with following track event will has its Duration value set
to 0, we don't want this to happen since it will prevent simultaneous
rendering of multiple track events.
This commit aims to address this issue.
ASS must only automatically break at ASCII spaces (\x20), but other
subtitle formats might expect more breaking oppurtinities.
Especially non-ASS subs in scripts, which typically do not use (ASCII)
spaces to seperate words, like e.g. CJK, might overflow the screen
if the conversion didn't insert additional linebreaks (ffmpeg does not).
Thus try to enable Unicode linebreaking for converted subs and the OSD
if libass is new enough. The feature may still be unavailable at runtime
if libass wasn't build with Unicode linebreaking support.
Using --sub-filter-regex-plain (default:no)
The ass-to-plaintext functionality already existed at sd_ass.c, but
it's internal and uses a private buffer type, so a trivial utility
wrapper was added with standard char*/bstr interface.
The plaintext can be multi-line, and the multi-line regexp flag is now
always set, but only affects plaintext (the ASS source is one line).
Pretty much identical to filter-regex but with JS expressions and
requires only JS support. Shares the filter-regex-* control options.
The target audience is Windows users - where filter-regex doesn't
work due to missing APIs, but mujs builds cleanly on Windows, and JS
is usually enabled in 3rd party Windows mpv builds.
Lua could have been used with similar effort, however, the JS regex
syntax is more extensive and also much more similar to POSIX.
Add two stand-alone function to help with the text-extraction task
which ass filters need. Makes it easier to add new filters without
cargo-culting this functionality.
Currently, on malformed event (which shouldn't happen), a warning is
printed when a filter tries to extract the text, so if few filters
are enabled, we'll get multiple warnings (like before) - not critical.
The regex filter now uses these utils, the SDH filter not yet.
This allows users to control whether full dialogue subtitles are displayed
with an audio track already in their preferred subtitle language.
Additionally, this improves handling for the forced flag, automatically
selecting between forced and unforced subtitle streams based on the user's
settings and the selected audio.
Options like --sub-ass-force-style and others could not be changed at
runtime (the changes didn't take any effect). Fix this by using the
brutal approach, and completely reinit the subtitle state when this
happens. Maybe a bit clunky, but for now I'd rather not put more effort
into this.
Fixes: #7689
See manpage additions. This was requested, sort of. Although what has
been requested might be something completely different. So this is
speculative.
This also changes sub_get_text() to return an allocated copy, because
the buffer shit was too damn messy.
Making OSD/subtitle bitmaps refcounted was planend a longer time ago,
e.g. the sub_bitmaps.packed field (which refcounts the subtitle bitmap
data) was added in 2016. But nothing benefited much from it, because
struct sub_bitmaps was usually stack allocated, and there was this weird
callback stuff through osd_draw().
Make it possible to get actually refcounted subtitle bitmaps on the OSD
API level. For this, we just copy all subtitle data other than the
bitmaps with sub_bitmaps_copy(). At first, I had planned some fancy
refcount shit, but when that was a big mess and hard to debug and just
boiled to emulating malloc(), I made it a full allocation+copy. This
affects mostly the parts array. With crazy ASS subtitles, this parts
array can get pretty big (thousands of elements or more), in which case
the extra alloc/copy could become performance relevant. But then again
this is just pure bullshit, and I see no need to care. In practice, this
extra work most likely gets drowned out by libass murdering a single
core (while mpv is waiting for it) anyway. So fuck it.
I just wanted this so draw_bmp.c requires only a single call to render
everything. VOs also can benefit from this, because the weird callback
shit isn't necessary anymore (simpler code), but I haven't done anything
about it yet. In general I'd hope this will work towards simplifying the
OSD layer, which is prerequisite for making actual further improvements.
I haven't tested some cases such as the "overlay-add" command. Maybe it
crashes now? Who knows, who cares.
In addition, it might be worthwhile to reduce the code duplication
between all the things that output subtitle bitmaps (with repacking,
image allocation, etc.), but that's orthogonal.
Works as ad-filter. I had some more plans, for example replacing
matching text with different text, but for now it's dropping matches
only. There's a big warning in the manpage that I might change
semantics. For example, I might turn it into a primitive sed.
In a sane world, you'd probably write a simple script that processes
downloaded subtitles before giving them to mpv, and avoid all this
complexity. But we don't live in a sane world, and the sooner you learn
this, the happier you will be. (But I also want to run this on muxed
subtitles.)
This is pretty straightforward. We use POSIX regexes, which are readily
available without additional pain or dependencies. This also means it's
(apparently) not available on win32 (MinGW). The regex list is because I
hate big monolithic regexes, and this makes it slightly better.
Very superficially tested.
llyyr
Kacper Michajłow
Dudemanguy
Christoph Heinrich
Guido Cella
dyphire
karelrooted
Mohammad AlSaleh
rcombs
Lypheo
Jan Ekström
Frédéric Brière
Oneric
VincentVerdynanta
Avi Halachmi (:avih)
wm4
Wessel Dankers