Simple enough to do. May have mixed results. Typically, bitmap subtitles
will have a tight bounding box around the rendered text. But if for
example there is text on the top and bottom, it may be a single big
bitmap with a large transparent area between top and bottom. In
particular, DVD subtitles are really just a single screen-sized
RLE-encoded bitmap, though libavcodec will crop off transparent areas.
Like with sd_ass, you can't move subtitles _down_ if they are already in
their origin position. This could probably be improved, but I don't want
to deal with that right now.
See manpage additions. This is a huge hack. You can bet there are shit
tons of bugs. It's literally forcing square pegs into round holes.
Hopefully, the manpage wall of text makes it clear enough that the whole
shit can easily crash and burn. (Although it shouldn't literally crash.
That would be a bug. It possibly _could_ start a fire by entering some
sort of endless loop, not a literal one, just something where it tries
to do work without making progress.)
(Some obvious bugs I simply ignored for this initial version, but
there's a number of potential bugs I can't even imagine. Normal playback
should remain completely unaffected, though.)
How this works is also described in the manpage. Basically, we demux in
reverse, then we decode in reverse, then we render in reverse.
The decoding part is the simplest: just reorder the decoder output. This
weirdly integrates with the timeline/ordered chapter code, which also
has special requirements on feeding the packets to the decoder in a
non-straightforward way (it doesn't conflict, although a bugmessmass
breaks correct slicing of segments, so EDL/ordered chapter playback is
broken in backward direction).
Backward demuxing is pretty involved. In theory, it could be much
easier: simply iterating the usual demuxer output backward. But this
just doesn't fit into our code, so there's a cthulhu nightmare of shit.
To be specific, each stream (audio, video) is reversed separately. At
least this means we can do backward playback within cached content (for
example, you could play backwards in a live stream; on that note, it
disables prefetching, which would lead to losing new live video, but
this could be avoided).
The fuckmess also meant that I didn't bother trying to support
subtitles. Subtitles are a problem because they're "sparse" streams.
They need to be "passively" demuxed: you don't try to read a subtitle
packet, you demux audio and video, and then look whether there was a
subtitle packet. This means to get subtitles for a time range, you need
to know that you demuxed video and audio over this range, which becomes
pretty messy when you demux audio and video backwards separately.
Backward display is the most weird (and potentially buggy) part. To
avoid that we need to touch a LOT of timing code, we negate all
timestamps. The basic idea is that due to the navigation, all
comparisons and subtractions of timestamps keep working, and you don't
need to touch every single of them to "reverse" them.
E.g.:
bool before = pts_a < pts_b;
would need to be:
bool before = forward
? pts_a < pts_b
: pts_a > pts_b;
or:
bool before = pts_a * dir < pts_b * dir;
or if you, as it's implemented now, just do this after decoding:
pts_a *= dir;
pts_b *= dir;
and then in the normal timing/renderer code:
bool before = pts_a < pts_b;
Consequently, we don't need many changes in the latter code. But some
assumptions inhererently true for forward playback may have been broken
anyway. What is mainly needed is fixing places where values are passed
between positive and negative "domains". For example, seeking and
timestamp user display always uses positive timestamps. The main mess is
that it's not obvious which domain a given variable should or does use.
Well, in my tests with a single file, it suddenly started to work when I
did this. I'm honestly surprised that it did, and that I didn't have to
change a single line in the timing code past decoder (just something
minor to make external/cached text subtitles display). I committed it
immediately while avoiding thinking about it. But there really likely
are subtle problems of all sorts.
As far as I'm aware, gstreamer also supports backward playback. When I
looked at this years ago, I couldn't find a way to actually try this,
and I didn't revisit it now. Back then I also read talk slides from the
person who implemented it, and I'm not sure if and which ideas I might
have taken from it. It's possible that the timestamp reversal is
inspired by it, but I didn't check. (I think it claimed that it could
avoid large changes by changing a sign?)
VapourSynth has some sort of reverse function, which provides a backward
view on a video. The function itself is trivial to implement, as
VapourSynth aims to provide random access to video by frame numbers (so
you just request decreasing frame numbers). From what I remember, it
wasn't exactly fluid, but it worked. It's implemented by creating an
index, and seeking to the target on demand, and a bunch of caching. mpv
could use it, but it would either require using VapourSynth as demuxer
and decoder for everything, or replacing the current file every time
something is supposed to be played backwards.
FFmpeg's libavfilter has reversal filters for audio and video. These
require buffering the entire media data of the file, and don't really
fit into mpv's architecture. It could be used by playing a libavfilter
graph that also demuxes, but that's like VapourSynth but worse.
There are 3 packet reading functions in the demux API, which all
function completely differently. One of them, demux_read_packet(), has
only 1 caller, which is in dec_sub.c. Change this caller to use
demux_read_packet_async() instead. Since it really wants to do a
blocking call, setup some proper waiting. This uses mp_dispatch_queue,
because even though it's overkill, it needs the least code.
In practice, waiting actually never happens. This code is only called on
code paths where everything is already read into memory (libavformat's
subtitle demuxers simply behave this way). It's still a bit of a
"coincidence", so implement it properly anyway.
If suubtitle decoder init fails, we still need to unset the demuxer
wakeup callback. Add a sub_destroy() call to the failure path. This also
happens to fix a missed pthread_mutex_destroy() call (in practice this
was a nop, or a memory leak on BSDs).
UB-sanitizer complains that we shift bits into the sign (when a is
used). Change it to unsigned, which in theory is more correct and
silences the warning.
Doesn't matter in practice, both the "bug" and the fix have 0 impact.
It would always autodetect it based on the passed style block,
but as we are defining it - we might as well define it always.
(As far as I can see all decoders in libavcodec utilize 4+ style
blocks)
Manual changes done:
* Merged the interface-changes under the already master'd changes.
* Moved the hwdec-related option changes to video/decode/vd_lavc.c.
Only printable ASCII characters were considered to be valid texts. Make
it possible that UTF-8 contents are also considered valid.
This does not make the SDH subtitle filter support non-English
languages. This just prevents the filter from blindly marking lines that
have only UTF-8 characters as empty.
Fixes#6502
libavcodec normally drops subtitle lines that fail a check for invalid
UTF-8 (their check is slightly broken too, by the way). This was always
annoying and inconvenient, but now there is a mechanism to prevent
it from doing this. Requires newst libavcodec.
Remove them from the big MPOpts struct and move them to their sub
structs. In the places where their fields are used, create a private
copy of the structs, instead of accessing the semi-deprecated global
option struct instance (mpv_global.opts) directly.
This actually makes accessing these options finally thread-safe. They
weren't even if they should have for years. (Including some potential
for undefined behavior when e.g. the OSD font was changed at runtime.)
This is mostly transparent. All options get moved around, but most users
of the options just need to access a different struct (changing sd.opts
to a different type changes a lot of uses, for example).
One thing which has to be considered and could cause potential
regressions is that the new option copies must be explicitly updated.
sub_update_opts() takes care of this for example.
Another thing is that writing to the option structs manually won't work,
because the changes won't be propagated to other copies. Apparently the
only affected case is the implementation of the sub-step command, which
tries to change sub_delay. Handle this one explicitly (osd_changed()
doesn't need to be called anymore, because changing the option triggers
UPDATE_OSD, and updates the OSD as a consequence). The way the option
value is propagated is rather hacky, but for now this will do.
It was split at least across osd.c and sd_ass.c/sd_lavc.c. sd_lavc.c
actually ignored most of the more obscure subtitle timing things.
There's no reason for this - just move it all to dec_sub.c (mostly from
sd_ass.c, because it has some of the most complex stuff).
Now timestamps are transformed as they enter or leave dec_sub.c.
There appear to have been some subtle mismatches about how subtitle
timestamps were transformed, e.g. sd_functions.accepts_packet didn't
apply the subtitle speed to the timestamp. This patch should fix them,
although it's not clear if they caused actual misbehavior.
The semantics of SD_CTRL_SUB_STEP are slightly changed, which is the
reason for the changes in command.c and sd_lavc.c.
I've decided that MP_TRACE means “noisy spam per frame”, whereas
MP_DBG just means “more verbose debugging messages than MSGL_V”.
Basically, MSGL_DBG shouldn't create spam per frame like it currently
does, and MSGL_V should make sense to the end-user and provide mostly
additional informational output.
MP_DBG is basically what I want to make the new default for --log-file,
so the cut-off point for MP_DBG is if we probably want to know if for
debugging purposes but the user most likely doesn't care about on the
terminal.
Also, the debug callbacks for libass and ffmpeg got bumped in their
verbosity levels slightly, because being external components they're a
bit less relevant to mpv debugging, and a bit too over-eager in what
they consider to be relevant information.
I exclusively used the "try it on my machine and remove messages from
MSGL_* until it does what I want it to" approach of refactoring, so
YMMV.
The OpenType Font File specification recommends that "Collection fonts
that use CFF or CFF2 outlines should have an .OTC extension." mpv
should accept .otc as a fallback extension for font detection should
the mimetype detection fail.
IETF RFC8081 added the "font" top-level media type,
including font/ttf, font/otf, font/sfnt, and also
font/collection. These font formats are all supported
by mpv/libass but they are not accepted as valid
Matroska mime types. mpv can load them via file extension
and they work as expected, so files using the new types
should not trigger a warning from mpv.
The current invocation of bstr_cut is as good as no cutting at all.
Almost the entire header is reread in every iteration of the loop.
I don't know how many styles libavcodec tends to generate, but if
(now or in the future) it generates many, then this loop is slow
for no good reason. If anything, the code would be more clear and
have the same performance if it didn't call bstr_cut at all.
The intention here (and the sensible thing regardless) seems to be
to skip the part of the string that bstr_find has already looked
through and found nothing. This commit additionally skips the whole
substring, because overlapping matches are impossible.
In some cases, demux_mkv will detect a start time slightly above 0, but
there might still be a subtitle starting at exactly 0. When the player
rebases the timestamps to assumed start time, the subtitle will have a
slightly negative timestamp in the end. libavcodec's subtitle converter
turns this into a larger number due to underflow. Fix by clamping
subtitles always to 0, which may or may not be what you want.
At least it fixes#5047.
The new_segment field was used to track the decoder data flow handler of
timeline boundaries, which are used for ordered chapters etc. (anything
that sets demuxer_desc.load_timeline). This broke seeking with the
demuxer cache enabled. The demuxer is expected to set the new_segment
field after every seek or segment boundary switch, so the cached packets
basically contained incorrect values for this, and the decoders were not
initialized correctly.
Fix this by getting rid of the flag completely. Let the decoders instead
compare the segment information by content, which is hopefully enough.
(In theory, two segments with same information could perhaps appear in
broken-ish corner cases, or in an attempt to simulate looping, and such.
I preferred the simple solution over others, such as generating unique
and stable segment IDs.)
We still add a "segmented" field to make it explicit whether segments
are used, instead of doing something silly like testing arbitrary other
segment fields for validity.
Cached seeking with timeline stuff is still slightly broken even with
this commit: the seek logic is not aware of the overlap that segments
can have, and the timestamp clamping that needs to be performed in
theory to account for the fact that a packet might contain a frame that
is always clipped off by segment handling. This can be fixed later.
If a VO-area option changes, gl_video_resize() is called
unconditionally. This function does something even if the size does not
change (at least it discards buffered frames for interpolation), which
can lead to stutter when you keep firing option change events during
playback.
Check for an actual resize, and if nothing changes, exit early.
Lua scripts can call osd_set_external() early (before the VO window is
created and obj->vo_res is filled), in which case the PlayResX field
would be set to nonsense, and libass would print a pointless warning.
There's an easy and a hard fix: either just go on and pass dummy values
to libass (basically like before, just clamp them to avoid the values
which make libass print the warning). Or attempt to update the PlayRes
field to correct values on rendering (since at rendering time, you
always know the screen size and the correct values). Do the latter.
Since various things use PlayRes for scaling things, this might still
not be fully ideal. This is a general problem with the async scripting
interface.
This API isn't deprecated (yet?), but it's still inferior and harder to
use than avcodec_free_context().
Leave the call only in 1 case in af_lavcac3enc.c, where we apparently
seriously close and reopen the encoder for whatever reason.
List of changes:
1. Rename `signfs` to `scale`, to better match what it actually does
(force --sub-scale to apply to ASS subtitles), and fix the blatantly
wrong documentation (it actually specifically does *not* apply to
signs)
2. Rename `--sub-ass-style-override` to `--sub-ass-override` to help
reduce confusion between it and `--sub-ass-force-style`, as well as
pointing out that it doesn't necessarily actually override styles.
(The new `scale` option, for example, only sets
ASS_OVERRIDE_BIT_FONT_SIZE, but not ASS_OVERRIDE_BIT_STYLE)
3. Mention that `--sub-ass-override` is generally sort of smart about
only overriding dialog, not signs.
All contributors of the code used for these files agreed to the LGPL
relicensing.
There are some unaccounted contributors, but all of their code was
completely removed before. (The only exception is one contributor whose
only line left was "#include <string.h>". I don't know if that's
copyrightable, but it wasn't needed anyway, so just remove it.)
These files started out as libvo/sub.* (renamed to sub/sub.*, then
renamed again to sub/osd.*). They used to contain code for rendering
the OSD (as in, actual pixel manipulation and text layouting). But
later all this code was dropped, and libass was used to render the OSD
instead. Actual subtitle rendering was reimplemented in other files
(the old subtitle rendering path is completely gone).
One potential problem are the option declarations, which makes this
harder, as these options involve more history. But it turns out most of
them were reimplemented since 80270218cb, rather than taken from old
code. (Although not all - but the rest covered by relicensing
agreements.)
This also affects osd_state.h, which was apparently incorrectly implied
to be LGPL.
All contributors have agreed.
Compared to sd_ass.c, this has a pretty simple history:
av_sub.c -> sub/av_sub.c -> sub/sd_lavc.c
At one point, some code from spudec.c was added to it, but it was
removed again later.
All contributors of the code used for sd_ass.c agreed to the LGPL
relicensing. Some code has a very chaotic history, due to MPlayer
subtitle handling being awful, chaotic, and having been refactored a
dozen of times. Most of the subtitle code was actually rewritten from
scratch (a few times), and the initial sd_ass.c was pretty tiny. So we
should be fine, but it's still a good idea to look at this closely.
Potentially problematic cases of old code leaking into sd_ass.c are
mentioned below.
Some code originates from demux_mkv. Most of this was added by eugeni,
and later moved into mplayer.c or mpcommon.c. The old demux_mkv ASS/SSA
subtitle code is somewhat dangerous from a legal perspective, because it
involves 2 patches by a certain Tristan/z80, who disagreed with LGPL,
and who told us to "rewrite" parts we need. These patches were for
converting the ASS packet data to the old MPlayer text subtitle data
structures. None of that survived in the current code base.
Moving the subtitle handling out of demux_mkv happened in the following
commits: bdb6a07d2a, de73d4dd97, 61e4a80191. The code by
z80 was removed in b44202b69f.
At this time, the z80 code was located in mplayer.c and subreader.c.
The code was fully removed, being unnecessary due to the entire old
subtitle rendering code being removed. This adds a ass_to_plaintext(),
function, which replaces the old ASS tag stripping code in
sub_add_text(), which was based on the z80 code. The new function was
intended to strip ASS tags in a correct way, instead of somehow
dealing with other subtitle types (like HTML-style SRT tags), so it
was written from scratch.
Another potential issue is the --sub-fix-timing option is based on
-overlapsub added in d459e64463. But the implementation is new, and
no code from that commit is used in sd_ass.c. The new implementation
started out in 64b1374a44. (The following commit, bd45eb468c
removes the original code that was replaced.) The code was later
moved into sd_ass.c.
The --sub-fps option has a similar history.
Somewhat chaostic history: libass/ass_mp.* -> ass_mp.* -> sub/ass_mp.*
As far as I can tell, everyone who ever touched these files has agreed
to the relicensing.
When the format of the subtitle bitmaps changes, such as with taking
screenshots with vo_vaapi (RGBA for the VO vs. Y8 for screenshots), the
cache image obviously needs to be recreated.
Fixes#4325.
When doing harder filtering not require a space after : results
in lines with a clock (like 10:05) to be taken as a speaker label.
So require a space after : even when doing harder filtering as
missing space is very uncommon.
Some like to add text in parentheses in the speaker label,
like XXX (loud): or just (loud):
allow parentheses when doing harder filtering
Add subtitle filter to remove additions for deaf or hard-of-hearing
(SDH). This is for English, but may in part work for others too.
This is an ASS filter and the intention is that it can always be
enabled as it by default do not remove parts that may be normal text.
Harder filtering can be enabled with an additional option.
Signed-off-by: wm4 <wm4@nowhere>
This means the subtitles will show as "intended".
For some weird reason, --sub-ass-style-override is the option that
controls style override, which implies it's specific to ASS. While that
seems weird and doesn't always reflect reality, I don't care about that
now.
To make it easier for the eyes, multi line subtitles should
be left justified (for most languages).
This adds an option to define how subtitles are to be justified
inpendently of how they are aligned.
Also add option to enable --sub-justify to be applied on ASS subtitles.
A hacky, convoluted, half-working mess that attempts to cut off overlong
playlists.
It does so by relying on the ASS formatting rule that the font size is
specified in the virtual PlayResY resolution. This means we can
(normally) easily tell how many lines fit on the screen. On the other
hand, this does not work if the text is wrapped.
This as a kludge until a Better™ solution is available.
This core of this heuristic was once copied from MPlayer's spudec.c. I
think it was meant for the case when the resolution field was missing or
so.
I couldn't find a file for which this actually does something. On the
other hand, there are samples which actually have a smaller resolution
than 720x576, and which are broken by this old hack.
For subtitles that set no resolution (I'm not sure which codec/container
that would be), there's still the fallback on video resolution.
Just get rid of this hack. Also cleanup a bit. SD_CTRL_GET_RESOLUTION
hasn't been used since DVD menu removal. get_resolution() is left with 1
call site, and would be quite awkward to keep, so un-inline it.
The previous commit means subtitles were reinitialized on every seek
(even within a segment). This commit restores the old behavior.
To check whether the segment changed at all, we don't reset the current
start/end values. This assumes the decoder wrapper is always fed by a
stream which doesn't mix segment and non-segment packets, which is
currently always true.
If a VO is created, but no video is playing (i.e. --force-window is
used), then until now no subtitles were shown. This is because VO
subtitle display normally depends on video frame timing. If there are no
video frames, there can be no subtitles.
Change this and add some code to handle this situation specifically. Set
a subtitle PTS manually and request VO redrawing manually, which gets
the subtitles rendered somehow.
This is kind of shaky. The subtitles are essentially sampled at
arbitrary times (such as when new audio data is decoded and pushed to
the AO, or on user interaction). To make a it slightly more consistent,
force a completely arbitrary minimum FPS of 10.
Other solutions (such as creating fake video) would be more intrusive or
would require VO-level API changes.
Fixes#3684.
Rename the text subtitle options from --sub-text- to --sub-
and --ass- options to --sub-ass-.
The intention is to common sub options to prefixed --sub-
and special ASS option be seen as a special version of sub options.
The OSD options that work like the --sub- options are still named
--osd-.
Man page updated including a short note about renamed --sub-text-*
and --ass-* options to --sub-* and --sub-ass-*.
Secondary subtitle streams (to be shown on the top of the screen along
main subtitle stream) were shown with normal alignment. This is because
we tell libass to override the alignment style (a relatively recent
change, see commit 2f1eb49e). This would behave differently with old
libass versions too.
To escape the mess, just set the alignment explicitly with an override
tag instead of modifying the style.
Remove wrapper properties for OSD and video position updates, use the
new mechanism for them. We can mark the options directly. Update
behavior will work for more options (since I've casually marked more
affected options than the old less direct mechanism covered).
If --blend-subtitles=yes is given, vo_opengl will call osd_draw()
multiple times, once for subtitles, and once for OSD. This meant that
the want_redraw flag was reset before the OSD was rendered, which in
turn meant that update_osd() was never called. It seems like removing
the per-OSD object want_redraw wasn't such a good idea. Fix it by
reintroducing such a flag for OSDTYPE_OSD only.
Also, the want_redraw flag is now unused, so kill it.
Another regression caused by commit 9c9cf125. Fixes#3535.
This could in theory lead to missed updates if subtitles were switched
or external OSD overlays (via overlay-add) were updated. While the
change IDs of each of those were consistent, switching between two
separate OSD sources is not, and we have to explicitly trigger a change.
Regression since commit 9c9cf125. The new code is actually better,
because we do exactly what is needed, and don't just mess with the
update ID for libass-based OSD.
Remove the per-part force_redraw flags, and instead make the difference
between flagging dirty state and returning it to the player frontend
more explicit. The big issue is that 1. the OSD needs to know the dirty
state, and it should be cleared strictly when it is re-rendered
(force_redraw flag), and 2. the player core needs to be notified once,
and the notification must be reset (want_redraw flag).
The call in loadfile.c is replaced by making osd_set_sub() set the
change flag. Increasing the change flag on dirty state (the force_redraw
check in render_object()) should not be needed, because OSD part
renderers set it correctly (at least now).
Doing this just because someone pointed this out.
This is a bug fix, and the text alignment functionality probably got
lost sometime along the way.
For ASS subtitles, this could have unintended consequences, so it's hard
to get right - thus it's not applied to ASS subtitles.
For other text subtitles, this should be fine, though. It still works on
ASS subtitles as promised by the manpage if --no-sub-ass is used.
Whitelisting supported codecs is (probably) still better than just
allowing everything, given the weird FFmpeg API. I'm also assuming
Libav doesn't even have the codec ID, but I didn't check.
Also add a --teletext-page option, since otherwise it decodes every
teletext page and shows them in succession.
And yes, we can't use av_opt_set_int() - instead we have to set it as
string. Because FFmpeg's option system is terrible.
Like it's done for audio and video. Just to be uniform.
I'm sorry for deleting the anti-ffmpeg vitriol. It's still all true, but
since we decided to always set the timebase, the crappiness is isolated
to FFmpeg internals.
The accepts_packet packet callback is supposed to deal with subtitle
decoders which have only a small queue of current subtitle events (i.e.
sd_lavc.c), in case feeding it too many packets would discard events
that are still needed.
Normally, the number of subtitles that need to be preserved is estimated
by the rendering pts (get_bitmaps() argument). Rendering lags behind
decoding, so normally the rendering pts is smaller than the next video
frame pts, and we simply discard all subtitle events until the rendering
pts.
This breaks down in some annoying corner cases. One of them is seeking
backwards: the VO will still try to render the old PTS during seeks,
which passes a high PTS to the subtitle renderer, which in turn would
discard more subtitles than it should. There is a similar issue with
forward seeks. Add hacks to deal with those issues.
There should be a better way to deal with the essentially unknown
"rendering position", which is made worse by screenshots or rendering
with vf_sub. At the very least, we could handle seeks better, and e.g.
either force the VO not to re-render subs after seeks (ugly), or
introduce seek sequence numbers to distinguish attempts to render
earlier subtitles when a seek is done.
This has two reasons:
1. I tend to add new fields to this metadata, and every time I've done
so I've consistently forgotten to update all of the dozens of places in
which this colorimetry metadata might end up getting used. While most
usages don't really care about most of the metadata, sometimes the
intend was simply to “copy” the colorimetry metadata from one struct to
another. With this being inside a substruct, those lines of code can now
simply read a.color = b.color without having to care about added or
removed fields.
2. It makes the type definitions nicer for upcoming refactors.
In going through all of the usages, I also expanded a few where I felt
that omitting the “young” fields was a bug.
vo_vaapi is the only thing which can't scale RGBA on the GPU. (Other
cases of RGBA scaling are handled in draw_bmp.c for some reason.)
Move this code and get rid of the osd_conv_cache thing.
Functionally, nothing changes.
This affects VOs (or other code which render OSD) which does not support
the LIBASS format, but only RGBA. Instead of having a converter stage in
osd.c, make mp_ass_packer_pack() output directly in RGBA.
In general, this is work towards refcounted subtitle images.
Although we could keep the "converter" design, doing it this way seems
simpler, at least considering the current situation with only 2 OSD
formats. It also prevents copying & packing the data twice, which will
lead to better performance. (Although I guess this case is not important
at all.)
It also fixes --force-rgba-osd-rendering when used with vo_opengl,
vo_vdpau, and vo_direct3d.
The intention is to let mp_ass_packer_pack() produce different output
for the RGBA and LIBASS formats. VOs (or whatever generates the OSD)
currently do not signal a preferred format, and this mechanism just
exists to switch between RGBA and LIBASS formats correctly, preferring
LIBASS if the VO supports it.
Change all producer of libass images to packing the bitmaps into a
single larger bitmap directly when they're output. This is supposed to
help working towards refcounted sub bitmaps.
This will reduce performance for VOs like vo_xv, but not for vo_opengl.
vo_opengl simply will pick up the pre-packed sub bitmaps, and skip
packing them again. vo_xv will copy and pack the sub bitmaps
unnecessarily - but if we want sub bitmap refcounting, they'd have to be
copied anyway.
The packing code cannot be removed yet from vo_opengl, because there are
certain corner cases that still produce unpackad other sub bitmaps.
Actual refcounting will also require more work.
Since there are not many sub-rectangles, this doesn't cost too much. On
the other hand, it avoids frequent warnings with vo_xv.
Also, the second copy in mp_blur_rgba_sub_bitmap() can be dropped.
The previous few commits changed sd_lavc.c's output to packed RGB sub-
images. In particular, this means all sub-bitmaps are part of a larger,
single bitmap. Change the vo_opengl OSD code such that it can make use
of this, and upload the pre-packed image, instead of packing and copying
them again.
This complicates the upload code a bit (4 code paths due to messy PBO
handling). The plan is to make sub-bitmaps always packed, but some more
work is required to reach this point. The plan is to pack libass images
as well. Since this implies a copy, this will make it easy to refcount
the result.
(This is all targeted towards vo_opengl. Other VOs, vo_xv, vo_x11, and
vo_wayland in particular, will become less efficient. Although at least
vo_vdpau and vo_direct3d could be switched to the new method as well.)
The sub-bitmaps get extended by --sub-gauss, so we have to compute the
bounding box on the original subs. Not sure if this is really
eqauivalent to what the code did before, and I don't have the sample
anymore. (But this approach sure is a _shitty_ hack.)
Implement it directly in sd_lavc.c as well. Blurring requires extending
the size of the sub-images by the blur radius. Since we now want
sub_bitmaps to be packed into a single image, and we don't want to
repack for blurring, we add some extra padding to each sub-bitmap in the
initial packing, and then extend their size later. This relies on the
previous bitmap_packer commit, which always adds the padding in all
cases.
Since blurring is now done on parts of a large bitmap, the data pointers
can become unaligned, depending on their position. To avoid shitty
libswscale printing a dumb warning, allocate an extra image, so that the
blurring pass is done on two newly allocated images. (I don't find this
feature important enough to waste more time on it.)
The previous refactor accidentally broke this feature due to a logic bug
in osd.c. It didn't matter before it happened to break, and doesn't
matter now since the code paths are different.
Until now, subtitle renderers could export SUBBITMAP_INDEXED, which is a
8 bit per pixel with palette format. sd_lavc.c was the only renderer
doing this, and the result was converted to RGBA in every use-case
(except maybe when the subtitles were hidden.)
Change it so that sd_lavc.c converts to RGBA on its own. This simplifies
everything a bit, and the palette handling can be removed from the
common code.
This is also preparation for making subtitle images refcounted. The
"caching" in img_convert.c is a PITA in this respect, and needs to be
redone. So getting rid of some img_convert.c code is a positive side-
effect. Also related to refcounted subtitles is packing them into a
single mp_image. Fewer objects to refcount is easier, and for the libass
format the same will be done. The plan is to remove manual packing from
the VOs which need single images entirely.
Older ffmpeg releases don't have ffmpeg git commit
50401f5fb7d778583b03a13bc4440f71063d319d, which fixes ffmpeg's
pkt_timebase check to reject its default "unset" timebase as invalid.
The consequence was that all non-PGS bitmap subtitle timestamps were
forced to 0.
Of course this hit _only_ shitty distros using outdated/badly maintained
ffmpeg releases, so this is not worth working around. I've already
wasted a lot of time on analyzing this dumb issue, and it could be
useful for bisecting, so don't drop pre-3.0 ffmpeg just yet.
Fixes#3109.
libass's ass_process_chunk expects long long int for the timecode and
durations arguments, thus should use llrint instead of lrint.
This does not cause any problems on most platforms, but on cygwin, it
causes strange subtitle behaviour, like subtitles not showing, getting
stuck or old subtitles showing at the same time as new subtitles.
--sub-ass=no / --ass=no still work, but --ass-style-override=strip is
preferred now. With this change, --ass-style-override can control all
the types of style overriding.
There is an obscure feature which requires essentially reordering PTS
from different packets.
Unfortunately, libavcodec introduced a ridiculously shitty API for
this, which works very much unlike the audio/video API. Instead of
simply passing through the PTS, it wants to fuck with it for no reason,
and even worse, fucks with other fields and changes their semantivcs
(??????). This affects AVSubtitle.end_display_time. This probably will
cause issues for us, and I have no desire to find out whether it will.
Since only PGS requires this, and it happens not to use
end_display_time, do it for PGS only.
Fixes#3016.
Basically, this information is useless, because some muxers (hurr
libavformat) write bogus information anyway. This means if we e.g. see
PGS packets in mkv with duration explicitly set to 0, we must not trust
that value anyway. (The FFmpeg API problem is leaking into files, how
nice.)
In particular, this prevents subtitle packets from building up in the
subtitle queue if e.g. --vo=null is used. In this situation,
sub_get_bitmaps() is never called, and thus the segment never switched.
This also seems to help with flickering at segment switch boundaries (if
subs are supposed to be visible at the transition points).
In theory, this could trigger a switch too early, but the way VO and
subtitle renderer interact wrt. timing is a bit iffy anyway.
Glitches when resizing are still possible, but are reduced. Other VOs
could support this too, but don't need to do so.
(Totally avoiding glitches would be much more effort, and probably not
worth the trouble. How about you just watch the video the player is
playing, instead of spending your time resizing the window.)
No need to have them everywhere. The only exception/annoyance is
MAX_OSD_PARTS, which is now basically duplicated (and at runtime
initialization is checked with an assert()).
Until now, there was only 1 global ASS overlay that could be set by all
scripts. This was often perceived as bug when multiple scripts tried to
set their own ASS overlay.
This was kind of hard to solve because the script could set its own ASS
PlayResX/Y, which makes it impossible to share a single ASS_Renderer for
multiple scripts. The OSC unfortunately makes use of this feature (and
unfortunately can't be fixed because it's a POS), so we're stuck with
this complication.
Implement the worst-case solution and fix this by creating separate ASS
track and renderer objects for each script that wants to set an ASS
overlay.
The z-order is decided by the order the scripts set their text first.
This is essentially random, unless you do it at script init, and you
pass scripts in a specific order. Script initialization is currently
serialized (as a feature), so the first loaded script gets lowest
Z-order.
The Lua script API interestingly remains the same. (And also will remain
undocumented, unsupported, and potentially volatile.)
Instead of passing an explicit cache to the function, the res parameter
is used. Also, instead of replacing its contents, sub bitmaps are now
appended to it (all assuming the format doesn't actually change).
This is preparation for the following commits.
Do not scale OSD mouse input to the ASS OSD script resolution. The
original idea of this mechanism was that the user doesn't have to care
about the actual resolution of anything, and can just use the OSD
resolution consistently. But this made things worse.
Remove the implicit scaling, and always use the screen resolution.
(Except with --vo=xv, where additional scaling is forced upon
everything.)
Drop get_osd_resolution(). There is no replacement. Rename
get_screen_size() and get_screen_margins() to use "osd" instead of
"screen". For anything but --vo=xv these are equivalent, but with
--vo=xv the OSD resolution has additional implicit scaling.
Add code to osc.lua which emulates the old behavior.
Note that none of the changed functions were public API, so implicit
breakage of scripts which used it is just going to happen.
Subtitles can be preloaded, which means they're fully read and copied
into ASS_Track. This in turn is mainly for the sake of being able to do
subtitle seeking (when it comes down to it, subtitle seeking is the
cause for most trouble here).
Commit a714f8e92 broke preloaded subtitles which have events with
unknown duration, such as some MicroDVD samples. The event list gets
cleared on every seek, so the property of being preloaded obviously gets
lost.
Fix this by moving most of the preloading logic to dec_sub.c. If the
subtitle list gets cleared, they are not considered preloaded anymore,
and the logic for demuxed subtitles is used.
As another minor thing, preloadeding subtitles did neither disable the
demux stream, nor did it discard packets. Thus you could get queue
overflows in theory (harmless, but annoying). Fix this by explicitly
discarding packets in preloaded mode.
In summary, now the only difference between preloaded and normal
demuxing are:
1. a seek is issued, and all packets are read on start
2. during playback, discard the packets instead of feeding them to the
subtitle decoder
This is still petty annoying. It would be nice if maintaining the
subtitle index (and maybe a subtitle packet cache for instant subtitle
presentation when seeking back) could be maintained in the demuxer
instead. Half of all file formats with interleaved subtitles have
this anyway (mp4, mkv muxed with newer mkvmerge).
Although there is logic to prune subtitles as soon as they get too old
in this mode, this is not done for the _currently_ shown subtitles. Thus
explicitly clearing subtitles on seek is required to avoid duplicate
subtitles in certain cases when seeking.
Deals with broken mkv subtitle tracks generated by tvheadend. The subs
are srt, but without packet durations.
We need this logic for CCs anyway. CCs in particular will be unaffected
by this change because they are also marked with unknown duration. It
could be that there are actual demuxers outputting CCs - in this case,
we rely on the fact that they don't set a (meaningless) packet duration
(or we'd have to work that around).
Commit 8d4a179c made subtitle decoders pick up fonts strictly from the
same source file (i.e. the same demuxer).
It breaks some fucked up use-case, and 2 people on this earth complained
about the change because of this. Add it back.
This copies all attached fonts on each subtitle init. I considered
converting attachments to use refcounting, but it'd probably be much
more complex.
Since it's slightly harder to get a list of active demuxers with
duplicate removed, the prev_demuxer variable serves as a hack to achieve
almost the same thing, except in weird corner cases. (In which fonts
could be added twice.)
Use the mp_lavc_set_extradata() function instead of setting up the
extradata manually. This takes care of the corner case when
extradata_len is 0.
This apparently fixes#2888.
This happens only if the new segment wasn't read yet.
This is not quite proper and a problem with dec_sub.c internals.
Ideally, it'd wait with rendering until a new enough segment has been
read. Normally, the new segment is available immediately, so the end
will be automatically clipped by switching to the right segment in the
exact moment it's supposed to become effective.
Usually shouldn't cause any problems, though.
This uses a different method to piece segments together. The old
approach basically changes to a new file (with a new start offset) any
time a segment ends. This meant waiting for audio/video end on segment
end, and then changing to the new segment all at once. It had a very
weird impact on the playback core, and some things (like truly gapless
segment transitions, or frame backstepping) just didn't work.
The new approach adds the demux_timeline pseudo-demuxer, which presents
an uniform packet stream from the many segments. This is pretty similar
to how ordered chapters are implemented everywhere else. It also reminds
of the FFmpeg concat pseudo-demuxer.
The "pure" version of this approach doesn't work though. Segments can
actually have different codec configurations (different extradata), and
subtitles are most likely broken too. (Subtitles have multiple corner
cases which break the pure stream-concatenation approach completely.)
To counter this, we do two things:
- Reinit the decoder with each segment. We go as far as allowing
concatenating files with completely different codecs for the sake
of EDL (which also uses the timeline infrastructure). A "lighter"
approach would try to make use of decoder mechanism to update e.g.
the extradata, but that seems fragile.
- Clip decoded data to segment boundaries. This is equivalent to
normal playback core mechanisms like hr-seek, but now the playback
core doesn't need to care about these things.
These two mechanisms are equivalent to what happened in the old
implementation, except they don't happen in the playback core anymore.
In other words, the playback core is completely relieved from timeline
implementation details. (Which honestly is exactly what I'm trying to
do here. I don't think ordered chapter behavior deserves improvement,
even if it's bad - but I want to get it out from the playback core.)
There is code duplication between audio and video decoder common code.
This is awful and could be shareable - but this will happen later.
Note that the audio path has some code to clip audio frames for the
purpose of codec preroll/gapless handling, but it's not shared as
sharing it would cause more pain than it would help.
This covers source files which were added in mplayer2 and mpv times
only, and where all code is covered by LGPL relicensing agreements.
There are probably more files to which this applies, but I'm being
conservative here.
A file named ao_sdl.c exists in MPlayer too, but the mpv one is a
complete rewrite, and was added some time after the original ao_sdl.c
was removed. The same applies to vo_sdl.c, for which the SDL2 API is
radically different in addition (MPlayer supports SDL 1.2 only).
common.c contains only code written by me. But common.h is a strange
case: although it originally was named mp_common.h and exists in MPlayer
too, by now it contains only definitions written by uau and me. The
exceptions are the CONTROL_ defines - thus not changing the license of
common.h yet.
codec_tags.c contained once large tables generated from MPlayer's
codecs.conf, but all of these tables were removed.
From demux_playlist.c I'm removing a code fragment from someone who was
not asked; this probably could be done later (see commit 15dccc37).
misc.c is a bit complicated to reason about (it was split off mplayer.c
and thus contains random functions out of this file), but actually all
functions have been added post-MPlayer. Except get_relative_time(),
which was written by uau, but looks similar to 3 different versions of
something similar in each of the Unix/win32/OSX timer source files. I'm
not sure what that means in regards to copyright, so I've just moved it
into another still-GPL source file for now.
screenshot.c once had some minor parts of MPlayer's vf_screenshot.c, but
they're all gone.
demux_lavf.c leaked the complete subtitle data if it was put through
iconv.
lavc_conv.c leaked AVCodecContext.subtitle_header (set by libavcodec),
which is fixed by using avcodec_free_context(). It also leaked the
subtitle that was decoded last.
This is mainly a refactor. I'm hoping it will make some things easier
in the future due to cleanly separating codec metadata and stream
metadata.
Also, declare that the "codec" field can not be NULL anymore. demux.c
will set it to "" if it's NULL when added. This gets rid of a corner
case everything had to handle, but which rarely happened.
Slightly change how it is decided when a new packet should be read.
Switch to demux_read_packet_async(), and let the player "wait properly"
until required subtitle packets arrive, instead of blocking everything.
Move distinguishing the cases of passive and active reading into the
demuxer, where it belongs.
Just so I can remove a few lines from dec_sub.c.
This is slightly inelegant, as the whole subtitle file has to be read
into memory, converted at once in memory, and then provided to
libavformat in an awkward way by creating a memory stream instead of
using demuxer->stream. It also won't be possible to force the charset on
subtitles in binary container formats - but this wasn't exposed before,
and we just hope this won't be ever needed. (One motivation was fixing
broken files with non-UTF8 muxed.) It also won't be possible to change
the charset on the fly, but this was not exposed either.
Just simplify by removing parts not needed anymore. This includes
merging dec_sub allocation and initialization (since things making
initialization complicated were removed), or format support queries (it
simply tries to create a decoder, and if that fails, tries the next
one).
So that the video FPs is not required at initialization, and can be set
later.
(As for whether this MicroDVD crap is worth the trouble to handle it
"correctly": MicroDVD files are unfortunately still around, and in at
least one case using the video FPS seemed to help indeed.)
Keeping ASS_Renderers around for a potentially large number of subtitle
tracks could lead to excessive memory usage, especially since the libass
cache is broken (caches even unneeded data), and might consume up to
~500MB of memory for no reason.
This includes the case of switching ordered chapter boundaries. It will
now be recreated on each timeline part switch. This shouldn't be much of
a problem with modern libass. (Older libass versions use fontconfig for
memory fonts, and will be very slow to reinitialize memory fonts.)
Since commit 6d9cb893, subtitle state doesn't survive timeline switches
(ordered chapters etc.). So there is no point in caching the state per
sh_stream anymore (which would be required to deal with multiple
segments). Move the cache to struct track.
(Whether it's worth caching the subtitle state just for the situation
when subtitle tracks get reselected is questionable. But for now, it's
nice to have the subtitles immediately show up when reselecting a
subtitle.)
When crossing timeline boundaries (such as switching to a new segment or
chapter with ordered chapters), clear the internal text subtitle list.
This breaks the sub-seek command, but is otherwise not too harmful.
Fixes Sub-OC-test-final7.mkv. (The internal text subtitle list is
basically a cache to make subtitles show up at the right time when
seeking back.)
I suspect this was caused by 76fcef61. The sample file times subtitles
slightly before the video frame when it should show up. This is to avoid
problems with subtitles showing up a frame later than intended. It also
means that a subtitle which is supposed to show up on the start of a
timeline part boundary actually might first be shown in a different
part. Since we now manipulate the packet timestamps, instead of
manipulating timestamps after the subtitle decoder, this means this
subtitle event would have 2 timestamps, which our code of course does
not handle.
If the two parts come one after another, this would actually work (since
the subtitle would have the same timestamps in the old and new part),
but it breaks if the new part (which follows the old part in the
physical file) is has a completely different start time in the timeline.
Essentially, the trick used to time subtitles correctly is incompatible
with the way we cache subtitles (to make them survive seeks).
The simple solution is just clearing the cached subtitles when crossing
chapter boundaries.
Merge blend_src8_alpha and blend_src16_alpha into blend_src_alpha, and
the same for blend_const_alpha.
One thing that changes is that the vertical loop is now shared for both
code paths.
I think this is slightly easier to read, and it's a bit shorter as well.
This removes the need to define IMGFMT_GBRAP, which fixes compilation
with the current Libav release.
This also makes it automatically pick up a GBRP format with the same bit
width. (Unfortunately, it seems libswscale does not support conversion
to AV_PIX_FMT_GBRAP16, so our code falls back to 8 bit, removing
precision for video covered by subtitles in cases this code is used.)
Also, when the source video is e.g. 10 bit YUV, upsample to 16 bit.
Whether this is good or bad, it fixes behavior with alpha. Although I'm
not sure if the alpha range is really correct ([0,2^16-1] vs.
[0,255*256]). Keep in mind that libswscale doesn't even agree with the
way we do it.
This actually treats destination alpha correctly, and gives much better
results than before. I don't know if this is perfectly correct yet,
though. Slight difference with vo_opengl behavior suggests it might not
be.
Note that this does not affect VOs with true alpha support. vo_opengl
does not use this code at all, and does the alpha calculations in OpenGL
instead.
Converted subtitles use a different method to avoid adding repeated
packets as duplicate subtitle events. The state for this mechanism must
be cleared as well if --sub-clear-on-seek is used.
MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
Apparently, this was replaced by the SD_CTRL_SET_VIDEO_PARAMS set
dimensions. But I can't find out when this happened - possibly, these
fields were never used by sd_lavc.c, and only by the (long removed)
MPlayer dvdsub decoder.
The previous commit turned sd_lavc_conv from a sd_driver to
free-standing functions. Do the rename to reflect this change
separately to avoid confusing git's content tracking. (Or did
git solve this, making separating renames and content changes
unnecessary?)
It was stupid. The only thing that still effectively used it was
sd_lavc_conv - all other "filters" were the subtitle decoder/renderers
for text (sd_ass) and bitmap (sd_lavc) subtitles.
While having a subtitle filter chain was interesting (and actually
worked in almost the same way as the audio/video ones), I didn't
manage to use it in a meaningful way, and I couldn't e.g. factor
secondary features like fixing subtitle timing into filters.
Refactor the shit and drop unneeded things as it goes.
This affects non-ASS text subtitles (those which go through libavcodec's
subtitle converter), which are muxed with video/audio. (Typically srt
subs in mkv.)
The problem is that seeking in the file can send a subtitle packet to
the decoder multiple times. These packets are interlaved with video,
and thus can't be all read when opening the file. Rather, subtitle
packets can essentially be randomly skipped or repeated (by seeking).
Until recently, this was solved by scanning the libass event list for
duplicates. Then our builtin srt-to-ass converter was removed, and
the problem was handled by fully clearing the subtitle list on each
seek.
This resulted in sub-seek not working properly for this type of file.
Since the subtitle list was cleared on seek, it was not possible to
do e.g. sub-seeks to subtitles before the current playback position.
Fix this by not clearing the list, and intead explicitly rejecting
duplicate packets. We use the packet file position was unique ID for
subtitles; this is confirmed working for most file formats (although
it is slightly risky - new demuxers may not necessarily set the file
position to something unique, or at all).
The list of seen packets is sorted, and the lookup uses binary search.
This is to avoid quadratic complexity when subtitles are added in
bulks, such as when opening a text subtitle file.
In some places, the code has to be adjusted to pass through the packet
file position correctly.
With the FFmpeg subtitle decoder used for _all_ non-ASS text subtitle
format, this code is simply unused now.
Ironically, the FFmpeg subtitle decoder does not handle things correctly
in a bunch of cases. Should it turn out they actually matter, they will
have to hack back.
The extend_event one is a candidate, although even though there were
allegedly files which need it, I couldn't get samples from the user who
originally reported such files. As such, extend_event was only confirmed
to handle trailing events with no (endless) duration like with MicroDVD
and LRC, but FFmpeg "fudges" these anyway, so no special handling is
needed.
This code also had logic to handle seeking with muxed srt subtitles,
which made the sub-seek command work. But this has been broken before
this commit already. Currently, seeking with muxed srt subs will clear
all subtitles, as the broken FFmpeg ASS format output by the libavcodec
subtitle converters does not check for duplicates. Since the subtitles
are all cleared, ass_step_sub() can not work properly and sub-seek can
not seek to already seen subtitles.
Slightly simpler, and removes the need to pre-read all subtitle packets.
This still does the subtitle charset conversion on the packet level
(instead converting when parsing the file), so in theory this still
could provide a way to change the charset at runtime. But maybe even
this should be removed, as FFmpeg is somewhat likely to get its own
charset detection and conversion mechanism in the future. (Would have
to keep the subtitle file in memory to allow changing the charset on
the fly, I guess.)
The FFmpeg subtitle converter does the same. There used to be some
deficiencies in FFmpeg's code, but it seems at least some of them have
been fixed. There also used to be the timestamp issue (see previous
commit messages), but this doesn't matter anymore. So no reason to
keep this code - get rid of it.
This can be dropped for the same reasons as in the previous commits. It
removes MicroDVD conversion support on Libav, although MicroDVD files
couldn't be read in the first place ever since demux_subreader.c was
removed.
