Simple enough to do. May have mixed results. Typically, bitmap subtitles
will have a tight bounding box around the rendered text. But if for
example there is text on the top and bottom, it may be a single big
bitmap with a large transparent area between top and bottom. In
particular, DVD subtitles are really just a single screen-sized
RLE-encoded bitmap, though libavcodec will crop off transparent areas.
Like with sd_ass, you can't move subtitles _down_ if they are already in
their origin position. This could probably be improved, but I don't want
to deal with that right now.
UB-sanitizer complains that we shift bits into the sign (when a is
used). Change it to unsigned, which in theory is more correct and
silences the warning.
Doesn't matter in practice, both the "bug" and the fix have 0 impact.
Remove them from the big MPOpts struct and move them to their sub
structs. In the places where their fields are used, create a private
copy of the structs, instead of accessing the semi-deprecated global
option struct instance (mpv_global.opts) directly.
This actually makes accessing these options finally thread-safe. They
weren't even if they should have for years. (Including some potential
for undefined behavior when e.g. the OSD font was changed at runtime.)
This is mostly transparent. All options get moved around, but most users
of the options just need to access a different struct (changing sd.opts
to a different type changes a lot of uses, for example).
One thing which has to be considered and could cause potential
regressions is that the new option copies must be explicitly updated.
sub_update_opts() takes care of this for example.
Another thing is that writing to the option structs manually won't work,
because the changes won't be propagated to other copies. Apparently the
only affected case is the implementation of the sub-step command, which
tries to change sub_delay. Handle this one explicitly (osd_changed()
doesn't need to be called anymore, because changing the option triggers
UPDATE_OSD, and updates the OSD as a consequence). The way the option
value is propagated is rather hacky, but for now this will do.
It was split at least across osd.c and sd_ass.c/sd_lavc.c. sd_lavc.c
actually ignored most of the more obscure subtitle timing things.
There's no reason for this - just move it all to dec_sub.c (mostly from
sd_ass.c, because it has some of the most complex stuff).
Now timestamps are transformed as they enter or leave dec_sub.c.
There appear to have been some subtle mismatches about how subtitle
timestamps were transformed, e.g. sd_functions.accepts_packet didn't
apply the subtitle speed to the timestamp. This patch should fix them,
although it's not clear if they caused actual misbehavior.
The semantics of SD_CTRL_SUB_STEP are slightly changed, which is the
reason for the changes in command.c and sd_lavc.c.
This API isn't deprecated (yet?), but it's still inferior and harder to
use than avcodec_free_context().
Leave the call only in 1 case in af_lavcac3enc.c, where we apparently
seriously close and reopen the encoder for whatever reason.
All contributors have agreed.
Compared to sd_ass.c, this has a pretty simple history:
av_sub.c -> sub/av_sub.c -> sub/sd_lavc.c
At one point, some code from spudec.c was added to it, but it was
removed again later.
This core of this heuristic was once copied from MPlayer's spudec.c. I
think it was meant for the case when the resolution field was missing or
so.
I couldn't find a file for which this actually does something. On the
other hand, there are samples which actually have a smaller resolution
than 720x576, and which are broken by this old hack.
For subtitles that set no resolution (I'm not sure which codec/container
that would be), there's still the fallback on video resolution.
Just get rid of this hack. Also cleanup a bit. SD_CTRL_GET_RESOLUTION
hasn't been used since DVD menu removal. get_resolution() is left with 1
call site, and would be quite awkward to keep, so un-inline it.
Whitelisting supported codecs is (probably) still better than just
allowing everything, given the weird FFmpeg API. I'm also assuming
Libav doesn't even have the codec ID, but I didn't check.
Also add a --teletext-page option, since otherwise it decodes every
teletext page and shows them in succession.
And yes, we can't use av_opt_set_int() - instead we have to set it as
string. Because FFmpeg's option system is terrible.
Like it's done for audio and video. Just to be uniform.
I'm sorry for deleting the anti-ffmpeg vitriol. It's still all true, but
since we decided to always set the timebase, the crappiness is isolated
to FFmpeg internals.
The accepts_packet packet callback is supposed to deal with subtitle
decoders which have only a small queue of current subtitle events (i.e.
sd_lavc.c), in case feeding it too many packets would discard events
that are still needed.
Normally, the number of subtitles that need to be preserved is estimated
by the rendering pts (get_bitmaps() argument). Rendering lags behind
decoding, so normally the rendering pts is smaller than the next video
frame pts, and we simply discard all subtitle events until the rendering
pts.
This breaks down in some annoying corner cases. One of them is seeking
backwards: the VO will still try to render the old PTS during seeks,
which passes a high PTS to the subtitle renderer, which in turn would
discard more subtitles than it should. There is a similar issue with
forward seeks. Add hacks to deal with those issues.
There should be a better way to deal with the essentially unknown
"rendering position", which is made worse by screenshots or rendering
with vf_sub. At the very least, we could handle seeks better, and e.g.
either force the VO not to re-render subs after seeks (ugly), or
introduce seek sequence numbers to distinguish attempts to render
earlier subtitles when a seek is done.
The intention is to let mp_ass_packer_pack() produce different output
for the RGBA and LIBASS formats. VOs (or whatever generates the OSD)
currently do not signal a preferred format, and this mechanism just
exists to switch between RGBA and LIBASS formats correctly, preferring
LIBASS if the VO supports it.
Since there are not many sub-rectangles, this doesn't cost too much. On
the other hand, it avoids frequent warnings with vo_xv.
Also, the second copy in mp_blur_rgba_sub_bitmap() can be dropped.
The previous few commits changed sd_lavc.c's output to packed RGB sub-
images. In particular, this means all sub-bitmaps are part of a larger,
single bitmap. Change the vo_opengl OSD code such that it can make use
of this, and upload the pre-packed image, instead of packing and copying
them again.
This complicates the upload code a bit (4 code paths due to messy PBO
handling). The plan is to make sub-bitmaps always packed, but some more
work is required to reach this point. The plan is to pack libass images
as well. Since this implies a copy, this will make it easy to refcount
the result.
(This is all targeted towards vo_opengl. Other VOs, vo_xv, vo_x11, and
vo_wayland in particular, will become less efficient. Although at least
vo_vdpau and vo_direct3d could be switched to the new method as well.)
The sub-bitmaps get extended by --sub-gauss, so we have to compute the
bounding box on the original subs. Not sure if this is really
eqauivalent to what the code did before, and I don't have the sample
anymore. (But this approach sure is a _shitty_ hack.)
Implement it directly in sd_lavc.c as well. Blurring requires extending
the size of the sub-images by the blur radius. Since we now want
sub_bitmaps to be packed into a single image, and we don't want to
repack for blurring, we add some extra padding to each sub-bitmap in the
initial packing, and then extend their size later. This relies on the
previous bitmap_packer commit, which always adds the padding in all
cases.
Since blurring is now done on parts of a large bitmap, the data pointers
can become unaligned, depending on their position. To avoid shitty
libswscale printing a dumb warning, allocate an extra image, so that the
blurring pass is done on two newly allocated images. (I don't find this
feature important enough to waste more time on it.)
The previous refactor accidentally broke this feature due to a logic bug
in osd.c. It didn't matter before it happened to break, and doesn't
matter now since the code paths are different.
Until now, subtitle renderers could export SUBBITMAP_INDEXED, which is a
8 bit per pixel with palette format. sd_lavc.c was the only renderer
doing this, and the result was converted to RGBA in every use-case
(except maybe when the subtitles were hidden.)
Change it so that sd_lavc.c converts to RGBA on its own. This simplifies
everything a bit, and the palette handling can be removed from the
common code.
This is also preparation for making subtitle images refcounted. The
"caching" in img_convert.c is a PITA in this respect, and needs to be
redone. So getting rid of some img_convert.c code is a positive side-
effect. Also related to refcounted subtitles is packing them into a
single mp_image. Fewer objects to refcount is easier, and for the libass
format the same will be done. The plan is to remove manual packing from
the VOs which need single images entirely.
Older ffmpeg releases don't have ffmpeg git commit
50401f5fb7d778583b03a13bc4440f71063d319d, which fixes ffmpeg's
pkt_timebase check to reject its default "unset" timebase as invalid.
The consequence was that all non-PGS bitmap subtitle timestamps were
forced to 0.
Of course this hit _only_ shitty distros using outdated/badly maintained
ffmpeg releases, so this is not worth working around. I've already
wasted a lot of time on analyzing this dumb issue, and it could be
useful for bisecting, so don't drop pre-3.0 ffmpeg just yet.
Fixes#3109.
There is an obscure feature which requires essentially reordering PTS
from different packets.
Unfortunately, libavcodec introduced a ridiculously shitty API for
this, which works very much unlike the audio/video API. Instead of
simply passing through the PTS, it wants to fuck with it for no reason,
and even worse, fucks with other fields and changes their semantivcs
(??????). This affects AVSubtitle.end_display_time. This probably will
cause issues for us, and I have no desire to find out whether it will.
Since only PGS requires this, and it happens not to use
end_display_time, do it for PGS only.
Fixes#3016.
Basically, this information is useless, because some muxers (hurr
libavformat) write bogus information anyway. This means if we e.g. see
PGS packets in mkv with duration explicitly set to 0, we must not trust
that value anyway. (The FFmpeg API problem is leaking into files, how
nice.)
This is mainly a refactor. I'm hoping it will make some things easier
in the future due to cleanly separating codec metadata and stream
metadata.
Also, declare that the "codec" field can not be NULL anymore. demux.c
will set it to "" if it's NULL when added. This gets rid of a corner
case everything had to handle, but which rarely happened.
Just simplify by removing parts not needed anymore. This includes
merging dec_sub allocation and initialization (since things making
initialization complicated were removed), or format support queries (it
simply tries to create a decoder, and if that fails, tries the next
one).
MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
It was stupid. The only thing that still effectively used it was
sd_lavc_conv - all other "filters" were the subtitle decoder/renderers
for text (sd_ass) and bitmap (sd_lavc) subtitles.
While having a subtitle filter chain was interesting (and actually
worked in almost the same way as the audio/video ones), I didn't
manage to use it in a meaningful way, and I couldn't e.g. factor
secondary features like fixing subtitle timing into filters.
Refactor the shit and drop unneeded things as it goes.
Image subtitles often use a "signaling" packet to set the end time of
the previous subtitle. As far as the libavcodec API is concerned, such
packets decode to empty AVSubtitles. Discard these after the end time of
the previous subtitle has been set.
Keep track of the per-subtitle end time better. This is for the sake of
improving sub_step/sub_seek. Without this, it would seek to the sub
before the previous sub, if the current sub has ended displaying.
Works roughly the same as the one in sd_ass for text subtitles. While
sub_step is very uninteresting, it comes for free with the support for
sub_seek.
The implementation is taken from ass_step_sub() from libass, with some
modifications
Until now, feeding packets to the decoder in advance was done for text
subtitles only. This was possible because libass buffers all subtitle
data anyway (in ASS_Track). sd_lavc, responsible for bitmap subs, does
not do this. But it can buffer a small number of subtitle frames ahead.
Enable this.
Repurpose the sub_accept_packets_in_advance(). Instead of "can take all
packets" it means "can take 1 packet" now. (The old meaning is still
needed locally in dec_sub.c; keep it there.) It asks the decoder whether
there is place for at least 1 subtitle packet. sd_lavc implements it and
returns true if its internal fixed-size subtitle queue still has a free
slot. (The implementation of this in dec_sub.c isn't entirely clean.
For one, decode_chain() ignores this mechanism, so it's implied that
bitmap subtitles do not use the subtitle filter chain in any advanced
way.)
Also fix 2 bugs in the sd_lavc queue handling. Subtitles must be checked
in reverse, because the first entry will often have endpts==NOPTS, which
would always match. alloc_sub() must cycle the queue buffer, because it
reuses memory allocations (like sub.imgs) by design.
Helps with broken vobsubs, which have an incorrect resolution header
set.
So we just extend the subtitle resolution to the video size, if the
video size is larger. This helps somewhat with readability, or makes
them visible at all. It should be a pretty safe change, because normally
no sub pictures are supposed to go outside of the area. It should make a
difference with broken files only.
The sample in question had a video resolution of 1888x1072, and a
subtitle resolution of 720x480. Note that always using video resolution
as subtitle resolution would break other files.
Probably makes users happy who want bitmap subtitles to show up in the
screen margins, and stops them from doing idiotic crap with vf_expand.
Fixes#2098.
There was a somewhat obscure optimization in the OSD and subtitle
rendering path: if only the position of the sub-images changed, and not
the actual image data, uploading of the image data could be skipped. In
theory, this could speed up things like scrolling subtitles.
But it turns out that even in the rare cases subtitles have such scrolls
or axis-aligned movement, modern libass rarely signals this kind of
change. Possibly this is because of sub-pixel handling and such, which
break this.
As such, it's a worthless optimization and just introduces additional
complexity and subtle bugs (especially in cases libass does the
opposite: incorrectly signaling a position change only, which happened
before). Remove this optimization, and rename bitmap_pos_id to
change_id.
Commit 87c13de6 added a fallback to video resolution if the subtitle
resolution is unknown. Apparently this fixed some broken files with
vobsubs.
This broke some DVB subtitles. Apparently .ts captures with 1920x1080
video resolution and 720x576 subtitles do exist. The sample at hand had
some streams with 720x576 resolution and no sub resolution set, and some
streams with 1920x1080 resolution and sub resolution set (both against
the same 1920x1080 video). My conclusion is that 720x576 is the only
reasonable fallback for DVB (but I can't be sure).
The fallback is removed for PGS too. I don't know about the PGS case; it
seems the sub resolution must always be set, so it shouldn't matter.
Fixes#1425.
Most image subtitle formats implicitly terminate the current subtitle
event with the next event (e.g. a new packet read from the demuxer will
instruct the subtitle render to stop display). If the subtitle event is
just trailing, it will be displayed forever. So there's no proper way
of doing this and we just apply an heuristic to avoid annoyances.
wm4
Stefano Pigozzi
cantabile
Dmitrij D. Czarkoff
Marcin Kurczewski