Slightly change how it is decided when a new packet should be read.
Switch to demux_read_packet_async(), and let the player "wait properly"
until required subtitle packets arrive, instead of blocking everything.
Move distinguishing the cases of passive and active reading into the
demuxer, where it belongs.
Just so I can remove a few lines from dec_sub.c.
This is slightly inelegant, as the whole subtitle file has to be read
into memory, converted at once in memory, and then provided to
libavformat in an awkward way by creating a memory stream instead of
using demuxer->stream. It also won't be possible to force the charset on
subtitles in binary container formats - but this wasn't exposed before,
and we just hope this won't be ever needed. (One motivation was fixing
broken files with non-UTF8 muxed.) It also won't be possible to change
the charset on the fly, but this was not exposed either.
Just simplify by removing parts not needed anymore. This includes
merging dec_sub allocation and initialization (since things making
initialization complicated were removed), or format support queries (it
simply tries to create a decoder, and if that fails, tries the next
one).
So that the video FPs is not required at initialization, and can be set
later.
(As for whether this MicroDVD crap is worth the trouble to handle it
"correctly": MicroDVD files are unfortunately still around, and in at
least one case using the video FPS seemed to help indeed.)
Keeping ASS_Renderers around for a potentially large number of subtitle
tracks could lead to excessive memory usage, especially since the libass
cache is broken (caches even unneeded data), and might consume up to
~500MB of memory for no reason.
This includes the case of switching ordered chapter boundaries. It will
now be recreated on each timeline part switch. This shouldn't be much of
a problem with modern libass. (Older libass versions use fontconfig for
memory fonts, and will be very slow to reinitialize memory fonts.)
Since commit 6d9cb893, subtitle state doesn't survive timeline switches
(ordered chapters etc.). So there is no point in caching the state per
sh_stream anymore (which would be required to deal with multiple
segments). Move the cache to struct track.
(Whether it's worth caching the subtitle state just for the situation
when subtitle tracks get reselected is questionable. But for now, it's
nice to have the subtitles immediately show up when reselecting a
subtitle.)
When crossing timeline boundaries (such as switching to a new segment or
chapter with ordered chapters), clear the internal text subtitle list.
This breaks the sub-seek command, but is otherwise not too harmful.
Fixes Sub-OC-test-final7.mkv. (The internal text subtitle list is
basically a cache to make subtitles show up at the right time when
seeking back.)
I suspect this was caused by 76fcef61. The sample file times subtitles
slightly before the video frame when it should show up. This is to avoid
problems with subtitles showing up a frame later than intended. It also
means that a subtitle which is supposed to show up on the start of a
timeline part boundary actually might first be shown in a different
part. Since we now manipulate the packet timestamps, instead of
manipulating timestamps after the subtitle decoder, this means this
subtitle event would have 2 timestamps, which our code of course does
not handle.
If the two parts come one after another, this would actually work (since
the subtitle would have the same timestamps in the old and new part),
but it breaks if the new part (which follows the old part in the
physical file) is has a completely different start time in the timeline.
Essentially, the trick used to time subtitles correctly is incompatible
with the way we cache subtitles (to make them survive seeks).
The simple solution is just clearing the cached subtitles when crossing
chapter boundaries.
Merge blend_src8_alpha and blend_src16_alpha into blend_src_alpha, and
the same for blend_const_alpha.
One thing that changes is that the vertical loop is now shared for both
code paths.
I think this is slightly easier to read, and it's a bit shorter as well.
This removes the need to define IMGFMT_GBRAP, which fixes compilation
with the current Libav release.
This also makes it automatically pick up a GBRP format with the same bit
width. (Unfortunately, it seems libswscale does not support conversion
to AV_PIX_FMT_GBRAP16, so our code falls back to 8 bit, removing
precision for video covered by subtitles in cases this code is used.)
Also, when the source video is e.g. 10 bit YUV, upsample to 16 bit.
Whether this is good or bad, it fixes behavior with alpha. Although I'm
not sure if the alpha range is really correct ([0,2^16-1] vs.
[0,255*256]). Keep in mind that libswscale doesn't even agree with the
way we do it.
This actually treats destination alpha correctly, and gives much better
results than before. I don't know if this is perfectly correct yet,
though. Slight difference with vo_opengl behavior suggests it might not
be.
Note that this does not affect VOs with true alpha support. vo_opengl
does not use this code at all, and does the alpha calculations in OpenGL
instead.
Converted subtitles use a different method to avoid adding repeated
packets as duplicate subtitle events. The state for this mechanism must
be cleared as well if --sub-clear-on-seek is used.
MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
Apparently, this was replaced by the SD_CTRL_SET_VIDEO_PARAMS set
dimensions. But I can't find out when this happened - possibly, these
fields were never used by sd_lavc.c, and only by the (long removed)
MPlayer dvdsub decoder.
The previous commit turned sd_lavc_conv from a sd_driver to
free-standing functions. Do the rename to reflect this change
separately to avoid confusing git's content tracking. (Or did
git solve this, making separating renames and content changes
unnecessary?)
It was stupid. The only thing that still effectively used it was
sd_lavc_conv - all other "filters" were the subtitle decoder/renderers
for text (sd_ass) and bitmap (sd_lavc) subtitles.
While having a subtitle filter chain was interesting (and actually
worked in almost the same way as the audio/video ones), I didn't
manage to use it in a meaningful way, and I couldn't e.g. factor
secondary features like fixing subtitle timing into filters.
Refactor the shit and drop unneeded things as it goes.
This affects non-ASS text subtitles (those which go through libavcodec's
subtitle converter), which are muxed with video/audio. (Typically srt
subs in mkv.)
The problem is that seeking in the file can send a subtitle packet to
the decoder multiple times. These packets are interlaved with video,
and thus can't be all read when opening the file. Rather, subtitle
packets can essentially be randomly skipped or repeated (by seeking).
Until recently, this was solved by scanning the libass event list for
duplicates. Then our builtin srt-to-ass converter was removed, and
the problem was handled by fully clearing the subtitle list on each
seek.
This resulted in sub-seek not working properly for this type of file.
Since the subtitle list was cleared on seek, it was not possible to
do e.g. sub-seeks to subtitles before the current playback position.
Fix this by not clearing the list, and intead explicitly rejecting
duplicate packets. We use the packet file position was unique ID for
subtitles; this is confirmed working for most file formats (although
it is slightly risky - new demuxers may not necessarily set the file
position to something unique, or at all).
The list of seen packets is sorted, and the lookup uses binary search.
This is to avoid quadratic complexity when subtitles are added in
bulks, such as when opening a text subtitle file.
In some places, the code has to be adjusted to pass through the packet
file position correctly.
With the FFmpeg subtitle decoder used for _all_ non-ASS text subtitle
format, this code is simply unused now.
Ironically, the FFmpeg subtitle decoder does not handle things correctly
in a bunch of cases. Should it turn out they actually matter, they will
have to hack back.
The extend_event one is a candidate, although even though there were
allegedly files which need it, I couldn't get samples from the user who
originally reported such files. As such, extend_event was only confirmed
to handle trailing events with no (endless) duration like with MicroDVD
and LRC, but FFmpeg "fudges" these anyway, so no special handling is
needed.
This code also had logic to handle seeking with muxed srt subtitles,
which made the sub-seek command work. But this has been broken before
this commit already. Currently, seeking with muxed srt subs will clear
all subtitles, as the broken FFmpeg ASS format output by the libavcodec
subtitle converters does not check for duplicates. Since the subtitles
are all cleared, ass_step_sub() can not work properly and sub-seek can
not seek to already seen subtitles.
Slightly simpler, and removes the need to pre-read all subtitle packets.
This still does the subtitle charset conversion on the packet level
(instead converting when parsing the file), so in theory this still
could provide a way to change the charset at runtime. But maybe even
this should be removed, as FFmpeg is somewhat likely to get its own
charset detection and conversion mechanism in the future. (Would have
to keep the subtitle file in memory to allow changing the charset on
the fly, I guess.)
The FFmpeg subtitle converter does the same. There used to be some
deficiencies in FFmpeg's code, but it seems at least some of them have
been fixed. There also used to be the timestamp issue (see previous
commit messages), but this doesn't matter anymore. So no reason to
keep this code - get rid of it.
This can be dropped for the same reasons as in the previous commits. It
removes MicroDVD conversion support on Libav, although MicroDVD files
couldn't be read in the first place ever since demux_subreader.c was
removed.
This restored timestamps when demuxing srt subtitles in Libav, which
was important for avoiding slightly overlapping subtitles. Since the
way this works was changed, there is no real reason to maintain proper
timestamps anymore on this level - this can be dropped without issues.
This has no reason to be there. Put the functionality into another
function instead. While we're at it, also adjust for possible accuracy
issues with high bit depth YUV (matters for rendering subtitles into
screenshots only).
Fixes a reported sample, that has a sign interrupted by a few frames
(for which --sub-fix-timing would remove the wanted gap).
The list of tags in has_overrides() is taken from libass. It has a
similar function (which even checks whether the tag are within the { }
delimiters). Unfortunately, this function is not public, so we just have
a simpler one which does roughly the same. It doesn't matter that this
function sometimes returns false positives.
The awkward "preprocess" step of putting the subtitles through single
filters before doing something else was made unnecessary by the recent
changes.
(Fun fact: I originally planned to move these extra things, like fixing
subtitle gaps/overlaps, to filters - but this would suffer from various
complications, and moving them to the renderers seems much simpler.)
I feel like it's better there. Note that there is no reduced
functionality, as bitmaps subs (i.e. not handled by sd_ass.c) were never
fully read on init, and thus never went through sub_read_all_packets().
On the other hand, this might lead to confusion, as --sub-fps etc. will
now also affect muxed subtitles (which makes not much sense).
Instead of messing with the subtitle packet timestamps, do it on output.
We work on the libass event list. If there is an unwanted gap or
overlap, we render the timestamp at another position where there is no
gap or overlap.
This is somewhat more robust, and even works with demuxed subs (to some
degree - depends whether the subs are prefected soon enough).
It's active even for native ASS subs. I wonder if this is a problem with
extended type setting. If it is, the heuristic that tries to avoid
interrupting such cases has to be improved.
While it probably would be ideal to do this after the subtitle decoder,
certain aspects are at least currently handled better in this place.
Image subtitles often use a "signaling" packet to set the end time of
the previous subtitle. As far as the libavcodec API is concerned, such
packets decode to empty AVSubtitles. Discard these after the end time of
the previous subtitle has been set.
Keep track of the per-subtitle end time better. This is for the sake of
improving sub_step/sub_seek. Without this, it would seek to the sub
before the previous sub, if the current sub has ended displaying.
Works roughly the same as the one in sd_ass for text subtitles. While
sub_step is very uninteresting, it comes for free with the support for
sub_seek.
The implementation is taken from ass_step_sub() from libass, with some
modifications
Until now, feeding packets to the decoder in advance was done for text
subtitles only. This was possible because libass buffers all subtitle
data anyway (in ASS_Track). sd_lavc, responsible for bitmap subs, does
not do this. But it can buffer a small number of subtitle frames ahead.
Enable this.
Repurpose the sub_accept_packets_in_advance(). Instead of "can take all
packets" it means "can take 1 packet" now. (The old meaning is still
needed locally in dec_sub.c; keep it there.) It asks the decoder whether
there is place for at least 1 subtitle packet. sd_lavc implements it and
returns true if its internal fixed-size subtitle queue still has a free
slot. (The implementation of this in dec_sub.c isn't entirely clean.
For one, decode_chain() ignores this mechanism, so it's implied that
bitmap subtitles do not use the subtitle filter chain in any advanced
way.)
Also fix 2 bugs in the sd_lavc queue handling. Subtitles must be checked
in reverse, because the first entry will often have endpts==NOPTS, which
would always match. alloc_sub() must cycle the queue buffer, because it
reuses memory allocations (like sub.imgs) by design.
Commit 2b07d3eb merged progbar and OSD text renderer into one ASS_Track,
but it confused the styles. Specifically, if both progbar and OSD are
visible, the create_ass_track() call made by the progbar code will reset
the style adjusted by the OSD text code.
Change create_ass_track() not to add any styles. Instead let the caller
manage the styles. They are now referenced by name, and lazily added if
they don't exist yet. This is also much cleaner.
Reduces memory usage and startup times. The implementation is a bit
weird, because both OSD parts have conflicting requirements on the used
ASS styles.
If a second subtitle is shown, it should be forced to display on top
of the screen. This was working only if --no-ass was passed, because
otherwise the subtitle was rendered normally (i.e. usually on the
bottom).
This was used with --no-sub-ass (aka --no-ass). This option (which is
not yet removed) strips all styling from the subtitles, and renders them
as plaintext only. For some reason, it originally seemed convenient to
reuse all the OSD text rendering code (osd_libass.c). While this was
indeed simple, it had a bad influence on the rest of the code. For
example, it had to decide whether to go through the OSD code path, or
the proper subtitle renderer in sd_ass.c.
Kill the OSD subtitle renderer. Reimplement --no-sub-ass and also
"secondary" subtitles in sd_ass.c. fill_plaintext() contains some rather
minor code duplication with osd_libass.c for setting up a dummy
ASS_Event and escaping the stripped text. Since sd_ass.c already has to
handle "normal" text subtitles, and has code for stripping ASS tags,
this remains all relatively simple.
Remove all the unnecessary crap from the rest of the code.
Use the demux_set_ts_offset() added in the previous commit to base each
timeline segment to use timestamps according to its relative position
within the overall timeline. As a consequence we don't need to care
about these timestamps anymore, and everything becomes simpler.
(Another minor but delicious nugget of sanity.)
av_free_packet() got finally deprecated. Use av_packet_unref() instead,
which has almost the same semantics, has existed for a while, and is
available in all FFmpeg and Libav versions we support.
If the aspect ratio of the video resolution and the subtitle resolution
(the implied subtitle coordinate system) mismatch, the subtitles
obviously can't be overlayed over the video perfectly. Either you get
video that can't be covered by subtitles, or the subtitles could go
beyond the video. We don't want to stretch the subtitle to compensate
for the aspect ratio, because it would look terrible.
Until now, mpv used to fit the subtitle rectangle into the video
rectangle (letterboxing/pillarboxing). This looks odd with some sample
files with subtitle canvas being wider than the video. Also, mpc-hc
displays them in a better way. vlc stretches them, which looks bad.
While you probably can't win this game with all those broken files
around, pick the mpc-hc method to handle this.
Helps with broken vobsubs, which have an incorrect resolution header
set.
So we just extend the subtitle resolution to the video size, if the
video size is larger. This helps somewhat with readability, or makes
them visible at all. It should be a pretty safe change, because normally
no sub pictures are supposed to go outside of the area. It should make a
difference with broken files only.
The sample in question had a video resolution of 1888x1072, and a
subtitle resolution of 720x480. Note that always using video resolution
as subtitle resolution would break other files.
This AVPacket field was a hack against the fact that the duration field
was merely an int (too small for things like subtitle durations). Newer
libavcodec drops this field and makes duration 64 bit.
This was in sub/, because the code used to be specific to subtitles. It
was extended to automatically load external audio files too, and moving
the file and renaming it was long overdue.
Until now, most OSD objects created the associated ASS_Renderer instance
as soon as possible, even if nothing was going to be rendered. Maybe
this was even intentional.
Change this for the sake of lowering resource usage, and strictly
initialize ASS_Renderer only when it's really needed.
For the OSC, initialization has to be forced, because of the insane
mechanism for translating mouse coordinates to OSD coordinates.
Drop the whitelist. It's annoying to maintain. Instead, accept any
subtitle decoder. Since this code path will now also be taken by bitmap
subtitle decoders not whitelisted by sd_lavc.c, add a warning when
bitmap subtitles are decoded. (To reduce or increase potential user
confusion.)
To some degree, a whitelist is needed to distinguish text and bitmap
subtitles. FFmpeg has an API to distinguish them in a generic way to
some degree, but Libav doesn't. So we just stick with this for now.
The FFmpeg can officially not distinguish between unknown subtitle
durations, and subtitle durations being 0. (It documents the value 0
meaning unknown duration.)
In practice, at least the LRC demuxer signals unknown subtitle durations
with a negative value.
Assume negative durations mean unknown duration. Show subtitles with
unknown duration forever. Unless there's a subtitle event following it;
then reset the duration so that it ends on the new subtitle event.
Fixes#2244.
Nobody wanted to restore this, so it gets the boot.
If anyone still wants to volunteer to restore menu support, this would
be welcome. (I might even try it myself if I feel masochistic and like
wasting a lot of time for nothing.) But if it does get restored, it
should be done differently. There were many stupid things about how it
was done. For example, it somehow tried to pull mp_nav_events through
all the layers (including needing to "buffer" them in the demuxer),
which was needlessly complicated. It could be done simpler.
This code was already inactive, so this commit actually changes nothing.
Also keep in mind that normal DVD/BD playback still works.
uchardet is written in C++, and thus doesn't appreciate the value of
using static strings, and internally stores the guessed charset as
allocated std::string. Add a minimal hack to deal with this. (I don't
appreciate that the code is potentially harder to understand by
returning either a static or allocated string, but I do appreciate for
not having to litter the existing code with strdups.)
Probably makes users happy who want bitmap subtitles to show up in the
screen margins, and stops them from doing idiotic crap with vf_expand.
Fixes#2098.
Each subtitle track gets its own decoder instance (sd_ass). But they use
a shared ASS_Renderer. This is done mainly because of fontconfig.
Initializing fontconfig is very slow when using it with memory fonts, so
there's a practical need to cache this memory font state, which is done
by not creating separate ASS_Renderers. This is very dirty and very
evil, but we probably can't get rid of it any time soon.
The shared ASS_Renderer was not properly synchronized. While the program
logic guarantees that only one sd_ass instance is visible at a time,
there are other interactions that require synchronization. In
particular, I suspect concurrent execution of mp_ass_configure_fonts()
and sd_ass.get_bitmaps cause issues in a newer libass development
branch.
So here's a shitty hack that hopefully fixes things, hopefully only
until libass becomes less dependent on fontconfig.
MPlayer traditionally had completely separate sh_ structs for
audio/video/subs, without a good way to share fields. This meant that
fields shared across all these headers had to be duplicated. This commit
deduplicates essentially the last remaining duplicated fields.
Because gcc (and clang) is a goddamn PITA and unnecessarily warns if
the universal initializer for structs is used (like mp_image x = {})
and the first member of the struct is also a struct, move the w/h
fields to the top.
They are redundant. They were used by draw_bmp.c only, and only in a
special code path that 1. used fixed image formats, and 2. had image
sized perfectly aligned to chroma boundaries (so computing the chroma
width/height is trivial).
Prefer the builtin one again. libavcodec still uses the ASS packet
format that uses inline timestamps, so the packet timestamps are
ignored. This again leads to additional rounding of timestamps, because
the ASS storage format only has 10ms resolution (instead of 1ms
resolution like libass). This again can lead to unintentional overlaps
when converting subtitles. The internal MicroDVD converter avoids this,
because it always uses packet timestamps.
Remove the colorspace-related top-level options, add them to vf_format.
They are rather obscure and not needed often, so it's better to get them
out of the way. In particular, this gets rid of the semi-complicated
logic in command.c (most of which was needed for OSD display and the
direct feedback from the VO). It removes the duplicated color-related
name mappings.
This removes the ability to write the colormatrix and related
properties. Since filters can be changed at runtime, there's no loss of
functionality, except that you can't cycle automatically through the
color constants anymore (but who needs to do this).
This also changes the type of the mp_csp_names and related variables, so
they can directly be used with OPT_CHOICE. This probably ended up a bit
awkward, for the sake of not adding a new option type which would have
used the previous format.
This has a number of user-visible changes:
1. A new flag blend-subtitles (default on for opengl-hq) to control this
behavior.
2. The OSD itself will not be color managed or affected by
gamma controls. To get subtitle CMS/gamma, blend-subtitles must be
used.
3. When enabled, this will make subtitles be cleanly interpolated by
:interpolation, and also dithered etc. (just like the normal output).
Signed-off-by: wm4 <wm4@nowhere>
There's literally no reason why these functions have to be inline (they
might be performance critical, but then the function call overhead isn't
going to matter at all).
Uninline them and move them to mp_image.c. Drop the header file and fix
all uses of it.
There was a somewhat obscure optimization in the OSD and subtitle
rendering path: if only the position of the sub-images changed, and not
the actual image data, uploading of the image data could be skipped. In
theory, this could speed up things like scrolling subtitles.
But it turns out that even in the rare cases subtitles have such scrolls
or axis-aligned movement, modern libass rarely signals this kind of
change. Possibly this is because of sub-pixel handling and such, which
break this.
As such, it's a worthless optimization and just introduces additional
complexity and subtle bugs (especially in cases libass does the
opposite: incorrectly signaling a position change only, which happened
before). Remove this optimization, and rename bitmap_pos_id to
change_id.
Update options like --sub-text-margin-y at runtime. These are somewhat
of a problem, because ass_set_selective_style_override() (intentionally)
does not override them. This should be fixed in libass (by providing
additional override modes), but for now this will do.
Also change the signature of mp_ass_configure(), so we can get access to
the track.
Additionally, drop the redundant setting of the style Alignment (it's
overwritten by mp_ass_set_style()).
See #1622 (again).
Overwrite the alignment applied by the OSD style. Additionally, remove
the initialization of the Alignment field in create_ass_track(); the
value is always overwritten by mp_ass_set_style() later.
Fixes#1626.
There was some logic to set certain things on init only. Not sure why
this was done (saving some cheap calculations?) - but since the next
call would override these style settings by applying the usual subtitle
style, I don't think this was intended.
Basically abuse the style override mechanism meant for ASS
(mp_ass_set_style()) to update text subtitle styling at runtime too.
This even has the advantage that the style will be overridden, even if
the text subtitle converted (like sd_lavc_conv.c) dares to add a fixed
style in the styles section.
Probably helps with #1622.
Now --ass-use-margins doesn't apply to normal subtitles anymore. This is
probably the inverse from the mpv behavior users expected so far, and
thus a breaking change, so rename the option, that the user at least has
a chance to lookup the option and decide whether the new behavior is
wanted or not.
The basic idea here is:
- plain text subtitles should have a certain useful defalt behavior,
like actually using margins
- ASS subtitles should never be broken by default
- ASS subtitles should look and behave like plaintext subtitles if
the --ass-style-override=force option is used
This also subtly changes --sub-scale-with-window and adds the --ass-
scale-with-window option. Since this one isn't so important, don't
bother with compatibility.
You can set in which "corner" the OSD and subtitles are shown. I'd
prefer it a bit more general (so you could set the alignment using
a factor), but the libass API does not provide this.
Until now, they used exactly the same defaults for the styling options.
The defaults were shared, so it was impossible to have different
defaults. Change this. This requires duplicating the full default
struct, even for settings that are the same. The list of options is
still shared, though.
Commit 87c13de6 added a fallback to video resolution if the subtitle
resolution is unknown. Apparently this fixed some broken files with
vobsubs.
This broke some DVB subtitles. Apparently .ts captures with 1920x1080
video resolution and 720x576 subtitles do exist. The sample at hand had
some streams with 720x576 resolution and no sub resolution set, and some
streams with 1920x1080 resolution and sub resolution set (both against
the same 1920x1080 video). My conclusion is that 720x576 is the only
reasonable fallback for DVB (but I can't be sure).
The fallback is removed for PGS too. I don't know about the PGS case; it
seems the sub resolution must always be set, so it shouldn't matter.
Fixes#1425.
Although the line count increases, this is better for making sure
everything is handled consistently for all users of the mp_csp_params
stuff.
This also makes sure mp_csp_params is always initialized with
MP_CSP_PARAMS_DEFAULTS (for consistency).
wm4
Aman Gupta
Marcin Kurczewski
Niklas Haas
Ben Boeckel
Jaime Marquínez Ferrándiz