Old code used to use libass' recoding feature, which is a copy of the
old MPlayer code. We dropped that a few commits ago. Unfortunately,
this made it impossible to load some subtitle files, like UTF-16 files.
Make .ass loading respect -subcp again. We do this by recoding the
probe buffer to UTF-8, and then trying to load it normally. (Yep.)
Since UTF-16 in particular will effectively half the probe buffer size,
double the probe size.
demux_libass.c allows us to make subtitle format detection part of the
normal file loading process. libass has no probe function, but trying to
load the start of a file (the first 4 KB) is good enough. Hope that
libass can even handle random binary input gracefully without printing
stupid log messages, and that the libass parser doesn't accept too many
non-ASS files as input.
This doesn't handle the -subcp option correctly yet. This will be fixed
later.
The default correct-pts mode depended on which demuxer was opened last.
Often this is the subtitle demuxer. The correct-pts mode should be
decided on the demuxer for video instead.
subreader.c (before this commit renamed to demux_subreader.c) was
special cased to the -sub option. The plan is using the normal demuxer
codepath for all subtitle formats (so we can prefer libavformat demuxers
for most formats).
There are some subtle changes. The probe size is restricted to 32 KB
(instead of unlimitted + giving up after 100 lines of input). For
formats like MicroDVD, the video FPS isn't used anymore, because it's
not available on the subtitle demuxer level. Instead, hardcode it to
23.976 FPS (libavformat seems to do the same). The user can probably
still use -sub-fps to fix the timing. Checking the file extension for
".utf"/".utf8"/".utf-8" is simply removed (seems worthless, was in the
way, and I've never seen this anywhere).
Simpler, reduces the amount of copying.
We still have to malloc+memcpy the probe buffer though, because padding
with FF_INPUT_BUFFER_PADDING_SIZE is required by libavformat.
If a subtitle is external, read it completely and add all subtitle
events in advance when the subtitle track is selected. This is done
for text subtitles only. (Note that subreader.c and subtitles loaded
with libass are different and don't have anything to do with this
commit.)
Seems like a completely unnecessary complication. Instead, always add a
1 byte padding (could be extended if a caller needs it), and clear it.
Also add some documentation. There was some, but it was outdated and
incomplete.
The stream EOF flag should only be set when trying to read past the end
of the file (relatively similar to unix files). Always clear the EOF
flag on seeking. Trying to set it "properly" (depending whether data is
available at seek destination or not) might be an ok idea, but would
require attention to too many special cases. I suspect before this
commit (and in MPlayer etc. too), the EOF flag wasn't handled
consistently when the stream position was at the end of the file.
Fix one special case in ebml.c and stream_skip(): this function couldn't
distinguish between at-EOF and past-EOF either.
This function was called in various places. Most time, it was used
before a seek. In other cases, the purpose was apparently resetting
the EOF flag. As far as I can see, this makes no sense anymore. At
least the stream_reset() calls paired with stream_seek() are completely
pointless. A seek will either seek inside the buffer (and reset the
EOF flag), or do an actual seek and reset all state.
This branch heavily refactors the subtitle code (both loading and
rendering), and adds support for a few new formats through FFmpeg.
We don't remove any of the old code yet. There are still some subtleties
related to subreader.c to be resolved: code page detection & conversion,
timing post-processing, UTF-16 subtitle support, support for the -subfps
option. Also, SRT reading and loading ASS via libass should be turned
into proper demuxers. (SRT is needed because Libav's is gravely broken,
and we want ASS loading via libass to cover full libass format support.
Both should be demuxers which are probed _before_ libavformat, so that
all subtitles can be loaded through the demuxer infrastructure, and
libavformat subtitles don't need to be treated in a special way.)
Before this, subtitle packets were returned as data ptr/len pairs, and
mplayer.c got the rest (pts and duration) directly from the demuxer
data structures. Then mplayer.c reassembled the packet data structure
again.
Pass packets directly instead. The mplayer.c side stays a bit awkward,
because the (now by default unused) DVD path keeps getting in the way.
In demux.c there's lots of weird stuff (3 functions that read packets,
really?), but we want to keep the code equivalent for now to avoid
hitting weird issues and corner cases.
Makes WebVTT actually work.
Also simplify the logic for setting duration. Only the subtitle path
uses the packet duration, so the checks for STREAM_SUB as well as the
keyframe flag are redundant.
Apparently duration and convergence_duration are the same thing, but
convergence_duration was added as Matroska-specific hack to get a higher
value range (int vs. int64_t) with high resolution Matroska timebases.
For us it doesn't matter, because double floats are used for timestamps
and durations.
This means subassconvert.c is split in sd_srt.c and sd_microdvd.c. Now
this code is involved in the sub conversion chain like sd_movtext is.
The invocation of the converter in sd_ass.c is removed.
This requires some other changes to make the new sub converter code work
with loading external subtitles. Until now, subtitles loaded via
subreader.c was assumed to be in plaintext, or for some formats, in ASS
(except in -no-ass mode). Then these were added to an ASS_Track. Change
this so that subtitles are always in their original format (as far as
decoders/converters for them are available), and turn every sub event
read by subreader.c as packet to the dec_sub.c subtitle chain.
This removes differences between external/demuxed and -ass/-no-ass code
paths further.
Internally, stream_dvd.c returned DEMUXER_TYPE_MPEG_PS, and the same
value was hardcoded to enforced usage of demux_lavf in demux.c. But
"-demuxer mpegps" basically did the same, so that switch was broken
for this format. Undo this and don't request a demuxer in stream_dvd.c.
demux_lavf.c is (probably) good enough to probe correctly with DVD.
Otherwise, we'd actually have to do something completely different to
force the libavformat demuxer.
Make the sub decoder stuff independent from sh_sub (except for
initialization of course). Sub decoders now access a struct sd only,
instead of getting access to sh_sub. The glue code in dec_sub.c is
similarily independent from osd.
Some simplifications are made. For example, the switch_id stuff is
unneeded: the frontend code just has to make sure to call osd_changed()
any time subtitles are switched.
This is also preparation for introducing subtitle converters. It's much
cleaner to completely separate demuxer header/renderer glue/decoders
for this purpose, especially since sub converters might completely
change how demuxer headers have to be interpreted.
Also pass data as demux_packets. Currently, this doesn't help much, but
libavcodec converters might need scary stuff like packet side data, so
it's perhaps better to go with passing packets.
Subtitle files are opened in mplayer.c, not using the demuxer
infrastructure in general. Pretend that this is not the case (outside of
the loading code) by opening a pseudo demuxer that does nothing. One
advantage is that the initialization code is now the same, and there's
no confusion about what the difference between track->stream,
track->sh_sub and mpctx->sh_sub is supposed to be.
This is a bit stupid, and it would be much better if there were proper
subtitle demuxers (there are many in recent FFmpeg, but not Libav). So
for now this is just a transition to a more proper architecture. Look
at demux_sub like an artifical limb: it's ugly, but don't hate it - it
helps you to get on with your life.
This unifies the subtitle rendering path. Now all subtitle rendering
goes through sd_ass.c/sd_lavc.c/sd_spu.c.
Before that commit, the spudec.h functions were used directly in
mplayer.c, which introduced many special cases. Add sd_spu.c, which is
just a small wrapper connecting the new subtitle render API with the
dusty old vobsub decoder in spudec.c.
One detail that changes is that we always pass the palette as extra
data, instead of passing the libdvdread palette as pointer to spudec
directly. This is a bit roundabout, but actually makes the code simpler
and more elegant: the difference between DVD and non-DVD dvdsubs is
reduced.
Ideally, we would just delete spudec.c and use libavcodec's DVD sub
decoder. However, DVD playback with demux_mpg produces packets
incompatible to lavc. There are incompatibilities the other way around
as well: packets from libavformat's vobsub demuxer are incompatible to
spudec.c. So we define a new subtitle codec name for demux_mpg subs,
"dvd_subtitle_mpg", which only sd_spu can decode.
There is actually code in spudec.c to "assemble" fragments into complete
packets, but using the whole spudec.c is easier than trying to move this
code into demux_mpg to fix subtitle packets.
As additional complication, Libav 9.x can't decode DVD subs correctly,
so use sd_spu in that case as well.
The new wavpack packet format (see previous commit) doesn't work with
older libavcodec versions, so disable the new code in this case.
The version numbers are only approximate, since the libavcodec version
wasn't bumped with the wavpack change, but it's close enough.
Libav introduced a silent API breakage by changing what wavpack packets
the libavcodec decoder accepts. Originally the libavcodec codec accepted
Matroska-style wavpack packets. Libav commit 9b6f47c removed this
capability from the libavcodec code, and added code to libavformat's
Matroska demuxer to "rearrange" wavpack packets. Since demux_mkv still
sent Matroska-style packets, playback failed.
Fix this by "rearranging" packets in demux_mkv as well by copying
libavformat's code. (The best kind of fix.)
Tested with [CCCP]_Mega_Lossless_Audio_Test.mkv, as well as with a
sample generated by mkvmerge.
The core deselected all streams on initialization, and then selected the
streams it actually wanted. This was no problem for
demux_mkv/demux_lavf, but old demuxers (like demux_asf) could lose some
packets. The problem is that these demuxers can buffer some data on
initialization, which then is flushed on track switching. Fix this by
explicitly avoiding deselecting a wanted stream.
When AAC is streamed over HTTP, using libavformat defaults is
pathetically slow. One solution for that is skipping probing and using
the mimetype to identify that it's AAC instead. This is what we did
before this commit (and ffmpeg does it too, but their logic is too
"inaccessible" for mpv).
This is still pretty fragile though. Make it a bit more robust by
requiring minimal probing. A probescore of 25 is reached after feeding
2 KB to libavformat (instead of > 500 KB for the normal probescore), so
use that. This is done only when streaming AAC from HTTP to reduce the
possibility of weird breakages for other formats.
Also reduce analyzeduration. The default analyzeduration will make
libavformat read lots of data, which makes playback start slow. So we
set analyzeduration to a low value. On the other hand, doing that for
other formats is risky, because there are unspecified effects with
certain "strange" formats (like transport streams). So we do this only
if we're streaming AAC from HTTP as well.
tl;dr libavformat is shit for media players
This can control whether demux_lavf should use the HTTP mime type to
determine the format, instead of probing the data with the libavformat
API. Do this to allow easier debugging in case the mimetype is
incorrect. (This is done only for AAC streams right now.)
Playing Youtube videos often requires an additional seek to the end of
the file. This flushes the stream cache. The reason for the seek is
reading the cues (seek index). This poses the question why Google is
muxing its files in such a way, since nothing in Matroska mandates that
cues are located at the end of the file, but we want to handle this
situation better anyway.
The seek index is not needed for normal playback, only for seeking.
This commit changes header parsing such that the index is not read on
initialization in order to avoid the additional stream-level seek.
Instead, read the index on the first demuxer-level seek, when the seek
index is actually needed.
If the cues are at the beginning of the file, they are read immediately
as part of the normal header reading process. This commit changes
behavior only if cues are outside of the header (i.e. not in the area
between EBML header and clusters), and linked by a SeekHead. Other
level 1 elements linked by the SeekHead might still cause seeks to the
end of the file, although that seems to be rare.
Before this commit, the demuxer would in theory accept multiple cues
elements (and append its contents to the index in the order as
encountered during reading). According to the Matroska specification,
there can be only one cues element in the segment, so this seems like
an overcomplication.
Change it so that redundant elements are ignored, like with all other
unique header elements. This makes implementing deferred reading of the
cues element easier.
The sequence of avcodec_alloc_context3() / avcodec_copy_context() /
avcodec_close() / av_free() leaks some memory. So don't copy the context
and use it directly.
Originally avcodec_copy_context() was used to guarantee that libavformat
can't update the fields of the context during demuxing in order to make
things a little more robust, but it's not strictly needed, and
ffmpeg/ffplay don't do this anyway. Still might make the situation worse
should we move demuxing into a separate thread, though.
Using -demuxer mpegts -correct-pts triggered the assertion in
ds_get_packet2(). This is not surprising, because the correct-pts code
was changed to accept _complete_ packets, while all the old demuxers
(including the mpegts demuxer) require you to use "partial" packet
reads, together with the video_read_frame(). (That function actually
parses video frames, so fragments of the original "packets" can be fed
to the decoder.)
However, it returns out demux_ts packet's are mostly useable. demux_ts
still adds an offset (i.e. ds->buffer_pos != 0) to the packets when
calling internal parser functions, such as in parse_es.c. While this is
unclean design due to mplayer's old video demuxing/decoding path, it can
be easily be made work by modifying the packet as returned by
ds_get_packet2(). We also have to change the packet freeing code, as
demux_packet->buffer doesn't have to point to the start of the memory
allocation anymore.
MPlayer handles this "correctly" because it doesn't have a function that
reads a complete packet.
Nobody uses this, and this is an absolute waste of time. Even the user
who reported this turned out to have produced a sample manually.
Sample produced with:
wget http://diracvideo.org/download/test-streams/raw/vts/vts.LD-8Mb.drc
mkvmerge -o dirac.mkv vts.LD-8Mb.drc
mkvmerge writes a sort of broken aspect ratio. libavformat interprets it
as 1:1 PAR, while demux_mkv thinks this is a 1:1 DAR. Maybe libavformat
is more correct here.
This helps passing the channel layout correctly from decoder to audio
filter chain. (Because that part "reuses" the demuxer level codec
parameters, which is very disgusting.)
Note that ffmpeg stuff already passed the channel layout via
mp_copy_lav_codec_headers(). So other than easier dealing with the
demuxer/decoder parameters mess, there's no real advantage to doing
this.
Make the --channels option accept a channel map. Since simple numbers
map to standard layouts with the given number of channels, this is
downwards compatible. Likewise for demux_rawaudio.
Allow the stream layer to report chapter times. Extend stream_dvd to do
this. I'm not 100% sure whether the re-used code is bug-free (because it
was used for slave-mode and/or debugging only).
MAke the frontend do time-based seeks when switching DVD chapters. I'm
not sure if there's a real reason STREAM_CTRL_SEEK_TO_CHAPTER exists
(maybe/hopefully not), but we will see.
Note that querying chapter times in demuxer_chapter_time() with the new
STREAM_CTRL_GET_CHAPTER_TIME could be excessively slow, especially with
the cache enabled. The frontend likes to query chapter times very often.
Additionally, stream_dvd uses some sort of quadratic algorithm to list
times for all chapters. For this reason, we try to query all chapters on
start (after the demuxer is opened), and add the chapters to the demuxer
chapter list. demuxer_chapter_time() will get the time from that list,
instead of asking the stream layer over and over again.
This assumes stream_dvd knows the list of chapters at the start, and
also that the list of chapters never changes during playback. This
seems to be true, and the only exception, switching DVD titles, is not
supported at runtime (and doesn't need to be supported).
The frontend doesn't use this.
Also use double for returning the chapter times. Everything uses double
for times, and there's no reason to use float here.
These were found by the cppcheck and scan-build static analyzers. Most
of these aren't interesting (the 2 previous commits fix some interesting
cases found by these analyzers), and they don't nearly fix all warnings.
(Most of the unfixed warnings are spam, things MPlayer never cared
about, or false positives.)
Also, mark demuxer as not capable if DVD playback is done. The problem
with DVD is that playback time (stream_pts) is not reported frame-exact,
and the time is a "guess" at best.
With the commit "demux_lavf: fix DEMUXER_CTRL_RESYNC", DVD playback
seems to work nicely with demux_lavf, and maybe works even better than
with demux_mpg.
The old demuxer can be forced with: --demuxer=mpegps
If no regressions surface, demux_mpg.c will be deleted later.
This used the libavformat current position, instead of the mp stream
(which reflects current DVD/Bluray read position). This was broken,
because libavformat won't update its position by calling the user's
stream callbacks, negating the whole point of DEMUXER_CTRL_RESYNC.
Now DVD playback with libavformat seems to work relatively well.
demux_mpg did the same, and doing this in demux_lavf fixes DVD playback
when using this demuxer.
Additionally this might make bluray work better in the future (but for
now, bluray playback doesn't change as it doesn't report stream PTS yet).
DVD playback uses a demuxer that signals to the frontend that timestamp
resets are possible. This made the frontend calculate the OSD playback
position based on the byte position and the total size of the stream.
This actually broke DVD playback position display. Since DVD reports a
a linear playback position, we don't have to rely on the demuxer
reported position, so disable this functionality in case of DVD
playback. This reverts the OSD behavior with DVD to the old behavior.
The stream ID handling as it was changed in commit 654c34f was still
a little bit insane, and caused a regression with the cover art hack
(the stream set in demux->video->sh was incorrect for demux_lavf).
Simplify by always using stream_index for demux_stream->id, and getting
rid of that tid thing. It turns out that the id for subtitles isn't
special either (maybe demux_ts.c was the only thing left that required
this).
This check was always false:
if (num == EBML_UINT_INVALID)
Fix it by using the proper type for the num variable.
This case actually doesn't really matter, and this is just for hiding
the warning and for being 100% correct.
When trying to seek before the start of the file, which usually happens
when using the arrow keys to seek to the start of the file, external
libavformat demuxed subtitles will be invisible. This is because seeking
in the external subtitle file fails, so the subtitle demuxer is left in
a random state.
This is actually similar to the normal seeking path, which has some
fallback code to handle this situation. Add such code to the subtitle
seeking path too.
(Normally, all demuxer support av_seek_frame(), except subtitles, which
support avformat_seek_file() only. The latter was meant to be the "new"
seeking API, but this never really took off, and using it normally seems
to cause worse seeking behavior. Or maybe we just use it incorrectly,
nobody really knows.)
Get rid of the 1-char subtitle type field. Use sh_stream->codec instead
just like audio and video do. Use codec names as defined by libavcodec
for simplicity, even if they're somewhat verbose and annoying.
Note that ffmpeg might switch to "ass" as codec name for ASS, so we
don't bother with the current silly "ssa" name.
MP_INPUT_BUFFER_PADDING_SIZE and FF_INPUT_BUFFER_PADDING_SIZE are both
16. The doxygen for FF_INPUT_BUFFER_PADDING_SIZE says only the first 23
bits must to be 0, but this is probably a lie.
This removes the stream handling mess by using a single list for all
stream types.
One consequence is that new streams are always set to AVDISCARD_ALL,
which could be an issue if packets are read before initializing other
streams. However, this doesn't seem to an issue for various reasons,
so we don't do anything about it.
The new code strictly assumes that libavformat never removes or
reorders streams once added to AVFormatContext->streams. Undefined
behavior will result if it does.
mkv_track_t now references sh_stream directly, instead of using an ID.
Also remove all accesses to demux_stream (demuxer->video etc.).
Remove some slave-mode things on the way, like "ID_SID_..." messages.
Some preparations to simplify demux_mkv and demux_lavf.
struct demux_stream manages state for each stream type that is being
demuxed (audio/video/sub). demux_stream is rather annoying, especially
the id and sh members, which are often used by the demuxers to determine
current stream and so on. Demuxers don't really have to access this,
except for testing whether a stream is selected and to add packets.
Add a new_sh_stream(), which allows creating streams without having the
caller specify any kind of stream ID. Demuxers should just use sh_stream
pointers, instead of multiple kinds of IDs and indexes.
Since demux_mkv queries the demuxer state when reading packets, track
switching is completely passive. Cycling etc. is done by the frontend.
As result, all track switching code can be removed.
Matroska files can contain multiple segments, which are literally
further Matroska files appended to the main file. They can be referenced
by segment linking.
While this is an extraordinarily useless and dumb feature, we support it
for the hell of it.
This is implemented by adding a further demuxer parameter for skipping
segments. When scanning for linked segments, each file is opened
multiple times, until there are no further segments found. Each segment
will have a separate demuxer instance (with a separate file handle
etc.).
It appears the Matroska spec. has an even worse feature for segments:
live streaming can completely reconfigure the stream by starting a new
segment. We won't add support for it, because there are 0 people on this
earth who think Matroska life streaming is a good idea. (As opposed to
serving Matroska/WebM files via HTTP.)
Matroska segment linking allows abusing Matroska files as playlists
without any actual video/audio/sub data, making files without any
clusters still useful for the frontend.
Relative seeks backwards didn't work too well with incomplete files, or
other files that are missing the seek index. The problem was that the
on-the-fly seek index generation simply added cluster positions as seek
entries. While this is perfectly fine, the seek code had no information
about the location of video key frames. For example, a 5 second long
cluster can have only 1 video key frame, which is located 4 seconds into
the cluster. Seeking backwards by one second while still located in the
same cluster would select this cluster as seek target again. Decoding
would resume with the key frame, giving the impression that seeking is
"stuck" at this frame.
Make the generated index aware of key frame and track information, so
that video can always be seeked in an idea way. This also uses the
normal block parsing code for indexing the clusters, instead of the
suspicious looking special code. (This code didn't parse the Matroska
elements correctly, but was fine for files with normal structure. Files
with corrupted clusters or clusters formatted for streaming were not
handled properly.)
Skipping is now quite a bit slower (takes about twice as long as
before), but it removes the special cased skipping code, and it's still
much faster (at least twice as fast) than libavformat. It needs to do
more I/O (no more skipping entire clusters, all data is read), and has
more CPU usage (more data needs to be parsed).
Move most code from demux_mkv_fill_buffer() to read_next_block(). The
former is supposed to read raw blocks, while ..fill_buffer() reads
blocks and turns them into packets.
Somehow this was setup such that a BlockGroup can be incrementally
read (at least in theory). This makes no sense, as BlockGroup can
contain only one Block (despite its name). There's no need to read
this incrementally, and makes the code confusing for no gain.
Read all the BlockGroup sub-elements with a single function call,
without keeping global state for BlockGroup parsing.
The code for reading block data was duplicated. Move it into a function.
Instead of returning on error (possibly due to corrupt data) and
signalling EOF, continue by trying to find the next block. This makes
error handling slightly simpler too, because you don't have to care
about freeing the current block. We could still signal EOF in this case,
but trying to resync sounds better for dealing with corrupted files.
Matroska files prepared for streaming have clusters with unknown size.
These files are pretty rare, see e.g. test4.mkv from the official
Matroska test file collection.
The end positions of the current cluster and block were managed by
tracking their size and how much of them were read, instead of just
using the absolute end positions.
I'm not sure about the reasons why this code was originally written
this way. One obvious concern is reading from pipes and such, but the
stream layers hides this. stream_tell(s) works even when reading from
pipes. It's also a fast call, and doesn't involve the stream
implementation or syscalls. Keeping track of the cluster/block end is
simpler and there's no reason why this wouldn't work.
