In insane files with a very huge number of subtitle events, and if the
--demuxer-mkv-subtitle-preroll option is given, seeking can still
overflow the packet queue. Normally, the subtitle_preroll variable
specifies the maximum number of packets that can be added. But once this
number is reached, the normal seeking behavior is enabled, which will
add all subtitle packets with the right timestamps to the packet queue.
At this point the next video keyframe can still be quite far away, with
enough subtitle packets on the way to overflow the packet queue.
Fix this by always setting an upper limit of subtitle packets read
during seeking. This should provide additional robustness even if the
preroll option is not used.
This means that even with normal seeking, at most 500 subtitle packets
are demuxed. Packets after that are discarded.
One slightly questionable aspect of this commit is that subtitle_preroll
is never reset in audio-only mode, but that is probably ok.
Retrieve per-chapter metadata, but don't do much with it. We just make
the metadata of the _current_ chapter available as chapter-metadata
property. Returning the full chapter list with metadata would be no
problem, except that the property interface isn't really good with
structured data, so it's not available for now.
Not sure if it's worth it, but it was requested via github issue #201.
Consider the cluster used for prerolling contains an insane amount of
subtitle packets. Then the demuxer packet queue would be full of
subtitle packets, and demux.c would refuse to read any further packets -
including video and audio packets, resulting in EOF. Since everything
involving Matroska and subtitles is 100% insane, this can actually
happen.
Fix this by putting a limit on the number of subtitle packets read by
preroll, and throw away any further packets if the limit is exceeded. If
this happens, the preroll mechanism will stop working, but the player's
operation is unaffected otherwise.
The way this was added to FFmpeg is less than ideal, because it requires
text parsing in the Matroska demuxer. But in order to use the FFmpeg
webvtt-to-ass converter, we still have to mimic this in some way. We do
this by putting the parsing into sd_lavc_conv.c, before the subtitle
packet is passed to libavcodec. At least this keeps the ugliness out of
unrelated code.
There is some change that FFmpeg will fix their design eventually.
Instead of rewriting the parsing code, we simply borrow it from FFmpeg's
Matroska demuxer.
Originally, the objective of this commit was changing --edition to be
1-based, but this was cancelled. I'm still leaving the change to
demux_mkv.c though, which is now only of cosmetic nature.
In general, this warning can hint to actual bugs. We don't enable it
yet, because it would conflict with some unmerged code, and we should
check with clang too (this commit was done by testing with gcc).
This fixes the sample RA_missing_timestamps.mkv. Pretty funny how this
code got it almost right, but not quite, so it was broken all these
years. And then, after everyone stopped caring, someone comes and fixes
it. (By the way, I know absolutely nothing about realaudio.)
This fixes playback of the sample linked by FFmpeg ticket 2508. The fix
follows ffmpeg commit 6158a3b (although it's not exactly the same).
The problem here is that the file contains an apparently non-sense
DefaultDuration value. DefaultDuration for audio tracks is used to
derive PTS values for packets with no timestamps, like they can happen
with frames inside a laced block. So the first packet of a SimpleBlock
will have a correct PTS, while the PTS values of the following packets
are calculated using DefaultDuration, and thus are broken.
This leads to seemingly ok playback, but broken A/V sync. Not using the
DefaultDuration value will leave the PTS values of these packets unset,
and the audio decoder can derive them from the output instead.
The fix more or less uses a heuristic to detect the broken case: if the
sample rate is 8 KHz (Matroska default, can assume unset), and the codec
is AC3 (as the broken file did), don't use it. I'm not sure why this
should be done only for AC3, maybe the muxing application (mkvmerge
v4.9.1) has known issues with AC3. AC3 also doesn't support 8 KHz as
sample rate natively.
(By the way, I'm not sure why we should honor the DefaultDuration at all
for audio. It doesn't seem to be needed. You can't seek to these frames,
and decoders should always be able to produce perfect PTS values by
adding the duration of the decoded audio to the first PTS.)
Matroska has an output sample rate (OutputSamplingFrequency), which in
theory should be forced instead of whatever the decoder outputs. But it
appears no software (other than mplayer2 and mpv until now) actually
respects this. Even worse, there were broken files around, which played
correctly with (in theory) broken software, but not mplayer2/mpv. Hacks
were added to our code to play these files correctly, but they didn't
catch all cases.
Simplify this by doing what everyone else does, and always use the
decoder's sample rate instead. In particular, we try to handle all
sample rate issues like libavformat's Matroska demuxer does.
All demuxers make a reasonable effort to set packet timestamps, and thus
support correct-pts mode. This commit also implicitly switches
demux_rawvideo to correct-pts mode.
We still allow demuxers to disable correct-pts mode in theory.
Get rid of the strange and messy reliance on DEMUXER_TYPE_ constants.
Instead of having two open functions for the demuxer callbacks (which
somehow are both optional, but you can also decide to implement both...),
just have one function. This function takes a parameter that tells the
demuxer how strictly it should check for the file headers. This is a
nice simplification and allows more flexibility.
Remove the file extension code. This literally did nothing (anymore).
Change demux_lavf so that we check our other builtin demuxers first
before libavformat tries to guess by file extension.
Move codec_tags.h include to demux_mkv.c, because this is the only file
which still uses it.
Move new_sh_stream() to demux.h, because this is more proper.
Generally remove all accesses to demux_stream from all the code, except
inside of demux.c. Make it completely private to demux.c.
This simplifies the code because it removes an extra concept. In demux.c
it is reduced to a simple packet queue. There were other uses of
demux_stream, but they were removed or are removed with this commit.
Remove the extra "ds" argument to demux fill_buffer callback. It was
used by demux_avi and the TV pseudo-demuxer only.
Remove usage of d_video->last_pts from the no-correct-pts code. This
field contains the last PTS retrieved after a packet that is not NOPTS.
We can easily get this value manually because we read the packets
ourselves. Reuse sh_video->last_pts to store the packet PTS values. It
was used only by the correct-pts code before, and like d_video->last_pts,
it is reset on seek. The behavior should be exactly the same.
These separate arrays were used by the old demuxers and are not needed
anymore. We can simplify track switching as well.
One interesting thing is that stream/tv.c (which is a demuxer) won't
respect --no-audio anymore. It will probably work as expected, but it
will still open an audio device etc. - this is because track selection
is now always done with the runtime track switching mechanism. Maybe
the TV code could be updated to do proper runtime switching, but I
can't test this stuff.
Delete demux_avi, demux_asf, demux_mpg, demux_ts. libavformat does
better than them (except in rare corner cases), and the demuxers have
a bad influence on the rest of the code. Often they don't output
proper packets, and require additional audio and video parsing. Most
work only in --no-correct-pts mode.
Remove them to facilitate further cleanups.
The new wavpack packet format (see previous commit) doesn't work with
older libavcodec versions, so disable the new code in this case.
The version numbers are only approximate, since the libavcodec version
wasn't bumped with the wavpack change, but it's close enough.
Libav introduced a silent API breakage by changing what wavpack packets
the libavcodec decoder accepts. Originally the libavcodec codec accepted
Matroska-style wavpack packets. Libav commit 9b6f47c removed this
capability from the libavcodec code, and added code to libavformat's
Matroska demuxer to "rearrange" wavpack packets. Since demux_mkv still
sent Matroska-style packets, playback failed.
Fix this by "rearranging" packets in demux_mkv as well by copying
libavformat's code. (The best kind of fix.)
Tested with [CCCP]_Mega_Lossless_Audio_Test.mkv, as well as with a
sample generated by mkvmerge.
Playing Youtube videos often requires an additional seek to the end of
the file. This flushes the stream cache. The reason for the seek is
reading the cues (seek index). This poses the question why Google is
muxing its files in such a way, since nothing in Matroska mandates that
cues are located at the end of the file, but we want to handle this
situation better anyway.
The seek index is not needed for normal playback, only for seeking.
This commit changes header parsing such that the index is not read on
initialization in order to avoid the additional stream-level seek.
Instead, read the index on the first demuxer-level seek, when the seek
index is actually needed.
If the cues are at the beginning of the file, they are read immediately
as part of the normal header reading process. This commit changes
behavior only if cues are outside of the header (i.e. not in the area
between EBML header and clusters), and linked by a SeekHead. Other
level 1 elements linked by the SeekHead might still cause seeks to the
end of the file, although that seems to be rare.
Before this commit, the demuxer would in theory accept multiple cues
elements (and append its contents to the index in the order as
encountered during reading). According to the Matroska specification,
there can be only one cues element in the segment, so this seems like
an overcomplication.
Change it so that redundant elements are ignored, like with all other
unique header elements. This makes implementing deferred reading of the
cues element easier.
Nobody uses this, and this is an absolute waste of time. Even the user
who reported this turned out to have produced a sample manually.
Sample produced with:
wget http://diracvideo.org/download/test-streams/raw/vts/vts.LD-8Mb.drc
mkvmerge -o dirac.mkv vts.LD-8Mb.drc
mkvmerge writes a sort of broken aspect ratio. libavformat interprets it
as 1:1 PAR, while demux_mkv thinks this is a 1:1 DAR. Maybe libavformat
is more correct here.
This helps passing the channel layout correctly from decoder to audio
filter chain. (Because that part "reuses" the demuxer level codec
parameters, which is very disgusting.)
Note that ffmpeg stuff already passed the channel layout via
mp_copy_lav_codec_headers(). So other than easier dealing with the
demuxer/decoder parameters mess, there's no real advantage to doing
this.
Make the --channels option accept a channel map. Since simple numbers
map to standard layouts with the given number of channels, this is
downwards compatible. Likewise for demux_rawaudio.
These were found by the cppcheck and scan-build static analyzers. Most
of these aren't interesting (the 2 previous commits fix some interesting
cases found by these analyzers), and they don't nearly fix all warnings.
(Most of the unfixed warnings are spam, things MPlayer never cared
about, or false positives.)
This check was always false:
if (num == EBML_UINT_INVALID)
Fix it by using the proper type for the num variable.
This case actually doesn't really matter, and this is just for hiding
the warning and for being 100% correct.
Get rid of the 1-char subtitle type field. Use sh_stream->codec instead
just like audio and video do. Use codec names as defined by libavcodec
for simplicity, even if they're somewhat verbose and annoying.
Note that ffmpeg might switch to "ass" as codec name for ASS, so we
don't bother with the current silly "ssa" name.
mkv_track_t now references sh_stream directly, instead of using an ID.
Also remove all accesses to demux_stream (demuxer->video etc.).
Remove some slave-mode things on the way, like "ID_SID_..." messages.
Since demux_mkv queries the demuxer state when reading packets, track
switching is completely passive. Cycling etc. is done by the frontend.
As result, all track switching code can be removed.
Matroska files can contain multiple segments, which are literally
further Matroska files appended to the main file. They can be referenced
by segment linking.
While this is an extraordinarily useless and dumb feature, we support it
for the hell of it.
This is implemented by adding a further demuxer parameter for skipping
segments. When scanning for linked segments, each file is opened
multiple times, until there are no further segments found. Each segment
will have a separate demuxer instance (with a separate file handle
etc.).
It appears the Matroska spec. has an even worse feature for segments:
live streaming can completely reconfigure the stream by starting a new
segment. We won't add support for it, because there are 0 people on this
earth who think Matroska life streaming is a good idea. (As opposed to
serving Matroska/WebM files via HTTP.)
Matroska segment linking allows abusing Matroska files as playlists
without any actual video/audio/sub data, making files without any
clusters still useful for the frontend.
Relative seeks backwards didn't work too well with incomplete files, or
other files that are missing the seek index. The problem was that the
on-the-fly seek index generation simply added cluster positions as seek
entries. While this is perfectly fine, the seek code had no information
about the location of video key frames. For example, a 5 second long
cluster can have only 1 video key frame, which is located 4 seconds into
the cluster. Seeking backwards by one second while still located in the
same cluster would select this cluster as seek target again. Decoding
would resume with the key frame, giving the impression that seeking is
"stuck" at this frame.
Make the generated index aware of key frame and track information, so
that video can always be seeked in an idea way. This also uses the
normal block parsing code for indexing the clusters, instead of the
suspicious looking special code. (This code didn't parse the Matroska
elements correctly, but was fine for files with normal structure. Files
with corrupted clusters or clusters formatted for streaming were not
handled properly.)
Skipping is now quite a bit slower (takes about twice as long as
before), but it removes the special cased skipping code, and it's still
much faster (at least twice as fast) than libavformat. It needs to do
more I/O (no more skipping entire clusters, all data is read), and has
more CPU usage (more data needs to be parsed).
Move most code from demux_mkv_fill_buffer() to read_next_block(). The
former is supposed to read raw blocks, while ..fill_buffer() reads
blocks and turns them into packets.
Somehow this was setup such that a BlockGroup can be incrementally
read (at least in theory). This makes no sense, as BlockGroup can
contain only one Block (despite its name). There's no need to read
this incrementally, and makes the code confusing for no gain.
Read all the BlockGroup sub-elements with a single function call,
without keeping global state for BlockGroup parsing.
The code for reading block data was duplicated. Move it into a function.
Instead of returning on error (possibly due to corrupt data) and
signalling EOF, continue by trying to find the next block. This makes
error handling slightly simpler too, because you don't have to care
about freeing the current block. We could still signal EOF in this case,
but trying to resync sounds better for dealing with corrupted files.
Matroska files prepared for streaming have clusters with unknown size.
These files are pretty rare, see e.g. test4.mkv from the official
Matroska test file collection.
The end positions of the current cluster and block were managed by
tracking their size and how much of them were read, instead of just
using the absolute end positions.
I'm not sure about the reasons why this code was originally written
this way. One obvious concern is reading from pipes and such, but the
stream layers hides this. stream_tell(s) works even when reading from
pipes. It's also a fast call, and doesn't involve the stream
implementation or syscalls. Keeping track of the cluster/block end is
simpler and there's no reason why this wouldn't work.
Incomplete files don't have a valid index, because the index is usually
located near the end of a file. In this case, an index is created on the
fly during demuxing, or when seeks are done.
This used a completely different code path, which leads to unnecessary
complications and code duplication. Use the normal index data structure
instead. The seeking code at the end of seek_creating_index() (in this
commit renamed to create_index_until()) is removed. The normal seek code
does the same thing instead.
No subtitle selected was supposed to disable the preroll logic
completely. However, the packet skipping logic was not properly enabled,
so the demuxer would still return subtitle packets from before the seek
target timecode. This shouldn't matter at all in practice, but fixing
this makes the code clearer.
Makes sure that seeking to a given time position shows the subtitle at
that position. This can fail if the subtitle packet is not close enough
to the seek target. Always enabled for hr-seeks, and can be manually
enabled for normal seeks with --mkv-subtitle-preroll.
This helps displaying subtitles correctly with ordered chapters. When
switching ordered chapter segments, a seek is performed. If the subtitle
is timed slightly before the start of the segment, it normally won't be
demuxed. This is a problem with all seeks, but in this case normal
playback is affected. Since switching segments always uses hr-seeks,
the code added by this commit is always active in this situation.
If no subtitles are selected or the subtitles come from an external
file, the demuxer should behave exactly as before this commit.
Commit 546ae23 fixed aspect ratio if the DisplayWidth or DisplayHeight
elements were missing. However, some bogus files [1] can have these
elements present in the file, but set to 0. Use 1:1 pixel aspect for
such files.
[1] https://ffmpeg.org/trac/ffmpeg/ticket/2424
Commit ac1c5e6 (demux_mkv: improve robustness against broken files)
added code to skip to the next cluster on error conditions. However,
reaching normal EOF triggers this code as well, so explicitly check
for EOF before this happens. Note that the EOF flag is only set _after_
reading the last byte, so EOF needs to be checked after the fact. (Or
in other words, we must check for EOF after the ebml_read_id() call.)
(To answer the question why reading packets actually reaches EOF, even
if there's the seek index between the last packet and the end of the
file: the cluster reading code skips the seeking related EBML elements
as normal part of operation, so it hits EOF gracefully when trying to
find the next cluster.)
Fixes test7.mkv from the Matroska test file collection, as well as some
real broken files I've found in the wild. (Unfortunately, true recovery
requires resetting the decoders and playback state with a manual seek,
but it's still better than just exiting.)
If there are broken EBML elements, try harder to skip them correctly.
Do this by searching for the next cluster element. The cluster element
intentionally has a long ID, so it's a suitable element for
resynchronizing (mkvmerge does something similar).
We know that data is corrupt if the ID or length fields of an element
are malformed. Additionally, if skipping an unknown element goes past
the end of the file, we assume it's corrupt and undo the seek. Do this
because it often happens that corrupt data is interpreted as correct
EBML elements. Since these elements will have a ridiculous values in
their length fields due to the large value range that is possible
(0-2^56-2), they will go past the end of the file. So instead of
skipping them (which would result in playback termination), try to
find the next cluster instead. (We still skip unknown elements that
are within the file, as this is needed for correct operation. Also, we
first execute the seek, because we don't really know where the file
ends. Doing it this way is better for unseekable streams too, because
it will still work in the non-error case.)
This is done as special case in the packet reading function only. On
the other hand, that's the only part of the file that's read after
initialization is done.
Fixes test4.mkv from the Matroska test file collection.
demux_mkv_open() contains a loop that reads header elements. It starts
by reading the EBML element ID with ebml_read_id(). If there is broken
data in the header, ebml_read_id() might return EBML_ID_INVALID.
However, that is not handled specially, and the code for handling
unknown tags is invoked. This reads the EBML element length in order to
skip data, which, if the EBML ID is broken, is entirely random. This
caused a seek beyond the end of the file, making the demuxer fail.
So don't skip any data if the EBML ID was invalid, and simply try to
read the next element. ebml_read_id() reads at least one byte, so the
parsing loop won't get stuck.
All in all this is rather questionable, but since this affects error
situations only, makes behavior a bit more robust (no random seeks), and
actually fixes at least one sample, it's ok.
libavformat's demuxer handled this.
FFmpeg recently changed how it writes Opus-in-Matroska to match
the A_OPUS/EXPERIMENTAL name that mkvmerge uses, with the caveat
that things will change and compatibility with old files can get
worked out when the spec is finalized.
This adds both A_OPUS and A_OPUS/EXPERIMENTAL so that *hopefully*
it can play both the newer files that use A_OPUS/EXPERIMENTAL, and
older ones muxed by FFmpeg that were simply A_OPUS, since this is
also what FFmpeg seems to be doing to handle the situation.
The percent position is used for the OSD, the status line, and for the
OSD bar (shown on seeks). By default, the PTS of the last demuxed packet
was used to calculate it. This led to a "jumpy" display when the
percentage value (casted to int) was changing. The reasons for this were
the presence of video frame reordering (packet PTS is not monotonic), or
getting PTS values from different streams (like audio/subs).
Since these rely on PTS values and correct file durations anyway,
simplify it by calculating it with the current playback position in
mplayer.c instead.
Also move the lang field into the general stream header. (SH_COMMON is
an old hack to "share" code between audio/video/sub headers.)
There should be no functional changes, other than not printing stream
info in verbose mode or with slave mode. (The frontend already prints
stream info, and this is just a leftover when individual demuxers did
this, and slave mode remains broken.)
Use codec names instead of FourCCs to identify codecs. Rewrite how
codecs are selected and initialized. Now each decoder exports a list
of decoders (and the codec it supports) via add_decoders(). The order
matters, and the first decoder for a given decoder is preferred over
the other decoders. E.g. all ad_mpg123 decoders are preferred over
ad_lavc, because it comes first in the mpcodecs_ad_drivers array.
Likewise, decoders within ad_lavc that are enumerated first by
libavcodec (using av_codec_next()) are preferred. (This is actually
critical to select h264 software decoding by default instead of vdpau.
libavcodec and ffmpeg/avconv use the same method to select decoders by
default, so we hope this is sane.)
The codec names follow libavcodec's codec names as defined by
AVCodecDescriptor.name (see libavcodec/codec_desc.c). Some decoders
have names different from the canonical codec name. The AVCodecDescriptor
API is relatively new, so we need a compatibility layer for older
libavcodec versions for codec names that are referenced internally,
and which are different from the decoder name. (Add a configure check
for that, because checking versions is getting way too messy.)
demux/codec_tags.c is generated from the former codecs.conf (minus
"special" decoders like vdpau, and excluding the mappings that are the
same as the mappings libavformat's exported RIFF tables). It contains
all the mappings from FourCCs to codec name. This is needed for
demux_mkv, demux_mpg, demux_avi and demux_asf. demux_lavf will set the
codec as determined by libavformat, while the other demuxers have to do
this on their own, using the mp_set_audio/video_codec_from_tag()
functions. Note that the sh_audio/video->format members don't uniquely
identify the codec anymore, and sh->codec takes over this role.
Replace the --ac/--vc/--afm/--vfm with new --vd/--ad options, which
provide cover the functionality of the removed switched.
Note: there's no CODECS_FLAG_FLIP flag anymore. This means some obscure
container/video combinations (e.g. the sample Film_200_zygo_pro.mov)
are played flipped. ffplay/avplay doesn't handle this properly either,
so we don't care and blame ffmeg/libav instead.
Select the generic raw video decoder in codecs.cfg ("MPrv" FourCC),
which forces the generic lavc raw video decoder "rawvideo". This means
all FourCCs understood by lavc rawvideo are supported, not just whatever
has codecs.cfg entries.
Something produces corrupt Matroska files with audio tracks that have
SamplingFrequency set to 44100 and OutputSamplingFrequency to 96000,
when the correct playback rate is 44100. Add a special case for this
44100/96000 combination and override it to 44100/44100; it's unlikely
that anyone would ever want to use this 44100/96000 combination for
real in valid files.
Reinitialize sh_audio->samplesize and sample_format before falling back
to another audio decoder (some decoders rely on default values). Remove
code setting these fields from demux_mkv and demux_lavf (no decoder
should depend on demuxer-set values for these fields).
Conflicts:
audio/decode/ad_lavc.c
Merged from mplayer2 commit 6b9567. The changes to ad_lavc.c are not
merged, as they are very specific to the mplayer2 libavresample hack;
we deplanarize manually, so we can't get unsupported sample formats
yet (except on raw audio with "pcm_f64le", as we don't support
AV_SAMPLE_FMT_DBL in the audio chain).
Finish renaming directories and moving files. Adjust all include
statements to make the previous commit compile.
The two commits are separate, because git is bad at tracking renames
and content changes at the same time.
Also take this as an opportunity to remove the separation between
"common" and "mplayer" sources in the Makefile. ("common" used to be
shared between mplayer and mencoder.)
Tis drops the silly lib prefixes, and attempts to organize the tree in
a more logical way. Make the top-level directory less cluttered as
well.
Renames the following directories:
libaf -> audio/filter
libao2 -> audio/out
libvo -> video/out
libmpdemux -> demux
Split libmpcodecs:
vf* -> video/filter
vd*, dec_video.* -> video/decode
mp_image*, img_format*, ... -> video/
ad*, dec_audio.* -> audio/decode
libaf/format.* is moved to audio/ - this is similar to how mp_image.*
is located in video/.
Move most top-level .c/.h files to core. (talloc.c/.h is left on top-
level, because it's external.) Park some of the more annoying files
in compat/. Some of these are relicts from the time mplayer used
ffmpeg internals.
sub/ is not split, because it's too much of a mess (subtitle code is
mixed with OSD display and rendering).
Maybe the organization of core is not ideal: it mixes playback core
(like mplayer.c) and utility helpers (like bstr.c/h). Should the need
arise, the playback core will be moved somewhere else, while core
contains all helper and common code.
wm4
Stefano Pigozzi
Bin Jin
eng
Stephen Hutchinson
Uoti Urpala