This is supposed to help making data flow easier and wakeup handling
more efficient. Once that change is done, reading a packet on any
stream won't have to wakeup and poll all decoders (which helps reducing
the mess even if all decoders are on the same thread).
This also improves the accuracy of wakeups by tracking better whether
a wakeup is needed.
If you play a video with an external audio track, and do backwards
keyframe seeks, then audio can be missing. This is because a backwards
seek can end up way before the seek target (this is just how this seek
mode works). The audio file will be seeked at the correct seek target
(since audio usually has a much higher seek granularity), which results
in silence being played until the video reaches the originally intended
seek target.
There was a hack in audio.c to deal with this. Replace it with a
different hack. The new hack probably works about as well as the old
hack, except it doesn't add weird crap to the audio resync path (which
is some of the worst code here, so this is some nice preparation for
rewriting it). As a more practical advantage, it doesn't discard the
audio demuxer packet cache. The old code did, which probably ruined
seeking in youtube DASH streams.
A non-hacky solution would be handling external files in the demuxer
layer. Then chaining the seeks would be pretty easy. But we're pretty
far from that, because it would either require intrusive changes to the
demuxer layer, or wouldn't be flexible enough to load/unload external
files at runtime. Maybe later.
Export them as explicitly undocumented debugging fields for the
"demuxer-cache-state" property.
Should be somewhat helpful to debug "wtf is the demuxer" doing
situations better, especially when seeking. It also becomes visible how
long the demuxer is blocked on an "old" seek when you keep seeking while
the first seek hasn't finished.
This will help with things like livestreams.
As a minor detail, subtitles are excluded, because they sometimes have
"unused" events after video and audio ends. To avoid this annoying
corner case, just ignore them.
Before this change and before the seekable stream cache became a thing,
we could possibly seek using the stream cache. But we couldn't know
whether the seek would succeed. We knew the available byte range, but
could in general not tell whether a demuxer would stay within the range
when trying to seek to a specific time position. We preferred to have
safe defaults, so seeking in streams that were detected as unseekable
were not honored. We allowed overriding this via --force-seekable=yes,
in which case it depended on your luck whether the seek would work, or
the player crapped its pants.
With the demuxer packet cache, we can tell exactly whether a seek will
work (at least if there's only 1 seek range). We can just let seeks go
through. Everything to allow this is already in place, and this commit
just moves around some minor things.
Note that the demux_seek() return value was not used before, because low
level (i.e. network level) seeks are usually asynchronous, and if they
fail, the state is pretty much undefined. We simply repurpose the return
value to signal whether cache seeking worked. If it didn't, we can just
resume playback normally, because demuxing continues unaffected, and no
decoder are reset.
This should be particularly helpful to people who for some reason stream
data into stdin via streamlink and such.
It seems like there's nothing stopping from sub-demuxers from keeping
packets in the cache, even if it's completely pointless. The top-most
demuxer (demux_timeline) already takes care of caching, so sub-demuxers
only waste space and time with this.
Add a function that can disable the packet cache even at runtime and
after packets are read. (It's not clear whether it really can happen
that packets are read before demux_timeline gets the sub-demuxers, but
there's no reason to make it too fragile.) Call it on all sub-demuxers.
For this to work, it seems we have to move the code for setting the
seekable_cache flag to before demux_timeline is potentially initialized,
because otherwise the cache would be reenabled if the demuxer triggering
timeline support is a timeline segment itself (e.g. ordered chapters).
Until now, the demuxer cache was limited to a single range. Extend this
to multiple range. Should be useful for slow network streams.
This commit changes a lot in the internal demuxer cache logic, so
there's a lot of room for bugs and regressions. The logic without
demuxer cache is mostly untouched, but also involved with the code
changes. Or in other words, this commit probably fucks up shit.
There are two things which makes multiple cached ranges rather hard:
1. the need to resume the demuxer at the end of a cached range when
seeking to it
2. joining two adjacent ranges when the lowe range "grows" into it (and
resuming the demuxer at the end of the new joined range)
"Resuming" the demuxer means that we perform a low level seek to the end
of a cached range, and properly append new packets to it, without adding
packets multiple times or creating holes due to missing packets.
Since audio and video never line up exactly, there is no clean "cut"
possible, at which you could resume the demuxer cleanly (for 1.) or
which you could use to detect that two ranges are perfectly adjacent
(for 2.). The way how the demuxer interleaves multiple streams is also
unpredictable. Typically you will have to expect that it randomly allows
one of the streams to be ahead by a bit, and so on.
To deal with this, we have heuristics in place to detect when one packet
equals or is "behind" a packet that was demuxed earlier. We reuse the
refresh seek logic (used to "reread" packets into the demuxer cache when
enabling a track), which checks for certain packet invariants.
Currently, it observes whether either the raw packet position, or the
packet DTS is strictly monotonically increasing. If none of them are
true, we discard old ranges when creating a new one.
This heavily depends on the file format and the demuxer behavior. For
example, not all file formats have DTS, and the packet position can be
unset due to libavformat not always setting it (e.g. when parsers are
used).
At the same time, we must deal with all the complicated state used to
track prefetching and seek ranges. In some complicated corner cases, we
just give up and discard other seek ranges, even if the previously
mentioned packet invariants are fulfilled.
To handle joining, we're being particularly dumb, and require a small
overlap to be confident that two ranges join perfectly. (This could be
done incrementally with as little overlap as 1 packet, but corner cases
would eat us: each stream needs to be joined separately, and the cache
pruning logic could remove overlapping packets for other streams again.)
Another restriction is that switching the cached range will always
trigger an asynchronous low level seek to resume demuxing at the new
range. Some users might find this annoying.
Dealing with interleaved subtitles is not fully handled yet. It will
clamp the seekable range to where subtitle packets are.
Even though only 1 seek range is supported at the time.
Other than preparation for possibly future features, the main gain is
actually that we finally separate the reporting for the buffering, and
the seek ranges. These can be subtly different, so it's good to have a
clear separation.
This commit also fixes that the ts_reader wasn't rebased to the start
time, which could make the player show "???" for buffered cache amount
in some .ts files and others (especially at the end, when ts_reader
could become higher than ts_max). It also fixes writing the cache-end
field in the demuxer-cache-state property: it checked ts_start against
NOPTS, which makes no sense.
ts_start was never used (except for the bug mentioned above), so get rid
of it completely. This also makes it convenient to move the segment
check for last_ts to the demux_add_packet() function.
Seems like most code dealing with this was for setting it in redundant
cases. Now SEEK_BACKWARD is redundant, and SEEK_FORWARD is the odd one
out.
Also fix that SEEK_FORWARD was not correctly unset in try_seek_cache().
In demux_mkv_seek(), make the arbitrary decision that a video stream is
not required for the subtitle prefetch logic to be active. We might want
subtitles with long duration even with audio only playback, or if the
file is used as external subtitle.
This improves upon the previous commit, and partially rewrites it (and
other code). It does:
- disable the seeking within cache by default, and add an option to
control it
- mess with the buffer estimation reporting code, which will most likely
lead to funny regressions even if the new features are not enabled
- add a back buffer to the packet cache
- enhance the seek code so you can seek into the back buffer
- unnecessarily change a bunch of other stuff for no reason
- fuck up everything and vomit ponies and rainbows
This should actually be pretty usable. One thing we should add are some
properties to report the proper buffer state. Then the OSC could show a
nice buffer range. Also configuration of the buffers could be made
simpler. Once this has been tested enough, it can be enabled by default,
and might replace the stream cache's byte ringbuffer.
In addition it may or may not be possible to keep other buffer ranges
when seeking outside of the current range, but that would be much more
complex.
As usual, the history of these files is a bit murky. It starts with the
initial commit. (At which some development had already been done,
according to the AUTHORS and ChangeLog files at the time, we should be
but covered with relicensing agreements, though.) then it goes on with
complete lack of modularization, which was cleaned up later (cd68e161).
As usual, we don't consider the copyright of the stuff that has been
moved out cleanly.
There were also contributions to generic code by people who could not be
reached or who did not agree to the relicensing, but this was all
removed.
The only patches that we could not relicense and which were still in the
current code in some form are from Dénes Balatoni: 422b0d2a, 32937181.
We could not reach him, so commits f34e1a0d and 18905298 remove his
additions. It still leaves the demux_control() declaration itself, but
we don't consider it copyrightable. It's basically an idiom that existed
in MPlayer before that change, applied to the demuxer struct. (We even
went as far as making sure to remove all DEMUXER_CTRLs the original
author added.)
Commit be54f481 might be a bit of a corner case, but this was rewritten,
and we consider the old copyright removed long ago.
Similar purpose as f34e1a0dee.
Somehow this is much more natural too, and needs less code.
This breaks runtime updates to duration. This could easily be fixed, but
no important demuxer does this anyway. Only demux_raw and demux_disc
might (the latter for BD/DVD). For the latter it might actually have
some importance when changing titles at runtime (I guess?), but guess
what, I don't care.
This is more uniform, and potentially gets rid of some past copyrights.
It might be that this subtly changes caching behavior (it seems before
this, it synced to the demuxer if the length was unknown, which is not
what we want.)
We use the metadata provided by youtube-dl to sort-of implement
fragmented DASH streaming.
This is all a bit hacky, but hopefully a makeshift solution until
libavformat has proper mechanisms. (Although in danger of being one
of those temporary hacks that become permanent.)
This was excessively useless, and I want my time back that was needed to
explain users why they don't want to use it.
It captured the byte stream only, and even for types of streams it was
designed for (like transport streams), it was rather questionable.
As part of the removal, un-inline demux_run_on_thread() (which has only
1 call-site now), and sort of reimplement --stream-dump to write the
data directly instead of using the removed capture code.
(--stream-dump is also very useless, and I struggled coming up with an
explanation for it in the manpage.)
Disabling cache readahead by default until at least 1 track is selected
is mainly for external files and such, where you don't want them to use
up resources until they're actually used.
It doesn't make sense to disable the cache for the demuxer opened for
prefetch. Also, it's fine to let it do that for the main file too (doing
or not doing it is of little consequence). That saves us from having to
distinguish them.
Don't access MPOpts directly, and always use the new m_config.h
functions for accessing them in a thread-safe way.
The goal is eventually removing the mpv_global.opts field, and the
demuxer/stream-layer specific hack that copies MPOpts to deal with
thread-safety issues.
This moves around a lot of options. For one, we often change the
physical storage location of options to make them more localized,
but these changes are not user-visible (or should not be). For
shared options on the other hand it's better to do messy direct
access, which is worrying as in that somehow renaming an option
or changing its type would break code reading them manually,
without causing a compilation error.
This is for text subtitles. libavformat currently always reads text
subtitles completely on init. This means the underlying stream is
useless and will consume resources for various reasons (network
connection, file handles, cache memory).
Take care of this by closing the underlying stream if we think the
demuxer has read everything. Since libavformat doesn't export whether it
did (or whether it may access the stream again in the future), we rely
on a whitelist. Also, instead of setting the stream to NULL or so, set
it to an empty dummy stream. This way we don't have to litter the code
with NULL checks.
demux_lavf.c needs extra changes, because it tries to do clever things
for the sake of subtitle charset conversion.
The main reason we keep the demuxer etc. open is because we fell for
libavformat being so generic, and we tried to remove corresponding
special-cases in the higher-level player code. Some of this is forced
due to ass/srt mkv/mp4 demuxing being very similar to external text
files. In the future it might be better to do this in a more
straight-forward way, such as reading text subtitles into libass and
then discarding the demuxer entirely, but for aforementioned reasons
this could be more of a mess than the solution introduced by this
commit.
Probably fixes#3456.
Cleaner and makes it easier to change the underlying stream.
mp_property_stream_capture() still directly accesses it directly via
demux_run_on_thread(). This is evil, but still somewhat sane and is not
getting into the way here.
Not sure if I got all field accesses.
...and ignore it. The main purpose is for retrieving per-track
replaygain tags. Other than that per-track tags are not used or accessed
by the playback core yet.
The demuxer infrastructure is still not really good with that whole
synchronization thing (at least in part due to being inherited from
mplayer's single-threaded architecture). A convoluted mechanism is
needed to transport the tags from demuxer thread to user thread. Two
factors contribute to the complexity: tags can change during playback,
and tracks (i.e. struct sh_stream) are not duplicated per thread.
In particular, we update the way replaygain tags are retrieved. We first
try to use per-track tags (common in Matroska) and global tags
(effectively formats like mp3). This part fixes#3405.
Remove the explicit whitelisting of formats for refresh seeks. Instead,
check whether the packet position is somewhat reliable during demuxing.
If there are packets without position, or the packet position is not
monotonically increasing, then do not use them for refresh seeks.
This does not make sure of some requirements, such as deterministic
seeks. If that happens, mpv will mess up a bit on stream switching.
Also, add another method that uses DTS to identify packets, and prefer
it to the packet position method. Even if there's a demuxer which
randomizes packet positions, it hardly can do that with DTS. The DTS
method is not always available either, though. Some formats do not have
a DTS, and others are not always strictly monotonic (possibly due to
libavformat codec parsing and timestamp determination issues).
When switching tracks, we normally have the problem that data gets lost
due to readahead buffering. (Which in turn is because we're stubborn and
instruct the demuxers to discard data on unselected streams.) The
demuxer layer has a hack that re-reads discarded buffered data if a
stream is enabled mid-stream, so track switching will seem instant.
A somewhat similar problem is when all tracks of an external files were
disabled - when enabling the first track, we have to seek to the target
position.
Handle these with the same mechanism. Pass the "current time" to the
demuxer's stream switch function, and let the demuxer figure out what to
do. The demuxer will issue a refresh seek (if possible) to update the
new stream, or will issue a "normal" seek if there was no active stream
yet.
One case that changes is when a video/audio stream is enabled on an
external file with only a subtitle stream active, and the demuxer does
not support rrefresh seeks. This is a fuzzy case, because subtitles are
sparse, and the demuxer might have skipped large amounts of data. We
used to seek (and send the subtitle decoder some subtitle packets
twice). This case is sort of obscure and insane, and the fix would be
questionable, so we simply don't care.
Should mostly fix#3392.
This pause stuff is bothersome and is needed only for a few corner-
cases. This commit removes it from the demuxer public API and replaces
it with a demux_run_on_thread() function and refactors the code which
needed demux_pause(). The next commit will change the implementation.
Ever since a change in mplayer2 or so, relative seeks were translated to
absolute seeks before sending them to the demuxer in most cases. The
only exception in current mpv is DVD seeking.
Remove the SEEK_ABSOLUTE flag; it's not the implied default. SEEK_FACTOR
is kept, because it's sometimes slightly useful for seeking in things
like transport streams. (And maybe mkv files without duration set?)
DVD seeking is terrible because DVD and libdvdnav are terrible, but
mostly because libdvdnav is terrible. libdvdnav does not expose seeking
with seek tables. (Although I know xbmc/kodi use an undocumented API
that is not declared in the headers by dladdr()ing it - I think the
function is dvdnav_jump_to_sector_by_time().) With the current mpv
policy if not giving a shit about DVD, just revert our half-working seek
hacks and always use dvdnav_time_search(). Relative seeking might get
stuck sometimes; in this case --hr-seek=always is recommended.
This uses a different method to piece segments together. The old
approach basically changes to a new file (with a new start offset) any
time a segment ends. This meant waiting for audio/video end on segment
end, and then changing to the new segment all at once. It had a very
weird impact on the playback core, and some things (like truly gapless
segment transitions, or frame backstepping) just didn't work.
The new approach adds the demux_timeline pseudo-demuxer, which presents
an uniform packet stream from the many segments. This is pretty similar
to how ordered chapters are implemented everywhere else. It also reminds
of the FFmpeg concat pseudo-demuxer.
The "pure" version of this approach doesn't work though. Segments can
actually have different codec configurations (different extradata), and
subtitles are most likely broken too. (Subtitles have multiple corner
cases which break the pure stream-concatenation approach completely.)
To counter this, we do two things:
- Reinit the decoder with each segment. We go as far as allowing
concatenating files with completely different codecs for the sake
of EDL (which also uses the timeline infrastructure). A "lighter"
approach would try to make use of decoder mechanism to update e.g.
the extradata, but that seems fragile.
- Clip decoded data to segment boundaries. This is equivalent to
normal playback core mechanisms like hr-seek, but now the playback
core doesn't need to care about these things.
These two mechanisms are equivalent to what happened in the old
implementation, except they don't happen in the playback core anymore.
In other words, the playback core is completely relieved from timeline
implementation details. (Which honestly is exactly what I'm trying to
do here. I don't think ordered chapter behavior deserves improvement,
even if it's bad - but I want to get it out from the playback core.)
There is code duplication between audio and video decoder common code.
This is awful and could be shareable - but this will happen later.
Note that the audio path has some code to clip audio frames for the
purpose of codec preroll/gapless handling, but it's not shared as
sharing it would cause more pain than it would help.
This slightly changes behavior when seeking with external audio/subtitle
tracks if transport streams and mpeg files are played, as well as
behavior when seeking with such external tracks.
get_main_demux_pts() is evil because it always blocks on the demuxer (if
there isn't already a packet queued). Thus it could lock up the player,
which is a shame because all other possible causes have been removed.
The reduced "precision" when seeking in the ts/mpeg cases (where
SEEK_FACTOR is used, resulting in byte seeks instead of timestamp seeks)
might lead to issues. We should probably drop this heuristic. (It was
introduced because there is no other way to seek in files with PTS
resets with libavformat, but its value is still questionable.)
The demuxer infrastructure was originally single-threaded. To make it
suitable for multithreading (specifically, demuxing and decoding on
separate threads), some sort of tripple-buffering was introduced. There
are separate "struct demuxer" allocations. The demuxer thread sets the
state on d_thread. If anything changes, the state is copied to d_buffer
(the copy is protected by a lock), and the decoder thread is notified.
Then the decoder thread copies the state from d_buffer to d_user (again
while holding a lock). This avoids the need for locking in the
demuxer/decoder code itself (only demux.c needs an internal, "invisible"
lock.)
Remove the streams/num_streams fields from this tripple-buffering
schema. Move them to the internal struct, and protect them with the
internal lock. Use accessors for read access outside of demux.c.
Other than replacing all field accesses with accessors, this separates
allocating and adding sh_streams. This is needed to avoid race
conditions. Before this change, this was awkwardly handled by first
initializing the sh_stream, and then sending a stream change event. Now
the stream is allocated, then initialized, and then declared as
immutable and added (at which point it becomes visible to the decoder
thread immediately).
This change is useful for PR #2626. And eventually, we should probably
get entirely of the tripple buffering, and this makes a nice first step.
Most of this is explained in the DOCS additions.
This gives us slightly more sanity, because there is less interaction
between the various parts. The goal is getting rid of the video_offset
entirely.
The simplification extends to the user API. In particular, we don't need
to fix missing parts in the API, such as the lack for a seek command
that seeks relatively to the start time. All these things are now
transparent.
(If someone really wants to know the real timestamps/start time, new
properties would have to be added.)
Instead, force everyone to use the metadata struct and set a "title"
field. This is only a problem for the timeline producers, which set up
chapters manually. (They do this because a timeline is a separate
struct.)
This fixes the behavior of the chapter-metadata property, which never
returned a "title" property for e.g. ordered chapters.
Add --demuxer-max-packets and --demuxer-max-bytes, which control the
maximum size of the packet queue. These can be helpful to avoid
excessive memory usage.
Memory usage is the reason why there's a limit in the first place. If a
file is more or less broken, and audio and video don't line up, the
decoders will fill up the packet queue trying to read more audio or
video, and the maximum sizes are required to avoid unbounded memory
allocation. Being able to override the maximum sizes is useful; either
for restricting memory usage further, or enlarging the sizes when
attempting to play various broken files.
Instead of opening a stream and then a demuxer, do both at once with
demux_open_url().
This requires some awkward additions to demuxer_params, because there
are some weird features associated with opening the main file. E.g. the
relatively useless --stream-capture features requires enabling capturing
on the stream before the demuxer is opened, but on the other hand
shouldn't be done on secondary files like external subtitles.
Also relatively bad: since demux_open_url() returns just a demuxer
pointer or NULL, additional error reporting is done via demuxer_params.
Still, at least conceptually, it's ok, and simpler than before.
Nobody wanted to restore this, so it gets the boot.
If anyone still wants to volunteer to restore menu support, this would
be welcome. (I might even try it myself if I feel masochistic and like
wasting a lot of time for nothing.) But if it does get restored, it
should be done differently. There were many stupid things about how it
was done. For example, it somehow tried to pull mp_nav_events through
all the layers (including needing to "buffer" them in the demuxer),
which was needlessly complicated. It could be done simpler.
This code was already inactive, so this commit actually changes nothing.
Also keep in mind that normal DVD/BD playback still works.
The options don't change, but they're now declared and used privately by
demux_mkv.c. This also brings with it a minor refactor of the subpreroll
seek handling - merge the code from playloop.c into demux_mkv.c. The
change in demux.c is pretty much equivalent as well.
Check async abort notification. libavformat already do something
equivalent.
Before this commit, the demuxer could enter resync mode (and print silly
warning messages) when the stream stopped returning data because of an
abort.
Includes some logic for not starting the demuxer thread for fully read
subtitles. (Well, the cache will still waste _lots_ of resources, and
the cache always has to be created, because we don't know whether it'll
be needed _before_ opening the file.)
See #1597.
wm4
Aman Gupta
Marcin Kurczewski