I'm not sure about this, but it looks like a bug. If a stream didn't
have packets, but the joined range does, the stream should obviously
read the packets added by the joined range. Until now, due to
reader_head being NULL, reading was only resumed if a _new_ packet was
added by actual demuxing (in add_packet_locked()), which means the
stream would suddenly skip ahead, past the original end of the joined
range.
Change it so that it will pick up the new range.
Also, clear the skip_to_keyframe flag. Nothing useful can come from this
flag being set; in the first place, the first packet of a range (that
isn't the current range) should start with a keyframe. Some code
probably enforced it (although it's fuzzy).
Completely untested.
When doing a seek to the end of the cache, ds->skip_to_keyframe can be
set to true. Then some packets passed to add_packet_locked() may have to
be skipped. In some aspects, the skipped packet was still treated as if
it was going to be returned to the reader.
It almost doesn't matter though: it only caused a redundant wakeup_ds()
call, and could pass the packet to the stream recorder. Fix it anyway.
This fixes that there were weird delay ("buffering") when seeking into
the last part of a seekable range. The exact case which triggers it if
SEEK_FORWARD is used, and the seek pts is after the second-last
keyframe, but before the end of the range. In that case,
find_seek_target() returned NULL, and the cache layer waited until the
_next_ keyframe the underlying demuxer returned until resuming playback.
find_seek_target() returned NULL, because the last keyframe had
kf_seek_pts unset. This field contains the lowest PTS in the packet
range from the keyframe until the next keyframe (or EOF). For normal
seeks, this is needed because keyframes don't necessarily have the
minimum PTS in the packet range, so it needs to be computed by waiting
for all packets until the next keyframe (or EOF).
Strictly speaking, this behavior was correct, but it meant that the
caller would set ds->skip_to_keyframe, which waits for the next newly
demuxed keyframe. No packets were returned to the decoder until this
happened, usually resulting in the frontend entering "buffering" mode.
What it really needs to do is returning the last keyframe in the cache.
In this situation, the seek target points in the middle of the last
completely cached packet range (as delimited by keyframes), and
SEEK_FORWARD is supposed to skip to the next keyframe. This is in line
with the basic assumptions the packet cache makes (e.g. the keyframe
flag means it's possible to start decoding, and the frames decoded from
it and following packets will strictly have PTS values above the
previous keyframe range). This means in this situation the kf_seek_pts
value doesn't matter either.
So fix this situation by explicitly detecting it and then returning the
last cached keyframe.
Should the search loop look at all packets, instead of only keyframe
ones? This would mean it can know that it's within the last keyframe
range (without looking at queue->seek_end). Maybe this would be a bit
more natural for the SEEK_FORWARD case, but due to PTS reordering it
doesn't sound like a useful thing to do.
Should skip_to_keyframe be checked by the code that sets kf_seek_pts to
a known value? This wouldn't help too much; the frontend would still go
into "buffering" mode for no reason until the packet range is completed,
although it would resume from the correct range.
Should a NULL return always unconditionally use keyframe_latest? This
makes sense because the seek PTS is usually already in the cached range,
so this is the only case that should happen. But there are scary special
cases, like sparse subtitle streams, or other uses of find_seek_target()
which could be out of range now or in future. Basically, don't "risk"
it.
One other potential problem with this is that the "adjust seek target"
code will be disabled in this case. It checks kf_seek_pts, and if it's
unset, the adjustment is not done. Maybe this could be changed to use
the queue's seek_end time, but I'm not sure if this is fully kosher. On
the other hand, I think the main use for this adjustment is with
backwards seeks, so this shouldn't matter.
A previous commit dealing with audio/video stream merging mentioned how
seeking forward entered "buffering" mode for unknown reasons; this
commit fixes this issue.
demux_timeline doesn't do any transport accesses itself. The slave
demuxers do this (these will actually access the stream layer and
perform e.g. network accesses). As a consequence, demux_timeline always
reported 0 bytes read, and network speed display didn't work.
Fix this by awkwardly reporting the amount of read bytes upwards. This
is not very nice, and requires explicit calls whenever the slave "might"
have read data.
Due to the way the reporting is done, it only works if the slaves do not
run demuxer threads, which makes things even less nice. (Fortunately
they don't anyway, because it would be a waste of resources.) Some
identifiers contain the word "hack" as a warning.
Some of the stupidity comes from the fact that demux.c itself resets the
stats randomly in order to calculate the bytes_per_second value, which
is useless for a slave, but of course is still done, because demux.c
itself is not aware of whether it's on the slave or top-level layer.
Unfortunately, this must do.
In theory, the demuxer thread/cache layer should be separated from
demuxer implementations. This would get rid of all the awkwardness and
nonsense. For example, the only threading involved would be the caching
layer, completely separate from demuxers themselves. It'd be the only
thing calculates speed rates for the player frontend, too (instead of
doing it for each demuxer, even if unused).
It was an ugly hack, and the next commit will make it even uglier.
Slightly reduce the ugliness to prevent death of too many brain cells,
though it's still an ugly hack.
The cleanup is really minor, but I guess the following commit would be
much worse otherwise. In particular, this commit checks accesses
(instead of having a public field with evil access rules), which should
avoid misunderstandings and incorrect use. Strictly speaking, the added
field is redundant, but the next commit complicates it a bit.
The only thing left is the notification for track switching. Just get
rid of that.
There's probably no real reason to get rid of control(), but why not. I
think I was actually trying to do some real work but fuck that.
Subtitles (and a few other file types, like playlists) are not streamed,
but fully read on opening. This means keeping the file handle or network
socket open is a waste of resources and could cause other weird
behavior. This is why there's a hack to close them after opening.
Change this hack to make the demuxer itself do this, which is less
weird. (Until recently, demuxer->stream ownership was more complex,
which is why it was done this way.)
There is some evil shit due to a huge ownership/lifetime mess of various
objects. Especially EDL (the currently only nested demuxer case)
requires being careful about mp_cancel and passing down stream pointers.
As one defensive programming measure, stop accessing the "stream"
variable in open_given_type(), even where it would still work. This
includes removing a redundant line of code, and removing the peak call,
which should not be needed anymore, as the remaining demuxers do this
mostly correctly.
I always wanted to get rid of this, because it makes the ownership rules
for the stream pointer really awkward. demux_edl.c was the only
remaining user of this. Replace it with a semi-clever idea: the init
segment shit can be used to pass the "file" contents as memory block,
and "memory://" itself provides an empty stream. I have no idea if this
actually works, because I didn't immediately find a test stream (would
have to be some youtube DASH shit).
Instead of going through those weird DEMUXER_CTRLs, query this
information directly. I'm not sure which kind of brain damage made me
use CTRLs for these. Since there are no other DEMUXER_CTRLs that make
sense for the frontend, remove the remaining infrastructure for them
too.
The stream size return was the only thing that still required doing
STREAM_CTRLs from frontend through the demuxer layer. This can be done
much easier, so rip it out. Also rip out the now unused infrastructure
for STREAM_CTRLs via demuxer layer.
Apparently this was so that when playing a video file from a .rar file,
it would load external subtitles with the same name (instead of looking
for mpv's rar:// mangled URL). This was requested on github almost 5
years ago. Seems like a weird feature, and I don't care. Drop it,
because it complicates some in progress change.
Linux analog TV support (via tv://) was excessively complex, and
whenever I attempted to use it (cameras or loopback devices), it didn't
work well, or would have required some major work to update it. It's
very much stuck in the analog past (my favorite are the frequency tables
in frequencies.c for analog TV channels which don't exist anymore).
Especially cameras and such work fine with libavdevice and better than
tv://, for example:
mpv av://v4l2:/dev/video0
(adding --profile=low-latency --untimed even makes it mostly realtime)
Adding a new input layer that targets such "modern" uses would be
acceptable, if anyone is interested in it. The old TV code is just too
focused on actual analog TV.
DVB is rather obscure, but has an active maintainer, so don't remove it.
However, the demux/stream ctrl layer must go, so remove controls for
channel switching. Most of these could be reimplemented by using the
normal method for option runtime changes.
This removes anything related to DVD/BD/CD that negatively affected the
core code. It includes trying to rewrite timestamps (since DVDs and
Blurays do not set packet stream timestamps to playback time, and can
even have resets mid-stream), export of chapters, stream languages,
export of title/track lists, and all that.
Only basic seeking is supported. It is very much possible that seeking
completely fails on some discs (on some parts of the timeline), because
timestamp rewriting was removed.
Note that I don't give a shit about optical media. If you want to watch
them, rip them. Keeping some bare support for DVD/BD is the most I'm
going to do to appease the type of lazy, obnoxious users who will care.
There are other players which are better at optical discs.
Manual changes done:
* Merged the interface-changes under the already master'd changes.
* Moved the hwdec-related option changes to video/decode/vd_lavc.c.
The seek range update was to early and did not take the removed head
packets into account. And therefore missed that the queue was not
BOF anymore.
This led to not be able to backward seek before the first packet of
the first seek range.
Fix it by moving the seek range update after the possible removal and
the change of the BOF flag.
Fixes: #6522
Commit e392d6610d modified the native
demuxer to use track gain as a fallback for album gain if the latter is
not present. This commit makes functionally equivalent changes in the
libavformat demuxer.
If the number of chapters is 0, the chapter list can be NULL. clang
complains that we pass NULL to qsort(). This is yet another pointless UB
that exists for no reason other than wasting your time.
--record-file is nice, but only sometimes. If you watch some sort of
livestream which you want to record, it's actually much nicer not to
record what you're currently "seeing", but anything you're receiving.
The demuxer cache is the only cache now. Might need another change to
combat seeking failures in mp4 etc. The only bad thing is the loss of
cache-speed, which was sort of nice to have.
When the current packet queue was completely empty, and EOF was reached,
the queue->is_eof flag was not correctly set to true. Change this by
reading ds->eof to check whether the stream is considered EOF. We also
need to make sure update_seek_ranges() is called in this case, so change
the code to simply call it when queue->is_eof changes.
Also, read_packet() needs to call adjust_seek_range_on_packet() if
ds->eof changes. In that case, the decoder also needs to be notified
about EOF. So both of these should be called when ds->eof changes to
true. (Other code outside of this function deals with the case when
ds->eof is changed to false.)
In addition, this code was kind of shoddy about calling wakeup_ds()
correctly. It looks like there was an inverted condition, and sent a
wakeup to the decoder only when ds->eof was already true, which is
obviously bogus. The final EOF case tried to be somehow clever about
checking in->last_eof for notifying the codec, which is sort of OK, but
seems to be strictly worse than just checking whether ds->eof changed.
Fix these things.
This will enable the player core to terminate the demuxers in a "nicer"
way without having to block on network. If it just used demux_free(), it
would either have to block on network, or like currently, essentially
kill all I/O forcefully.
The API is slightly awkward, because demuxer lifetime is bound to its
allocation. On the other hand, changing that would also be awkward, and
introduce weird in-between states that would have to be handled in tons
of places.
Currently unused, to be user later.
Alway give each demuxer its own mp_cancel instance. This makes
management of the mp_cancel things much easier. Also, instead of having
add/remove functions for mp_cancel slaves, replace them with a simpler
to use set_parent function. Remove cancel_and_free_demuxer(), which had
mpctx as parameter only to check an assumption. With this commit,
demuxers have their own mp_cancel, so add demux_cancel_and_free() which
makes use of it.
Them being separate is just dumb. Replace them with a single
demux_free() function, and free its stream by default. Not freeing the
stream is only needed in 1 special case (demux_disc.c), use a special
flag to not free the stream in this case.
The properties/commands touched in this commit are all for obscure
special inputs (BD/DVD/DVB/TV), and they all block on the demuxer/stream
layer. For network streams, this blocking is very unwelcome. They will
affect playback and probably introduce pauses and frame drops. The
player can even freeze fully, and the logic that tries to make playback
abortable even if frozen complicates the player.
Since the mentioned accesses are not needed for network streams, but
they will block on network streams even though they're going to fail,
add a flag that coarsely enables/disables these accesses. Essentially it
establishes a whitelist of demuxers/streams which support them.
In theory you could to access BD/DVD images over network (or add such
support, I don't think it's a thing in mpv). In these cases these
controls still can block and could even "freeze" the player completely.
Writing to the "program" and "cache-size" properties still can block
even for network streams. Just don't use them if you don't want freezes.
It seems a bit inappropriate to have dumped this into stream.c, even if
it's roughly speaking its main user. At least it made its way somewhat
unfortunately to other components not related to the stream or demuxer
layer at all.
I'm too greedy to give this weird helper its own file, so dump it into
thread_tools.c.
Probably a somewhat pointless change.
If a stream starts later than the others at the start of the file, it
shouldn't restrict the seek range to the time stamp where it begins.
This is similar to the previous commit, just for the other end.
Normally, the seek range is the minimum overlap of the cached ranges of
each stream. But if one of the streams ends earlier, this leads to the
seek range getting cut off, even if you could seek there.
Change it so that EOF streams cannot restrict the end of the seek range.
They can only extend it. This is the opposite from not-EOF streams, so
they need to be handled separately. In particular, they get exluded from
normal end range calculation, but when full EOF is reached, all streams
are EOF, and the maximum end time can be used to set the seek end time.
(In theory we could also take the max with the demuxer signaled total
file duration, but let's not for now.)
Also, if a stream is completely empty, essentially skip it, instead of
considering the range unseekable. (Also, we don't need to mess with
seek_start in this case, because it will be NOPTS and is skipped
anyway.)
When the current packet queue was completely empty, and EOF was reached,
the queue->is_eof flag was not correctly set to true. Change this by
reading ds->eof to check whether the stream is considered EOF. We also
need to make sure update_seek_ranges() is called in this case, so change
the code to simply call it when queue->is_eof changes.
Also, read_packet() needs to call adjust_seek_range_on_packet() if
ds->eof changes. In that case, the decoder also needs to be notified
about EOF. So both of these should be called when ds->eof changes to
true. (Other code outside of this function deals with the case when
ds->eof is changed to false.)
In addition, this code was kind of shoddy about calling wakeup_ds()
correctly. It looks like there was an inverted condition, and sent a
wakeup to the decoder only when ds->eof was already true, which is
obviously bogus. The final EOF case tried to be somehow clever about
checking in->last_eof for notifying the codec, which is sort of OK, but
seems to be strictly worse than just checking whether ds->eof changed.
Fix these things.
Fixes several issues playing back mpegts with video streams marked
as having "still images". For example, see this video which has
frames only every 6s: https://s3.amazonaws.com/tmm1/music-choice.ts
Changes include:
- start playback right away, without waiting for first video frame
- do not consider the sparse video stream in demuxer underrun detection
- do not require multiple video frames for the VO
- use audio as the master stream for demuxer metadata events
- use audio stream for playback time
Signed-off-by: Aman Gupta <aman@tmm1.net>
This makes ICY title changes show up at approximately the correct time,
even if the demuxer buffer is huge. (It'll still be wrong if the stream
byte cache contains a meaningful amount of data.)
It should have the same effect for mid-stream metadata changes in e.g.
OGG (untested).
This is still somewhat fishy, but in parts due to ICY being fishy, and
FFmpeg's metadata change API being somewhat fishy. For example, what
happens if you seek? With FFmpeg AVFMT_EVENT_FLAG_METADATA_UPDATED and
AVSTREAM_EVENT_FLAG_METADATA_UPDATED we hope that FFmpeg will correctly
restore the correct metadata when the first packet is returned.
If you seke with ICY, we're out of luck, and some audio will be
associated with the wrong tag until we get a new title through ICY
metadata update at an essentially random point (it's mostly inherent to
ICY). Then the tags will switch back and forth, and this behavior will
stick with the data stored in the demuxer cache. Fortunately, this can
happen only if the HTTP stream is actually seekable, which it usually is
not for ICY things. Seeking doesn't even make sense with ICY, since you
can't know the exact metadata location. Basically ICY metsdata sucks.
Some complexity is due to a microoptimization: I didn't want additional
atomic accesses for each packet if no timed metadata is used. (It
probably doesn't matter at all.)
This fixes an issue where captions stop rendering after an
in-demuxer-cache seek, because the demuxer keeps waiting to find
a keyframe (ds->skip_to_keyframe set to true in execute_cache_seek).
When this happens, network calls are forcibly aborted (more or less),
but demuxers might keep going, as most of them do not check for forced
exits properly. This can possibly lead to broken packets being added.
Also do not attempt to read more packets in this situation.
Also do not print a stream open failed message if opening was aborted
anyway.
Since the demuxer cache addition, ds->queue->head can actually be set to
non-NULL, but the decoder can still be at EOF (with no packets to come).
This made it report an unknown buffered size, instead of 0. Fix this by
checking the decoder part of the packet queue instead.
Probably doesn't matter much, but fixes an annoying "???" on the CLI
status line in some situations.
It's a mess: mp3 files have user tags as global metadata (because the
id3v2 tag is global and there is only 1 stream), while OGG files have it
per-track (because it's per-stream on the lowest level). mpv needs to
try to make something nice out of the mess.
It did so by trying to detect audio-only OGG files, and then copying the
per-stream metadata to the global metadata. Make the heuristic for
detecting this slightly more clever, so it works for files with extra,
unrelated streams, like the awful libavformat cover art hack.
Fixes#5577.
Reduce backward/forward from 400MB/400MB to 50MB/150MB. Too many
complaints about high memory usage.
Note that external tracks (like ytdl DASH with external audio tracks)
will double the amounts, because an external track uses its own demuxer
and cache.
This is supposed to help making data flow easier and wakeup handling
more efficient. Once that change is done, reading a packet on any
stream won't have to wakeup and poll all decoders (which helps reducing
the mess even if all decoders are on the same thread).
This also improves the accuracy of wakeups by tracking better whether
a wakeup is needed.
And use it for 2 demuxer options. It could be used for more options
later. (Though the --cache options can not use this, because they use KB
as base unit.)
It was actually already implemented as ta_dup_ptrtype(), but that seems
like a clunky name. Also we still use the talloc_ names throughout the
source, and I'd rather use an old name instead of a mixing inconsistent
naming conventions.
If you play a video with an external audio track, and do backwards
keyframe seeks, then audio can be missing. This is because a backwards
seek can end up way before the seek target (this is just how this seek
mode works). The audio file will be seeked at the correct seek target
(since audio usually has a much higher seek granularity), which results
in silence being played until the video reaches the originally intended
seek target.
There was a hack in audio.c to deal with this. Replace it with a
different hack. The new hack probably works about as well as the old
hack, except it doesn't add weird crap to the audio resync path (which
is some of the worst code here, so this is some nice preparation for
rewriting it). As a more practical advantage, it doesn't discard the
audio demuxer packet cache. The old code did, which probably ruined
seeking in youtube DASH streams.
A non-hacky solution would be handling external files in the demuxer
layer. Then chaining the seeks would be pretty easy. But we're pretty
far from that, because it would either require intrusive changes to the
demuxer layer, or wouldn't be flexible enough to load/unload external
files at runtime. Maybe later.
Similar to 1eec7d2315, but for the beginning of the stream (named BOF in
this commit).
We can know this only if demuxing actually started from the beginning.
If there is a seek to the beginning (even if you use --start=-1000), we
don't know in general whether the demuxer truly returns the start of the
file. We could probably make a heuristic with assuming that this is what
happens if the seek target is before the start time or so, but this is
not included in this commit.
libavformat's cover art hack (aka attached pictures) breaks the ability
of the demuxer cache to keep multiple seek ranges. This happens because
the cover art packet has neither position nor timestamp, and libavformat
gives us the packet even though we intended to drop it.
The cover art hack works by adding the cover art packet to the read
packet stream once when demuxing starts (or after seeks). mpv treats
cover art in a similar way internally, but we have to compensate for
libavformat's shortcomings, and add the cover art packet ourselves when
we need it. So we don't want libavformat to return the packet.
We normally prevent this in demux_lavc.c/select_tracks() and explicitly
disable cover art streams. (We add it in dequeue_packet() instead.) But
libavformat will actually add the cover art packet even if we disable
the cover art stream, because it adds it at initialization time, and
does not bother to check again in av_read_frame() (apparently). The
packet is actually read, and upsets the demuxer cache logic. In
addition, this also means we probably decoded the cover art picture
twice in some situations.
Fix this by explicitly checking/discarding this in yet another place.
(Screw this hack...)
The impact was that you couldn't exactly seek to the join point with a
keyframe seek, even though there was a keyframe. This commit fixes it by
preserving the necessary metadata that got lost on cached range joining.
This is so absurdly obscure that it gets a longer code comment.
This warning was printed when the demuxer cache tried to join two
adjacent seek ranges, but failed if the last keyframe in the second
range was within the (overlapping) first range. This is a weird corner
case which to support probably would not be worth it.
So this code just printed a warning and discarded the second range. As
it turns out, this can happen relatively often if you seek a lot, and
the seek ranges are very tiny (such as consisting of only 1 keyframe).
Dropping the second range in these cases is OK and probably cheaper than
trying to actually join them. Change the warning to verbose level.
(It seems this could actually be "supported", because if keyframe_latest
is not set, there will be no other keyframes, so it could just be unset,
with the exception that q1->keyframe_latest in the code below must not
be overwritten. But still, too much trouble for a special case that
likely does not matter, and it would have to be tested too.)
This means if the user tries to seek past EOF, and we know EOF was seen
already, then use a cached seek, instead of triggering a low level seek.
This requires some annoying tracking, but seems pretty simple otherwise.
One advantage of doing this is that if the user tries to do this kind of
seek, there's no unnecessary waiting for a reaction by network (and in
most cases, redundant downloading of data just to discard it again).
Another is that this avoids creating overlapping seek ranges: previously, the
low level seek would naturally create a new range. Then it would read and add
data from the end of the stream due to the low level demuxer not being able to
seek to the target and selecting the last seek point before the end of the
stream. Consequently, this new range would overlap with the previous cached
range. But since the cache joining code is written such that you join the
current range with the _next_ range (instead of the previous as it would be
needed in this case), the overlapping ranges were left alone, until seeking back
to the previous range. That was ugly, sort of harmless, and could happen in
other cases, but this avoidable case was pretty easy to trigger.
Export them as explicitly undocumented debugging fields for the
"demuxer-cache-state" property.
Should be somewhat helpful to debug "wtf is the demuxer" doing
situations better, especially when seeking. It also becomes visible how
long the demuxer is blocked on an "old" seek when you keep seeking while
the first seek hasn't finished.
update_seek_ranges() has some special code that attempts to correctly
adjust seek ranges for subtitle tracks. (Subtitle are a nightmare for
seek ranges, because they are sparse, so using the packet list is not
enough to reliably determine the valid cached range.)
This had code like this inside the modified if statement:
range->seek_start = MP_PTS_MAX(range->seek_start, <something>);
If seek_start is NOPTS, then seek_start will be set to <something>,
breaking some other code that checks seek_start for NOPTS to see if it's
empty. Fix this by explicitly checking whether seek_start is NOPTS
before adjusting it.
The crash happened in prune_old_packets() because the range was marked
as non-empty, yet there was no packet in it to prune. This was with
files with muxed subtitles, when seeking back to the start. This should
not happen anymore with the change. Also add an assert() to
check_queue_consistency() that checks for this specific case.
There's still some mess. In theory, subtitle tracks could be completely
empty, yet their seek range would span the entire file. Seek range
tracking of subtitle files is slightly broken (even before this change).
Some of this should probably be revisited later, including not just
using seek_start to determine whether a seek range should be pruned due
to being empty.
This will help with things like livestreams.
As a minor detail, subtitles are excluded, because they sometimes have
"unused" events after video and audio ends. To avoid this annoying
corner case, just ignore them.
Before this change and before the seekable stream cache became a thing,
we could possibly seek using the stream cache. But we couldn't know
whether the seek would succeed. We knew the available byte range, but
could in general not tell whether a demuxer would stay within the range
when trying to seek to a specific time position. We preferred to have
safe defaults, so seeking in streams that were detected as unseekable
were not honored. We allowed overriding this via --force-seekable=yes,
in which case it depended on your luck whether the seek would work, or
the player crapped its pants.
With the demuxer packet cache, we can tell exactly whether a seek will
work (at least if there's only 1 seek range). We can just let seeks go
through. Everything to allow this is already in place, and this commit
just moves around some minor things.
Note that the demux_seek() return value was not used before, because low
level (i.e. network level) seeks are usually asynchronous, and if they
fail, the state is pretty much undefined. We simply repurpose the return
value to signal whether cache seeking worked. If it didn't, we can just
resume playback normally, because demuxing continues unaffected, and no
decoder are reset.
This should be particularly helpful to people who for some reason stream
data into stdin via streamlink and such.
This log line tells us why the demuxer is trying to read more, which us
useful when debugging queue overflows. Probably barely useful, but I
think keeping that flag separately also makes the code slightly easier
to understand.
This fixes weird behavior in the following case:
- open a file
- make sure the max. demuxer forward cache is smaller than the
file's video track
- make sure the max. readahead duration is larger than the file's
duration
- disable the audio track
- seek to the beginning of the file
- once the cache has filled enable the audio track
- a queue overflow warning should appear
(- looking at the seek ranges is also interesting)
The queue overflow warning happens because the packed queue for the
video track will use up the full quota set by --demuxer-max-bytes. When
the audio track is enabled, reading an audio packet would technically
overflow the packet cache by the size of whatever packet is read next.
This means the demuxer signals EOF to the decoder, and once playback has
consumed enough video packets so that audio packets can be read again,
the decoder resumes from EOF. This interacts badly with A/V
synchronization and the whole thing can randomly crap itself until audio
has fully recovered.
We didn't care about this so far, but we want to raise the readahead
duration to something very high, so that the demuxer cache is fully
used. This means this case can be hit quite quickly by switching audio
or subtitle tracks, and is not really an obscure corner case anymore.
Fix this by always losing all cache. Since the cache can't be used
anyway until the newly selected track has been read, this is not much of
a disadvantage. The only thing that could be brought up is that
unselecting the track again could resume operation normally. (Maybe this
would be useful if network died completely without chance of recovery.
Then you could watch the already buffered video anyway by deselecting
the audio track again.) But given the headaches, this seems like the
better solution.
Unfortunately this requires adding new new strange fields and strangely
fragmenting state management functions again. I'm sure whoever works on
this in the future will hate me. Currently it seems like the lesser
evil, and much simpler and robust than the other potential solutions.
In case this needs to be revisited, here is a reminder for readers from
the future what alternative solutions were considered, without those
disadvantages:
A first attempted solution allowed the demuxer to buffer some additional
packets on track switching. This would allow it to read enough data to
feed the decoder at least. But it was still awkward, as it didn't allow
the demuxer to continue prefetching the newly selected track. It also
barely worked, because you could make the forward buffer "over full" by
seeking back with seekable cache enabled, and then it couldn't read
packets anyway.
As alternative solution, we could always demux and cache all tracks,
even if they're deselected. This would also not require a network-level
seek for the "refresh" logic (it's the thing that lets the video decoder
continue as if nothing happened, while actually seeking back in the
stream to get the missing audio packets, in the case of enabling a
previously disabled audio track). But it would also possibly waste
network and memory resources, depending on what the user actually wants.
A second solution would just account the queue sizes for each stream
separately. We could freely fill up the audio packet queue, even if the
video queue is full. Since the demuxer API returns interleaved packets
and doesn't let you predict which packet type comes next, this is not as
simple as it sounds, but it'd probably tie in nicely with the "refresh"
logic.
A third solution would be removing buffered video packets from the end
of the packet queue. Since the "refresh" logic gets these anyway, there
is no reason to keep them if they prevent the audio packet queue from
catching up with the video one. But this would require additional logic,
would interact badly with a bunch of other corner cases. And as far as
the code goes, it's rather complex, because all the logic is written
with FIFO behavior in mind (including the fact that the packet queue is
a singly linked list with no backwards links, making removal from the
end harder).
It seems like there's nothing stopping from sub-demuxers from keeping
packets in the cache, even if it's completely pointless. The top-most
demuxer (demux_timeline) already takes care of caching, so sub-demuxers
only waste space and time with this.
Add a function that can disable the packet cache even at runtime and
after packets are read. (It's not clear whether it really can happen
that packets are read before demux_timeline gets the sub-demuxers, but
there's no reason to make it too fragile.) Call it on all sub-demuxers.
For this to work, it seems we have to move the code for setting the
seekable_cache flag to before demux_timeline is potentially initialized,
because otherwise the cache would be reenabled if the demuxer triggering
timeline support is a timeline segment itself (e.g. ordered chapters).
This fixes missing audio when cycling through audio tracks with anything
that uses nested demuxers, such as demux_timeline, which us used for
EDL, --merge-files, ordered chapters, and youtube-dl pseudo DASH
support. When this bug happened, reenabling an audio track would lead to
silence for the duration of the readahead amount.
The underlying reason is the incorrectly updated buffered range on track
switch. It accidentally included the amount covered by the deselected
stream. But the cause of the observed effect was that demux_timeline
issued a refresh seek to the underlying slave demuxer, which in turn
thought it could do a cache seek, because the seek range still included
everything.
update_stream_selection_state() calls update_seek_ranges() to update the
seek ranges after a track switch. When reenabling the track, ds->eager
was set to false during update_seek_ranges(), which made it think the
stream was sparse, and thus it didn't restrict the current seek range
(making later code think everything was buffered). Fix this by moving
some code, so we first update the ds->eager flag, then the seek ranges.
Also verbose log the low level stream selection calls.
Always display the duration as "unknown" if the duration is known. Also
fix that at least demux_lavf reported unknown duration as 0 (fix by
setting the default to unknown in demux.c).
Remove the dumb _u formatter function, and use a different approach to
avoiding displaying "unknown" as playback time on playback start (set
last_seek_pts for that).
If the backbuffer is much larger than the forward buffer, and if you
join a small range with a large range (larger than the forward buffer),
then the seek issues to the end of the range after joining will overflow
the queue.
Normally, read_more will be false when the forward buffer is full, but
the resume seek after joining will set need_refresh to true, which
forces more reading and thus triggers the overfloe warning.
Attempt to fix this by not setting read_more to true on refresh seeks.
Set prefetch_more instead. read_more will still be set if an A/V stream
has no data.
This doesn't help with the following problems related to using refresh
seeks for track switching:
- If the forward buffer is full, then enabling another track will
obviously immediately overflow the queue, and immediately lead to
marking the new track as having no more data (i.e. EOF). We could cut
down the forward buffer or so, but there's no simple way to implement
it. Another possibility would be dropping all buffers and trying to
resume again, but this would likely be complex as well.
- Subtitle tracks will not even show a warning (because they are sparse,
and we have no way of telling whether a packet is missing, or there's
just no packet near the current position). Before this commit,
enabling an empty subtitle track would probably have overflown the
queue, because ds->refreshing was never set to true. Possibly this
could be solved by determining a demuxer read position, which would
reflect until which PTS all subtitle packets should have been demuxed.
The forward buffer limit was intended as a last safeguard to avoid
excessive memory usage against badly interleaved files or decoders going
crazy (up to reading the whole into memory and OOM'ing the user's
system). It's not good at all to limit prefetch. Possibly solutions
include having another smaller limit for prefetch, or maybe having only
a total buffer limit, and discarding back buffer if more data has to be
read. The current solution is making the forward buffer larger than the
forward duration (--cache-secs) would require, but of course this
depends on the stream's bitrate.
The option for enabling it has now an "auto" choice, which is the
default, and which will enable it if the media is thought to be via
network or if the stream cache is enabled (same logic as --cache-secs).
Also bump the --cache-secs default from 10 to 120.
Some back buffer is required to make the immediate forward range
seekable. This is because the back buffer limit is strictly enforced.
Just set a rather high back buffer by default. It's not use if
--demuxer-seekable-cache is disabled, so this is without risk.
Limit the number of cached ranges to MAX_SEEK_RANGES, which is the same
number of maximally exported seek ranges. It makes no sense to keep
them, as the user won't see them anyway. Remove the smallest ones to
enforce the limit if the number grows too high.
Helps a little bit, I guess. But in general, t(h)rashing the cache kills
us anyway.
This has a fixed number of index entries. Entries are added/removed as
packets go through the packet queue. Only keyframes after index_distance
seconds are added. If there are too many keyframe packets, the existing
index is reduced by half, and index_distance is doubled. This should
provide somewhat even spacing between the entries.
The packet queue is sorted, so we can stop the search if we have found a
packet, and the next packet in the queue has a higher PTS than the seek
PTS (for the sake of SEEK_FORWARD, we still consider the first packet
with a higher PTS).
Also, as a mostly cosmetic change, but which might be "faster", check
target for NULL, instead of target_diff for a magic float value.
Subtitle streams are sparse, and no overlap is required to correctly
join two cached ranges. This was not correctly handled iff the two
ranges had different subtitle packets close to the join point.
demux_add_packet() must completely ignore any packets that are added
while a queued seek is not initiated yet.
The main issue is that after setting in->seeking==true, the central lock
is released, and it can take "a while" until it's reacquired on the
demux thread and the seek is actually initiated. During that time,
packets could be read and added, that have nothing to do with the new
state.
If subtitles are part of the stream, determining the seekable range
becomes harder. Subtitles are sparse, and can have packets in irregular
intervals, or even completely lack packets. The usual logic of computing
the seek range by the min/max packet timestamps fails.
Solve this by making the only assumption we can make: subtitle packets
are implicitly demuxed along with other packets. We also assume perfect
interleaving for this, but you really can't do anything with sparse
packets that makes sense without this assumption.
One special case is if we prune sparse packets within the current
seekable range. Obviously this should limit the seekable range to after
the pruned packet.
Instead of weirdly deciding this on every packet read and having the
code far away from where it's actually needed, just run it where it's
actually needed.
A typical idiom for calling functions that remove something from the
array being iterated, but it's not needed here. I have no idea why this
was ever done.
Setting ds->refreshing for unselected streams could lead to a
nonsensical queue overflow warning, because read_packet() took it as a
reason to continue reading.
Also add some more information to the queue overflow warning (even if
that one doesn't have anything to do with this bug - it was for
unselected streams only).
This fixes an endless loop with threading disabled, such as for example
when playing a file with an external subtitle file, and seeking to the
beginning. Something will set in->seeking, but the seek is never
executed, which made demux_read_packet() loop endlessly. (External
subtitles will use non-threaded mode for whatever reasons.)
Fix this by by making the unthreaded code to execute the worker thread
body, which reduces the difference in logic.
Until now, the demuxer cache was limited to a single range. Extend this
to multiple range. Should be useful for slow network streams.
This commit changes a lot in the internal demuxer cache logic, so
there's a lot of room for bugs and regressions. The logic without
demuxer cache is mostly untouched, but also involved with the code
changes. Or in other words, this commit probably fucks up shit.
There are two things which makes multiple cached ranges rather hard:
1. the need to resume the demuxer at the end of a cached range when
seeking to it
2. joining two adjacent ranges when the lowe range "grows" into it (and
resuming the demuxer at the end of the new joined range)
"Resuming" the demuxer means that we perform a low level seek to the end
of a cached range, and properly append new packets to it, without adding
packets multiple times or creating holes due to missing packets.
Since audio and video never line up exactly, there is no clean "cut"
possible, at which you could resume the demuxer cleanly (for 1.) or
which you could use to detect that two ranges are perfectly adjacent
(for 2.). The way how the demuxer interleaves multiple streams is also
unpredictable. Typically you will have to expect that it randomly allows
one of the streams to be ahead by a bit, and so on.
To deal with this, we have heuristics in place to detect when one packet
equals or is "behind" a packet that was demuxed earlier. We reuse the
refresh seek logic (used to "reread" packets into the demuxer cache when
enabling a track), which checks for certain packet invariants.
Currently, it observes whether either the raw packet position, or the
packet DTS is strictly monotonically increasing. If none of them are
true, we discard old ranges when creating a new one.
This heavily depends on the file format and the demuxer behavior. For
example, not all file formats have DTS, and the packet position can be
unset due to libavformat not always setting it (e.g. when parsers are
used).
At the same time, we must deal with all the complicated state used to
track prefetching and seek ranges. In some complicated corner cases, we
just give up and discard other seek ranges, even if the previously
mentioned packet invariants are fulfilled.
To handle joining, we're being particularly dumb, and require a small
overlap to be confident that two ranges join perfectly. (This could be
done incrementally with as little overlap as 1 packet, but corner cases
would eat us: each stream needs to be joined separately, and the cache
pruning logic could remove overlapping packets for other streams again.)
Another restriction is that switching the cached range will always
trigger an asynchronous low level seek to resume demuxing at the new
range. Some users might find this annoying.
Dealing with interleaved subtitles is not fully handled yet. It will
clamp the seekable range to where subtitle packets are.
libavcodec can't deal with them, because its API doesn't distinguish
between 0 sized packets and sending EOF. As such, keeping them doesn't
do any good, ever. This actually fixes some obscure mkv sample (see
previous commit).
This adds a bunch of stuff (mostly unused or redundant) as preparation
for supporting multiple seek ranges. Actual support is probably still
far away.
One change that messes deeper with the actual code is that we account
for total buffered bytes instead of just the backwards bytes now. This
way, prune_old_packets() doesn't have to iterate over all seek ranges to
determine whether something needs pruning.
The main purpose of this commit is avoiding any hidden O(n^2) algorithms
in the code for pruning the demuxer cache, and for determining the
seekable boundaries of the cache. The old code could loop over the whole
packet queue on every packet pruned in certain corner cases.
There are two ways how to reach the goal:
1) commit a cardinal sin
2) do everything incrementally
The cardinal sin is adding an extra field to demux_packet, which caches
the determined seekable range for a keyframe range. demux_packet is a
rather general data structure and thus shouldn't have any fields that
are not inherent to its use, and are only needed as an implementation
detail of code using it. But what are you gonna do, sue me?
In the future, demux.c might have its own packet struct though. Then the
other existing cardinal sin (the "next" field, from MPlayer times) could
be removed as well.
This commit also changes slightly how the seek end is determined. There
is a note on the manpage in case anyone finds the new behavior
confusing. It's somewhat cleaner and might be needed for supporting
multiple ranges (although that's unclear).
The demuxer cache seeking mechanism looks at keyframe ranges to
determine the earlierst PTS of a packet. Instead of looping over all
packets twice (once to find the next keyframe, a second time to find the
seek PTS), do it in one go.
For that basically turn recompute_keyframe_target_pts() into an
iteration functionn. Functionality should be unchanged with this commit.
The base_ts field is used to guess the decoder position, and when set to
NOPTS, it just read ahead arbitrarily. Also demux_add_packet() sets
base_ts to the new timestamp when appending a packet, which would also
make it readahead by a too large amount.
Fix this by setting base_ts after a seek. This assumes that normally, a
cached seek target will always have the timestamp set. This is actually
not quite clear (as it calls recompute_keyframe_target_pts(), which
looks at multiple packets), but maybe it works well enough.
Don't do any of the extra work related to pruning the backbuffer if
demuxer cache seeking is disabled.
As a small extra, always prune packets with no timestamps immediately,
or queue heads that are not keyframes. (Both of them would be pruned
from the backbuffer by the normal logic anyway.)
If fulfilling --demuxer-readahead-secs requires more memory than allowed
by --demuxer-max-bytes, the queue obviously overflows. But the warning
is normally only for the case when trying to find the next video or
audio packet fails, and decoding can't continue.
Use a separate variable for determining whether to prefetch, and if the
queue has overflown, skip the message. In fact, skip the EOF setting and
waking up the decoder thread as well, because the EOF flag should not be
(have been) set in this situation, and waking up the reader thread helps
only if the EOF state changed.
In a shit show of subtle corner case interactions, making the demuxer
cache buffer the entire file can display a small buffered time if
subtitles are enabled. The reason is that some subtitle decoders may
read in advance infinitely, i.e. they read the entire subtitle stream.
Then, since the other streams (audio/video) have logically reached EOF,
and the subtitle stream is set to ds->active==true. This will have to be
fixed properly later to account buffering for subtitle-only files
(another corner case) correctly, but for now this is less annoying.
We don't hope to auto-detect them at load time, as that would be too
much of a pain - even FFmpeg requires fetching and parsing of video
packets, and exposes the information only via deprecated API.
But there still needs to be a way to select them by default. This is
also needed to get the first CC packet at all (without seeking back).
This commit also attempts to clean up locking a bit, which is a PITA,
but it's better be careful & clean.
Even though only 1 seek range is supported at the time.
Other than preparation for possibly future features, the main gain is
actually that we finally separate the reporting for the buffering, and
the seek ranges. These can be subtly different, so it's good to have a
clear separation.
This commit also fixes that the ts_reader wasn't rebased to the start
time, which could make the player show "???" for buffered cache amount
in some .ts files and others (especially at the end, when ts_reader
could become higher than ts_max). It also fixes writing the cache-end
field in the demuxer-cache-state property: it checked ts_start against
NOPTS, which makes no sense.
ts_start was never used (except for the bug mentioned above), so get rid
of it completely. This also makes it convenient to move the segment
check for last_ts to the demux_add_packet() function.
Avoids that cache seeking is not possible with files that have audio
frames with no timestamps (such as avi, sometimes mkv sub-packets from
lacing). These would set back_pts (first seekable PTS) to NOPTS, and
thus disable cache seeking completely. Instead, prune such packets until
we find one with timestamps.
One corner case is that the new next good packet might be in the forward
cache. In this case we defer dropping until the next time this code is
run, and the reader position has possibly moved past the drop point.
In theory, start/ts_min could be set to NOPTS, in which case
"pts < start" for a valid pts would always evaluate to false.
Also remove the redundant "in-cache seek is possible.." message, as
there's always another message when cache seeks are done.
The seek range computation ignored streams with no timestamps. For
things like buffer estimation this is OK and wanted, but the seek range
needs to be conservative.
Which parts of the queue are considered forward or backward cache
depends on ds->reader_header. The packet at ds->reader_head, as well as
all packets following it (via the ->next field) are considered forward.
The fw_packs/fw_bytes/bw_bytes must be updated accordingly.
This broke in demux_add_packet(), when ds->reader_head was _not_ set on
the first packet (or before). This could happen since commit
05ae571241, which can require skipping packets (so they immediately end
up in the backbuffer).
With the timeline code, a packet at the start or end of a segment can
refer to an invisible frame. So it doesn't extend the seek range, and
the timestamp should be clipped to the actual segment range.
Also restructure recompute_keyframe_target_pts() to be hopefully less
confusing.
Restores some behavior from before the demuxer cache changes, though
affects mostly just OSD display. The unknown queue state is reserved for
streams with missing or messed up timestamps.
This fixes .cue files with audio files that contain attached pictures to
some degree. demux_timeline.c just discarded packets with unset index,
so the picture was never fed to the decoder.
Although seeking past the cached range will trigger a low level seek, a
seek into the region between cache end and last video key frame would
simply seek to the video key frame. This meant that you could get
"stuck" at the end of the file instead of terminating playback when
trying to seek past the end.
One change is that we fix this by _actually_ allowing SEEK_FORWARD to
seek past the last video keyframe in find_seek_target().
In that case, or otherwise seeking to cache buffer end, it could happen
that we set ds->reader_head=NULL if the seek target is after the current
packet. We allow this, because the end of the cached region is defined
by the existence of "any" packet, not necessarily a key frame. Seeking
there still makes sense, because we know that there is going to be more
packets (or EOF) that satisfy the seek target.
The problem is that just resuming demuxing with reader_head==NULL will
simply return any packets that come its way, even non-keyframe ones.
Some decoders will produce ugly soup in this case. (In practice, this
was not a problem, because seeking at the end of the cached region was
rare before this commit, and also some decoders like h264 will skip
broken frames by default anyway.)
So the other change of this commit is to enable key frame skipping.
As a nasty implementation detail, we use a separate flag, instead of
setting reader_head to the first key frame encounted (reader_head being
NULL can happen after a normal seek or on playback start, and then we
want to mirror the underlying demuxer behavior, for better or worse).
This change is relatively untested, so you get to keep the pieces for
yourself.
Seems like most code dealing with this was for setting it in redundant
cases. Now SEEK_BACKWARD is redundant, and SEEK_FORWARD is the odd one
out.
Also fix that SEEK_FORWARD was not correctly unset in try_seek_cache().
In demux_mkv_seek(), make the arbitrary decision that a video stream is
not required for the subtitle prefetch logic to be active. We might want
subtitles with long duration even with audio only playback, or if the
file is used as external subtitle.
This improves upon the previous commit, and partially rewrites it (and
other code). It does:
- disable the seeking within cache by default, and add an option to
control it
- mess with the buffer estimation reporting code, which will most likely
lead to funny regressions even if the new features are not enabled
- add a back buffer to the packet cache
- enhance the seek code so you can seek into the back buffer
- unnecessarily change a bunch of other stuff for no reason
- fuck up everything and vomit ponies and rainbows
This should actually be pretty usable. One thing we should add are some
properties to report the proper buffer state. Then the OSC could show a
nice buffer range. Also configuration of the buffers could be made
simpler. Once this has been tested enough, it can be enabled by default,
and might replace the stream cache's byte ringbuffer.
In addition it may or may not be possible to keep other buffer ranges
when seeking outside of the current range, but that would be much more
complex.
More the ignore_eof field to the internal demux_stream struct. This is
relatively messy, because the internal struct exists only once the
stream is created, and after that setting the ignore_eof flag is a race
condition. We could bother with adding demux_add_sh_stream() parameters
for this, but let's not. So in theory a tiny race condition is
introduced, which can never be triggered since all demux API functions
are called by the playback thread only anyway.
Fix that ts_offset is accessed without log (this was introduced much
earlier by myself).
Introduce an alternative way of avoiding the annoying EOF reached
messages by not resetting the EOF flags for CC streams when a CC packet
is added. This makes the second commit in the PR which added the
original fix unnecessary.
As another cosmetic change merge the check in cached_demux_control()
into a single if().
In the future, the CC pseudo-stream should probably be replaced with an
entire pseudo-demuxer or such, which would avoid some of the messiness
(or maybe not, we don't know yet).
As usual, the history of these files is a bit murky. It starts with the
initial commit. (At which some development had already been done,
according to the AUTHORS and ChangeLog files at the time, we should be
but covered with relicensing agreements, though.) then it goes on with
complete lack of modularization, which was cleaned up later (cd68e161).
As usual, we don't consider the copyright of the stuff that has been
moved out cleanly.
There were also contributions to generic code by people who could not be
reached or who did not agree to the relicensing, but this was all
removed.
The only patches that we could not relicense and which were still in the
current code in some form are from Dénes Balatoni: 422b0d2a, 32937181.
We could not reach him, so commits f34e1a0d and 18905298 remove his
additions. It still leaves the demux_control() declaration itself, but
we don't consider it copyrightable. It's basically an idiom that existed
in MPlayer before that change, applied to the demuxer struct. (We even
went as far as making sure to remove all DEMUXER_CTRLs the original
author added.)
Commit be54f481 might be a bit of a corner case, but this was rewritten,
and we consider the old copyright removed long ago.
Similar purpose as f34e1a0dee.
Somehow this is much more natural too, and needs less code.
This breaks runtime updates to duration. This could easily be fixed, but
no important demuxer does this anyway. Only demux_raw and demux_disc
might (the latter for BD/DVD). For the latter it might actually have
some importance when changing titles at runtime (I guess?), but guess
what, I don't care.
This is more uniform, and potentially gets rid of some past copyrights.
It might be that this subtly changes caching behavior (it seems before
this, it synced to the demuxer if the length was unknown, which is not
what we want.)
It's all explained in the DOCS changes. Although this option was always
kind of obscure and pointless. Until it is removed, the only reason for
setting it would be to raise the static default limit, so change its
default to INT_MAX so that it does nothing by default.
Instead of enabling it only when a stream-cache is enabled, also try to
enable it independently from that if the demuxer is marked as
is_network.
Also add some code to the EDL code, so EDLs containing network streams
are automatically cached this way.
Extend the OSD info line so that it shows the demuxer cache in this case
(more or less).
I didn't find where or whether options.rst describes how the demuxer
cache is enabled, so no changes there.
"uncached_stream" is a pretty bad name. It could be mistaken for a
boolean, and then its meaning would be inverted. Rename it.
Also add a "caching" field, which signals that the stream is a cache or
reads from a cache. This is easier to understand and more flexible.
This was excessively useless, and I want my time back that was needed to
explain users why they don't want to use it.
It captured the byte stream only, and even for types of streams it was
designed for (like transport streams), it was rather questionable.
As part of the removal, un-inline demux_run_on_thread() (which has only
1 call-site now), and sort of reimplement --stream-dump to write the
data directly instead of using the removed capture code.
(--stream-dump is also very useless, and I struggled coming up with an
explanation for it in the manpage.)
Disabling cache readahead by default until at least 1 track is selected
is mainly for external files and such, where you don't want them to use
up resources until they're actually used.
It doesn't make sense to disable the cache for the demuxer opened for
prefetch. Also, it's fine to let it do that for the main file too (doing
or not doing it is of little consequence). That saves us from having to
distinguish them.
Cover art handling is a disgusting hack that causes a mess in all
components. And this will stay this way. This is the Xth time I've
changed cover art handling, and that will probably also continue.
But change the code such that cover art is injected into the demux
packet stream, instead of having an explicit special case it in the
decoder glue code. (This is somewhat more similar to the cover art hack
in libavformat.)
To avoid that the over art picture is decoded again on each seek, we
need some additional "caching" in player/video.c. Decoding it after each
seek would work as well, but since cover art pictures can be pretty
huge, it's probably ok to invest some lines of code into caching it.
One weird thing is that the cover art packet will remain queued after
seeks, but that is probably not an issue.
In exchange, we can drop the dec_video.c code, which is pretty
convenient for one of the following commits. This code duplicates a
bunch of lower-level decode calls and does icky messing with this weird
state stuff, so I'm glad it goes away.
It has only 1 caller, and is too far appart within the file. I think it
used to have multiple callers, but now it just doesn't make any sense to
keep it separate anymore.
This deals with the estimation of buffered packets, which is used mostly
for display, but also things like pausing on low buffer levels.
If a stream is fully EOF (no more packets), we don't want to include it
in the total buffer amount. This also means we should make ds->eof less
flaky and more stable, so don't reset it in ds_get_packets() (this
function reset ds->eof just to retrigger a packet read attempt - we can
have this slightly simpler). This somewhat fixes buffering display when
e.g. issuing a refresh seek after re-enabling audio/video when playing
with subtitles only.
When switching a subtitle track, the subtitle wasn't necessarily
updated, especially when playback was paused.
Some awfully subtle and complex interactions here.
First off (and not so subtle), the subtitle decoder will read packets
only on explicit update_subtitles() calls, which, if video is active, is
called only when a new video frame is shown. (A simply video frame
redraw doesn't trigger this.) So call it explicitly. But only if
playback is "initialized", i.e. not when it does initial track selection
and decoder init, during which no packets should be read.
The second issue is that the demuxer thread simply will not read new
packets just because a track was switched, especially if playback is
paused. That's fine, but if a refresh seek is to be done, it really
should do this. So if there's either 1. a refresh seek requested, or 2.
a refresh seek ongoing, then read more packets.
Note that it's entirely possible that we overflow the packet queue with
this in unpredicated weird corner cases, but the queue limit will still
be enforced, so this shouldn't make the situation worse.
Don't access MPOpts directly, and always use the new m_config.h
functions for accessing them in a thread-safe way.
The goal is eventually removing the mpv_global.opts field, and the
demuxer/stream-layer specific hack that copies MPOpts to deal with
thread-safety issues.
This moves around a lot of options. For one, we often change the
physical storage location of options to make them more localized,
but these changes are not user-visible (or should not be). For
shared options on the other hand it's better to do messy direct
access, which is worrying as in that somehow renaming an option
or changing its type would break code reading them manually,
without causing a compilation error.
This is for text subtitles. libavformat currently always reads text
subtitles completely on init. This means the underlying stream is
useless and will consume resources for various reasons (network
connection, file handles, cache memory).
Take care of this by closing the underlying stream if we think the
demuxer has read everything. Since libavformat doesn't export whether it
did (or whether it may access the stream again in the future), we rely
on a whitelist. Also, instead of setting the stream to NULL or so, set
it to an empty dummy stream. This way we don't have to litter the code
with NULL checks.
demux_lavf.c needs extra changes, because it tries to do clever things
for the sake of subtitle charset conversion.
The main reason we keep the demuxer etc. open is because we fell for
libavformat being so generic, and we tried to remove corresponding
special-cases in the higher-level player code. Some of this is forced
due to ass/srt mkv/mp4 demuxing being very similar to external text
files. In the future it might be better to do this in a more
straight-forward way, such as reading text subtitles into libass and
then discarding the demuxer entirely, but for aforementioned reasons
this could be more of a mess than the solution introduced by this
commit.
Probably fixes#3456.
Cleaner and makes it easier to change the underlying stream.
mp_property_stream_capture() still directly accesses it directly via
demux_run_on_thread(). This is evil, but still somewhat sane and is not
getting into the way here.
Not sure if I got all field accesses.
It doesn't necessarily have to mean anything bad.
We're still too lazy to provide any more detailed information (e.g.
whether this happened to likely bad interleaving, excessive amount of
packets like with some ASS subs, or that the readahead user option is
limitted by the packet queue size).
When an ogg track upodates metadata, we have to perform a complicated
runtime update due to the demux.c architecture. A detail was broken and
an array was allocated with the previous number of streams, which
usually led to invalid memory write accesses at least on the first
update.
See github commit comment on commit b9ba9a89.
If the PEAK tag is invalid, return an error.
Make the error signalling conventions more uniform by strictly returning
a negative value on error, and treating >=0 as success.
The demuxer layer usually doesn't log per-stream information, and even
the replaygain information was logged only if it came from tags.
So log it in af_volume instead.
...and ignore it. The main purpose is for retrieving per-track
replaygain tags. Other than that per-track tags are not used or accessed
by the playback core yet.
The demuxer infrastructure is still not really good with that whole
synchronization thing (at least in part due to being inherited from
mplayer's single-threaded architecture). A convoluted mechanism is
needed to transport the tags from demuxer thread to user thread. Two
factors contribute to the complexity: tags can change during playback,
and tracks (i.e. struct sh_stream) are not duplicated per thread.
In particular, we update the way replaygain tags are retrieved. We first
try to use per-track tags (common in Matroska) and global tags
(effectively formats like mp3). This part fixes#3405.
Remove the explicit whitelisting of formats for refresh seeks. Instead,
check whether the packet position is somewhat reliable during demuxing.
If there are packets without position, or the packet position is not
monotonically increasing, then do not use them for refresh seeks.
This does not make sure of some requirements, such as deterministic
seeks. If that happens, mpv will mess up a bit on stream switching.
Also, add another method that uses DTS to identify packets, and prefer
it to the packet position method. Even if there's a demuxer which
randomizes packet positions, it hardly can do that with DTS. The DTS
method is not always available either, though. Some formats do not have
a DTS, and others are not always strictly monotonic (possibly due to
libavformat codec parsing and timestamp determination issues).
If the packet read function returns, and EOF was detected, and a seek
was issued in the meantime, then don't use the EOF result. The seek will
be processed later, and reset the EOF state anyway.
The main effect is probably that we don't return EOF to the decoders
(which the playback core resets before issuing the seek), and that we
won't log an EOF message.
Not important, but slightly more correct.
When switching tracks, we normally have the problem that data gets lost
due to readahead buffering. (Which in turn is because we're stubborn and
instruct the demuxers to discard data on unselected streams.) The
demuxer layer has a hack that re-reads discarded buffered data if a
stream is enabled mid-stream, so track switching will seem instant.
A somewhat similar problem is when all tracks of an external files were
disabled - when enabling the first track, we have to seek to the target
position.
Handle these with the same mechanism. Pass the "current time" to the
demuxer's stream switch function, and let the demuxer figure out what to
do. The demuxer will issue a refresh seek (if possible) to update the
new stream, or will issue a "normal" seek if there was no active stream
yet.
One case that changes is when a video/audio stream is enabled on an
external file with only a subtitle stream active, and the demuxer does
not support rrefresh seeks. This is a fuzzy case, because subtitles are
sparse, and the demuxer might have skipped large amounts of data. We
used to seek (and send the subtitle decoder some subtitle packets
twice). This case is sort of obscure and insane, and the fix would be
questionable, so we simply don't care.
Should mostly fix#3392.
Instead of having a separate for each, which also requires separate
additional caching in the demuxer. (The demuxer adds an indirection,
since STREAM_CTRLs are not thread-safe.)
Since this includes the cache speed, this should fix#3003.
This is simpler, because it doesn't have to wait from both threads for
synchronization.
Apart from being simpler/cleaner, this serves vague plans to stop/start
the demuxer thread itself automatically on demand (for the purpose of
reducing unneeded resource usage).
This pause stuff is bothersome and is needed only for a few corner-
cases. This commit removes it from the demuxer public API and replaces
it with a demux_run_on_thread() function and refactors the code which
needed demux_pause(). The next commit will change the implementation.
Commit 503c6f7f essentially removed timestamps from "laces" (Block sub-
divisions), which means many audio packets will have no timestamp.
There's no reason why bitrate calculation can't just delayed to a point
when the next timestamp is known.
Fixes#2903 (no audio bitrate with mkv files).
Ever since a change in mplayer2 or so, relative seeks were translated to
absolute seeks before sending them to the demuxer in most cases. The
only exception in current mpv is DVD seeking.
Remove the SEEK_ABSOLUTE flag; it's not the implied default. SEEK_FACTOR
is kept, because it's sometimes slightly useful for seeking in things
like transport streams. (And maybe mkv files without duration set?)
DVD seeking is terrible because DVD and libdvdnav are terrible, but
mostly because libdvdnav is terrible. libdvdnav does not expose seeking
with seek tables. (Although I know xbmc/kodi use an undocumented API
that is not declared in the headers by dladdr()ing it - I think the
function is dvdnav_jump_to_sector_by_time().) With the current mpv
policy if not giving a shit about DVD, just revert our half-working seek
hacks and always use dvdnav_time_search(). Relative seeking might get
stuck sometimes; in this case --hr-seek=always is recommended.
If a stream is marked as EOF (due to no packets found in reach), then we
need to wakeup the decoder. This is important especially if no packets
are found at the start of the file, so the A/V sync logic actually
starts playback, instead of waiting for packets that will never come.
(It would randomly start playback when running the playback loop due to
arbitrary external events like user input.)
This uses a different method to piece segments together. The old
approach basically changes to a new file (with a new start offset) any
time a segment ends. This meant waiting for audio/video end on segment
end, and then changing to the new segment all at once. It had a very
weird impact on the playback core, and some things (like truly gapless
segment transitions, or frame backstepping) just didn't work.
The new approach adds the demux_timeline pseudo-demuxer, which presents
an uniform packet stream from the many segments. This is pretty similar
to how ordered chapters are implemented everywhere else. It also reminds
of the FFmpeg concat pseudo-demuxer.
The "pure" version of this approach doesn't work though. Segments can
actually have different codec configurations (different extradata), and
subtitles are most likely broken too. (Subtitles have multiple corner
cases which break the pure stream-concatenation approach completely.)
To counter this, we do two things:
- Reinit the decoder with each segment. We go as far as allowing
concatenating files with completely different codecs for the sake
of EDL (which also uses the timeline infrastructure). A "lighter"
approach would try to make use of decoder mechanism to update e.g.
the extradata, but that seems fragile.
- Clip decoded data to segment boundaries. This is equivalent to
normal playback core mechanisms like hr-seek, but now the playback
core doesn't need to care about these things.
These two mechanisms are equivalent to what happened in the old
implementation, except they don't happen in the playback core anymore.
In other words, the playback core is completely relieved from timeline
implementation details. (Which honestly is exactly what I'm trying to
do here. I don't think ordered chapter behavior deserves improvement,
even if it's bad - but I want to get it out from the playback core.)
There is code duplication between audio and video decoder common code.
This is awful and could be shareable - but this will happen later.
Note that the audio path has some code to clip audio frames for the
purpose of codec preroll/gapless handling, but it's not shared as
sharing it would cause more pain than it would help.
Slightly helps with timeline stuff, like EDL. There is no need to keep
network (or even just disk I/O) busy for all segments at the same time,
because 1. the data won't be needed any time soon, and 2. will probably
be discarded anyway if the stream is seeked when segment is resumed.
Partially fixes#2692.
This is mainly a refactor. I'm hoping it will make some things easier
in the future due to cleanly separating codec metadata and stream
metadata.
Also, declare that the "codec" field can not be NULL anymore. demux.c
will set it to "" if it's NULL when added. This gets rid of a corner
case everything had to handle, but which rarely happened.
This slightly changes behavior when seeking with external audio/subtitle
tracks if transport streams and mpeg files are played, as well as
behavior when seeking with such external tracks.
get_main_demux_pts() is evil because it always blocks on the demuxer (if
there isn't already a packet queued). Thus it could lock up the player,
which is a shame because all other possible causes have been removed.
The reduced "precision" when seeking in the ts/mpeg cases (where
SEEK_FACTOR is used, resulting in byte seeks instead of timestamp seeks)
might lead to issues. We should probably drop this heuristic. (It was
introduced because there is no other way to seek in files with PTS
resets with libavformat, but its value is still questionable.)
Slightly change how it is decided when a new packet should be read.
Switch to demux_read_packet_async(), and let the player "wait properly"
until required subtitle packets arrive, instead of blocking everything.
Move distinguishing the cases of passive and active reading into the
demuxer, where it belongs.
The demuxer infrastructure was originally single-threaded. To make it
suitable for multithreading (specifically, demuxing and decoding on
separate threads), some sort of tripple-buffering was introduced. There
are separate "struct demuxer" allocations. The demuxer thread sets the
state on d_thread. If anything changes, the state is copied to d_buffer
(the copy is protected by a lock), and the decoder thread is notified.
Then the decoder thread copies the state from d_buffer to d_user (again
while holding a lock). This avoids the need for locking in the
demuxer/decoder code itself (only demux.c needs an internal, "invisible"
lock.)
Remove the streams/num_streams fields from this tripple-buffering
schema. Move them to the internal struct, and protect them with the
internal lock. Use accessors for read access outside of demux.c.
Other than replacing all field accesses with accessors, this separates
allocating and adding sh_streams. This is needed to avoid race
conditions. Before this change, this was awkwardly handled by first
initializing the sh_stream, and then sending a stream change event. Now
the stream is allocated, then initialized, and then declared as
immutable and added (at which point it becomes visible to the decoder
thread immediately).
This change is useful for PR #2626. And eventually, we should probably
get entirely of the tripple buffering, and this makes a nice first step.
All of these are supported by FFmpeg now. It was disabled by default
too (with FFmpeg).
If compiled against Libav, mpv will lose the ability to read some
subtitle formats (but the most important ones, srt and ass, still should
work).
Most of this is explained in the DOCS additions.
This gives us slightly more sanity, because there is less interaction
between the various parts. The goal is getting rid of the video_offset
entirely.
The simplification extends to the user API. In particular, we don't need
to fix missing parts in the API, such as the lack for a seek command
that seeks relatively to the start time. All these things are now
transparent.
(If someone really wants to know the real timestamps/start time, new
properties would have to be added.)
This loaded external .ass files via libass. libavformat's .ass reader is
now good enough, so use that instead.
Apparently libavformat still doesn't support fonts embedded into text
.ass files, but support for this has been accidentally broken in mpv for
a while anyway. (And only 1 person complained.)
This makes the bitrate properties unavailable, instead of
returning 0 when:
1. No track is selected, or
2. Not enough packets have been read to have a bitrate estimate yet
Make handling of metadata slightly more generic, and add reading of the
"PERFORMER" fields. There are some more fields, but for now let's leave
it at this.
TRACK-specific PERFORMER fields have to be read from the per-chapter
metadata (somewhat obscure).
Fixes#2328.
This works similar to the existing .rar support, but uses libarchive.
libarchive supports a number of formats, including zip and (most of)
rar.
Unfortunately, seeking does not work too well. Most libarchive readers
do not support seeking, so it's emulated by skipping data until the
target position. On backwards seek, the file is reopened. This works
fine on a local machine (and if the file is not too large), but will
perform not so well over network connection.
This is disabled by default for now. One reason is that we try
libarchive on every file we open, before trying libavformat, and I'm not
sure if I trust libarchive that much yet. Another reason is that this
breaks multivolume rar support. While libarchive supports seeking in
rar, and (probably) supports multivolume archive, our support of
libarchive (probably) does not. I don't care about multivolume rar, but
vocal users do.
Instead, force everyone to use the metadata struct and set a "title"
field. This is only a problem for the timeline producers, which set up
chapters manually. (They do this because a timeline is a separate
struct.)
This fixes the behavior of the chapter-metadata property, which never
returned a "title" property for e.g. ordered chapters.
Add --demuxer-max-packets and --demuxer-max-bytes, which control the
maximum size of the packet queue. These can be helpful to avoid
excessive memory usage.
Memory usage is the reason why there's a limit in the first place. If a
file is more or less broken, and audio and video don't line up, the
decoders will fill up the packet queue trying to read more audio or
video, and the maximum sizes are required to avoid unbounded memory
allocation. Being able to override the maximum sizes is useful; either
for restricting memory usage further, or enlarging the sizes when
attempting to play various broken files.
Remove --demuxer-readahead-packets and --demuxer-readahead-bytes. These
were a bit useless. They could force a minimum packet queue size, but
controlling the queue size with --demuxer-readahead-secs is much nicer.
It's fairly certain nobody ever used these options.
Instead of opening a stream and then a demuxer, do both at once with
demux_open_url().
This requires some awkward additions to demuxer_params, because there
are some weird features associated with opening the main file. E.g. the
relatively useless --stream-capture features requires enabling capturing
on the stream before the demuxer is opened, but on the other hand
shouldn't be done on secondary files like external subtitles.
Also relatively bad: since demux_open_url() returns just a demuxer
pointer or NULL, additional error reporting is done via demuxer_params.
Still, at least conceptually, it's ok, and simpler than before.
Nobody wanted to restore this, so it gets the boot.
If anyone still wants to volunteer to restore menu support, this would
be welcome. (I might even try it myself if I feel masochistic and like
wasting a lot of time for nothing.) But if it does get restored, it
should be done differently. There were many stupid things about how it
was done. For example, it somehow tried to pull mp_nav_events through
all the layers (including needing to "buffer" them in the demuxer),
which was needlessly complicated. It could be done simpler.
This code was already inactive, so this commit actually changes nothing.
Also keep in mind that normal DVD/BD playback still works.
