The demuxer cache employs a strange method to make track switching
instant with caching enabled. Normally this would mean you have to wait
until the cache has played out (and you get new packets, including
packets from the newly selected track), or you have to perform a slow
high level seek (decoding video again etc.). The strange method is that
it performs a demuxer-level seek without a high level seek so it looks
like a continuous stream to the decoder, and the newly select stream
gets packets at the current playback position. This is called a refresh
seek.
This works only if some weird heuristics work. It needs a packet "unique
ID", for which it uses either dts or pts. The value must be strictly
monotonic increasing. If this doesn't work, the referesh seek can't be
executed, and the user has to wait until the end of the cache. Sometimes
there are files that simply do not work.
In the present case, there's actually a hack that we can extend. Packets
with unset position are likely generated by the parser, and the hack
which this commit touches simply attempts to make up a new (hopefully
unique) position value, even if the value itself makes no sense. It only
ha to be deterministic.
It turns out libavcodec sometimes output packets with repeating position
values. This commit tries to handle this case too with the same hack.
Fixes: #7498
Preparation for a future commit. The demuxer queues might be read from
other threads than the one to issue the seek, and passing SEEK_BLOCK
with such a seek will provide a convenient way to synchronize this.
In this case the video track has seek_start == seek_end, and due to the
"seek_start >= seek_end" condition, this was considered broken, and no
seek range was created, breaking cached seeking.
Fix this by allowing the case if they're equal, and a valid timestamp.
(NB: seeking backward in this will still jump to position 0, because it
is the video timestamp. This is unfortunately how it's supposed to work.
HR-seeks will also do this, but decode and skip the entire audio until
the seek target, so it will mostly appear to work.)
Remove some redundant fields that controlled or indicated whether the
demuxer was/could/should prefetch. Redefine how the eof/reading fields
work.
The in->eof field is now always valid, instead of weirdly being reset to
false in random situations. The in->reading field now corresponds to
whether the demuxer thread is working at all, and is reset if it stops
doing anything.
Also, I always found it stupid that dequeue_packet() forced the demuxer
thread to retry reading if it was EOF. This makes little sense, but was
probably added for files that are being appended to (running downloads).
It makes no sense, because if the cache really tried to read until file
EOF, it would encounter partial packets and throw errors, so all is lost
anyway. Plus stream_file now handles this better. So stop this behavior,
but add a temporary option that enables the old behavior.
I think checking for ds->eager when enabling prefetching never really
made sense (could be debated, but no, not really). On the other hand,
the change above exposed a missing wakeup in the backward demuxing code.
Some chances of regressions that could make it stuck in certain states
or so, or incorrect demuxer cache state reporting to the player
frontend.
A negative subtitle delay means that subtitles from the future should be
shown earlier. With muxed subtitles, subtitle packets are demuxed along
with audio and video packets. But since they are demuxed "lazily",
nothing guarantees that subtitle packets from the future are available
in time.
Typically, the user-observed effect is that subtitles do not appear at
all (or too late) with large negative --sub-delay values, but that using
--cache might fix this.
Make this behave better. Automatically extend read-ahead to as much as
needed by the subtitles. It seems it's the easiest to pass the subtitle
render timestamp to the demuxer in order to guarantee that everything is
read. This timestamp based approach might be fragile, so disable it if
no negative sub-delay is used.
As far as the player frontend part is concerned, this makes use of the
code path for external subtitles, which are not lazily demuxed, and may
already trigger waiting.
Fixes: #7484
Subtitle tracks are usually "lazy" (ds->eager=false), There are a number
of weird special cases associated with it. One of them is that they have
some sort of "temporary" EOF (to signal that there isn't a packet right
now, and the decoder should not block playback by waiting for more
packets). In a the next commit, I want to call mark_stream_eof() in case
of (some) of these temporary EOFs.
The problem is that mark_stream_eof() also calls the functions touched
by this commit. Basically they shouldn't do any complex work due to
these temporary EOFs (because they might happen very often). It turns
out that lazy tracks barely matter here: they do not extend the seek
range of a packet/EOF happens on them, they do not trigger seek range
joining, and they do not support backward demuxing.
This change should enable the following commit, while not causing any
behavior changes (i.e. bugs) with the current state.
These have ->segmented set (so the codec can be initialized properly),
but have no segment start or end times. This code was (probably) the
only thing which didn't handle this case.
While paused, the decoders typically stop reading data from the demuxer.
But for some reason, the file size is returned as a public field in
struct demuxer (wat...), and updated only when the packet reading
function is called. This caused the file size property to always return
the same value when paused, even though the demuxer thread was reading
new data, and the internal file size was updated.
Fix with a simple hack.
Instead of every packet. Doing it every packet led to the performance
regression mentioned in the fstat() commit. This should now be over, but
out of being careful, don't query the file size that often. This is only
used for user interface things, so this should not cause any problems.
For the sake of leaving the code compact, abuse another thing that is
updated only every second (speed statistics).
Add something that will access an URL embedded in EDL only when the
track it corresponds to is actually selected. This is meant to help with
ytdl_hook.lua and to improve loading speeds.
In theory, all this stuff is available to any mpv user, but discourage
using it, as it's so specialized towards ytdl_hook.lua, that there's
danger we'll just break this once ytdl_hook.lua stops using it, or
similar.
Mostly untested.
Until now, they were all just added to options.c (e.g. demux_mkv_conf).
This adds a mechanism which can be used to add future options in a
(very) slightly more elegant way.
When switching tracks, the data for the new track is missing by the
amount of data prefetched. This is because all demuxers return
interleaved data, and you can't just seek the switched track alone.
Normally, this would mean that the new track simply gets no data for a
while (e.g. silence if it's an audio track). To avoid this, mpv performs
a special "refresh seek" in the demuxer, which tries to resume demuxing
from an earlier position, in a way that does not disrupt decoding for
the non-changed tracks. (Could write a lot about the reasons for doing
something so relatively complex, and the alternatives and their
weaknesses, but let's not.)
This requires that the demuxer can tell whether a packet after a seek
was before or after a previously demuxed packet, sort of like an unique
ID. The code can use the byte position (pos) and the DTS for this. The
DTS is normally strictly monotonically increasing, the position in most
sane file formats too (notably not mp4, in theory).
The file at hand had DTS==NOPTS packets (which is fine, usually this
happens when PTS can be used instead, but the demux.c code structure
doesn't make it easy to use this), and pos==-1 at the same time. The
latter is what libavformat likes to return when the packet was produced
by a "parser" (or in other words, packets were split or reassembled),
and the packet has no real file position. That means the refresh seek
mechanism has no packet position and can't work.
Fix this by making up a pseudo-position if it's missing. This needs to
set the same value every time, which is why it does not work for
keyframe packets (which, by definition, could be a seek target).
Fixes: #7306 (sort of)
Demuxers can call demux_close_stream() to close the underlying stream if
it's not needed anymore. (Useful to release "heavy" resources like FDs
and sockets. Plus merely keeping a file open can have visible side
effects such as inability to unmount a filesystem or, on Windows, to do
anything with the file.)
Until now, this set demuxer->stream to a dummy stream, because most code
used to assume that the stream field is non-NULL. But this requirement
disappeared (in some cases, the stream field is already NULL), so stop
doing that. demux_lavf.c, one of the demuxers which calls this function,
still had some of this, though.
mpv has a very weak and very annoying policy that determines whether a
playlist should be used or not. For example, if you play a remote
playlist, you usually don't want it to be able to read local filesystem
entries. (Although for a media player the impact is small I guess.)
It's weak and annoying as in that it does not prevent certain cases
which could be interpreted as bad in some cases, such as allowing
playlists on the local filesystem to reference remote URLs. It probably
barely makes sense, but we just want to exclude some other "definitely
not a good idea" things, all while playlists generally just work, so
whatever.
The policy is:
- from the command line anything is played
- local playlists can reference anything except "unsafe" streams
("unsafe" means special stream inputs like libavfilter graphs)
- remote playlists can reference only remote URLs
- things like "memory://" and archives are "transparent" to this
This commit does... something. It replaces the weird stream flags with a
slightly clearer "origin" value, which is now consequently passed down
and used everywhere. It fixes some deviations from the described policy.
I wanted to force archives to reference only content within them, but
this would probably have been more complicated (or required different
abstractions), and I'm too lazy to figure it out, so archives are now
"transparent" (playlists within archives behave the same outside).
There may be a lot of bugs in this.
This is unfortunately a very noisy commit because:
- every stream open call now needs to pass the origin
- so does every demuxer open call (=> params param. gets mandatory)
- most stream were changed to provide the "origin" value
- the origin value needed to be passed along in a lot of places
- I was too lazy to split the commit
Fixes: #7274
The demuxer_id (exported in as "src-id" property) is supposed to be the
native stream ID, as it exists in the file, or -1 if that does not exist
(actually any negative value), or if it is unknown.
Until now, an ID was made up if it was missing. That seems like strange
non-sense, and I can't find the reason why it was done. But it was
probably for convenience by the EDL stuff or so.
Stop doing this. Fortunately, the src-id property was documented as
being unavailable if the ID is not known. Even the code for this was
present, it was just inactive until now. Extend input.rst with some
explanations.
Also fixing 3 other places where negative demuxer_id was ignored or
avoided.
The old code made it depend on ->seekable. If it isn't seekable, and
something discarded the data, then it'll just show an error message,
which will at least be somewhat informative. If no data was discarded,
the seek call is always a no-op.
There's a weird "timeline" condition in the old code; this doesn't
matter anymore, because timeline stuff does not pass streams down to
nested demuxers anymore.
This partially reverts commit a9d83eac40
("Remove optical disc fancification layers").
Mostly due to the timestamp crap, this was never really going to work.
The playback layer is sensitive to timestamps, and derives the playback
time directly from the low level packet timestamps. DVD/BD works
differently, and libdvdnav/libbluray do not make it easy at all to
compensate for this. Which is why it never worked well, but not doing it
at all is even more awful.
demux_disc.c tried this and rewrote packet timestamps from low level TS
to playback time. So restore demux_disc.c, which should bring behavior
back to the old often non-working but slightly better state.
I did not revert anything that affects components above the demuxer
layer. For example, the properties for switching DVD angles or listing
disc titles are still gone. (Disc titles could be reimplemented as
editions. But not by me.)
This commit modifies the reverted code a bit; this can't be avoided,
because the internal API changed quite a bit. The old seek resync in
demux_lavf.c (which was a hack) is replaced with a hack. SEEK_FORCE and
demux_params.external_stream are new additions.
Some of this could/should be further cleaned up. If you don't want
"proper" DVD/BD support to disappear, you should probably volunteer.
Now why am I wasting my time for this? Just because some idiot users are
too lazy to rip their ever-wearing out shitty physical discs? Then why
should I not be lazy and drop support completely? They won't even be
thankful for me maintaining this horrible garbage for no compensation.
This passed all streams to mp_recorder_create(), even disabled ones. The
disabled streams never get packets, so recorder.c eventually errors out
with unrelated-looking errors. The reason is that recorder.c waits for
packets to appear on other streams, which in turn is because libavformat
refuses to mux empty streams anyway.
recorder.c could call demux_stream_is_selected(), which would have made
the patch much smaller. But this feels like a bad idea, since recorder.c
should use sh_stream only for metadata (and not in an "active" way), nor
should it care what demux.c is currently doing with it. So make the API
user (demux.c) pass only the streams it really wants.
Fixes: #6999
Although this was sort of elegant, it just seems to complicate things
slightly. Originally, the API meant that you cache mp_recorder_sink
yourself (which would avoid the mess of passing an index around), but
that too seems slightly roundabout.
In a later change, I want to change the set of streams passed to
mp_recorder_create(), and then I'd have to keep track of the index for
each stream, which would suck. With this commit, I can just pass the
unambiguous sh_stream to it, and it will be guaranteed to match the
correct stream.
The disadvantages are barely worth discussing. It's a new linear search
per packet, but usually only 2 to 4 streams are active at a time. Also,
in theory a user could want to write 2 streams using the same sh_stream
(same metadata, just writing different packets or so), but in practice
this is never done.
in->eof is used as an indicator whether reading packets still makes
sense. (Without this, the prefetcher would obviously burn CPU by
retrying reading even though everything has been read.)
This was not reset properly after seeks were performed. It led to
getting stuck in at least one corner case: when enabling a track, the
demuxer would seek backwards to get new packets from the current
playback position ("refresh seeks"). But if playback was paused, and EOF
was previously reached, it would not try to read packers again due to
in->eof being false. There was not anything else that would make it
retry reading, so it was stuck in a weird underrun/buffering state.
Fixes: #6986
This was a leftover from commit b2752321 which fixed#6522 but after
the recent demux refactoring this fix is superseded by commit 0f6cda4ab.
Remove the redundant update call.
Before this commit, EDL or CUE files did not properly enable the cache
if they were on "slow" media (stream->streaming==true). This happened
because the stream is unset for demux_timeline, so the streaming flag
could not be queried anymore.
Fix this by adding this flag to struct demuxer, and propagate it exactly
like the is_network flag. is_network is not used for checking the cache
options anymore, and its main function seems to be something else.
Normal http streams set the streaming flag already.
This should fix#6958.
Helper for the ab-loop-dump-cache command, see manpage additions.
This is kind of shit. Not only is this a very "special" feature, but it
also vomits more messy code into the big and already bloated demux.c,
and the implementation is sort of duplicated with the dump-cache code.
(Except it's different.) In addition, the results sort of depend what a
video player would do with the dump-cache output, or what the user wants
(for example, a user might be more interested in the range of output
audio, instead of the video).
But hey, I don't actually need to justify it. I'm only justifying it for
fun.
That's right, and it's probably not the end of it. I'll just claim that
I have no idea how to create a proper user interface for this, so I'm
creating multiple partially-orthogonal, of which some may work better in
each of its special use cases.
Until now, there was --record-file. You get relatively good control
about what is muxed, and it can use the cache. But it sucks that it's
bound to playback. If you pause while it's set, muxing stops. If you
seek while it's set, the output will be sort-of trashed, and that's by
design.
Then --stream-record was added. This is a bit better (especially for
live streams), but you can't really control well when muxing stops or
ends. In particular, it can't use the cache (it just dumps whatever the
underlying demuxer returns).
Today, the idea is that the user should just be able to select a time
range to dump to a file, and it should not affected by the user seeking
around in the cache. In addition, the stream may still be running, so
there's some need to continue dumping, even if it's redundant to
--stream-record.
One notable thing is that it uses the async command shit. Not sure
whether this is a good idea. Maybe not, but whatever. Also, a user can
always use the "async" prefix to pretend it doesn't.
Much of this was barely tested (especially the reinterleaving crap),
let's just hope it mostly works. I'm sure you can tolerate the one or
other crash?
It makes some slight sense and helps with one of the following commits.
Also rename that other function to make it sound less similar to
find_seek_target().
Always set max_bytes_bw to 0 if seekable cache is disabled, instead at
the place of its use. This is the only use of it, so the commit should
not change any behavior.
(Alternatively, this could drop the max_bytes_bw variable, use the
option directly, and keep the old code that resets it on use of the
cache is disabled.)
Until now, the following could happen: if you set a 1GB forward cache,
and a 1GB backward cache, and you opened a 2GB file, it would prune away
the data cached at the start as playback progressed past the 50% mark.
With this commit, nothing gets pruned, because the total memory usage
will still be 2GB, which equals the total allowed memory usage of 1GB +
1GB.
There are no explicit buffers (every packet is malloc'ed and put into a
linked list), so it all comes down to buffer size computations. Both
reader and prune code use these sizes to decide whether a new packet
should be read / an old packet discarded. So just add the remaining free
"space" from the forward buffer to the available backward buffer. Still
respect if the back buffer is set to 0 (e.g. unseekable cache where it
doesn't make sense to keep old packets).
We need to make sure that the forward buffer can always append, as long
as the forward buffer doesn't exceed the set size, even if the back
buffer "borrows" free space from it. For this reason, always keep 1 byte
free, which is enough to allow it to read a new packet. Also, it's now
necessary to call pruning when adding a packet, to get back "borrowed"
space that may need to be free'd up after a packet has been added.
I refrained from doing the same for forward caching (making forward
cache use unused backward cache). This would work, but has a
disadvantage. Assume playback starts paused. Demuxing will stop once the
total allowed low total cache size is reached. When unpausing, the
forward buffer will slowly move to the back buffer. That alone will not
change the total buffer size, so demuxing remains stopped. Playback
would need to pass over data of the size of the back buffer until
demuxing resume; consider this unacceptable. Live playback would break
(or rather, would not resume in unintuitive ways), even normal streaming
may break if the server invalidates the URL due to inactivity. As an
alternative implementation, you could prune the back buffer immediately,
so the forward buffer can grow, but then the back buffer would never
grow. Also makes no sense.
As far as the user interface is concerned, the idea is that the limits
on their own aren't really meaningful, the purpose is merely to vaguely
restrict the cache memory usage. There could be just a single option to
set the total allowed memory usage, but the separate backward cache
controls the default ratio of backward/forward cache sizes. From that
perspective, it doesn't matter if the backward cache uses more of the
total buffer than assigned, if the forward buffer is complete.
The last_eof field is the last known EOF state from the underlying
demuxer. Normally, seeks reset it, because obviously if seek back into
the middle of the file, you don't want last_eof to have a "wrong" value
for a short time window (until a packet is read, which would reset the
field to its correct value).
This shouldn't happen during cache seeks, because you don't touch the
underlying demuxer state.
At first, I made this change because some other work in progress
required it. It turned out that it was unnecessary, but keep the change
anyway, since it's still correct and makes the logic cleaner.
Make most of the demuxer options runtime-changeable. This includes the
cache options and stream recording. The manpage documents some of the
possibly weird issues related to this.
In particular, the disk cache isn't shuffled around if the setting
changes at runtime.
Although this is not useful in general, it makes --stream-record work
with a certain video streaming service by a large dystopian company.
In the general case, this fails because normal muxing can, quite
obviously, not handle the segmented metadata in the packets. (There
isn't even a file format which could handle these, except possibly mp4.)
On the other hand, ytdl merely uses timeline/EDL to emulate DASH
streaming (unfortunately), which does not use the segmented stuff, and
stream recording will actually work.
Retarded webshit streaming protocols (well, DASH) chop a stream into
small fragments, and move unchanging header parts to an "init" fragment
to save some bytes (in the case at hand about 300 bytes for each
fragment that is 100KB-200KB, sure was worth it, fucking idiots).
Since mpv uses an even more retarded hack to inefficiently emulate DASH
through EDL, it opens a new demuxer for every fragment. Thus the
fragment needs to be virtually concatenated with the init fragment. (To
be fair, I'm not sure whether the alternative, reusing the demuxer and
letting it see a stream of byte-wise concatenated fragmenmts, would
actually be saner.)
demux_lavc.c contained a hack for this. Unfortunately, a certain shitty
streaming site by an evil company, that will bestow dytopia upon us soon
enough, sometimes serves webm based DASH instead of the expected mp4
DASH. And for some reason, libavformat's mkv demuxer can't handle the
init fragment or rejects it for some reason. Since I'd rather eat
mushrooms grown in Chernobyl than debugging, hacking, or (god no)
contributing to FFmpeg, and since Chernobyl is so far away, make it work
with our builtin mkv demuxer instead.
This is not hard. We just need to copy the hack in demux_lavf.c to
demux_mkv.c. Since I'm not _that_ much of a dumbfuck to actually do
this, remove the shitty gross demux_lavf.c hack, and replace it by a
slightly less bad generic implementation (stream_concat.c from the
previous commit), and use it on all demuxers. Although this requires
much more code, this frees demux_lavf.c from a hack, and doesn't require
adding a duplicated one to demux_mkv.c, so to the naive eye this seems
to be a much better outcome.
Regarding the code, for some reason stream_memory_open() is never meant
to fail, while stream_concat_open() can in extremely obscure situations,
and (currently) not in this case, but we handle failure of it anyway.
Yep.
Timeline (demux_timeline, for EDL and mkv ordered chapters) are a mess,
because it's the only nested demuxer case. Part of the mess comes from
shared struct stream pointers. This makes no sense, because the wrapper
(demux_timeline) doesn't have any business setting it.
Try to lessen it by not passing down streams. Instead, pass down NULL.
This prevents unintended interference, and tightens the ownership rules.
Now a demuxer always owns its stream.
On the other hand, demuxer->stream can now be NULL. This was never the
case before, and consequently there will be new bugs. At least they will
be spotted, because they've been bugs before.
struct stream is also used to access stream properties (such as whether
something is considered a network stream). Most of these have been
mirrored in struct demuxer (because the frontend has been forbidden to
access struct stream because of threading). But during initialization
was still used, so introduce an awkward struct parent_stream_info, which
unifies these.
Commit e0419fb181 changed demux_is_network_cached() to use
demuxer->stream->streaming instead of demuxer->is_network. To enable
timeline stuff to use the cache anyway, change it so that both flags can
contribute to it. The stream NULL-check is obviously due to changes in
this commit.
Instead of having to rely on the protocol matching, make a function that
creates a stream from a stream_info_t directly. Instead of going through
a weird indirection with STREAM_CTRL, add a direct argument for non-text
arguments to the open callback. Instead of creating a weird dummy
mpv_global, just pass an existing one from all callers. (The latter one
is just an artifact from the past, where mpv_global wasn't available
everywhere.)
Actually I just wanted a function that creates a stream without any of
that bullshit. This goal was slightly missed, since you still need this
heavy "constructor" just to setup a shitty struct with some shitty
callbacks.
The previous commit broke audio playback (maybe this is what 4. was
about?). But it wasn't the fault of the commit; it just exposed
pre-existing issues.
If the packet queue search can't get all packets, it checked
queue->is_bof to see whether there could be previous packets. But
obviously, is_bof can be set, even if the search start packet wasn't the
first one.
This was especially observable with audio, because audio by default
needs preroll packets, and plays artifacts if they're missing.
Fix by using the BOF playback condition for this purpose too.
In theory, backward demuxing does way too much work by doing a full
cache seek every time you need to step back through a packet queue. In
theory, it would be exceedingly more efficient to just iterate backwards
through the queue, but this is not possible because I'm too stingy to
add 8 bytes per packet for backlinks. (In theory, you could probably
come up with some sort of deque, that'd allow efficient iteration into
any direction, but other requirements make this tricky, and I'm
currently too dumb/lazy to do this. For example, the queue can grow to
millions of elements, all while packet pointers need to stay valid.)
Another possibility is to "locally" seek the queue. This still has less
overhead than a full seek.
Also, it just so happens that, as a side effect, this avoids performing
range merging, which commit f4b0e7e942 "broke". That wasn't a bug at
all, but since range joining is relatively slow, avoiding it is good.
This is really just a coincidental side effect, I'm not even quite sure
why it happens this way.
There are 4 ugly things about this change:
1. To get a keyframe "before" a certain one, we recompute the target
PTS, and then subtract 0.001 as arbitrary number to "fudge" it. This
isn't the first place where this is done, and hey, it wasn't my damn
idea that MPlayer should use floats for timestamps. (At first, it even
used 32 bit timestamps.)
2. This is the first time reader_head is reset to an earlier position
outside of the seek code. This might cause conceptual problems since
this code is now "duplicated".
3. In theory, find_backward_restart_pos() needs to be run again after
the backstep. Normally, the seek code calls it explicitly. We could call
it right in the new code, but then the damn function would be recursive.
We could shuffle the code around to make it a loop, but even then
there'd be an offchance into running into an unexpected corner case (aka
subtle bug), where it would loop forever. To avoid refactoring the code
and having to think too hard about it, make it deferred - add some new
state and the check_backward_seek() function for this. Great, even more
subtle mutable state for this backwards shit.
4. I forgot this one, but I can assure you, it's bad.
Without doubt someone will have to clean up this slightly in the future
(or rip it out), but it won't be me.
This code tries to determine the "current" position, which is used as
base for the seek target when it needs to seek back more (the point is
to prevent seeking back too far). But compute_keyframe_times() almost
computes the same thing, so use that. Unfortunately needs a forward
declaration.
("Almost", because it differs in some details that should not really
matter.)
demux_packet.next should not be used outside of demux.c, and in this
case it's a packet that was just passed to demux.c from the outside.
demux_packet.stream is already set by the demuxer, and this is assured
by the add_packet_locked() caller.
We don't care much about this case, because backward playback can fail
terribly without a good way to detect it, so this was fine.
However, this froze in certain situations. Reading from a subtitle file
for which backward demuxing failed could make it get stuck in
demux_read_packet_async() in unthreaded mode. (That we don't support
backwards subtitle decoding anyway doesn't matter for this.)
So aggressively disable backward demuxing to prevent worse in these
situations. The behavior will still be awful, because the frontend is
still in backwards playback mode, but at least it won't freeze.
wm4
Gunnar Marten