Commit 374600cec0 ("cache: propagate seek failures") changed
stream-level seeks to be done in the calling thread so possible errors
could be reported. This commit included some rationale why doing the
stream-level seeks synchronously was not a big issue. However, the
following fixup commit f4d62dc4a0 changed the seek code to always
synchronize with the cache thread and do seek handling there. This is
a problem because it affects even seeks within already cached data
(which require no stream-level seek). With a network server that sends
data in bursts, the cache thread can be blocked in a read call for a
long time; the added synchronization means that seeking within already
downloaded data can have unnecessary long delays.
Change cache_seek() to check whether the seek is expected to result in
a stream-level seek, and skip synchronization if it is not. This means
that seeks within the cached portion of the file now again happen
immediately without possibly waiting for network.
Signed-off-by: wm4 <wm4@nowhere>
All authors have agreed to the relicensing.
iive has insisted on "LGPL v3 or later", which makes the file LGPL v3.
His commits are the following: 84ec5775089b0d8c680f. All other
contributions are LGPL v2.1. I hope we can remove these changes
completely one day to make this file LGPL v2.1.
iive also authored commit 3934b160a8, but this code is completely gone
today. (fork() and shared memory use was removed completely in favor of
threads.)
This code used to check/free multiple things, so the argument to free()
was not always NULL. After the code was simplified, the free() became
redundant.
It's just wasted memory.
One corner case is when a file grows during playback, but this is rare
and usually happens on-disk only. The cache size was generally limited
before this change already, so no reason to care.
As an unrelated change, move the cache size info to the resize_cache()
function. There's really no reason not to do this, and it's slightly
more informative if the user changes the cache size at runtime.
The cache reader thread actually unlocks the mutex protecting the
underlying stream while reading from it. That's why other code goes out
of its way to run certain stream operations on the cache thread. Do the
same.
We could have this simpler by creating a mechanism that would "park" the
cache thread and make it wait for the lock (while we have it) in order
to gain exclusive access. This could be done in the future.
Eagerly execute seeks to the underlying stream in the cache seek
entrypoint itself. While asynchronous execution is a goal of the cache,
it doesn't matter too much for seeks. They always were executed within
the lock, so the reader was blocked anyway. It's not necessary to ensure
async. execution here either, because seeks are relatively rare, and the
demuxer can just stay blocked for a while.
Fixes: mpv http://samples.mplayerhq.hu/V-codecs/DIV5/ayaneshk-test.avi
Remove the attempted cleverness; keep it dumb.
This strictly calculates the average speed over an at least 1 second
window (longer if I/O blocks it).
Since this doesn't reset the speed anymore when reading stops by going
idle, the results might actually be more accurate now.
Tuning it in a way to be actually useful is too much effort.
As alternative, there's the "buffering" detection, which operates on a
much higher level. The only disadvantage is that it's harder to guess
for the user whether this is a network problem, or if e.g. libavformat
is probing too much data when opening a stream. Maybe the cache-speed
property is helpful here.
For now, do not remove the associated code, but just silence the
warning.
Fixes#3019.
Don't assume EOF if we didn't try to read anything in the first place.
Fixes regressions in particular with low cache sizes, which triggered
the other code paths more often.
Instead of having a separate for each, which also requires separate
additional caching in the demuxer. (The demuxer adds an indirection,
since STREAM_CTRLs are not thread-safe.)
Since this includes the cache speed, this should fix#3003.
Should reflect I/O speed.
This could go into the terminal status line. But I'm not sure how to put
it there, since it already uses too much space, so it's not there yet.
On read, it returns the name of the current DVB program,
on write, it triggers a channel-switch to the program
if it is found in the channel list of the currently active card.
Compared to the dvb-channel property which already exists
and is a pair of integers (card + channel number) this has the limitation
of not switching the card, but is probably of much more common use.
This causes weirdness with the "cache-size" property and option. Only
the read handler of the property included the backbuffer, while all
others did not. Make it consistent, and subtract the backbuffer size
from the cache size.
Fixes#2305.
Allow setting an arbitrary amount, instead of the fixed 50%.
This is nto striclty backwards compatible. The defaults don't change,
but the --cache/--cache-default options now set the readahead portion.
So in practice, users who configured this until now will see the
double amount of cache being used, _plus_ the 75MB default backbuffer
will be in use.
Currently, this is perfectly equivalent, because back_size is hardcoded
to buffer_size/2. But this fixes the logic for the case the back_size
can be configured freely.
Normally, the cache keeps 50% of the buffer for seeking backwards. Until
now, the cache just used the full buffer size at the beginning of a
file, because the 50% normally reserved for the backbuffer are unused.
This caused a problem: when streaming from http, the player would first
read about 150MB (default cache size), then stop until 75MB of the cache
has been played. (Until the 75MB position, the cache is fully used, so
nothing new can be read. After that, part of the backbuffer starts
getting unreserved, and can be used for readahead.) This long read pause
can cause the server to terminate the connection. Reconnecting may be
possible, but if youtube-dl is used, the media URL may have become
invalid.
Fix this by limiting readahead to 50% even if unnecessary. The only
exception is when the whole file would fit in the cache. In this case,
it won't matter if we can't reconnect, because the cache covers
everything anyway, and hopefully the cache will stay valid.
Likely fixes#2000.
There is not much of a reason to have these wrappers around. Use POSIX
standard functions directly, and use a separate utility function to take
care of the timespec calculations. (Course POSIX for using this weird
format for time values.)
An approximate measure to make it exit possibly slightly earlier.
Relatively speaking, some time will pass between cancellation and
the cache actually being requested to exit, so it's good if the
cache returns EOF immediately.
The caller can check for cache interruption instead. There's no need to
define special return values and such. It would be rather hard to make
waiting for the condition and stream cancellation atomic too (and
pointless, since the underlying stream will also be "cancelled" and exit
early), so nothing about cancellation being a separate call will change.
This put some effort into distinguishing between two messages to print -
all worthless. Even more so, this kept printing the message, which
doesn't feel overly useful either. (The message will be printed
repeatedly anyway if network recovers for a while and then gets stuck
again.)
All in all, the demuxer cache triggering the buffering state does a
better job here. But don't remove it completely, since knowing that the
network did nothing for a relatively short time is still useful.
If we're caching a stream with unknown size, and we reach EOF, then
consider the EOF position the file size. Typically makes sense when
reading from a pipe or a http connection that did not send a size.
This message will be printed relatively often once EOF is reached. In
some cases this is rather annoying, for example when playing HLS. (With
HLS, the stream is just a playlist file, while libavformat opens actual
media files without mpv's knowledge, so the cache is completely useless
and hits EOF instantly.)
That it retries reading is apparently a good thing: at least local files
can grow, and even after the player got the EOF, playback _could_ be
resumed by basically polling and detecting that there is more data. So
I'm not changing this behavior yet.
Fix return types and return values to make them more consistent. Some
reformatting and making code more concise.
In stream_reconnect(), avoid the additional mp_cancel_test() call by
moving the "connection lost" message below the mp_cancel_wait() call,
which effectively leads to the same behavior when the stream was already
canceled. (The goal is not to show the message in this case.)
Merge stream_seek_long() into stream_seek(). It was the only caller.
Always clear the eof flag on seeks.
Reduce access to stream internals in cache.c and stream_lavf.c.
...because everything is terrible.
strerror() is not documented as having to be thread-safe by POSIX and
C11. (Which is pretty much bullshit, because both mandate threads and
some form of thread-local storage - so there's no excuse why
implementation couldn't implement this in a thread-safe way. Especially
with C11 this is ridiculous, because there is no way to use threads and
convert error numbers to strings at the same time!)
Since we heavily use threads now, we should avoid unsafe functions like
strerror().
strerror_r() is in POSIX, but GNU/glibc deliberately fucks it up and
gives the function different semantics than the POSIX one. It's a bit of
work to convince this piece of shit to expose the POSIX standard
function, and not the messed up GNU one.
strerror_l() is also in POSIX, but only since the 2008 standard, and
thus is not widespread.
The solution is using avlibc (libavutil, by its official name), which
handles the unportable details for us, mostly. We avoid some pain.
Thanks to STREAM_CTRL_HAS_AVSEEK, we actually know whether CTRL_AVSEEK
is implemented at all, and we can avoid a blocking wait on the cache if
demux_lavf sends CTRL_AVSEEK even if it won't wait. I'm hoping this
can't currently happen, but why hope if we can explicitly prevent it.
It'll make us more robust against future changes in libavformat.
Basically, this will mark the demuxer as seekable with rtmp* and mmsh
protocols. These protocols have network-level time seeking, and whether
you can seek on the byte level does not matter.
Until now, seeking was typically only enabled because of the cache, and
a (nonsensical) warning was shown accordingly.
It still could happen that the server doesn't actually support thse
requests (or simply rejects them), so this is somewhat imperfect.
Especially with other components (libavcodec, OSX stuff), the thread
list can get quite populated. Setting the thread name helps when
debugging.
Since this is not portable, we check the OS variants in waf configure.
old-configure just gets a special-case for glibc, since doing a full
check here would probably be a waste of effort.
This mechanism originates from MPlayer's way of dealing with blocking
network, but it's still useful. On opening and closing, mpv waits for
network synchronously, and also some obscure commands and use-cases can
lead to such blocking. In these situations, the stream is asynchronously
forced to stop by "interrupting" it.
The old design interrupting I/O was a bit broken: polling with a
callback, instead of actively interrupting it. Change the direction of
this. There is no callback anymore, and the player calls
mp_cancel_trigger() to force the stream to return.
libavformat (via stream_lavf.c) has the old broken design, and fixing it
would require fixing libavformat, which won't happen so quickly. So we
have to keep that part. But everything above the stream layer is
prepared for a better design, and more sophisticated methods than
mp_cancel_test() could be easily introduced.
There's still one problem: commands are still run in the central
playback loop, which we assume can block on I/O in the worst case.
That's not a problem yet, because we simply mark some commands as being
able to stop playback of the current file ("quit" etc.), so input.c
could abort playback as soon as such a command is queued. But there are
also commands abort playback only conditionally, and the logic for that
is in the playback core and thus "unreachable". For example,
"playlist_next" aborts playback only if there's a next file. We don't
want it to always abort playback.
As a quite ugly hack, abort playback only if at least 2 abort commands
are queued - this pretty much happens only if the core is frozen and
doesn't react to input.
Don't reconnect to the cache (since the cached stream already handles
reconnection). This is necessary, because since commit 0b428e44 the
"streaming" field (which also controls whether attempting to reconnect
makes sense at all) is inherited to the cache stream wrapper.
Also, let the stream reset its own position on reconnect. This removes
some assumptions and messy handling from the reconnect function.
Make sure the cache is dropped on reconnect. This takes care of
readjusting the stream position if necessary. (Also drop the cache on
DVB channel switching commands.)
This didn't work, because the timebase was wrong. According to the
ffmpeg doxygen, if the stream index is -1 (which is what we used), the
timebase is AV_TIME_BASE. But this didn't work, and it really expected
the stream's timebase. Quite "surprising", since this feature
(avio_seek_time) is used by rtmp only.
Fixing this properly is too hard, so hack-fix our way around it.
STREAM_CTRL_SEEK_TO_TIME is also used by DVD/BD, so a new
STREAM_CTRL_AVSEEK is added. We simply pass-through the request
verbatim.
This adds a thread to the demuxer which reads packets asynchronously.
It will do so until a configurable minimum packet queue size is
reached. (See options.rst additions.)
For now, the thread is disabled by default. There are some corner cases
that have to be fixed, such as fixing cache behavior with webradios.
Note that most interaction with the demuxer is still blocking, so if
e.g. network dies, the player will still freeze. But this change will
make it possible to remove most causes for freezing.
Most of the new code in demux.c actually consists of weird caches to
compensate for thread-safety issues (with the previously single-threaded
design), or to avoid blocking by having to wait on the demuxer thread.
Most of the changes in the player are due to the fact that we must not
access the source stream directly. the demuxer thread already accesses
it, and the stream stuff is not thread-safe.
For timeline stuff (like ordered chapters), we enable the thread for the
current segment only. We also clear its packet queue on seek, so that
the remaining (unconsumed) readahead buffer doesn't waste memory.
Keep in mind that insane subtitles (such as ASS typesetting muxed into
mkv files) will practically disable the readahead, because the total
queue size is considered when checking whether the minimum queue size
was reached.
We used a complicated and approximate method to cache the stream
timestamp, which is basically per-byte. (To reduce overhead, it was only
cached per 8KB-block, so it was approximate.)
Simplify this, and read/keep the timestamp only on discontinuities. This
is when demux_disc.c actually needs the timestamp.
Note that caching is currently disabled for dvdnav, but we still read
the timestamp only after some data is read. libdvdread behaves well, but
I don't know about libbluray, and the previous code also read the
timestamp only after reading data, so try to keep it safe.
Also drop the start_time offset. It wouldn't be correct anymore if used
with the cache, and the idea behind it wasn't very sane either (making
the player to offset the initial playback time to 0).
DVD and Bluray (and to some extent cdda) require awful hacks all over
the codebase to make them work. The main reason is that they act like
container, but are entirely implemented on the stream layer. The raw
mpeg data resulting from these streams must be "extended" with the
container-like metadata transported via STREAM_CTRLs. The result were
hacks all over demux.c and some higher-level parts.
Add a "disc" pseudo-demuxer, and move all these hacks and special-cases
to it.
(Again.)
This time, we simply make it event-based, as it should be. This is done
for both demuxer metadata and stream metadata.
For some ogg-over-icy streams, 2 updates are reported on stream start.
This is because libavformat reports an update right on start, while
including the same info in the "static" metadata. I don't know if that's
a bug or a feature.
When resizing the cache, the buffer for the DVD timestamps is
initialized with 0. This causes the player to always return playback
position 0 with any file format (not just DVD), and also makes all
relative seeks relative to position 0. Fix this by clearing the
timestamps explicitly.
Closes#899.
CC: @mpv-player/stable
wm4
Uoti Urpala
Oliver Freyermuth
Marcin Kurczewski