Previously, all stream protocols were a static list in mpv. This is okay
for own builtin stuff, but for protocols that depend on ffmpeg it's not
so great. Support for certain protocols may or may not be enabled in a
user's ffmpeg and the protocol list that mpv generates should ideally
match this. Fix this by implementing a get_protocols method for
stream_lavf that will have different results depending on the ffmpeg mpv
is built against. We keep the safe and unsafe protocols separation. The
former is essentially a whitelist. Any protocol that is found in ffmpeg
but is not in the safe whitelist is considered unsafe. In the stream
list, ffmpeg is moved to the bottom so any possible protocols that are
added in the future don't automatically take precedence over any builtin
mpv ones.
Reading an entire file is a common operation, meanwhile
stream_file will happily open a directory. This doesn't match well.
Analogously for open_output_stream().
c784820454 introduced a bool option type
as a replacement for the flag type, but didn't actually transition and
remove the flag type because it would have been too much mundane work.
Recent versions of mpv have applied security checks to --playlist
that previously only existed if playlist files were played as an
input directly. This commit documents this change and how to work
around it, in the event that playlist files are trusted.
Add support for reading a byte range from a stream via
the `slice://` protocol.
Syntax is `slice://start[-end]@URL` where end is a maximum
(read until end or eof).
Size suffixes support in `m_option` is reused so they can
be used with start/end.
This can be very useful with e.g. large MPEGTS streams with
corruption or time-stamp jumps or other issues in them.
Signed-off-by: Mohammad AlSaleh <CE.Mohammad.AlSaleh@gmail.com>
Change it to strictly accept local paths only. No more http://, no more
$HOME expansion with "~/" or mpv config path expansion with "~~/". This
should behave as if passing a path directly to open().
Reduce annoying log noise to further facilitate it using as open()
replacement.
The buffer can be larger than the normal size when "peeking" is used
(such as done with some file formats, where a large number of bytes masy
need to be "peeked" at the beginning, because FFmpeg). Once normal
operation resumes, it's supposed to free this buffer again. Apparently
this didn't happen as intended, because normal reading did have no way
to discard back buffer before/while resizing the buffer. There's only a
path for discarding the back buffer when actually reading.
It seems like this unfortunately needs 2 code paths for discarding old
data. Just put it into stream_resize_buffer(), where it's rather
non-tricky (discarding can be done by adjusting the copy offset when
moving data to the new allocation). The function now drops old data if
it doesn't fit into the allocation. The caller must ensure that the new
size is sufficient; the function signature changes only so the size of
the implicitly guaranteed kept part can be checked with assert().
Change all OPT_* macros such that they don't define the entire m_option
initializer, and instead expand only to a part of it, which sets certain
fields. This requires changing almost every option declaration, because
they all use these macros. A declaration now always starts with
{"name", ...
followed by designated initializers only (possibly wrapped in macros).
The OPT_* macros now initialize the .offset and .type fields only,
sometimes also .priv and others.
I think this change makes the option macros less tricky. The old code
had to stuff everything into macro arguments (and attempted to allow
setting arbitrary fields by letting the user pass designated
initializers in the vararg parts). Some of this was made messy due to
C99 and C11 not allowing 0-sized varargs with ',' removal. It's also
possible that this change is pointless, other than cosmetic preferences.
Not too happy about some things. For example, the OPT_CHOICE()
indentation I applied looks a bit ugly.
Much of this change was done with regex search&replace, but some places
required manual editing. In particular, code in "obscure" areas (which I
didn't include in compilation) might be broken now.
In wayland_common.c the author of some option declarations confused the
flags parameter with the default value (though the default value was
also properly set below). I fixed this with this change.
This required libsmbclient, which is a heavy dependency, and as a
library, has all kinds of problems. For one, the API requires completely
unacceptable global state (in particular, leaks auth state), and is not
thread-safe (meaning concurrent reads to multiple files block each
other).
There are better replacements: you can use the Linux kernel's builtin
CIFS support, fusesmb, or contribute supoport for libdsm.
stream_seek() might somewhat show up in the profiler, even if it's a
no-OP, because of the MP_TRACE() call. I find this annoying. Otherwise,
this should be of no consequence, and should not break or improve
anything.
mpv has a very weak and very annoying policy that determines whether a
playlist should be used or not. For example, if you play a remote
playlist, you usually don't want it to be able to read local filesystem
entries. (Although for a media player the impact is small I guess.)
It's weak and annoying as in that it does not prevent certain cases
which could be interpreted as bad in some cases, such as allowing
playlists on the local filesystem to reference remote URLs. It probably
barely makes sense, but we just want to exclude some other "definitely
not a good idea" things, all while playlists generally just work, so
whatever.
The policy is:
- from the command line anything is played
- local playlists can reference anything except "unsafe" streams
("unsafe" means special stream inputs like libavfilter graphs)
- remote playlists can reference only remote URLs
- things like "memory://" and archives are "transparent" to this
This commit does... something. It replaces the weird stream flags with a
slightly clearer "origin" value, which is now consequently passed down
and used everywhere. It fixes some deviations from the described policy.
I wanted to force archives to reference only content within them, but
this would probably have been more complicated (or required different
abstractions), and I'm too lazy to figure it out, so archives are now
"transparent" (playlists within archives behave the same outside).
There may be a lot of bugs in this.
This is unfortunately a very noisy commit because:
- every stream open call now needs to pass the origin
- so does every demuxer open call (=> params param. gets mandatory)
- most stream were changed to provide the "origin" value
- the origin value needed to be passed along in a lot of places
- I was too lazy to split the commit
Fixes: #7274
stream_skip() semantics were kind of bad, especially after the recent
change to the stream code. Forward stream_skip() calls could still
trigger a seek and fail, even if it was supposed to actually skip data.
(Maybe the idea that stream_skip() should try to seek is worthless in
the first place.)
Rename it to stream_seek_skip() (takes absolute position now because I
think that's better), and make it always skip if the stream is marked as
forward.
While we're at it, make EOF detection more robust. I guess s->eof
shouldn't exist at all, since it's valid only "sometimes". It should be
removed... but not today. A 1-byte stream_read_peek() call is good to
get the s->eof flag set to a correct value.
It was set, but its value was never used. The stream cache used to use
it, but it was removed. It controlled how much data it tried to read
from the underlying stream at once.
The user can now control the buffer size with --stream-buffer-size,
which achieves a similar effect, because the stream will in the common
case read half of the buffer size at once. In fact, the new default size
is 128KB, i.e. 64KB read size, which is as much as stream_file and
stream_cb requested by default. stream_memory requested more, but it
doesn't matter anyway. Only stream_smb set a larger size with 128KB.
stream_read_peek() duplicated what stream_read_more() checks for anyway
(whether the forward buffer is large enough). This can be skipped by
making the stream_read_more() return value more consistent.
(Only half of the buffer is actually used in a useful way, see manpage
or commit which added the option.)
Might have some advantages with broken network filesystem drivers.
See: #6802
Was probably worthless, and I can't measure a difference anymore (I used
to be able and it still seemed worth doing so back then).
When the default buffer size is enlarged in the next commit, the inline
buffer probably won't even be useful in theory, because the data will
rarely be on the same page as the other stream fields. It surely makes
the inline buffer seem like a ridiculous micro-optimization. Farewell...
In some corner cases (see #6802), it can be beneficial to use a larger
stream buffer size. Use this as argument to rewrite everything for no
reason.
Turn stream.c itself into a ring buffer, with configurable size. The
latter would have been easily achievable with minimal changes, and the
ring buffer is the hard part. There is no reason to have a ring buffer
at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and
some subtle issues with demux_mkv.c wanting to seek back by small
offsets (the latter was handled with small stream_peek() calls, which
are unneeded now).
In addition, this turns small forward seeks into reads (where data is
simply skipped). Before this commit, only stream_skip() did this (which
also mean that stream_skip() simply calls stream_seek() now).
Replace all stream_peek() calls with something else (usually
stream_read_peek()). The function was a problem, because it returned a
pointer to the internal buffer, which is now a ring buffer with
wrapping. The new function just copies the data into a buffer, and in
some cases requires callers to dynamically allocate memory. (The most
common case, demux_lavf.c, required a separate buffer allocation anyway
due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_*
changes.
I'm not happy with this. There still isn't a good reason why there
should be a ring buffer, that is complex, and most of the time just
wastes half of the available memory. Maybe another rewrite soon.
It also contains bugs; you're an alpha tester now.
Add yet another variant of the stream open function. This time, make it
so that it's possible to add new open parameters in an extendable way,
which should put an end to having to change this every other year.
Effectively get rid of the overly special stream_create_instance()
function and use the new one instead, which requires changes in
stream_concat.c and stream_memory.c. The function is still in private in
stream.c, but I preferred to make the mentioned users go through the new
function for orthogonality. The error handling (mostly logging) was
adjusted accordingly.
This should not have any functional changes. (To preempt any excuses, I
didn't actually test stream_concat and stream_memory.)
Instead of having to rely on the protocol matching, make a function that
creates a stream from a stream_info_t directly. Instead of going through
a weird indirection with STREAM_CTRL, add a direct argument for non-text
arguments to the open callback. Instead of creating a weird dummy
mpv_global, just pass an existing one from all callers. (The latter one
is just an artifact from the past, where mpv_global wasn't available
everywhere.)
Actually I just wanted a function that creates a stream without any of
that bullshit. This goal was slightly missed, since you still need this
heavy "constructor" just to setup a shitty struct with some shitty
callbacks.
struct stream used to include the stream buffer, including peek buffer,
inline in the struct. It could not be resized, which means the maximum
peek size was set in stone. This meant demux_lavf.c could peek only so
much data.
Change it to use a dynamic buffer. Because it's possible, keep the
inline buffer for default buffer sizes (which are basically always used
outside of file opening). It's unknown whether it really helps with
anything. Probably not.
This is also the fallback plan in case we need something like the old
stream cache in order to deal with mp4 + unseekable http: the code can
now be easily changed to use any buffer size.
The intent of the line above the modified one code was raising the
amount of read data, so that many stream_peek() calls with small len
values would not degrade performance by effectively turning every
stream_peak() into an unbuffered read call to the stream implementation.
So this confusing looking MPMAX() was correct, but "chunk" could still
get beyond the buffer.
So just fix that and limit "chunk" correctly.
I'm not sure whether the commit referenced below accidentally removed
some intricate guarantee that this couldn't happen, since the code was
around since 2013. It could have relied on TOTAL_BUFFER_SIZE >
STREAM_BUFFER_SIZE. But not sure. I've rewritten all this code in my own
branch a year ago, so who knows.
Fixes: 162e0f5ad9Fixes: #6948
The write functionality is almost unused (only encoding 2-pass mode uses
it to write the log file). Moreover, it almost makes no sense to use
this in a not local scenario. This change is just to prevent people from
duplicating the short write logic across all streams that happen to
support writing. Mostly untested; local log file writing still works.
Linux analog TV support (via tv://) was excessively complex, and
whenever I attempted to use it (cameras or loopback devices), it didn't
work well, or would have required some major work to update it. It's
very much stuck in the analog past (my favorite are the frequency tables
in frequencies.c for analog TV channels which don't exist anymore).
Especially cameras and such work fine with libavdevice and better than
tv://, for example:
mpv av://v4l2:/dev/video0
(adding --profile=low-latency --untimed even makes it mostly realtime)
Adding a new input layer that targets such "modern" uses would be
acceptable, if anyone is interested in it. The old TV code is just too
focused on actual analog TV.
DVB is rather obscure, but has an active maintainer, so don't remove it.
However, the demux/stream ctrl layer must go, so remove controls for
channel switching. Most of these could be reimplemented by using the
normal method for option runtime changes.
This was possibly needed by libdvdread, and/or old CD drivers on some
system. It still works with on-filesystem DVD and BD test images, so
this can go.
stream_dvd.c contained large amounts of ancient, unmaintained code,
which has been historically moved to libdvdnav. Basically, it's full of
low level parsing of DVD on-disc structures.
Kill it for good. Users can use the remaining dvdnav support (which
basically operates in non-menu mode). Users have reported that
libdvdread sometimes works better, but this is just libdvdnav's problem
and not ours.
Seems to happen often with ytdl pseudo-DASH streams, so whatever. I
couldn't reproduce it and check what triggers it, I just remember seeing
the error message and found it annoying.
The demuxer cache is the only cache now. Might need another change to
combat seeking failures in mp4 etc. The only bad thing is the loss of
cache-speed, which was sort of nice to have.
It seems a bit inappropriate to have dumped this into stream.c, even if
it's roughly speaking its main user. At least it made its way somewhat
unfortunately to other components not related to the stream or demuxer
layer at all.
I'm too greedy to give this weird helper its own file, so dump it into
thread_tools.c.
Probably a somewhat pointless change.
When this happens, network calls are forcibly aborted (more or less),
but demuxers might keep going, as most of them do not check for forced
exits properly. This can possibly lead to broken packets being added.
Also do not attempt to read more packets in this situation.
Also do not print a stream open failed message if opening was aborted
anyway.
Kacper Michajłow
Dudemanguy
sfan5
Christoph Heinrich
Thomas Weißschuh
Leo Izen
Mohammad AlSaleh
wm4