In acac614032, I sort of cargoculted what
I had to do for m3u, but it's actually not needed. bstr_split_tok
unexpectedly doesn't modify the original string. So the line_dup
business in the ini parsing is not needed. Remove it. The part in
parse_ref_init isn't wrong but naming the variable "line_dup" instead of
"value" is stupid so adjust that. And finally, you can actually force a
codepage in mpv (add "+" before the codepage) which will cause every
line to be allocated memory including the header lines even though those
are obviously valid utf8 that should never need conversion. This wasn't
taken into account so add an extra pl_free_line in a couple of places to
make sure they are freed.
It's 2023 and people don't use UTF-8 for their m3u, ini, etc. files.
Well mpv already has the tools in place to try and guess other
codepages, so we might as well use it I guess. This change is pretty
awkward since we have to read line-by-line but mp_iconv_to_utf8 may
sometimes allocate memory. So sometimes the bstr needs to be freed and
sometimes not for every line. Also we need to make another copy of the
line on the stack since splitting by tokens and such will mess up the
original line which may possibly be allocated memory. The ugliness is
mostly hidden in pl_free_line, but it's still weird. Fixes#10911.
64959c450d solved the problems with resuming playback, so default to
--directory-mode=lazy because it's faster, especially on slow drives,
and results in smaller playlists.
Discovered by @christoph-heinrich in IRC.
09c701b797 added a fallback for headerless
m3u files. However, it requires that the bstr len be greater than 10.
This means that a m3u playlist with a single entry with a very short
filename (such as "test.flac") will not be recognized as a playlist
since the amount of data is too small. The reason for this restriction
is unexplained and really shouldn't matter given that the important
thing mpv should be doing is checking if the data is text. Instead,
loosen the check so that it only needs to be 2 or greater. This covers a
single byte filename and a line terminator.
Directories were always loaded recursively, which can be slow
(e.g. one of the subdirectories is a mounting point to a slow device)
and can unexpectedly expand into a massive playlist.
Due to the problems described in 503dada42f,
this defaults to recursive loading.
ref. https://github.com/mpv-player/mpv/issues/9652
Last time it was extended was de3ecc60 from 8K to 512K two years ago.
The issue currently is that youtube EDL files can get very big.
Size of about 520K (one line), was observed, at the time of writing:
mpv https://youtube.com/watch?v=DBzFQgSMHdQ --ytdl-format=299
ytdl_hook.lua is unaffected by this because EDL lists don't go through
the file reader at demux_playlist.c (where each line was limited to
512K before this commit), however, EDL files on disk which are
loaded with --playlist=file.edl do.
Increase the limit to 2M so that such EDL files can also be loaded
from disk.
Fixes#9186
Although a linked list was ideal at first, there are cases where it
sucks, and became increasingly awkward (with the mpv command API
preferring integer indexes to access the list). In future, we probably
want to add more playlist-related functionality, so better change it to
an array now.
An array isn't always ideal either. Since playlist entries are still
separate objects (because in some cases you need a stable "iterator" to
it), but you still need to efficiently get the next/previous playlist
entry, there's a pl_index field, that needs to be maintained. E.g.
adding an entry at the start of the playlist => update the pl_index
field for all other entries. Well, it's not really worth to do something
more complicated to avoid these things.
This commit is probably buggy as shit. It's not like I bothered to test
everything. That's _your_ role.
mpv has a very weak and very annoying policy that determines whether a
playlist should be used or not. For example, if you play a remote
playlist, you usually don't want it to be able to read local filesystem
entries. (Although for a media player the impact is small I guess.)
It's weak and annoying as in that it does not prevent certain cases
which could be interpreted as bad in some cases, such as allowing
playlists on the local filesystem to reference remote URLs. It probably
barely makes sense, but we just want to exclude some other "definitely
not a good idea" things, all while playlists generally just work, so
whatever.
The policy is:
- from the command line anything is played
- local playlists can reference anything except "unsafe" streams
("unsafe" means special stream inputs like libavfilter graphs)
- remote playlists can reference only remote URLs
- things like "memory://" and archives are "transparent" to this
This commit does... something. It replaces the weird stream flags with a
slightly clearer "origin" value, which is now consequently passed down
and used everywhere. It fixes some deviations from the described policy.
I wanted to force archives to reference only content within them, but
this would probably have been more complicated (or required different
abstractions), and I'm too lazy to figure it out, so archives are now
"transparent" (playlists within archives behave the same outside).
There may be a lot of bugs in this.
This is unfortunately a very noisy commit because:
- every stream open call now needs to pass the origin
- so does every demuxer open call (=> params param. gets mandatory)
- most stream were changed to provide the "origin" value
- the origin value needed to be passed along in a lot of places
- I was too lazy to split the commit
Fixes: #7274
stream_skip() semantics were kind of bad, especially after the recent
change to the stream code. Forward stream_skip() calls could still
trigger a seek and fail, even if it was supposed to actually skip data.
(Maybe the idea that stream_skip() should try to seek is worthless in
the first place.)
Rename it to stream_seek_skip() (takes absolute position now because I
think that's better), and make it always skip if the stream is marked as
forward.
While we're at it, make EOF detection more robust. I guess s->eof
shouldn't exist at all, since it's valid only "sometimes". It should be
removed... but not today. A 1-byte stream_read_peek() call is good to
get the s->eof flag set to a correct value.
In some corner cases (see #6802), it can be beneficial to use a larger
stream buffer size. Use this as argument to rewrite everything for no
reason.
Turn stream.c itself into a ring buffer, with configurable size. The
latter would have been easily achievable with minimal changes, and the
ring buffer is the hard part. There is no reason to have a ring buffer
at all, except possibly if ffmpeg don't fix their awful mp4 demuxer, and
some subtle issues with demux_mkv.c wanting to seek back by small
offsets (the latter was handled with small stream_peek() calls, which
are unneeded now).
In addition, this turns small forward seeks into reads (where data is
simply skipped). Before this commit, only stream_skip() did this (which
also mean that stream_skip() simply calls stream_seek() now).
Replace all stream_peek() calls with something else (usually
stream_read_peek()). The function was a problem, because it returned a
pointer to the internal buffer, which is now a ring buffer with
wrapping. The new function just copies the data into a buffer, and in
some cases requires callers to dynamically allocate memory. (The most
common case, demux_lavf.c, required a separate buffer allocation anyway
due to FFmpeg "idiosyncrasies".) This is the bulk of the demuxer_*
changes.
I'm not happy with this. There still isn't a good reason why there
should be a ring buffer, that is complex, and most of the time just
wastes half of the available memory. Maybe another rewrite soon.
It also contains bugs; you're an alpha tester now.
Instead of having to rely on the protocol matching, make a function that
creates a stream from a stream_info_t directly. Instead of going through
a weird indirection with STREAM_CTRL, add a direct argument for non-text
arguments to the open callback. Instead of creating a weird dummy
mpv_global, just pass an existing one from all callers. (The latter one
is just an artifact from the past, where mpv_global wasn't available
everywhere.)
Actually I just wanted a function that creates a stream without any of
that bullshit. This goal was slightly missed, since you still need this
heavy "constructor" just to setup a shitty struct with some shitty
callbacks.
Raise it from 8KB to 512KB.
Do this because ytdl_hook.lua generated a 40KB EDL file (from 80KB
youtube-dl JSON output), and putting it into a .m3u file for easier
debugging failed due to the size limit.
Subtitles (and a few other file types, like playlists) are not streamed,
but fully read on opening. This means keeping the file handle or network
socket open is a waste of resources and could cause other weird
behavior. This is why there's a hack to close them after opening.
Change this hack to make the demuxer itself do this, which is less
weird. (Until recently, demuxer->stream ownership was more complex,
which is why it was done this way.)
There is some evil shit due to a huge ownership/lifetime mess of various
objects. Especially EDL (the currently only nested demuxer case)
requires being careful about mp_cancel and passing down stream pointers.
As one defensive programming measure, stop accessing the "stream"
variable in open_given_type(), even where it would still work. This
includes removing a redundant line of code, and removing the peak call,
which should not be needed anymore, as the remaining demuxers do this
mostly correctly.
It seems a bit inappropriate to have dumped this into stream.c, even if
it's roughly speaking its main user. At least it made its way somewhat
unfortunately to other components not related to the stream or demuxer
layer at all.
I'm too greedy to give this weird helper its own file, so dump it into
thread_tools.c.
Probably a somewhat pointless change.
Directory-opening never worked on Windows because MSVCRT's open()
doesn't open directories and its fstat() doesn't recognise directory
handles. These are just MSVCRT restrictions, and the Windows API itself
has no problem with opening directories as file objects, so reimplement
mpv's mp_open and mp_stat to use the Windows API directly. This should
fix directory playback.
This also populates the st_dev and st_ino fields of struct stat, so
filesystem loop checking in demux_playlist.c should now work on Windows.
Fixes#4711
Because it's kind of dumb. (But not sure if it was worth the trouble.)
For stream_file.c, we add new explicit fields. The rest are rather
special uses and can be killed by comparing the stream impl. name.
The changes to DVD/BD/CD/TV are entirely untested.
demux_playlist.c recognizes if the source stream points to a directory,
and adds its directory entries. Until now, only 1 level was added.
During playback, further directory entries could be resolved as
directory paths were "played".
While this worked fine, it lead to frequent user confusion, because
playlist resuming and other things didn't work as expected. So just
recursively scan everything.
I'm unsure whether it's a good fix, but at least it gets rid of the
complaints. (And probably will add others.)
This covers source files which were added in mplayer2 and mpv times
only, and where all code is covered by LGPL relicensing agreements.
There are probably more files to which this applies, but I'm being
conservative here.
A file named ao_sdl.c exists in MPlayer too, but the mpv one is a
complete rewrite, and was added some time after the original ao_sdl.c
was removed. The same applies to vo_sdl.c, for which the SDL2 API is
radically different in addition (MPlayer supports SDL 1.2 only).
common.c contains only code written by me. But common.h is a strange
case: although it originally was named mp_common.h and exists in MPlayer
too, by now it contains only definitions written by uau and me. The
exceptions are the CONTROL_ defines - thus not changing the license of
common.h yet.
codec_tags.c contained once large tables generated from MPlayer's
codecs.conf, but all of these tables were removed.
From demux_playlist.c I'm removing a code fragment from someone who was
not asked; this probably could be done later (see commit 15dccc37).
misc.c is a bit complicated to reason about (it was split off mplayer.c
and thus contains random functions out of this file), but actually all
functions have been added post-MPlayer. Except get_relative_time(),
which was written by uau, but looks similar to 3 different versions of
something similar in each of the Unix/win32/OSX timer source files. I'm
not sure what that means in regards to copyright, so I've just moved it
into another still-GPL source file for now.
screenshot.c once had some minor parts of MPlayer's vf_screenshot.c, but
they're all gone.
That just makes no sense, but seems to be a somewhat common user error.
The detection is not perfect. It's conceivable that EXT-X-... headers
are used in normal m3u playlists. After all, HLS playlists are by
definition a compatible extension to m3u playlists, as stupid as it
sounds.
The user probably doesn't want these. Conveniently, this also skips the
unwanted "." and ".." entries.
(This code is triggered if the input stream is a directory - and it's in
demux_playlist.c because it's convenient.)
That's how mime types are.
(This makes redirection with a specific HLS URL work, because some idiot
thought it'd be a great idea to spell the mime type as
"application/x-mpegURL".)
If a directory is encountered, replace it with its contents in the
internal playlist.
This is messed into demux_playlist.c, because why not. STREAMTYPE_DIR
could be avoided by unconditonally trying opendir() in demux_playlist.c,
but it seems nicer not to do weird things like calling it on real files.
This does not work on Windows, because msvcrt is retarded.
With a recent cleanup, rar support was stuffed into demux_playlist.c
(because "opening" rar files pretty much just lists archive contents and
adds them to a playlist using a special rar:// protocol, which will
actually access the rar file contents).
Since demux_playlist.c is probed _after_ demux_lavf.c (and should/must
be), libavformat was given the chance to detect DTS streams embedded
within the rar file. This is not really what we want, and a regression
what happened before rar listing was moved to demux_playlist.c.
Fix it by moving the rar listing into its own pseudo-demuxer, and let ir
probe before demux_lavf.c.
(Yes, this feature still has users.)
It was possible to make the player play local files by putting rar://
links into remote playlists, and some other potentially unsafe things.
Redo the handling of it. Now the rar-redirector (the thing in
demux_playlist.c) sets disable_safety, which makes the player open any
playlist entries returned. This is fine, because it redirects to the
same file anyway (just with different selection/interpretation of the
contents). On the other hand, rar:// itself is now considered fully
unsafe, which means that it is ignored if found in normal playlists.
Refactors an older hack, which for some reason used a more complicated
way. This generates the playlist representing the contents of the rar
file in demux_playlist.c. The pseudo-demuxer could easily be separate
from the the playlist parsers (and in fact there's almost no shared
code), but I don't think this obscure feature deserves a separate file.
Sample files created with:
rar a -v20000k -m0 files.rar file1.mkv file1.mkv
Includes some logic for not starting the demuxer thread for fully read
subtitles. (Well, the cache will still waste _lots_ of resources, and
the cache always has to be created, because we don't know whether it'll
be needed _before_ opening the file.)
See #1597.
m3u files are normally just text files with a list of filenames. Nothing
actually mandates that there is a header. Until now, we've rejected such
files, because there's absolutely no way to detect them.
If nothing else claims the file, the extension is ".m3u", and if the
contents of the file look like text, then load it as m3u playlist. The
text heuristic is pretty cheap, but at least it should prevent trying
to load binary data as playlist. (Which would "work", but result in a
catastrophic user experience.)
Due to the text heuristic, UTF-16/32 files will be rejected (unless they
have a header), but I don't care.
If EOF is reached after reading a line, the EOF flag is set. This was a
problem for the m3u code, which checked for EOF _after_ reading a line,
which will discard the last line read.
Also fix a typo in an unrelated part of the file.
I'm not sure if this could be done in libavformat instead. Probably not,
because libavformat doesn't seem to have any mechanism for trying one
protocol and reverting (or redirecting) to another one if needed.
This commit is sort of a hack too, because it redirects the URL by
pretending the http:// link is a playlist containing the mmsh:// link.
The list of mime types is borrowed from MPlayer (which has completely
different code to handle this).
While I'm not very fond of "const", it's important for declarations
(it decides whether a symbol is emitted in a read-only or read/write
section). Fix all these cases, so we have writeable global data only
when we really need.
Caused failure to detect .pls files, because they were misdetected as
m3u. The problem is that "forcing playlist files" and "forcing a
specific playlist format" are not really treated separate, and in both
cases p->force is set to true. This made m3u detect all files as m3u
if --playlist was used. So correctly check whether the file format is
actually being probed or not.
Because the http playlist URL I had for testing claimed to be m3u by
file extension and mime type, but didn't have the header.
Note that this actually changes behavior only in the case the format is
detected by mime type. Then p->force will be set before calling the
parser, and the header becomes optional.
As of this commit, stream_read_line() can't actually error (except in
the case the passed in buffer is 0, which never happens here). This
commit is preparation for the following commit, which checks harder
whether the read data is actually text. Before this commit, an error
was treated as end-of-file, but the data read so far was considered
valid.
Dudemanguy
Guido Cella
Christoph Heinrich
Thomas Weißschuh
Avi Halachmi (:avih)
wm4
Philip Sequeira
James Ross-Gowan