It appears using lseek() to seek to the end and back to determine file
size is inefficient in some cases.
With CIFS, this restores the performance regression that happened when
the stream cache was removed (which called read() from a thread). This
is probably faster than the old code too, because it's the seeking that
was slowing down CIFS.
According to the user who tested this, the size caching does not help
with fstat() (although it did with the old method).
Fixes: #7408, #7152
Some cache logic in demux.c queries the raw byte stream size on every
packet read. This is because it reports the value to the user. (It has
to be polled like this because there is no change notification in most
underlying I/O APIs, and also the user can't just block on the demuxer
thread to update it explicitly.)
This causes a very high number of get_size calls with low packet sizes,
so cache the size, and update it on every read. Reads only happen
approximately all 64KB read with default settings, which is way less
frequent than every packet in such extreme cases.
In theory, this could in theory cause problems in some cases. Actually
this is whole commit complete non-sense, because why micro-optimize for
broken cases like patent troll codecs. I don't need to justify it
anyway.
As a minor detail, off_t is actually specified as signed, so the off_t
cast is never needed.
mpv has a very weak and very annoying policy that determines whether a
playlist should be used or not. For example, if you play a remote
playlist, you usually don't want it to be able to read local filesystem
entries. (Although for a media player the impact is small I guess.)
It's weak and annoying as in that it does not prevent certain cases
which could be interpreted as bad in some cases, such as allowing
playlists on the local filesystem to reference remote URLs. It probably
barely makes sense, but we just want to exclude some other "definitely
not a good idea" things, all while playlists generally just work, so
whatever.
The policy is:
- from the command line anything is played
- local playlists can reference anything except "unsafe" streams
("unsafe" means special stream inputs like libavfilter graphs)
- remote playlists can reference only remote URLs
- things like "memory://" and archives are "transparent" to this
This commit does... something. It replaces the weird stream flags with a
slightly clearer "origin" value, which is now consequently passed down
and used everywhere. It fixes some deviations from the described policy.
I wanted to force archives to reference only content within them, but
this would probably have been more complicated (or required different
abstractions), and I'm too lazy to figure it out, so archives are now
"transparent" (playlists within archives behave the same outside).
There may be a lot of bugs in this.
This is unfortunately a very noisy commit because:
- every stream open call now needs to pass the origin
- so does every demuxer open call (=> params param. gets mandatory)
- most stream were changed to provide the "origin" value
- the origin value needed to be passed along in a lot of places
- I was too lazy to split the commit
Fixes: #7274
It was set, but its value was never used. The stream cache used to use
it, but it was removed. It controlled how much data it tried to read
from the underlying stream at once.
The user can now control the buffer size with --stream-buffer-size,
which achieves a similar effect, because the stream will in the common
case read half of the buffer size at once. In fact, the new default size
is 128KB, i.e. 64KB read size, which is as much as stream_file and
stream_cb requested by default. stream_memory requested more, but it
doesn't matter anyway. Only stream_smb set a larger size with 128KB.
The demuxer cache is the only cache now. Might need another change to
combat seeking failures in mp4 etc. The only bad thing is the loss of
cache-speed, which was sort of nice to have.
Functions `write` and `smbc_write` are given a diminishing buffer of
incorrect constant size. After partial writes, the code would do another
write of the full original length, failing to subtract the amount
already written.
There is some code that checks a FD for whether it is a regular file or
not. If it's not a regular file, it e.g. enables use of poll() to avoid
blocking forever.
But this was done only for FDs that were open()ed by us, not from stdin
special handling or fd://. Consequently, " | mpv -" could block the
player. Fix this by moving the code and running for it on all FDs.
Also, set p->regular_file even on mingw.
Alway give each demuxer its own mp_cancel instance. This makes
management of the mp_cancel things much easier. Also, instead of having
add/remove functions for mp_cancel slaves, replace them with a simpler
to use set_parent function. Remove cancel_and_free_demuxer(), which had
mpctx as parameter only to check an assumption. With this commit,
demuxers have their own mp_cancel, so add demux_cancel_and_free() which
makes use of it.
The intention is to avoid that the parent mp_cancel retains the
internally allocated wakeup pipe. File FDs are a relatively scarce
resource, so try to avoid having too many. This might matter for
subtitle files, for which it is relatively likely that they are loaded
in large quantities.
demux_lavf.c will close the underlying stream for most subtitle files,
and now it will free the wakeup pipe too. Actually, there are currently
only 1 or 2 mp_cancel objects per mpv core, but this could change if
every external subtitle track gets its own mp_cancel in later commits.
It seems a bit inappropriate to have dumped this into stream.c, even if
it's roughly speaking its main user. At least it made its way somewhat
unfortunately to other components not related to the stream or demuxer
layer at all.
I'm too greedy to give this weird helper its own file, so dump it into
thread_tools.c.
Probably a somewhat pointless change.
There is some code that checks a FD for whether it is a regular file or
not. If it's not a regular file, it e.g. enables use of poll() to avoid
blocking forever.
But this was done only for FDs that were open()ed by us, not from stdin
special handling or fd://. Consequently, " | mpv -" could block the
player. Fix this by moving the code and running for it on all FDs.
Also, set p->regular_file even on mingw.
Do this because retrying reading on higher levels (like the demuxer)
usually causes tons of problems. A hack like this is simpler and could
allow to remove some of the higher level retry behavior.
This works by trying to detect whether the file is appended. If we reach
EOF, check if the file size changed compared to the initial value. If it
did, it means the file was appended at least once, and we set the
p->appending flag. If that flag is set, we simply retry reading more
data every time we encounter EOF. The only way to do this is polling,
and we poll for at most 10 times, after waiting for 200ms every time.
This has a messy history all back to the initial commit with multiple
refactors, but it seems almost all authors agreed.
Exceptions:
2aa6acd9747: patch by someone who could not be reached. Whether or not
this code is still in mpv is unknown, but the affected code was moved to
stream.c at one point anyway.
3859bbd9fef: not sure if this is a patch by the mentioned person (I
assume not) or the committer (who agreed to LGPL), but it seems the
change is too trivial to affect copyright. It seems even the FD check
can be dropped, which I'm doing in this commit.
58846451f0e: author doesn't reply. But reverting this and letting
someone who has never seen this commit before redo it would lead to
exactly the same code. So I'm claiming that the change is not
copyrightable.
Because it's kind of dumb. (But not sure if it was worth the trouble.)
For stream_file.c, we add new explicit fields. The rest are rather
special uses and can be killed by comparing the stream impl. name.
The changes to DVD/BD/CD/TV are entirely untested.
POSIX leaves poll() behavior on directories unspecified. While on
Linux, it seems to behave the same way as regular files (always
return immediately), this is not guaranteed. At least with OSX
10.12, it seems to wait, which essentially means that opening
directories will "hang".
Fixes#3530 and #3649.
Windows definitely supports Unix-style fd inheritance. This mostly
worked when launched from mpv.exe, though mpv should change the file
mode to O_BINARY. When launched from mpv.com, the wrapper must pass the
list of handles (stored in the undocumented lpReserved2 and cbReserved2
fields) to the mpv process.
Revert "win32: more wchar_t -> WCHAR replacements"
Revert "win32: replace wchar_t with WCHAR"
Doing a "partial" port of this makes no sense anymore from my
perspective. Revert the changes, as they're confusing without
context, maintenance, and progress. These changes were a bit
premature anyway, and might actually cause other issues
(locale neutrality etc. as it was pointed out).
This was essentially missing from commit 0b52ac8a.
Since L"..." string literals have the type wchar_t[], we can't use them
for UTF-16 strings. Use C11 u"..." string literals instead. These have
the type char16_t[], but we simply assume char16_t is the same
underlying type as WCHAR. In practice, they're both unsigned short.
For this reason use -std=c11 on Windows. Since Windows is a "special"
environment (we require either MinGW or Cygwin), we don't need to worry
too much about compiler compatibility.
If a directory is encountered, replace it with its contents in the
internal playlist.
This is messed into demux_playlist.c, because why not. STREAMTYPE_DIR
could be avoided by unconditonally trying opendir() in demux_playlist.c,
but it seems nicer not to do weird things like calling it on real files.
This does not work on Windows, because msvcrt is retarded.
...because everything is terrible.
strerror() is not documented as having to be thread-safe by POSIX and
C11. (Which is pretty much bullshit, because both mandate threads and
some form of thread-local storage - so there's no excuse why
implementation couldn't implement this in a thread-safe way. Especially
with C11 this is ridiculous, because there is no way to use threads and
convert error numbers to strings at the same time!)
Since we heavily use threads now, we should avoid unsafe functions like
strerror().
strerror_r() is in POSIX, but GNU/glibc deliberately fucks it up and
gives the function different semantics than the POSIX one. It's a bit of
work to convince this piece of shit to expose the POSIX standard
function, and not the messed up GNU one.
strerror_l() is also in POSIX, but only since the 2008 standard, and
thus is not widespread.
The solution is using avlibc (libavutil, by its official name), which
handles the unportable details for us, mostly. We avoid some pain.
stream_dvd.c includes a pseudo-protocol that recognizes .IFO files, and
plays them using libdvdread. This was relatively lazy, and could perhaps
easily trigger with files that just had the .ifo extension.
Make the checks stricter, and even probe the file header. Apparently the
first bytes in an .ifo file are always "DVDVIDEO-VTS", so check for
this.
Refuse to load the main "video_ts.ifo". The plan is to use stream_dvdnav
for it.
This also removes at least 1 memory leak.
Until now, you had to use --load-unsafe-playlists or --playlist to get
playlists loaded. Change this and always load playlists by default.
This still attempts to reject unsafe URLs. For example, trying to invoke
libavdevice pseudo-demuxer is explicitly prevented. Local paths and any
http links (and some more) are always allowed.
While I'm not very fond of "const", it's important for declarations
(it decides whether a symbol is emitted in a read-only or read/write
section). Fix all these cases, so we have writeable global data only
when we really need.
Also sneak in some cosmetics.
setmode() exists on Windows/msvcrt only, so there's no need for a
config test.
I couldn't reproduce the problem with seekable pipes on wine, so axe
it. (I'm aware that it still could be an issue on real Windows.)
For some reason, we support writeable streams. (Only encoding uses that,
and the use of it looks messy enough that I want to replace it with FILE
or avio today.)
It's a chaos: most streams do not actually check the mode parameter like
they should. Simplify it, and let streams signal availability of write
mode by setting a flag in the stream info struct.
Stop using it in most places, and prefer STREAM_CTRL_GET_SIZE. The
advantage is that always the correct size will be used. There can be no
doubt anymore whether the end_pos value is outdated (as it happens often
with files that are being downloaded).
Some streams still use end_pos. They don't change size, and it's easier
to emulate STREAM_CTRL_GET_SIZE using end_pos, instead of adding a
STREAM_CTRL_GET_SIZE implementation to these streams.
Make sure int64_t is always used for STREAM_CTRL_GET_SIZE (it was
uint64_t before).
Remove the seek flags mess, and replace them with a seekable flag. Every
stream must set it consistently now, and an assertion in stream.c checks
this. Don't distinguish between streams that can only be forward or
backwards seeked, since we have no such stream types.
Use NtQueryVolumeInformationFile instead of GetDriveType for detecting
remote filesystems on Windows. This has the advantage of working
directly on the file handle instead of needing a path and it works
unmodified in Cygwin where the previous code wouldn't understand Cygwin
paths or symlinks.
There is some risk in using NtQueryVolumeInformationFile, since it's an
internal function and its behaviour could change at any time or it could
be removed in a future version of Windows, however it's documented[1] in
the WDK and it's used successfully by Cygwin, so it should be fine. If
it's removed, the code should fail gracefully by treating all files as
local.
[1]: http://msdn.microsoft.com/en-us/library/windows/hardware/ff567070.aspx
Signed-off-by: wm4 <wm4@nowhere>
Detected 'protocols' are AFP, nfs, smb and webdav. This can be extended on
request.
This is currently only implemented for BSD systems (using fstatfs). This
addresses issue #558 on the above platforms.
This fixes two things:
1. Dropping files on the VO window will auto-load subtitles (since most
drag & drop code prefixes the filenames with 'file://', and the
subtitle auto-load code considers 'file://' non-local)
2. Fix behavior of the %x screenshot filename template (similar problem)
One could force all that code to special-case 'file://' URLs, but just
replacing the filename on playback start is simpler.
wm4
Aman Gupta
Anton Kindestam
Yclept Nemo
gall0ws
Philip Sequeira
sfan5
James Ross-Gowan
Ben Boeckel
Marcin Kurczewski
Stefano Pigozzi