demux_lavf.c leaked the complete subtitle data if it was put through
iconv.
lavc_conv.c leaked AVCodecContext.subtitle_header (set by libavcodec),
which is fixed by using avcodec_free_context(). It also leaked the
subtitle that was decoded last.
UTF-16 subtitles are special in that they are usually read by
libavformat directly, even though they are not in UTF-8. This is
explicitly handled convert_charset() and skips conversion to UTF-8.
There was a bug due to not resetting the file position: if conversion
happens, the actual stream is replaced with a memory stream containing
the converted data, but if conversion is skipped, the original stream
with the wrong file position is kept.
Fix by always opening a memory stream. (We _could_ seek back, but there
is a slight possibility of additional failure due to unseekable
streams.)
Also, don't enter conversion if the subtitle is detected as UTF-8
either.
Fixes#2700.
This is mainly a refactor. I'm hoping it will make some things easier
in the future due to cleanly separating codec metadata and stream
metadata.
Also, declare that the "codec" field can not be NULL anymore. demux.c
will set it to "" if it's NULL when added. This gets rid of a corner
case everything had to handle, but which rarely happened.
There are a lot of incorrectly encoded subtitles with .ass extension
and non-ass subtitles (srt, ssa) with such extension, so we need to
try codepage detection even for .ass.
Signed-off-by: wm4 <wm4@nowhere>
Just so I can remove a few lines from dec_sub.c.
This is slightly inelegant, as the whole subtitle file has to be read
into memory, converted at once in memory, and then provided to
libavformat in an awkward way by creating a memory stream instead of
using demuxer->stream. It also won't be possible to force the charset on
subtitles in binary container formats - but this wasn't exposed before,
and we just hope this won't be ever needed. (One motivation was fixing
broken files with non-UTF8 muxed.) It also won't be possible to change
the charset on the fly, but this was not exposed either.
The demuxer infrastructure was originally single-threaded. To make it
suitable for multithreading (specifically, demuxing and decoding on
separate threads), some sort of tripple-buffering was introduced. There
are separate "struct demuxer" allocations. The demuxer thread sets the
state on d_thread. If anything changes, the state is copied to d_buffer
(the copy is protected by a lock), and the decoder thread is notified.
Then the decoder thread copies the state from d_buffer to d_user (again
while holding a lock). This avoids the need for locking in the
demuxer/decoder code itself (only demux.c needs an internal, "invisible"
lock.)
Remove the streams/num_streams fields from this tripple-buffering
schema. Move them to the internal struct, and protect them with the
internal lock. Use accessors for read access outside of demux.c.
Other than replacing all field accesses with accessors, this separates
allocating and adding sh_streams. This is needed to avoid race
conditions. Before this change, this was awkwardly handled by first
initializing the sh_stream, and then sending a stream change event. Now
the stream is allocated, then initialized, and then declared as
immutable and added (at which point it becomes visible to the decoder
thread immediately).
This change is useful for PR #2626. And eventually, we should probably
get entirely of the tripple buffering, and this makes a nice first step.
MPlayer traditionally always used the display aspect ratio, e.g. 16:9,
while FFmpeg uses the sample (aka pixel) aspect ratio.
Both have a bunch of advantages and disadvantages. Actually, it seems
using sample aspect ratio is generally nicer. The main reason for the
change is making mpv closer to how FFmpeg works in order to make life
easier. It's also nice that everything uses integer fractions instead
of floats now (except --video-aspect option/property).
Note that there is at least 1 user-visible change: vf_dsize now does
not set the display size, only the display aspect ratio. This is
because the image_params d_w/d_h fields did not just set the display
aspect, but also the size (except in encoding mode).
Slightly simpler, and removes the need to pre-read all subtitle packets.
This still does the subtitle charset conversion on the packet level
(instead converting when parsing the file), so in theory this still
could provide a way to change the charset at runtime. But maybe even
this should be removed, as FFmpeg is somewhat likely to get its own
charset detection and conversion mechanism in the future. (Would have
to keep the subtitle file in memory to allow changing the charset on
the fly, I guess.)
av_free_packet() got finally deprecated. Use av_packet_unref() instead,
which has almost the same semantics, has existed for a while, and is
available in all FFmpeg and Libav versions we support.
This AVPacket field was a hack against the fact that the duration field
was merely an int (too small for things like subtitle durations). Newer
libavcodec drops this field and makes duration 64 bit.
At least Matroska files have a "forced" flag (in addition to the
"default" flag). Export this flag. Treat it almost like the default
flag, but with slightly higher priority.
MPlayer traditionally had completely separate sh_ structs for
audio/video/subs, without a good way to share fields. This meant that
fields shared across all these headers had to be duplicated. This commit
deduplicates essentially the last remaining duplicated fields.
Vobsubs come as .idx/.sub pair of files. The .idx file is the one that
should be opened, but the name of the .sub file is unknown. We can now
make our own guess what the name of that file is. In particular, improve
support with URLs (as these can have the file extension in the middle of
the filename string if there are HTTP parameters).
Note that this works only with newer ffmpeg versions, because the
recently added sub_name demuxer option is used for this.
Remove the old implementation for these properties. It was never very
good, often returned very innaccurate values or just 0, and was static
even if the source was variable bitrate. Replace it with the
implementation of "packet-video-bitrate". Mark the "packet-..."
properties as deprecated. (The effective difference is different
formatting, and returning the raw value in bits instead of kilobits.)
Also extend the documentation a little.
It appears at least some decoders (sipr?) need the
AVCodecContext.bit_rate field set, so this one is still passed through.
We handle picking out font attachments by mime type ourselves in a
higher level, so we really just want to use the mimetype. Also, Matroska
is currently the only code in libavformat which uses the fonts at all,
and we can drop use of the codec IDs completely.
Trying to handle such video is almost worthless, but it was requested by
at least 2 users.
If there are no timestamps, enable byte seeking by setting
ts_resets_possible. Use the video FPS (wherever it comes from) and the
audio samplerate for timing. The latter was already done by making the
first packet emit DTS=0; remove this again and do it "properly" in a
higher level.
This reverts commit c8f49be919.
Not needed anymore; fixed in all supported FFmpeg releases. Though I
could not test again, because all sample files are gone (oops).
Use the (relatively new) libavformat image format probing functionality,
instead of letting demux_mf guess by file extension and MIME type.
The libavformat support is weird, though. Traditionally, it uses an
absolutely terrible hack to detect images by extension, _and_ (which is
the horrible part) will randomly interpret parts of the filename as
specifiers for matching by number. So something like '%03d' will be
interpreted as placeholder for a frame number. The worst part is that
such character sequences can be perfectly valid and common in http URLs.
This is known as "image2" demuxer. The newer support, which probes by
examining the file header, is split into several format-specific
demuxers with names ending in "_pipe". So we check for such a name
suffix. (At this point we're doing fine-grained hacking around ffmpeg
weirdness, so a clean solution is impossible anyway until upstream
changes.)
Some of the hacks were not applied if the file format was forced. Commit
37a0c914 moved them to a table, which is checked with normal probing
only.
Fixes#1612 (DVD forces mpeg, which in turn has to export native stream
IDs specifically).
Do some code restructuring on the way. For example, the probescore can
simply be set to the correct initial value, instead of checking whether
it was set at all.
Whatever the hell that is. FFmpeg tries to open any files with .bin file
extension with this demuxer (unless it finds a better demuxer), and then
reads the whole damn file, along with spamming dumb crap.
Includes some logic for not starting the demuxer thread for fully read
subtitles. (Well, the cache will still waste _lots_ of resources, and
the cache always has to be created, because we don't know whether it'll
be needed _before_ opening the file.)
See #1597.
An attempt to make format-specifics more declarative. (In my opinion,
all of this should be either provided by libavformat, or should not be
needed.)
I'm still leaving many checks with matches_avinputformat_name(), because
they're so specific.
Also useful for the following commit.
The HLs protocol consists of a "playlist" main file, which mpv downloads
and passes to the HLS demuxer. The HLS demuxer actually requests segment
files containing media data on its own. The packets read from the
demuxer have a source file position set, but it's not from the main
file. This leads to a strange effect: as a last fallback, the player
will calculate the approximate playback position from the file
position/size ratio, and since the main file is tiny, this will always
show 100%. Fix this by resetting the packet file position.
This doesn't affect the case when HLS actually reports a duration.
This removes the delay when switching audio tracks in mkv or mp4 files.
Other formats are not enabled, because it's not clear whether the
demuxers fulfill the requirements listed in demux.h. (Many formats
definitely do not with libavformat.)
Background:
The demuxer packet cache buffers a certain amount of packets. This
includes only packets from selected streams. We discard packets from
other streams for various reasons. This introduces a problem: switching
to a different audio track introduces a delay. The delay is as big as
the demuxer packet cache buffer, because while the file was read ahead
to fill the packet buffer, the process of reading packets also discarded
all packets from the previously not selected audio stream. Once the
remaining packet buffer has been played, new audio packets are available
and you hear audio again.
We could probably just not discard packets from unselected streams. But
this would require additional memory and CPU resources, and also it's
hard to tell when packets from unused streams should be discarded (we
don't want to keep them forever; it'd be a memory leak).
We could also issue a player hr-seek to the current playback position,
which would solve the problem in 1 line of code or so. But this can be
rather slow.
So what we do in this commit instead is: we just seek back to the
position where our current packet buffer starts, and start demuxing from
this position again. This way we can get the "past" packets for the
newly selected stream. For streams which were already selected the
packets are simply discarded until the previous position is reached
again.
That latter part is the hard part. We really want to skip packets
exactly until the position where we left off previously, or we will skip
packets or feed packets to the decoder twice. If we assume that the
demuxer is deterministic (returns exactly the same packets after a seek
to a previous position), then we can try to check whether it's the same
packet as the one at the end of the packet buffer. If it is, we know
that the packet after it is where we left off last time.
Unfortunately, this is not very robust, and maybe it can't be made
robust. Currently we use the demux_packet.pos field as unique packet
ID - which works fine in some scenarios, but will break in arbitrary
ways if the basic requirement to the demuxer (as listed in the demux.h
additions) are broken. Thus, this is enabled only for the internal mkv
demuxer and the libavformat mp4 demuxer.
(libavformat mkv does not work, because the packet positions are not
unique. Probably could be fixed upstream, but it's not clear whether
it's a bug or a feature.)
Repurpose demuxer->filetype for this. It used to be used to print a
human readable format description; change it to a symbolic format name
and export it as property.
Unfortunately, libavformat has its own weird conventions, which are
reflected through the new property, e.g. the .mp4 case mentioned in the
manpage.
Fixes#1504.
Instead of defining a separate data structure in the core.
For some odd reason, demux_chapter exported the chapter time in
nano-seconds. Change that to the usual timestamps (rename the field
to make any code relying on this to fail compilation), and also remove
the unused chapter end time.
Basically, this will mark the demuxer as seekable with rtmp* and mmsh
protocols. These protocols have network-level time seeking, and whether
you can seek on the byte level does not matter.
Until now, seeking was typically only enabled because of the cache, and
a (nonsensical) warning was shown accordingly.
It still could happen that the server doesn't actually support thse
requests (or simply rejects them), so this is somewhat imperfect.
Apparently using the stream index is the best way to refer to the same
streams across multiple FFmpeg-using programs, even if the stream index
itself is rarely meaningful in any way.
For Matroska, there are some possible problems, depending how FFmpeg
actually adds streams. Normally they seem to match though.
Normally, we pass libavformat demuxers a wrapped mpv stream. But in some
cases, such as HLS and RTSP, we let libavformat open the stream itself.
In these cases, set typical network properties like useragent according
to the mpv options.
(We still don't set it for the cases where libavformat opens other
streams on its own, e.g. when opening the companion .sub file for .idx
files - not sure if we maybe should always set these options.)
Fixes opening some streams.
This means the HLS playlist will be opened twice, but that's not much of
a problem, considering it's pretty small, and HLS will make many other
http accesses anyway.
This code meant to flush demuxer internal buffers by doing a byte seek
to the current position. In theory this shouldn't drop any stream data.
However, if the stream positions mismatch, then avio_seek() (called by
av_seek_frame()) stops being a no-op, and might for example read some
data to skip to the seek target. (This can happen if the distance is
less than SHORT_SEEK_THRESHOLD.)
The positions get out of sync because we drop data at one point (which
is what we _want_ to do). Strictly speaking, the AVIOContext flushing is
done incorrectly, becuase pb->pos points to the start of the buffer, not
the current position. So we have to increment pb->pos by the buffered
amount.
Since there are other weird reasons why the positions might go out of
sync (such as stream_dvd.c dropping buffers itself), and they don't
necessarily need to be in sync in the first place unless AVIOContext has
nothing buffered internally, just use the sledgehammer approach and
correct the position manually.
Also run av_seek_frame() after this. Currently, it shouldn't read
anything, but who knows how that might change with future libavformat
development.
This whole change didn't have any observable effect for me, but I'm
hoping it fixes a reported problem.
When flushing the AVIOContext, make sure it can't seek back to discarded
data. buf_ptr is just the current read position, while buf_end - buffer
is the actual buffer size. Since mpegts.c is littered with seek calls,
it might be that the ability to seek could read
Mark the stream (which the demuxer uses) as not seekable. The cache can
enable seeking again (this behavior is sometimes useful for other
things). I think this should have had no bad influence in theory, since
seeking BD/DVD first does the "real" seek, then flushes libavformat and
reads new packets.
HLS streams as demuxed by libavformat have no track title metadata. So
show the HLS bitrate if no title is set. Could be useless or annoying,
so it's a bit controversial, I guess.
--hls-bitrate=min/max lets you select the min or max bitrate. That's it.
Something more sophisticated might be possible, but is probably not even
worth the effort.
bstr.c doesn't really deserve its own directory, and compat had just
a few files, most of which may as well be in osdep. There isn't really
any justification for these extra directories, so get rid of them.
The compat/libav.h was empty - just delete it. We changed our approach
to API compatibility, and will likely not need it anymore.
This is a simplification, because it lets us use the AVPacket
functions, instead of handling the details manually.
It also allows the libavcodec rawvideo decoder to use reference
counting, so it doesn't have to memcpy() the full image data. The change
in av_common.c enables this.
This change is somewhat risky, because we rely on the following AVPacket
implementation details and assumptions:
- av_packet_ref() doesn't access the input padding, and just copies the
data. By the API, AVPacket is always padded, and we violate this. The
lavc implementation would have to go out of its way to make this a
real problem, though.
- We hope that the way we make the AVPacket refcountable in av_common.c
is actually supported API-usage. It's hard to tell whether it is.
Of course we still use our own "old" demux_packet struct, just so that
libav* API usage is somewhat isolated.
Use OPT_KEYVALUELIST() for all places where AVOptions are directly set
from mpv command line options. This allows escaping values, better
diagnostics (also no more "pal"), and somehow reduces code size.
Remove the old crappy option parser (av_opts.c).
This happens apparently randomly with rtmp:// and after seeks. This
eventually leads to audio decoding returning an EOF status, which
basically disables audio sync. This will lead to audio desync, even if
audio decoding later "recovers" when the demuxer actually returns audio
packets.
Hack-fix this by special-casing EAGAIN.
This didn't work, because the timebase was wrong. According to the
ffmpeg doxygen, if the stream index is -1 (which is what we used), the
timebase is AV_TIME_BASE. But this didn't work, and it really expected
the stream's timebase. Quite "surprising", since this feature
(avio_seek_time) is used by rtmp only.
Fixing this properly is too hard, so hack-fix our way around it.
STREAM_CTRL_SEEK_TO_TIME is also used by DVD/BD, so a new
STREAM_CTRL_AVSEEK is added. We simply pass-through the request
verbatim.
The old FFmpeg API and the new Libav API disagree about mp4 display
rotation direction. Well, whatever, fix it trial-and-error-style.
CC: @mpv-player/stable: add
This adds a thread to the demuxer which reads packets asynchronously.
It will do so until a configurable minimum packet queue size is
reached. (See options.rst additions.)
For now, the thread is disabled by default. There are some corner cases
that have to be fixed, such as fixing cache behavior with webradios.
Note that most interaction with the demuxer is still blocking, so if
e.g. network dies, the player will still freeze. But this change will
make it possible to remove most causes for freezing.
Most of the new code in demux.c actually consists of weird caches to
compensate for thread-safety issues (with the previously single-threaded
design), or to avoid blocking by having to wait on the demuxer thread.
Most of the changes in the player are due to the fact that we must not
access the source stream directly. the demuxer thread already accesses
it, and the stream stuff is not thread-safe.
For timeline stuff (like ordered chapters), we enable the thread for the
current segment only. We also clear its packet queue on seek, so that
the remaining (unconsumed) readahead buffer doesn't waste memory.
Keep in mind that insane subtitles (such as ASS typesetting muxed into
mkv files) will practically disable the readahead, because the total
queue size is considered when checking whether the minimum queue size
was reached.
For OGG audio files, we usually merge the per-stream metadata back to
the file-global metadata. Don't do that for OGM, because with OGM most
metadata is actually per-stream.
Suggested by tholin on github issue #882.
This is not entirely clean, but the fields we're accessing might be
considered internal to libavformat. On the other hand, existence of the
fields is guaranteed by the ABI, and nothing in the libavformat doxygen
suggestes they're not allowed to be accessed.
CC: @mpv-player/stable
DVD and Bluray (and to some extent cdda) require awful hacks all over
the codebase to make them work. The main reason is that they act like
container, but are entirely implemented on the stream layer. The raw
mpeg data resulting from these streams must be "extended" with the
container-like metadata transported via STREAM_CTRLs. The result were
hacks all over demux.c and some higher-level parts.
Add a "disc" pseudo-demuxer, and move all these hacks and special-cases
to it.
(Again.)
This time, we simply make it event-based, as it should be. This is done
for both demuxer metadata and stream metadata.
For some ogg-over-icy streams, 2 updates are reported on stream start.
This is because libavformat reports an update right on start, while
including the same info in the "static" metadata. I don't know if that's
a bug or a feature.
It's unlikely that files with multiple audio tracks and with replaygain
actually happen, but this change might help avoid minor corner cases
with later changes.
Recently, libavformat added demuxers to open image files like normal
demuxers. This is a good thing, but for now they interfere with the
operation of demux_mf. Add them to the blacklist until there is a proper
solution.
(The list doesn't contain _all_ recognized image formats, just those
that might interfere with demux_mf.)
CC: @mpv-player/stable
This returned a stream error value directly to libavformat, which can't
make sense. For example STREAM_ERROR (0) means success in libavformat
error codes. (The meaning of the libavformat read_seek return value is
underdocumented too.)
Also clarify the semantics.
It seems --idx didn't do anything. Possibly it used to change how the
now removed legacy demuxers like demux_avi used to behave. Or maybe
it was accidental.
--forceidx basically becomes --index=force. It's possible that new
index modes will be added in the future, so I'm keeping it
extensible, instead of e.g. creating --force-index.
While I'm not very fond of "const", it's important for declarations
(it decides whether a symbol is emitted in a read-only or read/write
section). Fix all these cases, so we have writeable global data only
when we really need.
The i_bps members of the sh_audio and dev_video structs are mostly used
for displaying the average audio and video bitrates. Keeping them in
bits-per-second avoids truncating them to bytes-per-second and changing
them back lateron.
Stop using it in most places, and prefer STREAM_CTRL_GET_SIZE. The
advantage is that always the correct size will be used. There can be no
doubt anymore whether the end_pos value is outdated (as it happens often
with files that are being downloaded).
Some streams still use end_pos. They don't change size, and it's easier
to emulate STREAM_CTRL_GET_SIZE using end_pos, instead of adding a
STREAM_CTRL_GET_SIZE implementation to these streams.
Make sure int64_t is always used for STREAM_CTRL_GET_SIZE (it was
uint64_t before).
Remove the seek flags mess, and replace them with a seekable flag. Every
stream must set it consistently now, and an assertion in stream.c checks
this. Don't distinguish between streams that can only be forward or
backwards seeked, since we have no such stream types.
stream.start_pos was needed for optical media only, and (apparently) not
for very good reasons. Just get rid of it.
For stream_dvd, we don't need to do anything. Byte seeking was already
removed from it earlier.
For stream_cdda and stream_vcd, emulate the start_pos by offsetting the
stream pos as seen by the rest of mpv.
The bits in discnav.c and loadfile.c were for dealing with the code
seeking back to the start in demux.c. Handle this differently by
assuming the demuxer is always initialized with the stream at start
position, and instead seek back if initializing the demuxer fails.
Remove the --sb option, which worked by modifying stream.start_pos. If
someone really wants this option, it could be added back by creating a
"slice" stream (actually ffmpeg already has such a thing).
mp3 has a hack lowering the probescore for format detection. This is
because detecting mp3s is hard due to their nature, and the fact that
ID3v2 tags are sometimes several megabytes big.
When playing mp3 from network, the mime-type is usually set, and that
matches the format hack entry meant for webradios, overriding the normal
mp3 entry. This can lead to network mp3s not being detected. Lower the
network case to the same probescore as on-disk mp3s. The difference is
that for network mp3s, we don't load the full probe-buffer, and we lower
the amount of audio the demuxer will read to collect data on opening
(0.5 seconds instead of typically 5 seconds).
