Regression caused by commit 3b95dd47. Also see commit 4c25b000. We can
either use video_next_pts and add "delay", or we just use video_pts. Any
other combination breaks. The reason why the assumption that delay==0 at
this point was wrong exactly because after displaying the first video
frame (usually done before audio resync) a new frame might be "added"
immediately, resulting in a new video_next_pts and "delay", which will
still amount to video_pts.
Fixes#2770. (The reason why display-sync was blamed in this issue is
because enabling display-sync in the options forces a prefetch by 2
instead of 1 frames for seeks/playback restart, which triggers the
issue, even if display-sync is not actually enabled. In this case,
display-sync is never enabled because the frames have a unusually high
frame duration. This is also what exposed the initial desync issue.)
With the format left untouched, this would just try to reinit with a
spdif format again.
We're not clearing the format in reset_audio_state() so the audio chain
can be recreated any time without having to wait for a frame to be
decoded.
It doesn't need to be part of the big context, but is strictly part of
shuffling data from the audio filters to audio output, and thus belongs
into ao_chain.
It also turns out that clearing it in clear_audio_output_buffers() is
completely redundant.
(Of course ao_buffer is an abomination in the first place and shouldn't
exist at all.)
Similar to the video path. dec_audio.c now handles decoding only. It
also looks very similar to dec_video.c, and actually contains some of
the rewritten code from it. (A further goal might be unifying the
decoders, I guess.)
High potential for regressions.
Seems useless.
This only helped in one case: one audio stream in the sample
av_find_best_stream_fails.ts had a AC3 packets which couldn't be
decoded, and for which avcodec_decode_audio4() returned 0 forever. In
this specific case, playback will now not start, and you have to
deselect audio manually.
(If someone complains, the old behavior might be restored, but
differently.)
Also remove the stale "bitrate" field.
Eventually we want the VO be driven by a A->V filter, so a decoder
doesn't even have to exist. Some features definitely require a decoder
though (like reporting the decoder in use, hardware decoding, etc.), so
for each thing which accessed d_video, it has to be redecided if and how
it can access decoder state.
At least the "framedrop" property slightly changes semantics: you can
now always set this property, even if no video is active.
Some untested changes in this commit, but our bio-based distributed
test suite has to take care of this.
This is mainly a refactor. I'm hoping it will make some things easier
in the future due to cleanly separating codec metadata and stream
metadata.
Also, declare that the "codec" field can not be NULL anymore. demux.c
will set it to "" if it's NULL when added. This gets rid of a corner
case everything had to handle, but which rarely happened.
This is another attempt at making files with sparse video frames work
better.
The problem is that you generally can't know whether a jump in video
timestamps is just a (very) long video frame, or a timestamp reset. Due
to the existence of files with sparse video frames (new frame only every
few seconds or longer), every heuristic will be arbitrary (in general,
at least).
But we can use the fact that if video is continuous, audio should also
be continuous. Audio discontinuities can be easily detected, and if that
happens, reset some of the playback state.
The way the playback state is reset is rather radical (resets decoders
as well), but it's just better not to cause too much obscure stuff to
happen here. If the A/V sync code were to be rewritten, it should
probably strictly use PTS values (not this strange time_frame/delay
stuff), which would make it much easier to detect such situations and
to react to them.
Use the demux_set_ts_offset() added in the previous commit to base each
timeline segment to use timestamps according to its relative position
within the overall timeline. As a consequence we don't need to care
about these timestamps anymore, and everything becomes simpler.
(Another minor but delicious nugget of sanity.)
When the audio format is not known yet and the audio chain is still
initializing, filter reinit will fail. Normally, attempts to
reinitialize filters at this stage should be rare (e.g. user commands
editing the filter chain). But it sometimes happened with track
switching in combination with the video code calling
update_playback_speed() at arbitrary times.
Get rid of the message by not trying to change the filters for the sake
of playback speed update while decoding is still being initialized.
Actually, it didn't really require that before (most work was avoided),
but some bits had to be run anyway. Separate the speed change into a
light-weight function, which merely updates already created filters, and
a heavy-weight one which messes with filter insertion.
This also happens to fix the case where the filters would "forget" the
current speed (force resampling, change speed, hit a volume control to
force af_volume insertion - it will reset speed and desync).
Since we now always run the light-weight function, remove the
af_scaletempo verbose message that is printed on speed setting. Other
than that, all setters are cheap.
We still have a sample-based buffer between filters and audio outputs.
In order to avoid cutting frames into half (which can upset receivers),
we strictly need to align the boundaries on which we cut the audio.
Discontinuities (like toggling fullscreen) can cause multiple frames to
be dropped in succession, which sounds very weird. It's better to drop
some video frames instead to compensate for larger desyncs.
We roughly base it on the maximum allowed speed changes (audio change is
"additional" to the video change to account for deviations when playing
at max. video speed change).
It's not needed, because the additional data is not appended, but is the
total size of the audio buffer. The maximum size is the static audio
drop size (or twice, if the audio is duplicated).
The previous commit handled not falling back to normal decoding if the
AO was reloaded (I think...), and this tries to re-engage spdif pass-
through if it was previously falling back to normal decoding (e.g.
because it temporarily switched to an audio device incapable of
passthrough).
The manpage entry explains this.
(Maybe this option could be always enabled and removed. I don't quite
remember what valid use-cases there are for just disabling audio
entirely, other than that this is also needed for audio decoder init
failure.)
This should avoid unnecessary sleeping when audio playback start resync
has finished and goes into the normal playback state.
This is tricky; see e.g. commit 402fe381.
For video sync, we want separate playback speed controls for user-
requested speed and the "correction" speed for video timing. Further, we
use this separation to make sure only a resampler is inserted if
playback speed is only changed for video sync correction.
As of this commit, this is basically inactive code. It's just
preparation for the video sync code (the following commit).
Commit c5818046 fixed one case of audio EOF handling, and caused a new
one. This time, the ao_buffer doesn't actually contain everyting that
should be played - because if --end is used, only a part of it is
played. Of course this is stupid, and it will be changed later. For now,
this smaller change fixes the bug.
Fixes#2189.
time_frame is when the next video frame should be shown. It's normally
overwritten by the video timing code. This also says something about
"nosound mode" (--no-audio today), but at least these days we don't use
it at all if video is disabled.
Remove it; it likely has no function at all.
In paused mode, we never entered the audio EOF state. This shows e.g. in
--keep-open mode, which will not set the eof-reached property correctly.
Regression since commit c06cd1b9. This commit was the wrong fix. We need
to respect the buffer state, and pausing has nothing to do with this.
Fixes#2167.
Replace all the check macros with function calls. Give them all the
same case and naming schema.
Drop af_fmt2bits(). Only af_fmt2bps() survives as af_fmt_to_bytes().
Introduce af_fmt_is_pcm(), and use it in situations that used
!AF_FORMAT_IS_SPECIAL. Nobody really knew what a "special" format
was. It simply meant "not PCM".
This provides a new method for enabling spdif passthrough. The old
method via --ad (--ad=spdif:ac3 etc.) is deprecated. The deprecated
method will probably stop working at some point.
This also supports PCM fallback. One caveat is that it will lose at
least 1 audio packet in doing so. (I don't care enough to prevent this.)
(This is named after the old S/PDIF connector, because it uses the same
underlying technology as far as the higher level protoco is concerned.
Also, the user should be renamed that passthrough is backwards.)
This makes no sense, because the format can't be converted anyway. It
just sets up the filter chain init code, which will vomit a bunch of
useless and confusing messages. So uninit and fail explicitly when this
happens.
When starting in paused mode, no audio is written to the device at all,
because writing audio implicitly unpauses the AO. If the file is very
small, and all audio fits within the AO buffer, this accidentally
triggered the EOF condition. (In unpaused mode, it would write all
audio, end playback, and then wait until the AO has everything played.)
Commit 10915000 attempted to fix wasting CPU when resyncing and no new
data was actually coming from the demuxer. The fix assumed that at this
point it would have reached the sync point, but since the code attempts
weird incremental decoding, this wasn't actually true. So it broke
seeking in addition to removing the CPU waste.
Try something else. This time, we essentially only wakeup again if
data was read (i.e. audio_decode() returned successfully).
Thsi code path happens during seeking. If video is still being decoded
to get to the first video frame, audio has nothing to do, as it is
synchronized against the first video frame. We only want to wake up if
there's an actual state change.
Fixes#1958.
The af_add() function has a problem: if the inserted filter returns
AF_DETACH during init, the function will have a dangling pointer. Until
now this was avoided by making sure none of the used filters actually
return AF_DETACH, but it's getting infeasible.
Solve this by requiring passing an unique label to af_add(), which is
then used instead of the pointer.
Only reinit filters if it's actually needed. This is also slightly
easier to understand: if you look at the code, it should now be more
obvious why a reinit is needed (hopefully).
Precise seeking requires skipping audio, since the demuxer usually
doesn't seek precisely enough. There is a sanity check that prevents
skipping more than 300 seconds of audio. This still fails with very
large mp3s. For example, with a 1GB sized mp3 with Xing headers, entries
will be 4 MB apart on average, and occasionally much more.
Just bump the limit. I'm not even sure why it was added in the first
place; I suppose it's most important for files with real PTS resets.
When playback is started after seeking or opening a file, we need to
make sure audio and video line up exactly. This is done by cutting or
padding the audio stream to start on the video PTS.
This does not quite work with spdif: audio is compressed data, within a
spdif frame. There is no way to cut the audio "in between" the frames.
Cutting between the frames would just produce broken spdif packets, and
who knows how receivers will react to this (play noise?). But we still
can cut it in frame boundaries.
Unfortunately, we also insert 0 data for "silence" - we probably
shouldn't do this. Chances are the receiver will switch to PCM or so.
But for now this will have to do.
Note that this could be simplified somewhat, as soon as we work with
frames. See previous commit.
Handle the failure gracefully, instead of exploding and disabling audio.
Just set the speed back to 1.0.
Also remove the AF_DETACH from af_scaletempo. This actually created a
dangling pointer in af_add(), a tricky consequence of af_add()
reconfiguring the filter chain and the newly added filter using
AF_DETACH. Fortunately the AF_DETACH is not needed (and probably never
worked - it comes from MPlayer times, and MPlayer also disables audio
when trying to change speed with spdif).
Always use af_scaletempo if it's inserted, even if the option
--audio-pitch-correction=no is set.
Make sure all filters are reset on speed change. It's conceivable that
dynamic changes to the filter chain at runtime leave filters around
without resetting their speed parameters.
Also move the code to a separate function.
If the audio decoder was created, but no audio filter chain created yet
(still trying to decode a first audio frame), setting the "speed"
property could explode. It tried to recreate the filter chain, even
though no format was set yet.
This is inconvenient and should not happen.
Although the libraries we use for resampling (libavresample and
libswresample) do not support changing sampelrate on the fly, this makes
it easier to make sure no audio buffers are implicitly dropped. In fact,
this commit adds additional code to drain the resampler explicitly.
Changing speed twice without feeding audio in-between made it crash
with libavresample inc ertain cases (libswresample is fine). This is
probably a libavresample bug. Hopefully this will be fixed, and also I
attempted to workaround the situation that crashes it. (It seems to
point in direction of random memory corruption, though.)
In my opinion the artifacts created by af_scaletempo on extreme slowdown
(50% or so) are too bothersome - but users disagree. So use
af_scaletempo on any speed changes, not just on speedup.
This avoids potentially dropping some small amount of audio data
buffered in filters.
Reinit can be skipped only if the filter is af_scaletempo (which maps to
AF_CONTROL_SET_PLAYBACK_SPEED). The other case using af_lavrresample is
much more complicated due to filter chain politics.
Also, changing speed between 1.0 and something higher typically inserts
or removes the filter, so this obviously requires reinitialization. It
can be prevented by forcing the filter with --af=scaletempo.
I guess this was supposed to be some sort of optimization, but even
though it probably works, it's pretty meaningless and I couldn't measure
a difference. One special case killed.
mpctx->audio_delay always has the same value as opts->audio_delay. (This
was not the case a long time ago, when the audio-delay property didn't
actually write to opts->audio_delay. I think.)
Some files can have audio after video has ended, and playback of the
audio-only remainder is supposed to work just fine.
Seeking is broken-ish though. Not much can be done about this, since
it's the way demuxers work. Also, such files are obscure corner cases.
But enabling hr-seek for audio after video end can improve the situation
a lot.
This helps with issue #1533. The reported also provided a command line
to produce such a file:
ffmpeg -i image.jpg -i audio.flac -threads $(nproc) \
-c:v libvpx -crf 10 -qmin 5 -qmax 55 \
-vf scale=360:-1 -sws_flags lanczos -c:a libvorbis -ac 2 \
-b:a 128K out.webm
This was forgotten when the option was implemented, and makes this
option work as advertised.
Fixes#1473 (though the default behavior is probably still stupid).
This is a somewhat obscure situation, and happens only if audio starts
again after it has ended (in particular can happens with files where
audio starts later). It doesn't matter much whether audio starts
immediately or some milliseconds later, so simplify it.
When playing paused, the amount of decoded audio is limited to a small
amount (1 sample), because we don't write any audio to the AO when
paused. The small amount could trigger the case of the wanted audio
being too far in the future in the PTS sync code, which set the audio
status to STATUS_DRAINING, which in turn triggered the EOF code in the
next iteration. This was ok, but unfortunately, this triggered another
retry in order to check resuming from EOF by setting the status to
STATUS_SYNCING, which in turn lead to the busy loop by alternating
between the 2 states. So don't try resyncing while paused.
Since the PTS syncing code also calls ao_reset(), this could cause the
pulseaudio daemon to consume some CPU time as well.
This was caused by commit 33b57f55. Before that, the playloop was merely
run more often, but didn't cause any problems.
Fixes#1288.
We absolutely need to clear the AO reference in the mixer.
The audio_status must be changed to a state where no code assumes that
the AO is available. (It's allowed to do this blindly.)
This rewrites the audio decode loop to some degree. Audio filters don't
do refcounted frames yet, so af.c contains a hacky "emulation".
Remove some of the weird heuristic-heavy code in dec_audio.c. Instead of
estimating how much audio we need to filter, we always filter full
frames. Maybe this should be adjusted later: in case filtering increases
the volume of the audio data, we should try not to buffer too much
filter output by reducing the input that is fed at once.
For ad_spdif.c and ad_mpg123.c, we don't avoid extra copying yet - it
doesn't seem worth the trouble.
Use a pseudo-filter when changing speed with resampling, instead of
somehow changing a samplerate somewhere. This uses the same underlying
mechanism, but is a bit more structured and cleaner. It also makes some
of the following changes easier.
Since we now always use filters to change audio speed, move most of the
work set_playback_speed() does to recreate_audio_filters().
This is what you would expect. Before this commit, each
ao_request_reload() call would just queue a reload command, and then
recreate the AO for the number of times the function was called.
Instead of sending a command, introduce some sort of event retrieval
mechanism. At least for the reload case, use atomics, because we're too
lazy to setup an extra mutex.
This commit fixes a "cosmetic" user interface issue. Instead of
displaying the interpolated seek time on OSD, show the actual audio
time.
This is rather silly: when seeking in audio-only mode, it takes some
iterations until audio is "ready", but on the other hand, the audio
state machine is rather fickle, and fixing this cosmetic issue would be
intrusive. So just add a hack that paints over the ugly behavior as
perceived by the user. Probably the lesser evil.
It doesn't happen if video is enabled, because that mode sets the
current time immediately to video PTS. (Audio has to be synced to video,
so the code is a bit more complex.)
Fixes#1233.
The player was supposed to exit playback if both video and audio failed
to initialize (or if one of the streams was not selected when the other
stream failed). This didn't work; for one this check was missing from
one of the failure paths. And more importantly, both checked the
current_track array incorrectly.
Fix these issues, and move the failure handling code into a common
function.
CC: @mpv-player/stable
It possibly goes to sleep without actually starting to decode audio.
Possibly fixes a problem with --no-osc --no-video reported on IRC.
CC: @mpv-player/stable
Seems logical. For some reason, the player allows deselecting both audio
and video stream without quitting (a deliberate feature of which I have
no idea why it was added years ago), so this is needed.
Each subsystem (or similar thing) had an INITIALIZED_ flag assigned. The
main use of this was that you could pass a bitmask of these flags to
uninit_player(). Except in some situations where you wanted to
uninitialize nearly everything, this wasn't really useful. Moreover, it
was quite annoying that subsystems had most of the code in a specific
file, but the uninit code in loadfile.c (because that's where
uninit_player() was implemented).
Simplify all this. Remove the flags; e.g. instead of testing for the
INITIALIZED_AO flag, test whether mpctx->ao is set. Move uninit code
to separate functions, e.g. uninit_audio_out().
The messages "Audio: no audio" and "Video: no video" could be printed
twice each if initializing them failed. Prevent his silliness.
CC: @mpv-player/stable
Apparently this is what users want. When playing with normal speed,
nothing is done. When playing slower than normal, resampling is used
instead, because scaletempo (which does the pitch correction) adds
too many artifacts.
There's no real reason why audio_init_filter() should exist. Just use
af_init or af_reinit directly. (We lose a useless message; the same
information is printed in a quite close place with more details.)
Requires less code, and the way the filter chain is marked as having
failed to initialize allows just switching off audio instead of
crashing if trying to insert a volume filter in mixer.c fails, and
recreating the old filter chain fails too.
This would play some silence in case video was slower than audio. If
framedropping is already enabled, there's no other way to keep A/V
sync, short of changing audio playback speed (which would give worse
results). The --audiodrop option inserted silence if there was more
than 500ms desync.
This worked somewhat, but I think it was a silly idea after all. Whether
the playback experience is really bad or slightly worse doesn't really
matter. There also was a subtle bug with PTS handling, that apparently
caused A/V desync anyway at ridiculous playback speeds.
Just remove this feature; nobody is going to use it anyway.
Before this commit, there was AF_FORMAT_AC3 (the original spdif format,
used for AC3 and DTS core), and AF_FORMAT_IEC61937 (used for AC3, DTS
and DTS-HD), which was handled as some sort of superset for
AF_FORMAT_AC3. There also was AF_FORMAT_MPEG2, which used
IEC61937-framing, but still was handled as something "separate".
Technically, all of them are pretty similar, but may use different
bitrates. Since digital passthrough pretends to be PCM (just with
special headers that wrap digital packets), this is easily detectable by
the higher samplerate or higher number of channels, so I don't know why
you'd need a separate "class" of sample formats (AF_FORMAT_AC3 vs.
AF_FORMAT_IEC61937) to distinguish them. Actually, this whole thing is
just a mess.
Simplify this by handling all these formats the same way.
AF_FORMAT_IS_IEC61937() now returns 1 for all spdif formats (even MP3).
All AOs just accept all spdif formats now - whether that works or not is
not really clear (seems inconsistent due to earlier attempts to make
DTS-HD work). But on the other hand, enabling spdif requires manual user
interaction, so it doesn't matter much if initialization fails in
slightly less graceful ways if it can't work at all.
At a later point, we will support passthrough with ao_pulse. It seems
the PulseAudio API wants to know the codec type (or maybe not - feeding
it DTS while telling it it's AC3 works), add separate formats for each
codecs. While this reminds of the earlier chaos, it's stricter, and most
code just uses AF_FORMAT_IS_IEC61937().
Also, modify AF_FORMAT_TYPE_MASK (renamed from AF_FORMAT_POINT_MASK) to
include special formats, so that it always describes the fundamental
sample format type. This also ensures valid AF formats are never 0 (this
was probably broken in one of the earlier commits from today).
With e.g --start=-3 --audio-buffer=10 the decoder entered EOF state
before the initial sync was finished, entered STATUS_EOF, and just
started playing audio from a random position.
This doesn't handle seeking outside of the file, which is a different
case. E.g. --start=30:00 with audio and video enabled in a file shorter
than 30:00 will play a random last part of audio. This could perhaps be
fixed by using the hr-seek target for cutting audio, instead of the
video PTS, but that would be kind of intrusive, so don't do it for now.
The simpler solution, assuming audio EOF on video EOF, wouldn't work,
because we allow audio to start before video, or to last after video.
Somehow, there was a larger misunderstanding in the code: ao_buffer
does not need to be preserved over audio reinit for proper support of
gapless audio. The actual AO internal buffer takes care of this.
In fact, preserving ao_buffer just breaks audio resync. In the ordered
chapter case, end_pts is used, which means not all audio data in the
buffer is played, thus some data is left over when audio decoding
resumes on the next segment. This triggers some code that aborts resync
if there's "audio decoded" (ao_buffer contains something), but no PTS
is known (nothing was actually decoded yet).
Simplify, and always bind the output buffer to the decoder.
CC: @mpv-player/stable (maybe)
Probably no observable effect, but it's more correct. Setting audio to
EOF could have bad effects otherwise (anywhere the player logic for
example decides whether EOF was reached, and such).
Don't attempt to resync after speed changes. Note that most other cases
of audio reinit (like switching tracks etc.) still resync, but other
code paths take care of setting the audio_status accordingly.
This restores the old behavior of not trying to fix audio desync, which
was probably changed with commit 261506e3.
Note that the code as of now wasn't even entirely correct, since the A/V
sync values are slightly shifted. The dsync depends on the audio buffer
size, so a larger buffer size will show more extreme desync. Also see
mplayer2 commit 213a224e, which should fixed this - it was not merged
into mpv, because it disabled audio for too long, resulting in a worse
user experience. This is similar to the issue this commit attempts to
fix.
Fixes: #1042 (probably)
CC: @mpv-player-stable
This shouldn't change anything functionally.
Change the A/V desync message. --framedrop is enabled by default now, so
the text must be changed a little. I've never heard of audio outputs
messing up A/V sync recently, so remove that part.
Remove the unused ao_pts field.
Reorder 2 A/V sync related expressions so that they look the same.
In theory, timestamps can be negative, so we shouldn't just return -1
as special value.
Remove the separate code for clearing decode buffers; use the same code
that is used for normal seek reset.
Commit 5afc025c broke this. The reason is that mpctx->delay is updated
when a new video frame is added. This value is also needed to resync
audio, but it will be for the wrong PTS. They must be consistent with
each other, and if they aren't, initial sync will be off by N video
frames, which results at least in worse user experience.
This can be reproduced by for example heavily switching between normal
and 2x speed, or similar.
Fix by readding the video_next_pts field (keeping its use minimal,
instead of reverting the commit that removed it).
Apparently users prefer this behavior.
It was used for subtitles too, so move the code to calculate the video
offset into a separate function. Seeking also needs to be fixed.
Fixes#1018.
In encoding mode, the AO pretends to be infinitely fast (it will take
whatever we write, without ever rejecting input). Commit 261506e3 broke
this somehow. It turns out an old hack dealing with this was accidentally
dropped.
This is the hunk of code whose semantics were (partially) dropped:
if (mpctx->d_audio && (mpctx->restart_playback ? !video_left :
ao_untimed(mpctx->ao) && (mpctx->delay <= 0 ||
!video_left)))
{
int status = fill_audio_out_buffers(mpctx, endpts);
// Not at audio stream EOF yet
audio_left = status > -2;
}
This if condition is pretty wild, and it looked like it was pretty much
for audio-only mode, rather than subtle handling for encoding mode.
Basically move the code from playloop.c to video.c. The new function
write_video() now contains the code that was part of run_playloop().
There are no functional changes, except handling "new_frame_shown"
slightly differently. This is done so that we don't need new a new
MPContext field or a return value for write_video() to signal this
condition. Instead, it's handled indirectly.
This also reduces some code duplication with other parts of the code.
The changfe is mostly cosmetic, although there are also some subtle
changes in behavior. At least one change is that the big desync message
is now printed after every seek.
In situations when the demuxer reports EOF, but immediately "recovers"
after that and returns new data, it could happen that audio sync was
skipped. Deal with this by actually entering the EOF state, instead of
assuming this will happen later.
wm4
Dmitrij D. Czarkoff
Marcin Kurczewski