This fixes audio encoding crashing under ASan.
When extended_data != data, FFmpeg copies more pointers from
extended_data (= the number of channels) than there really
are for non-planar formats (= exactly 1), but that's not our fault.
Regardless, this commit makes it work in all common cases.
Changes:
- code refactored;
- mixer options removed;
- new mpv sound API used;
- add sound devices detect (mpv --audio-device=help will show all available devices);
- only OSSv4 supported now;
Tested on FreeBSD 12.2 amd64.
This makes the behavior of all control messages consistent,
fixing an inconsistency that has been with us since
4d8266c739 - which is the initial
rework of the polyaudio AO into the pulseaudio AO.
Muting the stream also directly triggers an update to the OSD.
When not waiting for the command completion this read of the mute
property may read the old state. A stale read.
Note that this somehow was not triggered on native Pulseaudio, but it is
an issue on Pipewire.
See https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/868
Set pcm state to SND_PCM_STATE_XRUN in case -EPIPE is received,
and handle this state as per the usual logic.
This way snd_pcm_prepare gets called, and the loop continued.
Inspired by a patch posted by malc_ on #mpv.
--audio-stream-silence is a shitty feature compensating for awful
consumer garbage, that mutes PCM at first to check whether it's
compressed audio, using formats advocated and owned by malicious patent
troll companies (who spend more money on their lawyers than paying any
technicians), wrapped in a wasteful way to make it constant bitrate
using a standard whose text is not freely available, and only rude users
want it. This feature has been carelessly broken, because it's
complicated and stupid. What would Jesus do? If not getting an aneurysm,
or pushing over tables with expensive A/V receivers on top of them, he'd
probably fix the feature. So let's take inspiration from Jesus Christ
himself, and do something as dumb as wasting some of our limited
lifetime on this incredibly stupid fucking shit.
This is tricky, because state changes like end-of-audio are supposed to
be driven by the AO driver, while playing silence precludes this. But it
seems code paths for "untimed" AOs can be reused.
But there are still problems. For example, underruns will just happen
normally (and stop audio streaming), because we don't have a separate
heuristic to check whether the buffer is "low enough" (as a consequence
of a network stall, but before the audio output itself underruns).
Create a central function which pumps data through the filter. This also
might fix bogus use of the filter API on flushing. (The filter is just
used for convenience, but I guess the overall result is still simpler.)
AVFrame doesn't have public code for pool allocation, so mpv does it
manually. AVFrame allocation is very tricky, so we added a bug.
This crashed with libopus encoding, but not some other audio codecs,
because the libopus libavcodec wrapper accesses AVFrame.data. Most code
tries to avoid accessing AVFrame.data and uses AVFrame.extended_data,
because using the former would subtly corrupt memory on more than 8
channels. The fact that this problem manifested only now shows that most
AVFrame consuming FFmpeg code indeed uses extended_data for audio.
It is now the AO's responsibility to handle period size alignment. The
ao->period_size alignment field is unused as of the recent audio
refactor commit. Remove it.
It turns out that ao_alsa shows extremely inefficient behavior as a
consequence of the removal of period size aligned writes in the
mentioned refactor commit. This is because it could get into a state
where it repeatedly wrote single samples (as small as 1 sample), and
starved the rest of the player as a consequence. Too bad. Explicitly
align the size in ao_alsa. Other AOs, which need this, should do the
same.
One reason why it broke so badly with ao_alsa was that it retried the
write() even if all reported space could be written. So stop doing that
too. Retry the write only if we somehow wrote less.
I'm not sure about ao_pulse.
Unused, was terrible garbage. It was (or at least its implementation
was) always a make-shift solution, and just gross bullshit. It is unused
now, so delete it.
This replaces the two buffers (ao_chain.ao_buffer in the core, and
buffer_state.buffers in the AO) with a single queue. Instead of having a
byte based buffer, the queue is simply a list of audio frames, as output
by the decoder. This should make dataflow simpler and reduce copying.
It also attempts to simplify fill_audio_out_buffers(), the function I
always hated most, because it's full of subtle and buggy logic.
Unfortunately, I got assaulted by corner cases, dumb features (attempt
at seamless looping, really?), and other crap, so it got pretty
complicated again. fill_audio_out_buffers() is still full of subtle and
buggy logic. Maybe it got worse. On the other hand, maybe there really
is some progress. Who knows.
Originally, the data flow parts was meant to be in f_output_chain, but
due to tricky interactions with the playloop code, it's now in the dummy
filter in audio.c.
At least this improves the way the audio PTS is passed to the encoder in
encoding mode. Now it attempts to pass frames directly, along with the
pts, which should minimize timestamp problems. But to be honest, encoder
mode is one big kludge that shouldn't exist in this way.
This commit should be considered pre-alpha code. There are lots of bugs
still hiding.
Allow mp_aframe_clip_timestamps() to discard a spdif frame if it's
entirely out of the timestamp range. Just a minor thing that might make
handling these dumb formats easier.
Previously get_state() would keep setting the cork status
while paused, but it only does for that after underflows now.
Correct this oversight by creating the stream corked for start()
to uncork it at a later time.
fixes#8026
FFmpeg expects those fields to be set on the AVFrame when
encoding audio, not doing so will cause the avcodec_send_frame
call to return EINVAL (at least in recent builds).
scaletempo2 is a new audio filter for playing back
audio at modified speed and is based on chromium
commit 51ed77e3f37a9a9b80d6d0a8259e84a8ca635259.
It sounds subjectively better than the existing
implementions scaletempo and rubberband.
When get_state() corks the stream after an underrun happens
priv->playing is incorrectly reset to true, which can cause the
player to miss the underrun entirely. Stop resetting priv->playing
during corking (but not uncorking) to fix this.
The underflow callback introduced in d27ad96 can be called
when the buffer is still full, causing playback to never
resume afterwards since get_state() reports free_samples == 0.
Fix this by fully resetting on underrun, which flushes
the stream and ensures free buffer space.
fixes#7874
The pull mode APIs were previously required to have thread-safe
ao_controls. However, locks were added in b83bdd1 for parity with push
mode. This introduced deadlocks in ao_wasapi.
Instead, only lock ao_control for the push mode APIs.
fixes#7787
See also #7832, #7811. We'll wait for feedback to see if those should
also be closed.
AOs which use the "push" API must set this field now. Actually, this was
sort of always required, but happened to work anyway. The future
intention is to use device_buffer as the pre-buffer amount, which has to
be available right before audio playback is started. "Pull" AOs really
need this too conceptually, just that the API is underspecified.
From what I can see, only ao_null did not do this yet.
Previously, device_buffer defaulted to 0 on pulse. This meant that
commit baa7b5c would always wait with a timeout of 0, leading to
high CPU usage for PulseAudio users.
By setting device_buffer to the number of samples per channel that
PulseAudio sets as its target, this commit fixes this behaviour.
The playback thread may obviously still fill the AO'S entire audio
buffer, which means it unset p->draining, which makes no sense and broke
ao_drain(). So just don't unset it here.
Not sure if this really fixes this, it was hard to reproduce. Regression
due to the recent changes. There are probably many more bugs like this.
Stupid asynchronous nightmare state machine. Give me a language that
supports formal verification (in presence of concurrency) or something.
I feel like this makes slightly more sense. At least it doesn't include
the potentially arbitrary constant latency that is generally included in
the delay value. Also, the buffer status doesn't matter - either we've
filled the entire buffer (then we can wait this long), or there's not
enough data anyway (then the core will wake up the thread if new data is
available).
But ultimately, we have to guess, unless the AO does notify us with
ao_wakeup_playthread().
Draining may now wait for no reason up to 1/4th of the total buffer
time. Shouldn't be a disimprovement in practice.
It's conceivable that ao->driver->reset() will make the audio API wait
for ao_read_data() (i.e. its audio callback) to return. Since we
recently moved the reset() call inside the same lock that ao_read_data()
acquires, this could deadlock. Whether this really happens depends on
how exactly the AO behaves. For example, ao_wasapi does not have this
problem. "Push" AOs are not affected either.
Fix by moving it outside of the lock. Assume ao->driver->start() will
not have this problem.
Could affect ao_sdl, ao_coreaudio (and similar rotten fruit AOs). I'm
unsure whether anyone experienced the problem in practice.
Instead of the relatively subtle underflow handling, simply signal
whether the stream is in a playing state. Should make it more robust.
Should affect ao_alsa and ao_pulse only (and ao_openal, but it's
broken).
For ao_pulse, I'm just guessing. How the hell do you query whether a
stream is playing? Who knows. Seems to work, judging from very
superficial testing.
Just a detail. If wrong (not unlikely because I'm just guessing my own
messy state machine), this will make the player freeze due to waiting
for something that never happens. Enjoy.
The feeder thread basically woke up the core and itself too often, and
caused some CPU overhead. This was caused by the recent buffer.c
changes.
For one, do not let ao_read_data() wake up the core, and instead rely on
the feeder thread's own buffer management. This is a bit strange, since
the change intended to unify the buffer management, but being more
consequent about it is better deferred to later, when the buffer
management changes again anyway. And also, the "more" condition in the
feeder thread seems outdated, or at least what made it make sense has
been destroyed, so do something that may or may not be better. In any
case, I'm still not getting underruns with ao_alsa, but the wakeup
hammering is gone.
This affects "pull" AOs only: ao_alsa, ao_pulse, ao_openal, ao_pcm,
ao_lavc. There are changes to the other AOs too, but that's only about
renaming ao_driver.resume to ao_driver.start.
ao_openal is broken because I didn't manage to fix it, so it exits with
an error message. If you want it, why don't _you_ put effort into it? I
see no reason to waste my own precious lifetime over this (I realize the
irony).
ao_alsa loses the poll() mechanism, but it was mostly broken and didn't
really do what it was supposed to. There doesn't seem to be anything in
the ALSA API to watch the playback status without polling (unless you
want to use raw UNIX signals).
No idea if ao_pulse is correct, or whether it's subtly broken now. There
is no documentation, so I can't tell what is correct, without reverse
engineering the whole project. I recommend using ALSA.
This was supposed to be just a simple fix, but somehow it expanded scope
like a train wreck. Very high chance of regressions, but probably only
for the AOs listed above. The rest you can figure out from reading the
diff.
wasapi/coreaudio/sdl were affected, alsa/pusle were not.
The confusion here was that resume() has different meaning with pull and
push AOs.
Fixes: #7772
Regression since the recent refactor. How did nobody notice?
This happened because the push code now calls the function for the pull
code. Both the former and latter apply the volume, so oops.
The recent change to the common code removed all calls to ->drain. It's
currently emulated via a timed sleep and polling ao_eof_reached(). That
is actually fallback code for AOs which lacked draining. I could just
readd the drain call, but it was a bad idea anyway. My plan to handle
this better is to require the AO to signal a underrun, even if
AOPLAY_FINAL_CHUNK is not set. Also reinstate not possibly waiting for
ao_lavc.c. ao_pcm.c did not have anything to handle this; whatever.
This is preparation to further cleanups (and eventually actual
improvements) of the audio output code.
AOs are split into two classes: pull and push. Pull AOs let an audio
callback of the native audio API read from a ring buffer. Push AOs
expose a function that works similar to write(), and for which we start
a "feeder" thread. It seems making this split was beneficial, because of
the different data flow, and emulating the one or other in the AOs
directly would have created code duplication (all the "pull" AOs had
their own ring buffer implementation before it was cleaned up).
Unfortunately, both types had completely separate implementations (in
pull.c and push.c). The idea was that little can be shared anyway. But
that's very annoying now, because I want to change the API between AO
and player.
This commit attempts to merge them. I've moved everything from push.c to
pull.c, the trivial entrypoints from ao.c to pull.c, and attempted to
reconcile the differences. It's a mess, but at least there's only one
ring buffer within the AO code now. Everything should work mostly the
same. Pull AOs now always copy the audio data under a lock; before this
commit, all ring buffer access was lock-free (except for the decoder
wakeup callback, which acquired a mutex). In theory, this is "bad", and
people obsessed with lock-free stuff will hate me, but in practice
probably won't matter. The planned change will probably remove this
copying-under-lock again, but who knows when this will happen.
One change for the push AOs now makes it drop audio, where before only a
warning was logged. This is only in case of AOs or drivers which exhibit
unexpected (and now unsupported) behavior.
This is a risky change. Although it's completely trivial conceptually,
there are too many special cases. In addition, I barely tested it, and
I've messed with it in a half-motivated state over a longer time, barely
making any progress, and finishing it under a rush when I already should
have been asleep. Most things seem to work, and I made superficial tests
with alsa, sdl, and encode mode. This should cover most things, but
there are a lot of tricky things that received no coverage. All this
text means you should be prepared to roll back to an older commit and
report your problem.
This mode drops or repeats audio data to adapt to video speed, instead
of resampling it or such. It was added to deal with SPDIF. The
implementation was part of fill_audio_out_buffers() - the entire
function is something whose complexity exploded in my face, and which I
want to clean up, and this is hopefully a first step.
Put it in a filter, and mess with the shitty glue code. It's all sort of
roundabout and illogical, but that can be rectified later. The important
part is that it works much like the resample or scaletempo filters.
For PCM audio, this does not work on samples anymore. This makes it much
worse. But for PCM you can use saner mechanisms that sound better. Also,
something about PTS tracking is wrong. But not wasting more time on
this.
sfan5
rim
Thomas Weißschuh
Dorian Rudolph
Jan Ekström
wm4
LAGonauta
ekisu
Dorian Rudolph
Kevin Mitchell
Nicolas F