This will make af_channels output a channel layout that is compatible
with any destination layout. Not sure if that's a good idea though,
since the way the AO choses a layout is perhaps less predictable. On the
other hand, using the old MPlayer standard layouts doesn't make much
sense either. We'll see whether this improves or breaks someone's use
case.
Apparently this stopped working after some planar changes (broken format
negotiation). Radically change option parsing in an incompatible way.
Suggest alternatives to this filter, since it barely has any importance
anymore.
Normally, audio decoder don't have a decoder delay, so the code was
fine. But FFmpeg supports multithreaded decoding for some audio codecs,
which introduces such a delay.
The delay means that we won't get decoded audio for the first few
packets, and that we need to do something to get the trailing audio
still buffered in the decoder when reaching EOF.
Two changes are needed to deal with the delay:
- If EOF is reached, pass a "flush" packet to the decoder to return the
buffered audio. Such a flush packet is automatically setup when
calling mp_set_av_packet() with a NULL packet.
- Use the PTS returned by the decoder, instead of the packet's. This is
important to get correct timestamps for decoded audio. Ignoring this
would result into offsetting the audio playback time by the decoder
delay. Note that we can still use the timestamp of the first packet
to get the timestamp for the start of the audio.
If the timebase is set, it's used for converting the packet timestamps.
Otherwise, the previous method of reinterpret-casting the mpv style
double timestamps to libavcodec style int64_t timestamps is used.
Also replace the kind of awkward mp_get_av_frame_pkt_ts() function by
mp_pts_from_av(), which simply converts timestamps in a way the old
function did. (Plus it takes a timebase parameter, similar to the
addition to mp_set_av_packet().)
Note that this should not change anything yet. The code in ad_lavc.c and
vd_lavc.c passes NULL for the timebase parameters. We could set
AVCodecContext.pkt_timebase and use that if we want to give libavcodec
"proper" timestamps.
This could be important for ad_lavc.c: some codecs (opus, probably mp3
and aac too) have weird requirements about doing decoding preroll on the
container level, and thus require adjusting the audio start timestamps
in some cases. libavcodec doesn't tell us how much was skipped, so we
either get shifted timestamps (by the length of the skipped data), or we
give it proper timestamps. (Note: libavcodec interprets or changes
timestamps only if pkt_timebase is set, which by default it is not.)
This would require selecting a timebase though, so I feel uncomfortable
with the idea. At least this change paves the way, and will allow some
testing.
There are some use cases for this. For example, you can use it to set
defaults of automatically inserted filters (like af_lavrresample). It's
also useful if you have a non-trivial VO configuration, and want to use
--vo to quickly change between the drivers without repeating the whole
configuration in the --vo argument.
This is needed so that new processes (created with fork+exec) don't
inherit open files, which can be important for a number of reasons.
Since O_CLOEXEC is relatively new (POSIX.1-2008, before that Linux
specific), we #define it to 0 in io.h to prevent compilation errors on
older/crappy systems. At least this is the plan.
input.c creates a pipe. For that, add a mp_set_cloexec() function (which
is based on Weston's code in vo_wayland.c, but more correct). We could
use pipe2() instead, but that is Linux specific. Technically, we have a
race condition, but it won't matter.
This partially reverts commit 7d152965. It turns out that at least some
ALSA drivers (at least snd-hda-intel) report incorrect audio delay with
non-native sample rates, even if the sample rate is only very slightly
different from the native one.
For example, 48000Hz is fine on my hda-intel system, while both 8000Hz
and 47999Hz lead to a delay off by 40ms (according to mpv's A/V
difference display), which suggests that something in ALSA is
calculating the delay using the wrong sample rate.
As an additional problem, with ALSA resampling enabled, using
48001Hz/float/2ch fails, while 49000Hz/float/2ch or 48001Hz/s16/2ch
work. With resampling disabled, all these cases work obviously, because
our own resampler doesn't just refuse any of these formats.
Since some people want to use the ALSA resampler (because it's highly
configurable, supports multiple backends, etc.), we still allow enabling
ALSA resampling with an ao_alsa suboption.
Previous code was using the values of the AudioChannelLabel enum directly to
create the channel bitmap. While this was quite smart it was pretty unreadable
and fragile (what if Apple changes the values of those enums?).
Change it to use a 'dumb' conversion table.
These used the suffix _resync_stream, which is a bit misleading. Nothing
gets "resynchronized", they really just reset state.
(Some audio decoders actually used to "resync" by reading packets for
resuming playback, but that's not the case anymore.)
Also move the function in dec_video.c to the top of the file.
The code stopped at kAudioChannelLabel_TopBackRight and missed mapping for
5 more channel labels. These are in a completely different order that the mpv
ones so they must be mapped manually.
This includes the case when lavc decodes audio with more than 8
channels, which our audio chain currently does not support.
the changes in ad_lavc.c are just simplifications. The code tried to
avoid overriding global parameters if it found something invalid, but
that is not needed anymore.
Resampling with non-ancient ALSA setups works fine, so there is no
need to keep this around. Furthermore, as of writing, the default
builtin resampler used by many ALSA setups (taken from libspeex)
actually has higher quality than the default resampling modes of
avresample and swresample.
Apparently just 5 packets is not enough for the initial audio decode
(which is needed to find the format). The old code (before the recent
refactor) appeared to use 5 packets, but there were apparently other
code paths which in the end amounted to more than 5 packets being read.
The sample that failed (see github issue #368) needed 9 packets.
Fixes#368.
This used to be needed to access the generic stream header from the
specific headers, which in turn was needed because the decoders had
access only to the specific headers. This is not the case anymore, so
this can finally be removed again.
Also move the "format" field from the specific headers to sh_stream.
sh_audio is supposed to contain file headers, not whatever was decoded.
Fix this, and write the decoded format to separate fields in the decoder
context, the dec_audio.decoded field. (Note that this field is really
only needed to communicate the audio format from decoder driver to the
generic code, so no other code accesses it.)
Move all state that basically changes during decoding or is needed in
order to manage decoding itself into a new struct (dec_audio).
sh_audio (defined in stheader.h) is supposed to be the audio stream
header. This should reflect the file headers for the stream. Putting the
decoder context there is strange design, to say the least.
The AF control commands used an elaborate and unnecessary organization
for the command constants. Get rid of all that and convert the
definitions to a simple enum. Also remove the control commands that
were not really needed, because they were not used outside of the
filters that implemented them.
And by "cleanup", I mean "remove". Actually, only remove the parts that
are redundant and doxygen noise. Move useful parts to the comment above
the function's implementation in the C source file.
When the decoder detects a format change, it overwrites the values
stored in sh_audio (this affects the members sample_format, samplerate,
channels). In the case when the old audio data still needs to be
played/filtered, the audio format as identified by sh_audio and the
format used for the decoder buffer can mismatch. In particular, they
will mismatch in the very unlikely but possible case the audio chain is
reinitialized while old data is draining during a format change.
Or in other words, sh_audio might contain the new format, while the
audio chain is still configured to use the old format.
Currently, the audio code (player/audio.c and init_audio_filters) access
sh_audio to get the current format. This is in theory incorrect for the
reasons mentioned above. Use the decoder buffer's format instead, which
should be correct at any point.
Commit 22b3f522 not only redid major aspects of audio decoding, but also
attempted to fix audio format change handling. Before that commit, data
that was already decoded but not yet filtered was thrown away on a
format change. After that commit, data was supposed to finish playing
before rebuilding filters and so on.
It was still buggy, though: the decoder buffer was initialized to the
new format too early, triggering an assertion failure. Move the reinit
call below filtering to fix this.
ad_mpg123.c needs to be adjusted so that it doesn't decode new data
before the format change is actually executed.
Add some more assertions to af_play() (audio filtering) to make sure
input data and configured format don't mismatch. This will also catch
filters which don't set the format on their output data correctly.
Regression due to planar_audio branch.
Simulate proper handling of AOPLAY_FINAL_CHUNK. Print when underruns
occur (i.e. running out of data). Add some options that control
simulated buffer and outburst sizes.
All this is useful for debugging and self-documentation. (Note that
ao_null always was supposed to simulate an ideal AO, which is the reason
why it fools people who try to use it for benchmarking video.)
This should allow it to select better fallback formats, instead of
picking the first encoder sample format if ao->format is not equal to
any of the encoder sample formats.
Not sure what is supposed to happen if the encoder provides no
compatible sample format (or no sample format list at all), but in this
case ao_lavc.c still fails gracefully.
The added function af_format_conversion_score() can be used to select
the best sample format to convert to in order to reduce loss and extra
conversion work.
It calculates a "loss" score when going from one format to another, and
for each conversion that needs to be done a certain score is subtracted.
Thus, if you have to convert from one format to a set of other formats,
you can calculate the score for each conversion, and pick the one with
the highest score.
Conversion between int and float is considered the worst case. One odd
consequence is that when converting from s32 to u8 or float, u8 will be
picked.
Test program used to develop this follows:
#define MAX_FMT 200
struct entry {
const char *name;
int score;
};
static int compentry(const void *px1, const void *px2)
{
const struct entry *x1 = px1;
const struct entry *x2 = px2;
if (x1->score > x2->score)
return 1;
if (x1->score < x2->score)
return -1;
return 0;
}
int main(int argc, char *argv[])
{
for (int n = 0; af_fmtstr_table[n].name; n++) {
struct entry entry[MAX_FMT];
int entries = 0;
for (int i = 0; af_fmtstr_table[i].name; i++) {
assert(i < MAX_FMT);
entry[entries].name = af_fmtstr_table[i].name;
entry[entries].score =
af_format_conversion_score(af_fmtstr_table[i].format,
af_fmtstr_table[n].format);
entries++;
}
qsort(&entry[0], entries, sizeof(entry[0]), compentry);
for (int i = 0; i < entries; i++) {
printf("%s -> %s: %d \n", af_fmtstr_table[n].name,
entry[i].name, entry[i].score);
}
}
}
These must be written even if there was no "final frame", e.g. due to
the player being exited with "q".
Although the issue is mostly of theoretical nature, as most audio codecs
don't need the final encoding calls with NULL data. Maybe will be more
relevant in the future.
This used to be in bytes, now it's in samples. Divide the value by 8
(assuming a typical audio format, float samples with 2 channels).
Fix some editing mistake or non-sense about the extra buffering added
(1<<x instead of x<<5).
Also sneak in a s/MPlayer/mpv/.
This changes option parsing as well as filter defaults slightly. The
default is now to encode to spdif (this is way more useful than writing
raw AC3 - what was this even useful for, other than writing broken ac3
-in-wav files?). The bitrate parameter is now always in kbps.
Apparently this was completely broken after commit 22b3f522. Basically,
this locked up immediately completely while decoding the first packet.
The reason was that the buffer calculations confused bytes and number of
samples. Also, EOF reporting was broken (wrong return code).
The special-casing of ad_mpg123 and ad_spdif (with DECODE_MAX_UNIT) is a
bit annoying, but will eventually be solved in a better way.
ao_null should simulate a "perfect" AO, but framestepping behaved quite
badly with it. Framstepping usually exposes problems with AOs dropping
their buffers on pause, and that's what happened here.
Most libavcodec decoders output non-interleaved audio. Add direct
support for this, and remove the hack that repacked non-interleaved
audio back to packed audio.
Remove the minlen argument from the decoder callback. Instead of
forcing every decoder to have its own decode loop to fill the buffer
until minlen is reached, leave this to the caller. So if a decoder
doesn't return enough data, it's simply called again. (In future, I
even want to change it so that decoders don't read packets directly,
but instead the caller has to pass packets to the decoders. This fits
well with this change, because now the decoder callback typically
decodes at most one packet.)
ad_mpg123.c receives some heavy refactoring. The main problem is that
it wanted to handle format changes when there was no data in the decode
output buffer yet. This sounds reasonable, but actually it would write
data into a buffer prepared for old data, since the caller doesn't know
about the format change yet. (I.e. the best place for a format change
would be _after_ writing the last sample to the output buffer.) It's
possible that this code was not perfectly sane before this commit,
and perhaps lost one frame of data after a format change, but I didn't
confirm this. Trying to fix this, I ended up rewriting the decoding
and also the probing.
libav* is generally freaking horrible, and might do bad things if the
data pointer passed to it are not aligned. One way to be sure that the
alignment is correct is allocating all pointers using av_malloc().
It's possible that this is not needed at all, though. For now it might
be better to keep this, since the mp_audio code is intended to replace
another buffer in dec_audio.c, which is currently av_malloc() allocated.
The original reason why this uses av_malloc() is apparently because
libavcodec used to directly encode into mplayer buffers, which is not
the case anymore, and thus (probably) doesn't make sense anymore.
(The commit subject uses the word "cargo cult", after all.)
Remove the awkward planarization. It had to be done because the AC3
encoder requires planar formats, but now we support them natively.
Try to simplify buffer management with mp_audio_buffer.
Improve checking for buffer overflows and out of bound writes. In
theory, these shouldn't happen due to AC3 fixed frame sizes, but being
paranoid is better.
The format negotiation is the same, except don't confusingly copy the
input format into af->data, just to overwrite it later. af->data should
alwass contain the output format, and the existing code was just a very
misguided use of the af_test_output() helper function.
Just set af->data to the output format immediately, and modify the input
format properly.
Also, if format negotiation fails (and needs another iteration), don't
initialize the libavcodec encoder.
Before this commit, the af_instance->mul/delay values were in bytes.
Using bytes is confusing for non-interleaved audio, so switch mul to
samples, and delay to seconds. For delay, seconds are more intuitive
than bytes or samples, because it's used for the latency calculation.
We also might want to replace the delay mechanism with real PTS
tracking inside the filter chain some time in the future, and PTS
will also require time-adjustments to be done in seconds.
For most filters, we just remove the redundant mul=1 initialization.
(Setting this used to be required, but not anymore.)
Since ao_openal simulates multi-channel audio by placing a bunch of
mono-sources in 3D space, non-interleaved audio is a perfect match for
it. We just have to remove the interleaving code.
ALSA supports non-interleaved audio natively using a separate API
function for writing audio. (Though you have to tell it about this on
initialization.) ALSA doesn't have separate sample formats for this,
so just pretend to negotiate the interleaved format, and assume that
all non-interleaved formats have an interleaved companion format.
Replace the code that used a single buffer with mp_audio_buffer. This
also enables non-interleaved output operation, although it's still
disabled, and no AO supports it yet.
Implementation wise, this could be much improved, such as using a
ringbuffer that doesn't require copying data all the time. This is
why we don't use mp_audio directly instead of mp_audio_buffer.
This comes with two internal AO API changes:
1. ao_driver.play now can take non-interleaved audio. For this purpose,
the data pointer is changed to void **data, where data[0] corresponds to
the pointer in the old API. Also, the len argument as well as the return
value are now in samples, not bytes. "Sample" in this context means the
unit of the smallest possible audio frame, i.e. sample_size * channels.
2. ao_driver.get_space now returns samples instead of bytes. (Similar to
the play function.)
Change all AOs to use the new API.
The AO API as exposed to the rest of the player still uses the old API.
It's emulated in ao.c. This is purely to split the commits changing all
AOs and the commits adding actual support for outputting N-I audio.
Allocate af_instance->data in generic code before filter initialization.
Every filter needs af->data (since it contains the output
configuration), so there's no reason why every filter should allocate
and free it.
Remove RESIZE_LOCAL_BUFFER(), and replace it with mp_audio_realloc_min().
Interestingly, most code becomes simpler, because the new function takes
the size in samples, and not in bytes. There are larger change in
af_scaletempo.c and af_lavcac3enc.c, because these had copied and
modified versions of the RESIZE_LOCAL_BUFFER macro/function.
No AO can handle these, so it would be a problem if they get added
later, and non-interleaved formats get accepted erroneously. Let them
gracefully fall back to other formats.
Most AOs actually would fall back, but to an unrelated formats. This is
covered by this commit too, and if possible they should pick the
interleaved variant if a non-interleaved format is requested.
Based on earlier work by Stefano Pigozzi.
There are 2 changes:
1. Instead of mp_audio.audio, mp_audio.planes[0] must be used.
2. mp_audio.len used to contain the size of the audio in bytes. Now
mp_audio.samples must be used. (Where 1 sample is the smallest unit
of audio that covers all channels.)
Also, some filters need changes to reject non-interleaved formats
properly.
Nothing uses the non-interleaved features yet, but this is needed so
that things don't just break when doing so.
This affects 64 bit floats and big endian integer PCM variants
(basically crap nobody uses). Possibly not all MS-muxed files work, but
I couldn't get or produce any samples.
Remove a bunch of format tags that are not needed anymore. Most of these
were used by demux_mov, which is long gone. Repurpose/abuse 'twos' as
mpv-internal tag for dealing with the PCM variants mentioned above.
Now to shift audio pts when outputting to e.g. avi, you need an explicit
facility to insert/remove initial samples, to avoid initial regions of
the video to be sped up/slowed down.
One such facility is the delay filter in libavfilter.
My main problem with this is that the output format will be incorrect.
(This doesn't matter right, because there are no samples output.)
This assumes all audio filters can deal with len==0 passed in for
filtering (though I wouldn't see why not).
A filter can still signal an error by returning NULL.
af_lavrresample has to be fixed, since resampling 0 samples makes
libavresample fail and return a negative error code. (Even though it's
not documented to return an error code!)
Apparently we were using FFmpeg-specific APIs. I have no idea whether
this code is correct on both FFmpeg and Libav (no examples, bad
doxygen... why do they even complaint aht people are using their APIs
incorrectly?), but it appears to work on FFmpeg. That was also the case
before commit ebc4ccb though, where it used internal libavformat
symbols.
Untested on Libav, Travis will tell us.
Set the PulseAudio stream title, just like the VO window title is set.
Refactor update_vo_window_title() so that we can use it for AOs too.
The ao_pulse.c bit is stolen from MPlayer.
In theory, af_volume could use separate volume levels for each channel.
But this was never used anywhere.
MPlayer implemented something similar before (svn r36498), but kept the
old path for some reason.
This member was redundant. sh_audio->sample_format indicates the sample
size already.
The TV code is a bit strange: the redundant sample size was part of the
internal TV interface. Assume it's really redundant and not something
else. The PCM decoder ignores the sample size anyway.
Also do some cosmetic changes, like merging definition and
initialization of local variables.
Remove an annoying debug mp_msg() from af_open(). It just printed the
command line parameters; if this is really needed, it could be added
to af.c instead (similar as to what vf.c does).
Helps with readability. Also remove the ctx_opt_set_* helper macros and
use av_opt_set_* directly (I think these macros were used because the
lines ended up too long, but this commit removes two indentation levels,
giving more space).
This should allow to make format negotiation much simpler, since it
takes the responsibility to compare actual input and accepted input
formats from the filters. It's also backwards compatible. Filters which
have expensive initialization still can use the old method.
wm4
bugmen0t
Stefano Pigozzi
Martin Herkt
Rudolf Polzer
William Light