We send a refcounted frame to the encoder, but then disrespect
refcounting rules and write to the frame data without making sure the
buffer is really writeable.
In theory this can lead to reallocation on every frame is the encoder
really keeps a reference. If we really cared, we could fix this by
providing a buffer pool. But then again, we don't care.
This code was supposed to adjust existing conversion filters (to make
them output a different format). But the code was just broken,
apparently a refactoring accident. It accessed af instead of af->prev.
The bug tended to add new conversion filters, even if an existing one
could have been used. (Can be tested by inserting a dummy lavrresample
filter followed by a format filter which forces conversion.)
In addition, it's probably better to return the actual error code if
reinitializing the filter fails. It would then respect an AF_FALSE
return value, which means format negotiation failed, instead of a
generic error.
af_volume has a volumedb sub-option, which allows the user to set an
explicit volume. Until recently, the player read back this value and
used it as initial softvol volume. But now it just overwrites it.
Instead of overwriting it, multiply the different gain values. Above
all, this will do the right thing if only softvol is used, or if the
user only adds the af_volume filter manually.
Normally, you want downmixing to happen first thing in the filter chain.
This is reflected in codec downmixing, which feeds the filter chain
downmixed audio in the first place. Doing this has the advantage of
needing less data to process. But the main motivation is that if there
is a drc filter in the chain, you want to process it the downmixed
audio.
Add an idiotic heuristic to achieve this. It tries to detect whether the
audio was indeed automatically downmixed (or upmixed). To detect what
the output format is going to be, it builds the filter chain normally,
and then retries with the heuristic applied (and for extra paranoia,
retries without the heuristic again if it fails to successfully rebuild
the filter chain for unknown reasons). This is simple and will work in
almost all cases.
Doing it in a more complete way is rather hard, because filters are so
generic. For example, we know absolutely nothing about the behavior of
af_lavfi, which creates an opaque filter graph with libavfilter. We
don't know why a filter would e.g. change the channel layout on its
output. (Our heuristic bails out in this case.) We're also slave to the
lowest common denominator of how our format negotiation works, and how
libavfilter's works.
In theory, we could make this mechanism explicit by introducing a
special dummy filter. The filter chain would then try to convert between
input and output formats at the dummy filter, which would give the user
more control over how downmix happens. On the other hand, the user could
just insert explicit conversion filters instead, so this would probably
have questionable value.
The algorithm and functionality is the same, but the code becomes much
simpler and easier to follow.
The assumption that there is only 1 conversion filter (lavrresample)
helps with the simplification, but the main change is to use the same
code for format/channels/rate. Get rid of the different AF_CONTROL_SET_*
controls, and change the af->data parameters directly. (af->data is
badly named, but essentially is a placeholder for the output format.)
Also, instead of trying to use the af_reinit() loop to init inserted
conversion filters or filters with changed output formats, do it inline,
and move the common code to a filter_reinit() function. This gets rid of
the awful retry variable.
In general, this should not change any runtime behavior.
This happens to be better for the af_volume filter (for softvol), and
saves some code too. It's "better" because you want to affect the
final filtered audio, such as after a manually inserted drc filter.
Instead of hardcoding what the libavcodec ac3 encoder expects, configure
it based on the AVCodec fields.
Unfortunately, it doesn't export the list of sample rates, so that is
done manually. This commit actually fixes the rate always to 48Khz. I
don't even know whether the other rates worked. (Possibly did, but
they'd still change the spdif parameters, and would work differently
from ad_spdif.c.)
The complex filter support that will be added makes much more complex
use of libavfilter, and I'm not going to bother with adding hacks to
keep libavfilter optional.
Remove flc-frc <-> sl<->sr. This was just plain wrong, and a mistaken
change to make 7.1 work properly on CoreAudio with 7.1(rear) layout.
Also see the following commit.
Add br-br <-> sl<->sr, because we decided that it makes sense.
Note that this "fudging" is applied only if the channel pairs are
replaced, i.e. they would get dropped and be replaced with silence. This
is done to compensate for libswresample's default rematrixing (which
takes care of some more common cases).
This also makes it refcounted, i.e. the new AVFrame will reference the
mp_audio buffers, instead of potentially forcing the consumer of the
AVFrame to copy the data.
All the extra code is for handling the >8 channels case, which requires
very messy dealing with the extended_ fields (not our fault).
This is probably the 3rd time the user-visible behavior changes. This
time, switch back because not normalizing seems to be the more expected
behavior from users.
This covers source files which were added in mplayer2 and mpv times
only, and where all code is covered by LGPL relicensing agreements.
There are probably more files to which this applies, but I'm being
conservative here.
A file named ao_sdl.c exists in MPlayer too, but the mpv one is a
complete rewrite, and was added some time after the original ao_sdl.c
was removed. The same applies to vo_sdl.c, for which the SDL2 API is
radically different in addition (MPlayer supports SDL 1.2 only).
common.c contains only code written by me. But common.h is a strange
case: although it originally was named mp_common.h and exists in MPlayer
too, by now it contains only definitions written by uau and me. The
exceptions are the CONTROL_ defines - thus not changing the license of
common.h yet.
codec_tags.c contained once large tables generated from MPlayer's
codecs.conf, but all of these tables were removed.
From demux_playlist.c I'm removing a code fragment from someone who was
not asked; this probably could be done later (see commit 15dccc37).
misc.c is a bit complicated to reason about (it was split off mplayer.c
and thus contains random functions out of this file), but actually all
functions have been added post-MPlayer. Except get_relative_time(),
which was written by uau, but looks similar to 3 different versions of
something similar in each of the Unix/win32/OSX timer source files. I'm
not sure what that means in regards to copyright, so I've just moved it
into another still-GPL source file for now.
screenshot.c once had some minor parts of MPlayer's vf_screenshot.c, but
they're all gone.
Prevents channels from being dropped, e.g. when going 7.1 -> 7.1(wide)
and similar cases. The reasoning here is that channel layouts over HDMI
don't work anyway, and not dropping a channel and playing it on a
slightly "wrong" (but expected) speaker is preferable to playing silence
on these speakers.
Do this to remove issues with ao_coreaudio. Frankly I'm not sure whether
our mapping (between CA and mpv/FFmpeg speakers) is correct, but on the
other hand due to the reasons stated above it's not all that meaningful.
Of course, only FFmpeg has av_clipd(), while Libav does not. (Nevermind
that it doesn't do much more than the mpv MPCLAMP() macro. Supposedly,
libavutil can provide optimized platform-specific versions for av_clip*,
but of course nothing actually does for av_clipf() or av_clipd().)
libswresample doesn't do it - although it should, but the patch is stuck
in limbo.
Probably reduces problems with artifacts on downmixing in some cases.
Just set the ratio directly by working around the intended semantics of
the API function. The silly rounding stuff we had isn't needed anymore
(and not entirely correct anyway).
Note that since the compensation is virtually active forever, we need to
reset if it's not needed. So always run this code to be sure to reset
it.
Also note that libswresample itself had a precision issue, until it
was fixed in FFmpeg commit 351e625d.
Actually, it didn't really require that before (most work was avoided),
but some bits had to be run anyway. Separate the speed change into a
light-weight function, which merely updates already created filters, and
a heavy-weight one which messes with filter insertion.
This also happens to fix the case where the filters would "forget" the
current speed (force resampling, change speed, hit a volume control to
force af_volume insertion - it will reset speed and desync).
Since we now always run the light-weight function, remove the
af_scaletempo verbose message that is printed on speed setting. Other
than that, all setters are cheap.
For some reason, the encoder didn't like that the AVPacket already had
fields set. I'm not quite sure, but this might just be invalid API
usage. Do it as it's recommended.
This reverts commit 4e358a9636.
Testing shows the channel pairs must indeed be swapped (details see
commit message of the reverted commit). Making the downmix code move
sl/sr to sdl/sdr is not an appropriate solution anymore, and it's
better to fix the unusual channel layout in ao_alsa.c directly.
(Not reverting the change in chmap.c; this is still correct.)
ao_alsa: attempt to fix 7.1 over HDMI
The last 2 channels of 7.1 (RLC/RRC in ALSA) were exported as sdl/sdr
instead of sl/sr (I don't even know why I chose sdl/sdr, but SL/SR
and RLC/RRC are different in the ALSA API). libsw/avresample do not
move the sl/sr channels to sdl/sdr when rematrixing, so silence was
sent for 2 channels. If my selection of sdl/sdr is essentially API
abuse, there's no reason why they should do this differently.
The mess here is really that ALSa doesn't map the HDMI layouts cleanly.
Most ALSA drivers export 7.1 in a way compatible to our expectations,
but Intel HDA/HDMI does not:
mpv/ffmpeg: fl-fr-fc-lfe-bl-br-sl-sr
ALSA/generic: FL FR FC LFE RL RR SL SR [1]
ALSA/HDMI: FL FR LFE FC RL RR RLC RRC [2]
The HDMI layout is layout 0x13 (going by CEA-861-B). The comment in
the kernel code has to be correct too. The early standard defines only
1 other layout, which replaces RLC/RRC with FRC/FLC - this probably
corresponds to what we call "7.1(wide)".
So it appears when ALSA requests RLC/RRC, we should feed it sl/sr.
To make it more complicated, Kodi/xbmc apparently also have to deal with
ALSA being special, but instead of sending sl/sr to RLC/RRC, they swap
the last two pairs of the layout, and send sl/sr to RL/RR and bl/br to
RLC/RRC. Or I might have misunderstood their code. I don't have a
7.1-capable A/V receiver, so I can't test this.
For now, go with the simpler solution, and wait until someone tests it.
If the speakers end up swapped, a completely different solution will be
needed.
[1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/sound/core/pcm_lib.c?id=refs/tags/v4.3#n2434
[2] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/sound/pci/hda/patch_hdmi.c?id=refs/tags/v4.3#n307
av_free_packet() got finally deprecated. Use av_packet_unref() instead,
which has almost the same semantics, has existed for a while, and is
available in all FFmpeg and Libav versions we support.
av_get_default_channel_layout() fails with channel counts larger than 8.
The channel layout doesn't need to make sense, so pick an arbitrary
fallback.
libswresample also has options for setting the channel counts directly,
but better not introduce new concepts in the code. Also, libavresample
doesn't have these options.
This code removes filters which can not take spdif inout. This was made
so that PCM filters are transparently dropped in spdif mode.
This entered an endless loop with:
--af=lavcac3enc:::2 --audio-channels=5.1
The forced number of output channels is incompatible with spdif. It's
trying to insert af_lavrresample as conversion filter to compensate for
it. Of course this doesn't work, which triggers the PCM filter removal.
Then it goes on normally - since the new state is exactly as before, it
will try the same thing again, forever.
Fix by reusing the retry counter, which is a very dumb but very
effective measure against these cases of filter negotiation failure. We
could try to be more clever (for example, if the removed filter is a
conversion filter, we can be sure this won't work, and error out
immediately). But better keep it simple and robust.
Small adjustments to the playback speed use swr_set_compensation()
to stretch the audio as it is required. But since large adjustments
are now handled by actually reinitializing libswresample, the small
adjustments get rounded off completely with typical frame sizes.
Compensate for this by accounting for the rounding error and keeping
track of fractional samples that should have been output to achieve
the correct ratio.
This fixes display sync mode behavior, which requires these adjustments
to be relatively accurate.
swr/avresample_set_compensation() was made for small speed adjustments.
Non-documentation says it should be used for changes not larger than 1%,
so reinitialize the sampler if the change is larger than that.
swr_set_compensation() changes the apparent sample rate on the fly (who
would have guessed). It is thus very well-suited for adjusting audio
speed on the fly during playback (like needed by the display-sync mode).
It skips the relatively slow resampler reinitialization.
If this doesn't work (libswresample soxr backend), then fall back to the
old method.
This flag was used by some filters and made sure none of these filters
were inserted twice. This triggers only if the user explicitly tries to
add multiple filters (and not e.g. due to auto-insertion), so at best
this warned the user from doing something potentially pointless. At
worst, it blocked some (mildly) legitimate use-cases. Get rid of it.
Also see #2322.
The reason MPlayer traditionally duplicated them all over the place is
that it wanted every component to be a self-contained library (e.g.
audio filters were in "libaf"). But this is not necessarily helpful, and
this change makes the following commit a bit simpler.
Not sure why struct af_resample_opts even exists. It seems useful to
group the fields set by user options. But storing the current format
conversion parameters doesn't seem very elegant, and having a separate
instance in the "ctx" field isn't helpful either.
Some users still use this filter, so the filter was going to be kept.
But I overlooked that libavfilter provides this filter. Remove the
redundant wrapper from mpv. Something like --af=lavfi=bs2b should work
and give exactly the same results.
All of these filters are considered not useful anymore by us. Some have
replacements in libavfilter (useable through af_lavfi).
af_center, af_extrastereo, af_karaoke, af_sinesuppress, af_sub,
af_surround, af_sweep: pretty simple and useless filters which probably
nobody ever wants.
af_ladspa: has a replacement in libavfilter.
af_hrtf: the algorithm doesn't work properly on most sources, and the
implementation was buggy and complicated. (The filter was inherited from
MPlayer; but even in mpv times we had to apply fixes that fixed major
issues with added noise.) There is a ladspa filter if you still want to
use it.
af_export: I'm not even sure what this is supposed to do. Possibly it
was meant for GUIs rendering audio visualizations, but it couldn't
really work well. For example, the size of the audio depended on the
samplerate (fixed number of samples only), and it couldn't retrieve the
complete audio, only fragments. If this is really needed for GUIs, mpv
should add native visualization, or a proper API for it.
This was a minor optimization to potentially avoid resampler
reconfiguration when the filter is reinitialized. But filter
reinitialization is a rare event, and the case when no reconfiguration
is needed is even rarer. As such, this is an unnecessary micro-
optimization and only adds potential for bugs.
This message bloats verbose log output if e.g. audio speed is frequently
readjusted, such as when syncing audio to video. So don't print the
message if only speed is changed. (This case requires reconfiguration,
but can't change the input/output channel maps.)
Also do not print the message if no remixing is done at all.
Some filter chains require a huge number of auto-inserted conversion
filters. There is an overly stupid safeguard against infinite filter
insertions, which counts the number of conversion filters inserted. This
triggered accidentally in this case. Fix by resetting this counter after
a non-conversion filter was successfully configured.
Replace all the check macros with function calls. Give them all the
same case and naming schema.
Drop af_fmt2bits(). Only af_fmt2bps() survives as af_fmt_to_bytes().
Introduce af_fmt_is_pcm(), and use it in situations that used
!AF_FORMAT_IS_SPECIAL. Nobody really knew what a "special" format
was. It simply meant "not PCM".
This avoids keeping "bad" state from previous reconfig calls, such as
the internal_sample_format option (which is set only on the first
reconfig call).
There's no advantage to keeping the resample contexts around anyway.
Basically, af_fix_format_conversion() behaves stupid you insert a
conversion filter that won't work, and adding back the conversion test
function is the simplest fix to it.
This attempted to find a minimal filter graph for a format conversion
involving multiple conversion filters. With the last 2 commits it
becomes dead code - remove it.
Now af_lavrresample can output 24 bit samples directly, by doing the
conversion "inline". Luckily, S32->S24 can be done in-place, so this
isn't too much work. But the output conversion logic (which seems to be
adding up) gets slightly more complicated again.
Normally this is done by af_convert24. But having multiple conversion
filters complicates some aspects of the filter chain. S24 output is the
only thing the code for multiple conversion filters is still needed for,
and getting rid of that is preferable.
If the code path for additional output conversion is active,
reorder_planes() is always called, even if the reorder_out array wasn't
filled. This is obviously wrong - always fill this array.
They are useless. Not only are they actually rarely in use; but
libavcodec doesn't even output them, as libavcodec has no such sample
formats for decoded audio.
Even if it should happen that we actually still need them (e.g. if doing
direct hardware output), there are better solutions. Swapping the sign
is a fast and lossless operation and can be done inplace, so AO actually
needing it could do this directly.
If you wonder why we keep U8 instead of S8: because libavcodec does it.
Until now, we didn't do this, because it required some effort, and
didn't seem to be necessary. It probably still isn't, but it sounds
like a good idea not to output arbitrary data on these channels.
The situation is complicated by the fact that just adding new channels
to a planar frame would require messing with buffers. So we would have
to allocate new buffers and add them to the frame. We could have to
maintain an extra buffer pool for this. Avoid this by being "clever",
and just allocate a frame with enough channels in the first place.
libav/swresample won't know about these channels and won't write to
them, but we can grab them in reorder_planes() and use them for the
NA channels.
This is better, because now we call swr_get_delay() with the output
samplerate, instead of with the input samplerate and then multiplying it
with the ratio and rounding it up.
This manually retrieved the remaining audio from the resampler. It
subtly missed a conversion which could leave to an unsubtle crash.
This could happen if reorder_planes() was supposed to insert NA
channels, and the resampler/actual output format were different.
Simplify it by reusing the normal drain path. One oddness is that
the filter will add an output frame outside of normal filtering,
but that should be fine.
This brings the volume control closer to what is percepted as linear
volume change.
Adjust the --softvol-max default to roughly the old maximum (roughly
doubles the gain).
Some time ago, a mechanism was added for automatically removing PCM-only
filters if the input format is spdif.
This could cause an infinite loop if the AO did not support spdif, but
was falling back to some PCM format. Then this code tried to remove the
last filter, which is a dummy filter for receiving and queuing filter
output. af_remove() simply fails gracefully in this case, so this
happens over and over again.
Fix by explicitly checking whether the filter to remove is a dummy
filter. (af_remove() also fails only if the dummy filters are attempted
to be removed - checking this directly is simpler.)
Some audio APIs explicitly require you to add dummy channels. These are
not rendered, and only exist for the sake of the audio API or hardware
strangeness. At least ALSA, Sndio, and CoreAudio seem to have them.
This commit is preparation for using them with ao_coreaudio.
The result is a bit messy. libavresample/libswresample don't have good
API for this; avresample_set_channel_mapping() is pretty useless.
Although in theory you can use it to add and remove channels, you
can't set the channel counts. So we do the ordering ourselves by making
sure the audio data is planar, and by swapping the plane pointers. This
requires lots of messiness to get the conversions in place. Also, the
input reordering is still done with the "old" method, and doesn't
support padded channels - hopefully this will never be needed. (I tried
to come up with cleaner solutions, but compared to my other attempts,
the final commit is not that bad.)
It could happen that a lavrresample filter would keep its old output
format when the decoder changed its output format. This simply happened
because the output format was never reset.
Normally, this was not an issue, because lavrresample filters only
inserted for format conversion were removed on format changes. But if
--no-audio-pitch-correction is set and playback speed is changed, then
there is a "permanent" lavrresample filter in the filter chain, which
shows this behavior.
Fix by explicitly resetting output formats for all filters which support
it.
Note: this can crash with libswresample in some cases. I'm not sure if
this is mpv's fault or libswresample's, but since it works with
libavresample, I'm going to assume it's not our's.
The af_add() function has a problem: if the inserted filter returns
AF_DETACH during init, the function will have a dangling pointer. Until
now this was avoided by making sure none of the used filters actually
return AF_DETACH, but it's getting infeasible.
Solve this by requiring passing an unique label to af_add(), which is
then used instead of the pointer.
libswresample doesn't normalize when remixing to a float format. This
will cause clipping due to float samples being out of the allowed range.
Fortunately this extremely bad default can be changed.
This does not happen with libavresample: it normalizes by default.
Fixes#1752.
--af=bs2b:help abort()ed because the default value of the "profile"
option is not represented by any choice. Fix it by adding an "unset"
choice. (It's a bit odd because there's already a "default" choice,
which is not default, but I don't care enough about this filter.)
Fixes#1712.
To handle seeking correctly, we need to flush the filter. libavfilter
does not support flushing, so we destroy and recreate it. We also need
to handle resume-after-EOF, because the mpv audio code sends an EOF
before and after seeking (the latter happens because the player drains
the filter chain in a generic way, which "causes" EOF).
Handle the failure gracefully, instead of exploding and disabling audio.
Just set the speed back to 1.0.
Also remove the AF_DETACH from af_scaletempo. This actually created a
dangling pointer in af_add(), a tricky consequence of af_add()
reconfiguring the filter chain and the newly added filter using
AF_DETACH. Fortunately the AF_DETACH is not needed (and probably never
worked - it comes from MPlayer times, and MPlayer also disables audio
when trying to change speed with spdif).
Although the libraries we use for resampling (libavresample and
libswresample) do not support changing sampelrate on the fly, this makes
it easier to make sure no audio buffers are implicitly dropped. In fact,
this commit adds additional code to drain the resampler explicitly.
Changing speed twice without feeding audio in-between made it crash
with libavresample inc ertain cases (libswresample is fine). This is
probably a libavresample bug. Hopefully this will be fixed, and also I
attempted to workaround the situation that crashes it. (It seems to
point in direction of random memory corruption, though.)
wm4
stepshal
Marcin Kurczewski
Kevin Mitchell