Long Term Prediction allows for prediction of spectral coefficients
via the previously decoded time-dependent samples. This feature
works well with harmonic content 2 or more frames long, like speech,
human or non-human, piano music or any constant tones at very low
bitrates.
It should be noted that the current coder is highly efficient and
the rate control system is unable to encode files at extremely
low bitrates (less than 14kbps seems to be impossible) so this
extension isn't capable of optimum operation. Dramatic difference
is observable with some types of audio and speech but for the most
part the audiable differences are subtle. The spectrum looks better
however so the encoder is able to harvest the additional bits that
this feature provies, should the user choose to enable it. So
it's best to enable this feature only if encoding at the absolutely
lowest bitrate that the encoder is capable of.
This finalizes merging of the work in the patches in ticket #2686.
Improvements to twoloop and RC logic are extensive.
The non-exhaustive list of twoloop improvments includes:
- Tweaks to distortion limits on the RD optimization phase of twoloop
- Deeper search in twoloop
- PNS information marking to let twoloop decide when to use it
(turned out having the decision made separately wasn't working)
- Tonal band detection and priorization
- Better band energy conservation rules
- Strict hole avoidance
For rate control:
- Use psymodel's bit allocation to allow proper use of the bit
reservoir. Don't work against the bit reservoir by moving lambda
in the opposite direction when psymodel decides to allocate more/less
bits to a frame.
- Retry the encode if the effective rate lies outside a reasonable
margin of psymodel's allocation or the selected ABR.
- Log average lambda at the end. Useful info for everyone, but especially
for tuning of the various encoder constants that relate to lambda
feedback.
Psy:
- Do not apply lowpass with a FIR filter, instead just let the coder
zero bands above the cutoff. The FIR filter induces group delay,
and while zeroing bands causes ripple, it's lost in the quantization
noise.
- Experimental VBR bit allocation code
- Tweak automatic lowpass filter threshold to maximize audio bandwidth
at all bitrates while still providing acceptable, stable quality.
I/S:
- Phase decision fixes. Unrelated to #2686, but the bugs only surfaced
when the merge was finalized. Measure I/S band energy accounting for
phase, and prevent I/S and M/S from being applied both.
PNS:
- Avoid marking short bands with PNS when they're part of a window
group in which there's a large variation of energy from one window
to the next. PNS can't preserve those and the effect is extremely
noticeable.
M/S:
- Implement BMLD protection similar to the specified in
ISO-IEC/13818:7-2003, Appendix C Section 6.1. Since M/S decision
doesn't conform to section 6.1, a different method had to be
implemented, but should provide equivalent protection.
- Move the decision logic closer to the method specified in
ISO-IEC/13818:7-2003, Appendix C Section 6.1. Specifically,
make sure M/S needs less bits than dual stereo.
- Don't apply M/S in bands that are using I/S
Now, this of course needed adjustments in the compare targets and
fuzz factors of the AAC encoder's fate tests, but if wondering why
the targets go up (more distortion), consider the previous coder
was using too many bits on LF content (far more than required by
psy), and thus those signals will now be more distorted, not less.
The extra distortion isn't audible though, I carried extensive
ABX testing to make sure.
A very similar patch was also extensively tested by Kamendo2 in
the context of #2686.
This commit removes the array which was made redundant with
the last commit. The current prediction system gets the
quantization error directly (and without the single-frame delay)
in the search_for_pred function.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit completely alters the algorithm of prediction.
The original commit which introduced prediction was completely
incorrect to even remotely care about what the actual coefficients
contain or whether any options were enabled. Not my actual fault.
This commit treats prediction the way the decoder does and expects
to do: like lossy encryption. Everything related to prediction now
happens at the very end but just before quantization and encoding
of coefficients. On the decoder side, prediction happens before
anything has had a chance to even access the coefficients.
Also the original implementation had problems because it actually
touched the band_type of special bands which already had their
scalefactor indices marked and it's a wonder the asserion wasn't
triggered when transmitting those.
Overall, this now drastically increases audio quality and you should
think about enabling it if you don't plan on playing anything encoded
on really old low power ultra-embedded devices since they might not
support decoding of prediction or AAC-Main. Though the specifications
were written ages ago and as times change so do the FLOPS.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit finalizes AAC-Main profile encoding support
by implementing all mandatory and optional tools available
in the specifications and current decoders.
The AAC-Main profile reqires that prediction support be
present (although decoders don't require it to be enabled)
for an encoder to be deemed capable of AAC-Main encoding,
as well as TNS, PNS and IS, all of which were implemented
with previous commits or earlier of this year.
Users are encouraged to test the new functionality using either
-profile:a aac_main or -aac_pred 1, the former of which will enable
the prediction option by default and the latter will change the
profile to AAC-Main. No other options shall be changed by enabling
either, it's currently up to the users to decide what's best.
The current implementation works best using M/S and/or IS,
so users are also welcome to enable both options and any
other options (TNS, PNS) for maximum quality.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit implements temporal noise shaping support in the
encoder, along with an -aac_tns option to toggle it on or off
(off by default for now). TNS will increase audio quality
and reduce quantization noise by applying a multitap FIR filter
across allowed coefficients and transmit side information to the
decoder so it could create an inverse filter.
Users are encouraged to test the new functionality by enabling
-aac_tns 1 during encoding.
No major bugs are observable at this time so after a while if no
new problems appear and if the current implementation is deemed
of high enough quality and stability it will be enabled by default,
possibly at the same time the encoder has its experimental flag
removed and becomes the standard aac encoder in ffmpeg.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This commit moves the quantizer to a separate header file.
This allows the quantizer to be used from a separate files outside
of aaccoder without having to put another function pointer and will
result in a slight speedup as the compiler can do more optimizations.
This is required for commits following.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Avoid clipping due to quantization noise to produce audible
artifacts, by detecting near-clipping signals and both attenuating
them a little and encoding escape-encoded bands (usually the
loudest) rounding towards zero instead of nearest, which tends to
decrease overall energy and thus clipping.
Currently fate tests measure numerical error so this change makes
tests using asynth (which are near clipping) report higher error
not less, because of window attenuation. Yet, they sound better,
not worse (albeit subtle, other samples aren't subtle at all).
Only measuring psychoacoustically weighted error would make for
a representative test, so that will be left for a future patch.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Add fixed point implementation of functions for generating tables
Signed-off-by: Nedeljko Babic <nedeljko.babic@imgtec.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This commit adds additional fields which are used by the native encoder to add intensity stereo support. It also adds some clarifying statements to the comments for the codebooks.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This commit adjusts the intial offset for PNS values, introduced
with commit f7f71b5795 earlier. This
commit shifts the value in such a way that no further offsets are
required in the aaccoder.c file. Earlier version of the PNS patch had 2 offsets in both the aaccoder and aacenc.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This commit implements support for writing the noise energy values used in PNS.
The difference between regular scalefactors and noise energy values is that the latter
require a small preamble (NOISE_PRE + energy_value_diff) to be written as the first
noise-containing band. Any following noise energy values use the previous one to
base their "diff" on. Ordinary scalefactors remain unchanged other than that they ignore the noise values.
This commit should not change anything by itself, the following commits will bring it in use.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This patch fixes a pointer arithmetic bug in adjust_frame_information that resulted in heavily corrupted audio when using M/S encoding. Also, a backup copy of untransformed coefficients has to be kept around or attempts at re-processing the frame (which happens when hevavily overspending bits during transients) will result in re-encoding of the coefficients and subsequent corruption of the resulting stream.
A/B testing shows the bug as corrected, but still cannot prove that M/S coding is a win at least in numbers. Limited listening tests do show improvement on M/S encoded samples in lower bitrates, but they're hidden among the other artifacts that remain to be corrected in the encoder.
Some of the regressions flagged in the report do show poor stereo image (but not buggy), so M/S encoding is clearly not good enough yet to be defaulted to auto.
In numbers, Patched against Unpatched, stereo_mode auto:
Files: 114
Bitrates: 6
Tests: 683
Serious Regressions: 0 (0%)
Regressions: 0 (0%)
Improvements: 227 (33%)
Big improvements: 92 (13%)
Worst regression - mybloodrusts.wv - 256k
- StdDev: 28.61 pSNR: -0.43 maxdiff: 1372.00
Best improvement - 60.wv - 384k
- StdDev: -369.57 pSNR: 45.02 maxdiff: -13322.00
Average - StdDev: -80.56 pSNR: 2.49 maxdiff: -8858.00
Patched against Unpatched stereo_mode ms_off shows no difference.
Patched stereo_mode auto vs Unpatched stereo_mode ms_off shows a small average improvement, just not too significant:
Serious Regressions: 0 (0%)
Regressions: 10 (1%)
Improvements: 45 (6%)
Big improvements: 2 (0%)
Worst regression - Illinois.wv - 256k
- StdDev: 33.20 pSNR: -2.03 maxdiff: 477.00
Best improvement - song_of_circomstances.flac - 384k
- StdDev: -3.97 pSNR: 7.61 maxdiff: -826.00
Average - StdDev: -10.25 pSNR: 0.20 maxdiff: -281.00
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'd615187f74ddf3413778a8b5b7ae17255b0df88e':
aacdec: Support for ER AAC ELD 480.
Conflicts:
libavcodec/aacdec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'e57daa876bf0cf50782550e366e589441cd8c2bd':
adpcm: decode directly to the user-provided AVFrame
ac3: decode directly to the user-provided AVFrame
aac: decode directly to the user-provided AVFrame
8svx: decode directly to the user-provided AVFrame
Conflicts:
libavcodec/8svx.c
libavcodec/ac3dec.c
libavcodec/adpcm.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '3d3cf6745e2a5dc9c377244454c3186d75b177fa':
aacdec: use float planar sample format for output
Conflicts:
libavcodec/aacdec.c
libavcodec/aacsbr.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '1cd432e167b1a80853760c89a33606e2b5f229c2':
configure: fix libcdio check
rtsp: Allow setting the reordering buffer size via an AVOption
rtsp: Vertically align a constant definition
rtp: Update the check for distinguishing between RTP and RTCP
aac: fix build with hardcoded tables
fate: dependencies for screen codec tests
riff: Move functions around to be covered by appropriate #ifdefs
Conflicts:
configure
tests/fate/screen.mak
Merged-by: Michael Niedermayer <michaelni@gmx.at>
aac_tablegen.h includes aac.h for the POW_SF2_ZERO definition, but
this also pulls in a raft of other headers, some of which are not
safe to use in code built with the host compiler.
Moving POW_SF2_ZERO to aac_tablegen_decl.h, where the declaration
of the array it relates to already resides, fixes the problems.
Signed-off-by: Mans Rullgard <mans@mansr.com>
* qatar/master:
yuv4mpeg: return proper error codes.
Give all anonymously typedeffed structs in headers a name
fate: Add parseutils test
parseutils-test: Drop random colors from parsing test
vf_pad/scale: use double precision for aspect ratios.
build: error on variable-length arrays
ppc: swscale: rework yuv2planeX_altivec()
ppc: fmtconvert: kill VLA in float_to_int16_interleave_altivec()
x86: dsputil: kill VLA in gmc_mmx()
libspeexenc: Updated commentary to reflect recent changes
libspeexenc: Add an option for enabling DTX
doc/APIchanges: fill in missing dates and hashes.
lavr: bump major to 1 and declare it stable.
lavr: change the type of the data buffers to uint8_t**.
lavc: deprecate the audio resampling API.
Conflicts:
cmdutils.h
configure
doc/APIchanges
ffplay.c
libavcodec/dwt.h
libavcodec/libspeexenc.c
libavfilter/vf_pad.c
libavfilter/vf_scale.c
libavformat/asf.h
tests/fate/libavutil.mak
tests/ref/fate/parseutils
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Japanese DTV uses some non standard extensions in AAC audio.
One example is 'dual mono', which combines two independent
audio into one stereo stream, storing them in left and right channels
respectively. Historically, dual mono audio has been used for
multi-lingual audio, one for local/native language, and another for english,
and usually the "main" (local language) channel should be output without
any user interactions.
The frames of those dual mono audio are allowed to set
ADTS channel_config field to 0, and just contain two SCE's *WITHOUT* PCE,
which is a non standard extension by Japanese DTV standard.
(ref. ARIB STD-B32 PartII 5.2.3)
This patch adds an AVPacket side data, AV_PKT_DATA_JP_DUALMONO,
which indicates that the AVPacket is likely to contain an audio frame
with the above dual mono extension, and has the parameter to specify
the desired channel selection in that case.
It also makes aacdec to detect dual mono and output just the desired
channel when this side data is attached.
Signed-off-by: Akihiro Tsukada <atsukada@users.sourceforge.net>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
float_dsp: ppc: add a separate header for Altivec function prototypes
ARM: fix float_dsp breakage from d5a7229
Add a float DSP framework to libavutil
PPC: Move types_altivec.h and util_altivec.h from libavcodec to libavutil
ARM: Move asm.S from libavcodec to libavutil
vc1dsp: mark put/avg_vc1_mspel_mc() always_inline
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Ganesh Ajjanagadde
Rostislav Pehlivanov
Claudio Freire
Djordje Pesut
Jovan Zelincevic
Michael Niedermayer
Anton Khirnov
Alex Converse
Benoit Fouet
Diego Biurrun
Mirjana Vulin
Justin Ruggles
Ronald S. Bultje
Mans Rullgard
Akihiro Tsukada