avcodec/wmavoice: Fix rounding and integer anomalies in calc_input_response()

Fixes: out of array access
Fixes: inf is outside the range of representable values of type 'int'
Fixes: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long'
Fixes: 19316/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAVOICE_fuzzer-5677369365102592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 38d3758444)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2019-12-14 15:27:44 +01:00
parent 20c0ea160e
commit 9f30473846
1 changed files with 4 additions and 2 deletions

View File

@ -636,12 +636,14 @@ static void calc_input_response(WMAVoiceContext *s, float *lpcs,
for (n = 0; n <= 64; n++) {
float pwr;
idx = FFMAX(0, lrint((max - lpcs[n]) * irange) - 1);
idx = lrint((max - lpcs[n]) * irange - 1);
idx = FFMAX(0, idx);
pwr = wmavoice_denoise_power_table[s->denoise_strength][idx];
lpcs[n] = angle_mul * pwr;
/* 70.57 =~ 1/log10(1.0331663) */
idx = (pwr * gain_mul - 0.0295) * 70.570526123;
idx = av_clipf((pwr * gain_mul - 0.0295) * 70.570526123, 0, INT_MAX / 2);
if (idx > 127) { // fall back if index falls outside table range
coeffs[n] = wmavoice_energy_table[127] *
powf(1.0331663, idx - 127);