avcodec/wmalosslessdec: Fix multiple integer overflows

Fixes: left shift of 3329 by 20 places cannot be represented in type 'int'
Fixes: signed integer overflow: -199378355 + -1948950833 cannot be represented in type 'int'
Fixes: 19837/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5752565837070336
Fixes: 19839/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5767483265122304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 422202516c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2020-01-09 02:06:36 +01:00
parent 109cab1487
commit 862c0048ec
1 changed files with 3 additions and 2 deletions

View File

@ -535,7 +535,8 @@ static int decode_channel_residues(WmallDecodeCtx *s, int ch, int tile_size)
i++;
}
for (; i < tile_size; i++) {
int quo = 0, rem, rem_bits, residue;
int rem, rem_bits;
unsigned quo = 0, residue;
while(get_bits1(&s->gb)) {
quo++;
if (get_bits_left(&s->gb) <= 0)
@ -774,7 +775,7 @@ static void revert_cdlms ## bits (WmallDecodeCtx *s, int ch, \
s->cdlms[ch][ilms].recent, \
FFALIGN(s->cdlms[ch][ilms].order, ROUND), \
WMASIGN(residue)); \
input = residue + (pred >> s->cdlms[ch][ilms].scaling); \
input = residue + (unsigned)(pred >> s->cdlms[ch][ilms].scaling); \
lms_update ## bits(s, ch, ilms, input); \
s->channel_residues[ch][icoef] = input; \
} \