avcodec/simple_idct: Fix runtime error: left shift of negative value -6395

Fixes: 633/clusterfuzz-testcase-4553133554401280

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2017-02-21 03:14:49 +01:00
parent 956472a323
commit 28dc6e7291

View File

@ -66,8 +66,8 @@ static inline void idct4col_put(uint8_t *dest, int line_size, const int16_t *col
a1 = col[8*2];
a2 = col[8*4];
a3 = col[8*6];
c0 = ((a0 + a2) << (CN_SHIFT - 1)) + (1 << (C_SHIFT - 1));
c2 = ((a0 - a2) << (CN_SHIFT - 1)) + (1 << (C_SHIFT - 1));
c0 = ((a0 + a2) * (1 << CN_SHIFT - 1)) + (1 << (C_SHIFT - 1));
c2 = ((a0 - a2) * (1 << CN_SHIFT - 1)) + (1 << (C_SHIFT - 1));
c1 = a1 * C1 + a3 * C2;
c3 = a1 * C2 - a3 * C1;
dest[0] = av_clip_uint8((c0 + c1) >> C_SHIFT);