From 979572365f2133f969f3f49ec6a99cc8739d2eba Mon Sep 17 00:00:00 2001 From: Ganesh Ajjanagadde Date: Fri, 9 Oct 2015 15:59:04 -0400 Subject: [PATCH] avcodec/ac3enc: fix undefined negative left shift This should fix the undefined behavior reported in: https://trac.ffmpeg.org/ticket/4727. I can reproduce this at runtime: simply stick in an abort call in asym_quant to check if c < 0 and run FATE. I don't know ac3 so I can't confirm if negative coefficients are intentional, but at the moment they clearly are according to FATE. This resolves the undefined behavior. Tested with FATE. Signed-off-by: Ganesh Ajjanagadde Signed-off-by: Michael Niedermayer --- libavcodec/ac3enc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/ac3enc.c b/libavcodec/ac3enc.c index 35e721a964..c8a0caa194 100644 --- a/libavcodec/ac3enc.c +++ b/libavcodec/ac3enc.c @@ -1183,7 +1183,7 @@ static inline int asym_quant(int c, int e, int qbits) { int m; - c = (((c << e) >> (24 - qbits)) + 1) >> 1; + c = (((c * (1<> (24 - qbits)) + 1) >> 1; m = (1 << (qbits-1)); if (c >= m) c = m - 1;