atrac3: avoid oversized shifting in decode_bytes()

When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.

    (0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))

Avoid oversized shifting.

CC: libav-stable@libav.org

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

(cherry picked from commit eba1ff3130)

Conflicts:
	libavcodec/atrac3.c
This commit is contained in:
Xi Wang 2013-03-15 06:31:21 -04:00 committed by Luca Barbato
parent 0076639965
commit 12d8ae2979
1 changed files with 5 additions and 2 deletions

View File

@ -185,7 +185,10 @@ static int decode_bytes(const uint8_t* inbuffer, uint8_t* out, int bytes){
off = (intptr_t)inbuffer & 3;
buf = (const uint32_t *)(inbuffer - off);
c = av_be2ne32((0x537F6103 >> (off*8)) | (0x537F6103 << (32-(off*8))));
if (off)
c = av_be2ne32((0x537F6103U >> (off * 8)) | (0x537F6103U << (32 - (off * 8))));
else
c = av_be2ne32(0x537F6103U);
bytes += 3 + off;
for (i = 0; i < bytes/4; i++)
obuf[i] = c ^ buf[i];