avformat/rpl: Use 64bit in bitrate computation and check it

Fixes: signed integer overflow: 777777776 * 4 cannot be represented in type 'int'
Fixes: 29102/clusterfuzz-testcase-minimized-ffmpeg_dem_RPL_fuzzer-6726188921913344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 29b244ffc1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2021-04-26 22:43:51 +02:00
parent d2e90cde29
commit 97d0d3ae61
1 changed files with 4 additions and 2 deletions

View File

@ -199,8 +199,10 @@ static int rpl_read_header(AVFormatContext *s)
ast->codecpar->bits_per_coded_sample = 4;
ast->codecpar->bit_rate = ast->codecpar->sample_rate *
ast->codecpar->bits_per_coded_sample *
ast->codecpar->channels;
(int64_t)ast->codecpar->channels;
if (ast->codecpar->bit_rate > INT64_MAX / ast->codecpar->bits_per_coded_sample)
return AVERROR_INVALIDDATA;
ast->codecpar->bit_rate *= ast->codecpar->bits_per_coded_sample;
ast->codecpar->codec_id = AV_CODEC_ID_NONE;
switch (audio_format) {