vqavideodev: Check image dimensions

Fixes out of heap array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3583c8706d)
Independently-Found-by: Fabian Yamaguchi
Fixes: CVE-2012-0947

Conflicts:

	libavcodec/vqavideo.c
This commit is contained in:
Michael Niedermayer 2012-03-22 23:43:37 +01:00
parent 9de0c8c60c
commit e70d202275

View File

@ -164,6 +164,11 @@ static av_cold int vqa_decode_init(AVCodecContext *avctx)
s->codebook = av_malloc(s->codebook_size);
s->next_codebook_buffer = av_malloc(s->codebook_size);
if (s->width % s->vector_width || s->height % s->vector_height) {
av_log(avctx, AV_LOG_ERROR, "Picture dimensions are not a multiple of the vector size\n");
return AVERROR_INVALIDDATA;
}
/* initialize the solid-color vectors */
if (s->vector_height == 4) {
codebook_index = 0xFF00 * 16;