avcodec/pnmenc: Check av_image_get_buffer_size()

Fixes the crash in ticket #10050.
Also ensure that we don't overflow before ff_get_encode_buffer().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This commit is contained in:
Andreas Rheinhardt 2022-11-19 02:03:05 +01:00
parent 14c6093528
commit 715bf3509a

View File

@ -42,7 +42,10 @@ static int pnm_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
int size = av_image_get_buffer_size(avctx->pix_fmt,
avctx->width, avctx->height, 1);
if ((ret = ff_get_encode_buffer(avctx, pkt, size + 200, 0)) < 0)
if (size < 0)
return size;
if ((ret = ff_get_encode_buffer(avctx, pkt, size + 200U, 0)) < 0)
return ret;
bytestream_start =