vulkan_h264: reject end_frame being called without valid session parameters

When seeking through MBAFF-coded H264, this can happen. Decoding calls end_frame
without calling start_frame. We are unable to decode this, as no frame
state has been set.

Happens for both VAAPI and Vulkan. Could be an issue elsewhere, hence
the individual commit.
This commit is contained in:
Lynne 2023-05-20 15:49:01 +02:00
parent 160a415e22
commit f701f088f4
No known key found for this signature in database
GPG Key ID: A2FEA5F03F034464
1 changed files with 4 additions and 0 deletions

View File

@ -508,10 +508,14 @@ static int vk_h264_end_frame(AVCodecContext *avctx)
const H264Context *h = avctx->priv_data;
H264Picture *pic = h->cur_pic_ptr;
H264VulkanDecodePicture *hp = pic->hwaccel_picture_private;
FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data;
FFVulkanDecodePicture *vp = &hp->vp;
FFVulkanDecodePicture *rvp[H264_MAX_PICTURE_COUNT] = { 0 };
AVFrame *rav[H264_MAX_PICTURE_COUNT] = { 0 };
if (!dec->session_params)
return AVERROR(EINVAL);
for (int i = 0; i < vp->decode_info.referenceSlotCount; i++) {
H264Picture *rp = hp->ref_src[i];
H264VulkanDecodePicture *rhp = rp->hwaccel_picture_private;