avcodec/omx: fix xFramerate calculation

Integer overflow in the Q16 framerate calculation was sending
invalid values to the OMX encoder.

On the RPI4, this manifested as bitrate controls being ignored
on video streams with 60000/1001 framerates. Video streams with
30000/1001 framerates were not affected.

Signed-off-by: Aman Gupta <aman@tmm1.net>
This commit is contained in:
Aman Gupta 2019-08-29 16:00:50 -07:00
parent f4eb7d84a7
commit b022d9ba28
1 changed files with 2 additions and 2 deletions

View File

@ -473,9 +473,9 @@ static av_cold int omx_component_init(AVCodecContext *avctx, const char *role)
in_port_params.format.video.nFrameWidth = avctx->width;
in_port_params.format.video.nFrameHeight = avctx->height;
if (avctx->framerate.den > 0 && avctx->framerate.num > 0)
in_port_params.format.video.xFramerate = (1 << 16) * avctx->framerate.num / avctx->framerate.den;
in_port_params.format.video.xFramerate = (1LL << 16) * avctx->framerate.num / avctx->framerate.den;
else
in_port_params.format.video.xFramerate = (1 << 16) * avctx->time_base.den / avctx->time_base.num;
in_port_params.format.video.xFramerate = (1LL << 16) * avctx->time_base.den / avctx->time_base.num;
err = OMX_SetParameter(s->handle, OMX_IndexParamPortDefinition, &in_port_params);
CHECK(err);