avcodec/omx: fix xFramerate calculation

Integer overflow in the Q16 framerate calculation was sending
invalid values to the OMX encoder.

On the RPI4, this manifested as bitrate controls being ignored
on video streams with 60000/1001 framerates. Video streams with
30000/1001 framerates were not affected.

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit b022d9ba28)
This commit is contained in:
Aman Gupta 2019-08-29 16:00:50 -07:00
parent b4e9103709
commit 0f8e2a0b86
1 changed files with 2 additions and 2 deletions

View File

@ -473,9 +473,9 @@ static av_cold int omx_component_init(AVCodecContext *avctx, const char *role)
in_port_params.format.video.nFrameWidth = avctx->width;
in_port_params.format.video.nFrameHeight = avctx->height;
if (avctx->framerate.den > 0 && avctx->framerate.num > 0)
in_port_params.format.video.xFramerate = (1 << 16) * avctx->framerate.num / avctx->framerate.den;
in_port_params.format.video.xFramerate = (1LL << 16) * avctx->framerate.num / avctx->framerate.den;
else
in_port_params.format.video.xFramerate = (1 << 16) * avctx->time_base.den / avctx->time_base.num;
in_port_params.format.video.xFramerate = (1LL << 16) * avctx->time_base.den / avctx->time_base.num;
err = OMX_SetParameter(s->handle, OMX_IndexParamPortDefinition, &in_port_params);
CHECK(err);