avcodec/libx265: improve full range flag setting logic

Unlike libx264, libx265 does not have a separate "unspecified"/"auto"
default for color range, so we do always have to specify it.
Thus, we are required to handle the RGB case on the libavcodec
side to enable the correct value to be written out in in case
of RGB content with unspecified color range being received.

In other words:
1. If the user has set color range specifically, follow that.
2. If the user has not set color range specifically, set full
   range by default in case of RGB and YUVJ pixel formats.
This commit is contained in:
Jan Ekström 2021-08-17 23:35:38 +03:00
parent 7ca71b79f2
commit dbe40478e2
1 changed files with 9 additions and 4 deletions

View File

@ -181,10 +181,15 @@ static av_cold int libx265_encode_init(AVCodecContext *avctx)
ctx->params->vui.bEnableVideoSignalTypePresentFlag = 1;
ctx->params->vui.bEnableVideoFullRangeFlag = avctx->pix_fmt == AV_PIX_FMT_YUVJ420P ||
avctx->pix_fmt == AV_PIX_FMT_YUVJ422P ||
avctx->pix_fmt == AV_PIX_FMT_YUVJ444P ||
avctx->color_range == AVCOL_RANGE_JPEG;
if (avctx->color_range != AVCOL_RANGE_UNSPECIFIED)
ctx->params->vui.bEnableVideoFullRangeFlag =
avctx->color_range == AVCOL_RANGE_JPEG;
else
ctx->params->vui.bEnableVideoFullRangeFlag =
(av_pix_fmt_desc_get(avctx->pix_fmt)->flags & AV_PIX_FMT_FLAG_RGB) ||
avctx->pix_fmt == AV_PIX_FMT_YUVJ420P ||
avctx->pix_fmt == AV_PIX_FMT_YUVJ422P ||
avctx->pix_fmt == AV_PIX_FMT_YUVJ444P;
if ((avctx->color_primaries <= AVCOL_PRI_SMPTE432 &&
avctx->color_primaries != AVCOL_PRI_UNSPECIFIED) ||