avutil/video_enc_params: Combine overflow checks

This patch also fixes a -Wtautological-constant-out-of-range-compare
warning from Clang and a -Wtype-limits warning from GCC on systems
where size_t is 64bits and unsigned 32bits. The reason for this seems
to be that variable (whose value derives from sizeof() and can therefore
be known at compile-time) is used instead of using sizeof() directly in
the comparison.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit is contained in:
Andreas Rheinhardt 2021-02-14 19:43:56 +01:00
parent 1a555d3c60
commit 39df279c74

View File

@ -33,8 +33,7 @@ AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type,
size_t size;
size = sizeof(*par);
if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) ||
nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)
if (nb_blocks > (SIZE_MAX - size) / sizeof(AVVideoBlockParams))
return NULL;
size += sizeof(AVVideoBlockParams) * nb_blocks;