From 9b0f9003dfab6a230d46aaa94091bf509d889f37 Mon Sep 17 00:00:00 2001 From: Andreas Rheinhardt Date: Thu, 2 Apr 2020 23:07:55 +0200 Subject: [PATCH] avformat/matroskaenc: Make ebml_num_size() more robust Matroska (or actually EBML) uses variable-length numbers where only seven bits of every byte is usable for the length; the other bits encode the length of the variable-length number. So in order to find out how many bytes one needs to encode a given number one can use a loop like while (num >> 7 * bytes) bytes++; the Matroska muxer effectively did this. Yet it has a disadvantage: It is impossible for the result of a single right shift of an unsigned number with most significant bit set to be zero, because one can only shift by 0..(width - 1). On some architectures like x64 it is not even possible to do it with undefined right shifts in which case this leads to an infinite loop. This can be easily avoided by switching to a loop whose condition is (num >>= 7). The maximum value the so modified function can return is 10; any value > 8 is invalid and will now lead to an assert in put_ebml_num() or in start_ebml_master() (or actually in put_ebml_size_unknown()). Signed-off-by: Andreas Rheinhardt --- libavformat/matroskaenc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index e1ddf366d4..1a3db72043 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -194,9 +194,11 @@ static void put_ebml_size_unknown(AVIOContext *pb, int bytes) */ static int ebml_num_size(uint64_t num) { - int bytes = 1; - while ((num + 1) >> bytes * 7) + int bytes = 0; + num++; + do { bytes++; + } while (num >>= 7); return bytes; }