RepoMirrors/mpv
1
0
Fork 0
mirror of https://github.com/mpv-player/mpv synced 2024-12-18 12:55:16 +00:00
Code Issues Releases Wiki Activity

bstr: make UTF-8 check stricter

Browse Source 
Don't accept overlong sequences. Don't accept codepoints past the
maximum unicode codepoint. Don't accept the UTF-16 surrogate codepoints.

I'm not sure if there are more codepoints that are defined to be
invalid, but we just want to make libavcodec happy, so this is enough.

(libavcodec's subtitle converter checks for valid UTF-8 and throws up
and dies if it's not - now we want to use bstr_sanitize_utf8_latin1() to
force valid UTF-8, so the strictness of our UTF-8 parser has to match at
least that of the libavcodec's check.)

I'm not sure whether the min test is actually 100% correct.

Note that libavcodec also treats BOM codepoints as invalid. This is
definitely a bug: the BOM is really just "zero-width non-breaking space"
redefined by Microsoft, but it is perfectly valid to appear in the
middle of a string. Official Unicode has merely deprecated the old
usage of the BOM codepoint, and didn't make it illegal. Besides, the
string could be from the start of a file, so even this check doesn't
make sense even with libavcodec's insane logic. We don't copy this bug.
This commit is contained in:
wm4 2013-08-15 21:14:00 +02:00
parent acb51c9243
commit 00f735d5cb
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
mpvcore/bstr.c
View File

@ -279,6 +279,14 @@ int bstr_decode_utf8(struct bstr s, struct bstr *out_next)
codepoint = (codepoint << 6) | (tmp & ~0xC0);
s.start++; s.len--;
}
if (codepoint > 0x10FFFF || (codepoint >= 0xD800 && codepoint <= 0xDFFF))
return -1;
// Overlong sequences - check taken from libavcodec.
// (The only reason we even bother with this is to make libavcodec's
// retarded subtitle utf-8 check happy.)
unsigned int min = bytes == 2 ? 0x80 : 1 << (5 * bytes - 4);
if (codepoint < min)
return -1;
}
if (out_next)
*out_next = s;