libswresample: Prevent out of bounds.

We've been fuzzing torchvision with [sydr-fuzz](https://github.com/ispras/oss-sydr-fuzz)
and found out of bounds error in ffmpeg project at audioconvert.c:151.
To prevent error we need to fix checks for in and out fmt in swr_init.

Signed-off-by: Eli Kobrin <kobrineli@ispras.ru>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Eli Kobrin 2023-08-02 15:14:10 +03:00 committed by Michael Niedermayer
parent 0e9956a06e
commit 3e97d96e6f
No known key found for this signature in database
GPG Key ID: B18E8928B3948D64

View File

@ -196,11 +196,11 @@ av_cold int swr_init(struct SwrContext *s){
clear_context(s);
if(s-> in_sample_fmt >= AV_SAMPLE_FMT_NB){
if((unsigned) s-> in_sample_fmt >= AV_SAMPLE_FMT_NB){
av_log(s, AV_LOG_ERROR, "Requested input sample format %d is invalid\n", s->in_sample_fmt);
return AVERROR(EINVAL);
}
if(s->out_sample_fmt >= AV_SAMPLE_FMT_NB){
if((unsigned) s->out_sample_fmt >= AV_SAMPLE_FMT_NB){
av_log(s, AV_LOG_ERROR, "Requested output sample format %d is invalid\n", s->out_sample_fmt);
return AVERROR(EINVAL);
}