mirror of
https://git.ffmpeg.org/ffmpeg.git
synced 2024-12-28 18:32:22 +00:00
lavu/tx: clip when converting table values to fixed-point
INT32_MAX (2147483647) isn't exactly representable by a floating point value, with the closest being 2147483648.0. So when rescaling a value of 1.0, this could overflow when casting the 64-bit value returned from lrintf() into 32 bits. Unfortunately the properties of integer overflows don't match up well with how a Fourier Transform operates. So clip the value before casting to a 32-bit int. Should be noted we don't have overflows with the table values we're currently using. However, converting a Kaiser-Bessel window function with a length of 256 and a parameter of 5.0 to fixed point did create overflows. So this is more of insurance to save debugging time in case something changes in the future. The macro is only used during init, so it being a little slower is not a problem.
This commit is contained in:
parent
29993b2947
commit
91e1625db1
@ -85,7 +85,7 @@ typedef void FFTComplex;
|
||||
(dim) = (int)(((accu) + 0x40000000) >> 31); \
|
||||
} while (0)
|
||||
|
||||
#define RESCALE(x) (lrintf((x) * 2147483648.0))
|
||||
#define RESCALE(x) (av_clip64(lrintf((x) * 2147483648.0), INT32_MIN, INT32_MAX))
|
||||
|
||||
#define FOLD(x, y) ((int)((x) + (unsigned)(y) + 32) >> 6)
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user