avcodec/jpeg2000htdec: Consolidate jpeg2000 spec bits in jpeg2000_bitbuf_refill_backwards()

Code should make more sense now

Fixes: out of array access
Fixes: 58299/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6627570448465920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Pierre-Anthony Lemieux <pal@sandflow.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2023-08-02 02:01:34 +02:00
parent dcd0c79f7e
commit 9549712056
No known key found for this signature in database
GPG Key ID: B18E8928B3948D64
1 changed files with 4 additions and 20 deletions

View File

@ -144,36 +144,20 @@ static void jpeg2000_init_mel_decoder(MelDecoderState *mel_state)
static int jpeg2000_bitbuf_refill_backwards(StateVars *buffer, const uint8_t *array)
{
uint64_t tmp = 0;
int32_t position = buffer->pos;
int32_t position = buffer->pos - 4;
uint32_t new_bits = 32;
if (buffer->bits_left >= 32)
return 0; // enough data, no need to pull in more bits
/**
* We are reading bytes from end to start and need to handle being close to
* the end. Subtracting by 4 means we will read some of the bytes of the MEL
* byte stream since the MEL byte stream ends at the start of the VLC byte
* stream. This is okay as they are masked away since we check for cases
* where that occurs (when the position is less than 4).
*/
position -= 4;
tmp = AV_RB32(&array[position + 1]);
if (buffer->pos < 4){
/* mask un-needed bits if we are close to input end */
uint64_t mask = (1ull << (buffer->pos + 1) * 8) - 1;
tmp &= mask;
}
/**
* Unstuff bits. Load a temporary byte, which precedes the position we
* currently at, to ensure that we can also un-stuff if the stuffed bit is
* the bottom most bits.
*/
tmp <<= 8;
tmp |= array[buffer->pos + 1];
for(int i = FFMAX(0, position + 1); i <= buffer->pos + 1; i++)
tmp = 256*tmp + array[i];
if ((tmp & 0x7FFF000000) > 0x7F8F000000) {
tmp &= 0x7FFFFFFFFF;