Commit Graph

13922 Commits

Author SHA1 Message Date
Anton Khirnov
301761792a mpeg12: do not decode extradata more than once.
Fixes CVE-2012-2803.

(cherry picked from commit 5823686261)

Conflicts:

	libavcodec/mpeg12.c
	libavcodec/mpeg12.h
2013-02-02 09:54:16 +01:00
Anton Khirnov
440e98574b indeo4/5: check empty tile size in decode_mb_info().
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.

Based on a patch by Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2012-2800

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ae3da0ae55)

Conflicts:

	libavcodec/ivi_common.c
2013-02-02 09:54:16 +01:00
Anton Khirnov
604d72aa0d dfa: improve boundary checks in decode_dds1()
Fixes CVE-2012-2798

CC:libav-stable@libav.org
(cherry picked from commit d05f72c754)

Conflicts:

	libavcodec/dfa.c
2013-02-02 09:54:16 +01:00
Michael Niedermayer
03ddc26066 indeo5dec: Make sure we have had a valid gop header.
This prevents decoding happening on a half initialized context.

Fixes CVE-2012-2779

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 891918431d)

Conflicts:

	libavcodec/ivi_common.c
	libavcodec/ivi_common.h
2013-02-01 06:46:04 +01:00
Janne Grunau
801eff785a rv34: error out on size changes with frame threading
(cherry picked from commit cb7190cd2c)

Fixes: CVE-2012-2772 (according to Ubuntu)
2013-02-01 06:46:04 +01:00
Janne Grunau
dd0c5e0fa9 h264: check ref_count validity for num_ref_idx_active_override_flag
Fixes segfault in the fuzzed sample bipbop234.ts_s226407.
CC: libav-stable@libav.org
(cherry-picked from commit 6e5cdf2628)
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2013-01-12 19:36:38 +01:00
Janne Grunau
ad02537746 h264: check context state before decoding slice data partitions
Fixes mov_h264_aac__Demo_FlagOfOurFathers.mov.SIGSEGV.4e9.656.

Found-by: Mateusz "j00ru" Jurczyk
CC: libav-stable@libav.org
(cherry-picked from commit c1fcf563b1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:36:38 +01:00
Justin Ruggles
a39c6bf1b8 alacdec: do not be too strict about the extradata size
Sometimes the extradata has duplicate atoms, but that shouldn't prevent
decoding. Just ensure that it is at least 36 bytes as a sanity check.

CC: libav-stable@libav.org
(cherry picked from commit 68a04b0cce)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:34:10 +01:00
Victor Lopez
884a9b0d29 h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles
Fixes bug 396.

CC: libav-stable@libav.org
(cherry picked from commit 1c8bf3bfed)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:34:10 +01:00
Janne Grunau
4457e6137d h264: check sps.log2_max_frame_num for validity
Fixes infinite or long taking loop in frame num gap code in
the fuzzed sample bipbop234.ts_s223302.

CC: libav-stable@libav.org
(cherry picked from commit d7d6efe42b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:34:10 +01:00
Janne Grunau
5fa739e685 h264: enable low delay only if no delayed frames were seen
Dropping frames is undesirable but that is the only way by which the
decoder could return to low delay mode. Instead emit a warning and
continue with delayed frames.
Fixes a crash in fuzzed sample nasa-8s2.ts_s20033 caused by a larger
than expected has_b_frames value. Low delay keeps getting re-enabled
from a presumely broken SPS.

CC: libav-stable@libav.org
(cherry picked from commit 706acb558a)

Conflicts:

	libavcodec/h264.c
2013-01-12 19:32:24 +01:00
Alex Converse
b143844ea0 aacdec: Fix an off-by-one overwrite when switching to LTP profile from MAIN.
Found-by: pawlkt
CC: libav-stable@libav.org
Fixes: CVE-2012-5144
(cherry picked from commit 6d5b009267)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:27:42 +01:00
Luca Barbato
4ede95e69c vp6: properly fail on unsupported feature
Interlacing is not supported at all and mismanaged down the normal
codepaths causing possible buffer management issues.

Fixes: CVE-2012-2783
(cherry picked from commit be75fed975)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:27:29 +01:00
Reinhard Tartler
ce8910d861 h264: Fix parameters to ff_er_add_slice() call
s->mb_x is reset to zero a couple of lines above. It does not make
sense to call ff_er_add_slice() with 0 as endx when the end of the
macroblock row was reached. Fixes unnecessary and counterproductive
error resilience in https://bugzilla.libav.org/show_bug.cgi?id=394.

(cherry picked from commit e6160bda98)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:

	libavcodec/h264.c
2013-01-12 19:22:22 +01:00
Justin Ruggles
3d0c9c9af6 flacenc: ensure the order is within the min/max range in LPC order search
This fixes use of uninitialized values when the FLAC encoder uses the
2-level, 4-level, and 8-level search methods. Fixes failure of the
fate-flac-24-comp-8 test when run using valgrind.
(cherry picked from commit 3a2731cbd3)

Conflicts:

	libavcodec/flacenc.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Sami Pietila
bfbff1c748 vp8: reset loopfilter delta values at keyframes.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

(cherry picked from commit 0bf511d579)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Luca Barbato
7fd7950174 vp56: release frames on error
Fixes CVE-2012-2783

CC: libav-stable@libav.org

(cherry picked from commit f33b5ba63e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Luca Barbato
700fb8c8dd vp56: make parse_header return standard error codes
Returning 0 for failure is misleading.

CC: libav-stable@libav.org

(cherry picked from commit bb675d3ac6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Anton Khirnov
9f80712454 ivi_common: check that scan pattern is set before using it.
Fixes CVE-2012-2791.

CC: libav-stable@libav.org

(cherry picked from commit deabb52ab4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:20:27 +01:00
Alex Converse
549b8083d6 tiffenc: Check av_malloc() results.
(cherry picked from commit b92dfb56d4)

Conflicts:

	libavcodec/tiffenc.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Luca Barbato
ec6719f655 mpegaudiodec: fix short_start calculation
The value should be always 3, as it follows from the specification.

Fix a stack buffer overflow in exponents_from_scale_factors as reported
by asan. Thanks to Dale Curtis for the sample vector.
(cherry picked from commit 97cfa55eea)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Jindřich Makovička
11ecd8574a h264: avoid stuck buffer pointer in decode_nal_units
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org

Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 1a8c6917f6)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:38 +01:00
Franz Brauße
fb3189ce8b smacker audio: sign-extend the initial 16-bit predicted value
Fixes Bug #265

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 12cbbbb4ab)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:38 +01:00
Sean McGovern
dd14723602 wmapro: prevent division by zero when sample rate is unspecified
This fixes Bugzilla #327:

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit 3680b24351)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:38 +01:00
Thilo Borgmann
9474c93028 alsdec: fix number of decoded samples in first sub-block in BGMC mode.
Fixes CVE-2012-2790

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 66197988b1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Mans Rullgard
7e070cf202 alsdec: remove dead assignments
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 4ca6d206d1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Thilo Borgmann
1b48a426a9 alsdec: Fix out of ltp_gain_values read.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 97f0efbfb8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Michael Niedermayer
e3e369f696 alsdec: Check that quantized parcor coeffs are within range.
ALS spec:
	11.6.3.1.1 Quantization and encoding of parcor coefficients
	...
	In all cases the resulting quantized values ak are restricted to the range [-64,63].

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 5b051ec3bd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Michael Niedermayer
aa45b90804 alsdec: Check k used for rice decoder.
Values that fail this check will cause failure of decode_rice()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 23aae62c2c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:38 +01:00
Michael Niedermayer
6996a2f796 cavsdec: check for changing w/h.
Our decoder does not support changing w/h.

Fixes CVE-2012-2777 and CVE-2012-2784.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit c20a696306)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
4a636a5e43 wmaprodec: check num_vec_coeffs for validity
Fixes CVE-2012-2789

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 99f392a584)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
44da556815 lagarith: check count before writing zeros.
Fixes CVE-2012-2793

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b631e4ed64)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
aa097b4d5f indeo5: check tile size in decode_mb_info().
This prevents writing into a too small array if some parameters changed
without the tile being reallocated.

Fixes CVE-2012-2794

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 2d09cdbaf2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Janne Grunau
8148833193 indeo5: prevent null pointer dereference on broken files
Found by John Villamil <johnv@matasano.com>
(cherry picked from commit 366ac22ea5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
3c0f84402b indeo: check for invalid motion vectors
(cherry picked from commit cf61aaaca1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
601fa56582 indeo: clear allocated band buffers
(cherry picked from commit 23ba1503f2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
c0df6a24ce indeo: check custom Huffman tables for errors
(cherry picked from commit fe7a37c36f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Kostya Shishkov
2d63f9b4ef dfa: add some checks to ensure that decoder won't write past frame end
(cherry picked from commit 8099187e89)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Anton Khirnov
4c849c6991 dfa: check that the caller set width/height properly.
Fixes CVE-2012-2786.
(cherry picked from commit ee715f49a0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Aneesh Dogra
42c3a3719b bytestream: add a new set of bytestream functions with overread checking
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
2013-01-04 07:43:37 +01:00
Michael Niedermayer
7a0ff7566b avsdec: Set dimensions instead of relying on the demuxer.
The decode function assumes that the video will have those dimensions.

Fixes CVE-2012-2801

CC:libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 85f477935c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-04 07:43:37 +01:00
Janne Grunau
99008ba366 rv34: use AVERROR return values in ff_rv34_decode_frame()
Also adds an error message.
(cherry picked from commit 29330721b0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Martin Storsjö
a81c1ea2eb h263: Add ff_ prefix to nonstatic symbols
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit ddce8953a5)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Anton Khirnov
d3e2f35f7a bmpdec: only initialize palette for pal8.
Gray8 is not considered to be paletted, so this would cause an invalid
write.

Fixes bug 367.

CC: libav-stable@libav.org
(cherry picked from commit 8b78c2969a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:21 +01:00
Kostya Shishkov
e39fc137ae vc1dec: add flush function for WMV9 and VC-1 decoders
CC: libav-stable@libav.org
(cherry picked from commit 4dc8c8386e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-01-04 07:43:20 +01:00
Mans Rullgard
a2ae183a38 h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:20 +01:00
Alex Converse
d6e250abfc vorbis: Validate that the floor 1 X values contain no duplicates.
Duplicate values in this vector are explicitly banned by the Vorbis I spec
and cause divide-by-zero crashes later on.
(cherry picked from commit ecf79c4d3e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 9aaaeba45c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 09:40:46 +02:00
Justin Ruggles
61ece41372 vorbisenc: check all allocations for failure
(cherry picked from commit be8d812c96)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit e46cf805b1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 09:40:46 +02:00
Michael Niedermayer
b6ba39f931 alsdec: check opt_order.
Fixes out of array write in quant_cof.
Also make sure no invalid opt_order stays in the context.

Fixes CVE-2012-2775

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 9853e41aa0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit a1b127515b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 09:40:46 +02:00
Michael Niedermayer
899d95efe1 mpegvideo: Don't use ff_mspel_motion() for vc1
Using ff_mspel_motion assumes that s (a MpegEncContext
poiinter) really is a Wmv2Context.

This fixes crashes in error resilience on vc1/wmv3 videos.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 18f2d5cb9c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit da0c457663)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-10-06 09:40:46 +02:00