Commit Graph

57885 Commits

Author SHA1 Message Date
Michael Niedermayer 95fa91d975 postproc: fix qp count
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0b7e5d0d75)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer a90ed80d05 postproc/postprocess: fix quant store for fq mode
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 941aaa39e8)

Conflicts:

	tests/ref/fate/filter-pp3
(cherry picked from commit 705748caf3f6a4a3e74ad3d2fc547a5a0213a521)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Christophe Gisquet 3fd2ff1b4f utvideoenc: properly set slice height/last line
Mimic decoder and obey sampling.

Does not affect fate tests for utvideo.
Fixes ticket #3949.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb530dda7d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer 11313263ab swresample/swresample: fix sample drop loop end condition
Fixes Ticket3985

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f9fefa499f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer f335275c50 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer ea61dfe0ab avformat/mpegts: Check desc_len / get8() return code
Fixes out of array read
Fixes: signal_sigsegv_844d59_10_signal_sigsegv_a17bb7_366_mpegts_mpeg2video_mp2_dvbsub_topfield.rec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c3d7f00ee3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer 21808e2181 avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:53:10 +01:00
Michael Niedermayer bf2605c356 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer c10c71452f avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer 05d7e92e4f avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer 96357894ff avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer 4841b2759f avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Conflicts:

	libavcodec/pngdec.c
2014-11-28 18:44:38 +01:00
Michael Niedermayer 5d852f338d avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer ceb9d67a0a avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer ce9d497755 avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer b5298c464f avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer 9ec550c364 avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Conflicts:

	libavcodec/mjpegdec.c
2014-11-28 18:44:38 +01:00
Michael Niedermayer 23a22b0da9 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Reimar Döffinger d61a325a68 configure: add noexecstack to linker options if supported.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit b7082d953f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Michael Niedermayer c658f6c34d avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
lvqcl 8dd6075a7f avutil/x86/cpu: fix cpuid sub-leaf selection
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e58fc44649)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:38 +01:00
Philip DeCamp 9c61b44941 libavutil/opt: fix av_opt_set_channel_layout() to access correct memory address
Signed-off-by: Philip DeCamp <decamp@mit.edu>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 857fc0a71f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Benoit Fouet dc319a52f9 avformat/riffenc: Filter out "BottomUp" in ff_put_bmp_header()
Fixes Ticket1304

Commit message and extradata size bugfix by commiter
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6843b9dc78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Pascal Massimino 77367f2728 avcodec/webp: fix default palette color 0xff000000 -> 0x00000000
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5b3112996)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Gianluigi Tiesi 34ef754854 avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
James Almer 84487650e2 x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Pascal Massimino 5ecc4a644a libavcodec/webp: treat out-of-bound palette index as translucent black
See https://code.google.com/p/webp/issues/detail?id=206
for a description of the problem/fix.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This patch makes the decoder follow the recommendation of the spec.
There is some disagreement (see "[FFmpeg-devel] [PATCH]: libavcodec/webp")
about what would be best to be written in the spec, so in case the spec
is changed again, this potentially would need to be amended or reverted
(cherry picked from commit 4fd21d58a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Katerina Barone-Adesi 36ec1c2c55 apetag: Fix APE tag size check
The size variable is (correctly) unsigned, but is passed to several functions
which take signed parameters, such as avio_read, sometimes after having
numbers added to it. So ensure that size remains within the bounds that
these functions can handle.

CC: libav-stable@libav.org
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c5560e72d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 5e4a821b8a tools/crypto_bench: fix build when AV_READ_TIME is unavailable
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a99134f1a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 5bd45a1b27 avformat/m4vdec: Check for non startcode 00 00 00 sequences in probe
Fixes miss detection of PCM as m4v
Fixes Ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c1835c52a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer cd190f0c3f avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 31f5d5f690 avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 7644b29222 avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer f7086be79a avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334ba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 72f0d13802 avformat/swfdec: Do not change the pixel format
This is currently not supported
Fixes part of Ticket 3539

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2430304df)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 255ebf3aff avformat/swfdec: Use side data to communicate w/h changes to the decoder
Fixes reading from freed data
Fixes part of Ticket3539

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1c55d0ff32)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a9734e7d3017ffc9539eaac2a8acce3ad427f746)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Mika Raento cb10e05ff0 segment: don't access outside seg->frames array
Fixes wrong number of segments output and undefined memory access.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58e0402e02)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
wm4 17f30ab6a1 oggdec: fix invalid free on error
The read_packet callback passes a pointer to a stack-allocated AVPacket.
Attempting to free it with av_free() makes no sense.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b173f5c155)

Conflicts:

	libavformat/oggdec.c
(cherry picked from commit a82401b7284e05e2cf6fb89aeed53738af06a7fd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 8c8950f982 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Timothy Gu 0f8863df86 bktr: Fix Fabrice's name
Signed-off-by: Timothy Gu <timothygu99@gmail.com>

This file with the incorrect name was added after the name was fixed in all other files.
This is thus fixing a mistake

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 25cb697d0c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer cb8f645faf avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer 6ab793c2b9 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet 80b6632b36 wavpackenc: proper buffer allocation
The allocation didn't account for headers, that can be easily 79 bytes.
As a result, buffers allocated for a few samples (e.g. 5 in the original
bug) could be undersized.

Fixed ticket #2881.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2ba58bec20)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet 656bf0ca79 wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf53)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet 407982b8f9 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet 92096acc0a proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Christophe Gisquet e1ed566c1c proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bcc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer c9da441eb4 avcodec/snow: fix null pointer dereference in cleanup after allocation failure
Fixes: snowf.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a162146ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Michael Niedermayer f405267493 avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00
Anton Khirnov f7ed48938a cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 18:44:37 +01:00