Commit Graph

1256 Commits

Author SHA1 Message Date
Michael Niedermayer b53adef07b h264: reset first_field when current_picture_ptr is reset
Fixes NULL pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-11 23:14:51 +01:00
Michael Niedermayer 1894302a44 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  cmdutils: update copyright year to 2013
  h264: check SPS entries directly to detect pixel format changes
  forgotten changelogs for 9_beta2

Conflicts:
	Changelog
	cmdutils.c
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-03 13:22:45 +01:00
Janne Grunau 9ac44ad9d0 h264: check SPS entries directly to detect pixel format changes
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.
2013-01-03 11:09:00 +01:00
Janne Grunau e9fd51b0d6 h264: check SPS entries directly to detect pixel format changes
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-02 02:28:57 +01:00
Michael Niedermayer d69238e991 Merge commit 'f3298f12997eb4b7ad203766f768f92e3dd72a2a'
* commit 'f3298f12997eb4b7ad203766f768f92e3dd72a2a':
  Return proper error code after av_log_ask_for_sample()
  configure: cosmetics: Separate hwaccel dependencies from decoders/encoders
  oggdec: check memory allocation

Conflicts:
	configure
	libavcodec/pictordec.c
	libavformat/anm.c
	libavformat/oggdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-24 15:09:58 +01:00
Diego Biurrun f3298f1299 Return proper error code after av_log_ask_for_sample() 2012-12-23 18:56:56 +01:00
Michael Niedermayer 985aa0be82 h264: Detect POC inconsistencies and try to handle them reasonably
Improves the file from Ticket2050

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-22 18:35:43 +01:00
Michael Niedermayer a41bf09d9c Merge commit '6906b19346ae8a330bfaa1c16ce535be10789723'
* commit '6906b19346ae8a330bfaa1c16ce535be10789723':
  lavc: add missing files for arm
  lavc: introduce VideoDSPContext

Conflicts:
	configure
	libavcodec/arm/dsputil_init_armv5te.c
	libavcodec/dsputil.c
	libavcodec/dsputil.h
	libavcodec/dsputil_template.c
	libavcodec/h264.c
	libavcodec/mpegvideo.h
	libavcodec/mpegvideo_enc.c
	libavcodec/x86/dsputil_mmx.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-21 17:18:43 +01:00
Ronald S. Bultje 8c53d39e7f lavc: introduce VideoDSPContext
Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2012-12-20 13:40:45 +01:00
Michael Niedermayer a9275b4f69 h264: Fix code to also handle pix format changes.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-20 12:56:47 +01:00
Michael Niedermayer 8525fa7c2c h264: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-20 04:22:21 +01:00
Michael Niedermayer ca4dd3810e h264: use must_reinit to simplify code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer af0fafcb6f h264: decode_update_thread_context() copy parameter sets before using them
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer e33811bd26 h264: remove reschange blocking
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer c2dfb1e37c h264: support frame size changes with multi threading
Based on code by Janne Grunau

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer 8ea4a5533f h264: move h264_set_parameter_from_sps() call up
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer afc03268b4 h264: move list_count and current_slice reset to flush_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer 971c469a0d h264: split flush_change() out of flush_dpb()
Based on a patch by Janne Grunau
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer 25a0af51da h264: factor get_pixel_format() out
Based on patch by Janne Grunau

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer 642a655f7d Merge commit 'f1d8763a02b5fce9a7d9789e049d74a45b15e1e8'
* commit 'f1d8763a02b5fce9a7d9789e049d74a45b15e1e8':
  mpegvideo: allocate scratch buffers after linesize is known

Conflicts:
	libavcodec/mpegvideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 15:44:59 +01:00
Janne Grunau 9e696d2e5f h264: support frame parameter changes during frame-mt
Fixes CVE-2012-2782.
2012-12-18 19:55:10 +01:00
Janne Grunau f1d8763a02 mpegvideo: allocate scratch buffers after linesize is known
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.

Fixes fuzzed sample nasa-8s2.ts_s244342.
2012-12-18 19:48:30 +01:00
Michael Niedermayer 14f79ba18f h264: remove redundant parts of old slice in extradata code.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-18 14:52:44 +01:00
Michael Niedermayer 99321d1b03 h264: merge old and new "slice in extradata" checks
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-18 14:50:12 +01:00
Michael Niedermayer 09b4ae8331 Merge commit '61c6eef5456f2bc8b1dc49a0a759c975551cea29'
* commit '61c6eef5456f2bc8b1dc49a0a759c975551cea29':
  h264: prevent decoding of slice NALs in extradata
  doxy: Clarify what avpriv_set_pts_info does

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-18 14:38:10 +01:00
Janne Grunau 61c6eef545 h264: prevent decoding of slice NALs in extradata
It is not posible to call get_buffer during frame-mt codec
initialization. Libavformat might pass huge amounts of data as
extradata after parsing broken files. The 'extradata' for the fuzzed
sample sample_varPAR_s5374_r001-02.avi is 2.8M large and contains
multiple slices.
2012-12-18 11:01:14 +01:00
Michael Niedermayer 7973a07590 h264: Improve first slice and slice type checks
This prevents a null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-17 01:08:19 +01:00
Michael Niedermayer dde4832b64 Merge commit '27c8337e595a058347150269d5c2c48281e4285b'
* commit '27c8337e595a058347150269d5c2c48281e4285b':
  h264-mt: handle NAL_DPAs before calling ff_thread_finish_setup
  lavr: move AudioMix struct definition to audio_mix.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-16 13:09:30 +01:00
Michael Niedermayer d7599bd8e2 h264: dont mess with frame gaps on second fields.
Fixes assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-16 00:28:36 +01:00
Janne Grunau 27c8337e59 h264-mt: handle NAL_DPAs before calling ff_thread_finish_setup
Since a NAL_DPA can start a new frame it has to be handled before
ff_thread_finish_setup is called.
2012-12-15 19:06:37 +01:00
Michael Niedermayer a01fe55077 Merge commit 'c0dc57f1264dad1e121772d03abdb9e14ed8857f'
* commit 'c0dc57f1264dad1e121772d03abdb9e14ed8857f':
  asyncts: merge two conditions
  x86inc: fully concatenate tokens to fix macro expansion for nasm
  h264: initialize frame-mt context copies properly

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:43:46 +01:00
Michael Niedermayer 3b5c0f5e36 h264: remove low_delay/has_b_frame setting code from nal loop
This code is now executed in h264_set_parameter_from_sps()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:22:19 +01:00
Michael Niedermayer b9d887c225 Merge commit '072be3e8969f24113d599444be4d6a0ed04a6602'
* commit '072be3e8969f24113d599444be4d6a0ed04a6602':
  h264: set parameters from SPS whenever it changes
  asyncts: cosmetics: reindent

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:17:51 +01:00
Janne Grunau 072be3e896 h264: set parameters from SPS whenever it changes
Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.
2012-12-13 21:02:42 +01:00
Janne Grunau 0eae920c3c h264: initialize frame-mt context copies properly 2012-12-13 21:02:42 +01:00
Michael Niedermayer c3bb3334f6 h264: dont try to allocate scratchpad if linesize is not known
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-10 20:52:14 +01:00
Michael Niedermayer 78ac7ee970 Merge commit '5d471b73d20616f5ac701ff62e5de49465cda264'
* commit '5d471b73d20616f5ac701ff62e5de49465cda264':
  rtpdec: K&R formatting and spelling cosmetics
  cosmetics: Fix dropable --> droppable typo

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-10 01:27:10 +01:00
Diego Biurrun ba0c898120 cosmetics: Fix dropable --> droppable typo 2012-12-09 13:36:11 +01:00
Michael Niedermayer e8ca7cfa4f h264: avoid calling idr() twice
Fixes rare race condition leading to null pointer dereferences.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-09 06:06:15 +01:00
Michael Niedermayer 857d7194ca Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: add a pointer for weighted prediction temporary buffer

Conflicts:
	libavcodec/h264.c
	libavcodec/h264.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-07 16:29:07 +01:00
Janne Grunau a394959bbe h264: add a pointer for weighted prediction temporary buffer
Reusing MpegEncContext's obmc_scratchpad for this becomes a mess with
adaptive frame-mt.
2012-12-07 11:43:28 +01:00
Janne Grunau 3ab5f7dc13 h264: slice-mt: check master context for valid current_picture_ptr
Fixes errors in slice based multithreading introduced in 0b300daad2.
2012-12-06 04:45:04 +01:00
Janne Grunau 24c62ea7a5 h264: slice-mt: get last_pic_dropable from master context
Fixes fate-h264-conformance-cvnlfi2_sony_h and smllwebdl.mkv from
https://github.com/OpenELEC/OpenELEC.tv/issues/1557 .
2012-12-06 04:44:57 +01:00
Janne Grunau 5945c7b35d h264: slice-mt: check master context for valid current_picture_ptr
Fixes errors in slice based multithreading introduced in 0b300daad2.

CC: libav-stable@libav.org
2012-12-05 23:16:37 +01:00
Janne Grunau a8cb1746c5 h264: slice-mt: get last_pic_dropable from master context
Fixes fate-h264-conformance-cvnlfi2_sony_h and smllwebdl.mkv from
https://github.com/OpenELEC/OpenELEC.tv/issues/1557 .

CC: libav-stable@libav.org
2012-12-05 23:16:37 +01:00
Michael Niedermayer fc1152de41 Merge commit 'df9b9567518f2840d79a4a96b447ebe1aa326408'
* commit 'df9b9567518f2840d79a4a96b447ebe1aa326408':
  lavc: fix decode_frame() third parameter semantics for video decoders

Conflicts:
	libavcodec/cscd.c
	libavcodec/eamad.c
	libavcodec/ffv1dec.c
	libavcodec/gifdec.c
	libavcodec/h264.c
	libavcodec/iff.c
	libavcodec/mjpegdec.c
	libavcodec/pcx.c
	libavcodec/vp56.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-05 17:20:19 +01:00
Anton Khirnov df9b956751 lavc: fix decode_frame() third parameter semantics for video decoders
It's got_frame, not data size
2012-12-04 21:45:36 +01:00
Michael Niedermayer 211372e86d Merge commit '0b300daad2f5cb59a7c06dde5ac701685e6edf16'
* commit '0b300daad2f5cb59a7c06dde5ac701685e6edf16':
  h264: error out on unset current_picture_ptr for h->current_slice > 0
  avprobe: report per stream bit rate if set by the decoder
  aac: avoid a memcpy in sbr_qmf_analysis

Conflicts:
	avprobe.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-03 01:56:47 +01:00
Janne Grunau 0b300daad2 h264: error out on unset current_picture_ptr for h->current_slice > 0
Fixes a segfault with fuzzed sample sample_varPAR_s11622_r001-02.avi.

CC: libav-stable@libav.org
2012-12-02 23:24:53 +01:00
Michael Niedermayer 936eaa89be h264: check for integer overflow, fix null pointer dereference
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-02 20:44:58 +01:00