Commit Graph

993 Commits

Author SHA1 Message Date
Mans Rullgard 0b711ca3f3 dsputil: drop non-compliant "fast" qpel mc functions
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2013-01-20 14:50:42 +01:00
Anton Khirnov 940b8b5861 h264: avoid pointless copying of ref lists
ref_list is constructed from other fields per slice when needed, so do
not copy it for both frame and slice threading.
default_ref_list is constructed per frame and still needs to be copied
to per-slice contexts for slice threading, but a copy is not needed for
frame threading.
2013-01-18 07:56:05 +01:00
Ronald S. Bultje fb845ffdd3 h264: add 3 pixels below for subpixel filter wait position
If the motion vector is at a subpixel position, we need 3 pixels below
the motion vector's wholepel position available, not 2, since the MC
filter is a sixtap filter for the hpel position, and then a bilin filter
for the qpel position.

This patch fixes highly irreproducible (0.1%) fate failures in frame 2
and 4 of h264-conformance-cama2_vtc_b (e.g. first P-frame, first field,
last line of MB x=40,y=2 and second field and last lines of MBs x=39-40,
y=3). These used pre-loopfilter instead of post-loopfilter data because
the await_progress() waited for one line too little in that field, and
the motion vector of these particular MBs happened to align exactly to a
position where that demonstrates the bug.

CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:00:34 +01:00
Anton Khirnov ea382767ad h264: fix ff_generate_sliding_window_mmcos() prototype.
It's been returning an error value since
bad446e251

Also check for the errors it returns.
2013-01-14 21:36:08 +01:00
Ronald S. Bultje bad446e251 h264: don't clobber mmco opcode tables for non-first slice headers.
Clobbering these tables will temporarily clobber the template used
as a basis for other threads to start decoding from. If the other
decoding thread updates from the template right at that moment,
subsequent threads will get invalid (or, usually, none at all) mmco
tables. This leads to invalid reference lists and subsequent decode
failures.

Therefore, instead, decode the mmco tables only for the first slice in
a field or frame. For other slices, decode the bits and ensure they
are identical to the mmco tables in the first slice, but don't ever
clobber the context state. This prevents other threads from using a
clobbered/invalid template as starting point for decoding, and thus
fixes decoding in these cases.

This fixes occasional (~1%) failures of h264-conformance-mr1_bt_a with
frame-multithreading enabled.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-14 19:20:47 +01:00
Michael Niedermayer 59d5680310 h264: Fix assignments in if()
Fixes null pointer dereference later, since if this function failed,
a positive return value was returned to the caller.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Martin Storsjö <martin@martin.st>
2013-01-14 13:12:38 +02:00
Janne Grunau 9ac44ad9d0 h264: check SPS entries directly to detect pixel format changes
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.
2013-01-03 11:09:00 +01:00
Diego Biurrun f3298f1299 Return proper error code after av_log_ask_for_sample() 2012-12-23 18:56:56 +01:00
Ronald S. Bultje 8c53d39e7f lavc: introduce VideoDSPContext
Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2012-12-20 13:40:45 +01:00
Janne Grunau 9e696d2e5f h264: support frame parameter changes during frame-mt
Fixes CVE-2012-2782.
2012-12-18 19:55:10 +01:00
Janne Grunau f1d8763a02 mpegvideo: allocate scratch buffers after linesize is known
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.

Fixes fuzzed sample nasa-8s2.ts_s244342.
2012-12-18 19:48:30 +01:00
Janne Grunau 61c6eef545 h264: prevent decoding of slice NALs in extradata
It is not posible to call get_buffer during frame-mt codec
initialization. Libavformat might pass huge amounts of data as
extradata after parsing broken files. The 'extradata' for the fuzzed
sample sample_varPAR_s5374_r001-02.avi is 2.8M large and contains
multiple slices.
2012-12-18 11:01:14 +01:00
Janne Grunau 27c8337e59 h264-mt: handle NAL_DPAs before calling ff_thread_finish_setup
Since a NAL_DPA can start a new frame it has to be handled before
ff_thread_finish_setup is called.
2012-12-15 19:06:37 +01:00
Janne Grunau 0eae920c3c h264: initialize frame-mt context copies properly 2012-12-13 21:02:42 +01:00
Janne Grunau 072be3e896 h264: set parameters from SPS whenever it changes
Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.
2012-12-13 21:02:42 +01:00
Diego Biurrun ba0c898120 cosmetics: Fix dropable --> droppable typo 2012-12-09 13:36:11 +01:00
Janne Grunau a394959bbe h264: add a pointer for weighted prediction temporary buffer
Reusing MpegEncContext's obmc_scratchpad for this becomes a mess with
adaptive frame-mt.
2012-12-07 11:43:28 +01:00
Janne Grunau 5945c7b35d h264: slice-mt: check master context for valid current_picture_ptr
Fixes errors in slice based multithreading introduced in 0b300daad2.

CC: libav-stable@libav.org
2012-12-05 23:16:37 +01:00
Janne Grunau a8cb1746c5 h264: slice-mt: get last_pic_dropable from master context
Fixes fate-h264-conformance-cvnlfi2_sony_h and smllwebdl.mkv from
https://github.com/OpenELEC/OpenELEC.tv/issues/1557 .

CC: libav-stable@libav.org
2012-12-05 23:16:37 +01:00
Anton Khirnov df9b956751 lavc: fix decode_frame() third parameter semantics for video decoders
It's got_frame, not data size
2012-12-04 21:45:36 +01:00
Janne Grunau 0b300daad2 h264: error out on unset current_picture_ptr for h->current_slice > 0
Fixes a segfault with fuzzed sample sample_varPAR_s11622_r001-02.avi.

CC: libav-stable@libav.org
2012-12-02 23:24:53 +01:00
Janne Grunau c1fcf563b1 h264: check context state before decoding slice data partitions
Fixes mov_h264_aac__Demo_FlagOfOurFathers.mov.SIGSEGV.4e9.656.

Found-by: Mateusz "j00ru" Jurczyk
CC: libav-stable@libav.org
2012-11-29 14:40:05 +01:00
Janne Grunau d5e83122bd h264: set Picture.owner2 to the current thread
This does not seem to have an effect currently. Fate-h264 passes with
THREADS=1..16 and both threading types as before. It fixes however a
segfault during error resilience with my adaptive-frame-mt patchset.
A picture in use during error resilience gets realloced in another
thread in the fuzzed sample sample_varPAR.avi_s226019.
2012-11-27 12:26:33 +01:00
Janne Grunau 6e5cdf2628 h264: check ref_count validity for num_ref_idx_active_override_flag
Fixes segfault in the fuzzed sample bipbop234.ts_s226407.

CC: libav-stable@libav.org
2012-11-27 12:26:33 +01:00
Janne Grunau 150b2361ca h264: add missing new line to log message 2012-11-27 12:26:33 +01:00
Janne Grunau 706acb558a h264: enable low delay only if no delayed frames were seen
Dropping frames is undesirable but that is the only way by which the
decoder could return to low delay mode. Instead emit a warning and
continue with delayed frames.
Fixes a crash in fuzzed sample nasa-8s2.ts_s20033 caused by a larger
than expected has_b_frames value. Low delay keeps getting re-enabled
from a presumely broken SPS.

CC: libav-stable@libav.org
2012-11-26 10:25:39 +01:00
Janne Grunau e6160bda98 h264: Fix parameters to ff_er_add_slice() call
s->mb_x is reset to zero a couple of lines above. It does not make
sense to call ff_er_add_slice() with 0 as endx when the end of the
macroblock row was reached. Fixes unnecessary and counterproductive
error resilience in https://bugzilla.libav.org/show_bug.cgi?id=394.

CC: libav-stable@libav.org
2012-11-16 13:18:28 +01:00
Janne Grunau 60b6b8c019 h264: always check ref_count for validity
Fixes a crash with zuffed files.
2012-11-16 13:18:28 +01:00
Diego Biurrun f75f4194d1 Restructure av_log_missing_feature message
Some invocations include a verb in the log message, others do not.  Yet
av_log_missing_feature expects callers to provide a verb.  Change the
function to include a verb instead and update the callers accordingly.
The result is a more natural function API and correct English in the
function invocations.
2012-10-09 11:39:26 +02:00
Anton Khirnov 716d413c13 Replace PIX_FMT_* -> AV_PIX_FMT_*, PixelFormat -> AVPixelFormat 2012-10-08 07:13:26 +02:00
Diego Biurrun 9c6cf7f2c9 avcodec: Drop silly and/or broken printf debug output 2012-10-01 10:24:28 +02:00
Jindřich Makovička 1a8c6917f6 h264: avoid stuck buffer pointer in decode_nal_units
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org

Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-09-29 19:31:17 +02:00
Michael Niedermayer 5e997688f8 h264: Remove an assert on current_picture_ptr being null
It is possible in various error paths as well as gap handling
that this has already been allocated. It is not clear why that
would be a problem with the current code, thus disable the
assert to avoid a common assert failure when asserts are enabled.

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-09-05 14:28:41 +03:00
Martin Storsjö 6f5b1a2ba4 h264: Check that the codec isn't null before accessing it
This fixes crashes introduced by 2e8f3cbcda, the codec can be null
when called from parsers.

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-08-28 18:15:38 +03:00
Uoti Urpala 2e8f3cbcda h264: vdpau: fix crash with unsupported colorspace
The h264_vdpau decoder crashed if output colorspace was not 8-bit 420.
Add a check to error out instead (current hardware does not support
other colorspaces, so successful decoding is not possible).

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-08-17 20:08:53 +03:00
Anton Khirnov 36ef5369ee Replace all CODEC_ID_* with AV_CODEC_ID_* 2012-08-07 16:00:24 +02:00
Ronald S. Bultje 58db34aa1e h264: refactor NAL decode loop
Write out the NAL decoding loops in full so that they are easier
to parse for a preprocessor without it having to be aware of macros
or other such things in C code.

This also makes the code more readable.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2012-07-23 16:27:53 +02:00
Mans Rullgard 28fff0d974 h264: use templates to avoid excessive inlining
Instead of inlining everything into ff_h264_hl_decode_mb(), use
explicit templating to create versions of the called functions
with constant parameters filled in.  This greatly speeds up
compilation of h264.c and reduces the code size without any
measurable impact on performance.

Compilation time for h264.c on an i7 goes from 30s to 5.5s.
Code size is reduced by 430kB.

Signed-off-by: Mans Rullgard <mans@mansr.com>
2012-07-05 11:50:18 +01:00
Diego Biurrun 09f211987c misc typo and wording fixes 2012-07-03 17:35:11 +02:00
Mans Rullgard 30f515091c h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
2012-06-09 15:08:52 +01:00
Ronald S. Bultje 732f9fcfe5 h264: additional protection against unsupported size/bitdepth changes.
Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-05-02 10:19:37 -07:00
Diego Biurrun 0becb07842 h264: Factorize declaration of mb_sizes array. 2012-04-05 17:17:22 +02:00
Anton Mitrofanov 14af74e9a0 h264: fix 4:2:2 PCM-macroblocks decoding
Fixes bug 239.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-04-04 19:34:39 +02:00
Diego Biurrun 3dde147ff9 cosmetics: Consistently place static, inline and av_cold attributes/keywords. 2012-04-04 14:54:13 +02:00
Diego Biurrun b691fd7a4d h264: drop ff_h264_ prefix from static function ff_h264_decode_rbsp_trailing() 2012-03-30 17:47:12 +02:00
Diego Biurrun 9ad80ef3db h264: Make ff_h264_decode_end() static, it is not used externally.
Also drop the now unnecessary ff_ prefix from its name.
2012-03-30 17:46:52 +02:00
Ronald S. Bultje 1e26a48fa2 h264: fix deadlocks on incomplete reference frame decoding.
If decoding a second complementary field, and the first was
decoded in our thread, mark decoding of that field as complete.
If decoding fails, mark the decoded field/frame as complete.
Do not allow switching between field modes or field/frame mode
between slices within the same field/frame. Ensure that two
subsequent fields cover top/bottom (rather than top/frame,
bottom/frame or such nonsense situations).

Fixes various deadlocks when decoding samples with errors in
reference frames.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-16 15:31:00 -07:00
Diego Biurrun e5d403720e h264: K&R formatting cosmetics
Also remove some disabled code and fix a few comment typos.
2012-03-16 20:35:37 +01:00
Ronald S. Bultje e0febda22d h264: stricter reference limit enforcement.
Progressive images can have only 16 references, error out if there are
more, since the data is almost certainly corrupt, and the invalid value
will lead to random crashes or invalid writes later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-14 13:24:49 -07:00
Michael Niedermayer 3aa661ec56 h264: improve parsing of broken AVC SPS
Parsing the entire NAL as SPS fixes decoding of some AVC bitstreams
with broken escaping. Since the size of the NAL unit is known and
checked against the buffer end we can parse it entirely without buffer
overreads.

Fixes playback of
http://streams.videolan.org/streams/mp4/Mr_MrsSmith-h264_aac.mp4

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2012-03-13 00:31:52 +01:00