Commit Graph

292 Commits

Author SHA1 Message Date
Michael Niedermayer 2140858524 avcodec/hevc: Fix integer overflow of entry_point_offset
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 22:34:30 +01:00
Clément Bœsch fb99ef0bd3 avcodec: use AV_OPT_TYPE_BOOL in a bunch of places 2015-12-04 15:37:05 +01:00
Michael Niedermayer dad354f38d avcodec/hevc: Check max ctb addresses for WPP
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 15:48:55 +01:00
Michael Niedermayer d85aa76115 avcodec/hevc: allocate entries unconditionally
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Michael Niedermayer ef9f7bbfa4 avcodec/hevc: Check entry_point_offsets
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 18:53:21 +01:00
Will Kelleher b1a32429ef hevc: Fix a53 caption extraction
Just realized my previous patch doesn't work quite right.  I uploaded a better
sample file that actually has visible captions to /incoming/hevc_cc.ts.  I
tested with that file doing hevc->x264 and it works.

This is basically an exact copy of the existing h264 logic.

Signed-off-by: Will Kelleher <wkelleher@gogoair.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 13:16:25 +01:00
Will Kelleher 7f7fa90f7b hevc: extract SEI caption data
Signed-off-by: Will Kelleher <wkelleher@gogoair.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-07 12:10:55 +01:00
Ganesh Ajjanagadde 4c96985af1 all: remove some casts of function pointer to void *
These casts are unnecessary, and may safely be removed.
Found by enabling -Wpedantic on clang 3.7.

Tested with FATE.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-10-24 15:14:59 -04:00
Anton Khirnov 8a73b8c5b4 cabac: Make cabac starts hardcoded
There's not much reason to generate such a small table at runtime.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2015-10-10 09:52:18 +02:00
Ronald S. Bultje 74e4948235 hevc: fix wpp threading deadlock.
Fixes ticket 4258.
2015-09-21 20:44:14 -04:00
Hendrik Leppkes 0118158efa hevc: properly handle no_rasl_output_flag when removing pictures from the DPB
Fixes ticket #4185.

Reviewed-By: Mickael Raulet <Mickael.Raulet@insa-rennes.fr>
Signed-off-by: Hendrik Leppkes <h.leppkes@gmail.com>
2015-09-20 00:17:32 +02:00
Hendrik Leppkes 151aa2ebff Merge commit '2268db2cd052674fde55c7d48b7a5098ce89b4ba'
* commit '2268db2cd052674fde55c7d48b7a5098ce89b4ba':
  lavu: Drop the {minus,plus}1 suffix from AVComponentDescriptor fields

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2015-09-08 16:35:28 +02:00
Vittorio Giovara 2268db2cd0 lavu: Drop the {minus,plus}1 suffix from AVComponentDescriptor fields
The new fields can be accessed directly and are more intelligible.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-09-07 12:37:47 +02:00
Timo Rothenpieler 1dd854e10f vaapi: Add hevc hwaccel support
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2015-08-27 17:05:44 +02:00
Hendrik Leppkes 001ea567ff Merge commit 'a1926a29fb4325afa46842883f197c74d4535c36'
* commit 'a1926a29fb4325afa46842883f197c74d4535c36':
  hevc: avoid invalid shifts of negative values

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2015-08-24 10:29:26 +02:00
Anton Khirnov d8ebb6157d hevcdsp: fix a function name
put_weighted_pred_avg should be put_unweighted_pred_avg, there is no
weighting there.
2015-08-21 08:46:05 +02:00
Anton Khirnov a1926a29fb hevc: avoid invalid shifts of negative values 2015-08-21 08:45:37 +02:00
wm4 94c0df79c7 lavc: propagate hwaccel errors
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.

Fix this by propagating end_frame errors.
2015-08-06 11:05:02 +02:00
Michael Niedermayer 444e9874a7 Merge commit 'def97856de6021965db86c25a732d78689bd6bb0'
* commit 'def97856de6021965db86c25a732d78689bd6bb0':
  lavc: AV-prefix all codec capabilities

Conflicts:
	cmdutils.c
	ffmpeg.c
	ffplay.c
	libavcodec/8svx.c
	libavcodec/aacenc.c
	libavcodec/ac3dec.c
	libavcodec/adpcm.c
	libavcodec/alac.c
	libavcodec/atrac3plusdec.c
	libavcodec/bink.c
	libavcodec/dnxhddec.c
	libavcodec/dvdec.c
	libavcodec/dvenc.c
	libavcodec/ffv1dec.c
	libavcodec/ffv1enc.c
	libavcodec/fic.c
	libavcodec/flacdec.c
	libavcodec/flacenc.c
	libavcodec/flvdec.c
	libavcodec/fraps.c
	libavcodec/frwu.c
	libavcodec/gifdec.c
	libavcodec/h261dec.c
	libavcodec/hevc.c
	libavcodec/iff.c
	libavcodec/imc.c
	libavcodec/libopenjpegdec.c
	libavcodec/libvo-aacenc.c
	libavcodec/libvorbisenc.c
	libavcodec/libvpxdec.c
	libavcodec/libvpxenc.c
	libavcodec/libx264.c
	libavcodec/mjpegbdec.c
	libavcodec/mjpegdec.c
	libavcodec/mpegaudiodec_float.c
	libavcodec/msmpeg4dec.c
	libavcodec/mxpegdec.c
	libavcodec/nvenc_h264.c
	libavcodec/nvenc_hevc.c
	libavcodec/pngdec.c
	libavcodec/qpeg.c
	libavcodec/ra288.c
	libavcodec/rv10.c
	libavcodec/s302m.c
	libavcodec/sp5xdec.c
	libavcodec/takdec.c
	libavcodec/tiff.c
	libavcodec/tta.c
	libavcodec/utils.c
	libavcodec/v210dec.c
	libavcodec/vp6.c
	libavcodec/vp9.c
	libavcodec/wavpack.c
	libavcodec/yop.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-27 22:50:18 +02:00
Vittorio Giovara def97856de lavc: AV-prefix all codec capabilities
Express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-07-27 15:24:58 +01:00
Michael Niedermayer afa97144cf Merge commit '077b55943330150db0eafd36bbee614697cabd98'
* commit '077b55943330150db0eafd36bbee614697cabd98':
  hevc: handle a NULL sps in set_sps() properly

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-13 16:41:09 +02:00
Michael Niedermayer d13fc98247 Merge commit '18156b53f9b642b71c182c5c9818175a61572d2b'
* commit '18156b53f9b642b71c182c5c9818175a61572d2b':
  hevc: do not pass an entire HEVCContext into export_stream_params()

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-13 16:31:10 +02:00
Anton Khirnov a062a55d37 hevc_parser: fix standalone build with the hevc decoder disabled
The parser depends on hevc_ps, which in turn needs some data tables.

Found-by: James Almer <jamrial@gmail.com>
2015-07-13 08:58:21 +02:00
Michael Niedermayer 99558270ed avcodec/hevc: Simplify skipped_bytes_pos code further
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-13 01:09:21 +02:00
Michael Niedermayer ad92410d90 avcodec/hevc: Move skipped_bytes_pos_nal to HEVCNAL, simplify code
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-13 00:43:25 +02:00
Michael Niedermayer bcc6c7bb65 avcodec/hevc: Move skipped_bytes_pos_size_nal into HAVCNAL
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-13 00:38:15 +02:00
Michael Niedermayer 5620ed3557 avcodec/hevc: Remove skipped_bytes_nal, simplify code
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-13 00:03:45 +02:00
Michael Niedermayer 93aa84c1ea Merge commit '7f78155dc6b65be55efb5309b6dd2bb33925a8c4'
* commit '7f78155dc6b65be55efb5309b6dd2bb33925a8c4':
  hevc: improve a comment

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-12 23:45:56 +02:00
Michael Niedermayer 4690a63632 Merge commit 'd82e1adc2019135a2fc45372e0ed0b5ef107cdd0'
* commit 'd82e1adc2019135a2fc45372e0ed0b5ef107cdd0':
  hevc: move splitting the packet into NALUs into a separate function

Conflicts:
	libavcodec/hevc.c
	libavcodec/hevc.h
	libavcodec/hevc_parse.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-12 23:44:24 +02:00
Michael Niedermayer 4c42c66935 Merge commit 'ae05b4865514fd71b5e9431e93aa0d03d7ba7751'
* commit 'ae05b4865514fd71b5e9431e93aa0d03d7ba7751':
  hevc: eliminate the second call to hls_nal_unit()

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-12 22:55:47 +02:00
Michael Niedermayer 760304e4c3 Merge commit 'd7bebe4805193783f0b6f292f9127a75709fb7d9'
* commit 'd7bebe4805193783f0b6f292f9127a75709fb7d9':
  hevc: skip invalid/ignored NALUs when splitting the packet

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-12 22:46:06 +02:00
Michael Niedermayer d5fcca83b9 Merge commit 'b11acd57326db6c2cc1475dd0bea2a06fbc85aa2'
* commit 'b11acd57326db6c2cc1475dd0bea2a06fbc85aa2':
  hevc: remove HEVCContext usage from hevc_ps

Conflicts:
	libavcodec/hevc.c
	libavcodec/hevc_cabac.c
	libavcodec/hevc_filter.c
	libavcodec/hevc_mvs.c
	libavcodec/hevc_ps.c
	libavcodec/hevc_refs.c
	libavcodec/hevcpred_template.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-12 22:28:54 +02:00
Anton Khirnov 077b559433 hevc: handle a NULL sps in set_sps() properly
This can happen in update_thread_context(), when the previous frame was
corrupted.
2015-07-12 18:57:10 +02:00
Anton Khirnov 18156b53f9 hevc: do not pass an entire HEVCContext into export_stream_params()
It only needs the parameter sets.
2015-07-12 18:52:02 +02:00
Anton Khirnov 7f78155dc6 hevc: improve a comment
That loop does the actual full decoding, so 'parse' can be misleading.
2015-07-12 18:15:40 +02:00
Anton Khirnov d82e1adc20 hevc: move splitting the packet into NALUs into a separate function
This function is independent of the decoding context, so we'll be able
to use it in the parser.
2015-07-12 18:15:39 +02:00
Anton Khirnov ae05b48655 hevc: eliminate the second call to hls_nal_unit()
Also, make hls_nal_unit() work only on the provided NAL unit, without
requiring a whole decoding context.

This will allow splitting this code for reuse by the parser.
2015-07-12 18:15:39 +02:00
Anton Khirnov d7bebe4805 hevc: skip invalid/ignored NALUs when splitting the packet
There is no need to wait until actually decoding the NALU. This will
allow to get rid of the second hls_nal_unit() call later.
2015-07-12 18:15:39 +02:00
Anton Khirnov b11acd5732 hevc: remove HEVCContext usage from hevc_ps
Factor out the parameter sets into a separate struct and use it instead.

This will allow us to reuse this code in the parser.
2015-07-12 18:15:39 +02:00
Andreas Cadhalpun 9f4c7397a2 hevc: check slice address length
It is used as get_bits argument and reading 0 bits doesn't make sense.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2015-07-12 18:15:39 +02:00
Andreas Cadhalpun 05cc8c8e4b hevc: check slice address length
It is used as get_bits argument and reading 0 bits isn't supported.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-10 21:28:56 +02:00
Michael Niedermayer 07ae8fa20e Merge commit '69ab9f53f901eac6a649e22d28cf093357870627'
* commit '69ab9f53f901eac6a649e22d28cf093357870627':
  hevc: split bitstream unescaping to a separate file

Conflicts:
	libavcodec/Makefile
	libavcodec/hevc.c

See: afa93d198a
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-09 05:06:48 +02:00
Michael Niedermayer 1d4194e696 Merge commit 'fd124d8357b1becfde3ac8d5e3320127cf97a5b7'
* commit 'fd124d8357b1becfde3ac8d5e3320127cf97a5b7':
  hevc_ps: split the code for parsing the SPS and exporting it into the context

Conflicts:
	libavcodec/hevc.c
	libavcodec/hevc_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-09 04:50:00 +02:00
Anton Khirnov 69ab9f53f9 hevc: split bitstream unescaping to a separate file
It will be useful in the QSV HEVC encoder.
2015-07-08 23:38:32 +02:00
Anton Khirnov fd124d8357 hevc_ps: split the code for parsing the SPS and exporting it into the context
This will be useful in the later commits, where we want to parse an SPS
without having a whole decoding context.
2015-07-08 23:36:22 +02:00
Philip Langdale 6e5e139fe3 avcodec/vdpau: Support for VDPAU accelerated HEVC decoding
This change introduces basic support for HEVC decoding through vdpau.
Right now, there are problems with the nvidia driver/library implementation
that mean that frames are incorrectly laid out in memory when they are
returned from the decoder, and it is normally impossible to recover the
complete decoded frame due to loss of data from alignment inconsistencies.

I obviously hope that nvidia will be fixing it in due course - I've verified
the problems exist with their example application.

As such, this support is not useful for any real world application, but I
believe that it is correct (with the caveat that the mangled frames may hide
problems) and will work properly once the nvidia problem is fixed.

Right now it appears that any file encoded by x265 or nvenc is decoded
correctly, but that's because these files don't use a bunch of HEVC
features.

Quick summary:

Features that seem to work:

1) Short Term References
2) Scaling Lists
3) Tiling

Features with known problems:

1) Long Term References

It's hard to tell what's going on here. After I read the nvidia example
app that does not set the IsLongTerm flag on LTRs, and changed my code,
a bunch of frames using LTR started to display correctly, but there
are still samples with glitches that are related to LTRs.

In terms of real world files, both x265 and nvenc only use short term
refs from this list. The divx encoder seems similar.

Signed-off-by: Philip Langdale <philipl@overt.org>
2015-06-25 20:06:47 -07:00
Philip Langdale 1b19d0c632 avcodec/hevc: Track long and short term RPS size for VDPAU
Today, we track the short term RPS size for DXVA, but only if the
SliceHeader RPS is being used. Otherwise it's left uninitialized.

NVIDIA's VDPAU implementation requires that the size be accurately
tracked even if an SPS RPS is being used. In this case, it's really
counting the size of the RPS idx information, but you end up with
mangled output if the value is not accurate.

VDPAU also needs the size of the long term RPS.

Signed-off-by: Philip Langdale <philipl@overt.org>
2015-06-13 21:09:24 -07:00
Ronald S. Bultje ade5684cda hevc: fix typo (mpv -> mvp).
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-06 12:57:20 +02:00
Michael Niedermayer 674b79148f hevc: Fix HWACCEL_MAX for D3D11
Signed-off-by: Martin Storsjö <martin@martin.st>
2015-06-01 11:36:22 +03:00
Michael Niedermayer 688147cfe2 avcodec/hevc: Fix HWACCEL_MAX for D3D11
Found-by: philipl
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-26 05:23:29 +02:00