Commit Graph

94933 Commits

Author SHA1 Message Date
Andreas Rheinhardt
5daaa64d8a avformat/apngenc: Add deinit function
Prevents memleaks when the trailer is never written (e.g. when there was
a write error when writing the header).

Fixes ticket #8347.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 26aa04991e)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 21:31:59 +02:00
Andreas Rheinhardt
07eb039e04 avcodec/hevc_mp4toannexb_bsf: Check NAL size against available input
The hevc_mp4toannexb bsf does not explicitly check whether a NAL unit
is so big that it extends beyond the end of the input packet; it does so
only implicitly by using the checked version of the bytestream2 API.
But this has downsides compared to real checks: It can lead to huge
allocations (up to 2GiB) even when the input packet is just a few bytes.
And furthermore it leads to uninitialized data being output.
So add a check to error out early if it happens.

Also check directly whether there is enough data for the length field.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ea1b71e82f)
2020-06-15 04:46:05 +02:00
Roman Arzumanyan
221e490d42 avcodec/nvenc: honor max bitrate in CQ mode
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2020-06-04 19:02:25 +02:00
Roman Arzumanyan
e388365754 avcodec/nvenc: zero avg and max bitrate in CQ mode
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2020-06-01 19:55:37 +02:00
Andreas Rheinhardt
bf7ed52c25 libavcodec/libvpxenc: Don't free user-provided AVPacket
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 26b4509690)
2020-05-23 20:29:17 +02:00
Andreas Rheinhardt
ef76e5acaa libavcodec/libmp3lame: Don't free user-provided AVPacket
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 7e6941e185)
2020-05-23 20:29:16 +02:00
Andreas Rheinhardt
cf813c09d7 avcodec/libopusenc: Don't free user-provided AVPacket
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b803993b6d)
2020-05-23 20:29:16 +02:00
Michael Niedermayer
d3b963cc41 Changelog: update
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:33:14 +02:00
Michael Niedermayer
989a2788c0 avcodec/pnmdec: Use unsigned for maxval rescaling
Fixes: signed integer overflow: 65535 * 55335 cannot be represented in type 'int'
Fixes: 21955/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGMYUV_fuzzer-5669206981083136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 49459aca47)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:29 +02:00
Michael Niedermayer
b7fdc2d02b avcodec/ivi: Clear got_p_frame before decoding a new frame using it
Fixes: assertion failure
Fixes: 21666/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO4_fuzzer-5706468994318336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1d633e6a0a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:29 +02:00
Michael Niedermayer
219c025463 avcodec/dsddec: Check channels
Fixes: division by zero
Fixes: 21677/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DSD_MSBF_fuzzer-5712547983654912
Fixes: 21751/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DSD_LSBF_fuzzer-5197097180856320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2570a8777e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
53bfdb6fdb avcodec/xvididct: Fix integer overflow in idct_row()
Fixes: signed integer overflow: -1238335488 + -1003634688 cannot be represented in type 'int'
Fixes: 21649/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-5112005765890048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 620236e4d2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
5df443f763 avcodec/wmalosslessdec: Fix integer overflows in revert_inter_ch_decorr()
Fixes: signed integer overflow: -717241856 + -1434459904 cannot be represented in type 'int'
Fixes: 21405/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5677143666458624

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e9a4c4fe99)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
209b429168 avcodec/cbs_jpeg: Fix infinite loop in cbs_jpeg_split_fragment()
Fixes: Timeout
Fixes: 21104/clusterfuzz-testcase-minimized-ffmpeg_BSF_TRACE_HEADERS_fuzzer-5129580475318272

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a3dc67c984)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
ab9074c1c3 avformat/mpegenc: Fix integer overflow with AV_NOPTS_VALUE
Fixes: signed integer overflow: -9223372036854775808 - 45000 cannot be represented in type 'long'
Fixes: ticket8187

Found-by: Suhwan
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9874815b1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
52510a50f7 avformat/swfenc: Fix integer overflow in frame rate handling
Fixes: signed integer overflow: 30000299 * 256 cannot be represented in type 'int'
Fixes: ticket8184

Found-by: Suhwan
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 31f956acad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
719deee9a7 avformat/aadec: Check toc_size to contain the minimum to demuxer uses
Fixes: out of array access
Fixes: stack-buffer-overflow-READ-0x0831fff1

Found-by: GalyCannon <galycannon@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit daa2482871)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Michael Niedermayer
1653a2247f avcodec/cbs_h265_syntax_template: Limit num_long_term_pics more strictly
The limit is based on hevcdec.c
Fixes: 20854/clusterfuzz-testcase-minimized-ffmpeg_BSF_HEVC_METADATA_fuzzer-5160442882424832
Fixes: out of array access

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 435fa373d1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-21 15:32:28 +02:00
Peter Ross
1fc446d7e2 ffplay: set stream_index to -1 earlier to prevent segfault
Signed-off-by: Peter Ross <pross@xvid.org>
Reviewed-by: Marton Balint <cus@passwd.hu>
(cherry picked from commit 6cfb33f976)
2020-05-20 22:11:28 +02:00
Michael Niedermayer
137ceea997 Update for 4.2.3
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-05-20 19:44:32 +02:00
Dale Curtis
7c01f27701 avformat/mov: Free temp buffer upon negative sample_size error.
2d8d554f15 added a new error condition
to mov_read_stsz() but forgot to free a temporary buffer when it
occurs.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd0771c38c)
2020-05-20 19:42:47 +02:00
Andreas Rheinhardt
0a4e978f35 avformat/matroskadec: Improve forward compability
Matroska is built around the principle that a reader does not need to
understand everything in a file in order to be able to make use of it;
it just needs to ignore the data it doesn't know about.

Our demuxer typically follows this principle, but there is one important
instance where it does not: A Block belonging to a TrackEntry with no
associated stream is treated as invalid data (i.e. the demuxer will try
to resync to the next level 1 element because it takes this as a sign
that it has lost sync). Given that we do not create streams if we don't
know or don't support the type of the TrackEntry, this impairs this
demuxer's forward compability.

Furthermore, ignoring Blocks belonging to a TrackEntry without
corresponding stream can (in future commits) also be used to ignore
TrackEntries with obviously bogus entries without affecting the other
TrackEntries (by not creating a stream for said TrackEntry).

Finally, given that matroska_find_track_by_num() already emits its own
error message in case there is no TrackEntry with a given TrackNumber,
the error message (with level AV_LOG_INFO) for this can be removed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b577968cab)
2020-05-20 10:51:30 +02:00
Andreas Rheinhardt
26d1e77923 avformat/matroskadec: Don't discard valid packets
A Block (meaning both a Block in a BlockGroup as well as a SimpleBlock)
must have at least three bytes after the field containing the encoded
TrackNumber. So if there are <= 3 bytes, the Matroska demuxer would
skip this block, believing it to be an empty, but valid Block.

This might discard valid nonempty Blocks, namely if the track uses header
stripping. And certain definitely spec-incompliant Blocks don't raise
errors: Those with two or less bytes left after the encoded TrackNumber
and those with three bytes left, but with flags indicating that the Block
uses lacing as then there has to be further data describing the lacing.

Furthermore, zero-sized packets were still possible because only the
size of the last entry of a lace was checked.

This commit fixes this. All spec-compliant Blocks that contain data
(even if side data only) are now returned to the caller; spec-compliant
Blocks that don't contain anything are not returned.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit e471faf962)
2020-05-20 10:51:30 +02:00
Andreas Rheinhardt
588d258ea4 avformat/matroskaenc: Don't segfault when seekability changes
If the Matroska muxer's AVIOContext was unseekable when writing the
header, but is seekable when writing the trailer, the code for writing
the trailer presumes that a dynamic buffer exists and tries to update
its content in order to overwrite data that has already been
preliminarily written when writing the header, yet said buffer doesn't
exist as it has been written finally and not preliminarily when writing
the header (because of the unseekability it was presumed that one won't
be able to update the data anyway).

This commit adds a check for this and also for a similar situation
involving updating extradata with new data from packet side-data.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 8aabcf6c11)
2020-05-20 10:51:30 +02:00
Andreas Rheinhardt
f2117e1e9c avformat/utils: Fix memleaks
ff_read_packet had potential memleaks:
1. If av_packet_make_refcounted fails, it means that the packet is not
refcounted, but it could nevertheless carry side data and therefore
needs to be unreferenced.
2. If putting a packet on a packet list fails, it wasn't unreferenced.

Furthermore, read_frame_internal leaked a packet's (side) data if a
context update was required and failed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 66d5e43b9909804511a3e1fb65f6f0bbb0264eb3)
2020-05-20 10:51:30 +02:00
Andreas Rheinhardt
2c88423857 avformat/utils: Fix memleaks in avformat_open_input()
A demuxer might have allocated memory while reading the header. If
reading the header was successfull and an error happens before returning
(e.g. when queueing the attached pictures), the read_close function
would have never been called, so that all those allocations would leak.
This commit changes this.

Furthermore, there would be even more memleaks if the error level was
set to AV_EF_EXPLODE in case there is both metadata and id3v2 metadata.
This has been fixed, too.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
(cherry picked from commit e2307f4ff1)
2020-05-20 10:51:30 +02:00
Andreas Rheinhardt
15ccdea8b3 avfilter/vf_dedot: Fix leak of AVFrame if making it writable fails
Even in this scenario, the frame still contains references to data that
won't be freed if the frame isn't unreferenced. And the AVFrame itself
will leak, too.

Fixes Coverity issue #1441422.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 212077eda4)
2020-05-20 10:51:29 +02:00
Andreas Rheinhardt
126cd3821d avfilter/vf_paletteuse: Fix potential double-free of AVFrame
apply_palette() would free an AVFrame given to it only via an AVFrame *
(and not via AVFrame **) in three of its four exists (namely in the
normal path and in two error paths). So upon error the caller has no way
to know whether the frame has already been freed or not;
load_apply_palette(), the only caller, opted to free the frame in this
scenario.

This commit changes this by making apply_palette not freeing the frame
at all, which is left to load_apply_palette().

Fixes Coverity issue #1452434.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit adea33f46513821c111c602a0692b78315688c1b)
2020-05-20 10:51:29 +02:00
Andreas Rheinhardt
fa7d8d63db avformat/mov: Don't leak MOVFragmentStreamInfo on error
Fixes Coverity issue #1441933.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 43f58f2354bfab3819e44c1a97b0af75cc091226)
2020-05-20 10:51:29 +02:00
Andreas Rheinhardt
09d2b9b7be avformat/mov: Free encryption data on error
Fixes memleak and Coverity issue #1439587.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3999c4b374c2f3786137bd7e820dd1555fc20d90)
2020-05-20 10:51:29 +02:00
Andreas Rheinhardt
f23478aba2 fftools/ffmpeg: Free swresample dictionary during cleanup
Freeing this was forgotten in ad899522.

Fixes #8315 and #8316.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6f2a3958cf)
2020-05-20 10:51:29 +02:00
Matthieu Bouron
a7b0ab2a23 avcodec/mediacodec_wrapper: fix {input,output}_buffers global reference leak
Fixes ticket #8607.

Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
(cherry picked from commit 5216edbc54)
2020-05-20 09:35:58 +02:00
Andreas Rheinhardt
60d8a54432 avformat/webm_chunk: Close IO if writing header fails
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 2a78968849)
2020-05-20 04:15:00 +02:00
Andreas Rheinhardt
c70f788836 avcodec/cavsdsp: Fix undefined left shifts of negative numbers
Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version
of idct8_add is used and not e.g. the x86 SSE2 version).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes: left shift of negative value -107
Fixes: 20398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CAVS_fuzzer-5725389278412800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f0f2ab0c3)
2020-05-20 04:12:20 +02:00
Andreas Rheinhardt
3d9fd3ed69 avcodec/ra144enc: Fix invalid left shift of negative number
by replacing it with a multiplication. Said multiplication can't
overflow an int32_t because lpc_coefs is limited to 16 bit precision.

Fixes the FACE-test acodec-ra144 as well as part of #8217.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fb9af6f1)
2020-05-20 04:08:57 +02:00
Andreas Rheinhardt
dfe0d907c2 avcodec/adxenc: Avoid undefined left shift of negative numbers
Replace "((a << shift) + b) >> shift" by "a + (b >> shift)". This avoids
a left shift which also happens to trigger undefined behaviour in case "a"
is negative. This affected the FATE-tests acodec-adpcm-adx and
acodec-adpcm-adx-trellis; it also fixes ticket #8008.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59a9d65e0d)
2020-05-20 04:05:39 +02:00
Andreas Rheinhardt
4f17205a51 avcodec/adpcm: Fix undefined left shifts of negative numbers
Affected the adpcm-afc, adpcm-ea-1, adpcm-ea-2, adpcm-ea-maxis-xa,
adpcm-thp and ea-cdata FATE-tests. Also fixes ticket #8487.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ad8af51b7)
2020-05-20 04:02:44 +02:00
Andreas Rheinhardt
dc0ace722b avcodec/proresenc_anatoliy: Fix invalid left shift of negative number
This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).

(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 198081efb7)
2020-05-20 04:00:32 +02:00
Andreas Rheinhardt
e13e36f322 avformat/aviobuf: Honor avio_open[2] documentation
The documentation of both avio_open() as well as avio_open2() states
that on failure, the pointer to an AVIOContext given to this function
(via a pointer to a pointer to an AVIOContext) will be set to NULL. Yet
it didn't happen upon failure of ffurl_open_whitelist() or when allocating
the internal buffer failed. This commit changes this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 220846f74f)
2020-05-20 03:59:36 +02:00
Andreas Rheinhardt
d74e112ad8 avcodec/cinepakenc: Fix invalid shifts
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'.
Affected the FATE-tests vsynth1-cinepak, vsynth2-cinepak and
vsynth_lena-cinepak. Also fixes ticket #8220.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3650dcfc9)
2020-05-20 03:51:23 +02:00
Andreas Rheinhardt
02ac15ff35 avfilter/vf_xbr: Fix left shift of negative number
Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr,
filter-3xbr, filter-4xbr.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4294dc3589)
2020-05-20 03:49:13 +02:00
Andreas Rheinhardt
cbf4c7a6d8 avfilter/vf_hqx: Fix undefined left shifts of negative numbers
Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fa21194326)
2020-05-20 03:48:42 +02:00
Andreas Rheinhardt
33093d3782 avcodec/jpeg2000dwt: Fix undefined shifts of negative numbers
Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket #7983.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5cf593adcd)
2020-05-20 03:47:24 +02:00
Andreas Rheinhardt
5b26d86cde avcodec/ituh263dec: Fix undefined left shift of negative number
Fixes ticket #8160.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 324487b596)
2020-05-20 03:46:43 +02:00
Andreas Rheinhardt
0d5596a3fb avcodec/dnxhdenc: Fix undefined left shifts of negative numbers
Affected 61 FATE-tests: 60 vsynth tests and lavf-mxf_opatom.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b7f156e8cb)
2020-05-20 03:45:50 +02:00
Andreas Rheinhardt
4b75d960b6 swscale/utils: Fix invalid left shifts of negative numbers
Affected the FATE-tests vsynth_lena-dv-411, vsynth1-dv-411,
vsynth2-dv-411 and hevc-paramchange-yuv420p.yuv420p10.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e2646e23be)
2020-05-20 03:44:07 +02:00
Andreas Rheinhardt
b694403ef9 swscale/x86/swscale: Fix undefined left shifts of negative numbers
This affected many FATE-tests: The number of failing tests went down
from 663 to 344. (Both numbers exclude tests that failed because of
unaligned accesses in code that is inside #if HAVE_FAST_UNALIGNED.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 736c7c20e7)
2020-05-20 03:42:42 +02:00
Andreas Rheinhardt
a4a5abbad6 fftools/ffmpeg_opt: Fix signed integer overflow
Fixes ticket #8154.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b1fcba8dd)
2020-05-20 03:39:58 +02:00
Andreas Rheinhardt
4f5168e808 avcodec/exr: Fix undefined left shifts of negative numbers
Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b0f949906)
2020-05-20 03:35:10 +02:00
Andreas Rheinhardt
d58dab562b avformat/movenc: Fix undefined shift
Fixes the movenc FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 646799b42f)
2020-05-20 03:33:45 +02:00