ffmpeg/libswscale
Mans Rullgard 90540c2d5a x86: swscale: fix fragile memory accesses
To access data at multiple fixed offsets from a base address, this
code uses a single "m" operand and code of the form "32%0", relying on
the memory operand instantiation having no displacement, giving a final
result of the form "32(%rax)".  If the compiler uses a register and
displacement, e.g. "64(%rax)", the end result becomes "3264(%rax)",
which obviously does not work.

Replacing the "m" operands with "r" operands allows safe addition of a
displacement.  In theory, multiple memory operands could use a shared
base register with different index registers, "(%rax,%rbx)", potentially
making more efficient use of registers.  In the cases at hand, no such
sharing is possible since the addresses involved are entirely unrelated.

After this change, the code somewhat rudely accesses memory without
using a corresponding memory operand, which in some cases can lead to
unwanted "optimisations" of surrounding code.  However, the original
code also accesses memory not covered by a memory operand, so this is
not adding any defect not already present.  It is also hightly unlikely
that any such optimisations could be performed here since the memory
locations in questions are not accessed elsewhere in the same functions.

This fixes crashes with suncc.

Signed-off-by: Mans Rullgard <mans@mansr.com>
2012-08-13 14:51:52 +01:00
..
bfin bfin: libswscale: add const where appropriate to fix warnings 2012-06-20 02:15:33 +01:00
ppc swscale: Mark all init functions as av_cold 2012-07-23 01:30:05 +02:00
sparc swscale: Mark all init functions as av_cold 2012-07-23 01:30:05 +02:00
x86 x86: swscale: fix fragile memory accesses 2012-08-13 14:51:52 +01:00
Makefile Create version.h headers for libraries that lack them 2012-07-04 15:10:06 +02:00
colorspace-test.c swscale-test: Drop unused argc and argv arguments from main(). 2012-02-01 14:24:25 +01:00
input.c sws: fix planar RGB input conversions for 9/10/16 bpp. 2012-06-12 19:52:00 +02:00
libswscale.v Add symbol versioning for shared libraries 2010-01-16 04:48:55 +00:00
options.c swscale: remove "cpu flags" from -sws_flags description. 2012-03-04 06:52:06 -08:00
output.c swscale: Mark all init functions as av_cold 2012-07-23 01:30:05 +02:00
rgb2rgb.c swscale: Mark all init functions as av_cold 2012-07-23 01:30:05 +02:00
rgb2rgb.h rgb2rgb: rgb12to15() 2012-01-12 16:43:17 -08:00
rgb2rgb_template.c swscale: K&R formatting cosmetics (part I) 2012-02-21 22:53:40 +01:00
swscale-test.c swscale-test: fix stack overread. 2012-01-12 16:38:29 -08:00
swscale.c x86: build: replace mmx2 by mmxext 2012-08-03 22:51:05 +02:00
swscale.h x86: build: replace mmx2 by mmxext 2012-08-03 22:51:05 +02:00
swscale_internal.h swscale: K&R formatting cosmetics (part II) 2012-04-13 19:22:00 +02:00
swscale_unscaled.c Unscaled Planar RGB -> RGB support in swscale. 2012-04-03 13:06:42 -04:00
utils.c x86: rename libavutil/x86_cpu.h to libavutil/x86/asm.h 2012-08-09 00:58:20 +01:00
version.h x86: build: replace mmx2 by mmxext 2012-08-03 22:51:05 +02:00
yuv2rgb.c swscale: K&R formatting cosmetics (part I) 2012-02-21 22:53:40 +01:00