Mask all unused bits for packed pixel format instead of green and alpha mask only.

That fixes the case when converting 15-bit RGB/BGR to YUV and high bit is set
for input value(s).

Originally committed as revision 28916 to svn://svn.mplayerhq.hu/mplayer/trunk/libswscale
This commit is contained in:
Kostya Shishkov 2009-03-09 17:25:43 +00:00
parent 2500d8bea6
commit bcff32d1b4
1 changed files with 1 additions and 1 deletions

View File

@ -1663,7 +1663,7 @@ static inline void RENAME(name ## _half)(uint8_t *dstU, uint8_t *dstV, uint8_t *
{\
int pix0= ((type*)src)[2*i+0];\
int pix1= ((type*)src)[2*i+1];\
int g= (pix0&(maskg|maska))+(pix1&(maskg|maska));\
int g= (pix0&~(maskr|maskb))+(pix1&~(maskr|maskb));\
int b= ((pix0+pix1-g)&(maskb|(2*maskb)))>>shb;\
int r= ((pix0+pix1-g)&(maskr|(2*maskr)))>>shr;\
g&= maskg|(2*maskg);\