Change unsigned->signed and int->long, this fits the asm code better on 64

bit systems.
Also fixes several crashes because (long)-i is incorrect if i is unsigned.

Originally committed as revision 16740 to svn://svn.mplayerhq.hu/mplayer/trunk/postproc