Fix a typo that causes an assertion to always fail.

Reported by Alexander Bokovikov (openworld AT uralweb DOT ru)

Originally committed as revision 26257 to svn://svn.mplayerhq.hu/mplayer/trunk/libswscale