Disallow VLC coding with more than 8 bits as there are several bugs

in that code that could lead to broken files.
AC coding is unaffected.

Originally committed as revision 22638 to svn://svn.ffmpeg.org/ffmpeg/trunk