This prevents insane memory usage in case of insane input values.
Untested due to lack of a testcase that causes such insane allocation

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>