swr: check that there is enough information to do rematrixing when needed.

Fixes assertion failure.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>