dshow: fix return code when opening device

Successfully opening a device altered the ret variable, making the function
not cleanup properly and return an incorrect value for errors that happened
afterwards.

Reviewed-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>