ffmpeg_filter: check encoder before using it to set frame size.

Fix a segfault when no encoder is found for a default codec
with a complex filter graph.