It can take a long time before subtitles or data streams show up,
so we shouldn't wait for those before assuming we have all info
for streams.

Signed-off-by: Martin Storsjö <martin@martin.st>