The libopus encoder does the same thing and its better than
keeping track of when the empty flush frames appear.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>