swresample/dither: use integer arithmetic

This fixes a -Wabsolute-value reported by clang 3.5+ complaining about misuse of fabs() for integer absolute value.
An additional benefit is the removal of floating point calculations.

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>