[Mesa-dev] Improving precision of mod(x,y)

[Ian Romanick]

On 01/15/2015 10:05 PM, Iago Toral wrote:
> Hi,
> 
> We have 16 deqp tests that fail, at least on i965, because of
> insufficient precision of the mod GLSL function.
> 
> Mesa lowers mod(x,y) to y * fract(x,y) so there can be some precision
> lost due to fract operation. Since the result is multiplied by y the
> total precision lost usually grows together with the value of y.
> 
> Below are some examples to give an idea of the magnitude of this error.
> The values on the right represent the precision error for each case:
> 
> mod(-1.951171875, 1.9980468750) =>  0.0000000447
> mod(121.57, 13.29)              =>  0.0000023842
> mod(3769.12, 321.99)            =>  0.0000762939
> mod(3769.12, 1321.99)           =>  0.0001220703
> mod(-987654.125, 123456.984375) =>  0.0160663128
> mod( 987654.125, 123456.984375) =>  0.0312500000
> 
> As you see, for large enough values, the precision error becomes
> significant.
> 
> This can be fixed by lowering mod(x,y) to x - y * floor(x/y) instead,
> which is the suggested implementation in the GLSL docs. I have a local
> patch in my tree that does this and it does indeed fix the problem. the
> down side is that this implementation adds and extra ADD instruction to
> the generated code (besides replacing fract with floor, which I guess
> have similar cost).
> 
> Since this is a case where there is some trade-off to the fix, I wonder
> if we are interested in doing this or not. Is the precision fix worth
> the additional ADD?

I don't think mod is used so often that the extra instruction will wreck
performance for anyone.  The proposed change seems better.

> Iago
> 
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
>