[Mesa-dev] Improving precision of mod(x,y)

[Roland Scheidegger]

Am 15.01.2015 um 10:05 schrieb Iago Toral:
> Hi,
> 
> We have 16 deqp tests that fail, at least on i965, because of
> insufficient precision of the mod GLSL function.
> 
> Mesa lowers mod(x,y) to y * fract(x,y) so there can be some precision
> lost due to fract operation. Since the result is multiplied by y the
> total precision lost usually grows together with the value of y.
Did you mean fract(x/y) here?

> 
> Below are some examples to give an idea of the magnitude of this error.
> The values on the right represent the precision error for each case:
> 
> mod(-1.951171875, 1.9980468750) =>  0.0000000447
> mod(121.57, 13.29)              =>  0.0000023842
> mod(3769.12, 321.99)            =>  0.0000762939
> mod(3769.12, 1321.99)           =>  0.0001220703
> mod(-987654.125, 123456.984375) =>  0.0160663128
> mod( 987654.125, 123456.984375) =>  0.0312500000
> 
> As you see, for large enough values, the precision error becomes
> significant.
> 
> This can be fixed by lowering mod(x,y) to x - y * floor(x/y) instead,
> which is the suggested implementation in the GLSL docs. I have a local
> patch in my tree that does this and it does indeed fix the problem. the
> down side is that this implementation adds and extra ADD instruction to
> the generated code (besides replacing fract with floor, which I guess
> have similar cost).
> 
> Since this is a case where there is some trade-off to the fix, I wonder
> if we are interested in doing this or not. Is the precision fix worth
> the additional ADD?
> 

Well I can tell you that llvmpipe implements frc(x) as x - floor(x), so
this change looks good to me :-).
On a more serious note though, it looks to me like the cost of this
expression would be mostly dominated by the division, hence some add
more shouldn't be that bad. And if the test is legit, I don't think
there's much choice (unless you could make this optional for some old
glsl versions if they didn't require that much precision but even then
it's probably not worth bothering imho).

Roland