[Mesa-dev] Improving precision of mod(x,y)

[Iago Toral]

Thanks Roland and everyone else for the input. Based on the feedback
I'll include the fix in thew next round of deqp fixes.

Iago

On Thu, 2015-01-15 at 15:32 +0100, Roland Scheidegger wrote:
> Am 15.01.2015 um 10:05 schrieb Iago Toral:
> > Hi,
> > 
> > We have 16 deqp tests that fail, at least on i965, because of
> > insufficient precision of the mod GLSL function.
> > 
> > Mesa lowers mod(x,y) to y * fract(x,y) so there can be some precision
> > lost due to fract operation. Since the result is multiplied by y the
> > total precision lost usually grows together with the value of y.
> Did you mean fract(x/y) here?
> 
> > 
> > Below are some examples to give an idea of the magnitude of this error.
> > The values on the right represent the precision error for each case:
> > 
> > mod(-1.951171875, 1.9980468750) =>  0.0000000447
> > mod(121.57, 13.29)              =>  0.0000023842
> > mod(3769.12, 321.99)            =>  0.0000762939
> > mod(3769.12, 1321.99)           =>  0.0001220703
> > mod(-987654.125, 123456.984375) =>  0.0160663128
> > mod( 987654.125, 123456.984375) =>  0.0312500000
> > 
> > As you see, for large enough values, the precision error becomes
> > significant.
> > 
> > This can be fixed by lowering mod(x,y) to x - y * floor(x/y) instead,
> > which is the suggested implementation in the GLSL docs. I have a local
> > patch in my tree that does this and it does indeed fix the problem. the
> > down side is that this implementation adds and extra ADD instruction to
> > the generated code (besides replacing fract with floor, which I guess
> > have similar cost).
> > 
> > Since this is a case where there is some trade-off to the fix, I wonder
> > if we are interested in doing this or not. Is the precision fix worth
> > the additional ADD?
> > 
> 
> Well I can tell you that llvmpipe implements frc(x) as x - floor(x), so
> this change looks good to me :-).
> On a more serious note though, it looks to me like the cost of this
> expression would be mostly dominated by the division, hence some add
> more shouldn't be that bad. And if the test is legit, I don't think
> there's much choice (unless you could make this optional for some old
> glsl versions if they didn't require that much precision but even then
> it's probably not worth bothering imho).
> 
> Roland
> 
>