[Mesa-dev] [Bug 89586] Drivers/DRI/swrast

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=89586

--- Comment #38 from Dan Sebald <daniel.sebald at ieee.org> ---
0.1 tolerance is rather large (I'm guessing that applies to normalized
numbers).  In this case the discrepancy doesn't come close to that big.

I looked up ARB_shader_precision and I'm seeing tolerance on the order of 2 to
3 units of the last place, generally, with a factor larger tolerance (16 ULP)
for the pow() function.  Division is the source of the error in xfactor, which
has 2.5 ULP tolerance.

I can experiment with some of the C "next nearest number" routines and get a
feeling for the ULP.  Thanks.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20150325/31412b93/attachment.html>