[Mesa-dev] swrast: Correct pixel draw span endpoints computation, rid vertical lines

[Daniel J Sebald]

Hello mesa-dev,

I've created a changeset for the legacy swrast_dri.so driver which fixes 
vertical lines at the GL_MAX_TEXTURE_SIZE boundaries due to miscomputed 
column range computations.  Please consider.

Basically, I was very meticulous about the mathematical formulas in 
terms of rounding according to the OpenGL definition for both positive 
and negative xfactor and yfactor.  That fixed the vertical lines, but 
after creating a suggested Piglit test for glPixelZoom I noticed the 
driver still wasn't behaving according to some intricacies of the 
standard formulas.

The other advantage of being more detailed about the formulas is sort of 
sweeping numerical issues into a corner, i.e., the behavior of ceil() 
and floor() functions.  In order to pass the expected tests, I needed to 
add a tolerance of about 0.00004 to the rounding functions.  This isn't 
anywhere near the, say, ULP (units in the last place) called out in 
ARB-shader-precision, but it is the necessary value I've found when 
working with single-precision float arithmetic.  It's surprising how 
quickly precision can be lost by the division that is typically used in 
computing xfactor/yfactor before the user makes the glPixelZoom() call.

I've collected image files comparing the legacy driver and patched 
legacy, as well as Gallium driver here:

   https://bugs.freedesktop.org/show_bug.cgi?id=89586

The PNG files for Piglit test results, more than anything, should 
provide a good understanding of the issues.  The Gallium driver shows a 
slight discrepancy from the patched legacy driver.  I'm not saying which 
is correct, if any, just pointing out the slight difference.

Patch file is attached.  I'd be happy to answer any questions or tweak 
the code.

Regards,

Dan Sebald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mesa-vertical_lines-djs2015mar29.patch
Type: text/x-patch
Size: 13177 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20150330/4caa3c2e/attachment.bin>