[Mesa-dev] [PATCH] gallivm: honor d3d10 floating point rules for shadow comparisons

[sroland at vmware.com]

From: Roland Scheidegger <sroland at vmware.com>

d3d10 specifies ordered comparisons for everything but not_equal which is
unordered (http://msdn.microsoft.com/en-us/library/windows/desktop/cc308050.aspx).
OpenGL probably doesn't care.
---
 src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c |   20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c b/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c
index 337b6f7..4305c49 100644
--- a/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c
+++ b/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c
@@ -1484,12 +1484,26 @@ lp_build_sample_compare(struct lp_build_sample_context *bld,
     * should be converted to the depth format (quantization!) and comparison
     * then done in texture format.
     */
+
    /* result = (p FUNC texel) ? 1 : 0 */
-   res = lp_build_cmp(texel_bld, bld->static_sampler_state->compare_func,
-                      p, texel[chan]);
+   /*
+    * honor d3d10 floating point rules here, which state that comparisons
+    * are ordered except NOT_EQUAL which is unordered.
+    */
+   if (bld->static_sampler_state->compare_func != PIPE_FUNC_NOTEQUAL) {
+      res = lp_build_cmp_ordered(texel_bld, bld->static_sampler_state->compare_func,
+                                 p, texel[chan]);
+   }
+   else {
+      res = lp_build_cmp(texel_bld, bld->static_sampler_state->compare_func,
+                         p, texel[chan]);
+   }
    res = lp_build_select(texel_bld, res, texel_bld->one, texel_bld->zero);
 
-   /* XXX returning result for default GL_DEPTH_TEXTURE_MODE = GL_LUMINANCE */
+   /*
+    * returning result for default GL_DEPTH_TEXTURE_MODE = GL_LUMINANCE.
+    * This should be ok because sampler swizzle is applied on top of it.
+    */
    texel[0] =
    texel[1] =
    texel[2] = res;
-- 
1.7.9.5