[Mesa-dev] [PATCH 1/1] mesa: Properly set the fog scale (gl_Fog.scale) to +INF when fog start and end are equal.

[Henri Verbeet]

This was originally introduced by commit
ba47aabc9868b410cdfe3bc8b6d25a44a598cba2, but unfortunately the commit message
doesn't go into much detail about why +INF would be a problem here. I don't
see anything in the spec that would allow 1.0f here.

A similar issue exists for STATE_FOG_PARAMS_OPTIMIZED, but allowing infinity
there would potentially introduce NaNs where they shouldn't exist, depending
on the values of fog end and the fog coord. Since STATE_FOG_PARAMS_OPTIMIZED
is only used for fixed function (including ARB_fragment_program with fog
option), and the calculation there probably isn't very stable to begin with
when fog start and end are close together, it seems best to just leave it
alone.

This fixes a couple of Wine D3D tests. No piglit changes on Cayman.

Signed-off-by: Henri Verbeet <hverbeet at gmail.com>
---
 src/mesa/program/prog_statevars.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/src/mesa/program/prog_statevars.c b/src/mesa/program/prog_statevars.c
index f6073be..657c6e6 100644
--- a/src/mesa/program/prog_statevars.c
+++ b/src/mesa/program/prog_statevars.c
@@ -256,8 +256,7 @@ _mesa_fetch_state(struct gl_context *ctx, const gl_state_index state[],
       value[0] = ctx->Fog.Density;
       value[1] = ctx->Fog.Start;
       value[2] = ctx->Fog.End;
-      value[3] = (ctx->Fog.End == ctx->Fog.Start)
-         ? 1.0f : (GLfloat)(1.0 / (ctx->Fog.End - ctx->Fog.Start));
+      value[3] = (GLfloat)(1.0 / (ctx->Fog.End - ctx->Fog.Start));
       return;
    case STATE_CLIPPLANE:
       {
-- 
1.7.10.4