[Mesa-dev] [PATCH 1/1] mesa: Properly set the fog scale (gl_Fog.scale) to +INF when fog start and end are equal.

[Brian Paul]

On 08/17/2013 11:42 AM, Henri Verbeet wrote:
> This was originally introduced by commit
> ba47aabc9868b410cdfe3bc8b6d25a44a598cba2, but unfortunately the commit message
> doesn't go into much detail about why +INF would be a problem here. I don't
> see anything in the spec that would allow 1.0f here.
>
> A similar issue exists for STATE_FOG_PARAMS_OPTIMIZED, but allowing infinity
> there would potentially introduce NaNs where they shouldn't exist, depending
> on the values of fog end and the fog coord. Since STATE_FOG_PARAMS_OPTIMIZED
> is only used for fixed function (including ARB_fragment_program with fog
> option), and the calculation there probably isn't very stable to begin with
> when fog start and end are close together, it seems best to just leave it
> alone.
>
> This fixes a couple of Wine D3D tests. No piglit changes on Cayman.
>
> Signed-off-by: Henri Verbeet <hverbeet at gmail.com>
> ---
>   src/mesa/program/prog_statevars.c |    3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/src/mesa/program/prog_statevars.c b/src/mesa/program/prog_statevars.c
> index f6073be..657c6e6 100644
> --- a/src/mesa/program/prog_statevars.c
> +++ b/src/mesa/program/prog_statevars.c
> @@ -256,8 +256,7 @@ _mesa_fetch_state(struct gl_context *ctx, const gl_state_index state[],
>         value[0] = ctx->Fog.Density;
>         value[1] = ctx->Fog.Start;
>         value[2] = ctx->Fog.End;
> -      value[3] = (ctx->Fog.End == ctx->Fog.Start)
> -         ? 1.0f : (GLfloat)(1.0 / (ctx->Fog.End - ctx->Fog.Start));
> +      value[3] = (GLfloat)(1.0 / (ctx->Fog.End - ctx->Fog.Start));
>         return;
>      case STATE_CLIPPLANE:
>         {
>

I tested conform with this change and didn't see any regressions.

However, let's simply put 'f' on the 1.0 and remove the cast:

value[3] = 1.0f / (ctx->Fog.End - ctx->Fog.Start);

Tested-by: Brian Paul <brianp at vmware.com>
Reviewed-by: Brian Paul <brianp at vmware.com>