[Mesa-dev] [PATCH] glsl: Clamp vector indices when lowering to swizzles

[Kenneth Graunke]

On 11/07/2011 11:12 AM, Ian Romanick wrote:
> From: Ian Romanick<ian.d.romanick at intel.com>
>
> This prevents other code from seeing a swizzle of the 16th component
> of a vector, for example.
>
> NOTE: This is a candidate for the 7.11 branch.
>
> Signed-off-by: Ian Romanick<ian.d.romanick at intel.com>
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=42517
> ---
>   src/glsl/lower_vec_index_to_swizzle.cpp |   17 +++++++++++++++--
>   1 files changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/src/glsl/lower_vec_index_to_swizzle.cpp b/src/glsl/lower_vec_index_to_swizzle.cpp
> index c7630c2..ebc0833 100644
> --- a/src/glsl/lower_vec_index_to_swizzle.cpp
> +++ b/src/glsl/lower_vec_index_to_swizzle.cpp
> @@ -33,6 +33,7 @@
>   #include "ir_visitor.h"
>   #include "ir_optimization.h"
>   #include "glsl_types.h"
> +#include "main/macros.h"
>
>   /**
>    * Visitor class for replacing expressions with ir_constant values.
> @@ -76,8 +77,20 @@ ir_vec_index_to_swizzle_visitor::convert_vec_index_to_swizzle(ir_rvalue *ir)
>
>      void *ctx = ralloc_parent(ir);
>      this->progress = true;
> -   return new(ctx) ir_swizzle(deref->array,
> -			      ir_constant->value.i[0], 0, 0, 0, 1);
> +
> +   /* Page 40 of the GLSL 1.20 spec says:
> +    *
> +    *     "When indexing with non-constant expressions, behavior is undefined
> +    *     if the index is negative, or greater than or equal to the size of
> +    *     the vector."
> +    *
> +    * The ir_swizzle constructor gets angry if the index is negative or too
> +    * large.  For simplicity sake, just clamp the index to [0, size-1].
> +    */
> +   const int i = MIN2(MAX2(ir_constant->value.i[0], 0),
> +		      (deref->array->type->vector_elements - 1));
> +
> +   return new(ctx) ir_swizzle(deref->array, i, 0, 0, 0, 1);
>   }
>
>   ir_visitor_status

This seems like a sensible approach (since we can't generate an error at 
this point.)

Reviewed-by: Kenneth Graunke <kenneth at whitecape.org>