[Mesa-dev] [PATCH] sp_tex_sample: Fix segfault with fbo-cubemap.

[Brian Paul]

On 07/19/2012 10:55 AM, Olivier Galibert wrote:
> The cube sampler generates two-dimensional texture coordinates and
> hence passes NULL for the array for the third one.  The actual 2D
> sampler, lower in the pipe, knew not to used that array since it
> didn't need it.  But the samplers have become single-texel and the
> coordinate array dereference has been moved up one step, to a level
> where the code does not know only two coordinates are used.  Hence the
> segfault.
>
> The simplest fix by far is to add a third dummy coordinate array in
> the call to the next pipe step, which will be dereferenced to an
> harmless 0 which then will be happily ignored by the sampler.
>
> Signed-off-by: Olivier Galibert<galibert at pobox.com>
> ---
>   src/gallium/drivers/softpipe/sp_tex_sample.c |    7 ++++++-
>   1 file changed, 6 insertions(+), 1 deletion(-)
>
> Brown paper bag time.  I had tested with (I think) everything with
> "tex" in the name.  Guess what fbo-cubemap doesn't have in the name?
>
> Fixes 52250.
>
> diff --git a/src/gallium/drivers/softpipe/sp_tex_sample.c b/src/gallium/drivers/softpipe/sp_tex_sample.c
> index 292dc6e..2f6e272 100644
> --- a/src/gallium/drivers/softpipe/sp_tex_sample.c
> +++ b/src/gallium/drivers/softpipe/sp_tex_sample.c
> @@ -2090,6 +2090,11 @@ sample_cube(struct tgsi_sampler *tgsi_sampler,
>      unsigned j;
>      float ssss[4], tttt[4];
>
> +   /* Not actually used, but the intermediate steps that do the
> +    * dereferencing don't know it.
> +    */
> +   float pppp[4] = { 0, 0, 0, 0 };

static const float ...

> +
>      /*
>        major axis
>        direction    target                             sc     tc    ma
> @@ -2157,7 +2162,7 @@ sample_cube(struct tgsi_sampler *tgsi_sampler,
>       * is not active, this will point somewhere deeper into the
>       * pipeline, eg. to mip_filter or even img_filter.
>       */
> -   samp->compare(tgsi_sampler, ssss, tttt, NULL, c0, control, rgba);
> +   samp->compare(tgsi_sampler, ssss, tttt, pppp, c0, control, rgba);
>   }
>
>

Reviewed-by: Brian Paul <brianp at vmware.com>