[Mesa-dev] [PATCH v2] i965: fixed clamping in set_scissor_bits when the y is flipped

[Nanley Chery]

On Wed, Feb 20, 2019 at 03:08:29PM +0200, Eleni Maria Stea wrote:
> Calculating the scissor rectangle fields with the y flipped (0 on top)
> can generate negative values that will cause assertion failure later on
> as the scissor fields are all unsigned. We must clamp the bbox values
> again to make sure they don't exceed the fb_height. Also fixed a
> calculation error.
> 
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108999
> 
> v2:
>    - I initially clamped the values inside the if (Y is flipped) case
>    and I made a mistake in the calculation: the clamp of the bbox[2] should
>    be a check if (bbox[2] >= fbheight) bbox[2] = fbheight - 1 instead and I
>    shouldn't have changed the ScissorRectangleYMax calculation. As the
>    fixed code is equivalent with using CLAMP instead of MAX2 at the top of
>    the function when bbox[2] and bbox[3] are calculated, and the 2nd is more
>    clear, I replaced it. (Nanley Chery)
> ---
>  src/mesa/drivers/dri/i965/genX_state_upload.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/mesa/drivers/dri/i965/genX_state_upload.c b/src/mesa/drivers/dri/i965/genX_state_upload.c
> index dcdfb3c9292..dd695218fea 100644
> --- a/src/mesa/drivers/dri/i965/genX_state_upload.c
> +++ b/src/mesa/drivers/dri/i965/genX_state_upload.c
> @@ -2445,8 +2445,8 @@ set_scissor_bits(const struct gl_context *ctx, int i,
>  
>     bbox[0] = MAX2(ctx->ViewportArray[i].X, 0);
>     bbox[1] = MIN2(bbox[0] + ctx->ViewportArray[i].Width, fb_width);
> -   bbox[2] = MAX2(ctx->ViewportArray[i].Y, 0);
> -   bbox[3] = MIN2(bbox[2] + ctx->ViewportArray[i].Height, fb_height);
> +   bbox[2] = CLAMP(ctx->ViewportArray[i].Y, 0, fb_height);
> +   bbox[3] = CLAMP(bbox[2] + ctx->ViewportArray[i].Height, 0, fb_height);

The API guarantees that viewport height is positive, so we can leave the
calculation of bbox[3] unmodified.

>     _mesa_intersect_scissor_bounding_box(ctx, i, bbox);
>  
>     if (bbox[0] == bbox[1] || bbox[2] == bbox[3]) {
> -- 
> 2.20.1
>