[Mesa-dev] [PATCH] gallium/u_vbuf: Protect against overflow with large instance divisors.

[Brian Paul]

On 03/21/2018 11:06 AM, Eric Anholt wrote:
> GTF-GLES3.gtf.GL3Tests.instanced_arrays.instanced_arrays_divisor uses -1
> as a divisor,

Since GL's vertex attrib instance divisor is a GLuint, we can 
alternately say the divisor is ~0, right?


> so we would overflow to count=0 and upload no data,
> triggering the assert below.  We want to upload 1 element in this case,
> fixing the test on VC5.
> ---
>   src/gallium/auxiliary/util/u_vbuf.c | 7 ++++++-
>   1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/gallium/auxiliary/util/u_vbuf.c b/src/gallium/auxiliary/util/u_vbuf.c
> index 95d7990c6ca4..9073f3feed98 100644
> --- a/src/gallium/auxiliary/util/u_vbuf.c
> +++ b/src/gallium/auxiliary/util/u_vbuf.c
> @@ -936,7 +936,12 @@ u_vbuf_upload_buffers(struct u_vbuf *mgr,
>            size = mgr->ve->src_format_size[i];
>         } else if (instance_div) {
>            /* Per-instance attrib. */
> -         unsigned count = (num_instances + instance_div - 1) / instance_div;
> +         unsigned count = (num_instances + instance_div - 1);
> +
> +         if (count < num_instances)
> +            count = 0xffffffff;
> +         count /= instance_div;

I've been staring at this for a while but I'm still not sure I 
understand what's happening.  Can you add a comment at least?


> +
>            first += vb->stride * start_instance;
>            size = vb->stride * (count - 1) + mgr->ve->src_format_size[i];
>         } else {
>