[Mesa-dev] [PATCHv2 2/2] util/u_vbuf: correct map offset calculation for crazy offsets

[Marek Olšák]

Reviewed-by: Marek Olšák <marek.olsak at amd.com>

Marek

On Thu, Jan 23, 2014 at 3:47 AM, Ilia Mirkin <imirkin at alum.mit.edu> wrote:
> When the min_index is very large (or very negative), the multipliation
> can overflow 32 bits and result in an incorrect map pointer
> modification.
>
> Signed-off-by: Ilia Mirkin <imirkin at alum.mit.edu>
> ---
>
> I tested this by setting PIPE_CAP_USER_*_BUFFERS to 0 on nv50, and hacking the
> logic to say that unroll_indices = true, running with the
> draw-elements-base-vertex-neg user_varrays test.
>
>  src/gallium/auxiliary/util/u_vbuf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/gallium/auxiliary/util/u_vbuf.c b/src/gallium/auxiliary/util/u_vbuf.c
> index 52b360e..0c9c349 100644
> --- a/src/gallium/auxiliary/util/u_vbuf.c
> +++ b/src/gallium/auxiliary/util/u_vbuf.c
> @@ -370,7 +370,7 @@ u_vbuf_translate_buffers(struct u_vbuf *mgr, struct translate_key *key,
>
>        /* Subtract min_index so that indexing with the index buffer works. */
>        if (unroll_indices) {
> -         map -= vb->stride * min_index;
> +         map -= (ptrdiff_t)vb->stride * min_index;
>        }
>
>        tr->set_buffer(tr, i, map, vb->stride, ~0);
> --
> 1.8.3.2
>