Mesa (master): draw: check for integer overflows in instance computation

[Zack Rusin]

Module: Mesa
Branch: master
Commit: 88de009cc1ab838cd87416dcbbaa4f5b30173c71
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=88de009cc1ab838cd87416dcbbaa4f5b30173c71

Author: Zack Rusin <zackr at vmware.com>
Date:   Tue Jun 25 17:01:14 2013 -0400

draw: check for integer overflows in instance computation

Integers could easily overflow is the starting instance
was large enough. Instead of letting bogus counts through
set the instance to max if it overflown and let our
regular buffer overflow computation handle it.

Signed-off-by: Zack Rusin <zackr at vmware.com>

---

 src/gallium/auxiliary/draw/draw_llvm.c |    1 +
 src/gallium/auxiliary/draw/draw_pt.c   |    6 ++++++
 2 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/src/gallium/auxiliary/draw/draw_llvm.c b/src/gallium/auxiliary/draw/draw_llvm.c
index c1b4acd..33cccfe 100644
--- a/src/gallium/auxiliary/draw/draw_llvm.c
+++ b/src/gallium/auxiliary/draw/draw_llvm.c
@@ -768,6 +768,7 @@ generate_fetch(struct gallivm_state *gallivm,
                         lp_build_const_int32(
                            gallivm,
                            util_format_get_blocksize(velem->src_format)));
+   lp_build_printf(gallivm, "   instance_id = %u\n", instance_id);
    lp_build_printf(gallivm, "   stride = %u\n", stride);
    lp_build_printf(gallivm, "   buffer size = %u\n", buffer_size);
    lp_build_printf(gallivm, "   needed_buffer_size = %u\n", needed_buffer_size);
diff --git a/src/gallium/auxiliary/draw/draw_pt.c b/src/gallium/auxiliary/draw/draw_pt.c
index e0b8007..c4d06de 100644
--- a/src/gallium/auxiliary/draw/draw_pt.c
+++ b/src/gallium/auxiliary/draw/draw_pt.c
@@ -533,6 +533,12 @@ draw_vbo(struct draw_context *draw,
 
    for (instance = 0; instance < info->instance_count; instance++) {
       draw->instance_id = instance + info->start_instance;
+      /* check for overflow */
+      if (draw->instance_id < instance ||
+          draw->instance_id < info->start_instance) {
+         /* if we overflown just set the instance id to the max */
+         draw->instance_id = 0xffffffff;
+      }
 
       draw_new_instance(draw);