Mesa (mesa_7_6_branch): mesa: refine the error checking vbo_exec_DrawRangeElements()

[Brian Paul]

Module: Mesa
Branch: mesa_7_6_branch
Commit: 2655d437569c5bce7c56782792cbd4460b9f758b
URL:    http://cgit.freedesktop.org/mesa/mesa/commit/?id=2655d437569c5bce7c56782792cbd4460b9f758b

Author: Brian Paul <brianp at vmware.com>
Date:   Mon Sep 21 14:23:07 2009 -0600

mesa: refine the error checking vbo_exec_DrawRangeElements()

If the 'end' index is out of bounds issue a warning as before.  But instead
of just no-op'ing the draw call, examine the actual array indices to see
if they're OK.  If the max array index is out of bounds, issue another
warning and no-op the draw call.  Otherwise, draw normally.  This is a
debug build-only feature since it could impact performance.

This "fixes" the missing torus in the OGL Distilled / Picking demo.

---

 src/mesa/vbo/vbo_exec_array.c |   35 +++++++++++++++++++++++++++++++----
 1 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/src/mesa/vbo/vbo_exec_array.c b/src/mesa/vbo/vbo_exec_array.c
index 12911f5..3f0656a 100644
--- a/src/mesa/vbo/vbo_exec_array.c
+++ b/src/mesa/vbo/vbo_exec_array.c
@@ -677,9 +677,10 @@ vbo_exec_DrawRangeElements(GLenum mode,
       /* the max element is out of bounds of one or more enabled arrays */
       _mesa_warning(ctx, "glDraw[Range]Elements(start %u, end %u, count %d, "
                     "type 0x%x, indices=%p)\n"
-                    "\tindex=%u is out of bounds (max=%u)  "
-                    "Element Buffer %u (size %d)",
-                    start, end, count, type, indices, end,
+                    "\tend is out of bounds (max=%u)  "
+                    "Element Buffer %u (size %d)\n"
+                    "\tThis should probably be fixed in the application.",
+                    start, end, count, type, indices,
                     ctx->Array.ArrayObj->_MaxElement - 1,
                     ctx->Array.ElementArrayBufferObj->Name,
                     ctx->Array.ElementArrayBufferObj->Size);
@@ -689,7 +690,33 @@ vbo_exec_DrawRangeElements(GLenum mode,
 
       if (0)
          _mesa_print_arrays(ctx);
-      return;
+
+#ifdef DEBUG
+      /* 'end' was out of bounds, but now let's check the actual array
+       * indexes to see if any of them are out of bounds.  If so, warn
+       * and skip the draw to avoid potential segfault, etc.
+       */
+      {
+         GLuint max = _mesa_max_buffer_index(ctx, count, type, indices,
+                                             ctx->Array.ElementArrayBufferObj);
+         if (max >= ctx->Array.ArrayObj->_MaxElement) {
+            _mesa_warning(ctx, "glDraw[Range]Elements(start %u, end %u, "
+                          "count %d, type 0x%x, indices=%p)\n"
+                          "\tindex=%u is out of bounds (max=%u)  "
+                          "Element Buffer %u (size %d)\n"
+                          "\tSkipping the glDrawRangeElements() call",
+                          start, end, count, type, indices, max,
+                          ctx->Array.ArrayObj->_MaxElement - 1,
+                          ctx->Array.ElementArrayBufferObj->Name,
+                          ctx->Array.ElementArrayBufferObj->Size);
+            return;
+         }
+         /* XXX we could also find the min index and compare to 'start'
+          * to see if start is correct.  But it's more likely to get the
+          * upper bound wrong.
+          */
+      }
+#endif
    }
    else if (0) {
       _mesa_printf("glDraw[Range]Elements"