[Mesa-dev] [PATCH 06/14] gallium/util: return FALSE for intersection if there's empty rectangles

[Brian Paul]

From: Roland Scheidegger <sroland at vmware.com>

It isn't really obvious if intersection test should take into account empty
rectangles or if the caller should do it. But it looks like most callers
actually verified one of the rects but not the other, but since correctly
returning an empty rect that other rect could actually be empty leading to
more bugs. Hence just verify both rects for emptyness in the intersection
test itself which makes the code easier in the caller (though it will be
slower if the caller knows the rectangles are non-empty).

Reviewed-by: Zack Rusin <zackr at vmware.com>
---
 src/gallium/auxiliary/util/u_rect.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/gallium/auxiliary/util/u_rect.h b/src/gallium/auxiliary/util/u_rect.h
index b26f671..dea1e1e 100644
--- a/src/gallium/auxiliary/util/u_rect.h
+++ b/src/gallium/auxiliary/util/u_rect.h
@@ -42,6 +42,7 @@ struct u_rect {
 };
 
 /* Do two rectangles intersect?
+ * Note: empty rectangles are valid as inputs (and never intersect).
  */
 static inline boolean
 u_rect_test_intersection(const struct u_rect *a,
@@ -50,7 +51,11 @@ u_rect_test_intersection(const struct u_rect *a,
    return (!(a->x1 < b->x0 ||
              b->x1 < a->x0 ||
              a->y1 < b->y0 ||
-             b->y1 < a->y0));
+             b->y1 < a->y0 ||
+             a->x1 < a->x0 ||
+             a->y1 < a->y0 ||
+             b->x1 < b->x0 ||
+             b->y1 < b->y0));
 }
 
 /* Find the intersection of two rectangles known to intersect.
-- 
1.9.1