[Mesa-dev] [PATCH] mesa: loosen small matrix determinant check

[Brian Paul]

When computing a matrix inverse, if the determinant is too small we could hit
a divide by zero.  There's a check to prevent this (we basically give up on
computing the inverse and return the identity matrix.)  This patch loosens
this test to fix a lighting bug reported by Lars Henning Wendt.

v2: use abs(det) to handle negative values

NOTE: This is a candidate for the 8.0 branch.
---
 src/mesa/math/m_matrix.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/mesa/math/m_matrix.c b/src/mesa/math/m_matrix.c
index 02aedba..ffbdcdb 100644
--- a/src/mesa/math/m_matrix.c
+++ b/src/mesa/math/m_matrix.c
@@ -513,7 +513,7 @@ static GLboolean invert_matrix_3d_general( GLmatrix *mat )
 
    det = pos + neg;
 
-   if (det*det < 1e-25)
+   if (FABSF(det) < 1e-25)
       return GL_FALSE;
 
    det = 1.0F / det;
-- 
1.7.3.4