[Mesa-dev] [PATCH] mesa: loosen small matrix determinant check

[Brian Paul]

On Mon, Jul 30, 2012 at 7:39 AM, Patrick Baggett
<baggett.patrick at gmail.com> wrote:
>
>
> On Mon, Jul 30, 2012 at 4:31 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
>>
>> On Tue, 24 Jul 2012 11:31:59 -0600
>> Brian Paul <brianp at vmware.com> wrote:
>>
>> > When computing a matrix inverse, if the determinant is too small we
>> > could hit
>> > a divide by zero.  There's a check to prevent this (we basically give up
>> > on
>> > computing the inverse and return the identity matrix.)  This patches
>> > loosens
>> > this test to fix a lighting bug reported by Lars Henning Wendt.
>> >
>> > NOTE: This is a candidate for the 8.0 branch.
>> > ---
>> >  src/mesa/math/m_matrix.c |    2 +-
>> >  1 files changed, 1 insertions(+), 1 deletions(-)
>> >
>> > diff --git a/src/mesa/math/m_matrix.c b/src/mesa/math/m_matrix.c
>> > index 02aedba..ef377ee 100644
>> > --- a/src/mesa/math/m_matrix.c
>> > +++ b/src/mesa/math/m_matrix.c
>> > @@ -513,7 +513,7 @@ static GLboolean invert_matrix_3d_general( GLmatrix
>> > *mat )
>> >
>> >     det = pos + neg;
>> >
>> > -   if (det*det < 1e-25)
>> > +   if (det < 1e-25)
>> >        return GL_FALSE;
>> >
>> >     det = 1.0F / det;
>>
>> Hi,
>>
>> just a fly-by question; doesn't that break if determinant is negative?
>> I.e. reflection transformations.
>>
> Yeah, I think you need a fabsf() there.

v2 of the patch used abs() and was pushed a while ago.

-Brian