[Bug 697715] New: floating point precision problem in check test gst/gstvalue

[GStreamer (bugzilla.gnome.org)]

https://bugzilla.gnome.org/show_bug.cgi?id=697715
  GStreamer | gstreamer (core) | 1.x

           Summary: floating point precision problem in check test
                    gst/gstvalue
    Classification: Platform
           Product: GStreamer
           Version: 1.x
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: gstreamer (core)
        AssignedTo: gstreamer-bugs at lists.freedesktop.org
        ReportedBy: alexas at axis.com
         QAContact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---


The tests fail when building for 32 bit for us. I tracked the problem down to
this:

we do gst_date_time_new(....., 10.000001). This does g_date_time_new internally
on the same usec value. Problem is that g_date_time_new rounds the seconds
value _down_ to the closest usec. It's easy to think that we are in the clear
here. But 10.000001 in IEEE 754 representation is not quite 10.000001, more
like 10.0000009999999999. Rounding this value down gives 10.0. It's impossible
to represent 10.000001 exactly using IEEE 754, and we should never rely on the
given value to be exactly what it seems, it could just as well be slightly
higher or lower. Play around with this one if you'd like to convince yourself
that I am right ;)

http://www.h-schmidt.net/FloatConverter/

I think the reason that it doesn't happen on 64 bit build is because of
extended precision kicking in for some of the calculations. I haven't really
investigated this further though...

Question is, should we 'fix' (I managed to get it working by adding another
half usec to the values, hopefully making the representation become a little
over the usec boundry) the tests that obviously rely on a broken assumption:
that what you put in is what you get out. Or should we just skip these test, or
introduce some level of variance in the accepted results?

-- 
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.