[Libreoffice-bugs] [Bug 130725] New: ImpSvNumberInputScan::StringToDouble may produce inaccurate result

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=130725

            Bug ID: 130725
           Summary: ImpSvNumberInputScan::StringToDouble may produce
                    inaccurate result
           Product: LibreOffice
           Version: unspecified
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: medium
         Component: Calc
          Assignee: libreoffice-bugs at lists.freedesktop.org
          Reporter: mikekaganski at hotmail.com
                CC: erack at redhat.com

In Calc, put "0.0042" to A1, "0.0043" to A2, "=A2-A1" to A3; make sure to see
19 decimal places in A3.

Expected result:
> 0.0001000000000000000
Actual result:
> 0.0000999999999999994

The problem here is the entered string in A1 has been converted to double
0.0042000000000000006, which is not the nearest double-precision value for
decimal "0.0042": 0.0041999999999999997 is. The conversion of the entered
string to double happens in ImpSvNumberInputScan::StringToDouble, which handles
integer and fractional parts of the string separately, and then multiplies the
fractional part by a power 10 exponent. As shown, this does not guarantee the
closest possible representation, resulting in immediately observable
inaccuracies in following calculations.

A robust conversion is needed here.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20200217/fded735c/attachment.htm>