[Libreoffice-bugs] [Bug 130202] New: Floating-point number round-trip lossage

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=130202

            Bug ID: 130202
           Summary: Floating-point number round-trip lossage
           Product: LibreOffice
           Version: 6.3.4.2 release
          Hardware: x86-64 (AMD64)
                OS: Linux (All)
            Status: UNCONFIRMED
          Severity: normal
          Priority: medium
         Component: Calc
          Assignee: libreoffice-bugs at lists.freedesktop.org
          Reporter: phma at bezitopo.org

Enter the following numbers and formula:
A1: 31415926535897932384
A2: 31415926535897900000
A3: =A1-A2

Expected number in A3: 32768 (not 32384 because of floating-point roundoff).
Actual number in A3: 0.

Both cells display 3.14159265358979E+019 when edited. If you try to subtract
31415926535897932384 from 31415926535897967616, the latter number is turned
into 3.1415926535898E+019, but the difference is still 0. The ulp of those
numbers is 4096; it should be possible to enter two numbers that size whose
difference is 4096.

When a floating-point binary number is converted to decimal for later
conversion back to floating-point binary, it should be converted with enough
digits that conversion to floating-point binary results in the same number. In
this case, the number should be displayed in the edit box as
3.1415926535897932E+019.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20200126/82e9b594/attachment.htm>