[Libreoffice-bugs] [Bug 130202] Floating-point number round-trip lossage

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=130202

--- Comment #2 from Eike Rathke <erack at redhat.com> ---
The minus operator does an approxSub() within the order of magnitudes of the
two operands (i.e. ties/pulls to zero), as users expect 0.3-0.2-0.1 to be 0.0

If you use =RAWSUBTRACT(A1;A2) the result is 24576, which is just the "raw"
floating point subtraction of two values.

Adhoc I'm not sure if and what could be done about the input and display of
such large values, as the largest unambiguous integer value representable as
IEEE 754 double is 9007199254740991 or (2^53)-1.

Not directly related to bug 96918 as that was about displaying the
representable integer values accurately.

> When a floating-point binary number is converted to decimal for later
> conversion back to floating-point binary, it should be converted with
> enough digits that conversion to floating-point binary results in the
> same number. In this case, the number should be displayed in the edit
> box as 3.1415926535897932E+019.
That is wishful thinking. An IEEE 754 double can not be more accurate than
general 15 decimal digits, sometimes 16 and rarely 17 decimal digits. The
problem with unique strings convertible back to double is that they are even
less accurate and not what users expect. See
http://blog.reverberate.org/2016/02/06/floating-point-demystified-part2.html

You might also be interested in some other links under
https://erack.de/bookmarks/D.html#Computer_Arithmetic_and_IEEE_754 and
https://erack.de/bookmarks/D.html#010203

If you have a perfect solution I'm not aware of then please tell us.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20200218/0eba30c2/attachment-0001.htm>