[Libreoffice-bugs] [Bug 138920] Precision issue in calc

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=138920

--- Comment #10 from Jonny Grant <jg at jguk.org> ---
(In reply to Eike Rathke from comment #9)
> We're trying to round and due to the rounding even have some wrong display
> strings resulting of the rounding to 15 decimals, but that 0.269985590000001
> *is* already rounded; so yes, the entire display string conversion from
> binary IEEE 754 to decimal needs to be reworked, but any "user expects" and
> "easy fix" claim is just off.

Thank you for you reply. Feels like it's not IEEE rounding that's the issue,
it's LibreOffice code, as you say wrong precision is being used, and is then
even in the copy paste buffer!

Why doesn't the LibreOffice code just use the correct decimal place it already
knows?
8.9818 has 4 decimal places.
9.25178559 has 8 decimal places.

Seems simple to me, use the 8 decimal places LibreOffice already knows?

Here is another example of this issue:
=9.25168559 - 8.9818


0.269885589999999

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20201215/b56ab8c7/attachment.htm>