[Libreoffice-bugs] [Bug 134669] Calculation should show ZERO, but shows E-12 number

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=134669

Eike Rathke <erack at redhat.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |NOTABUG
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #2 from Eike Rathke <erack at redhat.com> ---
Repeated addition of binary floating point values leads to inaccuracy if values
aren't exactly representable, that's a fact of the IEEE 754 double format. If
in your document you widen columns D, F and H you'll see that already the
values in D22, F39 and H43 start to be slightly off. There are two measures to
calculate such series of repeated addition correctly, either round each
calculation to the desired decimals, here ROUND(...;2), or format all values to
two decimals with a number format 0.00 (which in this case already *displays*
the end result as 0.00) and maybe you'll also have to switch on General
Calculations "Precision as shown" under Tools -> Options -> Calc -> Calculate
if the repeated series gets longer with accumulated inaccuracy.

See also https://erack.de/bookmarks/D.html#010203

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20200709/5b467c31/attachment.htm>