[Libreoffice-bugs] [Bug 138424] calc: calculation: rounding of results: bad influence of unspecific rounding if terms in different dyadic ranges

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=138424

--- Comment #2 from b. <newbie-02 at gmx.de> ---
hello @erAck, 

sorry, maybe i (or you?) was lost in the nested 'non' and 'not's of: 

'Anyway, you hit an effect of the approxAdd() function used to normally
eliminate accuracy limitations when adding/subtracting not non-ambiguous
representable integer values with opposite signs of similar magnitudes by
scaling to 2^-48 and thus rounding off the least significant 4 bits to yield a
result of 0.0, i.e. to let 0.3-0.2-0.1==0.0.', 

cited acc.
https://ask.libreoffice.org/en/question/274247/calc-wrong-calculation-would-like-a-recheck/,

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20201202/6921a773/attachment.htm>