[Libreoffice-bugs] [Bug 138220] calc: calculation: roundup: rounddown: fail in 7.1 alpha

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=138220

--- Comment #6 from b. <newbie-02 at gmx.de> ---
@erAck: yes, but '=ROUNDUP(-999,13;11)' shouldn't end in
-999,13000000001000000000 either... right? but it does :-(  

and it's not just 'display string', see:  

=RAWSUBTRACT(ROUNDUP(-999,13;11);-999,13) -> -0,00000000001000444172  

vs.: 

'=RAWSUBTRACT(-999,13;-999,13)' -> 0,0000000000000000

and: 

'=RAWSUBTRACT(ROUND(-999,13;2);-999,13)' -> 0,00000000000000000000

there are two distinct issues:  

1. the patch by @Winfried Donkers -
https://gerrit.libreoffice.org/c/core/+/69762 - disables accuracy for values
with more than 12 significant digits, such is a very questionable strategy, i
would suggest to undo the patch and fix the errors in the elementary functions
instead (roundsig and subtraction)), 

2. something injects a small deviation into these calculations, imho it starts
with roundsig being inaccurate, see: 

'=RAWSUBTRACT(ROUNDSIG(-999,13;12);-999,13)' -> -0,00000000000011368684  

('RoundSignificant( fX, 12, fRes );' is used in the patch)

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20210204/b5f671d2/attachment.htm>