[Libreoffice-bugs] [Bug 138424] New: calc: calculation: rounding of results: bad influence of unspecific rounding if terms in different dyadic ranges

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=138424

            Bug ID: 138424
           Summary: calc: calculation: rounding of results: bad influence
                    of unspecific rounding if terms in different dyadic
                    ranges
           Product: LibreOffice
           Version: 3.5.1 release
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: medium
         Component: Calc
          Assignee: libreoffice-bugs at lists.freedesktop.org
          Reporter: newbie-02 at gmx.de

Description:
calc doe's some 'tie to zero' rounding, acc. @erAck intended for integer values
but imho affecting all calculations, 

this rounding sometimes has evil effects, e.g.
'=(2^53+31-2^53)+(2^53-31-2^53)+(2^53+31-2^53)+(2^53-31-2^53)' should
mathematical result in '0' (the subterms in brackets cancel each other out),
'fp-math' calculates '2' because above 2^53 IEEE 754 is limited to the 'even
integers' and some small rounding steps in, calc calculates '64' reg.
unneccessary rough rounding and either different effect of it below and above a
magnitude range border (2^53), or different effect for '+' vs. '-', 

Steps to Reproduce:
1. calculate '=(2^53+31-2^53)+(2^53-31-2^53)+(2^53+31-2^53)+(2^53-31-2^53)'
2.
3.

Actual Results:
64

Expected Results:
'0' or maximal tolerable '2', 


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 7.1.0.0.alpha1+ (x64)
Build ID: b61bf7c7cfcf97a5ade6d130873af146670bc2ee
CPU threads: 8; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default;
VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc:

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20201123/694dfbb6/attachment.htm>