<html> <head> <base href="https://bugs.documentfoundation.org/"> </head> <body> <div> <a class="bz_bug_link bz_status_NEW " title="NEW - calculation: basic math fail, !calc rounding wrong! dividing by inverse sometimes different from multiply, 7E10/1E-5 wrong," href="https://bugs.documentfoundation.org/show_bug.cgi?id=138360#c4">Comment # 4</a> on <a class="bz_bug_link bz_status_NEW " title="NEW - calculation: basic math fail, !calc rounding wrong! dividing by inverse sometimes different from multiply, 7E10/1E-5 wrong," href="https://bugs.documentfoundation.org/show_bug.cgi?id=138360">bug 138360</a> from <a class="email" href="mailto:erack@redhat.com" title="Eike Rathke <erack@redhat.com>"> Eike Rathke</a> <pre>(In reply to b. from <a href="show_bug.cgi?id=138360#c2">comment #2</a>) > 0,5 ULP precision at > granularity borders ... produces overlapping 'definition ranges'? as e.g. > 0,5 ULP of 2^53 reaches 'down' to including '2^53 - 1' Say what? I don't understand what you are trying to tell. > while 2^53-1 itself > covers the range from 2^53-1-0,5 to 2^53-1+0,4999~, It does not cover a range. In an IEEE-754 binary64 magnitude with a distance precision of 1, i.e. values in the interval [2^52,2^53) are integers, any value or result that would lie in between integer values is tied to the nearest representable value. There is no range between two integer values in that interval. Btw, it would help people and specifically developers reading (and maybe trying) your examples if you used '.' decimal point instead of ',' decimal comma. > that leads to some > ambiguity as e.g. 2^53-1+0,4 is falling into both 'ULP-precision-ranges', is > that handeled in a meaningful way? What both "ULP-precision-ranges"? 2^53-1+0.4 results in 2^53-1 as that is the nearest representable value. There is no ambiguity.</pre> </div> <hr> You are receiving this mail because: <ul> <li>You are the assignee for the bug.</li> </ul> </body> </html>