<html>
<head>
<base href="https://bugs.documentfoundation.org/">
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_UNCONFIRMED "
title="UNCONFIRMED - calculation: basic math fail, !calc rounding wrong! dividing by inverse sometimes different from multiply, 7E10/1E-5 wrong,"
href="https://bugs.documentfoundation.org/show_bug.cgi?id=138360">138360</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>calculation: basic math fail, !calc rounding wrong! dividing by inverse sometimes different from multiply, 7E10/1E-5 wrong,
</td>
</tr>
<tr>
<th>Product</th>
<td>LibreOffice
</td>
</tr>
<tr>
<th>Version</th>
<td>6.1.6.3 release
</td>
</tr>
<tr>
<th>Hardware</th>
<td>All
</td>
</tr>
<tr>
<th>OS</th>
<td>All
</td>
</tr>
<tr>
<th>Status</th>
<td>UNCONFIRMED
</td>
</tr>
<tr>
<th>Severity</th>
<td>normal
</td>
</tr>
<tr>
<th>Priority</th>
<td>medium
</td>
</tr>
<tr>
<th>Component</th>
<td>Calc
</td>
</tr>
<tr>
<th>Assignee</th>
<td>libreoffice-bugs@lists.freedesktop.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>newbie-02@gmx.de
</td>
</tr></table>
<p>
<div>
<pre>Description:
looking for some accuracy problems i spotted that calc's 'round' fails for
plenty values,
as similar happened with a user defined round routine i investigated further
and came to the point that:
'=7000000000000000*0,00001/0,00001' fails (6999~), while
'=7000000000000000/100000*100000' holds,
acc. school math the results should be equal, 'weitz' tells me 70000000000 /
0,00001 = 6,9999~,
question 1: weitz and calc may share some basic problem - buggy library, wrong
compiler options or similar,
question 2: if the error is 'unavoidable' would it be a good idea to use the
better of both formulas in calc? i 'assume' calcs rounding routine uses the
'wrong' of both formulas and thus fails sometimes ...
(my 'user space' rounding routine works fine since i exchanged that part,
before it had similar errors as calc),
question 3: may it be possible that this or similar errors, which can be easily
circumvented in the same or a similar way, are the cause of many other
'inaccuracies' in calc?
(fp-math isn't 'decimal-safe' as well as decimal math isn't 'real-world-safe',
but avoidable problems should be avoided and not sorted out as 'fp-problem, it
has to be that way',
earlier versions of calc were 'better', so i think that 'better' is possible,
didn't do extensive testing, but on a first glance even ex$el looks better,
and i think it is important, such elementary errors affect many other
calculations ...
Steps to Reproduce:
1. key '=ROUND(5e15+1;9)' in a cell,
2. hit enter,
3. observe result 5000000000000002,
4. try to explain this to a user with simple math knowledge
5. key '=70000000000/0,00001' in a cell,
6. observe result 6999999999999999,
7. if you want to say 'notabug, this is just fp-maths, learn first why it has
to be that way':
8. try the same steps in calc ver. 4.1.6.2, it is not! unavoidable,
9. call it a regression,
Actual Results:
5000000000000002,
6999999999999999
Expected Results:
5000000000000001,
7000000000000000
Reproducible: Always
User Profile Reset: Yes
Additional Info:
Version: 7.1.0.0.alpha1+ (x64)
Build ID: b61bf7c7cfcf97a5ade6d130873af146670bc2ee
CPU threads: 8; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default;
VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc:</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are the assignee for the bug.</li>
</ul>
</body>
</html>