<html>
<head>
<base href="https://bugs.documentfoundation.org/">
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_UNCONFIRMED "
title="UNCONFIRMED - Change Calc to display in appropriate precision"
href="https://bugs.documentfoundation.org/show_bug.cgi?id=130728">130728</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>Change Calc to display in appropriate precision
</td>
</tr>
<tr>
<th>Product</th>
<td>LibreOffice
</td>
</tr>
<tr>
<th>Version</th>
<td>6.0 all versions
</td>
</tr>
<tr>
<th>Hardware</th>
<td>All
</td>
</tr>
<tr>
<th>OS</th>
<td>All
</td>
</tr>
<tr>
<th>Status</th>
<td>UNCONFIRMED
</td>
</tr>
<tr>
<th>Severity</th>
<td>normal
</td>
</tr>
<tr>
<th>Priority</th>
<td>medium
</td>
</tr>
<tr>
<th>Component</th>
<td>Calc
</td>
</tr>
<tr>
<th>Assignee</th>
<td>libreoffice-bugs@lists.freedesktop.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>jg@jguk.org
</td>
</tr></table>
<p>
<div>
<pre>There's a well known issue, Calc will display this result as 1E-04
What was expected
Expected 0.0001
0.0042
0.0043
=A2-A1
1E-04
If we expand the column it presently shows show 0.0000999999999999994
What's gone wrong
There are various ways to fix this bug, the best is to calcuate accurately, and
not display raw Double precision.
A)
Calc seems to be displaying raw double precision, and not using it's own
accurate calculation of decimal numbers.
It's pretty easy to simply have a BigNum class with an int32 and an uint32
representing the decimal precision to avoid these issues.
B)
The alternative, is to select appropriate formatting
.
eg in this case it would be #,##0.0000
Personally I favour (A), just calculate and show exactly correct values.
My $5 Casio calculator displays the result correctly. There is no reason to
ever show a user 1E-04 or 0.0000999999999999994
Version: 6.0.7.3
Build ID: 1:6.0.7-0ubuntu0.18.04.10
CPU threads: 8; OS: Linux 4.15; UI render: default; VCL: gtk3;
Locale: en-GB (en_GB.UTF-8); Calc: group</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are the assignee for the bug.</li>
</ul>
</body>
</html>