<html>
<head>
<base href="https://bugs.documentfoundation.org/">
</head>
<body>
<p>
<div>
<b><a class="bz_bug_link
bz_status_NEW "
title="NEW - Precision issue in calc"
href="https://bugs.documentfoundation.org/show_bug.cgi?id=138920#c12">Comment # 12</a>
on <a class="bz_bug_link
bz_status_NEW "
title="NEW - Precision issue in calc"
href="https://bugs.documentfoundation.org/show_bug.cgi?id=138920">bug 138920</a>
from <span class="vcard"><a class="email" href="mailto:jg@jguk.org" title="Jonny Grant <jg@jguk.org>"> <span class="fn">Jonny Grant</span></a>
</span></b>
<pre>(In reply to Eike Rathke from <a href="show_bug.cgi?id=138920#c11">comment #11</a>)
<span class="quote">> You seem to not understand. Once the value is in the IEEE 754 binary format
> there are no "8 decimal places" to use.
>
> It's about sufficient precision and rounding when converting from binary
> floating-point to the decimal display string.
>
> See also <a href="https://erack.de/bookmarks/D.html#010203">https://erack.de/bookmarks/D.html#010203</a>
> Or if you want to dive deep into then
> <a href="https://erack.de/bookmarks/D.html#Computer_Arithmetic_and_IEEE_754">https://erack.de/bookmarks/D.html#Computer_Arithmetic_and_IEEE_754</a></span >
Dear Eike
My apologies if I was not clear. I only refer to the decimal display of the
IEEE 754 number. More specifically, when I press <Ctrl>+C, and <Ctrl>+V in an
email, i see 0.269985590000001 I'm only referring to the string conversion of
this number which is stored in floating-point format by Calc.
Calc needs to display as a string an appropriate number of decimal places to
for the result, and the copy&paste buffer should contain the same, that should
also be used so that copy&paste works as expected.
Here is an even simpler example :-
1) Type =10.1 - 9.9
2) Observe 0.2
3) <Ctrl>+C
4) Move to your text editor and press <Ctrl>+V
5) Observe 0.199999999999999
6) Switch back to Calc, resize the column
7) Observe 0.199999999999999
I'll attach another screenshot
Can the string conversion function take a hint on how many decimal places are
needed? In this example, it will never be more than 1 decimal place needed
right?</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are the assignee for the bug.</li>
</ul>
</body>
</html>