[Libreoffice-bugs] [Bug 130725] ImpSvNumberInputScan::StringToDouble may produce inaccurate result

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=130725

--- Comment #31 from b. <newbie-02 at gmx.de> ---
@Mike Kaganski's comment is not 'off topic', for the cause itself it's good
when even two stubborn opposites like him and me discuss things with each
other, 

he'd ask for a patch, i'm not a coder therefore i can only provide ideas or
recipes,  it's only one of plenty possible solutions (to propose the best i'd
need calc to calculate with less inaccuracies in e.g. rounding):  

calculating '=1,0000000000000007 - 1,0000000000000004' is 'difficult' with
fp-math and spreadsheets, the result '2,22044604925031E-16' is useless, 

calculating '=1 - 1 + 0,0000000000000007 - 0,0000000000000004' instead is some
more effort, but the result '3.0000000000000004E-16' is much better, rounded to
16 digits it's almost 'decimal accurate', 

such things don't require 'infinite ... in software ...' but only a change of
thinking from 

'fp-math is inaccurate, it has to stay that way' 

to 

'ok, there are some difficulties, what is the best way to deal with them'

above sample is solveable with a simple formula in a sheet: 

B2: '1,0000000000000007 (text here and 'value()' in the formula to get 17
digits visible) 
B3: '1,0000000000000004
B4:
'=INT(VALUE(B2))-INT(VALUE(B3))+ROUND(RAWSUBTRACT(VALUE(B2);INT(VALUE(B2)));16)-ROUND(RAWSUBTRACT(VALUE(B3);INT(VALUE(B3)));16)'
-> 3,00000000000000000000E-16

if you'd like the formula more general: 
'=INT(VALUE(B2))-INT(VALUE(B3))+ROUND(RAWSUBTRACT(VALUE(B2);INT(VALUE(B2)));16-LOG10(B2))-ROUND(RAWSUBTRACT(VALUE(B3);INT(VALUE(B3)));16-LOG10(B3))' 

solves plenty samples discussed in forums and 'ask-sites' as '=1234,12 - 1234'
or '=32000,12 - 31000,99'

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20201216/33d51853/attachment.htm>