[Libreoffice-bugs] [Bug 129606] Drag-fill series produces values like 6.00000000000001

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=129606

--- Comment #15 from b. <newbie-02 at gmx.de> ---
again i stumbled over a simple dragfill problem, (or two?), 

1.: 

B3: 6,9
B4: 6,8
select both cells and dragfill down, numbers like 6,29999999999999 starting in
B9, (visible in the input line or when displaying with more decimals), 
accumulating, e.g. to a deviation of -0,00000000000013 at -6,0

ver. 7.1 2020-09-05, win7x64, 

reopen? new bug? or forget about decimal math precision? 

(assumption - only assumption! - correcting the difference used as argument by
'fixed' rounding fails as the arguments loose precision with increasing
magnitude, and thus the deviation of the difference increases) 


2.: 

for this case the workaround with [sheet - fill cells] also fails, 0,1 and -0,1
are filled with 0,0999999999999996 and -~ instead, 

same ver. as above, 

covered with this bug? reopen? new bug? or forget about decimal math precision? 

(assumption - only assumption! - the errors 'more likely' become apparent when
applied to ranges with better (decimal or binary) precision, will say small
numbers)


suggestion: to find the 'true' increment between two decimal values (e.g. 6,9
and 6,8) you may legally round the decimal calculated from the 'precise'
fp-value (but 'fuzzy' regarding decimal - binary - decimal conversion) to the
max of the decimal places of the arguments (one decimal place in the 6,9 - 6,8
case), (is 'flexible rounding' an appr. term for that?), reg. school math rules
the decimal places of the result of '+' and '-' operations are limited like
that,  

with that you get an increment value as exact as when keyed into [sheet - fill
cells] and can concentrate on the 'fill cells' error, 

ok?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20200921/c1eb4a1c/attachment-0001.htm>