tests sc_array_functions_test fail due to floating point precision
winfried.libreoffice at gmail.com
Sun Dec 10 10:34:44 UTC 2017
>>> I've also opened the other test documents in
>>> sc/qa/unit/data/functions/array/fods/ using LibreOffice. The documents which
>>> fail are linest.fods and also logest.fods. The others succeed.
>> If your build completed ignoring these tests (i.e. produced a
>> usable office under ./instdir/), could you please open the
>> failing documents and tell us the cell addresses on the second
>> sheet where tests fail and the actual values they produced?
> Below are the numbers. Some remarks: Many errors are in the
> range of 1E-11 to 1E-12 but some are way larger >1E30. I assume
> this is due some division of a very small error.
> Has someone done a/some theoretical error analysis of the
> algorithms involved? (e.g. numeric (forward/backward) stability
> and error estimates similar to what is well-known for LU
> factorization and other algorithms.)
> Who/What provided the reference values -- or reformulated --
> what error do these numbers have (wrt to the mathematical
Looking at the numbers it would be better to use ROUNDSIG instead of ROUND.
ROUND(1.2345E30,4) has no effect, ROUNDSIG(1.2345E30,4) rounds to 4
significant digits, i.e. 1.234E30.
ROUNDSIG was only added to Calc last February, so most unit test
documents don't use it (yet).
More information about the LibreOffice