tests sc_array_functions_test fail due to floating point precision

[Milton Vandersloot]

Dear LibreOffice developers

I was told to report this bug here[2]. So here we go:

The test ArrayFunctionsTest::testArrayFormulasFODS (files sc/qa/unit/functions_test.cxx, sc/qa/unit/functions_array.cxx) fails the test document sc/qa/unit/data/functions/array/fods/linest.fods. Error message in the compile run is (Note that 1 = TRUE, 0 = FALSE in the document)


functions_test.cxx:43:Assertion
Test name: ArrayFunctionsTest::testArrayFormulasFODS
double equality assertion failed
- Expected: 1
- Actual  : 0
- Delta   : 1e-14


The issue seems to be (again) floating point precision. Indeed, I've compiled LibreOffice with FMA (fused multiply-add) instructions, if I disable them, i.e. passing -ffp-contract=off to GCC, then the tests succeed. Note that FMA has slightly different floating point precision than ordinary plus and mult [0, 1].

I've also opened the other test documents in sc/qa/unit/data/functions/array/fods/ using LibreOffice. The documents which fail are linest.fods and also logest.fods. The others succeed.


Some mentionable commits:

* Add logest.fods test: https://cgit.freedesktop.org/libreoffice/core/commit/?id=588ebef9e8d1e2c703b79f51f2a1c8004cd7d96d
* Add linest.fods test: https://cgit.freedesktop.org/libreoffice/core/commit/?id=0523e11a2530ed101c580f93f94a4180e9dc0c9e
* Disable tests on 32-bit x86-linux due to floating point weirdness: https://cgit.freedesktop.org/libreoffice/core/commit/?id=b9a27d5856f60688456762bfcc29c38670009254

Best
Milton

[0] https://en.wikipedia.org/wiki/Fused_multiply-add
[1] https://github.com/uclouvain/openjpeg/issues/1017
[2] https://bugs.documentfoundation.org/114051