Acceptable outcomes of SwarmSolverTest::testUnconstrained

Stephan Bergmann sbergman at redhat.com
Tue Feb 27 07:53:52 UTC 2018


SwarmSolverTest::testUnconstrained in sccomp/qa/unit/SwarmSolverTest.cxx 
has already been weakened in the past, 
<https://cgit.freedesktop.org/libreoffice/core/commit/?id=1fa761af825641da5c87f80c2a17135f92418960> 
"Ridiculously large delta for SwarmSolverTest::testUnconstrained for 
now" and 
<https://cgit.freedesktop.org/libreoffice/core/commit/?id=0c3444c9bcee093ad5976af8948138e6f2a97706> 
"Weaken SwarmSolverTest::testUnconstrained even further for now".  The 
first one has the following in its commit message: "suggestion by Toma┼ż 
Vajngerl was: 'Let's adapt the delta for now. Generally anything close 
to 3 should be acceptable as the algorithm greatly depends on random 
values.'"

Now <https://ci.libreoffice.org/job/lo_ubsan/833/console> failed with

> /home/tdf/lode/jenkins/workspace/lo_ubsan/sccomp/qa/unit/SwarmSolverTest.cxx:106:(anonymous namespace)::SwarmSolverTest::testUnconstrained
> double equality assertion failed
> - Expected: 3
> - Actual  : 94.6605927051114
> - Delta   : 0.9

Is that also an acceptable outcome, or does it indicate a bug somewhere 
that would need to be fixed?  What good is a test whose success 
criterion is the result of ad-hoc guesswork, instead of being determined 
precisely up-front when the test was written?

Can that test please be fixed properly, so that it would be actually useful?


More information about the LibreOffice mailing list