Acceptable outcomes of SwarmSolverTest::testUnconstrained
Stephan Bergmann
sbergman at redhat.com
Tue Feb 27 07:53:52 UTC 2018
SwarmSolverTest::testUnconstrained in sccomp/qa/unit/SwarmSolverTest.cxx
has already been weakened in the past,
<https://cgit.freedesktop.org/libreoffice/core/commit/?id=1fa761af825641da5c87f80c2a17135f92418960>
"Ridiculously large delta for SwarmSolverTest::testUnconstrained for
now" and
<https://cgit.freedesktop.org/libreoffice/core/commit/?id=0c3444c9bcee093ad5976af8948138e6f2a97706>
"Weaken SwarmSolverTest::testUnconstrained even further for now". The
first one has the following in its commit message: "suggestion by Tomaž
Vajngerl was: 'Let's adapt the delta for now. Generally anything close
to 3 should be acceptable as the algorithm greatly depends on random
values.'"
Now <https://ci.libreoffice.org/job/lo_ubsan/833/console> failed with
> /home/tdf/lode/jenkins/workspace/lo_ubsan/sccomp/qa/unit/SwarmSolverTest.cxx:106:(anonymous namespace)::SwarmSolverTest::testUnconstrained
> double equality assertion failed
> - Expected: 3
> - Actual : 94.6605927051114
> - Delta : 0.9
Is that also an acceptable outcome, or does it indicate a bug somewhere
that would need to be fixed? What good is a test whose success
criterion is the result of ad-hoc guesswork, instead of being determined
precisely up-front when the test was written?
Can that test please be fixed properly, so that it would be actually useful?
More information about the LibreOffice
mailing list