[Piglit] Silent Python exceptions
Jose Fonseca
jfonseca at vmware.com
Mon Dec 7 05:26:21 PST 2015
I just realized piglit has been producing lots of spurious failures.
The weird thing is that neither the JUnit nor JSON backend give any
explanation of why.
Inspecting results.json directly I found some clues:
"tests": {
"spec at glsl-1.10@execution at built-in-functions@fs-op-sub-vec4-float": {
"returncode": null,
"result": "fail",
"subtests": {
"__type__": "Subtests"
},
"out": "",
"dmesg": "",
"exception": "<type 'exceptions.ValueError'>invalid literal
for float(): 4.4.0",
"err": "",
"__type__": "TestResult",
"environment": "PIGLIT_SOURCE_DIR=\".\"
PIGLIT_PLATFORM=\"glx\"",
"command":
"/home/jfonseca/work/vmware/tests/piglit/bin/shader_runner
/home/jfonseca/work/vmware/tests/piglit/generated_tests/spec/glsl-1.10/execution/built-in-functions/fs-op-sub-vec4-float.shader_test
-auto",
"time": {
"start": 1449494145.586955,
"end": 0.0,
"__type__": "TimeAttribute"
}
}
},
So the problem is that some code is trying to do `float("4.4.0")`.
Is there any way to tell piglit framework to not trap and hide Python
exceptions, so they can be debugged?
Also, I'd much prefer that Python exceptions would cause the abortion of
whole testsuite, instead of being captured into the results.
IMO, the framework should be (or aim to be) rock solid -- exceptions in
the framework are not the same as test failures -- but rather
catastrophic failures which should never happen in practice.
Jose
More information about the Piglit
mailing list