[Piglit] Silent Python exceptions

Jose Fonseca jfonseca at vmware.com
Mon Dec 7 05:26:21 PST 2015


I just realized piglit has been producing lots of spurious failures.

The weird thing is that neither the JUnit nor JSON backend give any 
explanation of why.

Inspecting results.json directly I found some clues:

     "tests": {
 
"spec at glsl-1.10@execution at built-in-functions@fs-op-sub-vec4-float": {
             "returncode": null,
             "result": "fail",
             "subtests": {
                 "__type__": "Subtests"
             },
             "out": "",
             "dmesg": "",
             "exception": "<type 'exceptions.ValueError'>invalid literal 
for float(): 4.4.0",
             "err": "",
             "__type__": "TestResult",
             "environment": "PIGLIT_SOURCE_DIR=\".\" 
PIGLIT_PLATFORM=\"glx\"",
             "command": 
"/home/jfonseca/work/vmware/tests/piglit/bin/shader_runner 
/home/jfonseca/work/vmware/tests/piglit/generated_tests/spec/glsl-1.10/execution/built-in-functions/fs-op-sub-vec4-float.shader_test 
-auto",
             "time": {
                 "start": 1449494145.586955,
                 "end": 0.0,
                 "__type__": "TimeAttribute"
             }
         }
     },


So the problem is that some code is trying to do `float("4.4.0")`.


Is there any way to tell piglit framework to not trap and hide Python 
exceptions, so they can be debugged?


Also, I'd much prefer that Python exceptions would cause the abortion of 
whole testsuite, instead of being captured into the results.

IMO, the framework should be (or aim to be) rock solid -- exceptions in 
the framework are not the same as test failures -- but rather 
catastrophic failures which should never happen in practice.


Jose


More information about the Piglit mailing list