[Piglit] [PATCH 0/2] Add timeout support

[Tom Stellard]

On Wed, Mar 19, 2014 at 03:43:13PM +0000, Thomas Wood wrote:
> The following two patches add support for a timeout status and then implement a
> timeout mechanism in igt.py.
> 
> The "timeout" status is currently positioned just after "pass" in the status
> ordering. This means that a timeout status does not introduce a regression from
> other failure states, since the actual test status is unknown, but a change
> from timeout to fail is considered a regression.
>

What is the rationale for making timeout->fail a regression?  Based on
the way I use piglit, I think it would be better to have the reverse (i.e
fail->timeout is a regression), because in a real application it seems
like hanging the system is much worse than mis-rendering something.  Also,
the normal work flow for fixing a hanging test is first fix the hang
and then fix the test, so I think it makes sense to treat timeout->fail
as a fix.

-Tom

> Thomas Wood (2):
>   framework: add support for a "timeout" status
>   tests/igt.py: stop tests after 10 minutes
> 
>  framework/log.py                |  2 +-
>  framework/status.py             | 20 +++++----
>  framework/tests/status_tests.py |  4 +-
>  templates/index.css             |  5 ++-
>  tests/igt.py                    | 89 +++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 108 insertions(+), 12 deletions(-)
> 
> -- 
> 1.8.5.3
> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit