[Piglit] [PATCH 0/2] Add timeout support

[Daniel Vetter]

On Wed, Mar 19, 2014 at 09:15:17AM -0700, Tom Stellard wrote:
> On Wed, Mar 19, 2014 at 03:43:13PM +0000, Thomas Wood wrote:
> > The following two patches add support for a timeout status and then implement a
> > timeout mechanism in igt.py.
> > 
> > The "timeout" status is currently positioned just after "pass" in the status
> > ordering. This means that a timeout status does not introduce a regression from
> > other failure states, since the actual test status is unknown, but a change
> > from timeout to fail is considered a regression.
> >
> 
> What is the rationale for making timeout->fail a regression?  Based on
> the way I use piglit, I think it would be better to have the reverse (i.e
> fail->timeout is a regression), because in a real application it seems
> like hanging the system is much worse than mis-rendering something.  Also,
> the normal work flow for fixing a hanging test is first fix the hang
> and then fix the test, so I think it makes sense to treat timeout->fail
> as a fix.

Well in igt timeout is somewhat a normal condition on some machines
because on badly underpowered platforms some of the stress-tests just take
forever. We've tried to tune ofc but there's still tests that take
forever.

I think you make a good case that in general timeout should be treated
more severe than fail, so I'm ok if we switch this around. And we also
have that problem ofc when the driver hangs somewhere.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch