[Piglit] [PATCH] Add option '-r' to allow resume test after first test match filter

[Kenneth Graunke]

On 08/29/2011 08:47 PM, Ian Romanick wrote:
> On 08/27/2011 07:57 AM, Shuang He wrote:
>> This is useful when we want to resume test after tests are blocked by one case
> 
> I don't think this is the mechanism that we really want.  It's certainly
> not the mechanism that *I* want.  What would be better is to have it
> automatically figure out where to resume and continue appending to the
> existing results file.
> 
> In order to do this, I think it would have to emit some sort of "test
> started" entry to the log.  The resume would just skip any tests that
> already have a "started" entry in the log.  pigit-summary-html.py would
> also need to generate a new result type for tests that started but
> didn't finish.
> 
> I know Chad and Paul had some ideas about this, so you should talk with
> them before going further.

My thinking is:

1. Add pre-test and post-test hooks.  These would simply be python
   functions that people could fill out with whatever driver-specific
   checks they want to do.

2. Implement a post-test hook that implements checking for GPU hangs.
   This would obviously have to be done in a driver specific way.
   - If the GPU is hung, update the results for that test to "GPU hang".
   - Terminate the piglit run with an appropriate explanation.

3. Add a --resume/-r option that causes piglit-run.py to read the JSON
   results file back in (rather than overwriting it) and filter out any
   tests that already have results written.  Begin running the rest of
   the tests, appending the results to the existing file.

This shouldn't be too hard, and ought to work for everything except hard
system lockups (kernel OOPS)...and those are pretty rare.

The nice thing about this is that you could just re-run the same command
until you get a complete set of results.  You wouldn't need to manually
decide what to skip or where to start.

--Kenneth