[Piglit] [PATCH] Add option '-r' to allow resume test after first test match filter

[Shuang He]

On 2011/8/31 4:44, Kenneth Graunke wrote:
> On 08/29/2011 08:47 PM, Ian Romanick wrote:
>> On 08/27/2011 07:57 AM, Shuang He wrote:
>>> This is useful when we want to resume test after tests are blocked by one case
>> I don't think this is the mechanism that we really want.  It's certainly
>> not the mechanism that *I* want.  What would be better is to have it
>> automatically figure out where to resume and continue appending to the
>> existing results file.
>>
>> In order to do this, I think it would have to emit some sort of "test
>> started" entry to the log.  The resume would just skip any tests that
>> already have a "started" entry in the log.  pigit-summary-html.py would
>> also need to generate a new result type for tests that started but
>> didn't finish.
>>
>> I know Chad and Paul had some ideas about this, so you should talk with
>> them before going further.
> My thinking is:
>
> 1. Add pre-test and post-test hooks.  These would simply be python
>     functions that people could fill out with whatever driver-specific
>     checks they want to do.

This is what I was thinking of to have. It would be very great to have 
it. Pre-test and post-test hooks
are important for pre-condition and post-condition checks for test case.

>
> 2. Implement a post-test hook that implements checking for GPU hangs.
>     This would obviously have to be done in a driver specific way.
>     - If the GPU is hung, update the results for that test to "GPU hang".
>     - Terminate the piglit run with an appropriate explanation.

I was also thinking about to add timeout control to piglit, since I have 
met many times, that it will stuck at one test case for a very long time 
(at least I can't wait so long).
At present, we're using another guard process running side by side to 
apply timeout policy

>
> 3. Add a --resume/-r option that causes piglit-run.py to read the JSON
>     results file back in (rather than overwriting it) and filter out any
>     tests that already have results written.  Begin running the rest of
>     the tests, appending the results to the existing file.
>
> This shouldn't be too hard, and ought to work for everything except hard
> system lockups (kernel OOPS)...and those are pretty rare.

I don't know if piglit is assuring all test result are recorded in the 
sync way. If one case hang the system, and we can resume test after that 
case, then all of those should work. Or it may repeatedly running the 
system hang case :)

>
> The nice thing about this is that you could just re-run the same command
> until you get a complete set of results.  You wouldn't need to manually
> decide what to skip or where to start.

That's great to work this way

Thanks
     --Shuang

>
> --Kenneth