[igt-dev] [PATCH i-g-t v3 1/2] tests/kms_flip: Set duration for subtest from command line

[Mika Kahola]

On Thu, 2018-08-09 at 11:20 +0100, Chris Wilson wrote:
> Quoting Mika Kahola (2018-08-09 11:12:33)
> > 
> > To reduce the execution time of kms_flip test on CI, let's move
> > subtest
> > duration parameter as command line option. The default subtest
> > duration
> > is 3 seconds for test that require jitter computation and for the
> > rest
> > of the subtests are run only once.
> I disagree with having tests that change between invocations. It's
> far
> too easy for one to make a mistake and not configure the test
> correctly,
> completely fairly to reproduce whatever bug one was seeking.
> 
So, should I remove the option of setting the duration for the
subtests? Originally, we have timeouts ranging from 0 to 30 seconds,
depending on a test. Based on my tests on GLK, I found out that tests
that require some jitter computation yields the similar results if we
run it for 3 or 30 seconds. When running less than 3 seconds there are
differences in jitter results. That's why I suggested that 3 seconds
would be a sweetspot for the subtests.

Should I just use 3 seconds timeout for all subtests and discard the
possibility to set the timeout? 

> If you think both durations are interesting for different aspects,
> test
> both.
> -Chris
-- 
Mika Kahola - Intel OTC