[igt-dev] [PATCH i-g-t v3 1/2] tests/kms_flip: Set duration for subtest from command line

[Kahola, Mika]

On Thu, 2018-11-08 at 14:11 +0100, Maarten Lankhorst wrote:
> Op 08-11-18 om 13:45 schreef Kahola, Mika:
> > On Thu, 2018-11-08 at 12:47 +0100, Maarten Lankhorst wrote:
> > > Op 09-08-18 om 12:12 schreef Mika Kahola:
> > > > To reduce the execution time of kms_flip test on CI, let's move
> > > > subtest
> > > > duration parameter as command line option. The default subtest
> > > > duration
> > > > is 3 seconds for test that require jitter computation and for
> > > > the
> > > > rest
> > > > of the subtests are run only once.
> > > > 
> > > > v2: Run each subtest only once (default action)
> > > > v3: Reduces default timeout for tests that require jitter
> > > > computation (Ville)
> > > > 
> > > > Signed-off-by: Mika Kahola <mika.kahola at intel.com>
> > > 
> > > I think this is the wrong approach. What we really want to do is
> > > kill
> > > tests from kms_flip
> > > instead and move them to separate places.
> > > 
> > > Killing off all the interruptible tests would save 50% of the
> > > time.
> > > So all we have to do is
> > > making sure that we have tests that test the missing ioctl's in
> > > in
> > > kms_atomic_interruptible,
> > > and we would save 50% of the time.
> > 
> > kms_atomic_interruptible would probably be more logical place for
> > interrupt tests. I agree that it would reduce the execution time of
> > the
> > kms_flip test but in the end of the day we still need to run those
> > interruptible tests and therefore we would end up increasing the
> > execution time of the kms_atomic_interruptible test.
> > 
> > Do you feel that we could have overall reduction in test execution
> > if
> > we move these interruptible tests out from kms_flip? Maybe kms_flip
> > readability would improve and therefore worth the effort?
> 
> The kms_flip ones just double the execution time of the tests, and
> don't result in more coverage.
> 
> Testing the various ioctl's in a controlled fashion would result in
> better coverage and lower execution time.
> Just look at how kms_atomic_interruptible is structure, it should be
> easy to add more tests to it and actually
> guarantees that the ioctl being tested is interrupted.
> 
> I definitely hope that removing those tests increase readability,
> because of all the unrelated API test special
> cases being removed, and only actual flip related tests remaining.
Ok. I prepare a patch that removes interruptible tests from kms_flip.
Let's see how kms_flip looks after that change. I also look into adding
the removed tests to kms_atomic_interruptible. 
 
> 
> ~Maarten