[Intel-gfx] [PATCH] intel-ci: Add gem_exec_suspend/basic-S3/S4-devices to BAT

[Daniel Vetter]

On Mon, Oct 17, 2016 at 05:46:50PM +0300, Imre Deak wrote:
> On ma, 2016-10-17 at 16:32 +0200, Daniel Vetter wrote:
> > On Mon, Oct 17, 2016 at 03:39:04PM +0300, Imre Deak wrote:
> > > Add gem_exec_suspend/basic-s3-devices and basic-s4-devices subtests
> > > to
> > > BAT. At the same time remove basic-s4 from the list, which is atm
> > > implicitly disabled via HIBERNATION=n in kconfig. We would need at
> > > least
> > > basic S4 coverage provided by basic-s4-devices, which requires
> > > HIBERNATION=y.
> > > 
> > > Signed-off-by: Imre Deak <imre.deak at intel.com>
> > 
> > What's the impact on BAT runtime with this? 
> 
> I measured 8 sec for S3-devices and 9 sec for S4-devices on my APL.
> 
> > Afaik we're already over budget ... Where do you safe the time to
> > afford this?
> 
> I didn't, but we don't have any S4 coverage in CI atm and it's the
> minimum that can be added. The S3-devices subtest is not critical,
> although it would be useful for cases where we wouldn't get any logs
> for the full S3 test.

Yeah, I don't think a reduced S3 test in BAT is useful if  we have the
full one already. It's a good test for debugging, but not within the very
tight constraints we have for BAT.

Re S4: If it indeed improves coverage (i.e. calls our shutdown hooks and
all these S4-only callbacks) then adding it to BAT sounds reasonable.
Still there's the issue of where to get the machine time from. I really do
think you need to first trade in some speed-up here (or throw out some
other tests) before you can add more tests.

And yes I know that's not fun, but until we have CI running a more
complete test set (after BAT has done the initial sanity checking, to
avoid wasting machine time on broken patches), that's the reality we have
:(

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch