[Intel-gfx] [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

[Chris Wilson]

On Fri, May 27, 2016 at 01:31:10PM +0100, Tvrtko Ursulin wrote:
> 
> On 27/05/16 13:16, Chris Wilson wrote:
> >On Fri, May 27, 2016 at 01:10:07PM +0100, Tvrtko Ursulin wrote:
> >>
> >>On 27/05/16 12:58, Chris Wilson wrote:
> >>>On Fri, May 27, 2016 at 02:50:34PM +0300, Marius Vlad wrote:
> >>>>Signed-off-by: Marius Vlad <marius.c.vlad at intel.com>
> >>>
> >>>Nak. It's a race detector. Please suggest how to increase detection
> >>>rates.
> >>
> >>As a more or less well know TV personality would say - "it's better
> >>than nothing"! :))
> >>
> >>Seriously, under the new rules this is all we can do. Full
> >>gem-close-race will (will it?) run in the nightly run so a little
> >>bit more pain while bisecting breakages but those are the rules. We
> >>can give them a go and see how it works out.
> >
> >Wrong approach. Right approach would be to add a new test that reliably
> >detected a checklist of the most common races in under 1s. Nerfing a
> >test to make it useless makes BAT equally useless.
> >
> >So we are giving up on BAT?
> 
> If it is not possible to stuff everything into the allocated budget,
> and in cases where it may not be possible to come up with a 1s race
> detector, it makes sense to move the test out of BAT and into the
> nightly runs.
> 
> Agreement was that the time limits are hard limits so thats pretty
> much it. We can do this now and hit the targets and them add more
> tests if and when someone manages to implement them.

1s is totally arbitrary, setting it as a hard limit is ridiculous.

> I don't have a problem with that, B in BAT stands for basic anyway.

The most basic userspace appears far more complex than igt. Quite often
BAT passes, but a couple of seconds of dogfooding reveals an unusable
mess. Basic Acceptance Testing is failing in its task, and we seem to
heading down the path of making it fail harder.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre