[igt-dev] [PATCH i-g-t 6/9] tests/perf_pmu: PMU enable race test

[Tvrtko Ursulin]

On 05/02/2018 10:16, Chris Wilson wrote:
> Quoting Tvrtko Ursulin (2018-02-05 10:07:02)
>> I can wrap it in a 10x loop but until the PMU is fixed that means 10x
>> longer runtime as well which I wanted to avoid, and instead count on
>> execution in numbers across the shards.
> 
> Otoh, I want a test that we can reliably run locally to determine if we
> have a problem or not. A difficult balance. What should we aim for
> flip-floppy tests, >50% error rate? >10%? Certainly has to be >1% or we
> will struggle to notice the pattern. My finger in the air would say >33%.

I don't know how we do that since it will depend on so many factors. On 
my SKL GT2 and my kernel config, it is maybe around 10%. So if we cannot 
reach >33% what do we, don't add the test?

Regards,

Tvrtko