[igt-dev] [PATCH i-g-t] tests/perf: dramatically loosen expectations on oa-exponents

[Lionel Landwerlin]

On 30/03/2020 16:54, Chris Wilson wrote:
> Quoting Lionel Landwerlin (2020-03-30 13:48:34)
>> This test really quite flaky because it expects HW to emit reports
>> following pretty much exactly the specified timer. And there are lots
>> of reasons the HW might not be able to meet the timer deadline. The
>> design is such that it's expected to not always meet the timer.
>>
>> We just want to verify that the HW was configured properly, so loosen
>> the checks to half the reports (of 30) meeting the timer deadline.
>>
>> Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin at intel.com>
>> ---
>>   tests/perf.c | 10 ++++++++--
>>   1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/perf.c b/tests/perf.c
>> index 442d89fb..023829fc 100644
>> --- a/tests/perf.c
>> +++ b/tests/perf.c
>> @@ -1742,8 +1742,14 @@ test_oa_exponents(void)
>>   
>>                  igt_debug("matches=%u/%u\n", matches, n_timer_reports - 1);
>>   
>> -               /* Allow for a couple of errors. */
>> -               igt_assert_lte(n_timer_reports - 3, matches);
>> +               /*
>> +                * Expect half the reports to match the timing
>> +                * expectation. The results are quite erratic because
>> +                * the condition under which the HW reaches
>> +                * expectations depends on memory controller pressure
>> +                * etc...
>> +                */
>> +               igt_assert_lte(n_timer_reports / 2, matches);
> Acked-by: Chris Wilson <chris at chris-wilson.co.uk>
>
> This will check for gross errors and so detect if we misconfigure the
> HW.
>
> What you might like to try is to look at the distribution of intervals,
> and compare the median against your expectations.
> -Chris

Yeah... It's quite annoying, sometimes the unit skips a deadline and 
tries to make it up right after...

Last time I checked the median didn't really help.


-Lionel