[igt-dev] [PATCH i-g-t] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Chris Wilson]

Quoting Tvrtko Ursulin (2018-03-07 11:11:19)
> From: Tvrtko Ursulin <tvrtko.ursulin at intel.com>
> 
> We need to use absolute tolerance when asserting on percentages. Relative
> tolerance in this case is unfair and inaccurate since it's strictness
> varies with relative target busyness.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin at intel.com>
> Cc: Chris Wilson <chris at chris-wilson.co.uk>
> ---
>  tests/perf_pmu.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
> index 9ebffc64d1f1..8e547338b47c 100644
> --- a/tests/perf_pmu.c
> +++ b/tests/perf_pmu.c
> @@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
>         __sync_synchronize();
>  }
>  
> -#define div_round_up(a, b) (((a) + (b) - 1) / (b))
> +#define __assert_within(x, ref, tol_up, tol_down) \
> +       igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
> +                    (double)(x) >= ((double)(ref) - (tol_down)), \
> +                    "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
> +                    (double)(x), (double)(tol_up), (double)(tol_down), \
> +                    (double)(ref), #x, #ref)
> +
> +#define assert_within(x, ref, tolerance) \
> +       __assert_within(x, ref, tolerance, tolerance)
>  
>  static void
>  accuracy(int gem_fd, const struct intel_execution_engine2 *e,
> @@ -1571,7 +1579,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>  
>         /* Let the child run. */
>         read(link[0], &expected, sizeof(expected));
> -       assert_within_epsilon(expected, target_busy_pct/100., 0.05);
> +       assert_within(100.0 * expected, target_busy_pct, 5);
>  
>         /* Collect engine busyness for an interesting part of child runtime. */
>         fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
> @@ -1590,8 +1598,11 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>         igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
>                  __error(busy_r, expected), 100 * busy_r, 100 * expected);
>  
> -       assert_within_epsilon(busy_r, expected, 0.15);
> -       assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
> +       busy_r *= 100.0;
> +       expected *= 100.0;
> +
> +       assert_within(busy_r, expected, 2);
> +       assert_within(100.0 - busy_r, 100.0 - expected, 2);

The advantage of switching to absolute here is that we only need the
single test. Ok, using a factor of 100 here should make the output more
readable.

Kill the extra assert_within,
Reviewed-by: Chris Wilson <chris at chris-wilson.co.uk>

But I suspect we may need to relax the target for kasan, we will see in
a few days.
-Chris