[igt-dev] [i g t] tests/kms_setmode: basic Fix the testcase failure on 4K monitors

[Anand Moon]

From: Chris Wilson <chris at chris-wilson.co.uk>

On 4k monitors kms_setmode basic test case would like to fail with below
assertion, by checking the expected frame time should be less than max
scanline interval, we improve the test case so that 90% of frame times
fall within the one scanline on each side of expected mean.

CRTC[98] [Pipe A] Mode: 3840x2160 at 60Hz Connectors: DP-2[330] Expected frametime: 16668us; measured 16672.3us +- 1.805us accuracy 0.00% [0.07 scanlines]
(kms_setmode:14315) CRITICAL: Test assertion failure function check_timings, file ../tests/kms_setmode.c:544:
(kms_setmode:14315) CRITICAL: Failed assertion: fabs(mean - expected) < 1.718 * stddev
(kms_setmode:14315) CRITICAL: vblank interval differs from modeline! expected 16667.6us, measured 16672us +- 1.805us, difference 4.7us (2.6 sigma) Stack trace:
  #0 ../lib/igt_core.c:1734 __igt_fail_assert()
  #1 ../tests/kms_setmode.c:544 test_one_combination()
  #2 ../tests/kms_setmode.c:834 __real_main883()
  #3 ../tests/kms_setmode.c:883 main()
  #4 ../csu/libc-start.c:342 __libc_start_main()
  #5 [_start+0x2e]
Subtest basic failed.
**** DEBUG ****

Cc: Dorota Czaplejewicz <dorota.czaplejewicz at collabora.co.uk>
Signed-off-by: Chris Wilson <chris at chris-wilson.co.uk>
Signed-off-by: Anand Moon <anandx.ram.moon at intel.com>
---

And all the testcase regarding kms_setmode seems to passing on
5.8.x and 5.10-0-rcx kernel with following mode

Mode: 1920x1080 at 60Hz Connectors: HDMI-A-2
Mode: 1920x1080 at 60Hz Connectors: DP-1
Mode: 1920x1080 at 60Hz Connectors: DP-2
Mode: 2560x1440 at 60Hz Connectors: eDP-1
Mode: 3840x2160 at 60Hz Connectors: DP-2
---
 tests/kms_setmode.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
index a01010a1..d55326f2 100644
--- a/tests/kms_setmode.c
+++ b/tests/kms_setmode.c
@@ -525,10 +525,13 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
 	 * See:
 	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
 	 */
-	igt_assert_f(fabs(mean - expected) < 1.718 * stddev,
-		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma)\n",
-		     expected, mean, stddev,
-		     fabs(mean - expected), fabs(mean - expected) / stddev);
+	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
+			"vblank interval differs from modeline! expected %.1fus, measured"
+			" %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
+			expected, mean, stddev,
+			fabs(mean - expected),
+			fabs(mean - expected) / stddev,
+			fabs(mean - expected) / line_time(kmode));
 }
 
 static void test_crtc_config(const struct test_config *tconf,
-- 
2.29.2