[igt-dev] Must-Pass Test Suite for KMS drivers

[Daniel Vetter]

On Mon, 7 Nov 2022 at 10:43, Thomas Zimmermann <tzimmermann at suse.de> wrote:
>
> Hi
>
> Am 07.11.22 um 10:30 schrieb Maxime Ripard:
> > Hi Thomas,
> >
> > On Fri, Oct 28, 2022 at 09:31:38AM +0200, Thomas Zimmermann wrote:
> >> Am 24.10.22 um 14:43 schrieb maxime at cerno.tech:
> >>> I've discussing the idea for the past year to add an IGT test suite that
> >>> all well-behaved KMS drivers must pass.
> >>>
> >>> The main idea behind it comes from v4l2-compliance and cec-compliance,
> >>> that are being used to validate that the drivers are sane.
> >>>
> >>> We should probably start building up the test list, and eventually
> >>> mandate that all tests pass for all the new KMS drivers we would merge
> >>> in the kernel, and be run by KCi or similar.
> >>>
> >>> I did a first pass to create a draft of such a test-suite, which would
> >>> contain:
> >>>
> >>> igt at core_auth@basic-auth
> >>> igt at core_auth@getclient-master-drop
> >>> igt at core_auth@getclient-simple
> >>> igt at core_auth@many-magics
> >>> igt at core_getclient
> >>> igt at core_getstats
> >>> igt at core_getversion
> >>> igt at core_hotunplug@hotrebind-lateclose
> >>> igt at core_hotunplug@hotunbind-rebind
> >>> igt at core_hotunplug@unbind-rebind
> >>> igt at core_setmaster
> >>> igt at core_setmaster_vs_auth
> >>> igt at device_reset@unbind-reset-rebind
> >>> igt at drm_read
> >>> igt at dumb_buffer
> >>> igt at fbdev
> >>
> >> Maybe we make this test optional? At least sprd decided to not support fbdev
> >> at all.
> >
> > I'm not sure we need to make that test optional, or at least, we should
> > run it all the time, but skip it if there's no fbdev emulation, and not
> > report it as a failure?
>
> Sure. I just meant that fbdev support shouldn't be a blocker. If there,
> it should work of course.

Not supporting fbdev looks more like an accident than intention here,
and maybe a good reason to have these kind of must-past lists.
Eventually, once the i915-ism is worked out of igt well enough for a
set of tests at least. The drm/ci effort should help in building up
that list (by essentially extracting the common set of tests that
everyone passes and graduating that to the must-pass list for kms
conformance or whatever we'll call it).
-Daniel

>
> Best regards
> Thomas
>
> >
> > Maxime
>
> --
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Ivo Totev



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch