[igt-dev] Must-Pass Test Suite for KMS drivers

[Daniel Vetter]

On Mon, 7 Nov 2022 at 11:53, Thomas Zimmermann <tzimmermann at suse.de> wrote:
>
> Hi
>
> Am 07.11.22 um 11:26 schrieb Daniel Vetter:
> > On Mon, 7 Nov 2022 at 10:43, Thomas Zimmermann <tzimmermann at suse.de> wrote:
> >>
> >> Hi
> >>
> >> Am 07.11.22 um 10:30 schrieb Maxime Ripard:
> >>> Hi Thomas,
> >>>
> >>> On Fri, Oct 28, 2022 at 09:31:38AM +0200, Thomas Zimmermann wrote:
> >>>> Am 24.10.22 um 14:43 schrieb maxime at cerno.tech:
> >>>>> I've discussing the idea for the past year to add an IGT test suite that
> >>>>> all well-behaved KMS drivers must pass.
> >>>>>
> >>>>> The main idea behind it comes from v4l2-compliance and cec-compliance,
> >>>>> that are being used to validate that the drivers are sane.
> >>>>>
> >>>>> We should probably start building up the test list, and eventually
> >>>>> mandate that all tests pass for all the new KMS drivers we would merge
> >>>>> in the kernel, and be run by KCi or similar.
> >>>>>
> >>>>> I did a first pass to create a draft of such a test-suite, which would
> >>>>> contain:
> >>>>>
> >>>>> igt at core_auth@basic-auth
> >>>>> igt at core_auth@getclient-master-drop
> >>>>> igt at core_auth@getclient-simple
> >>>>> igt at core_auth@many-magics
> >>>>> igt at core_getclient
> >>>>> igt at core_getstats
> >>>>> igt at core_getversion
> >>>>> igt at core_hotunplug@hotrebind-lateclose
> >>>>> igt at core_hotunplug@hotunbind-rebind
> >>>>> igt at core_hotunplug@unbind-rebind
> >>>>> igt at core_setmaster
> >>>>> igt at core_setmaster_vs_auth
> >>>>> igt at device_reset@unbind-reset-rebind
> >>>>> igt at drm_read
> >>>>> igt at dumb_buffer
> >>>>> igt at fbdev
> >>>>
> >>>> Maybe we make this test optional? At least sprd decided to not support fbdev
> >>>> at all.
> >>>
> >>> I'm not sure we need to make that test optional, or at least, we should
> >>> run it all the time, but skip it if there's no fbdev emulation, and not
> >>> report it as a failure?
> >>
> >> Sure. I just meant that fbdev support shouldn't be a blocker. If there,
> >> it should work of course.
> >
> > Not supporting fbdev looks more like an accident than intention here,
> > and maybe a good reason to have these kind of must-past lists.
>
> No. Back then, I specifically asked the developer, Kevin Tang IIRC,
> about fbdev/console support and he replied that he has no intention of
> adding it.
>
> It's the only driver without fbdev, but as we don't have such a
> requirements AFAIK, I left it at that.

It does hurt a bit the sales pitch that a drm-kms driver is the
one-stop-shop for display driver needs, and so I'd only do this if
there's some technical reasons why fbdev just wont work (like no hw
support for formats fbdev supports or something), and not just because
the vendor has no need for this right now. Otoh it's generally just
one line to add if the driver works correctly, so ...

*shrug*

Cheers, Daniel

>
> Best regards
> Thomas
>
> > Eventually, once the i915-ism is worked out of igt well enough for a
> > set of tests at least. The drm/ci effort should help in building up
> > that list (by essentially extracting the common set of tests that
> > everyone passes and graduating that to the must-pass list for kms
> > conformance or whatever we'll call it).
> > -Daniel
> >
> >>
> >> Best regards
> >> Thomas
> >>
> >>>
> >>> Maxime
> >>
> >> --
> >> Thomas Zimmermann
> >> Graphics Driver Developer
> >> SUSE Software Solutions Germany GmbH
> >> Maxfeldstr. 5, 90409 Nürnberg, Germany
> >> (HRB 36809, AG Nürnberg)
> >> Geschäftsführer: Ivo Totev
> >
> >
> >
>
> --
> Thomas Zimmermann
> Graphics Driver Developer
> SUSE Software Solutions Germany GmbH
> Maxfeldstr. 5, 90409 Nürnberg, Germany
> (HRB 36809, AG Nürnberg)
> Geschäftsführer: Ivo Totev



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch