[igt-dev] Mandatory Test Suite for KMS Drivers?

[Maxime Ripard]

Hi Rodrigo,

On Thu, Mar 03, 2022 at 05:52:57AM -0500, Rodrigo Vivi wrote:
> On Thu, Mar 03, 2022 at 10:05:07AM +0100, Maxime Ripard wrote:
> > Hi,
> > 
> > Back at XDC we floated the idea of creating a test suite for IGT that we
> > expect any KMS driver to pass, similar to what v4l2-compliance and
> > cec-compliance provide for v4l2 and CEC respectively.
> > 
> > I was looking at the list of tests, and it's fairly massive, so it's not
> > clear to me what tests we could start this suite with. I can only assume
> > all the KMS (but the chamelium ones) and fbdev related ones would be a
> > good start?
> > 
> > What do you think?
> 
> I believe we should start with the group of the tests that we know that
> are reliably passing today on most of the platforms and then increase
> the list as the tests and drivers become more reliable.

I can see why that would be an objective too, but I'm not sure it would
cover mine. What I'd like this series to be is something we can ask
upfront to new drivers being submitted to make sure that they are sane.

Whether or not old drivers pass that bar is a bit irrelevant to that
objective (and this would actually create tasks for newcomers that are
looking for something to work on).

So, yeah, I don't mind having failing tests on older drivers, I kind of
even expect them to fail somehow. It would essentially be a bar to show
what any driver should strive for, not the lowest common denominator.

Does that make sense?

> For instance, many of these would be candidate to be filtered out for now
> https://intel-gfx-ci.01.org/tree/drm-next/index.html?testfilter=kms
> 
> compared to the whole view of kms tests:
> https://intel-gfx-ci.01.org/tree/drm-next/shards-all.html?testfilter=kms

So the set of tests that are always run would be the latter, right?

Maxime
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/igt-dev/attachments/20220308/40eedba1/attachment.sig>