[PATCH] RFC: Make igts for cross-driver stuff mandatory?

[Liviu Dudau]

On Fri, Oct 19, 2018 at 10:50:49AM +0200, Daniel Vetter wrote:
> Hi all,

Hi,

(Replying from my personal address as the work email seems to have let
this one go to /dev/null)

> 
> This is just to collect feedback on this idea, and see whether the
> overall dri-devel community stands on all this. I think the past few
> cross-vendor uapi extensions all came with igts attached, and
> personally I think there's lots of value in having them: A
> cross-vendor interface isn't useful if every driver implements it
> slightly differently.

I would like to also get some clarification on where we are standing on 
"tests vs 'real code'" stanza. Does making igt tests mandatory replace
the need for 'real code' or does it add to the list of requirements? If
the later, then I think the bar rises in terms of showing igts' 
usefulness / benefits.

> 
> I think there's 2 questions here:
> 
> - Do we want to make such testcases mandatory?

I'm a bit reluctant to make it so by fiat. I think that showing the
usefulness of having igts tests to newcomers (by adding with this patch
some more information about why IGT is a good place to add your testing to)
and getting more mature drivers to get tested under IGT on a regular basis
would make adoption of IGT for testing a community standard.

> 
> - If yes, are we there yet, or is there something crucially missing
>   still?

I'm just getting back into IGT by refreshing the writeback patches, but
by looking at the commit log I get the impression that there aren't that
many patches that target drivers other than Intel's. Are all the non-Intel
patches so generic that one doesn't need to specify a target driver for
those changes (in which case great, but then why is Intel's so different?),
or are the others not bothered with IGT support?

At the moment I'm a bit on the fence on this. Not having spent too much
time with IGT in the last 6 months, I'm probably closer to a newcomer in
my attitude towards IGT and at the moment I'm not clear on how to answer the
"Why?" and "What is in it for me?" questions.

Best regards,
Liviu

> 
> And of course there's a bunch of details to figure out. Like we
> probably want to also recommend the selftests/unit-tests in
> drivers/gpu/drm/selftest, since fairly often that's a much more
> effective approach to checking the details than from userspace.
> 
> Feedback and thoughts very much appreciated.
> 
> Cheers, Daniel
> ---
>  Documentation/gpu/drm-uapi.rst | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/Documentation/gpu/drm-uapi.rst b/Documentation/gpu/drm-uapi.rst
> index 4b4bf2c5eac5..91cf6e4b6303 100644
> --- a/Documentation/gpu/drm-uapi.rst
> +++ b/Documentation/gpu/drm-uapi.rst
> @@ -238,6 +238,13 @@ DRM specific patterns. Note that ENOTTY has the slightly unintuitive meaning of
>  Testing and validation
>  ======================
>  
> +Testing Requirements for userspace API
> +--------------------------------------
> +
> +New cross-driver userspace interface extensions, like new IOCTL, new KMS
> +properties, new files in sysfs or anything else that constitutes an API change
> +need to have driver-agnostic testcases in IGT for that feature.
> +
>  Validating changes with IGT
>  ---------------------------
>  
> -- 
> 2.19.1
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
             /`\
            / : |
   _.._     | '/
 /`    \    | /
|  .-._ '-"` (
|_/   /   o  o\
      |  == () ==
       \   -- /                       ______________________________________
       / ---<_              ________|                                      |_______
      |      \\             \       |  I would like to fix the world but   |      /
      | |     \\__           \      |   no one gives me the source code.   |     /
      / ;     |.__)          /      |______________________________________|     \
     (_/.-.   ;             /__________)                                (_________\
    { `|   \_/
     '-\   / |
        | /  |
       /  \  '-.
       \__|-----'