[igt-dev] [PATCH i-g-t 1/1] tests: Remove tools_test

[Jani Nikula]

On Fri, 14 Sep 2018, Daniel Vetter <daniel at ffwll.ch> wrote:
> On Wed, Sep 12, 2018 at 09:56:24AM +0300, Arkadiusz Hiler wrote:
>> On Wed, Sep 12, 2018 at 09:51:24AM +0300, Petri Latvala wrote:
>> > The test is basically just testing if the tools work properly with
>> > little relevance to testing if the kernel works properly. If the
>> > purpose was to test the kernel (or hardware), actual tests will be
>> > better suited for the purpose. If the purpose was to test if the tools
>> > work, sanity checks somewhere else for all tools instead of just two
>> > is better suited for the purpose.
>> > 
>> > In a nutshell, tools_test as such is fairly useless and has thus far
>> > only revealed problems in testing setups (incorrect paths etc) instead
>> > of problems in kernel or hardware.
>> > 
>> > Signed-off-by: Petri Latvala <petri.latvala at intel.com>
>> > Cc: Daniel Vetter <daniel.vetter at ffwll.ch>
>> > Cc: Arkadiusz Hiler <arkadiusz.hiler at intel.com>
>> Acked-by: Arkadiusz Hiler <arkadiusz.hiler at intel.com>
>
> Hm, not terribly happy about this. We have blown up the tools in trivial
> ways before, that's why I wrote these. And yes it's incomplete coverage.
> But we're also not going around deleting other igt tests, just because
> they're not yet fully covering a given area.

I think there's value in ensuring we don't break the tools from an ABI
perspective. But at the same time I don't think we need to test the
tools for every patch posted on the list. I.e. tool testing should be
separated from ABI testing.

So here's an idea. Extend the tools with a test subcommand or --test
parameter, and have the tools themselves check that the ABI they need
works. Centralize that information in the tools.

For example, 'intel_reg dump' is a ridiculously slow and potentially
dangerous way to test if intel_reg works. I've overlooked that in
tools_test; I would never have added something like that in intel_reg.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center