[Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest functionality.

[Morton, Derek J]

>
>-----Original Message-----
>From: Daniel Vetter [mailto:daniel.vetter at ffwll.ch] On Behalf Of Daniel Vetter
>Sent: Wednesday, January 27, 2016 1:39 PM
>To: Ville Syrjälä
>Cc: Morton, Derek J; intel-gfx at lists.freedesktop.org
>Subject: Re: [Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest functionality.
>
>On Wed, Jan 27, 2016 at 02:32:47PM +0200, Ville Syrjälä wrote:
>> On Wed, Jan 27, 2016 at 10:05:56AM +0000, Derek Morton wrote:
>> > Added support for specifying arbitary lists of subtests to run or to 
>> > exclude from being run by using : or ^ as a seperator.
>> > 
>> > :subtest1:subtest2: Will run subtest1 and subtest2 
>> > ^subtest1^subtest2^ will run all subtests except subtest1 and 
>> > subtest2
>> 
>> Hmm. Getting a bit complicated perhaps. Would it be simpler to just 
>> allow specifying the --r option multiple times? So we'd start with the 
>> full list of subtests, and each --r option would filter the list in 
>> some way?
>
>Also why not use piglit ... or what is this for?

We don't use piglet on android. Piglet does not know about adb. Piglet expects to be running on the system under test not on a separate host.

The main aim of this is because on android we are not testing a driver which is drm-nightly. The kernel / display driver used on android will not have all the features that are in the latest linux kernel. We keep hitting problems where new subtests get added to IGT to test features that are not yet in the android kernel. We run and report tests at a binary level as that is what the project managers expect. We wish to be able to run the latest versions of IGT to pick up bug fixes and useful test changes, but want a way of being able to exclude subtests that are not currently appropriate on android without having to exclude complete test binaries. The specific subtests which need to be excluded will differ depending on the HW (CHV vs BXT for example) and specific driver version in the build under test so we need a simple mechanism to specify the subtests to run or exclude (depending on which is more appropriate) for each test.

//Derek

>-Daniel
>--
>Daniel Vetter
>Software Engineer, Intel Corporation
>http://blog.ffwll.ch
>