[Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest functionality.
daniel at ffwll.ch
Wed Jan 27 07:42:40 PST 2016
On Wed, Jan 27, 2016 at 02:01:36PM +0000, Morton, Derek J wrote:
> >-----Original Message-----
> >From: Daniel Vetter [mailto:daniel.vetter at ffwll.ch] On Behalf Of Daniel Vetter
> >Sent: Wednesday, January 27, 2016 1:39 PM
> >To: Ville Syrjälä
> >Cc: Morton, Derek J; intel-gfx at lists.freedesktop.org
> >Subject: Re: [Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest functionality.
> >On Wed, Jan 27, 2016 at 02:32:47PM +0200, Ville Syrjälä wrote:
> >> On Wed, Jan 27, 2016 at 10:05:56AM +0000, Derek Morton wrote:
> >> > Added support for specifying arbitary lists of subtests to run or to
> >> > exclude from being run by using : or ^ as a seperator.
> >> >
> >> > :subtest1:subtest2: Will run subtest1 and subtest2
> >> > ^subtest1^subtest2^ will run all subtests except subtest1 and
> >> > subtest2
> >> Hmm. Getting a bit complicated perhaps. Would it be simpler to just
> >> allow specifying the --r option multiple times? So we'd start with the
> >> full list of subtests, and each --r option would filter the list in
> >> some way?
> >Also why not use piglit ... or what is this for?
> We don't use piglet on android. Piglet does not know about adb. Piglet
> expects to be running on the system under test not on a separate host.
This can be fixed, and iirc there's even been patches floating around to
run piglits remotely via adb.
> The main aim of this is because on android we are not testing a driver
> which is drm-nightly. The kernel / display driver used on android will
> not have all the features that are in the latest linux kernel. We keep
> hitting problems where new subtests get added to IGT to test features
> that are not yet in the android kernel. We run and report tests at a
> binary level as that is what the project managers expect. We wish to be
> able to run the latest versions of IGT to pick up bug fixes and useful
> test changes, but want a way of being able to exclude subtests that are
> not currently appropriate on android without having to exclude complete
> test binaries. The specific subtests which need to be excluded will
> differ depending on the HW (CHV vs BXT for example) and specific driver
> version in the build under test so we need a simple mechanism to specify
> the subtests to run or exclude (depending on which is more appropriate)
> for each test.
So a bunch of things:
- Reporting at the per-binary level. Still doesn't make sense, and really
imo not a technical issue. Worst case write shell scripts (or
autogenerate those) with the testcase groups.
- igts falling over when the kernel doesn't support a feature. This
shouldn't ever happen, igt testcases are suppose to skip when the
requirements aren't met. Please report any such cases so that we can fix
them up in upstream igt.
- Android folks breaking the libdrm abi isn't in your list, but comes up
fairly often, too.
Software Engineer, Intel Corporation
More information about the Intel-gfx