[Intel-gfx] [PATCH i-g-t 2/3] Unify handling of slow/combinatorial tests

[David Weinehall]

On Mon, Oct 26, 2015 at 03:59:24PM -0200, Paulo Zanoni wrote:
> 2015-10-26 15:30 GMT-02:00 David Weinehall <david.weinehall at linux.intel.com>:
> > On Mon, Oct 26, 2015 at 02:44:18PM -0200, Paulo Zanoni wrote:
> >> 2015-10-26 12:59 GMT-02:00 David Weinehall <david.weinehall at linux.intel.com>:
> >> > On Fri, Oct 23, 2015 at 11:50:46AM -0200, Paulo Zanoni wrote:
> >> >
> >> > [snip]
> >> >
> >> >> It's not clear to me, please clarify: now the tests that were
> >> >> previously completely hidden will be listed in --list-subtests and
> >> >> will be shown as skipped during normal runs?
> >> >
> >> > Yes.  Daniel and I discussed this and he thought listing all test
> >> > cases, even the slow ones, would not be an issue, since QA should
> >> > be running the default set not the full list
> >> > (and for that matter, shouldn't QA know what they are doing too? :P).
> >>
> >> If that's the case, I really think your patch should not touch
> >> kms_frontbuffer_tracking.c. The hidden subtests should not appear on
> >> the list. People shouldn't even have to ask themselves why they are
> >> getting 800 skips from a single testcase. Those are only for debugging
> >> purposes.
> >
> > Fair enough.  I'll try to come up with a resonable way to exclude them
> > from the list in a generic manner.  Because that's the whole point of
> > this exercise -- to standardise this rather than have every test case
> > implement its own method of choosing whether or not to run all tests.
> 
> Maybe instead of marking these tests as SKIP we could use some other
> flag. That would avoid the confusion between "skipped because some
> condition was not match but the test is useful" vs "skipped because
> the test is unnecessary".

I'd prefer a method that wouldn't require patching piglit.


Regards, David