[Intel-gfx] [PATCH i-g-t 2/3] Unify handling of slow/combinatorial tests

[David Weinehall]

On Fri, Oct 23, 2015 at 11:50:46AM -0200, Paulo Zanoni wrote:

[snip]

> It's not clear to me, please clarify: now the tests that were
> previously completely hidden will be listed in --list-subtests and
> will be shown as skipped during normal runs?

Yes.  Daniel and I discussed this and he thought listing all test
cases, even the slow ones, would not be an issue, since QA should
be running the default set not the full list
(and for that matter, shouldn't QA know what they are doing too? :P).

> For kms_frontbuffer_tracking, hidden tests are supposed to be just for
> developers who know what they are doing. I hide them behind a special
> command-line switch that's not used by QA because I don't want QA
> wasting time running those tests. One third of the
> kms_frontbuffer_tracking hidden tests only serve the purpose of
> checking whether there's a bug in kms_frontbuffer_track itself or not.
> For some other hidden tests, they are there just to help better debug
> in case some other non-hidden tests fail. Some other hidden tests are
> 100% useless and superfluous.

Shouldn't 100% useless and superfluous tests be excised completely?

> QA should only run the non-hidden tests.

Which is the default behaviour, AFAICT.

> So if some non-hidden test fails, the developers can use the hidden
> tests to help debugging.
>
> Besides, the "if (t.slow)" could have been moved to
> check_test_requirements(), making the code much simpler :)

Thanks for the suggestion.  Will modify the code accordingly.
That change does indeed simplify things quite a bit!


Kind regards, David