[Intel-gfx] [RFC i-g-t] igt: Test tagging support

[Tvrtko Ursulin]

On 29/06/2017 10:07, Jani Nikula wrote:
> On Thu, 29 Jun 2017, Daniel Vetter <daniel at ffwll.ch> wrote:
>> Just adding tags because tags sounds like a cool idea on top of all the
>> other things we do already still doesn't sound appealing to me.
> 
> Agreed.

Well it's not on top, but as a replacement, but anyway.

> So I always thought it was silly when we relied on test names to select
> tests to run or exclude in the past. I'm equally suspicious of adding
> any sorts of categories or tags within the test sources. I don't think a
> test source file should be changed if the category changes. Thus I think
> maintaining the categories outside of the tests is a good idea.
> 
> Currently we maintain the categories as test lists. I think you could
> call them tags just as well. Them being lists in files is just an
> implementation detail. If a test is included in a test list, then it's
> so tagged. If we wanted to, we could add more logic to running the tests
> based on which lists the tests are in. For example, "in extended and not
> in stress".

We started with the premise that tests lists are bad. :) If we have 
multiple lists we will need someone to keep an eye on test changes and 
keep all the lists up to date. To me that has a higher chance of failing 
that having tags together with the code. It is easier to remember if it 
stares at you from the screen while you are editing the test.

Regards,

Tvrtko