[Intel-gfx] Making IGT runnable by CI and developers

[Tvrtko Ursulin]

On 21/07/2017 16:52, Daniel Vetter wrote:
> On Fri, Jul 21, 2017 at 5:13 PM, Jani Nikula
> <jani.nikula at linux.intel.com> wrote:

[snip]

>> I agree the goal should be to run all tests by default. And this means
>> we should start being more critical of the tests we add.
>>
>> For stress tests I would like to look more into splitting up the tests
>> in a way that you could run one iteration fast (as part of default), and
>> repeat the tests for more stress and coverage. I don't know how feasible
>> this is, and if it requires carrying over state from one iteration to
>> other, but I like the goal of running also some of this by default. This
>> would better catch silly bugs in tests too. (We discussed this offline
>> with Martin and Tomi.)
> 
> I think right now, and for the near future (up to at least a year) the
> only time we'll run stress tests if developers need nastier testcases
> to help reproduce a bug locally. We simply don't have neither the CI
> nor the QA resources to run these tests. If we're making really great
> progress on overall quality and CI infrastructure (that needs budget
> we don't have right now) and pre-merge testing we might be able to
> start looking into running stress tests in CI or QA.
> 
> I think a good example is kms_frontbuffer_tracking --show-hidden,
> which Paulo used to debug issues on his own machine.

Just on this particular point - to make this facility generic would be a 
flavour of test tagging, so pretty much the same high level effect as 
the RFC I sent. I am not pushing it (again), just saying it would be 
effectively the same approach/effort, only that tags are more 
generic/flexible.

Regards,

Tvrtko