[PATCH i-g-t v6] igt-runner fact checking

[Peter Senna Tschudin]

On 19.11.2024 11:24, Luca Coelho wrote:
> On Tue, 2024-11-19 at 11:07 +0100, Peter Senna Tschudin wrote:
>>
>> On 19.11.2024 09:19, Luca Coelho wrote:
>>> On Mon, 2024-11-18 at 17:03 +0100, Peter Senna Tschudin wrote:
>>>>
>>>> On 18.11.2024 14:07, Luca Coelho wrote:
>>>>> This makes me start thinking... it would be so much easier to make all
>>>>> this in python.  Would it be possible to glue a python script to igt-
>>>>> runner (or whoever calls it) and get the data from dmesg, lsmod etc.?
>>>>> Or otherwise aren't there any libraries already used by IGT that
>>>>> include list handling?
>>>>
>>>> Absolutely no python here. Never! :-)
>>>
>>> Oh, well.  Can't really argue against dogmas...
>>
>> Wow, that was a low punch.
> 
> Oh, sorry for that, it was not my intention to punch you at all, much
> less a low punch. ;) But it seemed like a dry, I-don't-even-want-to-
> talk-about-it statement to me, without any explanations, which looks
> like a dogma to me...
> 
> 
>> This code is going to run about 1M times / day in our CI. There is history
>> that precedes me working on IGT of issues created by trying to mix c and
>> python in igt. The reasons for not trying python again are runtime overhead
>> and the history of such attempts in igt that just made a bunch of people and
>> our CI very upset.
> 
> Okay, now that's an explanation. :) It makes some sense, and obviously
> I'm not trying to change how the entire IGT framework is implemented. 
> But looking at all this C code doing something that can be done in way
> fewer lines in python, just got me wondering.
> 
> I think the previous IGT trials with python were probably not really
> done right, because I'm pretty sure it could be made in a way that
> would be very efficient.  Probably not having C invoke a shell to run
> python, but some proper bindings, or maybe even better, having python
> wrap around the C code.
> 
> In this specific case, I think you could neatly wrap some python around
> igt-runner to gather these facts.  But TBH I don't even know the
> details of how igt-runner and everything else is invoked in IGT...
> 
> In any case, if using python is an established no-no in IGT, all this
> conversation is moot.

The way to go may be to create a new Python framework that uses igt
under the hood. This will allow us to reuse all existing tests, will
empower us to do stuff like you want, and open testing to a broader
audience.

Another way may be to go even higher level than with something like
Robot Framework as front-end and having igt as the back end (with some
probable python glue code in between).

How do you feel about these?

> 
> I'll reply to the rest of your email later, when I find some more time
> again.

Thanks again. After sending you the reply I remembered the reason why I
am hiding errors. This code I am adding is like the alarm in your car. I
like the car alarm analogy because the main value of igt-facts will be
to detect really weird stuff like a gpu disappearing from the pci bus.

With the volume of tests we run, this will be a rare occurrence at best
which kind of match the number of times someone tries to steal your car.

I am fairly confident that you do not want an check engine light and
having your car powering itself off when the ICU detects an issue in the
alarm sub-system.

In the same way, I evaluated that I do not want to abort igt-runner
verbosely if something weird goes on with igt-facts. So I decided that
it is ok for the igt-facts to fail silently so that igt-runner will
continue to run and do it's thing.

Is it ok for igt-facts to fail silently or do you prefer something else?

Thanks!

> 
> 
> --
> Cheers,
> Luca.