[igt-dev] [PATCH i-g-t] runner: Treat dmesg warnings as pure warnings

[Joonas Lahtinen]

Quoting Chris Wilson (2018-11-21 11:18:43)
> Quoting Joonas Lahtinen (2018-11-21 08:07:30)
> > Quoting Ville Syrjälä (2018-11-14 15:53:26)
> > > On Wed, Nov 14, 2018 at 01:34:10PM +0000, Chris Wilson wrote:
> > > > Quoting Ville Syrjälä (2018-11-14 13:25:39)
> > > > > On Wed, Nov 14, 2018 at 01:56:53PM +0200, Petri Latvala wrote:
> > > > > > On Tue, Nov 13, 2018 at 05:25:47PM +0000, Chris Wilson wrote:
> > > > > > > I have whinged on for ages about the dmesg-warnings being an expected
> > > > > > > part of kernel testing (where else is the kernel meant to log its
> > > > > > > errors?) and should be treated the same as our stderr for the test. That
> > > > > > > is if a test fails, it fails and does not need to be conflated with
> > > > > > > whether or not there was a dmesg warning (just as the test saying why it
> > > > > > > failed on stderr does not need flagging), and that a passing test with a
> > > > > > > dmesg warning is simply a warn.
> > > > > > > 
> > > > > > > The effect is that we simply remove the "dmesg-" flagging from results
> > > > > > > names, as the err/dmesg output is simply collated for the error report
> > > > > > > already.
> > > > > > > 
> > > > > > > Signed-off-by: Chris Wilson <chris at chris-wilson.co.uk>
> > > > > > > Cc: Petri Latvala <petri.latvala at intel.com>
> > > > > > > Cc: Martin Peres <martin.peres at linux.intel.com>
> > > > > > 
> > > > > > 
> > > > > > Yep, now's a good a time as any to remove this cargo cult if ever.
> > > > > > 
> > > > > > Some acks from kernel people in general would be nice.
> > > > > > 
> > > > > > TODO:
> > > > > >  Martin: cibuglog filters ready for this
> > > > > >  Tomi et al: Visualization ready for this (tooltips etc)
> > > > > 
> > > > > Can I still see from the color/etc. whether there was some dmesg spew or
> > > > > not? I often glance at those and would prefer to not have to go through
> > > > > all the fails to find them.
> > > > 
> > > > It was never shown in the colour:
> > > > 
> > > >       warn/dmesg-warn == orange
> > > >       fail/dmesg-fail == red
> > > 
> > > Indeed. Which makes me wonder whether I've been overlooking some
> > > dmesg noise. Either that or it's just my memory playing tricks
> > > and telling me that I've mostly been looking at the orange stuff.
> > > 
> > > So let's consider this a feature request then. I would like to see
> > > from the overview which tests produced dmesg noise, regardless of
> > > whether the test otherwise succeeded or failed.
> > 
> > I think this is a fair change.
> > 
> > And for documentation purposes, what I proposed in IRC:
> > 
> > My suggestion was to add letter "D" to the matrix when there is dmesg
> > noise in the specific subtest (Similarly as there's letter T in
> > timeout).
> 
> I just don't understand what is "dmesg noise". dmesg is where I log my
> errors, there is no other suitable log.

My take on this is that it provides a quick differentation of:

"The IGT test fails, functionality lost"

"The IGT test passes, functionality is there, but makes kernel spew
error (recoverably)"

Both are errors, but there's informational value in seeing the
difference, at least to me.

Regards, Joonas

> -Chris