[Piglit] [PATCH 3/6] dmesg: setup dmesg to be able to mark a test warn or fail

[Daniel Vetter]

On Thu, Nov 14, 2013 at 10:35:03AM -0800, Dylan Baker wrote:
> On Thursday, November 14, 2013 05:34:38 PM Daniel Vetter wrote:
> > On Thu, Nov 14, 2013 at 08:04:20AM -0800, Dylan Baker wrote:
> > > On Thursday, November 14, 2013 02:52:41 PM Daniel Vetter wrote:
> > > > On Tue, Nov 12, 2013 at 07:54:01AM -0800, Dylan Baker wrote:
> > > > > This gives the dmesg class lists of statuses that will make a test a
> > > > > warn or a fail, it includes a few basic checks, namely i915 errors and
> > > > > that tests have not segfaulted.
> > > > > 
> > > > > Signed-off-by: Dylan Baker <baker.dylan.c at gmail.com>
> > > > > ---
> > > > > 
> > > > >  framework/dmesg.py    | 36 ++++++++++++++++++++++++++++++++----
> > > > >  framework/exectest.py | 22 +++++++++++++++-------
> > > > >  2 files changed, 47 insertions(+), 11 deletions(-)
> > > > > 
> > > > > diff --git a/framework/dmesg.py b/framework/dmesg.py
> > > > > index 9a23c14..edbea88 100644
> > > > > --- a/framework/dmesg.py
> > > > > +++ b/framework/dmesg.py
> > > > > @@ -22,6 +22,7 @@
> > > > > 
> > > > >  """ Module implementing classes for reading posix dmesg """
> > > > >  
> > > > >  import os
> > > > > 
> > > > > +import re
> > > > > 
> > > > >  import subprocess
> > > > >  from threads import synchronized_self
> > > > > 
> > > > > @@ -29,8 +30,10 @@ __all__ = ['Dmesg']
> > > > > 
> > > > >  # plain text list of statuses to be considered either a warn or a
> > > > >  fail,
> > > > >  any
> > > > >  # statuses not on this list will simply be ignored.
> > > > > 
> > > > > -WARN_STATUSES = []
> > > > > -FAIL_STATUSES = []
> > > > > +WARN_STATUSES = ['segfault']
> > > > > +FAIL_STATUSES = ['\[drm:.*\] \*ERROR\*',
> > > > > +                 '\[drm\] stuck on [a-zA-Z]* ring',
> > > > > +                 '\[drm\] GPU crash dump saved']
> > > > 
> > > > I think now that we filter out all the info/debug noise maybe we could
> > > > go
> > > > the other direction and blacklist a few of the remaining things from the
> > > > core kernel we don't care about. E.g.
> > > > 
> > > > [ 3867.022895] gem_evict_every (2671) used greatest stack depth: 2216
> > > > bytes
> > > > left
> > > > 
> > > > is a warn level message, but I don't care one bit about it (as long as
> > > > it
> > > > doesn't approach 0). But there's other warn level stuff which is fairly
> > > > interesting.
> > > > 
> > > > Just something to throw out there, I'm not sure what the best way would
> > > > be
> > > > to integrate dmesg reporting for piglit in general.
> > > > -Daniel
> > > 
> > > My personal problem with the dmesg code we have now (and with *just*
> > > blacklisting) is that I have an alps touchpad, it spams dmesg about 10
> > > times a minute, so I can't use dmesg reporting because of the massive
> > > number of false positives; we could use some combination of blacklisting
> > > and whitelisting however.
> > 
> > That sounds like we need a piglit cmdline option to supply a regex to
> > filter out crap like that ... Or is the alps touchpad driver so bad
> > there's not even a regex we could match it all against?
> > -Daniel
> 
> My concern is more that trying to filter out things we don't want seems like 
> an uphill battle that will become expensive quickly. First it's my touchpad, 
> then it's so and so's usb, and so on. I'm giong to ask some of the mesa guys 
> here in the office to weigh in with their thoughts, since they're all around 
> today, snice I'm basing interest on one developer's opinions.

That's kinda what the cmdline would be for, to get rid of machine-specific
stuff like your touchpad. I'm retesting igt on all the machines I have
here right now, and thus far the stack usage warning is the only offender
I've seend which wasn't a genuine issue. But yeah, more input should
definitely help.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch