[Piglit] [PATCH 3/6] dmesg: setup dmesg to be able to mark a test warn or fail

[Dylan Baker]

On Friday, November 15, 2013 06:28:41 PM Marek Olšák wrote:
> I'll gladly type --dmesg=radeon.
> 
> I have my own script for running piglit anyway, which calls piglit-run
> and piglit-summary-html, and the result and summary directories are
> named $(date)_$(time)_$(GLrenderer)_$(test_filters), and it can do a
> couple of other things.
> 
> Marek
> 
> On Fri, Nov 15, 2013 at 5:26 PM, Dylan Baker <baker.dylan.c at gmail.com> 
wrote:
> > On Friday, November 15, 2013 10:42:11 AM Daniel Vetter wrote:
> >> On Thu, Nov 14, 2013 at 10:35:03AM -0800, Dylan Baker wrote:
> >> > On Thursday, November 14, 2013 05:34:38 PM Daniel Vetter wrote:
> >> > > On Thu, Nov 14, 2013 at 08:04:20AM -0800, Dylan Baker wrote:
> >> > > > On Thursday, November 14, 2013 02:52:41 PM Daniel Vetter wrote:
> >> > > > > On Tue, Nov 12, 2013 at 07:54:01AM -0800, Dylan Baker wrote:
> >> > > > > > This gives the dmesg class lists of statuses that will make a
> >> > > > > > test
> >> > > > > > a
> >> > > > > > warn or a fail, it includes a few basic checks, namely i915
> >> > > > > > errors
> >> > > > > > and
> >> > > > > > that tests have not segfaulted.
> >> > > > > > 
> >> > > > > > Signed-off-by: Dylan Baker <baker.dylan.c at gmail.com>
> >> > > > > > ---
> >> > > > > > 
> >> > > > > >  framework/dmesg.py    | 36
> >> > > > > >  ++++++++++++++++++++++++++++++++----
> >> > > > > >  framework/exectest.py | 22 +++++++++++++++-------
> >> > > > > >  2 files changed, 47 insertions(+), 11 deletions(-)
> >> > > > > > 
> >> > > > > > diff --git a/framework/dmesg.py b/framework/dmesg.py
> >> > > > > > index 9a23c14..edbea88 100644
> >> > > > > > --- a/framework/dmesg.py
> >> > > > > > +++ b/framework/dmesg.py
> >> > > > > > @@ -22,6 +22,7 @@
> >> > > > > > 
> >> > > > > >  """ Module implementing classes for reading posix dmesg """
> >> > > > > >  
> >> > > > > >  import os
> >> > > > > > 
> >> > > > > > +import re
> >> > > > > > 
> >> > > > > >  import subprocess
> >> > > > > >  from threads import synchronized_self
> >> > > > > > 
> >> > > > > > @@ -29,8 +30,10 @@ __all__ = ['Dmesg']
> >> > > > > > 
> >> > > > > >  # plain text list of statuses to be considered either a warn
> >> > > > > >  or a
> >> > > > > >  fail,
> >> > > > > >  any
> >> > > > > >  # statuses not on this list will simply be ignored.
> >> > > > > > 
> >> > > > > > -WARN_STATUSES = []
> >> > > > > > -FAIL_STATUSES = []
> >> > > > > > +WARN_STATUSES = ['segfault']
> >> > > > > > +FAIL_STATUSES = ['\[drm:.*\] \*ERROR\*',
> >> > > > > > +                 '\[drm\] stuck on [a-zA-Z]* ring',
> >> > > > > > +                 '\[drm\] GPU crash dump saved']
> >> > > > > 
> >> > > > > I think now that we filter out all the info/debug noise maybe we
> >> > > > > could
> >> > > > > go
> >> > > > > the other direction and blacklist a few of the remaining things
> >> > > > > from
> >> > > > > the
> >> > > > > core kernel we don't care about. E.g.
> >> > > > > 
> >> > > > > [ 3867.022895] gem_evict_every (2671) used greatest stack depth:
> >> > > > > 2216
> >> > > > > bytes
> >> > > > > left
> >> > > > > 
> >> > > > > is a warn level message, but I don't care one bit about it (as
> >> > > > > long
> >> > > > > as
> >> > > > > it
> >> > > > > doesn't approach 0). But there's other warn level stuff which is
> >> > > > > fairly
> >> > > > > interesting.
> >> > > > > 
> >> > > > > Just something to throw out there, I'm not sure what the best way
> >> > > > > would
> >> > > > > be
> >> > > > > to integrate dmesg reporting for piglit in general.
> >> > > > > -Daniel
> >> > > > 
> >> > > > My personal problem with the dmesg code we have now (and with
> >> > > > *just*
> >> > > > blacklisting) is that I have an alps touchpad, it spams dmesg about
> >> > > > 10
> >> > > > times a minute, so I can't use dmesg reporting because of the
> >> > > > massive
> >> > > > number of false positives; we could use some combination of
> >> > > > blacklisting
> >> > > > and whitelisting however.
> >> > > 
> >> > > That sounds like we need a piglit cmdline option to supply a regex to
> >> > > filter out crap like that ... Or is the alps touchpad driver so bad
> >> > > there's not even a regex we could match it all against?
> >> > > -Daniel
> >> > 
> >> > My concern is more that trying to filter out things we don't want seems
> >> > like an uphill battle that will become expensive quickly. First it's my
> >> > touchpad, then it's so and so's usb, and so on. I'm giong to ask some
> >> > of
> >> > the mesa guys here in the office to weigh in with their thoughts, since
> >> > they're all around today, snice I'm basing interest on one developer's
> >> > opinions.
> >> 
> >> That's kinda what the cmdline would be for, to get rid of
> >> machine-specific
> >> stuff like your touchpad. I'm retesting igt on all the machines I have
> >> here right now, and thus far the stack usage warning is the only offender
> >> I've seend which wasn't a genuine issue. But yeah, more input should
> >> definitely help.
> >> -Daniel
> > 
> > So if I'm understanding correctly, you're asking users to craft regex on
> > the commandline every time they run piglit? that sounds like a nightmare.

Okay, now I'm confused. Are we debating include regex on the command line or 
exclude regex on the command line?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20131115/3a311166/attachment-0001.pgp>