[Piglit] [PATCH 3/6] dmesg: setup dmesg to be able to mark a test warn or fail

Fri Nov 15 15:57:08 PST 2013

I agree with Marke and took the same approach. In my work-in-progress v2 
patches I have added a message in the 'note' field to go along with a dmesg-* 
status if concurrency is on noting that cocurrency makes dmesg reporting 
unreliable.

On Saturday, November 16, 2013 12:49:41 AM Marek Olšák wrote:
> I decided to leave this decision to the user. Of course you should
> expect false negative results with concurrency, but that's acceptable
> for people who want piglit to finish in 15 minutes instead of 60 on
> quad-core.
> 
> Marek
> 
> On Fri, Nov 15, 2013 at 10:51 PM, Ken Phillis Jr <kphillisjr at gmail.com> 
wrote:
> > I believe it would be a good idea to modify the core behavior of the dmesg
> > flag to make sure that it can only be enabled when you are not running in
> > concurrent. This is because the wrong test could be flagged as causing the
> > dmesg message when it should be another test. This only has to deal with
> > the effects of synchronization.
> > 
> > On Fri, Nov 15, 2013 at 12:42 PM, Marek Olšák <maraeo at gmail.com> wrote:
> >> Why not to have both?
> >> 
> >> Marek
> >> 
> >> On Fri, Nov 15, 2013 at 7:33 PM, Dylan Baker <baker.dylan.c at gmail.com>
> >> 
> >> wrote:
> >> > On Friday, November 15, 2013 06:28:41 PM Marek Olšák wrote:
> >> >> I'll gladly type --dmesg=radeon.
> >> >> 
> >> >> I have my own script for running piglit anyway, which calls piglit-run
> >> >> and piglit-summary-html, and the result and summary directories are
> >> >> named $(date)_$(time)_$(GLrenderer)_$(test_filters), and it can do a
> >> >> couple of other things.
> >> >> 
> >> >> Marek
> >> >> 
> >> >> On Fri, Nov 15, 2013 at 5:26 PM, Dylan Baker <baker.dylan.c at gmail.com>
> >> > 
> >> > wrote:
> >> >> > On Friday, November 15, 2013 10:42:11 AM Daniel Vetter wrote:
> >> >> >> On Thu, Nov 14, 2013 at 10:35:03AM -0800, Dylan Baker wrote:
> >> >> >> > On Thursday, November 14, 2013 05:34:38 PM Daniel Vetter wrote:
> >> >> >> > > On Thu, Nov 14, 2013 at 08:04:20AM -0800, Dylan Baker wrote:
> >> >> >> > > > On Thursday, November 14, 2013 02:52:41 PM Daniel Vetter
> >> >> >> > > > 
> >> >> >> > > > wrote:
> >> >> >> > > > > On Tue, Nov 12, 2013 at 07:54:01AM -0800, Dylan Baker 
wrote:
> >> >> >> > > > > > This gives the dmesg class lists of statuses that will
> >> >> >> > > > > > make a
> >> >> >> > > > > > test
> >> >> >> > > > > > a
> >> >> >> > > > > > warn or a fail, it includes a few basic checks, namely
> >> >> >> > > > > > i915
> >> >> >> > > > > > errors
> >> >> >> > > > > > and
> >> >> >> > > > > > that tests have not segfaulted.
> >> >> >> > > > > > 
> >> >> >> > > > > > Signed-off-by: Dylan Baker <baker.dylan.c at gmail.com>
> >> >> >> > > > > > ---
> >> >> >> > > > > > 
> >> >> >> > > > > >  framework/dmesg.py    | 36
> >> >> >> > > > > >  ++++++++++++++++++++++++++++++++----
> >> >> >> > > > > >  framework/exectest.py | 22 +++++++++++++++-------
> >> >> >> > > > > >  2 files changed, 47 insertions(+), 11 deletions(-)
> >> >> >> > > > > > 
> >> >> >> > > > > > diff --git a/framework/dmesg.py b/framework/dmesg.py
> >> >> >> > > > > > index 9a23c14..edbea88 100644
> >> >> >> > > > > > --- a/framework/dmesg.py
> >> >> >> > > > > > +++ b/framework/dmesg.py
> >> >> >> > > > > > @@ -22,6 +22,7 @@
> >> >> >> > > > > > 
> >> >> >> > > > > >  """ Module implementing classes for reading posix dmesg
> >> >> >> > > > > > 
> >> >> >> > > > > > """
> >> >> >> > > > > > 
> >> >> >> > > > > >  import os
> >> >> >> > > > > > 
> >> >> >> > > > > > +import re
> >> >> >> > > > > > 
> >> >> >> > > > > >  import subprocess
> >> >> >> > > > > >  from threads import synchronized_self
> >> >> >> > > > > > 
> >> >> >> > > > > > @@ -29,8 +30,10 @@ __all__ = ['Dmesg']
> >> >> >> > > > > > 
> >> >> >> > > > > >  # plain text list of statuses to be considered either a
> >> >> >> > > > > > 
> >> >> >> > > > > > warn
> >> >> >> > > > > > 
> >> >> >> > > > > >  or a
> >> >> >> > > > > >  fail,
> >> >> >> > > > > >  any
> >> >> >> > > > > >  # statuses not on this list will simply be ignored.
> >> >> >> > > > > > 
> >> >> >> > > > > > -WARN_STATUSES = []
> >> >> >> > > > > > -FAIL_STATUSES = []
> >> >> >> > > > > > +WARN_STATUSES = ['segfault']
> >> >> >> > > > > > +FAIL_STATUSES = ['\[drm:.*\] \*ERROR\*',
> >> >> >> > > > > > +                 '\[drm\] stuck on [a-zA-Z]* ring',
> >> >> >> > > > > > +                 '\[drm\] GPU crash dump saved']
> >> >> >> > > > > 
> >> >> >> > > > > I think now that we filter out all the info/debug noise
> >> >> >> > > > > maybe we
> >> >> >> > > > > could
> >> >> >> > > > > go
> >> >> >> > > > > the other direction and blacklist a few of the remaining
> >> >> >> > > > > things
> >> >> >> > > > > from
> >> >> >> > > > > the
> >> >> >> > > > > core kernel we don't care about. E.g.
> >> >> >> > > > > 
> >> >> >> > > > > [ 3867.022895] gem_evict_every (2671) used greatest stack
> >> >> >> > > > > depth:
> >> >> >> > > > > 2216
> >> >> >> > > > > bytes
> >> >> >> > > > > left
> >> >> >> > > > > 
> >> >> >> > > > > is a warn level message, but I don't care one bit about it
> >> >> >> > > > > (as
> >> >> >> > > > > long
> >> >> >> > > > > as
> >> >> >> > > > > it
> >> >> >> > > > > doesn't approach 0). But there's other warn level stuff
> >> >> >> > > > > which is
> >> >> >> > > > > fairly
> >> >> >> > > > > interesting.
> >> >> >> > > > > 
> >> >> >> > > > > Just something to throw out there, I'm not sure what the
> >> >> >> > > > > best way
> >> >> >> > > > > would
> >> >> >> > > > > be
> >> >> >> > > > > to integrate dmesg reporting for piglit in general.
> >> >> >> > > > > -Daniel
> >> >> >> > > > 
> >> >> >> > > > My personal problem with the dmesg code we have now (and with
> >> >> >> > > > *just*
> >> >> >> > > > blacklisting) is that I have an alps touchpad, it spams dmesg
> >> >> >> > > > about
> >> >> >> > > > 10
> >> >> >> > > > times a minute, so I can't use dmesg reporting because of the
> >> >> >> > > > massive
> >> >> >> > > > number of false positives; we could use some combination of
> >> >> >> > > > blacklisting
> >> >> >> > > > and whitelisting however.
> >> >> >> > > 
> >> >> >> > > That sounds like we need a piglit cmdline option to supply a
> >> >> >> > > regex to
> >> >> >> > > filter out crap like that ... Or is the alps touchpad driver so
> >> >> >> > > bad
> >> >> >> > > there's not even a regex we could match it all against?
> >> >> >> > > -Daniel
> >> >> >> > 
> >> >> >> > My concern is more that trying to filter out things we don't want
> >> >> >> > seems
> >> >> >> > like an uphill battle that will become expensive quickly. First
> >> >> >> > it's my
> >> >> >> > touchpad, then it's so and so's usb, and so on. I'm giong to ask
> >> >> >> > some
> >> >> >> > of
> >> >> >> > the mesa guys here in the office to weigh in with their thoughts,
> >> >> >> > since
> >> >> >> > they're all around today, snice I'm basing interest on one
> >> >> >> > developer's
> >> >> >> > opinions.
> >> >> >> 
> >> >> >> That's kinda what the cmdline would be for, to get rid of
> >> >> >> machine-specific
> >> >> >> stuff like your touchpad. I'm retesting igt on all the machines I
> >> >> >> have
> >> >> >> here right now, and thus far the stack usage warning is the only
> >> >> >> offender
> >> >> >> I've seend which wasn't a genuine issue. But yeah, more input
> >> >> >> should
> >> >> >> definitely help.
> >> >> >> -Daniel
> >> >> > 
> >> >> > So if I'm understanding correctly, you're asking users to craft
> >> >> > regex
> >> >> > on
> >> >> > the commandline every time they run piglit? that sounds like a
> >> >> > nightmare.
> >> > 
> >> > Okay, now I'm confused. Are we debating include regex on the command
> >> > line or
> >> > exclude regex on the command line?
> >> 
> >> _______________________________________________
> >> Piglit mailing list
> >> Piglit at lists.freedesktop.org
> >> http://lists.freedesktop.org/mailman/listinfo/piglit
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20131115/5b406d1d/attachment.pgp>