[Piglit] [PATCH 3/6] dmesg: setup dmesg to be able to mark a test warn or fail

[Marek Olšák]

Why not to have both?

Marek

On Fri, Nov 15, 2013 at 7:33 PM, Dylan Baker <baker.dylan.c at gmail.com> wrote:
> On Friday, November 15, 2013 06:28:41 PM Marek Olšák wrote:
>> I'll gladly type --dmesg=radeon.
>>
>> I have my own script for running piglit anyway, which calls piglit-run
>> and piglit-summary-html, and the result and summary directories are
>> named $(date)_$(time)_$(GLrenderer)_$(test_filters), and it can do a
>> couple of other things.
>>
>> Marek
>>
>> On Fri, Nov 15, 2013 at 5:26 PM, Dylan Baker <baker.dylan.c at gmail.com>
> wrote:
>> > On Friday, November 15, 2013 10:42:11 AM Daniel Vetter wrote:
>> >> On Thu, Nov 14, 2013 at 10:35:03AM -0800, Dylan Baker wrote:
>> >> > On Thursday, November 14, 2013 05:34:38 PM Daniel Vetter wrote:
>> >> > > On Thu, Nov 14, 2013 at 08:04:20AM -0800, Dylan Baker wrote:
>> >> > > > On Thursday, November 14, 2013 02:52:41 PM Daniel Vetter wrote:
>> >> > > > > On Tue, Nov 12, 2013 at 07:54:01AM -0800, Dylan Baker wrote:
>> >> > > > > > This gives the dmesg class lists of statuses that will make a
>> >> > > > > > test
>> >> > > > > > a
>> >> > > > > > warn or a fail, it includes a few basic checks, namely i915
>> >> > > > > > errors
>> >> > > > > > and
>> >> > > > > > that tests have not segfaulted.
>> >> > > > > >
>> >> > > > > > Signed-off-by: Dylan Baker <baker.dylan.c at gmail.com>
>> >> > > > > > ---
>> >> > > > > >
>> >> > > > > >  framework/dmesg.py    | 36
>> >> > > > > >  ++++++++++++++++++++++++++++++++----
>> >> > > > > >  framework/exectest.py | 22 +++++++++++++++-------
>> >> > > > > >  2 files changed, 47 insertions(+), 11 deletions(-)
>> >> > > > > >
>> >> > > > > > diff --git a/framework/dmesg.py b/framework/dmesg.py
>> >> > > > > > index 9a23c14..edbea88 100644
>> >> > > > > > --- a/framework/dmesg.py
>> >> > > > > > +++ b/framework/dmesg.py
>> >> > > > > > @@ -22,6 +22,7 @@
>> >> > > > > >
>> >> > > > > >  """ Module implementing classes for reading posix dmesg """
>> >> > > > > >
>> >> > > > > >  import os
>> >> > > > > >
>> >> > > > > > +import re
>> >> > > > > >
>> >> > > > > >  import subprocess
>> >> > > > > >  from threads import synchronized_self
>> >> > > > > >
>> >> > > > > > @@ -29,8 +30,10 @@ __all__ = ['Dmesg']
>> >> > > > > >
>> >> > > > > >  # plain text list of statuses to be considered either a warn
>> >> > > > > >  or a
>> >> > > > > >  fail,
>> >> > > > > >  any
>> >> > > > > >  # statuses not on this list will simply be ignored.
>> >> > > > > >
>> >> > > > > > -WARN_STATUSES = []
>> >> > > > > > -FAIL_STATUSES = []
>> >> > > > > > +WARN_STATUSES = ['segfault']
>> >> > > > > > +FAIL_STATUSES = ['\[drm:.*\] \*ERROR\*',
>> >> > > > > > +                 '\[drm\] stuck on [a-zA-Z]* ring',
>> >> > > > > > +                 '\[drm\] GPU crash dump saved']
>> >> > > > >
>> >> > > > > I think now that we filter out all the info/debug noise maybe we
>> >> > > > > could
>> >> > > > > go
>> >> > > > > the other direction and blacklist a few of the remaining things
>> >> > > > > from
>> >> > > > > the
>> >> > > > > core kernel we don't care about. E.g.
>> >> > > > >
>> >> > > > > [ 3867.022895] gem_evict_every (2671) used greatest stack depth:
>> >> > > > > 2216
>> >> > > > > bytes
>> >> > > > > left
>> >> > > > >
>> >> > > > > is a warn level message, but I don't care one bit about it (as
>> >> > > > > long
>> >> > > > > as
>> >> > > > > it
>> >> > > > > doesn't approach 0). But there's other warn level stuff which is
>> >> > > > > fairly
>> >> > > > > interesting.
>> >> > > > >
>> >> > > > > Just something to throw out there, I'm not sure what the best way
>> >> > > > > would
>> >> > > > > be
>> >> > > > > to integrate dmesg reporting for piglit in general.
>> >> > > > > -Daniel
>> >> > > >
>> >> > > > My personal problem with the dmesg code we have now (and with
>> >> > > > *just*
>> >> > > > blacklisting) is that I have an alps touchpad, it spams dmesg about
>> >> > > > 10
>> >> > > > times a minute, so I can't use dmesg reporting because of the
>> >> > > > massive
>> >> > > > number of false positives; we could use some combination of
>> >> > > > blacklisting
>> >> > > > and whitelisting however.
>> >> > >
>> >> > > That sounds like we need a piglit cmdline option to supply a regex to
>> >> > > filter out crap like that ... Or is the alps touchpad driver so bad
>> >> > > there's not even a regex we could match it all against?
>> >> > > -Daniel
>> >> >
>> >> > My concern is more that trying to filter out things we don't want seems
>> >> > like an uphill battle that will become expensive quickly. First it's my
>> >> > touchpad, then it's so and so's usb, and so on. I'm giong to ask some
>> >> > of
>> >> > the mesa guys here in the office to weigh in with their thoughts, since
>> >> > they're all around today, snice I'm basing interest on one developer's
>> >> > opinions.
>> >>
>> >> That's kinda what the cmdline would be for, to get rid of
>> >> machine-specific
>> >> stuff like your touchpad. I'm retesting igt on all the machines I have
>> >> here right now, and thus far the stack usage warning is the only offender
>> >> I've seend which wasn't a genuine issue. But yeah, more input should
>> >> definitely help.
>> >> -Daniel
>> >
>> > So if I'm understanding correctly, you're asking users to craft regex on
>> > the commandline every time they run piglit? that sounds like a nightmare.
>
> Okay, now I'm confused. Are we debating include regex on the command line or
> exclude regex on the command line?