[Piglit] [PATCH 3/6] dmesg: setup dmesg to be able to mark a test warn or fail

Fri Nov 15 15:49:41 PST 2013

I decided to leave this decision to the user. Of course you should
expect false negative results with concurrency, but that's acceptable
for people who want piglit to finish in 15 minutes instead of 60 on
quad-core.

Marek

On Fri, Nov 15, 2013 at 10:51 PM, Ken Phillis Jr <kphillisjr at gmail.com> wrote:
> I believe it would be a good idea to modify the core behavior of the dmesg
> flag to make sure that it can only be enabled when you are not running in
> concurrent. This is because the wrong test could be flagged as causing the
> dmesg message when it should be another test. This only has to deal with the
> effects of synchronization.
>
>
> On Fri, Nov 15, 2013 at 12:42 PM, Marek Olšák <maraeo at gmail.com> wrote:
>>
>> Why not to have both?
>>
>> Marek
>>
>> On Fri, Nov 15, 2013 at 7:33 PM, Dylan Baker <baker.dylan.c at gmail.com>
>> wrote:
>> > On Friday, November 15, 2013 06:28:41 PM Marek Olšák wrote:
>> >> I'll gladly type --dmesg=radeon.
>> >>
>> >> I have my own script for running piglit anyway, which calls piglit-run
>> >> and piglit-summary-html, and the result and summary directories are
>> >> named $(date)_$(time)_$(GLrenderer)_$(test_filters), and it can do a
>> >> couple of other things.
>> >>
>> >> Marek
>> >>
>> >> On Fri, Nov 15, 2013 at 5:26 PM, Dylan Baker <baker.dylan.c at gmail.com>
>> > wrote:
>> >> > On Friday, November 15, 2013 10:42:11 AM Daniel Vetter wrote:
>> >> >> On Thu, Nov 14, 2013 at 10:35:03AM -0800, Dylan Baker wrote:
>> >> >> > On Thursday, November 14, 2013 05:34:38 PM Daniel Vetter wrote:
>> >> >> > > On Thu, Nov 14, 2013 at 08:04:20AM -0800, Dylan Baker wrote:
>> >> >> > > > On Thursday, November 14, 2013 02:52:41 PM Daniel Vetter
>> >> >> > > > wrote:
>> >> >> > > > > On Tue, Nov 12, 2013 at 07:54:01AM -0800, Dylan Baker wrote:
>> >> >> > > > > > This gives the dmesg class lists of statuses that will
>> >> >> > > > > > make a
>> >> >> > > > > > test
>> >> >> > > > > > a
>> >> >> > > > > > warn or a fail, it includes a few basic checks, namely
>> >> >> > > > > > i915
>> >> >> > > > > > errors
>> >> >> > > > > > and
>> >> >> > > > > > that tests have not segfaulted.
>> >> >> > > > > >
>> >> >> > > > > > Signed-off-by: Dylan Baker <baker.dylan.c at gmail.com>
>> >> >> > > > > > ---
>> >> >> > > > > >
>> >> >> > > > > >  framework/dmesg.py    | 36
>> >> >> > > > > >  ++++++++++++++++++++++++++++++++----
>> >> >> > > > > >  framework/exectest.py | 22 +++++++++++++++-------
>> >> >> > > > > >  2 files changed, 47 insertions(+), 11 deletions(-)
>> >> >> > > > > >
>> >> >> > > > > > diff --git a/framework/dmesg.py b/framework/dmesg.py
>> >> >> > > > > > index 9a23c14..edbea88 100644
>> >> >> > > > > > --- a/framework/dmesg.py
>> >> >> > > > > > +++ b/framework/dmesg.py
>> >> >> > > > > > @@ -22,6 +22,7 @@
>> >> >> > > > > >
>> >> >> > > > > >  """ Module implementing classes for reading posix dmesg
>> >> >> > > > > > """
>> >> >> > > > > >
>> >> >> > > > > >  import os
>> >> >> > > > > >
>> >> >> > > > > > +import re
>> >> >> > > > > >
>> >> >> > > > > >  import subprocess
>> >> >> > > > > >  from threads import synchronized_self
>> >> >> > > > > >
>> >> >> > > > > > @@ -29,8 +30,10 @@ __all__ = ['Dmesg']
>> >> >> > > > > >
>> >> >> > > > > >  # plain text list of statuses to be considered either a
>> >> >> > > > > > warn
>> >> >> > > > > >  or a
>> >> >> > > > > >  fail,
>> >> >> > > > > >  any
>> >> >> > > > > >  # statuses not on this list will simply be ignored.
>> >> >> > > > > >
>> >> >> > > > > > -WARN_STATUSES = []
>> >> >> > > > > > -FAIL_STATUSES = []
>> >> >> > > > > > +WARN_STATUSES = ['segfault']
>> >> >> > > > > > +FAIL_STATUSES = ['\[drm:.*\] \*ERROR\*',
>> >> >> > > > > > +                 '\[drm\] stuck on [a-zA-Z]* ring',
>> >> >> > > > > > +                 '\[drm\] GPU crash dump saved']
>> >> >> > > > >
>> >> >> > > > > I think now that we filter out all the info/debug noise
>> >> >> > > > > maybe we
>> >> >> > > > > could
>> >> >> > > > > go
>> >> >> > > > > the other direction and blacklist a few of the remaining
>> >> >> > > > > things
>> >> >> > > > > from
>> >> >> > > > > the
>> >> >> > > > > core kernel we don't care about. E.g.
>> >> >> > > > >
>> >> >> > > > > [ 3867.022895] gem_evict_every (2671) used greatest stack
>> >> >> > > > > depth:
>> >> >> > > > > 2216
>> >> >> > > > > bytes
>> >> >> > > > > left
>> >> >> > > > >
>> >> >> > > > > is a warn level message, but I don't care one bit about it
>> >> >> > > > > (as
>> >> >> > > > > long
>> >> >> > > > > as
>> >> >> > > > > it
>> >> >> > > > > doesn't approach 0). But there's other warn level stuff
>> >> >> > > > > which is
>> >> >> > > > > fairly
>> >> >> > > > > interesting.
>> >> >> > > > >
>> >> >> > > > > Just something to throw out there, I'm not sure what the
>> >> >> > > > > best way
>> >> >> > > > > would
>> >> >> > > > > be
>> >> >> > > > > to integrate dmesg reporting for piglit in general.
>> >> >> > > > > -Daniel
>> >> >> > > >
>> >> >> > > > My personal problem with the dmesg code we have now (and with
>> >> >> > > > *just*
>> >> >> > > > blacklisting) is that I have an alps touchpad, it spams dmesg
>> >> >> > > > about
>> >> >> > > > 10
>> >> >> > > > times a minute, so I can't use dmesg reporting because of the
>> >> >> > > > massive
>> >> >> > > > number of false positives; we could use some combination of
>> >> >> > > > blacklisting
>> >> >> > > > and whitelisting however.
>> >> >> > >
>> >> >> > > That sounds like we need a piglit cmdline option to supply a
>> >> >> > > regex to
>> >> >> > > filter out crap like that ... Or is the alps touchpad driver so
>> >> >> > > bad
>> >> >> > > there's not even a regex we could match it all against?
>> >> >> > > -Daniel
>> >> >> >
>> >> >> > My concern is more that trying to filter out things we don't want
>> >> >> > seems
>> >> >> > like an uphill battle that will become expensive quickly. First
>> >> >> > it's my
>> >> >> > touchpad, then it's so and so's usb, and so on. I'm giong to ask
>> >> >> > some
>> >> >> > of
>> >> >> > the mesa guys here in the office to weigh in with their thoughts,
>> >> >> > since
>> >> >> > they're all around today, snice I'm basing interest on one
>> >> >> > developer's
>> >> >> > opinions.
>> >> >>
>> >> >> That's kinda what the cmdline would be for, to get rid of
>> >> >> machine-specific
>> >> >> stuff like your touchpad. I'm retesting igt on all the machines I
>> >> >> have
>> >> >> here right now, and thus far the stack usage warning is the only
>> >> >> offender
>> >> >> I've seend which wasn't a genuine issue. But yeah, more input should
>> >> >> definitely help.
>> >> >> -Daniel
>> >> >
>> >> > So if I'm understanding correctly, you're asking users to craft regex
>> >> > on
>> >> > the commandline every time they run piglit? that sounds like a
>> >> > nightmare.
>> >
>> > Okay, now I'm confused. Are we debating include regex on the command
>> > line or
>> > exclude regex on the command line?
>> _______________________________________________
>> Piglit mailing list
>> Piglit at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/piglit
>
>