[Piglit] Newbie question: command to run tests that passed before?
mark.a.janes at intel.com
Wed Sep 7 23:24:11 UTC 2016
Dan Kegel <dank at kegel.com> writes:
> Hrm. A solution that involved Jenkins would be way overkill.
> I'm just looking for the little core idiom for running just the
> test of tests that passed in the, um, past.
Even if you don't want to set up a CI with jenkins, you can refer to the
automation that I've written at https://github.com/janesma/mesa_jenkins
Our approach is to list all failing/crashing tests in a config file, and
pass that to piglit. Piglit converts "known failing" tests to a "skip"
status. The only tests we can't run are the ones that have flaky
results or cause gpu hang.
> Maybe that's not the way people do it? Does everyone just compare
> logs against old logs?
> Thanks, and say hi to Ethyl for me.
> - Dan
> On Wed, Sep 7, 2016 at 4:08 PM, Ilia Mirkin <imirkin at alum.mit.edu> wrote:
>> Mark has set up a CI system at Intel, and I'm fairly sure did a
>> writeup about it and shared code, but I can't for the life of me
>> remember the details. Maybe a few Guinesses ago, but definitely not
>> now... hopefully he can share. [It involves Jenkins.]
>> On Wed, Sep 7, 2016 at 6:59 PM, Dan Kegel <dank at kegel.com> wrote:
>>> I'd like to use piglit as a regression test for my opengl test rigs.
>>> My plan is to run piglit a few times on each OS/card combo I need to support,
>>> get a list of tests that reliably pass on each platform, and then
>>> forevermore run just those tests.
>>> I know about the -t and -x options, and the --test-list option, and
>>> about writing test profiles,
>>> but am still not sure what the best way to generate a list of passing
>>> tests to pass to e.g. --test-list.
>>> Test names are a little hard for this newbie to identify in the output
>>> of piglit summary console.
>>> After looking at the source a bit, I tried generating a list of passed
>>> tests like this:
>>> ./piglit summary console results/quick | grep ': pass$' | sed
>>> 's/:.*//' | tr / @ > tests.list
>>> but that includes output
>>> spec@!opengl 1.1 at max-texture-size-level
>>> spec@!opengl 1.1 at max-texture-size@GL_PROXY_TEXTURE_1D-GL_RGBA16
>>> which leads to errors from --test-list tests.list like
>>> Fatal Error: Cannot reorder test: "spec@!opengl
>>> 1.1 at max-texture-size@GL_PROXY_TEXTURE_1D-GL_RGBA16", it is not in the
>>> p.s. Mesa Haswell is behaving very well on stock ubuntu 16.04, only 1%
>>> of quick tests fail, and there are no hangs or kernel complaints.
>>> Piglit mailing list
>>> Piglit at lists.freedesktop.org
More information about the Piglit