[Piglit] Newbie question: command to run tests that passed before?

[Dylan Baker]

Quoting Dylan Baker (2016-09-07 16:23:43)
> Quoting Dan Kegel (2016-09-07 15:59:21)
> > Hi!
> > I'd like to use piglit as a regression test for my opengl test rigs.
> > My plan is to run piglit a few times on each OS/card combo I need to support,
> > get a list of tests that reliably pass on each platform, and then
> > forevermore run just those tests.
> > 
> > I know about the -t and -x options, and the --test-list option, and
> > about writing test profiles,
> > but am still not sure what the best way to generate a list of passing
> > tests to pass to e.g. --test-list.
> > Test names are a little hard for this newbie to identify in the output
> > of piglit summary console.
> > 
> > After looking at the source a bit, I tried generating a list of passed
> > tests like this:
> >     ./piglit summary console results/quick | grep ': pass$' | sed
> > 's/:.*//' | tr / @ > tests.list
> > but that includes output
> > spec@!opengl 1.1 at max-texture-size-level
> > spec@!opengl 1.1 at max-texture-size@GL_PROXY_TEXTURE_1D-GL_RGBA16
> > 
> > which leads to errors from --test-list tests.list like
> > Fatal Error: Cannot reorder test: "spec@!opengl
> > 1.1 at max-texture-size@GL_PROXY_TEXTURE_1D-GL_RGBA16", it is not in the
> > profile.
> > 
> > Suggestions?
> 
> That's an interesting corner you've found. What's happening is that
> GL_PROXY_TEXTURE_1D-GL_RGBA16 is a subtest of 
> spec@!opengl 1.1 at max-texture-size, and thus isn't in the test list at
> all.
> 
> This is actually an odd corner of a known deficiency, the inability to
> filter on subtests. I've been thinking a lot about how to fix this, but
> there isn't a straight forward way to fix it (that I've been able to
> come up with). I guess I should bump this up on my list of things to do.
> 
> The easiest thing I can com up with off the top my head would be to
> write a little script to parse the output and output what you want.
> 
> Something like this should work (untested):
> 
> """
> import argparse
> 
> from framework import backends
> 
> def main():
>     parser = argparse.ArgumentParser()
>     parser.add_argument('file')
>     args = parser.parse_args()
> 
>     results = backends.load(args._file)

Oops, that should be 'args.file'. Just my auto-completion being
"helpful".

>     with open('myfile', 'w') as f:
>         for name, value in results.tests.items():
>             if value.result == 'pass':
>                 f.write(name)
>                 f.write('\n')
> 
> 
> if __name__ == '__main__':
>     main()
> """
> 
> Obviously you'll need to be in the piglit source directory or add it to
> your PYTHON_PATH to make this work, since it relies on using the piglit
> framework.
> 
> Dylan
> 
> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20160907/b500899c/attachment.sig>