[Piglit] Newbie question: command to run tests that passed before?

[Dylan Baker]

Quoting Dan Kegel (2016-09-07 15:59:21)
> Hi!
> I'd like to use piglit as a regression test for my opengl test rigs.
> My plan is to run piglit a few times on each OS/card combo I need to support,
> get a list of tests that reliably pass on each platform, and then
> forevermore run just those tests.
> 
> I know about the -t and -x options, and the --test-list option, and
> about writing test profiles,
> but am still not sure what the best way to generate a list of passing
> tests to pass to e.g. --test-list.
> Test names are a little hard for this newbie to identify in the output
> of piglit summary console.
> 
> After looking at the source a bit, I tried generating a list of passed
> tests like this:
>     ./piglit summary console results/quick | grep ': pass$' | sed
> 's/:.*//' | tr / @ > tests.list
> but that includes output
> spec@!opengl 1.1 at max-texture-size-level
> spec@!opengl 1.1 at max-texture-size@GL_PROXY_TEXTURE_1D-GL_RGBA16
> 
> which leads to errors from --test-list tests.list like
> Fatal Error: Cannot reorder test: "spec@!opengl
> 1.1 at max-texture-size@GL_PROXY_TEXTURE_1D-GL_RGBA16", it is not in the
> profile.
> 
> Suggestions?

That's an interesting corner you've found. What's happening is that
GL_PROXY_TEXTURE_1D-GL_RGBA16 is a subtest of 
spec@!opengl 1.1 at max-texture-size, and thus isn't in the test list at
all.

This is actually an odd corner of a known deficiency, the inability to
filter on subtests. I've been thinking a lot about how to fix this, but
there isn't a straight forward way to fix it (that I've been able to
come up with). I guess I should bump this up on my list of things to do.

The easiest thing I can com up with off the top my head would be to
write a little script to parse the output and output what you want.

Something like this should work (untested):

"""
import argparse

from framework import backends

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('file')
    args = parser.parse_args()

    results = backends.load(args._file)
    with open('myfile', 'w') as f:
        for name, value in results.tests.items():
            if value.result == 'pass':
                f.write(name)
                f.write('\n')


if __name__ == '__main__':
    main()
"""

Obviously you'll need to be in the piglit source directory or add it to
your PYTHON_PATH to make this work, since it relies on using the piglit
framework.

Dylan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20160907/5c3a1e14/attachment.sig>