[Piglit] Question about POST processing test profiles

[Dylan Baker]

Quoting Timothy Arceri (2017-05-10 18:27:49)
> On 10/05/17 14:44, Timothy Arceri wrote:
> > Hi Dylan/other python pros,
> > 
> > I'm trying to create a new no_error profile for running tests that are 
> > suitable for use with KHR_no_error. I want to flag KHR_no_error support 
> > in the c code of the piglit tests rather than adding a field to the 
> > python "all" profile as it seem much cleaner to me, and should be easier 
> > to automatically pick if it's safe to run or skip a shader_runner test.
> > 
> > Anyway to do this I want to base the no_error profile off the all 
> > profile and basically just append a "-khr_no_error" command line arg to 
> > all tests. However my sucking at python is getting in the way, I'm 
> > having trouble deciphering how the test groups/lists work.
> > 
> > Can someone please advise how I could add "-khr_no_error" to the command 
> > for each test?
> > 
> > Thanks for your time,
> > Tim
> 
> 
> Maybe I should give an example. So I imagined doing something like this:
> 
> from __future__ import (
>      absolute_import, division, print_function, unicode_literals
> )
> 
> from tests.all import profile as _profile
> from framework.test import GLSLParserTest
> from framework.test import PiglitGLTest
> 
> __all__ = ['profile']
> 
> profile = _profile.copy()  # pylint: disable=invalid-name
> 
> # Remove all parser tests, as they are compiler test
> profile.filters.append(lambda p, t: not isinstance(t, GLSLParserTest))
> profile.filters.append(lambda n, _: not n.startswith('asmparsertest'))
> 
> # The following code doesn't actually work but is an example of what I'd
> # like to do. As mentioned in my previous email I'm not sure how the
> # profiles are structured (finding the python hard to follow) and how I
> # can iterate over the tests.
> for group in profile.test_list:
>      for test in group:
>          if isinstance(test, PiglitGLTest):
>              test.command + ['-khr_no_error']

profile.test_list is just a dictionary (mapping) of 
{<str> test_name : <Test> instance}

With that in mind, I think what you want to do is (I haven't tested this at all):

for test in profile.test_list.itervalues():
    if isinstance(test, PiglitGLTest):
        test.command += ['-khr_no_error']


Although at the moment there isn't a public interface for altering Test.command,
I'll send a patch for that in a second.

You should also change the name of the test to include "khr_no_error", so that
this profile can be mixed with profiles containing the original variant, as well
as making the summary tools happy. I'd do something like:

from __future__ import (
     absolute_import, division, print_function, unicode_literals
)

import six

from tests.gpu import profile as _profile
from framework.test.shader_test import PiglitGLTest
form framework.profile import TestDict

__all__ = ['profile']

profile = _profile.copy()  # pylint: disable=invalid-name

# Save the old test_list, but create a new one to contain the modified tests
old_test_list = profile.test_list
profile.test_list = TestDict()

# Add a modified version of each test as a khr_no_error variant. Since we only
# add PiglitGLTest instances we've actively filter ShaderTest instances as well
# as those filtered by gpu.py
for name, test in six.iteritems(old_test_list):
    if isinstance(test, PiglitGLTest):
        profile.test_list['{} khr_no_error'.format(name)] = test
        test.command += ['-khr_no_error']

Dylan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: signature
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20170511/92dc2464/attachment.sig>