[Piglit] Question about POST processing test profiles

[Timothy Arceri]

On 12/05/17 03:57, Dylan Baker wrote:
> Quoting Timothy Arceri (2017-05-10 18:27:49)
>> On 10/05/17 14:44, Timothy Arceri wrote:
>>> Hi Dylan/other python pros,
>>>
>>> I'm trying to create a new no_error profile for running tests that are
>>> suitable for use with KHR_no_error. I want to flag KHR_no_error support
>>> in the c code of the piglit tests rather than adding a field to the
>>> python "all" profile as it seem much cleaner to me, and should be easier
>>> to automatically pick if it's safe to run or skip a shader_runner test.
>>>
>>> Anyway to do this I want to base the no_error profile off the all
>>> profile and basically just append a "-khr_no_error" command line arg to
>>> all tests. However my sucking at python is getting in the way, I'm
>>> having trouble deciphering how the test groups/lists work.
>>>
>>> Can someone please advise how I could add "-khr_no_error" to the command
>>> for each test?
>>>
>>> Thanks for your time,
>>> Tim
>>
>>
>> Maybe I should give an example. So I imagined doing something like this:
>>
>> from __future__ import (
>>       absolute_import, division, print_function, unicode_literals
>> )
>>
>> from tests.all import profile as _profile
>> from framework.test import GLSLParserTest
>> from framework.test import PiglitGLTest
>>
>> __all__ = ['profile']
>>
>> profile = _profile.copy()  # pylint: disable=invalid-name
>>
>> # Remove all parser tests, as they are compiler test
>> profile.filters.append(lambda p, t: not isinstance(t, GLSLParserTest))
>> profile.filters.append(lambda n, _: not n.startswith('asmparsertest'))
>>
>> # The following code doesn't actually work but is an example of what I'd
>> # like to do. As mentioned in my previous email I'm not sure how the
>> # profiles are structured (finding the python hard to follow) and how I
>> # can iterate over the tests.
>> for group in profile.test_list:
>>       for test in group:
>>           if isinstance(test, PiglitGLTest):
>>               test.command + ['-khr_no_error']
> 
> profile.test_list is just a dictionary (mapping) of
> {<str> test_name : <Test> instance}
> 
> With that in mind, I think what you want to do is (I haven't tested this at all):
> 
> for test in profile.test_list.itervalues():
>      if isinstance(test, PiglitGLTest):
>          test.command += ['-khr_no_error']
> 
> 
> Although at the moment there isn't a public interface for altering Test.command,
> I'll send a patch for that in a second.
> 
> You should also change the name of the test to include "khr_no_error", so that
> this profile can be mixed with profiles containing the original variant, as well
> as making the summary tools happy. I'd do something like:
> 
> from __future__ import (
>       absolute_import, division, print_function, unicode_literals
> )
> 
> import six
> 
> from tests.gpu import profile as _profile
> from framework.test.shader_test import PiglitGLTest
> form framework.profile import TestDict
> 
> __all__ = ['profile']
> 
> profile = _profile.copy()  # pylint: disable=invalid-name
> 
> # Save the old test_list, but create a new one to contain the modified tests
> old_test_list = profile.test_list
> profile.test_list = TestDict()
> 
> # Add a modified version of each test as a khr_no_error variant. Since we only
> # add PiglitGLTest instances we've actively filter ShaderTest instances as well
> # as those filtered by gpu.py
> for name, test in six.iteritems(old_test_list):
>      if isinstance(test, PiglitGLTest):
>          profile.test_list['{} khr_no_error'.format(name)] = test
>          test.command += ['-khr_no_error']
> 
> Dylan
> 

This works great. Thanks again :)