[Piglit] Question about POST processing test profiles

[Timothy Arceri]

On 12/05/17 05:09, Dylan Baker wrote:
> The test-command-setter on my github repo should provide all of the groundwork
> you need.

Thanks for your help, I'll take a look :)

> 
> Dylan
> 
> Quoting Dylan Baker (2017-05-11 10:57:32)
>> Quoting Timothy Arceri (2017-05-10 18:27:49)
>>> On 10/05/17 14:44, Timothy Arceri wrote:
>>>> Hi Dylan/other python pros,
>>>>
>>>> I'm trying to create a new no_error profile for running tests that are
>>>> suitable for use with KHR_no_error. I want to flag KHR_no_error support
>>>> in the c code of the piglit tests rather than adding a field to the
>>>> python "all" profile as it seem much cleaner to me, and should be easier
>>>> to automatically pick if it's safe to run or skip a shader_runner test.
>>>>
>>>> Anyway to do this I want to base the no_error profile off the all
>>>> profile and basically just append a "-khr_no_error" command line arg to
>>>> all tests. However my sucking at python is getting in the way, I'm
>>>> having trouble deciphering how the test groups/lists work.
>>>>
>>>> Can someone please advise how I could add "-khr_no_error" to the command
>>>> for each test?
>>>>
>>>> Thanks for your time,
>>>> Tim
>>>
>>>
>>> Maybe I should give an example. So I imagined doing something like this:
>>>
>>> from __future__ import (
>>>       absolute_import, division, print_function, unicode_literals
>>> )
>>>
>>> from tests.all import profile as _profile
>>> from framework.test import GLSLParserTest
>>> from framework.test import PiglitGLTest
>>>
>>> __all__ = ['profile']
>>>
>>> profile = _profile.copy()  # pylint: disable=invalid-name
>>>
>>> # Remove all parser tests, as they are compiler test
>>> profile.filters.append(lambda p, t: not isinstance(t, GLSLParserTest))
>>> profile.filters.append(lambda n, _: not n.startswith('asmparsertest'))
>>>
>>> # The following code doesn't actually work but is an example of what I'd
>>> # like to do. As mentioned in my previous email I'm not sure how the
>>> # profiles are structured (finding the python hard to follow) and how I
>>> # can iterate over the tests.
>>> for group in profile.test_list:
>>>       for test in group:
>>>           if isinstance(test, PiglitGLTest):
>>>               test.command + ['-khr_no_error']
>>
>> profile.test_list is just a dictionary (mapping) of
>> {<str> test_name : <Test> instance}
>>
>> With that in mind, I think what you want to do is (I haven't tested this at all):
>>
>> for test in profile.test_list.itervalues():
>>      if isinstance(test, PiglitGLTest):
>>          test.command += ['-khr_no_error']
>>
>>
>> Although at the moment there isn't a public interface for altering Test.command,
>> I'll send a patch for that in a second.
>>
>> You should also change the name of the test to include "khr_no_error", so that
>> this profile can be mixed with profiles containing the original variant, as well
>> as making the summary tools happy. I'd do something like:
>>
>> from __future__ import (
>>       absolute_import, division, print_function, unicode_literals
>> )
>>
>> import six
>>
>> from tests.gpu import profile as _profile
>> from framework.test.shader_test import PiglitGLTest
>> form framework.profile import TestDict
>>
>> __all__ = ['profile']
>>
>> profile = _profile.copy()  # pylint: disable=invalid-name
>>
>> # Save the old test_list, but create a new one to contain the modified tests
>> old_test_list = profile.test_list
>> profile.test_list = TestDict()
>>
>> # Add a modified version of each test as a khr_no_error variant. Since we only
>> # add PiglitGLTest instances we've actively filter ShaderTest instances as well
>> # as those filtered by gpu.py
>> for name, test in six.iteritems(old_test_list):
>>      if isinstance(test, PiglitGLTest):
>>          profile.test_list['{} khr_no_error'.format(name)] = test
>>          test.command += ['-khr_no_error']
>>
>> Dylan