[Piglit] [PATCH] junit.py: Avoid special test names.

[Mark Janes]

Jose Fonseca <jfonseca at vmware.com> writes:

> On 20/02/15 20:50, Mark Janes wrote:
>> We support using the test names as displayed in jenkins to start bisect
>> jobs.  If a test fails, you can copy/paste the name into a job, and it
>> will find the commit that broke it.
>>
>> With this change, special-cased test names can't be used with
>> --include-tests anymore.
>>
>
> Couldn't the bisect job easily trim trailing underscores to the test 
> name that is copy'n'pasted?

Yes, I could do that.  As it turns out, I won't need to.  Because we
use suffixes on all our tests, we will never have a test named "api"

The underscore will interfere with matching test names for
expected-failures in a config file.  Again, it won't matter for us, and
AFAIK we are the only ones using that feature.

>  > Instead of adding an underscore, can we change one of the characters to
>  > ".", so that the expression matching will still work?
>
> '.' is special for Jenkins -- it's equivalent to / in a path.  So I 
> suspect springing dots in the name will cause problems.

We currently append ".{hardware_suffix}" to each test name, so we can
run the same tests dozens of times on different platforms and display
all the results together.  The '.' is handled without error.

Since it won't matter to us, and it is such a tiny corner case for
everyone else, I'm fine with it.  

Reviewed-by: Mark Janes <mark.a.janes at intel.com>

> Another solution is to would be to dump the pristine name (or even 
> better, a properly quoted regex suitable to be passed in 
> -include-tests  ) in the junit test result (stdout or stderr). e.g.:
>
>    "....
>
>    To run only this test pass --include-tests '^foo/doo/boo$'"
>
>
> Jose
>
>
>>
>> -Mark
>>
>> Dylan Baker <baker.dylan.c at gmail.com> writes:
>>
>>> This looks fine to me, though I haven't tested it.
>>>
>>> Mark, does this seem good to you?
>>>
>>> Reviewed-by: Dylan Baker <baker.dylan.c at gmail.com>
>>>
>>> On Fri, Feb 20, 2015 at 11:53:50AM +0000, Jose Fonseca wrote:
>>>> For example, Jenkins was displaying its REST API page when one navigated
>>>> to  .../testReport/piglit.spec/ARB_occlusion_query2/api/ result.
>>>> ---
>>>>   framework/backends/junit.py | 10 +++++++++-
>>>>   1 file changed, 9 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/framework/backends/junit.py b/framework/backends/junit.py
>>>> index 839c99a..ddaf826 100644
>>>> --- a/framework/backends/junit.py
>>>> +++ b/framework/backends/junit.py
>>>> @@ -166,8 +166,16 @@ class JUnitBackend(FileBackend):
>>>>           # set different root names.
>>>>           classname = 'piglit.' + classname
>>>>
>>>> +        testname += self._test_suffix
>>>> +
>>>> +        # Jenkins will display special pages when the test has certain names.
>>>> +        # https://urldefense.proofpoint.com/v2/url?u=https-3A__jenkins-2Dci.org_issue_18062&d=AwIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=6iqiAc1uzBPZFCypdivM6WaoDheu37Q45KJTMIFulgo&s=3OKSPXH947_Gwt5zZ4v0DIJRLWf5-u6P4a2yc3cY1AU&e=
>>>> +        # https://urldefense.proofpoint.com/v2/url?u=https-3A__jenkins-2Dci.org_issue_19810&d=AwIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=6iqiAc1uzBPZFCypdivM6WaoDheu37Q45KJTMIFulgo&s=Kr5YcnwFlFnWhiOHklE4CT5QTR7A5PiMJFoNuK4dhaM&e=
>>>> +        if testname in ('api', 'search'):
>>>> +            testname += '_'
>>>> +
>>>>           # Create the root element
>>>> -        element = etree.Element('testcase', name=testname + self._test_suffix,
>>>> +        element = etree.Element('testcase', name=testname,
>>>>                                   classname=classname,
>>>>                                   time=str(data['time']),
>>>>                                   status=str(data['result']))
>>>> --
>>>> 2.1.0
>>>>
>>>> _______________________________________________
>>>> Piglit mailing list
>>>> Piglit at lists.freedesktop.org
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.freedesktop.org_mailman_listinfo_piglit&d=AwIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=6iqiAc1uzBPZFCypdivM6WaoDheu37Q45KJTMIFulgo&s=KdZJAJSf6EP4bKM8xp2zYCOpqQ8WY14LPCxx90LhrdY&e=