[Piglit] [PATCH] junit.py: Avoid special test names.

[Jose Fonseca]

On 20/02/15 22:18, Mark Janes wrote:
> Jose Fonseca <jfonseca at vmware.com> writes:
>
>> On 20/02/15 20:50, Mark Janes wrote:
>>> We support using the test names as displayed in jenkins to start bisect
>>> jobs.  If a test fails, you can copy/paste the name into a job, and it
>>> will find the commit that broke it.
>>>
>>> With this change, special-cased test names can't be used with
>>> --include-tests anymore.
>>>
>>
>> Couldn't the bisect job easily trim trailing underscores to the test
>> name that is copy'n'pasted?
>
> Yes, I could do that.  As it turns out, I won't need to.  Because we
> use suffixes on all our tests, we will never have a test named "api"
>
> The underscore will interfere with matching test names for
> expected-failures in a config file.  Again, it won't matter for us, and
> AFAIK we are the only ones using that feature.
>
>>   > Instead of adding an underscore, can we change one of the characters to
>>   > ".", so that the expression matching will still work?
>>
>> '.' is special for Jenkins -- it's equivalent to / in a path.  So I
>> suspect springing dots in the name will cause problems.
>
> We currently append ".{hardware_suffix}" to each test name, so we can
> run the same tests dozens of times on different platforms and display
> all the results together.  The '.' is handled without error.
>
> Since it won't matter to us, and it is such a tiny corner case for
> everyone else, I'm fine with it.

Ah, yes, if you use a suffix then testnames with api/search can never 
happen.

> Reviewed-by: Mark Janes <mark.a.janes at intel.com>

Thanks.

Jose

>
>> Another solution is to would be to dump the pristine name (or even
>> better, a properly quoted regex suitable to be passed in
>> -include-tests  ) in the junit test result (stdout or stderr). e.g.:
>>
>>     "....
>>
>>     To run only this test pass --include-tests '^foo/doo/boo$'"
>>
>>
>> Jose
>>
>>
>>>
>>> -Mark
>>>
>>> Dylan Baker <baker.dylan.c at gmail.com> writes:
>>>
>>>> This looks fine to me, though I haven't tested it.
>>>>
>>>> Mark, does this seem good to you?
>>>>
>>>> Reviewed-by: Dylan Baker <baker.dylan.c at gmail.com>
>>>>
>>>> On Fri, Feb 20, 2015 at 11:53:50AM +0000, Jose Fonseca wrote:
>>>>> For example, Jenkins was displaying its REST API page when one navigated
>>>>> to  .../testReport/piglit.spec/ARB_occlusion_query2/api/ result.
>>>>> ---
>>>>>    framework/backends/junit.py | 10 +++++++++-
>>>>>    1 file changed, 9 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/framework/backends/junit.py b/framework/backends/junit.py
>>>>> index 839c99a..ddaf826 100644
>>>>> --- a/framework/backends/junit.py
>>>>> +++ b/framework/backends/junit.py
>>>>> @@ -166,8 +166,16 @@ class JUnitBackend(FileBackend):
>>>>>            # set different root names.
>>>>>            classname = 'piglit.' + classname
>>>>>
>>>>> +        testname += self._test_suffix
>>>>> +
>>>>> +        # Jenkins will display special pages when the test has certain names.
>>>>> +        # https://urldefense.proofpoint.com/v2/url?u=https-3A__jenkins-2Dci.org_issue_18062&d=AwIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=6iqiAc1uzBPZFCypdivM6WaoDheu37Q45KJTMIFulgo&s=3OKSPXH947_Gwt5zZ4v0DIJRLWf5-u6P4a2yc3cY1AU&e=
>>>>> +        # https://urldefense.proofpoint.com/v2/url?u=https-3A__jenkins-2Dci.org_issue_19810&d=AwIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=6iqiAc1uzBPZFCypdivM6WaoDheu37Q45KJTMIFulgo&s=Kr5YcnwFlFnWhiOHklE4CT5QTR7A5PiMJFoNuK4dhaM&e=
>>>>> +        if testname in ('api', 'search'):
>>>>> +            testname += '_'
>>>>> +
>>>>>            # Create the root element
>>>>> -        element = etree.Element('testcase', name=testname + self._test_suffix,
>>>>> +        element = etree.Element('testcase', name=testname,
>>>>>                                    classname=classname,
>>>>>                                    time=str(data['time']),
>>>>>                                    status=str(data['result']))
>>>>> --
>>>>> 2.1.0
>>>>>
>>>>> _______________________________________________
>>>>> Piglit mailing list
>>>>> Piglit at lists.freedesktop.org
>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.freedesktop.org_mailman_listinfo_piglit&d=AwIBAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=zfmBZnnVGHeYde45pMKNnVyzeaZbdIqVLprmZCM2zzE&m=6iqiAc1uzBPZFCypdivM6WaoDheu37Q45KJTMIFulgo&s=KdZJAJSf6EP4bKM8xp2zYCOpqQ8WY14LPCxx90LhrdY&e=