[Piglit] Allow junit to be used for summary generation

[Dylan Baker]

On Fri, Oct 09, 2015 at 09:44:01AM -0700, Mark Janes wrote:
> Jose Fonseca <jfonseca at vmware.com> writes:
> 
> > On 09/10/15 01:21, baker.dylan.c at gmail.com wrote:
> >> This series updates the junit backend to allow it to properly load junit
> >> and convert it back into piglit's internal representation, thus allowing
> >> it to be summarized using the piglit summary tools
> >>
> >> There is still some work that needs to be done beyond this, most of the
> >> platform metadata isn't stored yet and restored, but I have a plan for
> >> that. I have some other refactoring work that I think will make that
> >> easier, and I'd like to get there before landing that.
> >>
> >> This is enough to be able to compare junit and json results using the
> >> console and html summaries.
> >>
> >> There is a caveat here, and that's patch 3. To compare json and junit we
> >> need to be able to restore the names of the junit tests to *exactly*
> >> what they were before, and currently we don't have a way to reverse the
> >> '.' -> '_' conversion. My proposal is to change '.' into '___', which is
> >> very unlikely in a real test name (though we could change it to almost
> >> anything that would be unique). This may break some existing setup
> >> (Mark, I think this will probably break some of our expected fail/crash
> >> data).
> >>
> >
> > I don't object this, but instead of this brittle testname (de)munge, 
> > have you considered/tried using an additional XML attribute with 
> > unmodified piglit test name?  I expect that Jenkins junit parser will 
> > just outright ignore it.
> 
> My recollection is that we found the junit parser to be strict:
> 
> https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd
> 
> >
> > Jose

I don't know if we tried, It definately allows some not strictly valid
junit to be passed to it (IIRC we didn't originally pass the total
number of tests in, and that is required per the SPEC).

If it will accept arbitrary tags that would be very nice. We might want
to guard them behind a --non-strict-junit switch (or inversely provide a
--strict-junit flag) in case that changes or someone whats to us a
consumer of junit that is strict.

I think I'll give it a try and see what happens.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20151009/8a9de334/attachment.sig>