[Piglit] [PATCH 17/19] framework: Add a JUnit backend

[Dylan Baker]

On Friday, August 29, 2014 12:36:58 PM Jose Fonseca wrote:
> 
> I don't think this attribute is actually necessary in practice.  Jenkins 
> will count tests when it processes the JUnit XML and stores it internally.
> 

Jenkins seems content to not have this value provided, but the XSD
does require it be valid. Because Jenkins doesn't seem to be concerned
with it I'm happy to just leave it as 0 for now, and fix it when it's
convenient. It doesn't seem that difficult to comply with and it might
save us (or someone else using JUnit to integrate with something other
than Jenkins) some headaches in the future.

> 
> >
> >       # Begin json.
> >       backend = framework.results.get_backend(args.backend)(
> > diff --git a/framework/results.py b/framework/results.py
> > index 741adc9..eabeb4d 100644
> > --- a/framework/results.py
> > +++ b/framework/results.py
> > @@ -26,11 +26,16 @@ import os
> >   import sys
> >   import abc
> >   import threading
> > +import posixpath
> >   from cStringIO import StringIO
> >   try:
> >       import simplejson as json
> >   except ImportError:
> >       import json
> > +try:
> > +    from lxml import etree
> > +except ImportError:
> > +    import xml.etree.cElementTree as etree
> >
> >   import framework.status as status
> >
> > @@ -43,7 +48,7 @@ __all__ = [
> >   ]
> >
> >   # A list of available backends
> > -BACKENDS = ['json']
> > +BACKENDS = ['json', 'junit']
> >
> >   # The current version of the JSON results
> >   CURRENT_JSON_VERSION = 1
> > @@ -349,6 +354,63 @@ class JSONBackend(Backend):
> >               self._write_dict_item(name, data)
> >
> >
> > +class JUnitBackend(Backend):
> > +    """ Backend that produces ANT JUnit XML
> > +
> > +    Based on the following schema:
> > +    https://urldefense.proofpoint.com/v1/url?u=https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-7.xsd&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=NMr9uy2iTjWVixC0wOcYCWEIYhfo80qKwRgdodpoDzA%3D%0A&m=214XzeIU6H5%2BhNCoizqfg3xjfdXgSseG1Fn0SVHv9oY%3D%0A&s=3ac0b84d76100043ccbe5216c44bbb5b14e24e3115f989ee5303fb6c246d78ff
> > +
> > +    """
> > +    # TODO: add fsync support
> > +
> > +    def __init__(self, dest, metadata, **options):
> > +        self._file = open(os.path.join(dest, 'results.xml'), 'w')
> > +
> > +        # Write initial headers and other data that etree cannot write for us
> > +        self._file.write('<?xml version="1.0" encoding="UTF-8" ?>\n')
> > +        self._file.write('<testsuites>\n')
> > +        self._file.write(
> > +            '<testsuite name="piglit" tests="{}">\n'.format(
> > +                metadata['test_count']))
> > +
> > +    def finalize(self, metadata=None):
> > +        self._file.write('</testsuite>\n')
> > +        self._file.write('</testsuites>\n')
> > +        self._file.close()
> 
> I'm not sure if there isn't one already, but you might want to add a 
> "try: ... finally: " construct at the top level to ensure that XML is 
> always valid, even if something bad happens halfway through.

We don't, but you're right we should have something for such a failure.
I think it belongs in another patch, I've added it to my list of things
to do.

> 
> > +
> > +    def write_test(self, name, data):
> > +        # Split the name of the test and the group (what junit refers to as
> > +        # classname), and replace piglits '/' seperated groups with '.', after
> > +        # replacing any '.' with '_' (so we don't get false groups)
> > +        classname, testname = posixpath.split(name)
> > +        assert classname
> > +        assert testname
> > +        classname = classname.replace('.', '_').replace('/', '.')
> 
> We might need to replace('\\', '.') too, for Windows' sake.

Done

> 
> > +        element = etree.Element('testcase', name=testname, classname=classname,
> > +                                time=str(data['time']),
> > +                                status=str(data['result']))
> > +
> > +        # Add stdout
> > +        out = etree.SubElement(element, 'system-out')
> > +        out.text = data['out']
> > +
> > +        # Add stderr
> > +        err = etree.SubElement(element, 'system-err')
> > +        err.text = data['err']
> > +
> > +        # Add relavent result value, if the result is pass then it doesn't need
> 
> relevant

Done

> 
> > +        # one of these statuses
> > +        if data['result'] == 'skip':
> > +            etree.SubElement(element, 'skipped')
> > +        elif data['result'] in ['warn', 'fail', 'dmesg-warn', 'dmesg-fail']:
> > +            etree.SubElement(element, 'failure')
> [...]
> 
> Otherwise looks good.
> 
> Jose

Thanks Jose
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140829/17495df6/attachment.sig>