<div dir="ltr">According to the schema it's allowed. The schema is in the piglit repo at "framework/tests/schema/junit-7.xsd"</div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Dec 5, 2015 at 4:36 AM, Jose Fonseca <span dir="ltr"><<a href="mailto:jfonseca@vmware.com" target="_blank">jfonseca@vmware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 05/12/15 00:21, <a href="mailto:baker.dylan.c@gmail.com" target="_blank">baker.dylan.c@gmail.com</a> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
From: Dylan Baker <<a href="mailto:baker.dylan.c@gmail.com" target="_blank">baker.dylan.c@gmail.com</a>><br>
<br>
Currently the JUnit backend has no way to represent subtests in such a<br>
way that they can be understood by jenkins and by the summary tools.<br>
<br>
Mark, Nanley and myself consulted and came up with several approaches,<br>
each with serious drawbacks:<br>
1. Print the subtest statuses into stdout or stderr nodes in the JUnit.<br>
This has the advantage of being simple, but has a problem with being<br>
shadowed by an expected-<status>. If subtest A fails, an expected<br>
fail can be entered. If subtest B also starts failing, no one will<br>
notice. This wont work<br>
2. Treat each subtest as a full test, and the test as a group. I have<br>
two reservations about this approach. It's different than the JSON<br>
for one, and there isn't a good way to turn the JUnit back into the<br>
piglit internal representation using this approach, which would make<br>
running JUnit results through the piglit status tools difficult. This<br>
would also massively inflate the size of the JSON results, and that's<br>
already becoming a problem for us.<br>
3. Create a main test entry, and then subtest entries as well, which<br>
pointed back to the original test as the parent in their stdout. This<br>
also has shadowing problems, and would still make the XML very large.<br>
<br>
The final approach taken was suggested by Nanely, to turn tests with<br>
subtests into a testsuite element, which could represent the shared<br>
values (stdout, stderr), and could hold individual testcases elements<br>
for each subtest. This solves the shadowing issue, and introduces less<br>
file size increase than the other ideas floated.<br>
</blockquote>
<br></div></div>
Having the sub-tests being a testsuite is OK, but I'm not sure nesting a testsuite inside a testsuite is reliable.<br>
<br>
I don't know if JUnit generated by other frameworks ever have that. And what exactly is the semantics for names.<br>
<br>
I wonder if it wouldn't be better to ensure that the `testsuite` tag is always a child from the root `testsuites` tag.<span class="HOEnZb"><font color="#888888"><br>
<br>
Jose</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Also adds test for the new feature.<br>
<br>
cc: <a href="mailto:mark.a.janes@intel.com" target="_blank">mark.a.janes@intel.com</a><br>
cc: <a href="mailto:jfonseca@vmware.com" target="_blank">jfonseca@vmware.com</a><br>
Signed-off-by: Dylan Baker <<a href="mailto:dylanx.c.baker@intel.com" target="_blank">dylanx.c.baker@intel.com</a>><br>
---<br>
framework/backends/junit.py | 174 +++++++++++++++++++++------<br>
framework/tests/junit_backends_tests.py | 206 +++++++++++++++++++++++++++++++-<br>
2 files changed, 343 insertions(+), 37 deletions(-)<br>
<br>
diff --git a/framework/backends/junit.py b/framework/backends/junit.py<br>
index 34df300..329fc4b 100644<br>
--- a/framework/backends/junit.py<br>
+++ b/framework/backends/junit.py<br>
@@ -117,8 +117,6 @@ class JUnitBackend(FileBackend):<br>
<br>
shutil.rmtree(os.path.join(self._dest, 'tests'))<br>
<br>
-<br>
-<br>
def _write(self, f, name, data):<br>
# Split the name of the test and the group (what junit refers to as<br>
# classname), and replace piglits '/' separated groups with '.', after<br>
@@ -135,11 +133,41 @@ class JUnitBackend(FileBackend):<br>
# set different root names.<br>
classname = 'piglit.' + classname<br>
<br>
- element = self.__make_case(testname, classname, data)<br>
+ if data.subtests:<br>
+ # If there are subtests treat the test as a suite instead of a<br>
+ # test, set system-out, system-err, and time on the suite rather<br>
+ # than on the testcase<br>
+ name='{}.{}'.format(classname, testname)<br>
+ element = etree.Element(<br>
+ 'testsuite',<br>
+ name=name,<br>
+ time=str(data.time.total))<br>
+<br>
+ out = etree.SubElement(element, 'system-out')<br>
+ out.text = data.command + '\n' + data.out<br>
+ err = etree.SubElement(element, 'system-err')<br>
+ err.text = data.err<br>
+ err.text += '\n\nstart time: {}\nend time: {}\n'.format(<br>
+ data.time.start, data.time.end)<br>
+<br>
+ for name, result in data.subtests.iteritems():<br>
+ sub = self.__make_subcase(name, result, err)<br>
+ out = etree.SubElement(sub, 'system-out')<br>
+ out.text = 'I am a subtest of {}'.format(name)<br>
+ element.append(sub)<br>
+<br>
+ for attrib, xpath in [('failures', './/testcase/failure'),<br>
+ ('errors', './/testcase/error'),<br>
+ ('skipped', './/testcase/skipped'),<br>
+ ('tests', './/testcase')]:<br>
+ element.attrib[attrib] = str(len(element.findall(xpath)))<br>
+<br>
+ else:<br>
+ element = self.__make_case(testname, classname, data)<br>
+<br>
f.write(etree.tostring(element))<br>
<br>
- def __make_case(self, testname, classname, data):<br>
- """Create a test case element and return it."""<br>
+ def __make_name(self, testname):<br>
# Jenkins will display special pages when the test has certain names,<br>
# so add '_' so the tests don't match those names<br>
# <a href="https://jenkins-ci.org/issue/18062" rel="noreferrer" target="_blank">https://jenkins-ci.org/issue/18062</a><br>
@@ -148,17 +176,68 @@ class JUnitBackend(FileBackend):<br>
if full_test_name in _JUNIT_SPECIAL_NAMES:<br>
testname += '_'<br>
full_test_name = testname + self._test_suffix<br>
+ return full_test_name<br>
+<br>
+ def __make_subcase(self, testname, result, err):<br>
+ """Create a <testcase> element for subtests.<br>
+<br>
+ This method is used to create a <testcase> element to nest inside of a<br>
+ <testsuite> element when that element represents a test with subtests.<br>
+ This differs from __make_case in that it doesn't add as much metadata<br>
+ to the <testcase>, since that was attached to the <testsuite> by<br>
+ _write, and that it doesn't handle incomplete cases, since subtests<br>
+ cannot have incomplete as a status (though that could change).<br>
+<br>
+ """<br>
+ full_test_name = self.__make_name(testname)<br>
+ element = etree.Element('testcase',<br>
+ name=full_test_name,<br>
+ status=str(result))<br>
+<br>
+ # replace special characters and make case insensitive<br>
+ lname = self.__normalize_name(testname)<br>
+<br>
+ expected_result = "pass"<br>
+<br>
+ if lname in self._expected_failures:<br>
+ expected_result = "failure"<br>
+ # a test can either fail or crash, but not both<br>
+ assert lname not in self._expected_crashes<br>
+<br>
+ if lname in self._expected_crashes:<br>
+ expected_result = "error"<br>
+<br>
+ self.__add_result(element, result, err, expected_result)<br>
+<br>
+ return element<br>
+<br>
+ def __make_case(self, testname, classname, data):<br>
+ """Create a <testcase> element and return it.<br>
+<br>
+ Specifically, this is used to create "normal" test case, one that<br>
+ doesn't contain any subtests. __make_subcase is used to create a<br>
+ <testcase> which belongs inside a nested <testsuite> node.<br>
+<br>
+ Arguments:<br>
+ testname -- the name of the test<br>
+ classname -- the name of the group (to use piglit terminology)<br>
+ data -- A TestResult instance<br>
+<br>
+ """<br>
+ full_test_name = self.__make_name(testname)<br>
<br>
# Create the root element<br>
- element = etree.Element('testcase', name=full_test_name,<br>
+ element = etree.Element('testcase',<br>
+ name=full_test_name,<br>
classname=classname,<br>
- # Incomplete will not have a time.<br>
time=str(data.time.total),<br>
status=str(data.result))<br>
<br>
# If this is an incomplete status then none of these values will be<br>
# available, nor<br>
if data.result != 'incomplete':<br>
+ expected_result = "pass"<br>
+<br>
# Add stdout<br>
out = etree.SubElement(element, 'system-out')<br>
out.text = data.out<br>
@@ -171,7 +250,8 @@ class JUnitBackend(FileBackend):<br>
err.text = data.err<br>
err.text += '\n\nstart time: {}\nend time: {}\n'.format(<br>
data.time.start, data.time.end)<br>
- expected_result = "pass"<br>
+<br>
+ element.extend([err, out])<br>
<br>
# replace special characters and make case insensitive<br>
lname = self.__normalize_name(classname, testname)<br>
@@ -184,29 +264,34 @@ class JUnitBackend(FileBackend):<br>
if lname in self._expected_crashes:<br>
expected_result = "error"<br>
<br>
- self.__add_result(element, data, err, expected_result)<br>
+ self.__add_result(element, data.result, err, expected_result)<br>
else:<br>
etree.SubElement(element, 'failure', message='Incomplete run.')<br>
<br>
return element<br>
<br>
@staticmethod<br>
- def __normalize_name(classname, testname):<br>
- name = (classname + "." + testname).lower()<br>
+ def __normalize_name(testname, classname=None):<br>
+ """Nomralize the test name to what is stored in the expected statuses.<br>
+ """<br>
+ if classname is not None:<br>
+ name = (classname + "." + testname).lower()<br>
+ else:<br>
+ name = testname.lower()<br>
name = name.replace("=", ".")<br>
name = name.replace(":", ".")<br>
return name<br>
<br>
@staticmethod<br>
- def __add_result(element, data, err, expected_result):<br>
- """Add a <skipped>, <failure>, or <error> if necissary."""<br>
+ def __add_result(element, result, err, expected_result):<br>
+ """Add a <skipped>, <failure>, or <error> if necessary."""<br>
res = None<br>
# Add relevant result value, if the result is pass then it doesn't<br>
# need one of these statuses<br>
- if data.result == 'skip':<br>
+ if result == 'skip':<br>
res = etree.SubElement(element, 'skipped')<br>
<br>
- elif data.result in ['fail', 'dmesg-warn', 'dmesg-fail']:<br>
+ elif result in ['fail', 'dmesg-warn', 'dmesg-fail']:<br>
if expected_result == "failure":<br>
err.text += "\n\nWARN: passing test as an expected failure"<br>
res = etree.SubElement(element, 'skipped',<br>
@@ -214,7 +299,7 @@ class JUnitBackend(FileBackend):<br>
else:<br>
res = etree.SubElement(element, 'failure')<br>
<br>
- elif data.result == 'crash':<br>
+ elif result == 'crash':<br>
if expected_result == "error":<br>
err.text += "\n\nWARN: passing test as an expected crash"<br>
res = etree.SubElement(element, 'skipped',<br>
@@ -229,7 +314,7 @@ class JUnitBackend(FileBackend):<br>
<br>
# Add the piglit type to the failure result<br>
if res is not None:<br>
- res.attrib['type'] = str(data.result)<br>
+ res.attrib['type'] = str(result)<br>
<br>
<br>
def _load(results_file):<br>
@@ -242,6 +327,27 @@ def _load(results_file):<br>
JUnit document.<br>
<br>
"""<br>
+ def populate_result(result, test):<br>
+ # This is the fallback path, we'll try to overwrite this with the value<br>
+ # in stderr<br>
+ result.time = results.TimeAttribute(end=float(test.attrib['time']))<br>
+ result.err = test.find('system-err').text<br>
+<br>
+ # The command is prepended to system-out, so we need to separate those<br>
+ # into two separate elements<br>
+ out = test.find('system-out').text.split('\n')<br>
+ result.command = out[0]<br>
+ result.out = '\n'.join(out[1:])<br>
+<br>
+ # Try to get the values in stderr for time<br>
+ if 'time start' in result.err:<br>
+ for line in result.err.split('\n'):<br>
+ if line.startswith('time start:'):<br>
+ result.time.start = float(line[len('time start: '):])<br>
+ elif line.startswith('time end:'):<br>
+ result.time.end = float(line[len('time end: '):])<br>
+ break<br>
+<br>
run_result = results.TestrunResult()<br>
<br>
splitpath = os.path.splitext(results_file)[0].split(os.path.sep)<br>
@@ -267,25 +373,25 @@ def _load(results_file):<br>
<br>
result.result = test.attrib['status']<br>
<br>
- # This is the fallback path, we'll try to overwrite this with the value<br>
- # in stderr<br>
- result.time = results.TimeAttribute(end=float(test.attrib['time']))<br>
- result.err = test.find('system-err').text<br>
+ populate_result(result, test)<br>
<br>
- # The command is prepended to system-out, so we need to separate those<br>
- # into two separate elements<br>
- out = test.find('system-out').text.split('\n')<br>
- result.command = out[0]<br>
- result.out = '\n'.join(out[1:])<br>
+ run_result.tests[name] = result<br>
<br>
- # Try to get the values in stderr for time<br>
- if 'time start' in result.err:<br>
- for line in result.err.split('\n'):<br>
- if line.startswith('time start:'):<br>
- result.time.start = float(line[len('time start: '):])<br>
- elif line.startswith('time end:'):<br>
- result.time.end = float(line[len('time end: '):])<br>
- break<br>
+ for test in tree.iterfind('testsuite'):<br>
+ result = results.TestResult()<br>
+ # Take the class name minus the 'piglit.' element, replace junit's '.'<br>
+ # separator with piglit's separator, and join the group and test names<br>
+ name = test.attrib['name'].split('.', 1)[1]<br>
+ name = name.replace('.', grouptools.SEPARATOR)<br>
+<br>
+ # Remove the trailing _ if they were added (such as to api and search)<br>
+ if name.endswith('_'):<br>
+ name = name[:-1]<br>
+<br>
+ populate_result(result, test)<br>
+<br>
+ for subtest in test.iterfind('testcase'):<br>
+ result.subtests[subtest.attrib['name']] = subtest.attrib['status']<br>
<br>
run_result.tests[name] = result<br>
<br>
diff --git a/framework/tests/junit_backends_tests.py b/framework/tests/junit_backends_tests.py<br>
index 7d5a3fc..ae18e3e 100644<br>
--- a/framework/tests/junit_backends_tests.py<br>
+++ b/framework/tests/junit_backends_tests.py<br>
@@ -29,6 +29,7 @@ try:<br>
from lxml import etree<br>
except ImportError:<br>
import xml.etree.cElementTree as etree<br>
+import mock<br>
import nose.tools as nt<br>
from nose.plugins.skip import SkipTest<br>
<br>
@@ -44,7 +45,7 @@ doc_formatter = utils.DocFormatter({'separator': grouptools.SEPARATOR})<br>
_XML = """\<br>
<?xml version='1.0' encoding='utf-8'?><br>
<testsuites><br>
- <testsuite name="piglit" tests="1"><br>
+ <testsuite name="piglit" tests="5"><br>
<testcase classname="piglit.foo.bar" name="a-test" status="pass" time="1.12345"><br>
<system-out>this/is/a/command\nThis is stdout</system-out><br>
<system-err>this is stderr<br>
@@ -53,6 +54,24 @@ time start: 1.0<br>
time end: 4.5<br>
</system-err><br>
</testcase><br>
+ <testsuite name="piglit.bar" time="1.234" tests="4" failures="1" skipped="1" errors="1"><br>
+ <system-err>this is stderr<br>
+<br>
+time start: 1.0<br>
+time end: 4.5<br>
+</system-err><br>
+ <system-out>this/is/a/command\nThis is stdout</system-out><br>
+ <testcase name="subtest1" status="pass"/><br>
+ <testcase name="subtest2" status="fail"><br>
+ <failed/><br>
+ </testcase><br>
+ <testcase name="subtest3" status="crash"><br>
+ <error/><br>
+ </testcase><br>
+ <testcase name="subtest4" status="skip"><br>
+ <skipped/><br>
+ </testcase><br>
+ </testsuite><br>
</testsuite><br>
</testsuites><br>
"""<br>
@@ -203,11 +222,12 @@ class TestJUnitLoad(utils.StaticDirectory):<br>
def setup_class(cls):<br>
super(TestJUnitLoad, cls).setup_class()<br>
cls.xml_file = os.path.join(cls.tdir, 'results.xml')<br>
-<br>
+<br>
with open(cls.xml_file, 'w') as f:<br>
f.write(_XML)<br>
<br>
cls.testname = grouptools.join('foo', 'bar', 'a-test')<br>
+ cls.subtestname = 'bar'<br>
<br>
@classmethod<br>
def xml(cls):<br>
@@ -270,7 +290,6 @@ class TestJUnitLoad(utils.StaticDirectory):<br>
"""backends.junit._load: Totals are calculated."""<br>
nt.ok_(bool(self.xml()))<br>
<br>
-<br>
@utils.no_error<br>
def test_load_file(self):<br>
"""backends.junit.load: Loads a file directly"""<br>
@@ -281,6 +300,48 @@ class TestJUnitLoad(utils.StaticDirectory):<br>
"""backends.junit.load: Loads a directory"""<br>
backends.junit.REGISTRY.load(self.tdir, 'none')<br>
<br>
+ def test_subtest_added(self):<br>
+ """backends.junit._load: turns secondlevel <testsuite> into test with stubtests"""<br>
+ xml = self.xml()<br>
+ nt.assert_in(self.subtestname, xml.tests)<br>
+<br>
+ def test_subtest_time(self):<br>
+ """backends.junit._load: handles time from subtest"""<br>
+ time = self.xml().tests[self.subtestname].time<br>
+ nt.assert_is_instance(time, results.TimeAttribute)<br>
+ nt.eq_(time.start, 1.0)<br>
+ nt.eq_(time.end, 4.5)<br>
+<br>
+ def test_subtest_out(self):<br>
+ """backends.junit._load: subtest stderr is loaded correctly"""<br>
+ test = self.xml().tests[self.subtestname].out<br>
+ nt.eq_(test, 'This is stdout')<br>
+<br>
+ def test_subtest_err(self):<br>
+ """backends.junit._load: stderr is loaded correctly."""<br>
+ test = self.xml().tests[self.subtestname].err<br>
+ nt.eq_(test, 'this is stderr\n\ntime start: 1.0\ntime end: 4.5\n')<br>
+<br>
+ def test_subtest_statuses(self):<br>
+ """backends.juint._load: subtest statuses are restored correctly<br>
+<br>
+ This is not implemented as separate tests or a generator becuase while<br>
+ it asserts multiple values, it is testing one peice of funcitonality:<br>
+ whether the subtests are restored correctly.<br>
+<br>
+ """<br>
+ test = self.xml().tests[self.subtestname]<br>
+<br>
+ subtests = [<br>
+ ('subtest1', 'pass'),<br>
+ ('subtest2', 'fail'),<br>
+ ('subtest3', 'crash'),<br>
+ ('subtest4', 'skip'),<br>
+ ]<br>
+<br>
+ for name, value in subtests:<br>
+ nt.eq_(test.subtests[name], value)<br>
+<br>
<br>
def test_load_file_name():<br>
"""backends.junit._load: uses the filename for name if filename != 'results'<br>
@@ -319,3 +380,142 @@ def test_load_default_name():<br>
test = backends.junit.REGISTRY.load(filename, 'none')<br>
<br>
nt.assert_equal(<a href="http://test.name" rel="noreferrer" target="_blank">test.name</a>, 'junit result')<br>
+<br>
+<br>
+class TestJunitSubtestWriting(object):<br>
+ """Tests for Junit subtest writing.<br>
+<br>
+ Junit needs to write out subtests as full tests, so jenkins will consume<br>
+ them correctly.<br>
+<br>
+ """<br>
+ __patchers = [<br>
+ mock.patch('framework.backends.abstract.shutil.move', mock.Mock()),<br>
+ ]<br>
+<br>
+ @staticmethod<br>
+ def _make_result():<br>
+ result = results.TestResult()<br>
+ result.time.end = 1.2345<br>
+ result.result = 'pass'<br>
+ result.out = 'this is stdout'<br>
+ result.err = 'this is stderr'<br>
+ result.command = 'foo'<br>
+ result.subtests['foo'] = 'skip'<br>
+ result.subtests['bar'] = 'fail'<br>
+ result.subtests['oink'] = 'crash'<br>
+<br>
+ test = backends.junit.JUnitBackend('foo')<br>
+ mock_open = mock.mock_open()<br>
+ with mock.patch('framework.backends.abstract.open', mock_open):<br>
+ with test.write_test(grouptools.join('a', 'group', 'test1')) as t:<br>
+ t(result)<br>
+<br>
+ # Return an xml object<br>
+ # This seems pretty fragile, but I don't see a better way to get waht<br>
+ # we want<br>
+ return etree.fromstring(mock_open.mock_calls[-3][1][0])<br>
+<br>
+ @classmethod<br>
+ def setup_class(cls):<br>
+ for p in cls.__patchers:<br>
+ p.start()<br>
+<br>
+ cls.output = cls._make_result()<br>
+<br>
+ @classmethod<br>
+ def teardown_class(cls):<br>
+ for p in cls.__patchers:<br>
+ p.stop()<br>
+<br>
+ def test_suite(self):<br>
+ """backends.junit.JUnitBackend.write_test: wraps the cases in a suite"""<br>
+ nt.eq_(self.output.tag, 'testsuite')<br>
+<br>
+ def test_cases(self):<br>
+ """backends.junit.JUnitBackend.write_test: has one <testcase> per subtest"""<br>
+ nt.eq_(len(self.output.findall('testcase')), 3)<br>
+<br>
+ @utils.nose_generator<br>
+ def test_metadata(self):<br>
+ """backends.junit.JUnitBackend.write_test: metadata written into the<br>
+ suite<br>
+<br>
+ """<br>
+ def test(actual, expected):<br>
+ nt.eq_(expected, actual)<br>
+<br>
+ descrption = ('backends.junit.JUnitBackend.write_test: '<br>
+ '{} is written into the suite')<br>
+<br>
+ if self.output.tag != 'testsuite':<br>
+ raise Exception('Could not find a testsuite!')<br>
+<br>
+ tests = [<br>
+ (self.output.find('system-out').text, 'this is stdout',<br>
+ 'system-out'),<br>
+ (self.output.find('system-err').text,<br>
+ 'this is stderr\n\nstart time: 0.0\nend time: 1.2345\n',<br>
+ 'system-err'),<br>
+ (self.output.attrib.get('name'), 'piglit.a.group.test1', 'name'),<br>
+ (self.output.attrib.get('time'), '1.2345', 'timestamp'),<br>
+ (self.output.attrib.get('failures'), '1', 'failures'),<br>
+ (self.output.attrib.get('skipped'), '1', 'skipped'),<br>
+ (self.output.attrib.get('errors'), '1', 'errors'),<br>
+ (self.output.attrib.get('tests'), '3', 'tests'),<br>
+ ]<br>
+<br>
+ for actual, expected, name in tests:<br>
+ test.description = descrption.format(name)<br>
+ yield test, actual, expected<br>
+<br>
+ def test_testname(self):<br>
+ """backends.junit.JUnitBackend.write_test: the testname should be the subtest name"""<br>
+ nt.ok_(self.output.find('testcase[@name="foo"]') is not None)<br>
+<br>
+ def test_fail(self):<br>
+ """Backends.junit.JUnitBackend.write_test: add <failure> if the subtest failed"""<br>
+ nt.eq_(len(self.output.find('testcase[@name="bar"]').findall('failure')), 1)<br>
+<br>
+ def test_skip(self):<br>
+ """Backends.junit.JUnitBackend.write_test: add <skipped> if the subtest skipped"""<br>
+ nt.eq_(len(self.output.find('testcase[@name="foo"]').findall('skipped')), 1)<br>
+<br>
+ def test_error(self):<br>
+ """Backends.junit.JUnitBackend.write_test: add <error> if the subtest crashed"""<br>
+ nt.eq_(len(self.output.find('testcase[@name="oink"]').findall('error')), 1)<br>
+<br>
+<br>
+class TestJunitSubtestFinalize(utils.StaticDirectory):<br>
+ @classmethod<br>
+ def setup_class(cls):<br>
+ super(TestJunitSubtestFinalize, cls).setup_class()<br>
+<br>
+ result = results.TestResult()<br>
+ result.time.end = 1.2345<br>
+ result.result = 'pass'<br>
+ result.out = 'this is stdout'<br>
+ result.err = 'this is stderr'<br>
+ result.command = 'foo'<br>
+ result.subtests['foo'] = 'pass'<br>
+ result.subtests['bar'] = 'fail'<br>
+<br>
+ test = backends.junit.JUnitBackend(cls.tdir)<br>
+ test.initialize(BACKEND_INITIAL_META)<br>
+ with test.write_test(grouptools.join('a', 'test', 'group', 'test1')) as t:<br>
+ t(result)<br>
+ test.finalize()<br>
+<br>
+ @utils.not_raises(etree.ParseError)<br>
+ def test_valid_xml(self):<br>
+ """backends.jUnit.JunitBackend.finalize: produces valid xml with subtests"""<br>
+ etree.parse(os.path.join(self.tdir, 'results.xml'))<br>
+<br>
+ def test_valid_junit(self):<br>
+ """backends.jUnit.JunitBackend.finalize: prodives valid junit with subtests"""<br>
+ if etree.__name__ != 'lxml.etree':<br>
+ raise SkipTest('Test requires lxml features')<br>
+<br>
+ schema = etree.XMLSchema(file=JUNIT_SCHEMA)<br>
+ xml = etree.parse(os.path.join(self.tdir, 'results.xml'))<br>
+ nt.ok_(schema.validate(xml), msg='xml is not valid')<br>
<br>
</blockquote>
<br>
</div></div></blockquote></div><br></div>