[Piglit] [PATCH 0/2] Handle subtests in JUnit output

[Mark Janes]

There are a few things with this patch that complicate my use of junit:

 1) the stdout for the parent test is in the stored in the testsuite tag
    in the junit xml.  However, I can't access this data from the
    jenkins ui.  If a subtest fails, I won't be able to write up a bug
    from the Jenkins UI.

 2) the subtests reference the parent test by descriptive name (eg "I am
    a subtest of imageAtomicMin"), but the user needs to find the test
    in the junit hierarchy
    (piglit.spec.arb_shader_image_load_store.atomicity)

 3) each subtest's testcase tag lacks the classname attribute, which
    lets me identify the test uniquely.  To use this format, I would
    have to check if each test is in a suite, then append the suite's
    name.

one option for making this usable is to keep the structure flat (no
nesting of suites).  The parent test stays the same as today, and the
subtests are simply added as subsequent tests at the same level.  The
stdout for a subtest refers to the parent test in stdout, so the user
can find it in the jenkins UI.

It's unfortunate to have to hack around the details of how Jenkins
displays tests.  Maybe there is a better option...

-Mark

baker.dylan.c at gmail.com writes:

> From: Dylan Baker <baker.dylan.c at gmail.com>
>
> This series changes the way the JUnit backend generates results for
> tests with subtests. Rather than generating them as a single test and
> ignoring the subtests it generates them as a testsuite, and then adds
> each subtest as a test. This allows subtests to be represented nicely
> and to avoid duplicating large amounts of data in the xml file.
>
> Dylan Baker (2):
>   framework/backends/junit.py: refactor _write into several helpers.
>   framework/backends/junit.py: Handle tests with subtests as testsuite
>     elements
>
>  framework/backends/junit.py             | 263 +++++++++++++++++++++++---------
>  framework/tests/junit_backends_tests.py | 206 ++++++++++++++++++++++++-
>  2 files changed, 393 insertions(+), 76 deletions(-)
>
> -- 
> 2.6.3
>
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit