[Piglit] [PATCH v3 0/4] Add support for subtests in JUnit as nested testsuite elements

[Mark Janes]

Before --junit-subtests is made the default, I need to be able to use
subtest names in:

 * --include-tests and --exclude-tests filters
 * [expected-failures] and [expected-crashes] sections of config files.

For filtering subtests, it is acceptable for include/exclude to apply to
the parent test.  I don't need to run a single subtest.

Series is

Reviewed-by: Mark Janes <mark.a.janes at intel.com>
Tested-by: Mark Janes <mark.a.janes at intel.com>

Dylan Baker <dylan at pnwbakers.com> writes:

> This series refactors the JUnit backend so that the _write attribute is
> actually a callable class, instead of a method. This gives greater
> flexibility to handle subtests in the JUnit, or continue ignoring them.
>
> The reason to ignore them is that some xUnit implementations don't
> handle them properly (the JUnit plugin for Jenkins does not, for
> example, while the xUnit plugin does). Since piglit has a more than one
> consumer of the JUnit backend, it makes sense to be able to select the
> new backend or not.
>
> To enable this features add the --junit-subtests switch to the command
> line (obviously when using the junit backend), and watch magic happen.
> Each test with subtests will be recorded as a testsuite element, and the
> stderr and stdout will be attached to the testsuite rather than the
> testcase. However, when using the xUnit plugin for Jenkins it is
> rendered correctly.
>
> Changes in v2:
>  - Add error message explaining what happened if result != pass.
>  - Merge two patches together to maintain behavior for bisecting.
>  - Add skipped tag to tests without subtests (when using junit-subtests).
>  - Add classname to subtests, like is done for full tests.
>  - Add additional unittests.
>  - Call junit_escape on the testname as well as the classname.
>  - Reword the help message for the junit-subtests switch.
>  - Add a new patch which fixes a bug masking status changes if the test becomes
>    skip.
>
> Changes in v3:
>  - Fix classname in subtests to include the test name.
>  - Fix spelling errors in commit messages.
>
> Dylan Baker (4):
>   framework/backends/junit.py: Split _write into a separate class.
>   framework/backends/junit.py: Add a writer class that handles subtests
>   framework: add command line switch to enable junit subtests
>   framework/backends/junit: Don't let skip hide status changes
>
>  framework/backends/junit.py                | 352 ++++++++++++++--------
>  framework/programs/run.py                  |   8 +-
>  unittests/framework/backends/test_junit.py | 192 +++++++++++-
>  3 files changed, 414 insertions(+), 138 deletions(-)
>
> -- 
> git-series 0.8.7
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/piglit