[Piglit] [PATCH 0/4] Add support for subtests in JUnit as nested testsuite elements

[Jose Fonseca]

Hi Dylan,

Thanks for the heads up.

To be honest, lately I haven't been paying much attention to Jenkins + 
piglit results (my focus has been elsewhere), and I am pretty much the 
only consumer of Piglit+Jenkins inside VMware nowadays (for llvmpipe 
regression testing.)  Other teams have other processes / test systems in 
place.

So I'm happy to let you guys to do what you think best.

Therefore the series is

Acked-by: Jose Fonseca <jfonseca at vmware.com>


And if Jenkin's xUnit parser gives better results than Junit parser, I'm 
happy to make the switch too.  Is it just a matter of choosing the Xunit 
parser and pick Junit?


Jose

On 05/08/16 01:02, Dylan Baker wrote:
> This series refactors the JUnit backend so that the _write attribute is
> actually a callable class, instead of a method. This gives greater
> flexibility to handle subtests in the JUnit, or continue ignoring them.
>
> The reason to ignore them is that some xUnit implementations don't
> handle them properly (the JUnit plugin for Jenkins does not, for
> example, while the xUnit plugin does). Since piglit has a more than one
> consumer of the JUnit backend, it makes sense to be able to select the
> new backend or not.
>
> To enable this features add the --junit-subtests switch to the command
> line (obviously when using the junit backned), and watch magic happen.
> Each test with subtests will be recorded as a testsuite element, and the
> stderr and stdout will be attached to the testsuite rather than the
> testcase. However, when using the xUnit plugin for Jenkins it is
> rendered correctly.
>
> Jose, Could you test the subtest mode and see if it's a viable path for
> you? If possible I'd like to lower the complexity of the code by just
> removing the non-subtest path, but if that's going to be too much work
> for you we'll leave it as-is.
>
> I've tested this on our Jenkins instance and it renders correctly.
>
> CC: jfonseca at vmware.com
> CC: mark.a.janes at intel.com
>
> Dylan Baker (4):
>   framework/backends/junit.py: Split _write into a separate class.
>   framework/backends/junit.py: Split _set_xml_err method
>   framework/backends/junit.py: Add a writer class that handles subtests
>   framework: add command line switch to enable junit subtests
>
>  framework/backends/junit.py                | 348 ++++++++++++++--------
>  framework/programs/run.py                  |   7 +-
>  unittests/framework/backends/test_junit.py | 124 +++++++-
>  3 files changed, 341 insertions(+), 138 deletions(-)
>