[Libreoffice-qa] Minutes of ESC call: 2015-07-30

[Norbert Thiebaud]

On Sun, Aug 2, 2015 at 11:25 AM, Markus Mohrhard
<markus.mohrhard at googlemail.com> wrote:


>> AI:     + will have a look at the CppUnit to implement 'expected failure'
>> (Jan-Marek)
>>             + Cpp logs are e.g. in
>> workdir/CppunitTest/sal_rtl_math.test.log
>
>
>
> That is already possible with cppunit. Instead of using CPPUNIT_TEST use
> CPPUNIT_TEST_FAIL which tells cppunit that the test is expected to fail with
> a cppunit exception being thrown (it is extensively used in the cppunit
> internal unit tests). The test will start to fail when none of the asserts
> fail anymore. Keep in mind that it might be dangerous to use this with more
> than one assertion as an unexpected one might fail.
>

That is not exactly what tha aim is...
the aim is to have a failure be repport as such and not stop
everything.. when a test is tagged as _can fail_ or something
this is when a test is added before the fix

also the minutes claim that cppunit log exist.. it is true they do,
but they are not exploitable.
I would like to be able to repport at the end of the build a nice recap

<module> <section> <nb_of_test> <#sucess> <#skipped> <#failed>
<for each failed
<space><testname> : FAILED (<optionally one line reason>)

the minute point to
workdir/CppunitTest/sal_rtl_math.test.log
which contains:
OK (3)

but  writerperfect_writer.test.log fro instance
contain 12027 lines.. most of it random trace messages, and I could
not find any way to parse (at all, even less reliably) that thing to
extract any useful information.

maybe it is just a matter of using
http://cppunit.sourceforge.net/doc/1.8.0/class_cpp_unit_1_1_xml_outputter.html
and then adding a (optional) post processing step in the build to
generate a nice summary....

Norbert