[Libreoffice-qa] Minutes of ESC call: 2015-07-30

Markus Mohrhard markus.mohrhard at googlemail.com
Sun Aug 2 14:20:22 PDT 2015


On Sun, Aug 2, 2015 at 9:01 PM, Norbert Thiebaud <nthiebaud at gmail.com>

> On Sun, Aug 2, 2015 at 11:25 AM, Markus Mohrhard
> <markus.mohrhard at googlemail.com> wrote:
> >> AI:     + will have a look at the CppUnit to implement 'expected
> failure'
> >> (Jan-Marek)
> >>             + Cpp logs are e.g. in
> >> workdir/CppunitTest/sal_rtl_math.test.log
> >
> >
> >
> > That is already possible with cppunit. Instead of using CPPUNIT_TEST use
> > CPPUNIT_TEST_FAIL which tells cppunit that the test is expected to fail
> with
> > a cppunit exception being thrown (it is extensively used in the cppunit
> > internal unit tests). The test will start to fail when none of the
> asserts
> > fail anymore. Keep in mind that it might be dangerous to use this with
> more
> > than one assertion as an unexpected one might fail.
> >
> That is not exactly what tha aim is...
> the aim is to have a failure be repport as such and not stop
> everything.. when a test is tagged as _can fail_ or something
> this is when a test is added before the fix

So you want to have this as non-fatal tests. Ok, that is currently missing
from cppunit but at least with the CPPUNIT_TEST_FAIL you can add tests
before the bugfix and just change it from CPPUNIT_TEST_FAIL to CPPUNIT_TEST
when the fix has been implemented. So it is a compromise as it is already
there and we can have a look how difficult it would be to properly
implement non-fatal tests in cppunit.

> also the minutes claim that cppunit log exist.. it is true they do,
> but they are not exploitable.
> I would like to be able to repport at the end of the build a nice recap
> <module> <section> <nb_of_test> <#sucess> <#skipped> <#failed>
> <for each failed
> <space><testname> : FAILED (<optionally one line reason>)
> the minute point to
> workdir/CppunitTest/sal_rtl_math.test.log
> which contains:
> OK (3)
> but  writerperfect_writer.test.log fro instance
> contain 12027 lines.. most of it random trace messages, and I could
> not find any way to parse (at all, even less reliably) that thing to
> extract any useful information.
> maybe it is just a matter of using
> http://cppunit.sourceforge.net/doc/1.8.0/class_cpp_unit_1_1_xml_outputter.html
> and then adding a (optional) post processing step in the build to
> generate a nice summary....

It is even easier. You can add another TestListener in
sal/cppunittester/cppunittester.cxx that logs all failures, and executed
tests. Of course as there is no support for non-fatal tests right now you
can't log them. It would at least give you an information how many tests
are executed. I think there is no post-processing necessary as you get all
the information the the xml outputter has already in the listener and
process is directly there.


> Norbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/libreoffice-qa/attachments/20150802/a3f57e8f/attachment.html>

More information about the Libreoffice-qa mailing list