Implement expected failures for CppUnit

Jan-Marek Glogowski glogow at fbihome.de
Wed Oct 7 02:37:57 PDT 2015


Hi.

I've implemented CPPUNIT_TEST_XFAIL to add test cases to a suite, which
are expected to fail. See

https://gerrit.libreoffice.org/#/c/19213/
https://gerrit.libreoffice.org/#/c/19214/
https://gerrit.libreoffice.org/#/c/19215/

The current approach unfortunately breaks the ABI and API. The API break
can be prevented, but I wanted to get some review of the approach.

One minor problem I'm struggling with is the accounting of the test in
src/cppunit/TestResultCollector.cpp and the resulting summary output of
src/cppunit/CompilerOutputter.cpp and src/cppunit/TextOutputter.cpp

So errors are a part of the failures, but for XFAIL a pass is actually a
failure. Currently XFAIL pass and failure are both handled as errors.

 <<  "Run: "  <<  m_result->runTests()  << "   "
 <<  "Failure total: "  <<  m_result->testFailuresTotal()  << "   "
 <<  "Failures: "  <<  m_result->testFailures()  << "   "
 <<  "Errors: "  <<  m_result->testErrors()  << "   "
 <<  "Expected: " << m_result->testFailuresExpected()  << "   "
 <<  "Unexpected passed: " << m_result->testUnexpectedPasses()

accounted as

  if ( failure.isError() )
    ++m_testErrors;
  if ( failure.isExpected() )
  {
    ++m_expectedFailures;
    if ( failure.isUnexpectedPass() )
      ++m_unexpectedPasses;
  }

Suggestions welcome.

Thanks for reviewing.

Jan-Marek


More information about the LibreOffice mailing list