'make check' fails when building with gcov code coverage
sbergman at redhat.com
Tue May 31 12:06:53 UTC 2022
On 31/05/2022 13:43, Maarten Hoes wrote:
> Anyway, if more tests fail sometimes without there being a 'real'
> failure, then I am not sure how to deal with that in relation to
> generating an lcov report for a full build. My initial idea was to not
> generate a report if 'make check' failed, but hearing this now makes me
> wonder if that would be a good approach. Perhaps it would be preferred
> just to run 'make -k check' and generate a report always, regardless of
> test failures ? Of course you could also choose to skip such tests, but
> that would lead to not representative results; and someone would have to
> manually keep the skip list up to date, which people will forget to do.
Occasionally failing tests are a well-known problem for LO (e.g.,
witness the "Jenkins / CI update: tests that failed more than twice in
last seven days" section in the weekly ESC minutes---many of those are
apparently spuriously failing tests), and there is no reason to assume
that your gcov builds would not also occasionally be affected by that.
So the general advice would be to ignore occasional failed builds (which
might not only fail due to spurious test failures, but also because e.g.
a build breaker got submitted by accident). If some specific tests
cause enough builds to fail to make that approach impractical, those
tests should get fixed. Or, as a last resort, get disabled for
known-failing build scenarios.
I don't think -k would be a good solution, as it would make it harder to
meaningfully interpret the generated data.
More information about the LibreOffice