[systemd-devel] [PATCH] Add test coverage and generate report with lcov

[Lennart Poettering]

On Tue, 05.02.13 23:56, Thomas H.P. Andersen (phomes at gmail.com) wrote:

> Adds test coverage and generates html reports. Configure with
> --enable-coverage and run make lcov.
> 
> make lcov-upload exists but it currently just rsyncs the report to
> ~/coverage. Perhaps we can set up a host for it and automate it later?

We could probably upload that to fdo...

> I have temporarily uploaded an example to look at here:
> http://people.gnome.org/~thomashpa/coverage/

Neat!

> Assuming that this is only useful for the core developers/buildbots I
> have not cluttered the code with workarounds for old versions for
> libtool, checks for non-gcc compilers, etc. I can add it you find it
> worth the extra lines in configure (or split it out to a m4 like
> gstreamer).
> 
> Note that this requires lcov 1.10. (rawhide is still at 1.9). The
> older versions have bugs that we hit with systemd. 1.10 also has the
> --no-external option that avoids including things from /usr in the
> report.

Hmm, given that this isn't even in rawhide, it might make sense to at
least check for lcov version in autoconf, as only test?

Otherwise looks pretty cool, Please go ahead anc commit.

> 
> I took inspiration from the gstreamer code.
> 
> >From 1693c939d7b5cb425a5af5ecc13187cd3980021a Mon Sep 17 00:00:00 2001
> From: Thomas Hindoe Paaboel Andersen <phomes at gmail.com>
> Date: Tue, 5 Feb 2013 22:49:26 +0100
> Subject: [PATCH] Add test coverage and generate report
> 
> Add support for creating a test coverage report using lcov.
> Enable the coverage with --enable-coverage.
> "make lcov" will create the report locally.
> "make lcov-upload" is intended to push the report to a server.
> It currently just rsyncs to ~/coverage.
> 
> Requires lcov version 1.10 to handle naming in systemd and to
> use the --no-external option.
> ---
>  Makefile.am  | 42 ++++++++++++++++++++++++++++++++++++++++++
>  configure.ac | 13 +++++++++++++
>  2 files changed, 55 insertions(+)
> 
> diff --git a/Makefile.am b/Makefile.am
> index 88662c0..f1c3de7 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -171,6 +171,11 @@ AM_CPPFLAGS = \
>  AM_CFLAGS = $(OUR_CFLAGS)
>  AM_LDFLAGS = $(OUR_LDFLAGS)
> 
> +if ENABLE_COVERAGE
> +       AM_CFLAGS += \
> +               $(COVERAGE_CFLAGS)
> +endif
> +
>  # ------------------------------------------------------------------------------
>  define move-to-rootlibdir
>      if test "$(libdir)" != "$(rootlibdir)"; then \
> @@ -1416,6 +1421,43 @@ test_sched_prio_LDADD = \
>      libsystemd-daemon.la
> 
>  # ------------------------------------------------------------------------------
> +## .PHONY so it always rebuilds it
> +.PHONY: lcov-reset lcov lcov-run lcov-report lcov-upload
> +
> +# run lcov from scratch, always
> +lcov-reset:
> +    $(MAKE) lcov-run
> +    $(MAKE) lcov-report
> +
> +# run lcov from scratch if the dir is not there
> +lcov:
> +    $(MAKE) lcov-reset
> +
> +if ENABLE_COVERAGE
> +# reset run coverage tests
> +lcov-run:
> +    @-rm -rf lcov
> +    lcov --directory . --zerocounters
> +    -$(MAKE) check
> +
> +# generate report based on current coverage data
> +lcov-report:
> +    mkdir lcov
> +    lcov --compat-libtool --base-directory . --directory .
> --no-external --capture --output-file lcov/lcov.info
> +    genhtml -t "systemd test coverage" -o lcov lcov/lcov.info
> +
> +lcov-upload: lcov
> +    rsync -rvz --delete lcov/* ~/coverage
> +
> +else
> +lcov-run:
> +    echo "Need to reconfigure with --enable-coverage"
> +
> +lcov-report:
> +    echo "Need to reconfigure with --enable-coverage"
> +endif
> +
> +# ------------------------------------------------------------------------------
>  systemd_initctl_SOURCES = \
>      src/initctl/initctl.c
> 
> diff --git a/configure.ac b/configure.ac
> index d94af7b..d237830 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -202,6 +202,18 @@ m4_pattern_forbid([^_?PKG_[A-Z_]+$],[*** pkg.m4
> missing, please install pkg-conf
>  PKG_CHECK_MODULES(DBUS, [dbus-1 >= 1.3.2])
> 
>  # ------------------------------------------------------------------------------
> +have_coverage=no
> +AC_ARG_ENABLE(coverage, AS_HELP_STRING([--enable-coverage], [enable
> test coverage]))
> +if test "x$enable_coverage" = "xyes"; then
> +       have_coverage=yes
> +       CC_CHECK_FLAGS_APPEND([with_coverage_cflags], [CFLAGS], [\
> +               -fprofile-arcs \
> +               -ftest-coverage])
> +       AC_SUBST([COVERAGE_CFLAGS], $with_coverage_cflags)
> +fi
> +AM_CONDITIONAL(ENABLE_COVERAGE, [test "$have_coverage" = "yes"])
> +
> +# ------------------------------------------------------------------------------
>  have_kmod=no
>  AC_ARG_ENABLE(kmod, AS_HELP_STRING([--disable-kmod], [disable
> loadable modules support]))
>  if test "x$enable_kmod" != "xno"; then
> @@ -871,6 +883,7 @@ AC_MSG_RESULT([
>          Python Headers:          ${have_python_devel}
>          man pages:               ${have_manpages}
>          gtk-doc:                 ${enable_gtk_doc}
> +        test coverage:           ${have_coverage}
>          Split /usr:              ${enable_split_usr}
>          SysV compatibility:      ${SYSTEM_SYSV_COMPAT}
> 


Lennart

-- 
Lennart Poettering - Red Hat, Inc.