Some thoughts about our tests and the build time

[Bjoern Michaelsen]

Heya Markus,

On Tue, May 17, 2016 at 02:37:02AM +0200, Markus Mohrhard wrote:
> The results for the initial build without building or executing the tests:
> 
> real    70m17.990s
> user    436m43.860s
> sys    28m3.680s
> 
> After that the results of a time make, therefore forcing the build of
> everything test related and executing the tests:
> 
> real    11m30.192s
> user    58m43.384s
> sys    1m36.876s

Did you by chance have an opportunity to also ran "make" from scratch on that
machine? I assume/hope it wont take the 70 min + 11 min = 82 min. a straight
addition would make one assume, because make will use idle job slots during a
full build.

IOW, these number suggest a 82min/70min = 17% overhead in real and a (436 +
58)min/436min = 13% overhead in CPU time -- but I assume the real overhead in a
build from scratch is smaller than both of that in the real world.

The most critical time I see from in all this is not anything build from
scratch anyway, but the for a simple
touch-one-cxx-recompile-relink-and-then-run-all-the-tests scenarios. So the:

> real    6m37.479s
> user    45m4.740s

as an _absolute_ is the key there, I guess.

Best,

Bjoern

P.S.: I would have assumed compiling/linking the tests to take much more time
than running the tests. But it seems with 45min/58min=77% -- most of the time
is indeed spend on running tests, not building them.

P.S.: For reference, the output of "time make build-nocheck" would be helpful
      too (aka a noop incremental build time overhead in make/dep parsing etc.)