Some thoughts about our tests and the build time
Markus Mohrhard
markus.mohrhard at googlemail.com
Tue May 17 14:25:35 UTC 2016
Hey Björn,
On Tue, May 17, 2016 at 11:21 AM, Bjoern Michaelsen <
bjoern.michaelsen at canonical.com> wrote:
> Heya Markus,
>
> On Tue, May 17, 2016 at 02:37:02AM +0200, Markus Mohrhard wrote:
> > The results for the initial build without building or executing the
> tests:
> >
> > real 70m17.990s
> > user 436m43.860s
> > sys 28m3.680s
> >
> > After that the results of a time make, therefore forcing the build of
> > everything test related and executing the tests:
> >
> > real 11m30.192s
> > user 58m43.384s
> > sys 1m36.876s
>
> Did you by chance have an opportunity to also ran "make" from scratch on
> that
> machine? I assume/hope it wont take the 70 min + 11 min = 82 min. a
> straight
> addition would make one assume, because make will use idle job slots
> during a
> full build.
>
real 83m37.618s
user 493m30.052s
sys 29m44.404s
And then the following time make (therefore again only executing tests):
real 9m16.741s
user 45m31.208s
sys 0m37.516s
>
> IOW, these number suggest a 82min/70min = 17% overhead in real and a (436 +
> 58)min/436min = 13% overhead in CPU time -- but I assume the real overhead
> in a
> build from scratch is smaller than both of that in the real world.
> The most critical time I see from in all this is not anything build from
> scratch anyway, but the for a simple
> touch-one-cxx-recompile-relink-and-then-run-all-the-tests scenarios. So
> the:
>
> > real 6m37.479s
> > user 45m4.740s
>
> as an _absolute_ is the key there, I guess.
>
So for me the problem is that on machines that are much slower it takes
quite some time to execute the tests and that we are going to see people
just skipping the tests if the cost-value relationship becomes too bad. As
mentioned I think we are right now still in an area where it is somewhat ok
but we know that we are adding more and more tests and that it will mean
that the ratio test time/build time will become bigger.
>
> Best,
>
> Bjoern
>
> P.S.: I would have assumed compiling/linking the tests to take much more
> time
> than running the tests. But it seems with 45min/58min=77% -- most of the
> time
> is indeed spend on running tests, not building them.
>
> P.S.: For reference, the output of "time make build-nocheck" would be
> helpful
> too (aka a noop incremental build time overhead in make/dep parsing
> etc.)
>
real 0m38.787s
user 0m16.300s
sys 0m0.896s
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice/attachments/20160517/4eef5c15/attachment.html>
More information about the LibreOffice
mailing list