[Libreoffice] Bug 40673 - unify build parallelization and use gnu makes --load-average feature where available

[Bjoern Michaelsen]

On Wed, 7 Sep 2011 04:02:36 -0500
Norbert Thiebaud <nthiebaud at gmail.com>
wrote:

> really ? ok so on my Mac I build with max-job=8 num-cpu=6... what
> value should I use to get the same result under the new scheme?

Counter question: Why would you want to get that result? max-job=8
num-cpu=6 gets you something unpredictable between 6 and 48 jobs. The
only way why it should work is because modern unix handles
overcommitment quite decently (RAM might be more of an issue, but not
with your machine). I think you would be just as fast with the new
defaults, if not faster.

> (today that means I get up to 8 jobs with up to 6 task per, except for
> tail_build where I got 8 -- note that tail_build tend to run alone)

I doubt you will ever run the 8 dmake dirs unless you compile binfilter.

> Now, yes,  if it works (I never tested it), the load average sound
> more promising than a pure -j n... but then if it works, then why even
> bother with -j n at all...
It works, but:
a) on distributed builders (icecream, distcc) the load of the host is
   not a good measure
b) having no limit on -j is risky on start as make will spawn jobs like
   mad, because most jobs are io-bound in the beginning
c) Windows has no sensible load measure

> if --with-sens-load is present then use the value specified or default
> to nb_cpu + 1 like you said, but then use -j without value...
see b)

Also -l will lessen the pain of overcommitment: have some old dmake dirs
running and gnu make will 'fill up' the available remaining free load
to optimal cpu-usage.

Best,

Bjoern

PS.:
Well, I took a look at the count of objects in our build -- ~60% are in
gbuild already, and of the remaining ones openssl, berkeleydb, icu,
autodoc (which has to die), connectivity and sal are big ones. openssl
and berkeleydb and icu are external ones which we should build with gnu
make paralellization too. And autodoc has to die (Did I say that
already?).


-- 
https://launchpad.net/~bjoern-michaelsen