[Libreoffice] --with-num-cpus/--with-max-jobs and gbuild

[Bjoern Michaelsen]

Hi Caolán,

On Fri, 25 Mar 2011 12:49:50 +0000
Caolán McNamara <caolanm at redhat.com> wrote:

> So, the gbuild stuff uses -j$(MAXPROCESS) and we have configure
> options tuned for build/dmake where
> --with-num-cpus is the number of -PX given to "build" and sets
> BUILD_NCPUS
> --with-max-jobs is the number of -PX given to "dmake" and sets
> BUILD_MAX_JOBS
> nothing sets MAXPROCESS

dmake sets MAXPROCESS to the number of jobs it got as a parameter. So
given a "build.pl --all -PNUM_CPUS -- -PMAX_JOBS", MAXPROCESS will be
MAX_JOBS.

> so, we could e.g. change MAXPROCESS to be BUILD_MAX_JOBS and -jX to
> the same X as dmake -X was.

Thats how it is currently as build.pl start GNU make via dmake. And if
you start GNU make directly you have better choices anyway.

> Or we could invert our current allocation
> of cpus and set the ncpus as the value for dmake && gmake, which is
> sort of the standard practice, i.e. make --with-num-cpus control
> dmake/gmake and set max-jobs as the no of parallel builds.
> 
> Thoughts ?

Another possibility (on unix only) would be, given a
"build.pl -PNUM_CPUS -- -PMAX_JOBS" to start the GNU makes with:

 make -srjNUM_CPUS*MAX_JOBS -lNUM_CPUS

The NUM_CPUS*MAX_JOBS multiplication would allow GNU make to start as
many jobs as the old build system was allowed to (in NUM_CPUS dirs),
but the -l switch would prevent it to overcommit the CPU, thus:
 - if build.pl/dmake are running few jobs (because it is bottlenecked in
   one dir) GNU make would tune up.
 - if build.pl/dmake are running many jobs (because it is not
   bottlenecked) GNU make would hold back.

It would thus try to "fill in" where ever possible. This works only on
unix, as Windows has no sensible load measurement.

Best Regards,

Bjoern

-- 
https://launchpad.net/~bjoern-michaelsen