linking performance (was: Re: OK to get rid of scaddins?)

[Mark Wielaard]

Hi,

Added Tom to the CC, see thread here:
http://thread.gmane.org/gmane.comp.documentfoundation.libreoffice.devel/24055/focus=24096
(Although it seems posts from non-subscribers, aka me, are not allowed,
 so you won't see my earlier reply there. I'll send it to you separately.)

On Tue, Feb 14, 2012 at 01:07:20PM +0100, Michael Stahl wrote:
> On 14/02/12 11:52, Michael Meeks wrote:
> > On Mon, 2012-02-13 at 16:06 +0100, Michael Stahl wrote:
> >>>> (of course i don't care if you do it for a special "merged libs" mode,
> >>>> but C++ development is already a sufficiently unproductive activity that
> >>>> we shouldn't make it even more so...)
> >>>
> >>> 	Is it necessary to build with full debug enabled ? how slow is it
> >>> really ? [ if it takes ten minutes - how slow is it to re-build with
> >>> just the bits you want symbols for & re-run whatever you're
> >>> debugging ?].
> >>
> >> i find it works quite well with 8GB of RAM, except that linking takes
> >> much longer (and you better not have 3 unit tests crash concurrently
> >> otherwise gdb will lock up the box for 15 minutes until OOM killer is
> >> invoked...).
> > 
> > 	So - can you give some concrete ideas of time & space it is taking to
> > link our shared libraries ? and also the growth in size that we get -
> > what is the stripped vs. non-stripped output ? Mark prolly has some
> > great insights as to how to improve that.
> 
> so the objects from gbuild modules with debug are about 6.5G:
> 
> > > du -sh workdir/unxlngx6/*Object
> > 1,1M	workdir/unxlngx6/CObject
> > 6,3G	workdir/unxlngx6/CxxObject
> > 960K	workdir/unxlngx6/GenCObject
> > 164M	workdir/unxlngx6/GenCxxObject
> 
> the libraries and executables linked from these, 2.9G:
> 
> > > du -sh workdir/unxlngx6/LinkTarget/
> > 2,9G	workdir/unxlngx6/LinkTarget/
> 
> i don't have a non-dbgutil tree to compare to.
> 
> >>> 	I wonder if the new 'gold' linker will help performance wise - have you
> >>> tried it ?
> >>
> >> no, but the problem is really the space that the object files take up:
> >> they don't fit all into RAM cache, and ld is blocked on I/O most of the
> >> time (in tail_build).
> > 
> > 	If we have > 8Gb of debug symbols per module there is a -real- problem
> > here;
> 
> the problem is more likely that in tail_build we first compile all the
> object files, and only after they have all been built they are linked
> into libraries/executables.
> that probably results in sub-optimal use of disk cache  :)
> 
> perhaps i'll look into fixing that in gbuild when i have some time, but
> LinkTarget is annoyingly complex so that would need some testing...
> 
> > Lubos was talking of using some more magic / smaller debug option
> > in the past: -gdwarf-4
> > 
> > 	Are you using that ? I believe we turned it off by default again for
> > some reason or other: potentially we want to add a check for a tolerably
> > recent toolchain and debugger on the system before defaulting to that
> > [ it supposedly saved 30% of the size ], but you need gdb 7.3 really.
> 
> no, AFAIR Lubos enabled that on master and there were some problems with
> it (gdb was horribly slow or something), so he disabled it again a week
> later.

That really needs some investigation (which is why I added Tom to the CC)
recent GDB versions improved startup performance a lot, so if enabling
DWARF4 would make GDB startup slower again (which surprises me) that
would be really bad. Especially since DWARF4 is now the default at least
on Fedora.

Could you give some more specifics about what was tried, what the original
startup times for gdb were and how much change you saw?

> > 	Either way, it sucks to hinder ourselves from creating a more efficient
> > library structure because of un-necessary performance problems in the
> > toolchain ;-)
> 
> well perhaps there are really 2 different target audiences, product
> builds need fast start-up, while development builds need fast re-build
> cycles...

Thanks,

Mark