1.6Gb of workdir/.../Dep - thoughts ...

[Norbert Thiebaud]

On Fri, Jun 1, 2012 at 5:10 AM, Michael Stahl <mstahl at redhat.com> wrote:
> On 01/06/12 11:16, Michael Meeks wrote:
>>
>>       And some thoughts on improving this; currently we add a ton of deps for
>> packages that are really internal and typically change en-masse or not
>> at all.
>>
>>       eg. 'boost' - it is installed, and then ~never changes again - people
>> don't edit a single boost header and expect a dependency clean
>> re-compile.
>>
>>       Ergo:
>>
>> find -type f | xargs grep 'solver.*boost/' | wc -l
>>
>>       4.5 million hits (of 21.5million - worth having).
>
> wow, that's a lot!
>
>>       IMHO we could - without significant loss of functionality reduce all
>> those deps to a single stamp file (which we prolly install anyway) in
>> the solver. Should be ~trivial to elide in our dep-re-writing anyway,
>> save > 300Mb of space etc.
>>
>>       Should just be a few lines in solenv/bin/concat-deps.c
>
> yes. just need to find the proper regex that doesn't have false positives :)

getting OUTDIR from the env and substituting $OUTDIR/inc/<module>/.*
with $WORKDIR/Package/<module>_inc should work for gbuild module
for dmake... maybe patching deliver.pl to touch
$WORKDIR/Package/<module>_inc when we deliver ?

Of course that add the formal obligation that the 'package' that
deliver the includes of a module be always named <module>_inc  which
is a soft requirement today I think

from a concat-dep.c pow, that means teaching it to manage per-target
list of subsitution (the goal is to have only one <module>_inc dep
instead of many *.h for that module...)
that mean among other things a hash creation/destroy per target with
dep i.e need to put hash object creation/destruction under memory pool
management, and being a bit creative to detect the above mention
pattern as we parse the input (want to avoid parsing it twice)


Norbert