[Mesa-dev] [PATCH 00/21] Reduce ir_variable memory usage

[Kenneth Graunke]

On Friday, May 30, 2014 11:07:03 AM Eric Anholt wrote:
> Ian Romanick <idr at freedesktop.org> writes:
> > This series reduces the memory usage of ir_variable quite significantly.
> > 
> > The first couple patches add a mechanism to determine the amount of
> > memory used by any kind of IR object.  This is used to collect the data
> > that is shown in the commit messages through the series.

I think that talloc can give you this type of information for basically free - 
I'd rather see an implementation of the ralloc API atop talloc, and then just 
turn the memory debugging on.  This is quite a bit of code, and incomplete...

> > Most of the rest of the patches rearrange data or store things in
> > smaller fields.  The two interesting "subseries" are:
> > 
> > Patches 9 through 15 and 20 through 21: Store short variable names in
> > otherwise "dead" space in the base class.  I didn't rebase these patches
> > to all be together because I didn't want to re-collect all the data. :)
> > A small amount more savings could be had here, but in the test case at
> > hand, it didn't appear worth the effort.  Adding
> 
> I'd rather this series didn't land.

I agree with Eric, here.  A couple of the patches are pretty obvious and 
reasonable, but a lot of them are...quite nasty.  Mashing strings between 
struct padding is just...eesh.  Sorry. :(

> Our bad memory waste for variables is mostly due to the fact that we
> hang our builtin variables off of the symbol table, so they never get
> freed even after dead code elimination.  These hacks (and there are some
> *nasty* hacks in here) are just slight improvements on that bad
> situation.
> 
> Instead, let's fix things to not keep the symbol table past AST time (we
> can global walk variables quickly by looking at the top level IR nodes
> when linking), and hang our variables off of the parser state like all
> the other IR at compile time.

Yeah, I was surprised to find that we allocate built-in variables using the 
symbol table as the memory context, and the symbol table lives for a long 
time.  It really only makes sense at AST->HIR time, where scopes exist.

After that, we just want a convenient way to find a global variable definition 
by name.  We can just make a function to do that, and free the whole symbol 
table, which would free all the dead variables.

> > Here's the punchline.  In a trimmed trace from dota2 on 32-bit,
> > ir_variable accounts for ~5.5MB before this series.  After this series,
> > it accounts for only ~4.5MB.
> > 
> > Before: IR MEM: variable usage / name / total: 4955496 915817 5871313
> > After:  IR MEM: variable usage / name / total: 4118280 644100 4762380
> 
> I *highly* recommend using valgrind's massif tool if you're looking at
> memory allocation.  Here's the peak allocation of my dota2 trace:

Massif is great.  There's a nice KDE-based visualizer for it as well.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20140530/b2204ac2/attachment.sig>