[Mesa-dev] [PATCH 2/3] st: Sweep NIR after linking phase to free held memory
Danylo Piliaiev
danylo.piliaiev at gmail.com
Thu Jul 19 10:13:48 UTC 2018
On 19.07.18 12:30, Timothy Arceri wrote:
>> On 19.07.18 11:47, Timothy Arceri wrote:
>>> On 19/07/18 08:31, Eric Anholt wrote:
>>>> Danylo Piliaiev <danylo.piliaiev at gmail.com> writes:
>>>>
>>>>> After optimization passes and many trasfromations most of memory
>>>>
>>>> "transformations"
>>>>
>>>>> NIR holds is a garbage which was being freed only after shader
>>>>> deletion.
>>>>
>>>> "is garbage"
>>>>
>>>>> Freeing it at the end of linking will save memory which would be
>>>>> useful
>>>>> in case there are a lot of complex shaders being compiled.
>>>>> The common case for this issue is 32bit game running under Wine.
>>>>>
>>>>> The cost of the optimization is around ~3-5% of compilation speed
>>>>> with complex shaders.
>>>>>
>>>>> Signed-off-by: Danylo Piliaiev <danylo.piliaiev at globallogic.com>
>>>>
>>>> This seems good, and I'm running it through the CTS now.
>>>>
>>>
>>> The problem is this does the sweep too early. We still do lots of
>>> work on NIR after this, I've thought about this a few times and it
>>> really seems we should be able to call a driver specific function
>>> from the st and pass it the IR so it can do what ever it wants
>>> (lowering opts etc) and spits it back out. Once this is done we
>>> should then call sweep and cache the IR.
>>>
>>> At the very least we should call sweep before we cache NIR rather
>>> than where this patch places it.
>> I debugged mesa once more and cannot see where else memory is
>> allocated for NIR after this sweep. Could you point it to me? After
>> this sweep the memory NIR holds never grows. Later this NIR is cloned
>> from and these cloned NIRs are being sweeped in other place.
>
> Ah yes you are right. We do clone it when creating variants I'd
> forgotten about that. In that case please ignore my comment.
Good, I thought I really missed something...
Also during my investigation of Mesa memory usage I wrote gdb pretty
printer which shows how much memory variable holds in its ralloc context
(with all its children), it was crudely written and at this moment has
sever limitation: x64 only, depends on internal malloc implementation
and other hardcoded things, also I wasn't able to nicely display
children of a variable. The reason that pretty printer was done this way
is that calling c function (e.g. malloc_usable_size) corrupts backtrace
somehow.
Example usage:
(gdb) source ralloc_info_pretty_printer.py
(gdb) backtrace
#0 brw_link_shader (ctx=0x5555558ca0a0 <size: 96432>,
shProg=0x555555d850c0 <size: 528>) at brw_link.cpp:320
#1 0x00007ffff2732b6f in _mesa_glsl_link_shader (ctx=0x5555558ca0a0
<size: 96432>, prog=0x555555d850c0 <size: 528>) at
program/ir_to_mesa.cpp:3174
#2 0x00007ffff25a1862 in link_program (no_error=false,
shProg=0x555555d850c0 <size: 528>, ctx=0x5555558ca0a0 <size: 96432>) at
main/shaderapi.c:1206
#3 link_program_error (ctx=0x5555558ca0a0 <size: 96432>,
shProg=0x555555d850c0 <size: 528>) at main/shaderapi.c:1286
#4 0x00007ffff25a2f00 in _mesa_LinkProgram (programObj=3) at
main/shaderapi.c:1778
#5 0x0000555555556de1 in main () at
/home/danylo/Projects/shader_compilation_memory_test/test.cpp:421
(gdb) p prog->nir
$1 = 0x555555dc2b20 <size: 241296>
If there is any interest in having it in Mesa I can clean it up. You can
find its code in attachment.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ralloc_info_pretty_printer.py
Type: text/x-python
Size: 2662 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20180719/022e1f24/attachment-0001.py>
More information about the mesa-dev
mailing list