[Mesa-dev] Improving ralloc performance for the GLSL compiler

[Emil Velikov]

On 31 August 2016 at 17:39, Marek Olšák <maraeo at gmail.com> wrote:
> On Wed, Aug 31, 2016 at 4:39 PM, Eero Tamminen
> <eero.t.tamminen at intel.com> wrote:
>> Hi,
>>
>> On 31.08.2016 16:37, Marek Olšák wrote:
>> ...
>>>
>>> OK. I'm afraid malloc/calloc/realloc/free from libjemalloc.so will
>>> conflict with libc and Steam.
>>
>>>
>>>
>>> When building jemalloc from source, there is an option to add a function
>>> prefix.
>>>
>>> There are basically two options at the moment:
>>> - my allocator: is faster, has much higher memory usage, but is simple
>>> to integrate.
>>> - jemalloc: should have reasonable memory usage, but it's not clear
>>> yet how to integrate it with Mesa.
>>
>>
>> If static library is used, symbols get resolved at build time, so that isn't
>> a problem.  If distro's static version isn't build with -fPIC, linking it to
>> a library wouldn't be nice though (code relocations would dirty text/code
>> memory mappings when library is loaded).
>
> My main concern (also to Emil) is that the _dri.so module is loaded
> too late and it shouldn't override malloc which may have already been
> used to allocate something and thus the existing allocations wouldn't
> be accepted by jemalloc's free(). Same for libGL.
>
> Example:
> ptr = malloc(..); // glibc
> dlopen("libGL") // contains jemalloc
> glXCreateContext
> free(ptr); // does it use glibc or jemalloc?
>
In case of Steam I'm leaning towards neither. From a quick look:
 - libcef.so (chrome embedded framework) part of Steam, _not_ the
runtime - pollutes global namespace with ~1.4K symbols amongst which
malloc and friends.
 - libtbbmalloc_proxy.so.2, part of the runtime, coming in both i386
and amd64 flavours - seems to be the preferred allocator.

Which one gets used is a good guess ;-)

As hinted earlier - I'm not against static linking things (for one
reason or another we have to do it) but I'd prefer if we don't import
another project into mesa.

The most important part imho - please confirm the metrics on about
your allocator/jemalloc in the scenarios mentioned by others.

Thanks
Emil