[Mesa-dev] [RFC] ralloc: use jemalloc for faster GLSL compilation

[Marek Olšák]

On Thu, Sep 29, 2016 at 8:19 PM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
> On 29 September 2016 at 18:48, Marek Olšák <maraeo at gmail.com> wrote:
>> On Thu, Sep 29, 2016 at 4:56 PM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>>> On 29 September 2016 at 11:48, Marek Olšák <maraeo at gmail.com> wrote:
>>>> On Thu, Sep 29, 2016 at 11:20 AM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
>>>>> On 28.09.2016 18:49, Marek Olšák wrote:
>>>>>>
>>>>>> From: Marek Olšák <marek.olsak at amd.com>
>>>>>>
>>>>>> More info about jemalloc:
>>>>>>    https://github.com/jemalloc/jemalloc/wiki/History
>>>>>>
>>>>>> Average from 3 takes compiling Alien Isolation shaders from GLSL to GCN
>>>>>> bytecode:
>>>>>>    glibc:    17.183s
>>>>>>    jemalloc: 15.558s
>>>>>>    diff:     -9.5%
>>>>>>
>>>>>> The diff is -10.5% for a full shader-db run.
>>>>>> ---
>>>>>>
>>>>>> TODO: The jemalloc dependency should be added to configure.ac before this.
>>>>>>
>>>>>> We can probably redirect all malloc/calloc/realloc/free calls in Mesa to
>>>>>> jemalloc. We can either use _mesa_jemalloc, etc. everywhere or we can
>>>>>> redirect calls to jemalloc using #define malloc _mesa_jemalloc, etc.
>>>>>>
>>>>>> Right now, I just use: export LDFLAGS=-ljemalloc
>>>>>
>>>>>
>>>>> Sounds good to me. It should probably be a configurable option, defaulting
>>>>> to jemalloc and failing if not available unless explicitly disabled.
>>>>
>>>> If it was a configurable option, almost nobody would use it. Let's
>>>> make it mandatory.
>>>>
>>> This combined with ...
>>>
>>>>>
>>>>> On the Gallium side of things, switching to jemalloc could be pretty
>>>>> straightforward via the macros in u_memory.h, once we know that they're
>>>>> actually used consistently (which we currently don't -- it would be nice to
>>>>> know how jemalloc and glibc malloc react when the calls are mixed).
>>>>
>>>> Redefining malloc/calloc/realloc/free/posix_memalign for all Mesa code
>>>> would be more robust.
>>>>
>>> ... this doesn't is not a wise move.
>>>
>>> Don't force jemalloc onto everyone without having an explicit ACK from
>>> a wide audience, please ? Considering the static/shared link (or w/o
>>> jemalloc all together) distributions will have their
>>> preferences/policies which won't align with my/your view.
>>
>> I guess we can have an option to disable jemalloc, but only if most
>> users won't use that option. The real problem is that the GLSL
>> compiler is alloc-bound and anybody wanting to use the GLSL compiler
>> should stay away from glibc's allocator.
>>
>> The GLSL compiler can be slowed down significantly by keeping 5x
>> LLVMContext in memory between compilations in radeonsi. The fact that
>> radeonsi can indirectly slow down the GLSL compiler (but not LLVM) is
>> a strong indication that we have a problem.
>>
> If the issue is present in only one driver, one might be looking at
> the wrong end. Alternatively others will also be in favour of this and
> things will flow naturally.

The improvement is even better with the Gallium noop driver. When I
tested noop, the compile time was reduced by 15%. I think radeonsi is
the driver that will benefit the least from it, not the most.

Marek