[Mesa-dev] [RFC] ralloc: use jemalloc for faster GLSL compilation

[Emil Velikov]

On 29 September 2016 at 18:48, Marek Olšák <maraeo at gmail.com> wrote:
> On Thu, Sep 29, 2016 at 4:56 PM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>> On 29 September 2016 at 11:48, Marek Olšák <maraeo at gmail.com> wrote:
>>> On Thu, Sep 29, 2016 at 11:20 AM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
>>>> On 28.09.2016 18:49, Marek Olšák wrote:
>>>>>
>>>>> From: Marek Olšák <marek.olsak at amd.com>
>>>>>
>>>>> More info about jemalloc:
>>>>>    https://github.com/jemalloc/jemalloc/wiki/History
>>>>>
>>>>> Average from 3 takes compiling Alien Isolation shaders from GLSL to GCN
>>>>> bytecode:
>>>>>    glibc:    17.183s
>>>>>    jemalloc: 15.558s
>>>>>    diff:     -9.5%
>>>>>
>>>>> The diff is -10.5% for a full shader-db run.
>>>>> ---
>>>>>
>>>>> TODO: The jemalloc dependency should be added to configure.ac before this.
>>>>>
>>>>> We can probably redirect all malloc/calloc/realloc/free calls in Mesa to
>>>>> jemalloc. We can either use _mesa_jemalloc, etc. everywhere or we can
>>>>> redirect calls to jemalloc using #define malloc _mesa_jemalloc, etc.
>>>>>
>>>>> Right now, I just use: export LDFLAGS=-ljemalloc
>>>>
>>>>
>>>> Sounds good to me. It should probably be a configurable option, defaulting
>>>> to jemalloc and failing if not available unless explicitly disabled.
>>>
>>> If it was a configurable option, almost nobody would use it. Let's
>>> make it mandatory.
>>>
>> This combined with ...
>>
>>>>
>>>> On the Gallium side of things, switching to jemalloc could be pretty
>>>> straightforward via the macros in u_memory.h, once we know that they're
>>>> actually used consistently (which we currently don't -- it would be nice to
>>>> know how jemalloc and glibc malloc react when the calls are mixed).
>>>
>>> Redefining malloc/calloc/realloc/free/posix_memalign for all Mesa code
>>> would be more robust.
>>>
>> ... this doesn't is not a wise move.
>>
>> Don't force jemalloc onto everyone without having an explicit ACK from
>> a wide audience, please ? Considering the static/shared link (or w/o
>> jemalloc all together) distributions will have their
>> preferences/policies which won't align with my/your view.
>
> I guess we can have an option to disable jemalloc, but only if most
> users won't use that option. The real problem is that the GLSL
> compiler is alloc-bound and anybody wanting to use the GLSL compiler
> should stay away from glibc's allocator.
>
> The GLSL compiler can be slowed down significantly by keeping 5x
> LLVMContext in memory between compilations in radeonsi. The fact that
> radeonsi can indirectly slow down the GLSL compiler (but not LLVM) is
> a strong indication that we have a problem.
>
If the issue is present in only one driver, one might be looking at
the wrong end. Alternatively others will also be in favour of this and
things will flow naturally.

> Since I happen to be responsible for delivering a well-performing
> driver for our hw across all installations, I guess you understand why
> I'm inclined to making jemalloc mandatory for ralloc at least. Using
> jemalloc for whole Mesa is not important and that idea can be dropped.
>
> One comment on distributions. Some distributions complained years ago
> that I made LLVM mandatory for r300g even though it worked fine
> without it (but too slow on some GPUs). I made it clear that if they
> didn't like the LLVM dependency, they were free to stop shipping that
> driver. That taught me that distributions are not always interested in
> delivering Mesa in the configuration and level of quality that was
> intended by Mesa developers. If there is an option to make something
> slower or worse, distributions shouldn't have the option, ever.
>
I see your point even before you stated it, but one should not enforce
their preference onto everyone. That is not how open-source works
afaict.

Yes, I've seen distros that patch out the llvm r300 "dependency",
surely you don't want this happening here as well ?

Afaict you're getting the stick out for distribution (maintainers)
while one ought to put a carrot at the end of it.

99% of the cases you don't want reports of people running patched mesa
where you have little to no idea what's happening. By forcing such
decisions you effectively give the finger to maintainers and a "do
apply local patches" card to everyone :-(

Simply add the an identifier to the vendor string and if one objects
about performance and is having "sub par" performance point them to
the wiki.


Thanks
Emil