[Mesa-dev] [PATCH 0/4] RadeonSI: Multithreaded shader compilation

[Grazvydas Ignotas]

On Sat, Jul 9, 2016 at 6:49 PM, Marek Olšák <maraeo at gmail.com> wrote:
> On Fri, Jul 8, 2016 at 3:20 AM, Timothy Arceri
> <timothy.arceri at collabora.com> wrote:
>> On Wed, 2016-06-29 at 18:32 +0200, Marek Olšák wrote:
>>> Hi,
>>>
>>> This series implements basic multithreaded LLVM shader compilation
>>> in a minimally invasive way. (+51 lines of code in the main patch)
>>>
>>> It doesn't help on-demand shader compilation, but it does improve
>>> loading and startup times by being able to saturate up to 4 CPU cores
>>> if given enough shaders to compile. A proper shader cache might make
>>> this redundant, but we don't have that now.
>>
>> Have you had a chance to take a look at my recent shader cache work
>> [1]? The glsl work is mostly done I'm now just cleaning up and fixing
>> up the fallback paths for when we have a cache miss. I'm not sure if
>> you guys will need the fallback path or not since you have a way
>> dealing with varients, you might be ok which would make things much
>> simpler.
>>
>> Anyway I don't think that getting something up and running would be a
>> large amount of work. Most of your time would likely be spent tweaking
>> the glsl to tgsi path to haveit to skip over the IR conversion and just
>> grab the required state.
>>
>> The two files you would find most interesting would be:
>>
>> src/compiler/glsl/shader_cache.cpp
>> src/mesa/drivers/dri/i965/brw_shader_cache.c
>>
>> Everything else (besided the cache code itself) is pretty much just
>> wiring things up to be called at the right time, or skipped.
>>
>>
>> [1] https://github.com/tarceri/Mesa_arrays_of_arrays.git shader-cache20
>
> No, I haven't looked at it.
>
> An on-disk shader cache would be nice to have, but it's not a pressing
> thing since we don't really get many apps compiling shaders before
> drawing. UE4 was doing that with GL 4.1 but not GL4.3, so that's cool.
> The only app where a shader cache would help a lot is Borderlands 2.
> Sadly, the same game would benefit from OpenGL driver multithreading
> even more, because once all shaders are compiled, the game is
> CPU-bound most of the time with frame rates as low as 20 on Core i5
> 3570.
>
> All in all, I need more user feedback to be sure if the on-disk shader
> cache would make any difference outside of Borderlands 2.

Having played with Timothy's i965 cache I can tell you there are
certainly more games affected. One of them is The Talos Principle
where difference is quite large (Serious Sam 3 probably too as the
engine is the same). Another one is Left 4 Dead 2 where it solves "do
something the first time" stalls. There are some bugs about older
Unreal engine games:
https://bugs.freedesktop.org/show_bug.cgi?id=96790
https://bugs.freedesktop.org/show_bug.cgi?id=92806

I do note that from subjective user's point of view, i965's shader
compilation seems to be a lot slower than r600g's. I don't know how
that all compares to radeonsi though as I have no way to test that.

Gražvydas