[Mesa-dev] [RFC] NIR serialization

[Nicolai Hähnle]

On 13.09.2017 11:54, Eero Tamminen wrote:
> Hi,
> 
> On 12.09.2017 09:55, Jordan Justen wrote:
>> On 2017-09-11 21:44:32, Timothy Arceri wrote:
>>> On 12/09/17 14:23, Ian Romanick wrote:
>>>> On 09/08/2017 01:59 AM, Kenneth Graunke wrote:
>>>>>
>>>>> We shouldn't use SPIR-V for the shader cache.
>>>>>
>>>>> The compilation process for GLSL is: GLSL -> GLSL IR -> NIR -> i965 
>>>>> IRs.
>>>>> Storing the content at one of those points, and later loading it and
>>>>> resuming the normal compilation process from that point...that's 
>>>>> totally
>>>>> reasonable.
>>>>>
>>>>> Having a fallback for "some things in the cache but not all the 
>>>>> variants
>>>>> we needed" suddenly take a different compilation pipeline, i.e. SPIR-V
>>>>> -> NIR -> ... seems risky.  It's a different compilation path that we
>>>>> don't normally use.  And one you'd only hit in limited circumstances.
>>>>> There's a lot of potential for really obscure bugs.
>>>>
>>>> Since we're going to expose exactly that path for GL_ARB_spirv / OpenGL
>>>> 4.6, we'd better make sure it works always.  Right?
>>>>
>>>> One nice thing about SPIR-V is that all of the handling of uniform
>>>> layouts, initial uniform values, attribute locations, etc. is already
>>>> serialized.  If I'm not mistaken, that was one of the big pain points
>>>> for all of the existing on-disk storage methods.  All of that has been
>>>> sorted out for SPIR-V, and we have to make it work anyway.
>>>
>>> Correct these are the main issues for the fallback path, however this is
>>> only used by i965 (exactly because an intermediate cache is missing).
>>> Using SPIR-V as the intermediate cache means we still need to convert to
>>> NIR and run all the opts, so I don't really see the advantage of caching
>>> to SPIR-V over NIR.
>>
>> For shader cache, hopefully we'll normally have the final program in
>> the cache, which means the 're-run opt passes' is probably not a big
>> factor. But, it still seems a fair point.
>>
>> I think the biggest advantage of having either nir or spir-v would be
>> not having to fallback to running the glsl compiler, right?
> 
> Shader cache is performance optimization aimed at reducing compile times.
> 
> When I earlier profiled it, 2/3 of the shader compilation work is done 
> at linking stage.

Caching NIR would completely eliminate the linking stage for shader 
variant compiles, since the NIR is already linked.

Cheers,
Nicolai


> Caching done at higher level than the linked binary shaders, may be of 
> questionable performance value, because the caching itself has also some 
> cost (extra code, disk access cache lookups etc).
> 
> Some profiling showing that doing caching at NIR/SPIRV level could 
> measurable help performance (compared to caching overhead), would be 
> appreciated.
> 
> However, if the main purpose of caching NIR/SPIRV is something else than 
> performance[1], then that's a different matter.  It just should be clear 
> what is the aim of this change.
> 
> 
>> Maybe we'll see which becomes available first? :)
> 
> 
>      - Eero
> 
> [1] Such as:
> * better code validation, or
> * ability to replace / tweak the cached files to manually test impact of 
> compiler optimizations before implementing them
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev


-- 
Lerne, wie die Welt wirklich ist,
Aber vergiss niemals, wie sie sein sollte.