[Mesa-dev] [RFC] NIR serialization

[Timothy Arceri]

On 08/09/17 18:59, Kenneth Graunke wrote:
> On Thursday, September 7, 2017 4:26:04 PM PDT Jordan Justen wrote:
>> On 2017-09-06 14:12:41, Daniel Schürmann wrote:
>>> Hello together!
>>> Recently, we had a small discussion (off the list) about the NIR
>>> serialization, which was previously discussed in [RFC] ARB_gl_spirv and
>>> NIR backend for radeonsi.
>>>
>>> As this topic could be interesting to more people, I would like to
>>> share, what was talked about so far (You might want to read from bottom up).
>>>
>>> TL;DR:
>>> - NIR serialization is in demand for shader cache
>>> - could be done either directly (NIR binary form) or via SPIR-V
>>> - Ian et al. are working on GLSL IR -> SPIR-V transformation, which
>>> could be adapted for a NIR -> SPIR-V pass
>>> - in NIR representation, some type information is lost
>>> - thus, a serialization via SPIR-V could NOT be a glslang alternative
>>> (otoh, the GLSL IR->SPIR-V pass could), but only for spirv-opt (if the
>>> output is valid SPIR-V)
>>
>> Ian,
>>
>> Tim was suggesting that we might look at serializing nir for the i965
>> shader cache. Based on this email, it sounds like serialized nir would
>> not be enough for the shader cache as some GLSL type info would be
>> lost. It sounds like GLSL IR => SPIR-V would be good enough. Is that
>> right?
>>
>> I don't think we have a strict requirement for the GLSL IR => SPIR-V
>> path for GL 4.6, right? So, this is more of a 'nice-to-have'?
>>
>> I'm not sure we'd want to make i965 shader cache depend on a
>> nice-to-have feature. (Unless we're pretty sure it'll be available
>> soon.)
>>
>> But, it would be nice to not have to fallback to compiling the GLSL
>> for i965 shader cache, so it would be worth waiting a little bit to be
>> able to rely on a SPIR-V serialization of the GLSL IR.
>>
>> What do you suggest?
>>
>> -Jordan
> 
> We shouldn't use SPIR-V for the shader cache.
> 
> The compilation process for GLSL is: GLSL -> GLSL IR -> NIR -> i965 IRs.
> Storing the content at one of those points, and later loading it and
> resuming the normal compilation process from that point...that's totally
> reasonable.
> 
> Having a fallback for "some things in the cache but not all the variants
> we needed" suddenly take a different compilation pipeline, i.e. SPIR-V
> -> NIR -> ... seems risky.  It's a different compilation path that we
> don't normally use.  And one you'd only hit in limited circumstances.
> There's a lot of potential for really obscure bugs.
> 
> Serializing NIR, and possibly a few auxiliary structures that we need,
> seems reasonable.  Although, just using the GLSL seemed reasonable to
> me as well, but I guess that's proven to be painful?

It's just more awkward and fragile then it should be, and requires a lot 
of disconnected code scattered everywhere from the driver backend to the 
GLSL IR compile. There are also still some unresolved issues in the 
latest i965 series. The main issue is uniforms, atomic bindings etc can 
change between the original compile and the fallback.  There is a big 
potential for really obscure bugs, it seemed simple enough when I 
started out but I wouldn't have gone down that road if I had realised 
what I was going to hit. Having tgsi made the gallium cache a breeze, 
and has all the other advantages others have pointed to for using nir 
such as smaller fallback delays, etc. Also besides the serialization 
code it should require very few code changes to the i965 driver.

> 
> --Ken
>