[Mesa-dev] [RFC] NIR serialization

[Rob Clark]

On Tue, Sep 12, 2017 at 4:39 PM, Jason Ekstrand <jason at jlekstrand.net> wrote:
> On Tue, Sep 12, 2017 at 11:09 AM, Jason Ekstrand <jason at jlekstrand.net>
> wrote:
>>
>> On Tue, Sep 12, 2017 at 10:12 AM, Ian Romanick <idr at freedesktop.org>
>> wrote:
>>>
>>> On 09/11/2017 11:17 PM, Kenneth Graunke wrote:
>>> > On Monday, September 11, 2017 9:23:05 PM PDT Ian Romanick wrote:
>>> >> On 09/08/2017 01:59 AM, Kenneth Graunke wrote:
>>> >>> Having a fallback for "some things in the cache but not all the
>>> >>> variants
>>> >>> we needed" suddenly take a different compilation pipeline, i.e.
>>> >>> SPIR-V
>>> >>> -> NIR -> ... seems risky.  It's a different compilation path that we
>>> >>> don't normally use.  And one you'd only hit in limited circumstances.
>>> >>> There's a lot of potential for really obscure bugs.
>>> >>
>>> >> Since we're going to expose exactly that path for GL_ARB_spirv /
>>> >> OpenGL
>>> >> 4.6, we'd better make sure it works always.  Right?
>>> >
>>> > In addition to the old pipeline:
>>> >
>>> > - GLSL from the app -> GLSL IR -> NIR -> i965 IR
>>> >
>>> > GL_ARB_spirv and OpenGL 4.6 add a second pipeline:
>>> >
>>> > - SPIR-V from the app -> NIR -> i965 IR
>>> >
>>> > Both of those absolutely have to work.  But these:
>>> >
>>> > - GLSL -> GLSL IR -> NIR -> SPIR-V -> NIR -> i965 IRs
>>> > - GLSL -> GLSL IR -> SPIR-V -> NIR -> i965 IRs
>>> >
>>> > aren't required to work, or even be supported.  It makes a lot of sense
>>> > to support them - both for testing purposes, and as an alternative to
>>> > glslang, for a broader tooling ecosystem.
>>> >
>>> > The thing that concerns me is that if you use SPIR-V for the cache, you
>>> > need these paths to not just work, but be _indistinguishable_ from one
>>> > another:
>>> >
>>> > - GLSL -> GLSL IR -> NIR -> ...
>>> > - GLSL -> GLSL IR -> NIR -> SPIR-V, then SPIR-V -> NIR -> ...
>>> >
>>> > Otherwise the original compile and partially-cached recompile might
>>> > have
>>> > different properties.  For example, if the the SPIR-V step messes with
>>> > variables or instruction ordering a little, it could trip up the loop
>>> > unroller so the original compiler gets unrolled, and the recompile from
>>> > partial cache doesn't get unrolled.  I don't want to have to debug
>>> > that.
>>>
>>> That is a very compelling argument.  If we want Mesa to be an
>>> alternative to glslang, I think we would like to have that property, but
>>> it's not a hard requirement for that use case.
>>
>>
>> I also find that argument rather compelling.  The SPIR-V -> NIR pass is
>> *not* a simple pass.  It does piles of lowering and things on-the-fly as
>> well as creating temporary variables for various things.  The best we could
>> hope to guarnatee would be that NIR -> SPIR-V -> NIR -> vars_to_ssa -> CSE
>> is idempotent.  Even that might be a bit of a stretch.
>
>
> I was talking to Jordan about this in person this morning and one other
> roadblock for using SPIR-V cropped up.  The place where we really want to
> cache the NIR for highest effectiveness would be right after calling
> brw_create_nir in brw_link_shader.  At this point in the process, a lot of
> lowering has been done to NIR intrinsics which have no SPIR-V equivalent.
> If we wanted to serialize the NIR as SPIR-V, we would either have to do it
> much higher up in the pipeline and lose quite a bit of the benefit, or add a
> NIR extension to SPIR-V that provides those extra intrinsics.  Adding such a
> SPIR-V extension and relevant headers and to/from NIR code is probably at
> least 2x the code of writing a nir_serialize pass.
>

not that it should be the deciding point, but one of these days when I
get a chance, I'd like to move some of the lowering done in nir->ir3
to nir->nir(+ir3 specific nir instructions/intrinsics).. which seems
like it would make serialization via spirv too early to be hugely
useful..

Maybe nir->spirv->nir round-tripping is useful for other reasons.. but
really nir_serialize shouldn't be that much work, and not enough to
care about it being duplicate effort with IR round tripping (which
imho, it isn't)

BR,
-R