<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hello together!<br>
Recently, we had a small discussion (off the list) about the NIR
serialization, which was previously discussed in [RFC]
ARB_gl_spirv and NIR backend for radeonsi.</p>
<div class="moz-forward-container">As this topic could be
interesting to more people, I would like to share, what was talked
about so far (You might want to read from bottom up).<br>
<br>
TL;DR:<br>
- NIR serialization is in demand for shader cache<br>
- could be done either directly (NIR binary form) or via SPIR-V<br>
- Ian et al. are working on GLSL IR -> SPIR-V transformation,
which could be adapted for a NIR -> SPIR-V pass<br>
- in NIR representation, some type information is lost<br>
- thus, a serialization via SPIR-V could NOT be a glslang
alternative (otoh, the GLSL IR->SPIR-V pass could), but only
for spirv-opt (if the output is valid SPIR-V)<br>
- now, the question is if this is worth the additional effort<br>
<br>
Kind regards,<br>
Daniel<br>
<br>
-------- Forwarded Message --------
<table class="moz-email-headers-table" border="0" cellspacing="0"
cellpadding="0">
<tbody>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">Subject:
</th>
<td>Re: NIR serialization</td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">Date: </th>
<td>Tue, 5 Sep 2017 11:00:31 -0700</td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">From: </th>
<td>Ian Romanick <a class="moz-txt-link-rfc2396E" href="mailto:idr@freedesktop.org"><idr@freedesktop.org></a></td>
</tr>
<tr>
<th align="RIGHT" nowrap="nowrap" valign="BASELINE">To: </th>
<td>Daniel Schürmann
<a class="moz-txt-link-rfc2396E" href="mailto:daniel.schuermann@campus.tu-berlin.de"><daniel.schuermann@campus.tu-berlin.de></a>, Nicolai
Hähnle <a class="moz-txt-link-rfc2396E" href="mailto:nhaehnle@gmail.com"><nhaehnle@gmail.com></a>, Timothy Arceri
<a class="moz-txt-link-rfc2396E" href="mailto:tarceri@itsqueeze.com"><tarceri@itsqueeze.com></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<pre>Sorry for taking so long to reply. It was a long holiday weekend in the
US, and I was away.
On 09/01/2017 05:03 AM, Daniel Schürmann wrote:
> A direct NIR binary serialization would also do the job (vc4/freedreno
> was mentioned as well).
> I only thought that SPIRV is preferable because
> - deserialization for free
> - cached shader size
> - spirv-opt and glslang alternative
>
> The term lossy doesn't make much sense to me with regard to
> optimizations: aren't all optimizations lossy?
By lossy I mean there is a significant semantic change. As soon as
GLSL IR is converted to NIR, Boolean types completely cease to exist.
They are replaced with integers that are either 0 or -1. Similarly, all
matrix types cease to exist. They are replaced by a set of vectors.
For the purpose of the on-disk cache, this probably doesn't matter. It
does mean that additional information about, for example, types of
uniforms has to be tracked. In a direct GLSL IR to SPIR-V translation,
type information is maintained, so the SPIR-V has all the necessary
information.
As a glslang replacement, maintaining type information is an absolute
requirement. Users will use other tools to introspect the SPIR-V shader
to find locations of uniforms, shader inputs, offsets of values in UBOs,
etc. If the types are changed in the SPIR-V shader that we emit, none
of that will work. I plan to enable retrieval of portable SPIR-V both
from a Mesa driver and the standalone GLSL compiler.
Right now SPIR-V binaries will be quite large. I have several ideas
that I plan to implement once we have OpenGL 4.6 done that should
dramatically reduce the size of SPIR-V... I'm actually hoping to present
that at FOSDEM.
> The primary goal would be the lossless NIR-SPIRV-NIR round-trip.
> Secondary, it would be desirable if we achieve valid SPIRV binaries
> which preserve the semantics of the original shader.
> And here is the question if this is possible with the type information
> that are available...
>
> Ian: can you hint me to your repository? I couldn't find it.
<a class="moz-txt-link-freetext" href="https://cgit.freedesktop.org/~idr/mesa/log/?h=emit-spirv">https://cgit.freedesktop.org/~idr/mesa/log/?h=emit-spirv</a>
> Kind regards,
>
> Daniel
>
>
> On 09/01/2017 12:16 PM, Nicolai Hähnle wrote:
>> In addition to using NIR-based optimizations, I believe Timothy
>> mentioned that a method for serializing NIR would help the shader disk
>> cache of i965. It would certainly help radeonsi if/when we switch to
>> the NIR backend, because we could compile new shader variants without
>> falling back all the way to GLSL. For that, a lossless NIR-SPIRV-NIR
>> path would do the job.
>>
>> Not that falling back all the way to GLSL from radeonsi is impossible,
>> but it would also require a whole bunch of new groundwork to be done
>> -- basically, we would need multi-threaded GLSL compilation and linking.
>>
>> Cheers,
>> Nicolai
>>
>>
>> On 01.09.2017 02:41, Ian Romanick wrote:
>>> I have been working on GLSL IR to SPIR-V. I have a bunch of stuff in
>>> the emit-spirv branch of my freedesktop.org tree. Once that is done, it
>>> should be pretty trivial to adapt it to NIR to SPIR-V, but I don't know
>>> how useful that would be for Mesa. Part of the problem is NIR loses a
>>> lot of information about types (bool and matrix types), so a
>>> SPIRV-NIR-SPIRV path would necessarily be lossy.
>>>
>>> On the flip side, GLSL IR lacks a huge number of optimizations that
>>> exist in NIR, so it's probably not a huge improvement over spirv-opt.
>>>
>>> On 08/26/2017 02:33 PM, Nicolai Hähnle wrote:
>>>> Hey Ian,
>>>>
>>>> Have you done any more concrete work on NIR serialization? See below...
>>>>
>>>> Cheers,
>>>> Nicolai
>>>>
>>>> On 26.08.2017 23:17, Daniel Schürmann wrote:
>>>>> Hello Nicolai,
>>>>>
>>>>> I'm a Master student (CS) from TU Berlin and currently writing an
>>>>> OpenMP backend for clang using SPIR-V/OpenCL for my thesis. As I'm
>>>>> interested in mesa and graphics driver development since long time, I
>>>>> would like to get involved a little bit.
>>>>>
>>>>> Recently, I read your [RFC] ARB_gl_spirv and NIR backend for radeonsi.
>>>>> You propose a serialization of NIR either directly or by using SPIR-V
>>>>> as binary format.
>>>>> I wanted to ask if you (or someone else) already started implementing
>>>>> this because it seems to be a nice task for me to get into mesa and
>>>>> NIR.
>>>>> I would rather try to use SPIR-V as binary format for two (not yet
>>>>> mentioned) reasons:
>>>>> - it could be used as a replacement for spirv-opt and draw some
>>>>> attention
>>>>> - I expect SPIR-V to be smaller than a raw (unoptimized) serialization
>>>>> of NIR.
>>>>>
>>>>> The round-trip NIR -> SPIRV -> NIR should preserve all information
>>>>> while SPIRV -> NIR -> SPIRV must preserve the semantic meaning. Please
>>>>> tell me, if your know about some hindering obstacles or if someone is
>>>>> already working on this.
>>>>>
[snip]
>>>>>
>>>>> Thank you for your time,
>>>>> kind regards,
>>>>>
>>>>> Daniel
</pre>
</div>
</body>
</html>