[Mesa-dev] [RFC] ARB_gl_spirv and NIR backend for radeonsi

[Marek Olšák]

On Sun, May 21, 2017 at 12:48 PM, Nicolai Hähnle <nhaehnle at gmail.com> wrote:
> Hi all,
>
> I've been looking into ARB_gl_spirv for radeonsi. I don't fancy re-inventing
> the ~8k LOC of src/compiler/spirv, and there's already a perfectly fine
> SPIR-V -> NIR -> LLVM compiler pipeline in radv, so I looked into re-using
> that.
>
> It's not entirely straightforward because radeonsi and radv use different
> "ABIs" for their shaders, i.e. prolog/epilog shader parts, different user
> SGPR allocations, descriptor loads work differently (obviously), and so on.
>
> Still, it's possible to separate the ABI from the meat of the NIR -> LLVM
> translation. So here goes...
>
>
> The Step-by-Step Plan
> =====================
>
> 1. Add an optional GLSL-to-NIR path (controlled by R600_DEBUG=nir) for very
> simple VS-PS pipelines.
>
> 2. Add GL_ARB_gl_spirv support to Mesa and test it on simple VS-PS
> pipelines.
>
> 3. Fill in all the rest:
> 3a. GL 4.x shader extensions (SSBOs, images, atomics, ...)
> 3b. Geometry and tessellation shaders
> 3c. Compute shaders
> 3d. Tests
>
> I've started with step 1 and got basic GLSL 1.30-level vertex shaders
> working via NIR. The code is here:
> https://cgit.freedesktop.org/~nh/mesa/log/?h=nir
>
> The basic approach is to introduce `struct ac_shader_abi' to capture the
> differences between radeonsi and radv. In the end, the entry point for NIR
> -> LLVM translation will simply be:
>
>    void ac_nir_translate(struct ac_llvm_context *ac,
>                          struct ac_shader_abi *abi,
>                          struct nir_shader *nir);
>
> Setting up the LLVM function with its parameters is still considered part of
> the driver.

This sounds good.

>
>
> Questions
> =========
>
> 1. How do we get good test coverage?
> ------------------------------------
> A natural candidate would be to add a SPIR-V execution mode for the piglit
> shader_runner. That is, use build scripts to extract shaders from
> shader_test files and feed them through glslang to get spv files, and then
> load those from shader_runner if a `-spirv' flag is passed on the command
> line.
>
> This immediately runs into the difficulty that GL_ARB_gl_spirv wants SSO
> linking semantics, and I'm pretty sure the majority of shader_test files
> don't support that -- if only because they don't set a location on the
> fragment shader color output.
>
> Some ideas:
> 1. Add a GL_MESA_spirv_link_by_name extension
> 2. Have glslang add the locations for us (probably difficult because glslang
> seems to be focused on one shader stage at a time.)
> 3. Hack something together in the shader_test-to-spv build scripts via
> regular expressions (and now we have two problems? :-) )
> 4. Other ideas?

We have plenty of GLSL SSO shader tests in shader-db, but we can only
compile-test them.

Initially I think we can convert a few shader tests to SSO manually
and use those.

>
>
> 2. What's the Gallium interface?
> --------------------------------
> Specifically, does it pass SPIR-V or NIR?
>
> I'm leaning towards NIR, because then specialization, mapping of uniform
> locations, atomics, etc. can be done entirely in st/mesa.
>
> On the other hand, Pierre Moreau's work passes SPIR-V directly. On the third
> hand, it wouldn't be the first time that clover does things differently.

If you passed SPIR-V to radeonsi and let radeonsi do SPIR-V -> NIR ->
LLVM, you wouldn't need the serialization capability in NIR. You can
just use SPIR-V as the shader binary and the major NIR disadvantage is
gone. Also, you won't have to touch GLSL-to-NIR, and the radeonsi
shader cache will continue working as-is.

However, I don't know how much GL awareness is required for doing
SPIR-V -> NIR in radeonsi. Additional GL-specific information might
have to be added to SPIR-V by st/mesa for the conversion to be doable.
You probably know better.

st/mesa or core Mesa just needs to fill gl_program, gl_shader, and
gl_shader_program by parsing SPIR-V and not relying on NIR. I don't
know how feasible that is, but it seems to be the only thing needed in
shared code.

That also answers the NIR vs TGSI debate for the shader cache. The
answer is: Neither.

Marek