[Mesa-dev] RFC: parallel shader compiles in mesa

[Ilia Mirkin]

On Sun, Nov 1, 2015 at 7:27 PM, Connor Abbott <cwabbott0 at gmail.com> wrote:
> There was already some work done for this (with i965) a while back:
>
> http://lists.freedesktop.org/archives/mesa-dev/2014-July/063008.html
>
> IIRC the reason it didn't get merged was that Chia never actually made
> sure that the compiler was threadsafe (despite some patches fixing
> some things).

That's a good point. I think it's safe to say that no matter how much
work I do fixing threading issues, some will be left over. I'm
thinking that threading will only be enabled when
ctx->Exceptions.ARB_parallel_shader_compile is enabled, and to start
with, it never will be. Developers will be able to test things by
overriding it to true, and when each driver team is comfortable that
its infrastructure (and the infra it depends on) is able to work
reliably, this can be flipped on.

IMHO it'd be useful to have some semi-working-but-not-completely-done
bits on this merged in before all the issues are ironed out. I think
background compiles are something everyone wants, and fixes will be
largely incremental. As long as everyone agrees with the general
approach I outlined. Basically it'll be:

Step 1: isolate compile_shader/link_program from most of the GL context state
Step 2: background them into separate threads and add machinery to
wait on those to complete when necessary
Step 3: fix the infinity bugs that result

I doubt that most of the resulting bugs will be things that will cause
significant changes to the work done in steps 1/2.

It seems that Chia added some measure of threading into the compiler
itself, which is deeper into the stack than I had in mind. I just
wanted to stick compile_shader/link_program into background threads.
Some measure of isolation will need to be done... e.g. gl_context's
are passed in, but in general all that's used from them is
Extensions/Consts/API, so encapsulating that into a const mini-context
will make sense, to avoid abuses down the line.

  -ilia