[virglrenderer-devel] Thoughts about TGSI-to-GLSL and improvements and beyond

[Gert Wollny]

Dear all, 

Erik and me collected a few thought about the TGSI to GLSL translation
code. Here is the result:

Current  problems and possible solution: 
----    
P: 
* using fixed-size buffers for some code creation 
* all concetations call  realloc 

S: 
* Use glib strings 
  + glib is a qemu dependency, so the addition is no real burden
    - except for non-qemu users? (like crosvm?) 
    + glib is widely used an probabably available somewhere in chromeos
    + crosvm uses rust, so doing it in rust would make sense there
  + supports growing strings as needed, 
    In order to avoid too many re-allocations a translation context
    can  hold a few buffers that grow as needed and are kept around
    With gcc and clang there is even an auto-pointer like feature for
    glib types

Alternative: implement a simple growing string-buffer ourselves
It's not that hard can be coupled with some domain-specifics, like
automatically generating the right level of indentation. 

In a more radical approach one could also think about using a real OO
language for this (C++, Rust, D,  ... )
-----

P: 
* lots of code duplication 
S: 
* replace the in-lined strings for build-ins with constants
* Move the duplicated code in the case chains into functions 
* use some object-oriented approach to handle the different shader
  types
* Lots of small, composable helpers to emit things?

----
P: 
* VS GLSL code needs to be patched based on FS (possibly also
TESS_EVAL    and GS could need this, but so far there is no test case)
S: 
* decouple parsing the TGSI from creating the GLSL 
  - This would add another layer with an additional IR
  + Can we do this on the mesa-side? Most gallium drivers don't finish
    off compiling shaders until draw. Perhaps we could do the same,
    and then have the "full perspective"?

----
P:
* TGSI might not be the best suited IR: 
  - Probably not going to get *that* much more love in the future, 
    due to NIR taking over
  - We already diverged from upstream
  - Untyped TGSI-registers leads to a *lot* of bitcasts.
    Can result in slow to compile shaders
    - and quite possibly slow to run, because a lot of on-line shader
      compilers don't optimize too much ... especially on mobile
  Note: NIR has the same problem ...but easier to figure out, due to
  SSA form

S:
+ Introduce a more reasonable IR? (see above)
+ Could we get away with sending GLSL to the guest instead of TGSI?
  - I guess one reason not to send the GLSL would be to not send
    literal strings to the host that could be used to exploit
    vulnerabilities in the host shader compiler 
+ How about SPIRV?
  Has a stable, actively maintained serialization, and SPIRV-Cross can
turn
  it back to GLSL in case the host lacks GL_ARB_spirv 
  (part of OpenGL 4.6)
----

That's about it, thanks for any commenst, 
Gert 

PS: The credit for thinking about the more advanced stuff like NIR and
SPIRV goes to Erik.