[Mesa-dev] [PATCH 0/4] RFC: gl_shader cache

[Tapani Pälli]

Hello;

This is RFC for shader cache implementation, goal is to optimize application
startup time. The implementation is based on idea that Kenneth had on a Mesa
branch for memory based cache. These patches implement a 'behind the scenes'
cache for shader compiler storing gl_shader struct to the disk. Later on my
goal is to implement OES_get_program_binary extension using some of this
implementation as the basis.

There are already number of TODO's and issues mentioned in commit msgs,
please check these first. Then, there are probably bugs and missing things
that I'm not aware of, please let me know about any of these. One big TODO
is to rewrite the serialization to happen to memory area first (not directly
to the disk), this will make serialization more robust with more centralized
error handling but also makes it more usable for OES_get_program_binary
implementation. I haven't paid much attention to the portability of this code
so I would be interested to know also how well this would work with windows
and others.

One big TODO/goal is to move this all to happen after linking to cache
all shaders in a gl program. This will give far better optimization for
startup time. Currently rough approx 1/4 time is spent during compilation
but much more after linking and optimizations done after linking (this is
based on analysis done with particular app and callgrind). I have already
somewhat promising results from this cache, for example compiling L4D2
shaders and shader-db runs are faster..

On my SNB desktop machine shader-db takes ~2.4secs without cache, with cache
it takes ~1.8secs (~2.7 when cache gets generated) (values are avg from 100
runs compiling all the shaders). Shader-db creates 266 blobs eating 46M of
disk space when using cache.

Any comments greatly appreciated!

Tapani Pälli (4):
  glsl: add ir_cache class for IR serialization (WIP)
  mesa: gl_shader_cache class (WIP)
  glsl: export populate_symbol_table
  glsl: use gl_shader_cache when compiling (WIP)

 src/glsl/Makefile.sources         |   2 +
 src/glsl/glsl_parser_extras.cpp   |  40 ++
 src/glsl/ir_cache.h               | 535 +++++++++++++++++++++
 src/glsl/ir_cache_serialize.cpp   | 603 ++++++++++++++++++++++++
 src/glsl/ir_cache_unserialize.cpp | 969 ++++++++++++++++++++++++++++++++++++++
 src/glsl/linker.cpp               |   2 +-
 src/glsl/linker.h                 |   3 +
 src/mesa/Makefile.sources         |   1 +
 src/mesa/main/context.c           |   2 +
 src/mesa/main/mtypes.h            |   4 +
 src/mesa/main/shadercache.cpp     | 198 ++++++++
 src/mesa/main/shadercache.h       |  68 +++
 12 files changed, 2426 insertions(+), 1 deletion(-)
 create mode 100644 src/glsl/ir_cache.h
 create mode 100644 src/glsl/ir_cache_serialize.cpp
 create mode 100644 src/glsl/ir_cache_unserialize.cpp
 create mode 100644 src/mesa/main/shadercache.cpp
 create mode 100644 src/mesa/main/shadercache.h

-- 
1.8.1.4