[Mesa-dev] [RFC] NIR serialization

[Jordan Justen]

On 2017-09-10 17:15:48, Timothy Arceri wrote:
> Ccing list.
> 
> On 11/09/17 09:50, Timothy Arceri wrote:
> > Hi Daniel,
> > 
> > Here is the code that does the caching of tgsi in Gallium.
> > 
> > https://cgit.freedesktop.org/mesa/mesa/tree/src/mesa/state_tracker/st_shader_cache.c 
> > 
> > 
> > Basically we just use the blob helpers to do the writing/reading.
> > 
> > https://cgit.freedesktop.org/mesa/mesa/tree/src/compiler/glsl/blob.c

Maybe this should be moved to src/util?

Don't we want something like:

bool nir_serialize(void *mem_ctx, const nir_shader *s, void **serialized,
                   uint32_t *size);

bool nir_deserialize(void *mem_ctx, void *serialized, uint32_t size,
                     nir_shader **s);

The blob interface usage is then just an implementation detail.

-Jordan

> > For gallium you will want to create an equivalent for nir. Something 
> > like st_shader_cache_nir.c
> > 
> > Once we have created the blob we simply pass it along with the sha we 
> > will use as the cache key off to the disk cache utils for 
> > compression/writing etc. e.g
> > 
> > disk_cache_put(st->ctx->Cache, sha1, blob->data, blob->size, NULL);
> > 
> > If you are doing development on radeonsi you could try using the env var 
> > R600_DEBUG=nir to enable the experimental nir path Nicolai added. This 
> > has OpenGL support up to 3.1 I believe.
> > 
> > To load the nir for example you would change the st_link_shader() 
> > function in st_glsl_to_tgsi.cpp to do something like this:
> > 
> >     /* Return early if we are loading the shader from on-disk cache */
> >     if (preferred_ir == PIPE_SHADER_IR_NIR) {
> >        if (st_load_nir_from_disk_cache(ctx, 
> > pdisk_cache_put(st->ctx->Cache, sha1, blob->data, blob->size, NULL);rog)) {
> >           return GL_TRUE;
> >        }
> >     } else {
> >        if (st_load_tgsi_from_disk_cache(ctx, prog)) {
> >           return GL_TRUE;
> >        }
> >     }
> > 
> > And to store you need to update the various functions in st_program.c:
> > 
> > To call a st_store_nir_in_disk_cache() in the nir return paths. See the 
> > calls to st_store_tgsi_in_disk_cache() for reference.
> > 
> > Hope this helps getting you started.
> > 
> > Tim
> > 
> > On 10/09/17 20:22, Daniel Schürmann wrote:
> >> Hello Jordan,
> >>
> >> I wanted to ask you, if you could provide me with an interface / 
> >> function header for nir_(de)serialize, e.g. void nir_serialize(const 
> >> nir_shader *s, FILE f), because I am not sure how the shader cache 
> >> starts the serialization. Does it work on file directly or some 
> >> buffer? What about compression?
> >>
> >> If cached shader size is not too important (I was told its not), I 
> >> have an idea for a quick approach which uses nir_clone for 
> >> deserialization I would like to test.
> >>
> >> Thanks in advance
> >> Daniel