[Mesa-dev] GLSL IR & TGSI on-disk shader cache

[Bas Nieuwenhuizen]

On Wed, Feb 8, 2017, at 11:25, Matt Turner wrote:
> On Wed, Feb 8, 2017 at 12:31 AM, Timothy Arceri <t_arceri at yahoo.com.au>
> wrote:
> > On Tue, 2017-02-07 at 23:58 +0100, Matt Turner wrote:
> >> On Tue, Feb 7, 2017 at 4:42 AM, Timothy Arceri <tarceri at itsqueeze.com
> >> > wrote:
> >> > This series adds support for a GLSL IR level and TGSI (OpenGL/st)
> >> > level caches.
> >> >
> >> > There are a few small bug fixes and a little bit of restructuring
> >> > to the GLSL IR patches to make things work for Gallium drivers vs
> >> > just i965 previously, but nothing too major.
> >> >
> >> > The biggest change is patch 35 which changes the mesa cache
> >> > directory to be more structured allowing us to not fallback to
> >> > recompiles at draw time if we realise a cache item was created
> >> > with an old version of Mesa. Patch 36 also cleans up old cache
> >> > files so we shouldn't get into a possition were we are constantly
> >> > evicting cache items because its never cleaned out.
> >> >
> >> > Even with the new directory structure I've still left in the
> >> > code that adds the Mesa version string to cache items because
> >> > users can do things like override the OpenGL version which will
> >> > change the output (in future we might want to just add this as
> >> > part of the sha1 input).
> >> >
> >> > This series does not include the patch that adds cache support
> >> > to the radeonsi backend, the main reason for this is that llvm
> >> > currently doesn't allow the version to be queried at runtime
> >> > (as far as I'm aware) although it seems like other are interested
> >> > in this feature [1] so I will follow up on that.
> >> >
> >> > The reason we don't just use the build time like radv is that we
> >> > will want something consistent accross distros to enable
> >> > distribution of precompiled shaders.
> >>
> >> I think I have a solution for this. I am traveling until next week,
> >> but I will send it as soon as I can.
> >
> > Are you able to provide a hint as to what you are suggesting? I've just
> > pinged the llvm about creating the api to query the version. Would be
> > interested to know if your idea will make the unnecessary.
> 
> Yes, (1) build with GNU ld's --build-id=sha1 flag, which embeds the
> sha1 of the object into a special .note.gnu.build-id ELF note section;
> (2) read that section using dl_iterate_phdr() and a little bit of
> parsing; (3) use sha1 plus perhaps other information to uniquely
> identify the runtime configuration.
> 
> I've written an example of how to do it here:
> http://git.mattst88.com/build-id/
> 
> That should definitely be better than the alternatives I've heard
> proposed (mtime of the .so, manually hashing the source directory,
> ...). I'm happy to implement this next week, or you can feel free to
> give it a shot.

Yeah,also  just figured that out during the week. The fundamental issue
here is that the SHA1 might come out different due to compiler
differences between distros, even if they are compiling exactly the same
version. This increases the number of versions of the cache that have to
be distributed if the application would like to distribute precompiled
caches.

- Bas 
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev