[Mesa-dev] [v2 6/6] mesa: OES_get_program_binary functionality

[Tapani Pälli]

On 11/06/2013 12:14 AM, Paul Berry wrote:
> On 1 November 2013 05:08, Tapani Pälli <tapani.palli at intel.com 
> <mailto:tapani.palli at intel.com>> wrote:
>
>     On 11/01/2013 12:38 PM, Erik Faye-Lund wrote:
>
>         On Fri, Nov 1, 2013 at 11:35 AM, Tapani Pälli
>         <tapani.palli at intel.com <mailto:tapani.palli at intel.com>> wrote:
>
>             On 11/01/2013 12:21 PM, Erik Faye-Lund wrote:
>
>
>                 Won't using the git-sha1 as a compatibility-criteria
>                 cause issues for
>                 developers with local changes? I'm not so worried
>                 about this for
>                 OES_get_program_binary itself, but once the
>                 shader-cache is in place
>                 it sounds like a potential source of difficult to
>                 track down
>                 misbehavior...
>
>
>             I agree it might be too aggressive criteria but it is hard
>             to come up with
>             better and as simple.
>
>         That's not my objection. My objection is that this might give
>         headaches for people with local modifications to the
>         glsl-compiler.
>         Local modifications does not affect the git-sha1.
>
>
>     For the automatic shader cache this headache could be helped a bit
>     with a environment variable or drirc setting that can be used
>     during development. On the other hand an automatic cache must work
>     in a transparent way so it should be always able to recover when
>     it fails, so one should only see it as 'slower than usual' (since
>     recompilation/relink required) sort of behaviour. The WIP of the
>     automatic cache I sent some time earlier also marked (renamed)
>     these 'problematic' cached shaders so that they can be detected on
>     further runs and cache can ignore those.
>
>     I agree that it might become problematic, on the other hand it is
>     also easy to just wipe ~/.cache/mesa and disable cache. Not sure
>     if Nvidia or Imagination try to handles these cases with their
>     cache implementations.
>
>
> I'm also concerned about this, especially for the automatic shader 
> cache.  During development, we frequently make small changes to the 
> front end, recompile, and then run a small test program, expecting our 
> changes to take effect.  I'm very worried about requiring developers 
> to remember to set an environment variable, change a drirc setting, or 
> wipe out a cache when making changes that haven't been committed yet.  
> Especially when the consequence of forgetting to do so is that the 
> change you were trying to make won't have any observed effect. That's 
> the sort of thing that leads people to spend hours chasing phantom bugs.
>
> How about if we have the cache mechanism include the modification time 
> of the mesa .so in the shader binary (in addition to the sha), and 
> reject shader binaries that don't have a matching modification time?  
> That way if mesa is recompiled, any previously-cached shaders will 
> automatically become invalid without the developer having to do any work.
>

My next plan for adding more validation was to add ir instruction sizes 
and a couple important struct sizes as part of the dump, this would 
allow some amount of sanity checking but mesa modification time addition 
sounds like better idea and would not add so much additional data.

>
> A related concern I have is: what happens if someone changes their 
> video card, or transplants their hard drive into a different machine 
> that has different graphics hardware?  Different back-ends have 
> different settings of ctx->ShaderCompilerOptions (some back-ends have 
> different values from one chipset to another), so even if the version 
> of Mesa is the same and there are no local developer changes, the 
> linked IR is not going to necessarily be the same if the graphics 
> hardware changes.
>
> To address this, I'd recommend that we also include the device's PCI 
> ID in the shader binary, and reject mismatched binaries.

Thanks, this is is a nice idea, currently there's also the vendor and 
renderer strings in the dump but so far these have been used only for 
debug purposes.

// Tapani

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20131106/305f09ec/attachment.html>