[Mesa-dev] Low-level infrastructure for the shader cache

[Carl Worth]

Hi folks,

This series adds a layer of code to store a cache of objects on
disk. Thanks to Kristian Høgsberg for the initial proof-of-concept
implementation here. I've take his original code and added my own
cleanups and documentation to the cache API. I've also fixed up a
couple of the items he had left as FIXMEs.

In this series, my cleaned-up API arrives first, so you won't see the
changes I've made there. But, for the implementation fixes, I've left
his code as originally written and have separate commits for my
fixes. I think this is the history we want for sake of review and
maintainability. The code compiles at all points, (and is never used),
so it's not like there are any regressions left lingering due to the
unsquashed history.

Like I said, None of the code here is actually used yet, (that will be
a future series of patches, where we'll finally have an actual shader
cache working). But I think the cache API and implementation can be
productively reviewed now.

There is one FIXME comment in the implementation here that will still
need to be addressed. I'm soliciting some thoughts from others on how
to do it. The issue is around the mmapped index file used for
random-cache replacement, and how to ensure things stay consistent
with multiple processes.

Kristian's comments in this area are as follows.

First, when mapping the index file:

   /* FIXME: We map this shared, which is a start, but we need to think about
    * how to make it multi-process safe. */
   cache->index = (unsigned char *)
      mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

Then, when reading an entry from the cache file, (and potentially
unlinking the old file being replaced):

   /* FIXME: We'll need an fsync here and think about races... maybe even need
    * an flock to avoid leaking files.  Or maybe fsync, then read back and
    * verify the entry is still ours, delete it if somebody else overwrote
    * it. */

So please follow up in replies if you have good ideas on how to best
address these comments.

Other than that, I think the code in this series is sound. Please do
let me know what you see that I've missed.

-Carl