[Mesa-dev] [RFC 07/10] mesa: add program blob cache functionality

[Jordan Justen]

On 2018-01-08 23:48:19, Tapani Pälli wrote:
> Cache set and get are called in similar fashion as what is happening
> with disk cache. Functionality requires ARB_get_program_binary and
> EGL_ANDROID_blob_cache support.
> 
> Signed-off-by: Tapani Pälli <tapani.palli at intel.com>
> ---
>  src/mesa/Makefile.sources          |   2 +
>  src/mesa/main/program_blob_cache.c | 141 +++++++++++++++++++++++++++++++++++++
>  src/mesa/main/program_blob_cache.h |  48 +++++++++++++
>  src/mesa/meson.build               |   2 +
>  src/mesa/program/ir_to_mesa.cpp    |   9 ++-
>  5 files changed, 201 insertions(+), 1 deletion(-)
>  create mode 100644 src/mesa/main/program_blob_cache.c
>  create mode 100644 src/mesa/main/program_blob_cache.h
> 
> diff --git a/src/mesa/Makefile.sources b/src/mesa/Makefile.sources
> index 53fa486364..bbcfdb425e 100644
> --- a/src/mesa/Makefile.sources
> +++ b/src/mesa/Makefile.sources
> @@ -177,6 +177,8 @@ MAIN_FILES = \
>         main/polygon.h \
>         main/program_binary.c \
>         main/program_binary.h \
> +       main/program_blob_cache.c \
> +       main/program_blob_cache.h \
>         main/program_resource.c \
>         main/program_resource.h \
>         main/querymatrix.c \
> diff --git a/src/mesa/main/program_blob_cache.c b/src/mesa/main/program_blob_cache.c
> new file mode 100644
> index 0000000000..0b3ea1a549
> --- /dev/null
> +++ b/src/mesa/main/program_blob_cache.c
> @@ -0,0 +1,141 @@
> +/*
> + * Mesa 3-D graphics library
> + *
> + * Copyright (C) 2018 Intel Corporation.  All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included
> + * in all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
> + * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + *
> + */
> +
> +#include "main/errors.h"
> +#include "main/mtypes.h"
> +#include "main/shaderobj.h"
> +#include "main/program_binary.h"
> +#include "util/mesa-sha1.h"
> +#include "compiler/glsl/program.h"
> +
> +#include "program_blob_cache.h"
> +
> +/* This is what Android EGL defines as the maxValueSize in egl_cache_t
> + * class implementation.
> + */
> +#define MAX_BLOB_SIZE 64 * 1024
> +
> +static void
> +generate_sha1_string(struct gl_context *ctx, struct gl_shader_program *shProg,
> +                     char *key)
> +{
> +   char *buf = create_shader_program_keystr(ctx, shProg);
> +   struct mesa_sha1 sha_ctx;
> +   unsigned char sha1str[20];
> +
> +   /* Add driver sha1 to the key string. */
> +   uint8_t driver_sha1[20];
> +   char driver_sha1buf[41];
> +
> +   ctx->Driver.GetProgramBinaryDriverSHA1(ctx, driver_sha1);
> +   _mesa_sha1_format(driver_sha1buf, driver_sha1);
> +   ralloc_asprintf_append(&buf, "%s", driver_sha1buf);
> +
> +   _mesa_sha1_init(&sha_ctx);
> +   _mesa_sha1_update(&sha_ctx, buf, strlen(buf));
> +   _mesa_sha1_final(&sha_ctx, sha1str);
> +   _mesa_sha1_format(key, sha1str);
> +
> +   ralloc_free(buf);
> +}
> +
> +void
> +_mesa_blob_cache_set(struct gl_context *ctx,
> +                     struct gl_shader_program *shProg)
> +{
> +   assert(shProg->data->LinkStatus == linking_success);
> +
> +   /* ARB_get_program_binary support required. */
> +   if (!ctx->blobCacheSet || !ctx->Driver.GetProgramBinaryDriverSHA1)
> +      return;
> +
> +   /* Skip cache for fixed-function programs and programs that use
> +    * transform feedback.
> +    */
> +   if (!shProg->Name || shProg->TransformFeedback.NumVarying > 0)
> +      return;
> +
> +   GLint length;
> +   _mesa_get_program_binary_length(ctx, shProg, &length);

I see this is built on GetProgramBinary. That might be fine, but what
if we utilized the blob read/write as an alternative storage for the
general disk shader cache?

It would allow us to skip compiling the GLSL.

I haven't looked much yet, but I didn't see anything that might
explain the texture issue you noted. It is possible that it might go
away if we instead shimmed this into the disk shader cache.

-Jordan