[Mesa-dev] [PATCH 03/88] glsl: Add initial functions to implement an on-disk cache
Brian Paul
brianp at vmware.com
Mon Sep 26 14:29:25 UTC 2016
On 09/23/2016 11:24 PM, Timothy Arceri wrote:
> From: Carl Worth <cworth at cworth.org>
>
> This code provides for an on-disk cache of objects. Objects are stored
> and retrieved via names that are arbitrary 20-byte sequences,
> (intended to be SHA-1 hashes of something identifying for the
> content). The directory used for the cache can be specified by means
> of environment variables in the following priority order:
>
> $MESA_GLSL_CACHE_DIR
> $XDG_CACHE_HOME/mesa
> <user-home-directory>/.cache/mesa
>
> By default the cache will be limited to a maximum size of 1GB. The
> environment variable:
>
> $MESA_GLSL_CACHE_MAX_SIZE
>
> can be set (at the time of GL context creation) to choose some other
> size. This variable is a number that can optionally be followed by
> 'K', 'M', or 'G' to select a size in kilobytes, megabytes, or
> gigabytes. By default, an unadorned value will be interpreted as
> gigabytes.
>
> The cache will be entirely disabled at runtime if the variable
> MESA_GLSL_CACHE_DISABLE is set at the time of GL context creation.
>
> Many thanks to Kristian Høgsberg <krh at bitplanet.net> for the initial
> implementation of code that led to this patch. In particular, the idea
> of using an mmapped file, (indexed by a portion of the SHA-1), for the
> efficent implementation of cache_has_key was entirely his
> idea. Kristian also provided some very helpful advice in discussions
> regarding various race conditions to be avoided in this code.
>
> Signed-off-by: Timothy Arceri <timothy.arceri at collabora.com>
> ---
> configure.ac | 3 +
> docs/envvars.html | 11 +
> src/compiler/Makefile.glsl.am | 10 +
> src/compiler/Makefile.sources | 4 +
> src/compiler/glsl/cache.c | 709 +++++++++++++++++++++++++++++++++++
> src/compiler/glsl/cache.h | 172 +++++++++
> src/compiler/glsl/tests/.gitignore | 1 +
> src/compiler/glsl/tests/cache_test.c | 416 ++++++++++++++++++++
> 8 files changed, 1326 insertions(+)
> create mode 100644 src/compiler/glsl/cache.c
> create mode 100644 src/compiler/glsl/cache.h
> create mode 100644 src/compiler/glsl/tests/cache_test.c
>
> diff --git a/configure.ac b/configure.ac
> index 0604ad9..7db31e4 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -1305,6 +1305,9 @@ if test "x$with_sha1" = "x"; then
> fi
> fi
> AM_CONDITIONAL([ENABLE_SHADER_CACHE], [test x$enable_shader_cache = xyes])
> +if test "x$enable_shader_cache" = "xyes"; then
> + AC_DEFINE([ENABLE_SHADER_CACHE], [1], [Enable shader cache])
> +fi
>
> case "$host_os" in
> linux*)
> diff --git a/docs/envvars.html b/docs/envvars.html
> index cf57ca5..2375145 100644
> --- a/docs/envvars.html
> +++ b/docs/envvars.html
> @@ -112,6 +112,17 @@ glGetString(GL_VERSION) for OpenGL ES.
> glGetString(GL_SHADING_LANGUAGE_VERSION). Valid values are integers, such as
> "130". Mesa will not really implement all the features of the given language version
> if it's higher than what's normally reported. (for developers only)
> +<li>MESA_GLSL_CACHE_DISABLE - if set, disables the GLSL shader cache
> +<li>MESA_GLSL_CACHE_MAX_SIZE - if set, determines the maximum size of
> +the on-disk cache of compiled GLSL programs. Should be set to a number
> +optionally followed by 'K', 'M', or 'G' to specify a size in
> +kilobytes, megabytes, or gigabytes. By default, gigabytes will be
> +assumed. And if unset, a maxium size of 1GB will be used.
> +<li>MESA_GLSL_CACHE_DIR - if set, determines the directory to be used
> +for the on-disk cache of compiled GLSL programs. If this variable is
> +not set, then the cache will be stored in $XDG_CACHE_HOME/.mesa (if
> +that variable is set), or else within .cache/mesa within the user's
> +home directory.
> <li>MESA_GLSL - <a href="shading.html#envvars">shading language compiler options</a>
> <li>MESA_NO_MINMAX_CACHE - when set, the minmax index cache is globally disabled.
> </ul>
> diff --git a/src/compiler/Makefile.glsl.am b/src/compiler/Makefile.glsl.am
> index b8225cb..80dfb73 100644
> --- a/src/compiler/Makefile.glsl.am
> +++ b/src/compiler/Makefile.glsl.am
> @@ -33,6 +33,7 @@ EXTRA_DIST += glsl/tests glsl/glcpp/tests glsl/README \
> TESTS += glsl/glcpp/tests/glcpp-test \
> glsl/glcpp/tests/glcpp-test-cr-lf \
> glsl/tests/blob-test \
> + glsl/tests/cache-test \
> glsl/tests/general-ir-test \
> glsl/tests/optimization-test \
> glsl/tests/sampler-types-test \
> @@ -47,6 +48,7 @@ check_PROGRAMS += \
> glsl/glcpp/glcpp \
> glsl/glsl_test \
> glsl/tests/blob-test \
> + glsl/tests/cache-test \
> glsl/tests/general-ir-test \
> glsl/tests/sampler-types-test \
> glsl/tests/uniform-initializer-test
> @@ -58,6 +60,11 @@ glsl_tests_blob_test_SOURCES = \
> glsl_tests_blob_test_LDADD = \
> glsl/libglsl.la
>
> +glsl_tests_cache_test_SOURCES = \
> + glsl/tests/cache_test.c
> +glsl_tests_cache_test_LDADD = \
> + glsl/libglsl.la
> +
> glsl_tests_general_ir_test_SOURCES = \
> glsl/tests/builtin_variable_test.cpp \
> glsl/tests/invalidate_locations_test.cpp \
> @@ -120,6 +127,9 @@ glsl_libglsl_la_SOURCES = \
> $(LIBGLSL_GENERATED_FILES) \
> $(LIBGLSL_FILES)
>
> +if ENABLE_SHADER_CACHE
> +glsl_libglsl_la_SOURCES += $(LIBGLSL_SHADER_CACHE_FILES)
> +endif
>
> glsl_libstandalone_la_SOURCES = \
> $(GLSL_COMPILER_CXX_FILES)
> diff --git a/src/compiler/Makefile.sources b/src/compiler/Makefile.sources
> index f5b4f9c..712b33a 100644
> --- a/src/compiler/Makefile.sources
> +++ b/src/compiler/Makefile.sources
> @@ -136,6 +136,10 @@ LIBGLSL_FILES = \
> glsl/s_expression.cpp \
> glsl/s_expression.h
>
> +LIBGLSL_SHADER_CACHE_FILES = \
> + glsl/cache.c \
> + glsl/cache.h
> +
> # glsl_compiler
>
> GLSL_COMPILER_CXX_FILES = \
> diff --git a/src/compiler/glsl/cache.c b/src/compiler/glsl/cache.c
> new file mode 100644
> index 0000000..c4c4fb7
> --- /dev/null
> +++ b/src/compiler/glsl/cache.c
> @@ -0,0 +1,709 @@
> +/*
> + * Copyright © 2014 Intel Corporation
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + */
> +
> +#include <ctype.h>
> +#include <string.h>
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <sys/file.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <sys/mman.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +#include <pwd.h>
> +#include <errno.h>
> +#include <dirent.h>
Some of those headers don't exist on Windows. All the
directory/file-related code will have to be inspected for Unix vs. Windows.
-Brian
> +
> +#include "util/u_atomic.h"
> +#include "util/mesa-sha1.h"
> +#include "util/ralloc.h"
> +#include "main/errors.h"
> +
[...]
More information about the mesa-dev
mailing list