[Mesa-dev] [PATCH 03/88] glsl: Add initial functions to implement an on-disk cache
Timothy Arceri
timothy.arceri at collabora.com
Mon Sep 26 23:08:05 UTC 2016
On Mon, 2016-09-26 at 08:29 -0600, Brian Paul wrote:
> On 09/23/2016 11:24 PM, Timothy Arceri wrote:
> >
> > From: Carl Worth <cworth at cworth.org>
> >
> > This code provides for an on-disk cache of objects. Objects are
> > stored
> > and retrieved via names that are arbitrary 20-byte sequences,
> > (intended to be SHA-1 hashes of something identifying for the
> > content). The directory used for the cache can be specified by
> > means
> > of environment variables in the following priority order:
> >
> > $MESA_GLSL_CACHE_DIR
> > $XDG_CACHE_HOME/mesa
> > <user-home-directory>/.cache/mesa
> >
> > By default the cache will be limited to a maximum size of 1GB. The
> > environment variable:
> >
> > $MESA_GLSL_CACHE_MAX_SIZE
> >
> > can be set (at the time of GL context creation) to choose some
> > other
> > size. This variable is a number that can optionally be followed by
> > 'K', 'M', or 'G' to select a size in kilobytes, megabytes, or
> > gigabytes. By default, an unadorned value will be interpreted as
> > gigabytes.
> >
> > The cache will be entirely disabled at runtime if the variable
> > MESA_GLSL_CACHE_DISABLE is set at the time of GL context creation.
> >
> > Many thanks to Kristian Høgsberg <krh at bitplanet.net> for the
> > initial
> > implementation of code that led to this patch. In particular, the
> > idea
> > of using an mmapped file, (indexed by a portion of the SHA-1), for
> > the
> > efficent implementation of cache_has_key was entirely his
> > idea. Kristian also provided some very helpful advice in
> > discussions
> > regarding various race conditions to be avoided in this code.
> >
> > Signed-off-by: Timothy Arceri <timothy.arceri at collabora.com>
> > ---
> > configure.ac | 3 +
> > docs/envvars.html | 11 +
> > src/compiler/Makefile.glsl.am | 10 +
> > src/compiler/Makefile.sources | 4 +
> > src/compiler/glsl/cache.c | 709
> > +++++++++++++++++++++++++++++++++++
> > src/compiler/glsl/cache.h | 172 +++++++++
> > src/compiler/glsl/tests/.gitignore | 1 +
> > src/compiler/glsl/tests/cache_test.c | 416 ++++++++++++++++++++
> > 8 files changed, 1326 insertions(+)
> > create mode 100644 src/compiler/glsl/cache.c
> > create mode 100644 src/compiler/glsl/cache.h
> > create mode 100644 src/compiler/glsl/tests/cache_test.c
> >
> > diff --git a/configure.ac b/configure.ac
> > index 0604ad9..7db31e4 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -1305,6 +1305,9 @@ if test "x$with_sha1" = "x"; then
> > fi
> > fi
> > AM_CONDITIONAL([ENABLE_SHADER_CACHE], [test x$enable_shader_cache
> > = xyes])
> > +if test "x$enable_shader_cache" = "xyes"; then
> > + AC_DEFINE([ENABLE_SHADER_CACHE], [1], [Enable shader cache])
> > +fi
> >
> > case "$host_os" in
> > linux*)
> > diff --git a/docs/envvars.html b/docs/envvars.html
> > index cf57ca5..2375145 100644
> > --- a/docs/envvars.html
> > +++ b/docs/envvars.html
> > @@ -112,6 +112,17 @@ glGetString(GL_VERSION) for OpenGL ES.
> > glGetString(GL_SHADING_LANGUAGE_VERSION). Valid values are
> > integers, such as
> > "130". Mesa will not really implement all the features of the
> > given language version
> > if it's higher than what's normally reported. (for developers
> > only)
> > +<li>MESA_GLSL_CACHE_DISABLE - if set, disables the GLSL shader
> > cache
> > +<li>MESA_GLSL_CACHE_MAX_SIZE - if set, determines the maximum size
> > of
> > +the on-disk cache of compiled GLSL programs. Should be set to a
> > number
> > +optionally followed by 'K', 'M', or 'G' to specify a size in
> > +kilobytes, megabytes, or gigabytes. By default, gigabytes will be
> > +assumed. And if unset, a maxium size of 1GB will be used.
> > +<li>MESA_GLSL_CACHE_DIR - if set, determines the directory to be
> > used
> > +for the on-disk cache of compiled GLSL programs. If this variable
> > is
> > +not set, then the cache will be stored in $XDG_CACHE_HOME/.mesa
> > (if
> > +that variable is set), or else within .cache/mesa within the
> > user's
> > +home directory.
> > <li>MESA_GLSL - <a href="shading.html#envvars">shading language
> > compiler options</a>
> > <li>MESA_NO_MINMAX_CACHE - when set, the minmax index cache is
> > globally disabled.
> > </ul>
> > diff --git a/src/compiler/Makefile.glsl.am
> > b/src/compiler/Makefile.glsl.am
> > index b8225cb..80dfb73 100644
> > --- a/src/compiler/Makefile.glsl.am
> > +++ b/src/compiler/Makefile.glsl.am
> > @@ -33,6 +33,7 @@ EXTRA_DIST += glsl/tests glsl/glcpp/tests
> > glsl/README \
> > TESTS += glsl/glcpp/tests/glcpp-test \
> > glsl/glcpp/tests/glcpp-test-cr-lf \
> > glsl/tests/blob-test \
> > + glsl/tests/cache-test \
> > glsl/tests/general-ir-test \
> > glsl/tests/optimization-test \
> > glsl/tests/sampler-types-test \
> > @@ -47,6 +48,7 @@ check_PROGRAMS +=
> > \
> > glsl/glcpp/glcpp \
> > glsl/glsl_test \
> > glsl/tests/blob-test \
> > + glsl/tests/cache-test \
> > glsl/tests/general-ir-test \
> > glsl/tests/sampler-types-test \
> > glsl/tests/uniform-initializer-test
> > @@ -58,6 +60,11 @@ glsl_tests_blob_test_SOURCES =
> > \
> > glsl_tests_blob_test_LDADD = \
> > glsl/libglsl.la
> >
> > +glsl_tests_cache_test_SOURCES = \
> > + glsl/tests/cache_test.c
> > +glsl_tests_cache_test_LDADD = \
> > + glsl/libglsl.la
> > +
> > glsl_tests_general_ir_test_SOURCES = \
> > glsl/tests/builtin_variable_test.cpp \
> > glsl/tests/invalidate_locations_test.cpp \
> > @@ -120,6 +127,9 @@ glsl_libglsl_la_SOURCES =
> > \
> > $(LIBGLSL_GENERATED_FILES) \
> > $(LIBGLSL_FILES)
> >
> > +if ENABLE_SHADER_CACHE
> > +glsl_libglsl_la_SOURCES += $(LIBGLSL_SHADER_CACHE_FILES)
> > +endif
> >
> > glsl_libstandalone_la_SOURCES = \
> > $(GLSL_COMPILER_CXX_FILES)
> > diff --git a/src/compiler/Makefile.sources
> > b/src/compiler/Makefile.sources
> > index f5b4f9c..712b33a 100644
> > --- a/src/compiler/Makefile.sources
> > +++ b/src/compiler/Makefile.sources
> > @@ -136,6 +136,10 @@ LIBGLSL_FILES = \
> > glsl/s_expression.cpp \
> > glsl/s_expression.h
> >
> > +LIBGLSL_SHADER_CACHE_FILES = \
> > + glsl/cache.c \
> > + glsl/cache.h
> > +
> > # glsl_compiler
> >
> > GLSL_COMPILER_CXX_FILES = \
> > diff --git a/src/compiler/glsl/cache.c b/src/compiler/glsl/cache.c
> > new file mode 100644
> > index 0000000..c4c4fb7
> > --- /dev/null
> > +++ b/src/compiler/glsl/cache.c
> > @@ -0,0 +1,709 @@
> > +/*
> > + * Copyright © 2014 Intel Corporation
> > + *
> > + * Permission is hereby granted, free of charge, to any person
> > obtaining a
> > + * copy of this software and associated documentation files (the
> > "Software"),
> > + * to deal in the Software without restriction, including without
> > limitation
> > + * the rights to use, copy, modify, merge, publish, distribute,
> > sublicense,
> > + * and/or sell copies of the Software, and to permit persons to
> > whom the
> > + * Software is furnished to do so, subject to the following
> > conditions:
> > + *
> > + * The above copyright notice and this permission notice
> > (including the next
> > + * paragraph) shall be included in all copies or substantial
> > portions of the
> > + * Software.
> > + *
> > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> > EXPRESS OR
> > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> > MERCHANTABILITY,
> > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
> > EVENT SHALL
> > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> > DAMAGES OR OTHER
> > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> > ARISING
> > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> > OTHER DEALINGS
> > + * IN THE SOFTWARE.
> > + */
> > +
> > +#include <ctype.h>
> > +#include <string.h>
> > +#include <stdlib.h>
> > +#include <stdio.h>
> > +#include <sys/file.h>
> > +#include <sys/types.h>
> > +#include <sys/stat.h>
> > +#include <sys/mman.h>
> > +#include <unistd.h>
> > +#include <fcntl.h>
> > +#include <pwd.h>
> > +#include <errno.h>
> > +#include <dirent.h>
>
> Some of those headers don't exist on Windows. All the
> directory/file-related code will have to be inspected for Unix vs.
> Windows.
>
> -Brian
Hi Brian,
Building shader cache support is a compile time option which I assume
you guys would not enable. For the time being the cache will only work
with the i965 driver so any modifications for enabling the cache on
Windows is not very useful at this point.
Tim
>
>
>
> >
> > +
> > +#include "util/u_atomic.h"
> > +#include "util/mesa-sha1.h"
> > +#include "util/ralloc.h"
> > +#include "main/errors.h"
> > +
>
> [...]
>
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
More information about the mesa-dev
mailing list