[Piglit] [PATCH 1/6] [RFC] arb_gpu_shader_fp64: add initial generated tests (v3)

Ilia Mirkin imirkin at alum.mit.edu
Fri Jul 25 17:14:22 PDT 2014


Something causes the tests to be generated in
generated_tests/ARB_gpu_shader_fp64 instead of arb_gpu_shader_fp64
like all the other tests (and like all.py expects... although it does
seem to find the tests anyways, weird).

Also, on your piglit branch, you've augmented the packDouble case, I
believe incorrectly. With

uniform double expected_doub 2.122e-311
uniform uvec2 given_uval 8519181 1000

It passes for me though. And all the other tests do too... While I
haven't reviewed the logic to generate these in detail, it does seem
to work reasonably. The tests pass on nvc0 with my impl (didn't test
against blob).

For the series, with the above 2 minor issues fixed (ARB -> arb in
generated dir name, fix your latest version of packDouble case),

Acked-by: Ilia Mirkin <imirkin at alum.mit.edu>


On Wed, Jul 23, 2014 at 6:56 PM, Dave Airlie <airlied at gmail.com> wrote:
> From: Dave Airlie <airlied at redhat.com>
>
> This creates a bunch of built-in tests for GLSL 4.00,
> corresponding to the builtin uniform and constant array size tests.
>
> I copied the test generators into new files, as I wasn't really sure how
> to separate the float and double generation, maybe some sort of force double
> mode to generate the correct code, would need advice of original author.
>
> v2: fix dmat issue
>
> v3: emit glsl-4.00 and ARB_gpu_shader_fp64 tests,
>     use a double precision value
>     fix double precision emitting bugs
>     fix generated constant test to work.
>
> Signed-off-by: Dave Airlie <airlied at redhat.com>
> ---
>  generated_tests/CMakeLists.txt                     |   12 +-
>  generated_tests/builtin_function_fp64.py           | 1151 ++++++++++++++++++++
>  generated_tests/gen_builtin_uniform_tests_fp64.py  |  648 +++++++++++
>  .../gen_constant_array_size_tests_fp64.py          |  269 +++++
>  4 files changed, 2079 insertions(+), 1 deletion(-)
>  create mode 100644 generated_tests/builtin_function_fp64.py
>  create mode 100644 generated_tests/gen_builtin_uniform_tests_fp64.py
>  create mode 100644 generated_tests/gen_constant_array_size_tests_fp64.py
>
> diff --git a/generated_tests/CMakeLists.txt b/generated_tests/CMakeLists.txt
> index 87311e9..6d27b3e 100644
> --- a/generated_tests/CMakeLists.txt
> +++ b/generated_tests/CMakeLists.txt
> @@ -83,6 +83,14 @@ piglit_make_generated_tests(
>         gen_outerproduct_tests.py
>         gen_outerproduct_template.mako)
>
> +piglit_make_generated_tests(
> +       builtin_uniform_tests_fp64.list
> +       gen_builtin_uniform_tests_fp64.py
> +       builtin_function_fp64.py)
> +piglit_make_generated_tests(
> +       constant_array_size_tests_fp64.list
> +       gen_constant_array_size_tests_fp64.py
> +       builtin_function_fp64.py)
>
>  # Add a "gen-tests" target that can be used to generate all the
>  # tests without doing any other compilation.
> @@ -103,4 +111,6 @@ add_custom_target(gen-tests ALL
>                 texture_lod_tests.list
>                 shader_bit_encoding_tests.list
>                 uniform-initializer_tests.list
> -               interpolation-qualifier-built-in-variable.list)
> +               interpolation-qualifier-built-in-variable.list
> +               builtin_uniform_tests_fp64.list
> +               constant_array_size_tests_fp64.list)
> diff --git a/generated_tests/builtin_function_fp64.py b/generated_tests/builtin_function_fp64.py
> new file mode 100644
> index 0000000..6b98ec7
> --- /dev/null
> +++ b/generated_tests/builtin_function_fp64.py
> @@ -0,0 +1,1151 @@
> +# coding=utf-8
> +#
> +# Copyright © 2011 Intel Corporation
> +#
> +# Permission is hereby granted, free of charge, to any person obtaining a
> +# copy of this software and associated documentation files (the "Software"),
> +# to deal in the Software without restriction, including without limitation
> +# the rights to use, copy, modify, merge, publish, distribute, sublicense,
> +# and/or sell copies of the Software, and to permit persons to whom the
> +# Software is furnished to do so, subject to the following conditions:
> +#
> +# The above copyright notice and this permission notice (including the next
> +# paragraph) shall be included in all copies or substantial portions of the
> +# Software.
> +#
> +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +# DEALINGS IN THE SOFTWARE.
> +
> +# This source file defines a set of test vectors that can be used to
> +# test GLSL's built-in functions and operators.  It is intended to be
> +# used by Python code that generates Piglit tests.
> +#
> +# The key export is the dictionary test_suite.  It contains an entry
> +# for each possible overload of every pure built-in function and
> +# operator.  By iterating through this dictionary you can find a set
> +# of test vectors for testing nearly every built-in GLSL function.
> +#
> +# The following functions are not included, since they are not pure,
> +# so they can't be tested using simple vectors:
> +# - dFdx()
> +# - dFdy()
> +# - fwidth()
> +# - ftransform()
> +# - Increment and decrement operators
> +#
> +# The following functions are not included, since they need to be
> +# tested in specialized ways:
> +# - modf(): not tested because it has an out parameter
> +# - isnan() and isinf(): not tested because special effort is required
> +#   to create values that cause these functions to return true.
> +#
> +# Also not tested are array subscripting, field/method selection,
> +# swizzling, the function call operator, assignment, and the sequence
> +# operator.
> +
> +import collections
> +import itertools
> +import numpy as np
> +
> +
> +# Floating point types used by Python and numpy
> +DOUBLE_TYPES = (float, np.float64, np.float32)
> +
> +class GlslBuiltinType(object):
> +    """Class representing a GLSL built-in type."""
> +    def __init__(self, name, base_type, num_cols, num_rows,
> +                 version_introduced):
> +        self.__name = name
> +        if base_type is not None:
> +            self.__base_type = base_type
> +        else:
> +            self.__base_type = self
> +        self.__num_cols = num_cols
> +        self.__num_rows = num_rows
> +        self.__version_introduced = version_introduced
> +
> +    @property
> +    def name(self):
> +        """The name of the type, as a string."""
> +        return self.__name
> +
> +    @property
> +    def base_type(self):
> +        """For vectors and matrices, the type of data stored in each
> +        element.  For scalars, equal to self.
> +        """
> +        return self.__base_type
> +
> +    @property
> +    def num_cols(self):
> +        """For matrices, the number of columns.  For vectors and
> +        scalars, 1.
> +        """
> +        return self.__num_cols
> +
> +    @property
> +    def num_rows(self):
> +        """For vectors and matrices, the number of rows.  For scalars,
> +        1.
> +        """
> +        return self.__num_rows
> +
> +    @property
> +    def is_scalar(self):
> +        return self.__num_cols == 1 and self.__num_rows == 1
> +
> +    @property
> +    def is_vector(self):
> +        return self.__num_cols == 1 and self.__num_rows != 1
> +
> +    @property
> +    def is_matrix(self):
> +        return self.__num_cols != 1
> +
> +    @property
> +    def version_introduced(self):
> +        """The earliest version of GLSL that this type appears in (as
> +        a string, e.g. 110).
> +        """
> +        return self.__version_introduced
> +
> +    def __str__(self):
> +        return self.__name
> +
> +    def __repr__(self):
> +        return 'glsl_{0}'.format(self.__name)
> +
> +
> +# Concrete declarations of GlslBuiltinType
> +glsl_bool   = GlslBuiltinType('bool',   None,       1, 1, 110)
> +glsl_bvec2  = GlslBuiltinType('bvec2',  glsl_bool,  1, 2, 110)
> +glsl_bvec3  = GlslBuiltinType('bvec3',  glsl_bool,  1, 3, 110)
> +glsl_bvec4  = GlslBuiltinType('bvec4',  glsl_bool,  1, 4, 110)
> +glsl_double = GlslBuiltinType('double', None,       1, 1, 400)
> +glsl_dvec2  = GlslBuiltinType('dvec2', glsl_double,  1, 2, 400)
> +glsl_dvec3  = GlslBuiltinType('dvec3', glsl_double,  1, 3, 400)
> +glsl_dvec4  = GlslBuiltinType('dvec4', glsl_double,  1, 4, 400)
> +glsl_dmat2   = GlslBuiltinType('dmat2', glsl_double, 2, 2, 400)
> +glsl_dmat3   = GlslBuiltinType('dmat3', glsl_double, 3, 3, 400)
> +glsl_dmat4   = GlslBuiltinType('dmat4', glsl_double, 4, 4, 400)
> +glsl_dmat2x2 = glsl_dmat2
> +glsl_dmat3x2 = GlslBuiltinType('dmat3x2', glsl_double, 3, 2, 400)
> +glsl_dmat4x2 = GlslBuiltinType('dmat4x2', glsl_double, 4, 2, 400)
> +glsl_dmat2x3 = GlslBuiltinType('dmat2x3', glsl_double, 2, 3, 400)
> +glsl_dmat3x3 = glsl_dmat3
> +glsl_dmat4x3 = GlslBuiltinType('dmat4x3', glsl_double, 4, 3, 400)
> +glsl_dmat2x4 = GlslBuiltinType('dmat2x4', glsl_double, 2, 4, 400)
> +glsl_dmat3x4 = GlslBuiltinType('dmat3x4', glsl_double, 3, 4, 400)
> +glsl_dmat4x4 = glsl_dmat4
> +
> +
> +# Named tuple representing the signature of a single overload of a
> +# built-in GLSL function or operator:
> +# - name is a name suitable for use in test filenames.  For functions,
> +#   this is the name of the function.  For operators, it is a short
> +#   description of the operator, beginning with "op", e.g. "op-plus".
> +# - template is a Python format string that can be used to construct
> +#   GLSL code that invokes the function or operator.
> +# - version_introduced earliest version of GLSL the test applies to
> +#   (as a string, e.g. 110).
> +# - rettype is the return type of the function or operator (as a
> +#   GlslBuiltinType).
> +# - argtypes is a tuple containing the types of each parameter (as
> +#   GlslBuiltinTypes).
> +#
> +# For example, the function
> +#
> +#   vec3 step(float edge, vec3 x)
> +#
> +# has a signature of
> +#
> +# Signature(name='step', template='step({0}, {1})',
> +#           version_introduced=110, rettype='vec3',
> +#           argtypes=('float', 'vec3'))
> +Signature = collections.namedtuple(
> +    'Signature',
> +    ('name', 'template', 'version_introduced', 'extension', 'rettype', 'argtypes'))
> +
> +
> +# Named tuple representing a single piece of test data for testing a
> +# built-in GLSL function:
> +# - arguments is a tuple containing the arguments to apply to the
> +#   function.  Each argument is of a type native to numpy (e.g.
> +#   numpy.float32 or numpy.ndarray)
> +# - result is the value the function is expected to return.  It is
> +#   also of a type native to numpy.
> +# - tolerance is a float64 representing how much deviation from the
> +#   result we expect, considering the floating point precision
> +#   requirements of GLSL and OpenGL.  The value may be zero for test
> +#   vectors involving booleans and integers.  If result is a vector or
> +#   matrix, tolerance should be interpreted as the maximum permissible
> +#   RMS error (as would be computed by the distance() function).
> +TestVector = collections.namedtuple(
> +    'TestVector', ('arguments', 'result', 'tolerance'))
> +
> +
> +def glsl_type_of(value):
> +    """Return the GLSL type corresponding to the given native numpy
> +    value, as a GlslBuiltinType.
> +    """
> +
> +    if isinstance(value, DOUBLE_TYPES):
> +        return glsl_double
> +    elif isinstance(value, (bool, np.bool_)):
> +        return glsl_bool
> +    else:
> +        if len(value.shape) == 1:
> +            # Vector
> +            vector_length = value.shape[0]
> +            assert 2 <= vector_length <= 4
> +            if value.dtype in DOUBLE_TYPES:
> +                return (glsl_dvec2, glsl_dvec3, glsl_dvec4)[vector_length - 2]
> +            elif value.dtype == bool:
> +                return (glsl_bvec2, glsl_bvec3, glsl_bvec4)[vector_length - 2]
> +            else:
> +                raise Exception(
> +                    'Unexpected vector base type {0}'.format(value.dtype))
> +        else:
> +            # Matrix
> +            assert value.dtype in DOUBLE_TYPES
> +            assert len(value.shape) == 2
> +            matrix_rows = value.shape[0]
> +            assert 2 <= matrix_rows <= 4
> +            matrix_columns = value.shape[1]
> +            assert 2 <= matrix_columns <= 4
> +            matrix_types = ((glsl_dmat2x2, glsl_dmat2x3, glsl_dmat2x4),
> +                            (glsl_dmat3x2, glsl_dmat3x3, glsl_dmat3x4),
> +                            (glsl_dmat4x2, glsl_dmat4x3, glsl_dmat4x4))
> +            return matrix_types[matrix_columns - 2][matrix_rows - 2]
> +
> +
> +def column_major_values(value):
> +    """Given a native numpy value, return a list of the scalar values
> +    comprising it, in column-major order."""
> +    if isinstance(value, np.ndarray):
> +        return list(np.reshape(value, -1, 'F'))
> +    else:
> +        return [value]
> +
> +
> +def glsl_constant(value):
> +    """Given a native numpy value, return GLSL code that constructs
> +    it."""
> +    column_major = np.reshape(np.array(value), -1, 'F')
> +    if column_major.dtype == bool:
> +        values = ['true' if x else 'false' for x in column_major]
> +    else:
> +        values = ['{0}lf'.format(repr(x)) for x in column_major]
> +    if len(column_major) == 1:
> +        return values[0]
> +    else:
> +        return '{0}({1})'.format(glsl_type_of(value), ', '.join(values))
> +
> +
> +def round_to_32_bits(value):
> +    """If value is a floating point type, round it down to 32 bits.
> +    Otherwise return it unchanged.
> +    """
> +    if isinstance(value, float):
> +        return np.float32(value)
> +    elif isinstance(value, np.ndarray) and value.dtype == np.float64:
> +        return np.array(value, dtype=np.float32)
> +    else:
> +        return value
> +
> +
> +def extend_to_64_bits(value):
> +    """If value is a floating point type, extend it to 64 bits.
> +    Otherwise return it unchanged.
> +    """
> +    if isinstance(value, np.float32):
> +        return np.float64(value)
> +    elif isinstance(value, np.ndarray) and value.dtype == np.float32:
> +        return np.array(value, dtype=np.float64)
> +    else:
> +        return value
> +
> +
> +# Dictionary containing the test vectors.  Each entry in the
> +# dictionary represents a single overload of a single built-in
> +# function.  Its key is a Signature tuple, and its value is a list of
> +# TestVector tuples.
> +#
> +# Note: the dictionary is initialized to {} here, but it is filled
> +# with test vectors by code later in this file.
> +test_suite = {}
> +
> +
> +# Implementation
> +# ==============
> +#
> +# The functions below shouldn't be necessary to call from outside this
> +# file.  They exist solely to populate test_suite with test vectors.
> +
> +# Functions that simulate GLSL built-in functions (in the cases where
> +# the GLSL built-in functions have no python or numpy equivalent, or
> +# in cases where there is a behavioral difference).  These functions
> +# return None if the behavior of the GLSL built-in is undefined for
> +# the given set of inputs.
> +def _multiply(x, y):
> +    x_type = glsl_type_of(x)
> +    y_type = glsl_type_of(y)
> +
> +    if x_type.is_vector and y_type.is_vector:
> +        # vector * vector is done componentwise.
> +        return x * y
> +    else:
> +        # All other cases are standard linear algebraic
> +        # multiplication, which numpy calls "dot".
> +        return np.dot(x, y)
> +
> +
> +def _divide(x, y):
> +    if any(y_element == 0 for y_element in column_major_values(y)):
> +        # Division by zero is undefined.
> +        return None
> +    return x / y
> +
> +
> +def _modulus(x, y):
> +    if any(x_element < 0 for x_element in column_major_values(x)):
> +        # Modulus operation with a negative first operand is
> +        # undefined.
> +        return None
> +    if any(y_element <= 0 for y_element in column_major_values(y)):
> +        # Modulus operation with a negative or zero second operand is
> +        # undefined.
> +        return None
> +    return x % y
> +
> +
> +def _lshift(x, y):
> +    if not all(0 <= y_element < 32 for y_element in column_major_values(y)):
> +        # Shifts by less than 0 or more than the number of bits in the
> +        # type being shifted are undefined.
> +        return None
> +    # When the arguments to << don't have the same signedness, numpy
> +    # likes to promote them to int64.  To avoid this, convert y to be
> +    # the same type as x.
> +    y_orig = y
> +    result = x << y
> +
> +    # Shifting should always produce a result with the same base type
> +    # as the left argument.
> +    assert glsl_type_of(result).base_type == glsl_type_of(x).base_type
> +
> +    return result
> +
> +
> +def _rshift(x, y):
> +    if not all(0 <= y_element < 32 for y_element in column_major_values(y)):
> +        # Shifts by less than 0 or more than the number of bits in the
> +        # type being shifted are undefined.
> +        return None
> +    # When the arguments to >> don't have the same signedness, numpy
> +    # likes to promote them to int64.  To avoid this, convert y to be
> +    # the same type as x.
> +    y_orig = y
> +    result = x >> y
> +
> +    # Shifting should always produce a result with the same base type
> +    # as the left argument.
> +    assert glsl_type_of(result).base_type == glsl_type_of(x).base_type
> +
> +    return result
> +
> +
> +def _equal(x, y):
> +    return all(column_major_values(x == y))
> +
> +
> +def _not_equal(x, y):
> +    return not _equal(x, y)
> +
> +
> +def _arctan2(y, x):
> +    if x == y == 0.0:
> +        return None
> +    return np.arctan2(y, x)
> +
> +
> +def _pow(x, y):
> +    if x < 0.0:
> +        return None
> +    if x == 0.0 and y <= 0.0:
> +        return None
> +    return np.power(x, y)
> +
> +
> +def _exp2(x):
> +    # exp2() is not available in versions of numpy < 1.3.0 so we
> +    # emulate it with power().
> +    return np.power(2, x)
> +
> +
> +def _trunc(x):
> +    # trunc() rounds toward zero.  It is not available in version
> +    # 1.2.1 of numpy so we emulate it with floor(), sign(), and abs().
> +    return np.sign(x) * np.floor(np.abs(x))
> +
> +
> +def _clamp(x, minVal, maxVal):
> +    if minVal > maxVal:
> +        return None
> +    return min(max(x, minVal), maxVal)
> +
> +
> +# Inefficient, but obvious
> +def _mid3(x, y, z):
> +    return np.sort([x, y, z])[1]
> +
> +def _smoothstep(edge0, edge1, x):
> +    if edge0 >= edge1:
> +        return None
> +    t = _clamp((x-edge0)/(edge1-edge0), 0.0, 1.0)
> +    return t*t*(3.0-2.0*t)
> +
> +
> +def _normalize(x):
> +    return x/np.linalg.norm(x)
> +
> +
> +def _faceforward(N, I, Nref):
> +    if np.dot(Nref, I) < 0.0:
> +        return N
> +    else:
> +        return -N
> +
> +
> +def _reflect(I, N):
> +    return I-2*np.dot(N, I)*N
> +
> +
> +def _refract(I, N, eta):
> +    k = 1.0-eta*eta*(1.0-np.dot(N, I)*np.dot(N, I))
> +    if k < 0.0:
> +        return I*0.0
> +    else:
> +        return eta*I-(eta*np.dot(N, I)+np.sqrt(k))*N
> +
> +
> +def _argument_types_match(arguments, argument_indices_to_match):
> +    """Return True if all of the arguments indexed by
> +    argument_indices_to_match have the same GLSL type.
> +    """
> +    types = [glsl_type_of(arguments[i]) for i in argument_indices_to_match]
> +    return all(x == types[0] for x in types)
> +
> +
> +def _strict_tolerance(arguments, result):
> +    """Compute tolerance using a strict interpretation of the GLSL and
> +    OpenGL standards.
> +
> +    From the GLSL 1.20 spec (4.1.4 "Floats"):
> +
> +      "As an input value to one of the processing units, a
> +      floating-point variable is expected to match the IEEE single
> +      precision floating-point definition for precision and dynamic
> +      range.  It is not required that the precision of internal
> +      processing match the IEEE floating-point specification for
> +      floating-point operations, but the guidelines for precision
> +      established by the OpenGL 1.4 specification must be met."
> +
> +    From the OpenGL 1.4 spec (2.1.1 "Floating-Point Computation"):
> +
> +      "We require simply that numbers' floating-point parts contain
> +      enough bits ... so that individual results of floating-point
> +      operations are accurate to about 1 part in 10^5."
> +
> +    A harsh interpretation of the above is that (a) no precision is
> +    lost in moving numbers into or out of the GPU, and (b) any
> +    built-in function constitutes a single operation, so therefore the
> +    error in applying any built-in function should be off by no more
> +    than 1e-5 times its theoretically correct value.
> +
> +    This is not the only possible interpretation, however.  Certain
> +    built-in functions, such as the cross product, are computed by a
> +    formula consisting of many elementary multiplications and
> +    additions, in which a large amount of cancellation sometimes
> +    occurs.  It's possible that these rules are meant to apply to
> +    those elementary multiplications and additions, and not the full
> +    built-in function. Other built-in functions, such as the trig
> +    functions, are typically implemented by a series approximation, in
> +    which 1 part in 10^5 accuracy seems like overkill.  See below for
> +    the tolerance computation we use on these other functions.
> +    """
> +    return 1e-5 * np.linalg.norm(result)
> +
> +
> +def _trig_tolerance(arguments, result):
> +    """Compute a more lenient tolerance bound for trig functions.
> +
> +    The GLSL and OpenGL specs don't provide any guidance as to the
> +    required accuracy of trig functions (other than the "1 part in
> +    10^5" general accuracy requirement, which seems like overkill for
> +    trig functions.
> +
> +    So the tolerance here is rather arbitrarily chosen to be either 1
> +    part in 10^3 or 10^-4, whichever is larger.
> +    """
> +    return max(1e-4, 1e-3 * np.linalg.norm(result))
> +
> +
> +def _cross_product_tolerance(arguments, result):
> +    """Compute a more lenient tolerance bound for cross product.
> +
> +    Since the computation of a cross product may involve a large
> +    amount of cancellation, an error tolerance of 1 part in 10^5
> +    (referred to the magnitude of the result vector) is overly tight.
> +
> +    So instead we allow the error to be 1 part in 10^5 referred to the
> +    product of the magnitudes of the arguments.
> +    """
> +    assert len(arguments) == 2
> +    return 1e-5 * np.linalg.norm(arguments[0]) * np.linalg.norm(arguments[1])
> +
> +
> +def _simulate_function(test_inputs, python_equivalent, tolerance_function):
> +    """Construct test vectors by simulating a GLSL function on a list
> +    of possible inputs, and return a list of test vectors.
> +
> +    test_inputs is a list of possible input sequences, each of which
> +    represents a set of arguments that should be applied to the
> +    function.
> +
> +    python_equivalent is the function to simulate--it should return
> +    None if the GLSL function returns undefined results for the given
> +    set of inputs, otherwise it should return the expected result.
> +    Input sequences for which python_equivalent returns None are
> +    ignored.
> +
> +    The function is simulated using 64 bit floats for maximum possible
> +    accuracy, but the output is rounded to 32 bits since that is the
> +    data type that we expect to get back form OpenGL.
> +
> +    tolerance_function is the function to call to compute the
> +    tolerance.  It should take the set of arguments and the expected
> +    result as its parameters.  It is only used for functions that
> +    return floating point values.
> +    """
> +    test_vectors = []
> +    for inputs in test_inputs:
> +        expected_output = python_equivalent(*[extend_to_64_bits(x) for x in inputs])
> +        if expected_output is not None:
> +            tolerance = np.float64(
> +                tolerance_function(inputs, expected_output))
> +            test_vectors.append(TestVector(inputs, expected_output, tolerance))
> +    return test_vectors
> +
> +
> +def _vectorize_test_vectors(test_vectors, scalar_arg_indices, vector_length):
> +    """Build a new set of test vectors by combining elements of
> +    test_vectors into vectors of length vector_length. For example,
> +    vectorizing the test vectors
> +
> +    [TestVector((10, 20), 30, tolerance), TestVector((11, 20), 31, tolerance)]
> +
> +    into vectors of length 2 would produce the result:
> +
> +    [TestVector((vec2(10, 11), vec2(20, 20)), vec2(30, 31), new_tolerance)].
> +
> +    Tolerances are combined in root-sum-square fashion.
> +
> +    scalar_arg_indices is a sequence of argument indices which should
> +    not be vectorized.  So, if scalar_arg_indices is [1] in the above
> +    example, the result would be:
> +
> +    [TestVector((vec2(10, 11), 20), vec2(30, 31), new_tolerance)].
> +    """
> +    def make_groups(test_vectors):
> +        """Group test vectors according to the values passed to the
> +        arguments that should not be vectorized.
> +        """
> +        groups = {}
> +        for tv in test_vectors:
> +            key = tuple(tv.arguments[i] for i in scalar_arg_indices)
> +            if key not in groups:
> +                groups[key] = []
> +            groups[key].append(tv)
> +        return groups
> +
> +    def partition_vectors(test_vectors, partition_size):
> +        """Partition test_vectors into lists of length partition_size.
> +        If partition_size does not evenly divide the number of test
> +        vectors, wrap around as necessary to ensure that every input
> +        test vector is included.
> +        """
> +        for i in xrange(0, len(test_vectors), partition_size):
> +            partition = []
> +            for j in xrange(partition_size):
> +                partition.append(test_vectors[(i + j) % len(test_vectors)])
> +            yield partition
> +
> +    def merge_vectors(test_vectors):
> +        """Merge the given set of test vectors (whose arguments and
> +        result are scalars) into a single test vector whose arguments
> +        and result are vectors.  For argument indices in
> +        scalar_arg_indices, leave the argument as a scalar.
> +        """
> +        arity = len(test_vectors[0].arguments)
> +        arguments = []
> +        for j in xrange(arity):
> +            if j in scalar_arg_indices:
> +                arguments.append(test_vectors[0].arguments[j])
> +            else:
> +                arguments.append(
> +                    np.array([tv.arguments[j] for tv in test_vectors]))
> +        result = np.array([tv.result for tv in test_vectors])
> +        tolerance = np.float64(
> +            np.linalg.norm([tv.tolerance for tv in test_vectors]))
> +        return TestVector(arguments, result, tolerance)
> +    vectorized_test_vectors = []
> +    groups = make_groups(test_vectors)
> +    for key in sorted(groups.keys()):
> +        test_vectors = groups[key]
> +        vectorized_test_vectors.extend(
> +            merge_vectors(partition)
> +            for partition in partition_vectors(test_vectors, vector_length))
> +    return vectorized_test_vectors
> +
> +
> +def _store_test_vector(test_suite_dict, name, glsl_version, extension, test_vector,
> +                       template=None):
> +    """Store a test vector in the appropriate place in
> +    test_suite_dict.  The dictionary key (which is a Signature tuple)
> +    is generated by consulting the argument and return types of the
> +    test vector, and combining them with name and glsl_version.
> +
> +    glsl_version is adjusted if necessary to reflect when the argument
> +    and return types were introduced into GLSL.
> +
> +    If template is supplied, it is used insted as the template for the
> +    Signature objects generated.
> +    """
> +    if template is None:
> +        arg_indices = xrange(len(test_vector.arguments))
> +        template = '{0}({1})'.format(
> +            name, ', '.join('{{{0}}}'.format(i) for i in arg_indices))
> +    rettype = glsl_type_of(test_vector.result)
> +    argtypes = tuple(glsl_type_of(arg) for arg in test_vector.arguments)
> +    adjusted_glsl_version = glsl_version
> +
> +    signature = Signature(
> +        name, template, adjusted_glsl_version, extension, rettype, argtypes)
> +    if signature not in test_suite_dict:
> +        test_suite_dict[signature] = []
> +    test_suite_dict[signature].append(test_vector)
> +
> +
> +def _store_test_vectors(test_suite_dict, name, glsl_version, extension,
> +                        test_vectors, template=None):
> +    """Store multiple test vectors in the appropriate places in
> +    test_suite_dict.
> +
> +    If template is supplied, it is used insted as the template for the
> +    Signature objects generated.
> +    """
> +    for test_vector in test_vectors:
> +        _store_test_vector(test_suite_dict, name, glsl_version, extension,
> +                           test_vector, template=template)
> +
> +
> +def make_arguments(input_generators):
> +    """Construct a list of tuples of input arguments to test.
> +
> +    input_generators is a list, the ith element of which is a sequence
> +    of values that are suitable for use as the ith argument of the
> +    function under test.
> +
> +    Output is a list, each element of which is a tuple of arguments to
> +    be passed to the function under test.  These values are produced
> +    by taking the cartesian product of the input sequences.
> +
> +    """
> +    return list(itertools.product(*input_generators))
> +
> +
> +def _make_componentwise_test_vectors(test_suite_dict):
> +    """Add test vectors to test_suite_dict for GLSL built-in
> +    functions that operate on vectors in componentwise fashion.
> +    Examples include sin(), cos(), min(), max(), and clamp().
> +    """
> +    # Make sure atan(x) and atan(x,y) don't misbehave for very large
> +    # or very small input values.
> +    atan_inputs = [0.0]
> +    for exponent in (-10, -1, 0, 1, 10):
> +        atan_inputs.append(pow(10.0, exponent))
> +        atan_inputs.append(-pow(10.0, exponent))
> +    # Make a similar set of inputs for acosh(), except don't use any
> +    # values < 1, since acosh() is only defined for x >= 1.
> +    acosh_inputs = [1.0 + x for x in atan_inputs if x >= 0]
> +    ints = [np.int32(x) for x in [-5, -2, -1, 0, 1, 2, 5]]
> +    uints = [np.uint32(x) for x in [0, 1, 2, 5, 34]]
> +    bools = [True, False]
> +
> +    def f(name, arity, python_equivalent,
> +          alternate_scalar_arg_indices, test_inputs,
> +          tolerance_function=_strict_tolerance):
> +
> +        """Create test vectors for the function with the given name
> +        and arity, which was introduced in the given glsl_version.
> +
> +        python_equivalent is a Python function which operates on scalars,
> +        and simulates the GLSL function.  This function should return None
> +        in any case where the output of the GLSL function is undefined.
> +
> +        If alternate_scalar_arg_indices is not None, also create test
> +        vectors for an alternate vectorized version of the function,
> +        in which some arguments are scalars.
> +        alternate_scalar_arg_indices is a sequence of the indices of
> +        the arguments which are scalars.
> +
> +        test_inputs is a list, the ith element of which is a list of
> +        values that are suitable for use as the ith argument of the
> +        function.
> +
> +        If tolerance_function is supplied, it is a function which
> +        should be used to compute the tolerance for the test vectors.
> +        Otherwise, _strict_tolerance is used.
> +        """
> +        scalar_test_vectors = _simulate_function(
> +            make_arguments(test_inputs), python_equivalent, tolerance_function)
> +        _store_test_vectors(
> +            test_suite_dict, name, 400, None, scalar_test_vectors)
> +        _store_test_vectors(
> +            test_suite_dict, name, 150, "ARB_gpu_shader_fp64", scalar_test_vectors)
> +        if alternate_scalar_arg_indices is None:
> +            scalar_arg_indices_list = [()]
> +        else:
> +            scalar_arg_indices_list = [(), alternate_scalar_arg_indices]
> +        for scalar_arg_indices in scalar_arg_indices_list:
> +            for vector_length in (2, 3, 4):
> +                _store_test_vectors(
> +                    test_suite_dict, name, 400, None,
> +                    _vectorize_test_vectors(
> +                        scalar_test_vectors, scalar_arg_indices,
> +                        vector_length))
> +                _store_test_vectors(
> +                    test_suite_dict, name, 150, "ARB_gpu_shader_fp64",
> +                    _vectorize_test_vectors(
> +                        scalar_test_vectors, scalar_arg_indices,
> +                        vector_length))
> +
> +    f('sqrt', 1, np.sqrt, None, [np.linspace(0.0, 2.0, 4)])
> +    f('inversesqrt', 1, lambda x: 1.0/np.sqrt(x), None,
> +      [np.linspace(0.1, 2.0, 4)])
> +    f('abs', 1, np.abs, None, [np.linspace(-1.5, 1.5, 5)])
> +    f('sign', 1, np.sign, None, [np.linspace(-1.5, 1.5, 5)])
> +    f('floor', 1, np.floor, None, [np.linspace(-2.0, 2.0, 4)])
> +    f('trunc', 1, _trunc, None, [np.linspace(-2.0, 2.0, 8)])
> +
> +    # Note: the direction of rounding used by round() is not specified
> +    # for half-integer values, so we test it over a range that doesn't
> +    # include exact half-integer values.  roundEven() is required to
> +    # round half-integer values to the nearest even integer, so we
> +    # test it over a range that does include exact half-integer
> +    # values.  In both cases, we can use numpy's round() function,
> +    # because it rounds half-integer values to even, and all other
> +    # values to nearest.
> +    f('round', 1, np.round, None, [np.linspace(-2.0, 2.0, 8)])
> +    f('roundEven', 1, np.round, None, [np.linspace(-2.0, 2.0, 25)])
> +
> +    f('ceil', 1, np.ceil, None, [np.linspace(-2.0, 2.0, 4)])
> +    f('fract', 1, lambda x: x-np.floor(x), None,
> +      [np.linspace(-2.0, 2.0, 4)])
> +    f('mod', 2, lambda x, y: x-y*np.floor(x/y), [1],
> +      [np.linspace(-1.9, 1.9, 4), np.linspace(-2.0, 2.0, 4)])
> +    f('min', 2, min, [1],
> +      [np.linspace(-2.0, 2.0, 4), np.linspace(-2.0, 2.0, 4)])
> +    f('max', 2, max, [1],
> +      [np.linspace(-2.0, 2.0, 4), np.linspace(-2.0, 2.0, 4)])
> +    f('clamp', 3, _clamp, [1, 2], [np.linspace(-2.0, 2.0, 4),
> +      np.linspace(-1.5, 1.5, 3), np.linspace(-1.5, 1.5, 3)])
> +    f('mix', 3, lambda x, y, a: x*(1-a)+y*a, [2],
> +      [np.linspace(-2.0, 2.0, 2), np.linspace(-3.0, 3.0, 2),
> +       np.linspace(0.0, 1.0, 4)])
> +    f('mix', 3, lambda x, y, a: y if a else x, None,
> +      [np.linspace(-2.0, 2.0, 2), np.linspace(-3.0, 3.0, 2), bools])
> +    f('step', 2, lambda edge, x: 0.0 if x < edge else 1.0, [0],
> +      [np.linspace(-2.0, 2.0, 4), np.linspace(-2.0, 2.0, 4)])
> +    f('smoothstep', 3, _smoothstep, [0, 1],
> +      [np.linspace(-1.9, 1.9, 4), np.linspace(-1.9, 1.9, 4),
> +       np.linspace(-2.0, 2.0, 4)])
> +
> +_make_componentwise_test_vectors(test_suite)
> +
> +
> +def _make_vector_relational_test_vectors(test_suite_dict):
> +    """Add test vectors to test_suite_dict for GLSL built-in functions
> +    that operate on vectors of floats, ints, or bools, but not on
> +    single floats, ints, or bools.  Examples include lessThan(),
> +    equal(), and not().
> +    """
> +    _default_inputs = {
> +        'v': np.linspace(-1.5, 1.5, 4),
> +        'b': np.array([False, True])
> +        }
> +
> +    def f(name, arity, python_equivalent, arg_types,
> +          tolerance_function=_strict_tolerance,
> +          extension=None):
> +        """Make test vectors for the function with the given name and
> +        arity, which was introduced in the given glsl_version.
> +
> +        python_equivalent is a Python function which operates on scalars,
> +        and simulates the GLSL function.
> +
> +        arg_types is a string containing 'v' if the function supports
> +        standard "vec" inputs, 'i' if it supports "ivec" inputs, and 'b'
> +        if it supports "bvec" inputs.  The output type of the function is
> +        assumed to be the same as its input type.
> +
> +        If tolerance_function is supplied, it is a function which
> +        should be used to compute the tolerance for the test vectors.
> +        Otherwise, _strict_tolerance is used.
> +        """
> +        for arg_type in arg_types:
> +            test_inputs = [_default_inputs[arg_type]]*arity
> +            scalar_test_vectors = _simulate_function(
> +                make_arguments(test_inputs), python_equivalent,
> +                tolerance_function)
> +            for vector_length in (2, 3, 4):
> +                _store_test_vectors(
> +                    test_suite_dict, name, 400, None,
> +                    _vectorize_test_vectors(
> +                        scalar_test_vectors, (), vector_length))
> +                _store_test_vectors(
> +                    test_suite_dict, name, 150, "ARB_gpu_shader_fp64",
> +                    _vectorize_test_vectors(
> +                        scalar_test_vectors, (), vector_length))
> +
> +    f('lessThan', 2, lambda x, y: x < y, 'v')
> +    f('lessThanEqual', 2, lambda x, y: x <= y, 'v')
> +    f('greaterThan', 2, lambda x, y: x > y, 'v')
> +    f('greaterThanEqual', 2, lambda x, y: x >= y, 'v')
> +    f('equal', 2, lambda x, y: x == y, 'v')
> +    f('notEqual', 2, lambda x, y: x != y, 'v')
> +
> +_make_vector_relational_test_vectors(test_suite)
> +
> +
> +def _make_vector_or_matrix_test_vectors(test_suite_dict):
> +    """Add test vectors to test_suite_dict for GLSL built-in functions
> +    that operate on vectors/matrices as a whole.  Examples include
> +    length(), dot(), cross(), normalize(), and refract().
> +    """
> +    def match_args(*indices):
> +        """Return a function that determines whether the type of the
> +        arguments at the given indices match.
> +
> +        For example:
> +
> +            match(1, 3)
> +
> +        is equivalent to:
> +
> +            lambda a, b, c, d: glsl_type_of(b) == glsl_type_of(d)
> +        """
> +        return lambda *args: _argument_types_match(args, indices)
> +
> +    def match_simple_binop(x, y):
> +        """Detemine whether the type of the arguments is compatible
> +        for a simple binary operator (such as '+').
> +
> +        Arguments are compatible if one is a scalar and the other is a
> +        vector/matrix with the same base type, or if they are the same
> +        type.
> +        """
> +        x_type = glsl_type_of(x)
> +        y_type = glsl_type_of(y)
> +        if x_type.base_type != y_type.base_type:
> +            return False
> +        if x_type.is_scalar or y_type.is_scalar:
> +            return True
> +        return x_type == y_type
> +
> +    def match_multiply(x, y):
> +        """Determine whether the type of the arguments is compatible
> +        for multiply.
> +
> +        Arguments are compatible if they are scalars, vectors, or
> +        matrices with the same base type, and the vector/matrix sizes
> +        are properly matched.
> +        """
> +        x_type = glsl_type_of(x)
> +        y_type = glsl_type_of(y)
> +        if x_type.base_type != y_type.base_type:
> +            return False
> +        if x_type.is_scalar or y_type.is_scalar:
> +            return True
> +        if x_type.is_vector and y_type.is_matrix:
> +            # When multiplying vector * matrix, the vector is
> +            # transposed to a row vector.  So its row count must match
> +            # the row count of the matrix.
> +            return x_type.num_rows == y_type.num_rows
> +        elif x_type.is_vector:
> +            assert y_type.is_vector
> +            # When multiplying vector * vector, the multiplication is
> +            # done componentwise, so the types must match exactly.
> +            return x_type == y_type
> +        else:
> +            assert x_type.is_matrix
> +            # When multiplying matrix * matrix or matrix * vector, a
> +            # standard linear algebraic multiply is used, so x's
> +            # column count must match y's row count.
> +            return x_type.num_cols == y_type.num_rows
> +
> +    def match_shift(x, y):
> +        """Determine whether the type of the arguments is compatible
> +        for shift operations.
> +
> +        Arguments are compatible if they are the same length or the
> +        first one is a vector and the second is a scalar.  Their base
> +        types need not be the same, but they both must be integral.
> +        """
> +        x_type = glsl_type_of(x)
> +        y_type = glsl_type_of(y)
> +        if x_type.base_type not in (glsl_int, glsl_uint):
> +            return False
> +        if y_type.base_type not in (glsl_int, glsl_uint):
> +            return False
> +        if y_type.is_scalar:
> +            return True
> +        assert not x_type.is_matrix
> +        assert not y_type.is_matrix
> +        return x_type.num_rows == y_type.num_rows
> +
> +    nz_doubles = [ -1.333333333333333259, 0.85]
> +    doubles = [0.0] + nz_doubles
> +    dvecs = [
> +        np.array([-0.10, -1.20]),
> +        np.array([-0.42, 0.48]),
> +        np.array([-1.333333333333333259, -0.85, -0.94]),
> +        np.array([1.67, 0.66, 1.87]),
> +        np.array([-1.65, 1.33, 1.93, 0.76]),
> +        np.array([0.80, -0.15, -0.51, 0.0])
> +        ]
> +    nz_doubles_dvecs = nz_doubles + dvecs
> +    dvec3s = [
> +        np.array([-0.03, -0.85, -0.94]),
> +        np.array([ -1.333333333333333259, 0.66, 1.87]),
> +        ]
> +
> +    norm_doubles_dvecs = [_normalize(x) for x in nz_doubles_dvecs]
> +    squaremats = [
> +        np.array([[ 1.60,  0.76],
> +                  [ 1.53, -1.00]]),  # mat2
> +        np.array([[-0.13, -0.87],
> +                  [-1.40,  1.40]]),  # mat2
> +        np.array([[-1.11,  1.67, -0.41],
> +                  [ 0.13,  1.09, -0.02],
> +                  [ 0.56,  0.95,  0.24]]),  # mat3
> +        np.array([[-1.69, -0.46, -0.18],
> +                  [-1.09,  1.75,  2.00],
> +                  [-1.53, -0.70, -1.47]]),  # mat3
> +        np.array([[-1.00, -0.55, -1.08,  1.79],
> +                  [ 1.77,  0.62,  0.48, -1.35],
> +                  [ 0.09, -0.71, -1.39, -1.21],
> +                  [-0.91, -1.82, -1.43,  0.72]]),  # mat4
> +        np.array([[ 0.06,  1.31,  1.52, -1.96],
> +                  [ 1.60, -0.32,  0.51, -1.84],
> +                  [ 1.25,  0.45,  1.90, -0.72],
> +                  [-0.16,  0.45, -0.88,  0.39]]),  # mat4
> +        ]
> +    mats = squaremats + [
> +        np.array([[ 0.09,  1.30,  1.25],
> +                  [-1.19,  0.08,  1.08]]),  # mat3x2
> +        np.array([[-0.36, -1.08, -0.60],
> +                  [-0.53,  0.88, -1.79]]),  # mat3x2
> +        np.array([[-0.46,  1.94],
> +                  [-0.45, -0.75],
> +                  [ 1.03, -0.50]]),  # mat2x3
> +        np.array([[ 1.38, -1.08],
> +                  [-1.27,  1.83],
> +                  [ 1.00, -0.74]]),  # mat2x3
> +        np.array([[ 1.81, -0.87,  0.81,  0.65],
> +                  [-1.16, -1.52,  0.25, -1.51]]),  # mat4x2
> +        np.array([[ 1.93, -1.63,  0.29,  1.60],
> +                  [ 0.49,  0.27,  0.14,  0.94]]),  # mat4x2
> +        np.array([[ 0.16, -1.69],
> +                  [-0.80,  0.59],
> +                  [-1.74, -1.43],
> +                  [-0.02, -1.21]]),  # mat2x4
> +        np.array([[-1.02,  0.74],
> +                  [-1.64, -0.13],
> +                  [-1.59,  0.47],
> +                  [ 0.30,  1.13]]),  # mat2x4
> +        np.array([[-0.27, -1.38, -1.41, -0.12],
> +                  [-0.17, -0.56,  1.47,  1.86],
> +                  [-1.85, -1.29,  1.77,  0.01]]),  # mat4x3
> +        np.array([[-0.47, -0.15,  1.97, -1.05],
> +                  [-0.20,  0.53, -1.82, -1.41],
> +                  [-1.39, -0.19,  1.62,  1.58]]),  # mat4x3
> +        np.array([[ 1.42, -0.86,  0.27],
> +                  [ 1.80, -1.74,  0.04],
> +                  [-1.88, -0.37,  0.43],
> +                  [ 1.37,  1.90,  0.71]]),  # mat3x4
> +        np.array([[-1.72,  0.09,  0.45],
> +                  [-0.31, -1.58,  1.92],
> +                  [ 0.14,  0.18, -0.56],
> +                  [ 0.40, -0.77,  1.76]]),  # mat3x4
> +        ]
> +
> +    dsquaredmats = [
> +        np.array([[ 1.60,  0.76],
> +                  [ 1.53, -1.00]]),  # mat2
> +        np.array([[-0.13, -0.87],
> +                  [-1.40,  1.40]]),  # mat2
> +        np.array([[-1.11,  1.67, -0.41],
> +                  [ 0.13,  1.09, -0.02],
> +                  [ 0.56,  0.95,  0.24]]),  # mat3
> +        np.array([[-1.69, -0.46, -0.18],
> +                  [-1.09,  1.75,  2.00],
> +                  [-1.53, -0.70, -1.47]]),  # mat3
> +        np.array([[-1.00, -0.55, -1.08,  1.79],
> +                  [ 1.77,  0.62,  0.48, -1.35],
> +                  [ 0.09, -0.71, -1.39, -1.21],
> +                  [-0.91, -1.82, -1.43,  0.72]]),  # mat4
> +        np.array([[ 0.06,  1.31,  1.52, -1.96],
> +                  [ 1.60, -0.32,  0.51, -1.84],
> +                  [ 1.25,  0.45,  1.90, -0.72],
> +                  [-0.16,  0.45, -0.88,  0.39]]),  # mat4
> +        ]
> +    dmats = dsquaredmats + [
> +        np.array([[ 0.09,  1.30,  1.25],
> +                  [-1.19,  0.08,  1.08]]),  # mat3x2
> +        np.array([[-0.36, -1.08, -0.60],
> +                  [-0.53,  0.88, -1.79]]),  # mat3x2
> +        np.array([[-0.46,  1.94],
> +                  [-0.45, -0.75],
> +                  [ 1.03, -0.50]]),  # mat2x3
> +        np.array([[ 1.38, -1.08],
> +                  [-1.27,  1.83],
> +                  [ 1.00, -0.74]]),  # mat2x3
> +        np.array([[ 1.81, -0.87,  0.81,  0.65],
> +                  [-1.16, -1.52,  0.25, -1.51]]),  # mat4x2
> +        np.array([[ 1.93, -1.63,  0.29,  1.60],
> +                  [ 0.49,  0.27,  0.14,  0.94]]),  # mat4x2
> +        np.array([[ 0.16, -1.69],
> +                  [-0.80,  0.59],
> +                  [-1.74, -1.43],
> +                  [-0.02, -1.21]]),  # mat2x4
> +        np.array([[-1.02,  0.74],
> +                  [-1.64, -0.13],
> +                  [-1.59,  0.47],
> +                  [ 0.30,  1.13]]),  # mat2x4
> +        np.array([[-0.27, -1.38, -1.41, -0.12],
> +                  [-0.17, -0.56,  1.47,  1.86],
> +                  [-1.85, -1.29,  1.77,  0.01]]),  # mat4x3
> +        np.array([[-0.47, -0.15,  1.97, -1.05],
> +                  [-0.20,  0.53, -1.82, -1.41],
> +                  [-1.39, -0.19,  1.62,  1.58]]),  # mat4x3
> +        np.array([[ 1.42, -0.86,  0.27],
> +                  [ 1.80, -1.74,  0.04],
> +                  [-1.88, -0.37,  0.43],
> +                  [ 1.37,  1.90,  0.71]]),  # mat3x4
> +        np.array([[-1.72,  0.09,  0.45],
> +                  [-0.31, -1.58,  1.92],
> +                  [ 0.14,  0.18, -0.56],
> +                  [ 0.40, -0.77,  1.76]]),  # mat3x4
> +        ]
> +    def f(name, arity, python_equivalent,
> +          filter, test_inputs, tolerance_function=_strict_tolerance,
> +          template=None):
> +        """Make test vectors for the function with the given name and
> +        arity, which was introduced in the given glsl_version.
> +
> +        python_equivalent is a Python function which simulates the GLSL
> +        function.  This function should return None in any case where the
> +        output of the GLSL function is undefined.  However, it need not
> +        check that the lengths of the input vectors are all the same.
> +
> +        If filter is not None, it will be called with each set of
> +        arguments, and test cases will only be generated if the filter
> +        returns True.
> +
> +        test_inputs is a list, the ith element of which is a list of
> +        vectors and/or scalars that are suitable for use as the ith
> +        argument of the function.
> +
> +        If tolerance_function is supplied, it is a function which
> +        should be used to compute the tolerance for the test vectors.
> +        Otherwise, _strict_tolerance is used.
> +
> +        If template is supplied, it is used insted as the template for
> +        the Signature objects generated.
> +        """
> +        test_inputs = make_arguments(test_inputs)
> +
> +        if filter is not None:
> +            test_inputs = \
> +                [arguments for arguments in test_inputs if filter(*arguments)]
> +        _store_test_vectors(
> +            test_suite_dict, name, 400, None,
> +            _simulate_function(
> +                test_inputs, python_equivalent, tolerance_function),
> +            template=template)
> +        _store_test_vectors(
> +            test_suite_dict, name, 150, "ARB_gpu_shader_fp64",
> +            _simulate_function(
> +                test_inputs, python_equivalent, tolerance_function),
> +            template=template)
> +
> +    f('op-add', 2, lambda x, y: x + y, match_simple_binop,
> +      [doubles+dvecs+dmats,
> +       doubles+dvecs+dmats],
> +      template='({0} + {1})')
> +    f('op-sub', 2, lambda x, y: x - y, match_simple_binop,
> +      [doubles+dvecs+dmats,
> +       doubles+dvecs+dmats],
> +      template='({0} - {1})')
> +    f('op-mult', 2, _multiply, match_multiply,
> +      [doubles+dvecs+dmats,
> +       doubles+dvecs+dmats],
> +      template='({0} * {1})')
> +    f('op-div', 2, _divide, match_simple_binop,
> +      [doubles+dvecs+dmats,
> +       doubles+dvecs+dmats],
> +      template='({0} / {1})')
> +    f('length', 1, np.linalg.norm, None, [doubles+dvecs])
> +    f('distance', 2, lambda x, y: np.linalg.norm(x-y), match_args(0, 1),
> +      [doubles+dvecs, doubles+dvecs])
> +    f('dot', 2, np.dot, match_args(0, 1), [doubles+dvecs, doubles+dvecs])
> +    f('cross', 2, np.cross, match_args(0, 1), [dvec3s, dvec3s],
> +      _cross_product_tolerance)
> +    f('normalize', 1, _normalize, None, [nz_doubles_dvecs])
> +    f('faceforward', 3, _faceforward, match_args(0, 1, 2),
> +      [doubles+dvecs, doubles+dvecs, doubles+dvecs])
> +    f('reflect', 2, _reflect, match_args(0, 1),
> +      [doubles+dvecs, norm_doubles_dvecs])
> +    f('refract', 3, _refract, match_args(0, 1),
> +      [norm_doubles_dvecs, norm_doubles_dvecs, [0.5, 2.0]])
> +    f('matrixCompMult', 2, lambda x, y: x*y, match_args(0, 1),
> +      [dmats, dmats])
> +    f('outerProduct', 2, np.outer, None, [dvecs, dvecs])
> +    f('transpose', 1, np.transpose, None, [dmats])
> +
> +    f('inverse', 1, np.linalg.inv, None, [dsquaredmats])
> +
> +    f('determinant', 1, np.linalg.det, None, [dsquaredmats])
> +_make_vector_or_matrix_test_vectors(test_suite)
> +
> +
> +def _check_signature_safety(test_suite_dict):
> +    """As a final safety check, verify that for each possible
> +    combination of name and argtypes, there is exactly one
> +    signature.
> +    """
> +    name_argtype_combos = set()
> +    for signature in test_suite_dict:
> +        name_argtype_combo = (signature.name, signature.argtypes, signature.extension)
> +        if name_argtype_combo in name_argtype_combos:
> +            raise Exception(
> +                'Duplicate signature found for {0}'.format(name_argtype_combo))
> +        name_argtype_combos.add(name_argtype_combo)
> +_check_signature_safety(test_suite)
> diff --git a/generated_tests/gen_builtin_uniform_tests_fp64.py b/generated_tests/gen_builtin_uniform_tests_fp64.py
> new file mode 100644
> index 0000000..84ecf9f
> --- /dev/null
> +++ b/generated_tests/gen_builtin_uniform_tests_fp64.py
> @@ -0,0 +1,648 @@
> +# coding=utf-8
> +#
> +# Copyright © 2011 Intel Corporation
> +#
> +# Permission is hereby granted, free of charge, to any person obtaining a
> +# copy of this software and associated documentation files (the "Software"),
> +# to deal in the Software without restriction, including without limitation
> +# the rights to use, copy, modify, merge, publish, distribute, sublicense,
> +# and/or sell copies of the Software, and to permit persons to whom the
> +# Software is furnished to do so, subject to the following conditions:
> +#
> +# The above copyright notice and this permission notice (including the next
> +# paragraph) shall be included in all copies or substantial portions of the
> +# Software.
> +#
> +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +# DEALINGS IN THE SOFTWARE.
> +
> +# Generate a set of shader_runner tests for every overloaded version
> +# of every built-in function, based on the test vectors computed by
> +# builtin_function.py.
> +#
> +# In each set of generated tests, one test exercises the built-in
> +# function in each type of shader (vertex, geometry, and fragment).
> +# In all cases, the inputs to the built-in function come from
> +# uniforms, so that the effectiveness of the test won't be
> +# circumvented by constant folding in the GLSL compiler.
> +#
> +# The tests operate by invoking the built-in function in the
> +# appropriate shader, applying a scale and offset so that the expected
> +# values are in the range [0.25, 0.75], and then outputting the result
> +# as a solid rgba color, which is then checked using shader_runner's
> +# "probe rgba" command.
> +#
> +# For built-in functions whose result type is a matrix, the test
> +# checks one column at a time.
> +#
> +# This program outputs, to stdout, the name of each file it generates.
> +# With the optional argument --names-only, it only outputs the names
> +# of the files; it doesn't generate them.
> +
> +from builtin_function_fp64 import *
> +import abc
> +import numpy
> +import optparse
> +import os
> +import os.path
> +import sys
> +
> +
> +def compute_offset_and_scale(test_vectors):
> +    """Compute scale and offset values such that for each result in
> +    test_vectors, (result - offset) * scale is in the range [0.25,
> +    0.75], and scale is less than or equal to 1.0.  These values are
> +    used to transform the test vectors so that their outputs can be
> +    stored in gl_FragColor without overflow.
> +    """
> +    low = min(numpy.min(tv.result) for tv in test_vectors)
> +    hi = max(numpy.max(tv.result) for tv in test_vectors)
> +    span = hi - low
> +    center = (hi + low)/2.0
> +    span *= 2.0
> +    if span < 1.0:
> +        span = 1.0
> +    offset = center - span/2.0
> +    scale = 1.0/span
> +    return offset, scale
> +
> +
> +def shader_runner_format(values):
> +    """Format the given values for use in a shader_runner "uniform" or
> +    "probe rgba" command.  Bools are converted to 0's and 1's, and
> +    values are separated by spaces.
> +    """
> +    transformed_values = []
> +    for value in values:
> +        if isinstance(value, (bool, np.bool_)):
> +            transformed_values.append(int(value))
> +        else:
> +            transformed_values.append(value)
> +    return ' '.join(repr(x) for x in transformed_values)
> +
> +
> +def shader_runner_type(glsl_type):
> +    """Return the appropriate type name necessary for binding a
> +    uniform of the given type using shader_runner's "uniform" command.
> +    Boolean values and vectors are converted to ints, and square
> +    matrices are written in "matNxN" form.
> +    """
> +    if glsl_type.base_type == glsl_bool:
> +        if glsl_type.is_scalar:
> +            return 'int'
> +        else:
> +            return 'ivec{0}'.format(glsl_type.num_rows)
> +    if glsl_type.is_matrix:
> +        return 'dmat{0}x{1}'.format(glsl_type.num_cols, glsl_type.num_rows)
> +    else:
> +        return str(glsl_type)
> +
> +
> +class Comparator(object):
> +    """Base class which abstracts how we compare expected and actual
> +    values.
> +    """
> +    __metaclass__ = abc.ABCMeta
> +
> +    def make_additional_declarations(self):
> +        """Return additional declarations, if any, that are needed in
> +        the shader program.
> +        """
> +        return ''
> +
> +    @abc.abstractmethod
> +    def make_result_handler(self, invocation, output_var):
> +        """Return the shader code that is needed to produce the result
> +        and store it in output_var.
> +
> +        invocation is the GLSL code to compute the output of the
> +        built-in function.
> +        """
> +
> +    @abc.abstractmethod
> +    def make_result_test(self, test_num, test_vector):
> +        """Return the shader_runner test code that is needed to test a
> +        single test vector.
> +        """
> +
> +    def testname_suffix(self):
> +        """Return a string to be used as a suffix on the test name to
> +        distinguish it from tests using other comparators."""
> +        return ''
> +
> +
> +class BoolComparator(Comparator):
> +    """Comparator that tests functions returning bools and bvecs by
> +    converting them to floats.
> +
> +    This comparator causes code to be generated in the following form:
> +
> +        rettype result = func(args);
> +        output_var = vec4(result, 0.0, ...);
> +    """
> +    def __init__(self, signature):
> +        assert not signature.rettype.is_matrix
> +        self.__signature = signature
> +        self.__padding = 4 - signature.rettype.num_rows
> +
> +    def make_result_handler(self, invocation, output_var):
> +        statements = '  {0} result = {1};\n'.format(
> +            self.__signature.rettype, invocation)
> +        statements += '  {0} = vec4(result{1});\n'.format(
> +            output_var, ', 0.0' * self.__padding)
> +        return statements
> +
> +    def convert_to_float(self, value):
> +        """Convert the given vector or scalar value to a list of
> +        floats representing the expected color produced by the test.
> +        """
> +        value = value*1.0  # convert bools to floats
> +        value = column_major_values(value)
> +        value += [0.0] * self.__padding
> +        return value
> +
> +    def make_result_test(self, test_num, test_vector, draw):
> +        test = draw
> +        test += 'probe rgba {0} 0 {1}\n'.format(
> +            test_num,
> +            shader_runner_format(self.convert_to_float(test_vector.result)))
> +        return test
> +
> +
> +class BoolIfComparator(Comparator):
> +    """Comparator that tests functions returning bools by evaluating
> +    them inside an if statement.
> +
> +    This comparator causes code to be generated in the following form:
> +
> +        if (func(args))
> +          output_var = vec4(1.0, 1.0, 0.0, 1.0);
> +        else
> +          output_var = vecp(0.0, 0.0, 1.0, 1.0);
> +    """
> +    def __init__(self, signature):
> +        assert signature.rettype == glsl_bool
> +        self.__padding = 4 - signature.rettype.num_rows
> +
> +    def make_result_handler(self, invocation, output_var):
> +        statements = '  if({0})\n'.format(invocation)
> +        statements += '    {0} = vec4(1.0, 1.0, 0.0, 1.0);\n'.format(
> +            output_var)
> +        statements += '  else\n'
> +        statements += '    {0} = vec4(0.0, 0.0, 1.0, 1.0);\n'.format(
> +            output_var)
> +        return statements
> +
> +    def convert_to_float(self, value):
> +        """Convert the given vector or scalar value to a list of
> +        floats representing the expected color produced by the test.
> +        """
> +        if value:
> +            return [1.0, 1.0, 0.0, 1.0]
> +        else:
> +            return [0.0, 0.0, 1.0, 1.0]
> +
> +    def make_result_test(self, test_num, test_vector, draw):
> +        test = draw
> +        test += 'probe rgba {0} 0 {1}\n'.format(
> +            test_num,
> +            shader_runner_format(self.convert_to_float(test_vector.result)))
> +        return test
> +
> +    def testname_suffix(self):
> +        return '-using-if'
> +
> +
> +class IntComparator(Comparator):
> +    """Comparator that tests functions returning ints or ivecs using a
> +    strict equality test.
> +
> +    This comparator causes code to be generated in the following form:
> +
> +        rettype result = func(args);
> +        output_var = result == expected ? vec4(0.0, 1.0, 0.0, 1.0)
> +                                        : vec4(1.0, 0.0, 0.0, 1.0);
> +    """
> +    def __init__(self, signature):
> +        self.__signature = signature
> +
> +    def make_additional_declarations(self):
> +        return 'uniform {0} expected;\n'.format(self.__signature.rettype)
> +
> +    def make_result_handler(self, invocation, output_var):
> +        statements = '  {0} result = {1};\n'.format(
> +            self.__signature.rettype, invocation)
> +        statements += '  {v} = {cond} ? {green} : {red};\n'.format(
> +            v=output_var, cond='result == expected',
> +            green='vec4(0.0, 1.0, 0.0, 1.0)',
> +            red='vec4(1.0, 0.0, 0.0, 1.0)')
> +        return statements
> +
> +    def make_result_test(self, test_num, test_vector, draw):
> +        test = 'uniform {0} expected {1}\n'.format(
> +            shader_runner_type(self.__signature.rettype),
> +            shader_runner_format(column_major_values(test_vector.result)))
> +        test += draw
> +        test += 'probe rgba {0} 0 0.0 1.0 0.0 1.0\n'.format(test_num)
> +        return test
> +
> +
> +class FloatComparator(Comparator):
> +    """Comparator that tests functions returning floats or vecs using a
> +    strict equality test.
> +
> +    This comparator causes code to be generated in the following form:
> +
> +        rettype result = func(args);
> +        output_var = distance(result, expected) <= tolerance
> +                     ? vec4(0.0, 1.0, 0.0, 1.0) : vec4(1.0, 0.0, 0.0, 1.0);
> +    """
> +    def __init__(self, signature):
> +        self.__signature = signature
> +
> +    def make_additional_declarations(self):
> +        decls = 'uniform double tolerance;\n'
> +        decls += 'uniform {0} expected;\n'.format(self.__signature.rettype)
> +        return decls
> +
> +    def make_indexers(self):
> +        """Build a list of strings which index into every possible
> +        value of the result.  For example, if the result is a vec2,
> +        then build the indexers ['[0]', '[1]'].
> +        """
> +        if self.__signature.rettype.num_cols == 1:
> +            col_indexers = ['']
> +        else:
> +            col_indexers = ['[{0}]'.format(i)
> +                            for i in xrange(self.__signature.rettype.num_cols)]
> +        if self.__signature.rettype.num_rows == 1:
> +            row_indexers = ['']
> +        else:
> +            row_indexers = ['[{0}]'.format(i)
> +                            for i in xrange(self.__signature.rettype.num_rows)]
> +        return [col_indexer + row_indexer
> +                for col_indexer in col_indexers
> +                for row_indexer in row_indexers]
> +
> +    def make_result_handler(self, invocation, output_var):
> +        statements = '  {0} result = {1};\n'.format(
> +            self.__signature.rettype, invocation)
> +        # Can't use distance when testing itself, or when the rettype
> +        # is a matrix.
> +        if self.__signature.name == 'distance' or \
> +                self.__signature.rettype.is_matrix:
> +            statements += '  {0} residual = result - expected;\n'.format(
> +                self.__signature.rettype)
> +            statements += '  double error_sq = {0};\n'.format(
> +                ' + '.join(
> +                    'residual{0} * residual{0}'.format(indexer)
> +                    for indexer in self.make_indexers()))
> +            condition = 'error_sq <= tolerance * tolerance'
> +        else:
> +            condition = 'distance(result, expected) <= tolerance'
> +        statements += '  {v} = {cond} ? {green} : {red};\n'.format(
> +            v=output_var, cond=condition, green='vec4(0.0, 1.0, 0.0, 1.0)',
> +            red='vec4(1.0, 0.0, 0.0, 1.0)')
> +        return statements
> +
> +    def make_result_test(self, test_num, test_vector, draw):
> +        test = 'uniform {0} expected {1}\n'.format(
> +            shader_runner_type(self.__signature.rettype),
> +            shader_runner_format(column_major_values(test_vector.result)))
> +        test += 'uniform double tolerance {0}\n'.format(
> +            shader_runner_format([test_vector.tolerance]))
> +        test += draw
> +        test += 'probe rgba {0} 0 0.0 1.0 0.0 1.0\n'.format(test_num)
> +        return test
> +
> +
> +class ShaderTest(object):
> +    """Class used to build a test of a single built-in.  This is an
> +    abstract base class--derived types should override test_prefix(),
> +    make_vertex_shader(), make_fragment_shader(), and other functions
> +    if necessary.
> +    """
> +    __metaclass__ = abc.ABCMeta
> +
> +    def __init__(self, signature, test_vectors, use_if):
> +        """Prepare to build a test for a single built-in.  signature
> +        is the signature of the built-in (a key from the
> +        builtin_function.test_suite dict), and test_vectors is the
> +        list of test vectors for testing the given builtin (the
> +        corresponding value from the builtin_function.test_suite
> +        dict).
> +
> +        If use_if is True, then the generated test checks the result
> +        by using it in an if statement--this only works for builtins
> +        returning bool.
> +        """
> +        self._signature = signature
> +        self._test_vectors = test_vectors
> +        if use_if:
> +            self._comparator = BoolIfComparator(signature)
> +        elif signature.rettype.base_type == glsl_bool:
> +            self._comparator = BoolComparator(signature)
> +        elif signature.rettype.base_type == glsl_double:
> +            self._comparator = FloatComparator(signature)
> +        else:
> +            raise Exception('Unexpected rettype {0}'.format(signature.rettype))
> +
> +    def glsl_version(self):
> +        return self._signature.version_introduced
> +
> +    def draw_command(self):
> +        if self.glsl_version() >= 140:
> +            return 'draw arrays GL_TRIANGLE_FAN 0 4\n'
> +        else:
> +            return 'draw rect -1 -1 2 2\n'
> +
> +    def make_additional_requirements(self):
> +        """Return a string that should be included in the test's
> +        [require] section.
> +        """
> +        return ''
> +
> +    @abc.abstractmethod
> +    def test_prefix(self):
> +        """Return the prefix that should be used in the test file name
> +        to identify the type of test, e.g. "vs" for a vertex shader
> +        test.
> +        """
> +
> +    @abc.abstractmethod
> +    def make_vertex_shader(self):
> +        """Return the vertex shader for this test."""
> +
> +    def make_geometry_shader(self):
> +        """Return the geometry shader for this test (or None if this
> +        test doesn't require a geometry shader).  No need to
> +        reimplement this function in classes that don't use geometry
> +        shaders.
> +        """
> +        return None
> +
> +    def make_geometry_layout(self):
> +        """Return the geometry layout for this test (or None if this
> +        test doesn't require a geometry layout section).  No need to
> +        reimplement this function in classes that don't use geometry
> +        shaders.
> +        """
> +        return None
> +
> +    @abc.abstractmethod
> +    def make_fragment_shader(self):
> +        """Return the fragment shader for this test."""
> +
> +    def make_test_shader(self, additional_declarations, prefix_statements,
> +                         output_var, suffix_statements):
> +        """Generate the shader code necessary to test the built-in.
> +        additional_declarations is a string containing any
> +        declarations that need to be before the main() function of the
> +        shader.  prefix_statements is a string containing any
> +        additional statements than need to be inside the main()
> +        function of the shader, before the built-in function is
> +        called.  output_var is the variable that the result of the
> +        built-in function should be assigned to, after conversion to a
> +        vec4.  suffix_statements is a string containing any additional
> +        statements that need to be inside the main() funciton of the
> +        shader, after the built-in function is called.
> +        """
> +        shader = ''
> +        if self._signature.extension:
> +            shader += '#extension GL_{0} : require\n'.format(self._signature.extension)
> +        shader += additional_declarations
> +        for i in xrange(len(self._signature.argtypes)):
> +            shader += 'uniform {0} arg{1};\n'.format(
> +                self._signature.argtypes[i], i)
> +        shader += self._comparator.make_additional_declarations()
> +        shader += '\n'
> +        shader += 'void main()\n'
> +        shader += '{\n'
> +        shader += prefix_statements
> +        invocation = self._signature.template.format(
> +            *['arg{0}'.format(i)
> +              for i in xrange(len(self._signature.argtypes))])
> +        shader += self._comparator.make_result_handler(invocation, output_var)
> +        shader += suffix_statements
> +        shader += '}\n'
> +        return shader
> +
> +    def make_test(self):
> +        """Make the complete shader_runner test file, and return it as
> +        a string.
> +        """
> +        test = ''
> +        for test_num, test_vector in enumerate(self._test_vectors):
> +            for i in xrange(len(test_vector.arguments)):
> +                test += 'uniform {0} arg{1} {2}\n'.format(
> +                    shader_runner_type(self._signature.argtypes[i]),
> +                    i, shader_runner_format(
> +                        column_major_values(test_vector.arguments[i])))
> +            # Note: shader_runner uses a 250x250 window so we must
> +            # ensure that test_num <= 250.
> +            test += self._comparator.make_result_test(
> +                test_num % 250, test_vector, self.draw_command())
> +        return test
> +
> +    def make_vbo_data(self):
> +        # Starting with GLSL 1.40/GL 3.1, we need to use VBOs and
> +        # vertex shader input bindings for our vertex data instead of
> +        # the piglit drawing utilities and gl_Vertex.
> +        if self.glsl_version() < 140:
> +            return ""
> +        vbo = '[vertex data]\n'
> +        vbo += 'vertex/float/2\n'
> +        vbo += '-1.0 -1.0\n'
> +        vbo += ' 1.0 -1.0\n'
> +        vbo += ' 1.0  1.0\n'
> +        vbo += '-1.0  1.0\n'
> +        vbo += '\n'
> +        return vbo
> +
> +    def filename(self):
> +        argtype_names = '-'.join(
> +            str(argtype) for argtype in self._signature.argtypes)
> +        if self._signature.extension:
> +            subdir = self._signature.extension
> +        else:
> +            subdir = 'glsl-{0:1.2f}'.format(float(self.glsl_version()) / 100)
> +        return os.path.join(
> +            'spec', subdir, 'execution', 'built-in-functions',
> +            '{0}-{1}-{2}{3}.shader_test'.format(
> +                self.test_prefix(), self._signature.name, argtype_names,
> +                self._comparator.testname_suffix()))
> +
> +    def generate_shader_test(self):
> +        """Generate the test and write it to the output file."""
> +        shader_test = '[require]\n'
> +        shader_test += 'GLSL >= {0:1.2f}\n'.format(
> +            float(self.glsl_version()) / 100)
> +        shader_test += self.make_additional_requirements()
> +        shader_test += '\n'
> +        shader_test += '[vertex shader]\n'
> +        shader_test += self.make_vertex_shader()
> +        shader_test += '\n'
> +        gs = self.make_geometry_shader()
> +        if gs:
> +            shader_test += '[geometry shader]\n'
> +            shader_test += gs
> +            shader_test += '\n'
> +        gl = self.make_geometry_layout()
> +        if gl:
> +            shader_test += '[geometry layout]\n'
> +            shader_test += gl
> +            shader_test += '\n'
> +        shader_test += '[fragment shader]\n'
> +        shader_test += self.make_fragment_shader()
> +        shader_test += '\n'
> +        shader_test += self.make_vbo_data()
> +        shader_test += '[test]\n'
> +        shader_test += self.make_test()
> +        filename = self.filename()
> +        dirname = os.path.dirname(filename)
> +        if not os.path.exists(dirname):
> +            os.makedirs(dirname)
> +        with open(filename, 'w') as f:
> +            f.write(shader_test)
> +
> +
> +class VertexShaderTest(ShaderTest):
> +    """Derived class for tests that exercise the built-in in a vertex
> +    shader.
> +    """
> +    def test_prefix(self):
> +        return 'vs'
> +
> +    def make_vertex_shader(self):
> +        if self.glsl_version() >= 140:
> +            return self.make_test_shader(
> +                'in vec4 vertex;\n' +
> +                'out vec4 color;\n',
> +                '  gl_Position = vertex;\n',
> +                'color', '')
> +        else:
> +            return self.make_test_shader(
> +                'varying vec4 color;\n',
> +                '  gl_Position = gl_Vertex;\n',
> +                'color', '')
> +
> +    def make_fragment_shader(self):
> +        shader = '''varying vec4 color;
> +
> +void main()
> +{
> +  gl_FragColor = color;
> +}
> +'''
> +        return shader
> +
> +
> +class GeometryShaderTest(ShaderTest):
> +    """Derived class for tests that exercise the built-in in a
> +    geometry shader.
> +    """
> +    def test_prefix(self):
> +        return 'gs'
> +
> +    def glsl_version(self):
> +        return max(150, ShaderTest.glsl_version(self))
> +
> +    def make_vertex_shader(self):
> +        shader = ''
> +        shader += "in vec4 vertex;\n"
> +        shader += "out vec4 vertex_to_gs;\n"
> +
> +        shader += "void main()\n"
> +        shader += "{\n"
> +        shader += "     vertex_to_gs = vertex;\n"
> +        shader += "}\n"
> +
> +        return shader
> +
> +    def make_geometry_shader(self):
> +        additional_declarations = ''
> +        additional_declarations += 'layout(triangles) in;\n'
> +        additional_declarations \
> +            += 'layout(triangle_strip, max_vertices = 3) out;\n'
> +        additional_declarations += 'in vec4 vertex_to_gs[3];\n'
> +        additional_declarations += 'out vec4 color;\n'
> +        return self.make_test_shader(
> +            additional_declarations,
> +            '  vec4 tmp_color;\n',
> +            'tmp_color',
> +            '  for (int i = 0; i < 3; i++) {\n'
> +            '    gl_Position = vertex_to_gs[i];\n'
> +            '    color = tmp_color;\n'
> +            '    EmitVertex();\n'
> +            '  }\n')
> +
> +    def make_fragment_shader(self):
> +        shader = '''varying vec4 color;
> +
> +void main()
> +{
> +  gl_FragColor = color;
> +}
> +'''
> +        return shader
> +
> +
> +class FragmentShaderTest(ShaderTest):
> +    """Derived class for tests that exercise the built-in in a
> +    fragment shader.
> +    """
> +    def test_prefix(self):
> +        return 'fs'
> +
> +    def make_vertex_shader(self):
> +        shader = ""
> +        if self.glsl_version() >= 140:
> +            shader += "in vec4 vertex;\n"
> +
> +        shader += "void main()\n"
> +        shader += "{\n"
> +        if self.glsl_version() >= 140:
> +            shader += "        gl_Position = vertex;\n"
> +        else:
> +            shader += "        gl_Position = gl_Vertex;\n"
> +        shader += "}\n"
> +
> +        return shader
> +
> +    def make_fragment_shader(self):
> +        return self.make_test_shader('', '', 'gl_FragColor', '')
> +
> +
> +def all_tests():
> +    for use_if in [False, True]:
> +        for signature, test_vectors in sorted(test_suite.items()):
> +            if use_if and signature.rettype != glsl_bool:
> +                continue
> +            yield VertexShaderTest(signature, test_vectors, use_if)
> +            yield GeometryShaderTest(signature, test_vectors, use_if)
> +            yield FragmentShaderTest(signature, test_vectors, use_if)
> +
> +
> +def main():
> +    desc = 'Generate shader tests that test built-in functions using uniforms'
> +    usage = 'usage: %prog [-h] [--names-only]'
> +    parser = optparse.OptionParser(description=desc, usage=usage)
> +    parser.add_option(
> +        '--names-only',
> +        dest='names_only',
> +        action='store_true',
> +        help="Don't output files, just generate a list of filenames to stdout")
> +    options, args = parser.parse_args()
> +    for test in all_tests():
> +        if not options.names_only:
> +            test.generate_shader_test()
> +        print test.filename()
> +
> +
> +if __name__ == '__main__':
> +    main()
> diff --git a/generated_tests/gen_constant_array_size_tests_fp64.py b/generated_tests/gen_constant_array_size_tests_fp64.py
> new file mode 100644
> index 0000000..031d719
> --- /dev/null
> +++ b/generated_tests/gen_constant_array_size_tests_fp64.py
> @@ -0,0 +1,269 @@
> +# coding=utf-8
> +#
> +# Copyright © 2011 Intel Corporation
> +#
> +# Permission is hereby granted, free of charge, to any person obtaining a
> +# copy of this software and associated documentation files (the "Software"),
> +# to deal in the Software without restriction, including without limitation
> +# the rights to use, copy, modify, merge, publish, distribute, sublicense,
> +# and/or sell copies of the Software, and to permit persons to whom the
> +# Software is furnished to do so, subject to the following conditions:
> +#
> +# The above copyright notice and this permission notice (including the next
> +# paragraph) shall be included in all copies or substantial portions of the
> +# Software.
> +#
> +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +# DEALINGS IN THE SOFTWARE.
> +
> +# Generate a pair of glsl parser tests for every overloaded version of
> +# every built-in function, which test that the built-in functions are
> +# handled properly when applied to constant arguments inside an array
> +# size declaration.
> +#
> +# In each pair of generated tests, one test exercises the built-in
> +# function in vertex shaders, and the other exercises it in fragment
> +# shaders.
> +#
> +# This program outputs, to stdout, the name of each file it generates.
> +# With the optional argument --names-only, it only outputs the names
> +# of the files; it doesn't generate them.
> +
> +from builtin_function_fp64 import *
> +import abc
> +import optparse
> +import os
> +import os.path
> +
> +
> +class ParserTest(object):
> +    """Class used to build a test of a single built-in.  This is an
> +    abstract base class--derived types should override test_suffix(),
> +    output_var(), and other functions if necessary.
> +    """
> +
> +    def __init__(self, signature, test_vectors):
> +        """Prepare to build a test for a single built-in.  signature
> +        is the signature of the built-in (a key from the
> +        builtin_function.test_suite dict), and test_vectors is the
> +        list of test vectors for testing the given builtin (the
> +        corresponding value from the builtin_function.test_suite
> +        dict).
> +        """
> +        self.__signature = signature
> +        self.__test_vectors = test_vectors
> +
> +    def glsl_version(self):
> +        if self.__signature.version_introduced < 120:
> +            # Before version 1.20, built-in function invocations
> +            # weren't allowed in constant expressions.  So even if
> +            # this built-in was introduced prior to 1.20, test it in
> +            # version 1.20.
> +            return 120
> +        else:
> +            return self.__signature.version_introduced
> +
> +    def version_directive(self):
> +        return '#version {0}\n'.format(self.glsl_version())
> +
> +    def additional_declarations(self):
> +        """Return a string containing any additional declarations that
> +        should be placed after the version directive.  Returns the
> +        empty string by default.
> +        """
> +        return ''
> +
> +    def additional_extensions(self):
> +        """Return a list (or other iterable) containing any additional
> +        extension requirements that the test has.  Returns the empty
> +        list by default.
> +        """
> +        return []
> +
> +    @abc.abstractmethod
> +    def test_suffix(self):
> +        """Return the suffix that should be used in the test file name
> +        to identify the type of shader, e.g. "vert" for a vertex
> +        shader test.
> +        """
> +
> +    def make_condition(self, test_vector):
> +        """Generate a GLSL constant expression that should evaluate to
> +        true if the GLSL compiler's constant evaluation produces the
> +        correct result for the given test vector, and false if not.
> +        """
> +        invocation = self.__signature.template.format(
> +            *[glsl_constant(x) for x in test_vector.arguments])
> +        if self.__signature.rettype.base_type == glsl_double:
> +            # Test floating-point values within tolerance
> +            if self.__signature.name == 'distance':
> +                # Don't use the distance() function to test itself.
> +                return '{0} <= {1} && {1} <= {2}'.format(
> +                    test_vector.result - test_vector.tolerance,
> +                    invocation,
> +                    test_vector.result + test_vector.tolerance)
> +            elif self.__signature.rettype.is_matrix:
> +                # We can't apply distance() to matrices.  So apply it
> +                # to each column and root-sum-square the results.  It
> +                # is safe to use pow() here because its behavior is
> +                # verified in the pow() tests.
> +                terms = []
> +                for col in xrange(self.__signature.rettype.num_cols):
> +                    terms.append('(distance({0}[{1}], {2}) * distance({0}[{1}], {2}))'.format(
> +                        invocation, col,
> +                        glsl_constant(test_vector.result[:, col])))
> +                rss_distance = ' + '.join(terms)
> +                sq_tolerance = test_vector.tolerance * test_vector.tolerance
> +                return '{0} <= {1}'.format(
> +                    rss_distance, glsl_constant(sq_tolerance))
> +            else:
> +                return 'distance({0}, {1}) <= {2}'.format(
> +                    invocation, glsl_constant(test_vector.result),
> +                    glsl_constant(test_vector.tolerance))
> +        else:
> +            # Test non-floating point values exactly
> +            assert not self.__signature.rettype.is_matrix
> +            if self.__signature.name == 'equal':
> +                # Don't use the equal() function to test itself.
> +                assert self.__signature.rettype.is_vector
> +                terms = []
> +                for row in xrange(self.__signature.rettype.num_rows):
> +                    terms.append('{0}[{1}] == {2}'.format(
> +                        invocation, row,
> +                        glsl_constant(test_vector.result[row])))
> +                return ' && '.join(terms)
> +            elif self.__signature.rettype.is_vector:
> +                return 'all(equal({0}, {1}))'.format(
> +                    invocation, glsl_constant(test_vector.result))
> +            else:
> +                return '{0} == {1}'.format(
> +                    invocation, glsl_constant(test_vector.result))
> +
> +    def make_shader(self):
> +        """Generate the shader code necessary to test the built-in."""
> +        shader = self.version_directive()
> +        if self.__signature.extension:
> +            shader += '#extension GL_{0} : require\n'.format(self.__signature.extension)
> +        shader += self.additional_declarations()
> +        shader += '\n'
> +        shader += 'void main()\n'
> +        shader += '{\n'
> +        lengths = []
> +        for i, test_vector in enumerate(self.__test_vectors):
> +            shader += '  double[{0} ? 1 : -1] array{1};\n'.format(
> +                self.make_condition(test_vector), i)
> +            lengths.append('array{0}.length()'.format(i))
> +        shader += '  {0} = dvec4({1});\n'.format(
> +            self.output_var(), ' + '.join(lengths))
> +        shader += '}\n'
> +        return shader
> +
> +    def filename(self):
> +        argtype_names = '-'.join(
> +            str(argtype) for argtype in self.__signature.argtypes)
> +        if self.__signature.extension:
> +            subdir = self.__signature.extension
> +        else:
> +            subdir = 'glsl-{0:1.2f}'.format(float(self.glsl_version()) / 100)
> +        return os.path.join(
> +            'spec', subdir, 'compiler', 'built-in-functions',
> +            '{0}-{1}.{2}'.format(
> +                self.__signature.name, argtype_names, self.test_suffix()))
> +
> +    def generate_parser_test(self):
> +        """Generate the test and write it to the output file."""
> +        parser_test = '/* [config]\n'
> +        parser_test += ' * expect_result: pass\n'
> +        parser_test += ' * glsl_version: {0:1.2f}\n'.format(
> +            float(self.glsl_version()) / 100)
> +        req_extensions = list(self.additional_extensions())
> +        if req_extensions:
> +            parser_test += ' * require_extensions: {0}\n'.format(
> +                ' '.join(req_extensions))
> +        parser_test += ' * [end config]\n'
> +        parser_test += ' *\n'
> +        parser_test += ' * Check that the following test vectors are constant'\
> +                       'folded correctly:\n'
> +        for test_vector in self.__test_vectors:
> +            parser_test += ' * {0} => {1}\n'.format(
> +                self.__signature.template.format(
> +                    *[glsl_constant(arg) for arg in test_vector.arguments]),
> +                glsl_constant(test_vector.result))
> +        parser_test += ' */\n'
> +        parser_test += self.make_shader()
> +        filename = self.filename()
> +        dirname = os.path.dirname(filename)
> +        if not os.path.exists(dirname):
> +            os.makedirs(dirname)
> +        with open(filename, 'w') as f:
> +            f.write(parser_test)
> +
> +
> +class VertexParserTest(ParserTest):
> +    """Derived class for tests that exercise the built-in in a vertex
> +    shader.
> +    """
> +    def test_suffix(self):
> +        return 'vert'
> +
> +    def output_var(self):
> +        return 'gl_Position'
> +
> +
> +class GeometryParserTest(ParserTest):
> +    """Derived class for tests that exercise the built-in in a geometry
> +    shader.
> +    """
> +    def glsl_version(self):
> +       return max(150, ParserTest.glsl_version(self))
> +
> +    def test_suffix(self):
> +        return 'geom'
> +
> +    def output_var(self):
> +        return 'gl_Position'
> +
> +
> +class FragmentParserTest(ParserTest):
> +    """Derived class for tests that exercise the built-in in a fagment
> +    shader.
> +    """
> +    def test_suffix(self):
> +        return 'frag'
> +
> +    def output_var(self):
> +        return 'gl_FragColor'
> +
> +
> +def all_tests():
> +    for signature, test_vectors in sorted(test_suite.items()):
> +        yield VertexParserTest(signature, test_vectors)
> +        yield GeometryParserTest(signature, test_vectors)
> +        yield FragmentParserTest(signature, test_vectors)
> +
> +
> +def main():
> +    desc = 'Generate shader tests that test built-in functions using constant'\
> +           'array sizes'
> +    usage = 'usage: %prog [-h] [--names-only]'
> +    parser = optparse.OptionParser(description=desc, usage=usage)
> +    parser.add_option('--names-only',
> +                      dest='names_only',
> +                      action='store_true',
> +                      help="Don't output files, just generate a list of"
> +                           "filenames to stdout")
> +    options, args = parser.parse_args()
> +
> +    for test in all_tests():
> +        if not options.names_only:
> +            test.generate_parser_test()
> +        print test.filename()
> +
> +
> +if __name__ == '__main__':
> +    main()
> --
> 1.8.5.3
>
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit


More information about the Piglit mailing list