[Piglit] [PATCH v4] framework: Add a vulkan tests profile

Mon Nov 5 10:17:59 UTC 2018

This patch and the second patch of the v3 patch series are:

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias at igalia.com>

Thanks,

Sam

On Monday, 5 November 2018 11:04:44 (CET) Neil Roberts wrote:
> This searches for files named *.vk_shader_test in the tests/vulkan
> directory and runs them with VkRunner. VkRunner is executed as an
> external dependency. It is found either with the vkrunner:bin config
> option, by setting the PIGLIT_VKRUNNER_BINARY environment variable, or
> just in the search path.
> 
> v2: Move VkShaderTest to piglit_test.py and rename to VkRunnerTest.
>     Add future imports. Remove unused import.
> v3: Support the PIGLIT_VKRUNNER_BINARY variable to specify the
>     location of VkRunner.
> v4: Add documentation to the README. Add an option in piglit.conf to
>     set the binary location. (Suggested by Samuel Iglesias)
> ---
>  README.md                     | 17 ++++++++++++++++-
>  framework/test/piglit_test.py | 23 +++++++++++++++++++++++
>  piglit.conf.example           |  4 ++++
>  tests/vulkan.py               | 33 +++++++++++++++++++++++++++++++++
>  4 files changed, 76 insertions(+), 1 deletion(-)
>  create mode 100644 tests/vulkan.py
> 
> diff --git a/README.md b/README.md
> index 2ae0febc6..c75c5f6c3 100644
> --- a/README.md
> +++ b/README.md
> @@ -47,6 +47,8 @@ Optionally, you can install the following:
>      (https://simplejson.readthedocs.org/en/latest/)
>    - jsonstreams. A JSON stream writer for python.
>      (https://jsonstreams.readthedocs.io/en/stable/)
> +  - VkRunner. A shader script testing tool for Vulkan.
> +    (https://github.com/igalia/vkrunner)
> 
>  For Python 2.x you can install the following to add features, these are
>  unnecessary for python3:
> @@ -285,6 +287,13 @@ behaves.
>      When this variable is true in python then any timeouts given by tests
>      will be ignored, and they will run until completion or they are killed.
> 
> +  - `PIGLIT_VKRUNNER_BINARY`
> +
> +    Can be used to override the path to the vkrunner executable for
> +    running Vulkan shader tests. Alternatively the config option
> +    vkrunner:bin can be used instead. If neither are set then vkrunner
> +    will be searched for in the search path.
> +
> 
>  ### 3.2 Note
> 
> @@ -338,7 +347,13 @@ The following test sets are currently available:
>      opencv and oclconform.
> 
> 
> -### 4.3 External Integration
> +### 4.3 Vulkan tests
> +
> +  - **vulkan.py** This suite contains all Vulkan tests. Note that
> +    currently all of the Vulkan tests require VkRunner. If it is not
> +    installed then all of the tests will be skipped.
> +
> +### 4.4 External Integration
> 
>    - **xts.py** Support for running the X Test Suite using piglit.
>    - **igt.py** Support for running Intel-gpu-tools test suite using piglit.
> diff --git a/framework/test/piglit_test.py b/framework/test/piglit_test.py
> index f52915d18..c80e355d1 100644
> --- a/framework/test/piglit_test.py
> +++ b/framework/test/piglit_test.py
> @@ -44,6 +44,7 @@ __all__ = [
>      'PiglitCLTest',
>      'PiglitGLTest',
>      'PiglitBaseTest',
> +    'VkRunnerTest',
>      'CL_CONCURRENT',
>      'ROOT_DIR',
>      'TEST_BIN_DIR',
> @@ -229,3 +230,25 @@ class CLProgramTester(PiglitCLTest):
>          command = super(CLProgramTester, self).command
>          command.insert(1, os.path.join(ROOT_DIR, self.filename))
>          return command
> +
> +
> +class VkRunnerTest(PiglitBaseTest):
> +    """ Make a PiglitTest instance for a VkRunner shader test file """
> +
> +    def __init__(self, filename):
> +        vkrunner_bin = os.environ.get('PIGLIT_VKRUNNER_BINARY')
> +
> +        if vkrunner_bin is None:
> +            vkrunner_bin = core.PIGLIT_CONFIG.safe_get(
> +                'vkrunner', 'bin', fallback='vkrunner')
> +
> +        super(VkRunnerTest, self).__init__(
> +            [vkrunner_bin, filename],
> +            run_concurrent=True)
> +
> +    @PiglitBaseTest.command.getter
> +    def command(self):
> +        # This is overriden because we don’t want PiglitBaseTest to
> +        # prepend TEST_BIN_DIR so that it will look for vkrunner in
> +        # the search path.
> +        return self._command
> diff --git a/piglit.conf.example b/piglit.conf.example
> index 5d1ff5474..1877187df 100644
> --- a/piglit.conf.example
> +++ b/piglit.conf.example
> @@ -187,6 +187,10 @@ run_test=./%(test_name)s
>  ; Default: True
>  ;process isolation=True
> 
> +[vkrunner]
> +; Path to the VkRunner executable
> +; bin=/home/neil/local/bin/vkrunner
> +
>  [expected-failures]
>  ; Provide a list of test names that are expected to fail.  These tests
>  ; will be listed as passing in JUnit output when they fail.  Any
> diff --git a/tests/vulkan.py b/tests/vulkan.py
> new file mode 100644
> index 000000000..7058f3108
> --- /dev/null
> +++ b/tests/vulkan.py
> @@ -0,0 +1,33 @@
> +#!/usr/bin/env python
> +# -*- coding: utf-8 -*-
> +"""All Vulkan tests that come with piglit, using default settings."""
> +
> +from __future__ import (
> +    absolute_import, division, print_function, unicode_literals
> +)
> +
> +import os
> +
> +from framework.profile import TestProfile
> +from framework import grouptools
> +from framework.test.piglit_test import VkRunnerTest
> +from .py_modules.constants import TESTS_DIR, GENERATED_TESTS_DIR
> +
> +__all__ = ['profile']
> +
> +profile = TestProfile()
> +
> +# Find and add all shader tests.
> +for basedir in [TESTS_DIR, GENERATED_TESTS_DIR]:
> +    _basedir = os.path.join(basedir, 'vulkan')
> +    for dirpath, _, filenames in os.walk(_basedir):
> +        groupname = grouptools.from_path(os.path.relpath(dirpath,
> _basedir)) +        for filename in filenames:
> +            testname, ext = os.path.splitext(filename)
> +            if ext != '.vk_shader_test':
> +                continue
> +            test = VkRunnerTest(os.path.join(dirpath, filename))
> +            group = grouptools.join(groupname, testname)
> +            assert group not in profile.test_list, group
> +
> +            profile.test_list[group] = test

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.freedesktop.org/archives/piglit/attachments/20181105/c44e0cfd/attachment.sig>