[Piglit] [PATCH 00/34] Hybridize build system for python2 and python3

[Dylan Baker]

This series hybridizes the python generator framework (both for tests
and for dispatch) to work with both python2 and python3.

This makes use of the six module (which is a new dependency that you
probably already have installed, a lot of python modules depend on it),
to craft code that produces as close to the same output as is possible.
There are limitations, python3 numeric types have limitless precision,
unlike their python2 counterparts, and there isn't anything sane that
can be done about it except to use numpy, which is a bigger project than
I want to take on at the moment.

The goal here is that we can get everything except the test framework to
be hybridized, and then make a python2 branch, with the assumption that
cherry-picking anything that isn't a framework change will be safe. To
that end I'm willing to ensure that any changes to the output format
that are relevant get ported to the python2 framework, as do bug fixes.

This is motivated by three things. First, we really want a reliable
timeout mechanism, and python3 provides that (see my
wip/python3-complete branch at my github) Second, linux distros are
starting to make the transition toward python3. And finally, python3.3
(the version we're targeting) starts to add new features that are useful
for us, and has added more in python 3.4, and will continue to do so in
later versions of python3.x. Finally, the code is generally clean enough
at this point that we can make the transition with very little pain, the
code is largely in a shape to make the jump.

This series does not turn on python3 (even with a switch), it simply
lays the groundwork. The second series I mentioned (which still needs
cleanup) finishes out the transition and begins to make use of python3
features. Since this doesn't impede the use of python2, I don't think
it's a problem to land, there are maybe 5 lines of code that are dead
without python3 support. You can manually turn it on with a one line
change to CMakeList.txt, as explained later in this message.

I have tested quick.py and there is no difference in results if the
output is generated with python3 or with python2. I cannot get a machine
with OpenCL running (but I did modify the generators), and they same to
produce the same out put when diffed, but I would really appreciate if
one of the OpenCL guys could give them a whirl and make sure that they
don't change the results.

If you want to test this out you'll likely need to clear the piglit mako
cache it's located in <system tmp directory>/piglit-<username>

To use python3 you'll also need to change find_package(PythonInterp 2.7
REQUIRED) in <root>/CMakeList.txt to find_package(PythonInterp 3.3
REQUIRED). I don't think that any of the cl generators use mako, so I
don't think you'll need to clear that cache, but you will need to delete
the cmake cache to get the new python version. This will not affect the
runner version, just the python build version.

This is available at my github:
https://github.com/dcbaker/piglit submit/python3

Dylan Baker (34):
  gen_builtin_packing_tests.py: use __future__ division
  gen_builtin_uniform_tests.py: use __future__ functions
  gen_builtin_uniform_tests_fp64.py: use __future__ functions
  gen_constant_array_size_tests.py: use __future__ functions
  gen_constant_array_size_tests_fp64.py: use __future__ functions.
  generated_tests: use future division in mako templates
  Add dependency on Python six
  generators: python2/3 hybridize gen_outerproduct_tests.py
  gen_non-lvalue_tests.py: make python2/3 compatible
  gen_texture_query_lod_tests.py: make pyhthon2/3 hybrid
  generators: interpolation-qualifier... python2/3 hybridize
  builtin_function*py: convert to python3
  gen_shader_precision_tests.py: python2/3 hybridize
  gen_uniform_initializer_tests.py: hybridize for python2 and python3
  gen_constant_array_size_tests.py: python2/3 hybridize
  gen_constant_array_size_tests_fp64.py: python2/3 hybridize
  gen_interpolation_tests.py: hybridize python 2/3
  gen_builtin_uniform_tests_fp64: hybridize python2/3
  gen_builtin_uniform_tests.py: python2/3 hybridize
  gen_builtin_packing_tests.py: python2/3 hybridize
  gen_shader_bit_encoding_tests.py: python 2/3 hybridize
  registry/gl.py: drop 2.6 support
  registry/gl.py: Don't try to compare None and not-None
  registry/gl.py: add hash methods for classes that need to be hashed
  registry generation: Finish hybridizing
  generators: rename cl generators so they can be imported
  generators: use if __name__ == __main__ pattern in cl generators
  gen_cl_int_builtins.py: Replace use of Long type
  cl tests: move non-generated tests out of the generated tests dir
  genclbuiltins.py: remove tabs
  genclbuiltins.py: add MIT header
  gen_cl_int_builtins.py: use __future__ division
  genclbuiltins.py: hybridize for python3
  gen_cl_store_tests.py: hybridize for python 2/3

 CMakeLists.txt                                     |  1 +
 cmake/Modules/FindPythonSix.cmake                  | 22 ++++++
 generated_tests/CMakeLists.txt                     |  8 +--
 generated_tests/builtin_function.py                | 27 +++++--
 generated_tests/builtin_function_fp64.py           | 33 +++++++--
 generated_tests/gen_builtin_packing_tests.py       |  4 +-
 generated_tests/gen_builtin_uniform_tests.py       | 16 +++--
 generated_tests/gen_builtin_uniform_tests_fp64.py  | 22 +++---
 ...e-cl-int-builtins.py => gen_cl_int_builtins.py} |  9 ++-
 ...cl-math-builtins.py => gen_cl_math_builtins.py} |  3 +-
 ...l-builtins.py => gen_cl_relational_builtins.py} |  3 +-
 ...ate-cl-store-tests.py => gen_cl_store_tests.py} | 84 ++++++++++++----------
 generated_tests/gen_constant_array_size_tests.py   |  9 ++-
 .../gen_constant_array_size_tests_fp64.py          | 11 +--
 generated_tests/gen_interpolation_tests.py         |  8 ++-
 generated_tests/gen_non-lvalue_tests.py            |  2 +-
 generated_tests/gen_outerproduct_tests.py          |  4 +-
 generated_tests/gen_shader_bit_encoding_tests.py   | 10 +--
 generated_tests/gen_shader_precision_tests.py      | 14 ++--
 generated_tests/gen_texture_query_lod_tests.py     |  8 ++-
 generated_tests/gen_uniform_initializer_tests.py   | 22 +++---
 generated_tests/genclbuiltins.py                   | 32 +++++++--
 .../interpolation-qualifier-built-in-variable.py   | 10 +--
 generated_tests/templates/__init__.py              |  5 ++
 .../fs_pack.shader_test.mako                       |  7 +-
 .../fs_unpack.shader_test.mako                     |  7 +-
 .../vs_pack.shader_test.mako                       |  7 +-
 .../vs_unpack.shader_test.mako                     |  7 +-
 .../template.shader_test.mako                      | 18 +++--
 .../template.shader_test.mako                      |  3 +-
 .../templates/gen_shader_precision_tests/fs.mako   | 11 +--
 .../templates/gen_shader_precision_tests/gs.mako   | 11 +--
 .../templates/gen_shader_precision_tests/vs.mako   | 11 +--
 .../template.glsl_parser_test.mako                 |  2 +-
 registry/gl.py                                     | 68 +++++++++---------
 tests/cl.py                                        |  2 +
 .../cl/store/store-kernels-global.inc              |  0
 .../cl/store/store-kernels-local.inc               |  0
 tests/util/gen_dispatch.py                         | 14 +++-
 39 files changed, 349 insertions(+), 186 deletions(-)
 create mode 100644 cmake/Modules/FindPythonSix.cmake
 rename generated_tests/{generate-cl-int-builtins.py => gen_cl_int_builtins.py} (99%)
 rename generated_tests/{generate-cl-math-builtins.py => gen_cl_math_builtins.py} (99%)
 rename generated_tests/{generate-cl-relational-builtins.py => gen_cl_relational_builtins.py} (99%)
 rename generated_tests/{generate-cl-store-tests.py => gen_cl_store_tests.py} (60%)
 rename {generated_tests => tests}/cl/store/store-kernels-global.inc (100%)
 rename {generated_tests => tests}/cl/store/store-kernels-local.inc (100%)

-- 
2.3.0