[Mesa-dev] [PATCH 0/4] glsl/tests: completely broken or strange design ?

[Emil Velikov]

Hi all,

As some of you may know we have a bunch of interesting tests in glsl.

A bit earlier I've spotted that they are competely broken, as they would
happily return "success" even when 0 tests were generated.

As an example I will take optimisation-test, which this series sort of
fixes. Here is the design flow:
 - optimisation-test (SH1) assumes that we're in the right location and 
opens a python script (PY1).
 - if found (only in corner cases) - the PY1 is executed and generates 
a bunch of bash scripts (SH2) which are stored in the srcdir (fails on
make distcheck).
 - PY1 also hardcodes the location of the glsl_test binary, based on 
some magically derived value (hint srcdir not used and wrong to use)
 - the return status of PY1 is unchecked.
 - thus as PY1 fails (and we often do) no tests are generated and SH1 
reports "SUCCESS"
 - alternatively, SH1 executes SH2 to produce the to-be-tested files
 - then it (SH1) proceeds to execute PY2
 - latter of which decodes and saves the [to a temporary location] the
template and the file under test and calls "diff" on the resulting
files.

With the following patches I've addressed all usecases, but the fact that
srcdir is RO, thus 3/4 will cause "make distcheck" to fail (and rightfully so).

I believe all of us can see how fragile/suboptimal/you name it the
design is and it need a complete rework.

Conteplating about giving a shot myself, but I suspect all of it will end up
in the bin, now that we have Dylan back ;-)

So the questions are:
 - Do we care about those tests ?
 - Dylan any interest in giving it a stab ?

Thanks
Emil

Cc: Dylan Baker <dylan at pnwbakers.com>
Cc: Kenneth Graunke <kenneth at whitecape.org>
Cc: Matt Turner <mattst88 at gmail.com>

Emil Velikov (4):
  glsl/tests/optimisation-test: make sure that $PYTHON2 is set/available
  glsl/tests/optimisation-test: ensure that compare_ir is available
  glsl/tests/optimization-test: error out if we fail to generate any
    tests
  glsl/tests/optimization-test: correctly manage srcdir/pwd and co

 src/compiler/glsl/tests/optimization-test | 50 +++++++++++++++++++++++++++----
 1 file changed, 44 insertions(+), 6 deletions(-)

-- 
2.11.0